From lp_benchmark_robot at intel.com Tue Sep 1 11:04:23 2015 From: lp_benchmark_robot at intel.com (lp_benchmark_robot) Date: Tue, 1 Sep 2015 09:04:23 +0000 Subject: [Python-checkins] Benchmark Results for Python Default 2015-09-01 Message-ID: <078AA0FFE8C7034097F90205717F504611D77EE7@IRSMSX102.ger.corp.intel.com> Results for project python_default-nightly, build date 2015-09-01 06:02:01 commit: afdfb9d53bec97c2e64166718424b9098ad17350 revision date: 2015-08-31 18:45:11 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v3.4.3, with hash b4cbecbc0781e89a309d03b60a1f75f8499250e6 from 2015-02-25 12:15:33+00:00 ------------------------------------------------------------------------------------------ benchmark RSD* change since change since current rev with last run v3.4.3 regrtest PGO ------------------------------------------------------------------------------------------ :-) django_v2 0.18601% 0.09676% 8.23460% 17.20786% :-( pybench 0.13592% 0.03344% -2.42500% 8.25125% :-( regex_v8 2.87124% -0.00005% -3.17370% 5.04940% :-| nbody 0.11478% 0.50369% -0.82900% 9.08337% :-| json_dump_v2 0.26878% -0.32432% -1.37670% 12.03586% :-| normal_startup 0.75209% 0.02933% -0.07303% 4.98845% ------------------------------------------------------------------------------------------ Note: Benchmark results are measured in seconds. * Relative Standard Deviation (Standard Deviation/Average) Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. No license (express or implied, by estoppel or otherwise) to any intellectual property rights is granted by this document. Intel disclaims all express and implied warranties, including without limitation, the implied warranties of merchantability, fitness for a particular purpose, and non-infringement, as well as any warranty arising from course of performance, course of dealing, or usage in trade. This document may contain information on products, services and/or processes in development. Contact your Intel representative to obtain the latest forecast, schedule, specifications and roadmaps. The products and services described may contain defects or errors known as errata which may cause deviations from published specifications. Current characterized errata are available on request. (C) 2015 Intel Corporation. From python-checkins at python.org Tue Sep 1 11:20:54 2015 From: python-checkins at python.org (raymond.hettinger) Date: Tue, 01 Sep 2015 09:20:54 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_Improve_tutori?= =?utf-8?q?al_suggestion_for_looping_techniques?= Message-ID: <20150901092048.65555.32945@psf.io> https://hg.python.org/cpython/rev/af97d596da40 changeset: 97563:af97d596da40 branch: 2.7 parent: 97560:79afd50396c5 user: Raymond Hettinger date: Tue Sep 01 02:20:44 2015 -0700 summary: Improve tutorial suggestion for looping techniques files: Doc/tutorial/datastructures.rst | 20 ++++++++++---------- 1 files changed, 10 insertions(+), 10 deletions(-) diff --git a/Doc/tutorial/datastructures.rst b/Doc/tutorial/datastructures.rst --- a/Doc/tutorial/datastructures.rst +++ b/Doc/tutorial/datastructures.rst @@ -664,18 +664,18 @@ gallahad the pure robin the brave -To change a sequence you are iterating over while inside the loop (for -example to duplicate certain items), it is recommended that you first make -a copy. Looping over a sequence does not implicitly make a copy. The slice -notation makes this especially convenient:: +It is sometimes tempting to change a list while you are looping over it; +however, it is often simpler and safer to create a new list instead. :: - >>> words = ['cat', 'window', 'defenestrate'] - >>> for w in words[:]: # Loop over a slice copy of the entire list. - ... if len(w) > 6: - ... words.insert(0, w) + >>> import math + >>> raw_data = [56.2, float('NaN'), 51.7, 55.3, 52.5, float('NaN'), 47.8] + >>> filtered_data = [] + >>> for value in raw_data: + ... if not math.isnan(value): + ... filtered_data.append(value) ... - >>> words - ['defenestrate', 'cat', 'window', 'defenestrate'] + >>> filtered_data + [56.2, 51.7, 55.3, 52.5, 47.8] .. _tut-conditions: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 1 11:33:28 2015 From: python-checkins at python.org (raymond.hettinger) Date: Tue, 01 Sep 2015 09:33:28 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_Improve_tutori?= =?utf-8?q?al_suggestion_for_looping_techniques?= Message-ID: <20150901093325.40621.8026@psf.io> https://hg.python.org/cpython/rev/447d8e6b17b9 changeset: 97564:447d8e6b17b9 branch: 3.5 parent: 97561:833db9e2ed14 user: Raymond Hettinger date: Tue Sep 01 02:33:02 2015 -0700 summary: Improve tutorial suggestion for looping techniques files: Doc/tutorial/datastructures.rst | 20 ++++++++++---------- 1 files changed, 10 insertions(+), 10 deletions(-) diff --git a/Doc/tutorial/datastructures.rst b/Doc/tutorial/datastructures.rst --- a/Doc/tutorial/datastructures.rst +++ b/Doc/tutorial/datastructures.rst @@ -612,18 +612,18 @@ orange pear -To change a sequence you are iterating over while inside the loop (for -example to duplicate certain items), it is recommended that you first make -a copy. Looping over a sequence does not implicitly make a copy. The slice -notation makes this especially convenient:: +It is sometimes tempting to change a list while you are looping over it; +however, it is often simpler and safer to create a new list instead. :: - >>> words = ['cat', 'window', 'defenestrate'] - >>> for w in words[:]: # Loop over a slice copy of the entire list. - ... if len(w) > 6: - ... words.insert(0, w) + >>> import math + >>> raw_data = [56.2, float('NaN'), 51.7, 55.3, 52.5, float('NaN'), 47.8] + >>> filtered_data = [] + >>> for value in raw_data: + ... if not math.isnan(value): + ... filtered_data.append(value) ... - >>> words - ['defenestrate', 'cat', 'window', 'defenestrate'] + >>> filtered_data + [56.2, 51.7, 55.3, 52.5, 47.8] .. _tut-conditions: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 1 11:33:28 2015 From: python-checkins at python.org (raymond.hettinger) Date: Tue, 01 Sep 2015 09:33:28 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_merge?= Message-ID: <20150901093325.40621.22161@psf.io> https://hg.python.org/cpython/rev/0073f17ed1f8 changeset: 97565:0073f17ed1f8 parent: 97562:afdfb9d53bec parent: 97564:447d8e6b17b9 user: Raymond Hettinger date: Tue Sep 01 02:33:20 2015 -0700 summary: merge files: Doc/tutorial/datastructures.rst | 20 ++++++++++---------- 1 files changed, 10 insertions(+), 10 deletions(-) diff --git a/Doc/tutorial/datastructures.rst b/Doc/tutorial/datastructures.rst --- a/Doc/tutorial/datastructures.rst +++ b/Doc/tutorial/datastructures.rst @@ -612,18 +612,18 @@ orange pear -To change a sequence you are iterating over while inside the loop (for -example to duplicate certain items), it is recommended that you first make -a copy. Looping over a sequence does not implicitly make a copy. The slice -notation makes this especially convenient:: +It is sometimes tempting to change a list while you are looping over it; +however, it is often simpler and safer to create a new list instead. :: - >>> words = ['cat', 'window', 'defenestrate'] - >>> for w in words[:]: # Loop over a slice copy of the entire list. - ... if len(w) > 6: - ... words.insert(0, w) + >>> import math + >>> raw_data = [56.2, float('NaN'), 51.7, 55.3, 52.5, float('NaN'), 47.8] + >>> filtered_data = [] + >>> for value in raw_data: + ... if not math.isnan(value): + ... filtered_data.append(value) ... - >>> words - ['defenestrate', 'cat', 'window', 'defenestrate'] + >>> filtered_data + [56.2, 51.7, 55.3, 52.5, 47.8] .. _tut-conditions: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 2 01:40:00 2015 From: python-checkins at python.org (victor.stinner) Date: Tue, 01 Sep 2015 23:40:00 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Move_assertion_inside_=5FP?= =?utf-8?q?yTime=5FObjectToTimeval=28=29?= Message-ID: <20150901233957.106006.28510@psf.io> https://hg.python.org/cpython/rev/5602d2094d2e changeset: 97567:5602d2094d2e user: Victor Stinner date: Wed Sep 02 00:50:43 2015 +0200 summary: Move assertion inside _PyTime_ObjectToTimeval() Change also _PyTime_FromSeconds() assertion to ensure that the _PyTime_t type is used. files: Modules/_datetimemodule.c | 1 - Python/pytime.c | 20 ++++++++++++++------ 2 files changed, 14 insertions(+), 7 deletions(-) diff --git a/Modules/_datetimemodule.c b/Modules/_datetimemodule.c --- a/Modules/_datetimemodule.c +++ b/Modules/_datetimemodule.c @@ -4100,7 +4100,6 @@ if (_PyTime_ObjectToTimeval(timestamp, &timet, &us, _PyTime_ROUND_FLOOR) == -1) return NULL; - assert(0 <= us && us <= 999999); return datetime_from_timet_and_us(cls, f, timet, (int)us, tzinfo); } diff --git a/Python/pytime.c b/Python/pytime.c --- a/Python/pytime.c +++ b/Python/pytime.c @@ -101,7 +101,8 @@ _PyTime_ObjectToDenominator(PyObject *obj, time_t *sec, long *numerator, double denominator, _PyTime_round_t round) { - assert(denominator <= LONG_MAX); + assert(denominator <= (double)LONG_MAX); + if (PyFloat_Check(obj)) { double d = PyFloat_AsDouble(obj); return _PyTime_DoubleToDenominator(d, sec, numerator, @@ -149,14 +150,20 @@ _PyTime_ObjectToTimespec(PyObject *obj, time_t *sec, long *nsec, _PyTime_round_t round) { - return _PyTime_ObjectToDenominator(obj, sec, nsec, 1e9, round); + int res; + res = _PyTime_ObjectToDenominator(obj, sec, nsec, 1e9, round); + assert(0 <= *nsec && *nsec < SEC_TO_NS ); + return res; } int _PyTime_ObjectToTimeval(PyObject *obj, time_t *sec, long *usec, _PyTime_round_t round) { - return _PyTime_ObjectToDenominator(obj, sec, usec, 1e6, round); + int res; + res = _PyTime_ObjectToDenominator(obj, sec, usec, 1e6, round); + assert(0 <= *usec && *usec < SEC_TO_US ); + return res; } static void @@ -170,12 +177,13 @@ _PyTime_FromSeconds(int seconds) { _PyTime_t t; + t = (_PyTime_t)seconds; /* ensure that integer overflow cannot happen, int type should have 32 bits, whereas _PyTime_t type has at least 64 bits (SEC_TO_MS takes 30 bits). */ - assert((seconds >= 0 && seconds <= _PyTime_MAX / SEC_TO_NS) - || (seconds < 0 && seconds >= _PyTime_MIN / SEC_TO_NS)); - t = (_PyTime_t)seconds * SEC_TO_NS; + assert((t >= 0 && t <= _PyTime_MAX / SEC_TO_NS) + || (t < 0 && t >= _PyTime_MIN / SEC_TO_NS)); + t *= SEC_TO_NS; return t; } -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 2 01:40:00 2015 From: python-checkins at python.org (victor.stinner) Date: Tue, 01 Sep 2015 23:40:00 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Refactor_pytime=2Ec?= Message-ID: <20150901233957.38877.14784@psf.io> https://hg.python.org/cpython/rev/b367c1446af0 changeset: 97566:b367c1446af0 user: Victor Stinner date: Wed Sep 02 00:49:16 2015 +0200 summary: Refactor pytime.c Move code to convert double timestamp to subfunctions. files: Python/pytime.c | 117 ++++++++++++++++++++--------------- 1 files changed, 67 insertions(+), 50 deletions(-) diff --git a/Python/pytime.c b/Python/pytime.c --- a/Python/pytime.c +++ b/Python/pytime.c @@ -61,43 +61,51 @@ } static int +_PyTime_DoubleToDenominator(double d, time_t *sec, long *numerator, + double denominator, _PyTime_round_t round) +{ + double intpart, err; + /* volatile avoids unsafe optimization on float enabled by gcc -O3 */ + volatile double floatpart; + + floatpart = modf(d, &intpart); + if (floatpart < 0) { + floatpart = 1.0 + floatpart; + intpart -= 1.0; + } + + floatpart *= denominator; + if (round == _PyTime_ROUND_CEILING) { + floatpart = ceil(floatpart); + if (floatpart >= denominator) { + floatpart = 0.0; + intpart += 1.0; + } + } + else { + floatpart = floor(floatpart); + } + + *sec = (time_t)intpart; + err = intpart - (double)*sec; + if (err <= -1.0 || err >= 1.0) { + error_time_t_overflow(); + return -1; + } + + *numerator = (long)floatpart; + return 0; +} + +static int _PyTime_ObjectToDenominator(PyObject *obj, time_t *sec, long *numerator, double denominator, _PyTime_round_t round) { assert(denominator <= LONG_MAX); if (PyFloat_Check(obj)) { - double d, intpart, err; - /* volatile avoids unsafe optimization on float enabled by gcc -O3 */ - volatile double floatpart; - - d = PyFloat_AsDouble(obj); - floatpart = modf(d, &intpart); - if (floatpart < 0) { - floatpart = 1.0 + floatpart; - intpart -= 1.0; - } - - floatpart *= denominator; - if (round == _PyTime_ROUND_CEILING) { - floatpart = ceil(floatpart); - if (floatpart >= denominator) { - floatpart = 0.0; - intpart += 1.0; - } - } - else { - floatpart = floor(floatpart); - } - - *sec = (time_t)intpart; - err = intpart - (double)*sec; - if (err <= -1.0 || err >= 1.0) { - error_time_t_overflow(); - return -1; - } - - *numerator = (long)floatpart; - return 0; + double d = PyFloat_AsDouble(obj); + return _PyTime_DoubleToDenominator(d, sec, numerator, + denominator, round); } else { *sec = _PyLong_AsTime_t(obj); @@ -221,29 +229,38 @@ #endif static int +_PyTime_FromFloatObject(_PyTime_t *t, double value, _PyTime_round_t round, + long to_nanoseconds) +{ + /* volatile avoids unsafe optimization on float enabled by gcc -O3 */ + volatile double d, err; + + /* convert to a number of nanoseconds */ + d = value; + d *= to_nanoseconds; + + if (round == _PyTime_ROUND_CEILING) + d = ceil(d); + else + d = floor(d); + + *t = (_PyTime_t)d; + err = d - (double)*t; + if (fabs(err) >= 1.0) { + _PyTime_overflow(); + return -1; + } + return 0; +} + +static int _PyTime_FromObject(_PyTime_t *t, PyObject *obj, _PyTime_round_t round, long to_nanoseconds) { if (PyFloat_Check(obj)) { - /* volatile avoids unsafe optimization on float enabled by gcc -O3 */ - volatile double d, err; - - /* convert to a number of nanoseconds */ + double d; d = PyFloat_AsDouble(obj); - d *= to_nanoseconds; - - if (round == _PyTime_ROUND_CEILING) - d = ceil(d); - else - d = floor(d); - - *t = (_PyTime_t)d; - err = d - (double)*t; - if (fabs(err) >= 1.0) { - _PyTime_overflow(); - return -1; - } - return 0; + return _PyTime_FromFloatObject(t, d, round, to_nanoseconds); } else { #ifdef HAVE_LONG_LONG -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 2 01:52:48 2015 From: python-checkins at python.org (victor.stinner) Date: Tue, 01 Sep 2015 23:52:48 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2323517=3A_Add_=22h?= =?utf-8?q?alf_up=22_rounding_mode_to_the_=5FPyTime_API?= Message-ID: <20150901235248.24923.80929@psf.io> https://hg.python.org/cpython/rev/abeb625b20c2 changeset: 97568:abeb625b20c2 user: Victor Stinner date: Wed Sep 02 01:43:56 2015 +0200 summary: Issue #23517: Add "half up" rounding mode to the _PyTime API files: Include/pytime.h | 5 +- Lib/test/test_time.py | 64 +++++++++++++++++++++++++- Modules/_testcapimodule.c | 4 +- Python/pytime.c | 64 ++++++++++++++++++++++---- 4 files changed, 122 insertions(+), 15 deletions(-) diff --git a/Include/pytime.h b/Include/pytime.h --- a/Include/pytime.h +++ b/Include/pytime.h @@ -30,7 +30,10 @@ _PyTime_ROUND_FLOOR=0, /* Round towards infinity (+inf). For example, used for timeout to wait "at least" N seconds. */ - _PyTime_ROUND_CEILING + _PyTime_ROUND_CEILING=1, + /* Round to nearest with ties going away from zero. + For example, used to round from a Python float. */ + _PyTime_ROUND_HALF_UP } _PyTime_round_t; /* Convert a time_t to a PyLong. */ diff --git a/Lib/test/test_time.py b/Lib/test/test_time.py --- a/Lib/test/test_time.py +++ b/Lib/test/test_time.py @@ -30,8 +30,11 @@ ROUND_FLOOR = 0 # Round towards infinity (+inf) ROUND_CEILING = 1 + # Round to nearest with ties going away from zero + ROUND_HALF_UP = 2 -ALL_ROUNDING_METHODS = (_PyTime.ROUND_FLOOR, _PyTime.ROUND_CEILING) +ALL_ROUNDING_METHODS = (_PyTime.ROUND_FLOOR, _PyTime.ROUND_CEILING, + _PyTime.ROUND_HALF_UP) class TimeTestCase(unittest.TestCase): @@ -753,11 +756,11 @@ (123.0, 123 * SEC_TO_NS), (-7.0, -7 * SEC_TO_NS), - # nanosecond are kept for value <= 2^23 seconds + # nanosecond are kept for value <= 2^23 seconds, + # except 2**23-1e-9 with HALF_UP (2**22 - 1e-9, 4194303999999999), (2**22, 4194304000000000), (2**22 + 1e-9, 4194304000000001), - (2**23 - 1e-9, 8388607999999999), (2**23, 8388608000000000), # start loosing precision for value > 2^23 seconds @@ -790,24 +793,36 @@ # Conversion giving different results depending on the rounding method FLOOR = _PyTime.ROUND_FLOOR CEILING = _PyTime.ROUND_CEILING + HALF_UP = _PyTime.ROUND_HALF_UP for obj, ts, rnd in ( # close to zero ( 1e-10, 0, FLOOR), ( 1e-10, 1, CEILING), + ( 1e-10, 0, HALF_UP), (-1e-10, -1, FLOOR), (-1e-10, 0, CEILING), + (-1e-10, 0, HALF_UP), # test rounding of the last nanosecond ( 1.1234567899, 1123456789, FLOOR), ( 1.1234567899, 1123456790, CEILING), + ( 1.1234567899, 1123456790, HALF_UP), (-1.1234567899, -1123456790, FLOOR), (-1.1234567899, -1123456789, CEILING), + (-1.1234567899, -1123456790, HALF_UP), # close to 1 second ( 0.9999999999, 999999999, FLOOR), ( 0.9999999999, 1000000000, CEILING), + ( 0.9999999999, 1000000000, HALF_UP), (-0.9999999999, -1000000000, FLOOR), (-0.9999999999, -999999999, CEILING), + (-0.9999999999, -1000000000, HALF_UP), + + # close to 2^23 seconds + (2**23 - 1e-9, 8388607999999999, FLOOR), + (2**23 - 1e-9, 8388607999999999, CEILING), + (2**23 - 1e-9, 8388608000000000, HALF_UP), ): with self.subTest(obj=obj, round=rnd, timestamp=ts): self.assertEqual(PyTime_FromSecondsObject(obj, rnd), ts) @@ -875,18 +890,33 @@ FLOOR = _PyTime.ROUND_FLOOR CEILING = _PyTime.ROUND_CEILING + HALF_UP = _PyTime.ROUND_HALF_UP for ns, tv, rnd in ( # nanoseconds (1, (0, 0), FLOOR), (1, (0, 1), CEILING), + (1, (0, 0), HALF_UP), (-1, (-1, 999999), FLOOR), (-1, (0, 0), CEILING), + (-1, (0, 0), HALF_UP), # seconds + nanoseconds (1234567001, (1, 234567), FLOOR), (1234567001, (1, 234568), CEILING), + (1234567001, (1, 234567), HALF_UP), (-1234567001, (-2, 765432), FLOOR), (-1234567001, (-2, 765433), CEILING), + (-1234567001, (-2, 765433), HALF_UP), + + # half up + (499, (0, 0), HALF_UP), + (500, (0, 1), HALF_UP), + (501, (0, 1), HALF_UP), + (999, (0, 1), HALF_UP), + (-499, (0, 0), HALF_UP), + (-500, (0, 0), HALF_UP), + (-501, (-1, 999999), HALF_UP), + (-999, (-1, 999999), HALF_UP), ): with self.subTest(nanoseconds=ns, timeval=tv, round=rnd): self.assertEqual(PyTime_AsTimeval(ns, rnd), tv) @@ -929,18 +959,33 @@ FLOOR = _PyTime.ROUND_FLOOR CEILING = _PyTime.ROUND_CEILING + HALF_UP = _PyTime.ROUND_HALF_UP for ns, ms, rnd in ( # nanoseconds (1, 0, FLOOR), (1, 1, CEILING), + (1, 0, HALF_UP), (-1, 0, FLOOR), (-1, -1, CEILING), + (-1, 0, HALF_UP), # seconds + nanoseconds (1234 * MS_TO_NS + 1, 1234, FLOOR), (1234 * MS_TO_NS + 1, 1235, CEILING), + (1234 * MS_TO_NS + 1, 1234, HALF_UP), (-1234 * MS_TO_NS - 1, -1234, FLOOR), (-1234 * MS_TO_NS - 1, -1235, CEILING), + (-1234 * MS_TO_NS - 1, -1234, HALF_UP), + + # half up + (499999, 0, HALF_UP), + (499999, 0, HALF_UP), + (500000, 1, HALF_UP), + (999999, 1, HALF_UP), + (-499999, 0, HALF_UP), + (-500000, -1, HALF_UP), + (-500001, -1, HALF_UP), + (-999999, -1, HALF_UP), ): with self.subTest(nanoseconds=ns, milliseconds=ms, round=rnd): self.assertEqual(PyTime_AsMilliseconds(ns, rnd), ms) @@ -966,18 +1011,31 @@ FLOOR = _PyTime.ROUND_FLOOR CEILING = _PyTime.ROUND_CEILING + HALF_UP = _PyTime.ROUND_HALF_UP for ns, ms, rnd in ( # nanoseconds (1, 0, FLOOR), (1, 1, CEILING), + (1, 0, HALF_UP), (-1, 0, FLOOR), (-1, -1, CEILING), + (-1, 0, HALF_UP), # seconds + nanoseconds (1234 * US_TO_NS + 1, 1234, FLOOR), (1234 * US_TO_NS + 1, 1235, CEILING), + (1234 * US_TO_NS + 1, 1234, HALF_UP), (-1234 * US_TO_NS - 1, -1234, FLOOR), (-1234 * US_TO_NS - 1, -1235, CEILING), + (-1234 * US_TO_NS - 1, -1234, HALF_UP), + + # half up + (1499, 1, HALF_UP), + (1500, 2, HALF_UP), + (1501, 2, HALF_UP), + (-1499, -1, HALF_UP), + (-1500, -2, HALF_UP), + (-1501, -2, HALF_UP), ): with self.subTest(nanoseconds=ns, milliseconds=ms, round=rnd): self.assertEqual(PyTime_AsMicroseconds(ns, rnd), ms) diff --git a/Modules/_testcapimodule.c b/Modules/_testcapimodule.c --- a/Modules/_testcapimodule.c +++ b/Modules/_testcapimodule.c @@ -2646,7 +2646,9 @@ static int check_time_rounding(int round) { - if (round != _PyTime_ROUND_FLOOR && round != _PyTime_ROUND_CEILING) { + if (round != _PyTime_ROUND_FLOOR + && round != _PyTime_ROUND_CEILING + && round != _PyTime_ROUND_HALF_UP) { PyErr_SetString(PyExc_ValueError, "invalid rounding"); return -1; } diff --git a/Python/pytime.c b/Python/pytime.c --- a/Python/pytime.c +++ b/Python/pytime.c @@ -60,6 +60,17 @@ #endif } +static double +_PyTime_RoundHalfUp(double x) +{ + if (x >= 0.0) + x = floor(x + 0.5); + else + x = ceil(x - 0.5); + return x; +} + + static int _PyTime_DoubleToDenominator(double d, time_t *sec, long *numerator, double denominator, _PyTime_round_t round) @@ -75,7 +86,9 @@ } floatpart *= denominator; - if (round == _PyTime_ROUND_CEILING) { + if (round == _PyTime_ROUND_HALF_UP) + floatpart = _PyTime_RoundHalfUp(floatpart); + else if (round == _PyTime_ROUND_CEILING) { floatpart = ceil(floatpart); if (floatpart >= denominator) { floatpart = 0.0; @@ -124,7 +137,9 @@ double d, intpart, err; d = PyFloat_AsDouble(obj); - if (round == _PyTime_ROUND_CEILING) + if (round == _PyTime_ROUND_HALF_UP) + d = _PyTime_RoundHalfUp(d); + else if (round == _PyTime_ROUND_CEILING) d = ceil(d); else d = floor(d); @@ -247,7 +262,9 @@ d = value; d *= to_nanoseconds; - if (round == _PyTime_ROUND_CEILING) + if (round == _PyTime_ROUND_HALF_UP) + d = _PyTime_RoundHalfUp(d); + else if (round == _PyTime_ROUND_CEILING) d = ceil(d); else d = floor(d); @@ -333,7 +350,19 @@ _PyTime_Divide(_PyTime_t t, _PyTime_t k, _PyTime_round_t round) { assert(k > 1); - if (round == _PyTime_ROUND_CEILING) { + if (round == _PyTime_ROUND_HALF_UP) { + _PyTime_t x, r; + x = t / k; + r = t % k; + if (Py_ABS(r) >= k / 2) { + if (t >= 0) + x++; + else + x--; + } + return x; + } + else if (round == _PyTime_ROUND_CEILING) { if (t >= 0) return (t + k - 1) / k; else @@ -359,8 +388,10 @@ _PyTime_AsTimeval_impl(_PyTime_t t, struct timeval *tv, _PyTime_round_t round, int raise) { + const long k = US_TO_NS; _PyTime_t secs, ns; int res = 0; + int usec; secs = t / SEC_TO_NS; ns = t % SEC_TO_NS; @@ -392,20 +423,33 @@ res = -1; #endif - if (round == _PyTime_ROUND_CEILING) - tv->tv_usec = (int)((ns + US_TO_NS - 1) / US_TO_NS); + if (round == _PyTime_ROUND_HALF_UP) { + _PyTime_t r; + usec = (int)(ns / k); + r = ns % k; + if (Py_ABS(r) >= k / 2) { + if (ns >= 0) + usec++; + else + usec--; + } + } + else if (round == _PyTime_ROUND_CEILING) + usec = (int)((ns + k - 1) / k); else - tv->tv_usec = (int)(ns / US_TO_NS); + usec = (int)(ns / k); - if (tv->tv_usec >= SEC_TO_US) { - tv->tv_usec -= SEC_TO_US; + if (usec >= SEC_TO_US) { + usec -= SEC_TO_US; tv->tv_sec += 1; } if (res && raise) _PyTime_overflow(); - assert(0 <= tv->tv_usec && tv->tv_usec <= 999999); + assert(0 <= usec && usec <= 999999); + + tv->tv_usec = usec; return res; } -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 2 01:57:38 2015 From: python-checkins at python.org (victor.stinner) Date: Tue, 01 Sep 2015 23:57:38 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2323517=3A_datetime?= =?utf-8?q?=2Edatetime=2Efromtimestamp=28=29_and?= Message-ID: <20150901235732.65942.80224@psf.io> https://hg.python.org/cpython/rev/b690bf218702 changeset: 97569:b690bf218702 user: Victor Stinner date: Wed Sep 02 01:57:23 2015 +0200 summary: Issue #23517: datetime.datetime.fromtimestamp() and datetime.datetime.utcfromtimestamp() now rounds to nearest with ties going away from zero, instead of rounding towards minus infinity (-inf), as Python 2 and Python older than 3.3. files: Misc/NEWS | 5 +++++ Modules/_datetimemodule.c | 2 +- 2 files changed, 6 insertions(+), 1 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -17,6 +17,11 @@ Library ------- +- Issue #23517: datetime.datetime.fromtimestamp() and + datetime.datetime.utcfromtimestamp() now rounds to nearest with ties going + away from zero, instead of rounding towards minus infinity (-inf), as Python + 2 and Python older than 3.3. + - Issue #23552: Timeit now warns when there is substantial (4x) variance between best and worst times. Patch from Serhiy Storchaka. diff --git a/Modules/_datetimemodule.c b/Modules/_datetimemodule.c --- a/Modules/_datetimemodule.c +++ b/Modules/_datetimemodule.c @@ -4098,7 +4098,7 @@ long us; if (_PyTime_ObjectToTimeval(timestamp, - &timet, &us, _PyTime_ROUND_FLOOR) == -1) + &timet, &us, _PyTime_ROUND_HALF_UP) == -1) return NULL; return datetime_from_timet_and_us(cls, f, timet, (int)us, tzinfo); -- Repository URL: https://hg.python.org/cpython From lp_benchmark_robot at intel.com Wed Sep 2 09:37:25 2015 From: lp_benchmark_robot at intel.com (lp_benchmark_robot) Date: Wed, 2 Sep 2015 07:37:25 +0000 Subject: [Python-checkins] Benchmark Results for Python Default 2015-09-02 Message-ID: <078AA0FFE8C7034097F90205717F504611D780F7@IRSMSX102.ger.corp.intel.com> Results for project python_default-nightly, build date 2015-09-02 06:02:09 commit: b690bf218702993e739a2c243470c9452cf77a8c revision date: 2015-09-02 02:57:23 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v3.4.3, with hash b4cbecbc0781e89a309d03b60a1f75f8499250e6 from 2015-02-25 12:15:33+00:00 ------------------------------------------------------------------------------------------ benchmark RSD* change since change since current rev with last run v3.4.3 regrtest PGO ------------------------------------------------------------------------------------------ :-) django_v2 0.43596% -1.46885% 6.88670% 19.37111% :-( pybench 0.15683% -0.24531% -2.67626% 9.94772% :-( regex_v8 3.00295% -0.36440% -3.54967% 4.93305% :-| nbody 0.08780% -0.27069% -1.10193% 9.56219% :-( json_dump_v2 0.28831% -1.51084% -2.90834% 14.38039% :-| normal_startup 0.68411% 0.15604% -0.34170% 5.16650% ------------------------------------------------------------------------------------------ Note: Benchmark results are measured in seconds. * Relative Standard Deviation (Standard Deviation/Average) Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. No license (express or implied, by estoppel or otherwise) to any intellectual property rights is granted by this document. Intel disclaims all express and implied warranties, including without limitation, the implied warranties of merchantability, fitness for a particular purpose, and non-infringement, as well as any warranty arising from course of performance, course of dealing, or usage in trade. This document may contain information on products, services and/or processes in development. Contact your Intel representative to obtain the latest forecast, schedule, specifications and roadmaps. The products and services described may contain defects or errors known as errata which may cause deviations from published specifications. Current characterized errata are available on request. (C) 2015 Intel Corporation. From python-checkins at python.org Wed Sep 2 10:40:58 2015 From: python-checkins at python.org (victor.stinner) Date: Wed, 02 Sep 2015 08:40:58 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2323517=3A_Fix_=5FP?= =?utf-8?q?yTime=5FObjectToDenominator=28=29?= Message-ID: <20150902084058.14256.94365@psf.io> https://hg.python.org/cpython/rev/700303850cd7 changeset: 97571:700303850cd7 user: Victor Stinner date: Wed Sep 02 10:37:46 2015 +0200 summary: Issue #23517: Fix _PyTime_ObjectToDenominator() * initialize numerator on overflow error ensure that numerator is smaller than * denominator. files: Python/pytime.c | 37 ++++++++++++++++++------------------- 1 files changed, 18 insertions(+), 19 deletions(-) diff --git a/Python/pytime.c b/Python/pytime.c --- a/Python/pytime.c +++ b/Python/pytime.c @@ -60,6 +60,7 @@ #endif } +/* Round to nearest with ties going away from zero (_PyTime_ROUND_HALF_UP). */ static double _PyTime_RoundHalfUp(double x) { @@ -81,32 +82,31 @@ floatpart = modf(d, &intpart); if (floatpart < 0) { - floatpart = 1.0 + floatpart; + floatpart += 1.0; intpart -= 1.0; } floatpart *= denominator; if (round == _PyTime_ROUND_HALF_UP) floatpart = _PyTime_RoundHalfUp(floatpart); - else if (round == _PyTime_ROUND_CEILING) { + else if (round == _PyTime_ROUND_CEILING) floatpart = ceil(floatpart); - if (floatpart >= denominator) { - floatpart = 0.0; - intpart += 1.0; - } + else + floatpart = floor(floatpart); + if (floatpart >= denominator) { + floatpart -= denominator; + intpart += 1.0; } - else { - floatpart = floor(floatpart); - } + assert(0.0 <= floatpart && floatpart < denominator); *sec = (time_t)intpart; + *numerator = (long)floatpart; + err = intpart - (double)*sec; if (err <= -1.0 || err >= 1.0) { error_time_t_overflow(); return -1; } - - *numerator = (long)floatpart; return 0; } @@ -123,9 +123,9 @@ } else { *sec = _PyLong_AsTime_t(obj); + *numerator = 0; if (*sec == (time_t)-1 && PyErr_Occurred()) return -1; - *numerator = 0; return 0; } } @@ -167,7 +167,7 @@ { int res; res = _PyTime_ObjectToDenominator(obj, sec, nsec, 1e9, round); - assert(0 <= *nsec && *nsec < SEC_TO_NS ); + assert(0 <= *nsec && *nsec < SEC_TO_NS); return res; } @@ -177,7 +177,7 @@ { int res; res = _PyTime_ObjectToDenominator(obj, sec, usec, 1e6, round); - assert(0 <= *usec && *usec < SEC_TO_US ); + assert(0 <= *usec && *usec < SEC_TO_US); return res; } @@ -444,12 +444,11 @@ tv->tv_sec += 1; } + assert(0 <= usec && usec < SEC_TO_US); + tv->tv_usec = usec; + if (res && raise) _PyTime_overflow(); - - assert(0 <= usec && usec <= 999999); - - tv->tv_usec = usec; return res; } @@ -484,7 +483,7 @@ } ts->tv_nsec = nsec; - assert(0 <= ts->tv_nsec && ts->tv_nsec <= 999999999); + assert(0 <= ts->tv_nsec && ts->tv_nsec < SEC_TO_NS); return 0; } #endif -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 2 10:40:58 2015 From: python-checkins at python.org (victor.stinner) Date: Wed, 02 Sep 2015 08:40:58 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2323517=3A_Reintrod?= =?utf-8?q?uce_unit_tests_for_the_old_PyTime_API_since_it=27s_still?= Message-ID: <20150902084058.32111.19323@psf.io> https://hg.python.org/cpython/rev/03c97bb04cd2 changeset: 97572:03c97bb04cd2 user: Victor Stinner date: Wed Sep 02 10:39:40 2015 +0200 summary: Issue #23517: Reintroduce unit tests for the old PyTime API since it's still used. files: Lib/test/test_time.py | 154 ++++++++++++++++++++++++++++++ 1 files changed, 154 insertions(+), 0 deletions(-) diff --git a/Lib/test/test_time.py b/Lib/test/test_time.py --- a/Lib/test/test_time.py +++ b/Lib/test/test_time.py @@ -727,6 +727,9 @@ @unittest.skipUnless(_testcapi is not None, 'need the _testcapi module') class TestPyTime_t(unittest.TestCase): + """ + Test the _PyTime_t API. + """ def test_FromSeconds(self): from _testcapi import PyTime_FromSeconds for seconds in (0, 3, -456, _testcapi.INT_MAX, _testcapi.INT_MIN): @@ -1041,5 +1044,156 @@ self.assertEqual(PyTime_AsMicroseconds(ns, rnd), ms) + at unittest.skipUnless(_testcapi is not None, + 'need the _testcapi module') +class TestOldPyTime(unittest.TestCase): + """ + Test the old _PyTime_t API: _PyTime_ObjectToXXX() functions. + """ + def setUp(self): + self.invalid_values = ( + -(2 ** 100), 2 ** 100, + -(2.0 ** 100.0), 2.0 ** 100.0, + ) + + @support.cpython_only + def test_time_t(self): + from _testcapi import pytime_object_to_time_t + + # Conversion giving the same result for all rounding methods + for rnd in ALL_ROUNDING_METHODS: + for obj, time_t in ( + # int + (0, 0), + (-1, -1), + + # float + (1.0, 1), + (-1.0, -1), + ): + self.assertEqual(pytime_object_to_time_t(obj, rnd), time_t) + + + # Conversion giving different results depending on the rounding method + FLOOR = _PyTime.ROUND_FLOOR + CEILING = _PyTime.ROUND_CEILING + HALF_UP = _PyTime.ROUND_HALF_UP + for obj, time_t, rnd in ( + (-1.9, -2, FLOOR), + (-1.9, -2, HALF_UP), + (-1.9, -1, CEILING), + (1.9, 1, FLOOR), + (1.9, 2, HALF_UP), + (1.9, 2, CEILING), + ): + self.assertEqual(pytime_object_to_time_t(obj, rnd), time_t) + + # Test OverflowError + rnd = _PyTime.ROUND_FLOOR + for invalid in self.invalid_values: + self.assertRaises(OverflowError, + pytime_object_to_time_t, invalid, rnd) + + def test_timeval(self): + from _testcapi import pytime_object_to_timeval + + # Conversion giving the same result for all rounding methods + for rnd in ALL_ROUNDING_METHODS: + for obj, timeval in ( + # int + (0, (0, 0)), + (-1, (-1, 0)), + + # float + (-1.0, (-1, 0)), + (-1.2, (-2, 800000)), + (-1e-6, (-1, 999999)), + (1e-6, (0, 1)), + ): + with self.subTest(obj=obj, round=rnd, timeval=timeval): + self.assertEqual(pytime_object_to_timeval(obj, rnd), + timeval) + + # Conversion giving different results depending on the rounding method + FLOOR = _PyTime.ROUND_FLOOR + CEILING = _PyTime.ROUND_CEILING + HALF_UP = _PyTime.ROUND_HALF_UP + for obj, timeval, rnd in ( + (-1e-7, (-1, 999999), FLOOR), + (-1e-7, (0, 0), CEILING), + (-1e-7, (0, 0), HALF_UP), + + (1e-7, (0, 0), FLOOR), + (1e-7, (0, 1), CEILING), + (1e-7, (0, 0), HALF_UP), + + (0.4e-6, (0, 0), HALF_UP), + (0.5e-6, (0, 1), HALF_UP), + (0.6e-6, (0, 1), HALF_UP), + + (0.9999999, (0, 999999), FLOOR), + (0.9999999, (1, 0), CEILING), + (0.9999999, (1, 0), HALF_UP), + ): + with self.subTest(obj=obj, round=rnd, timeval=timeval): + self.assertEqual(pytime_object_to_timeval(obj, rnd), timeval) + + rnd = _PyTime.ROUND_FLOOR + for invalid in self.invalid_values: + self.assertRaises(OverflowError, + pytime_object_to_timeval, invalid, rnd) + + @support.cpython_only + def test_timespec(self): + from _testcapi import pytime_object_to_timespec + + # Conversion giving the same result for all rounding methods + for rnd in ALL_ROUNDING_METHODS: + for obj, timespec in ( + # int + (0, (0, 0)), + (-1, (-1, 0)), + + # float + (-1.0, (-1, 0)), + (-1e-9, (-1, 999999999)), + (1e-9, (0, 1)), + (-1.2, (-2, 800000000)), + ): + with self.subTest(obj=obj, round=rnd, timespec=timespec): + self.assertEqual(pytime_object_to_timespec(obj, rnd), + timespec) + + # Conversion giving different results depending on the rounding method + FLOOR = _PyTime.ROUND_FLOOR + CEILING = _PyTime.ROUND_CEILING + HALF_UP = _PyTime.ROUND_HALF_UP + for obj, timespec, rnd in ( + (-1e-10, (-1, 999999999), FLOOR), + (-1e-10, (0, 0), CEILING), + (-1e-10, (0, 0), HALF_UP), + + (1e-10, (0, 0), FLOOR), + (1e-10, (0, 1), CEILING), + (1e-10, (0, 0), HALF_UP), + + (0.4e-9, (0, 0), HALF_UP), + (0.5e-9, (0, 1), HALF_UP), + (0.6e-9, (0, 1), HALF_UP), + + (0.9999999999, (0, 999999999), FLOOR), + (0.9999999999, (1, 0), CEILING), + (0.9999999999, (1, 0), HALF_UP), + ): + with self.subTest(obj=obj, round=rnd, timespec=timespec): + self.assertEqual(pytime_object_to_timespec(obj, rnd), timespec) + + # Test OverflowError + rnd = FLOOR + for invalid in self.invalid_values: + self.assertRaises(OverflowError, + pytime_object_to_timespec, invalid, rnd) + + if __name__ == "__main__": unittest.main() -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 2 10:41:02 2015 From: python-checkins at python.org (victor.stinner) Date: Wed, 02 Sep 2015 08:41:02 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Backed_out_changeset_b690b?= =?utf-8?q?f218702?= Message-ID: <20150902084058.130452.89199@psf.io> https://hg.python.org/cpython/rev/30454ef98e81 changeset: 97570:30454ef98e81 user: Victor Stinner date: Wed Sep 02 10:10:26 2015 +0200 summary: Backed out changeset b690bf218702 Issue #23517: the change broke test_datetime. datetime.timedelta() rounding mode must also be changed, and test_datetime must be updated for the new rounding mode (half up). files: Misc/NEWS | 5 ----- Modules/_datetimemodule.c | 2 +- 2 files changed, 1 insertions(+), 6 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -17,11 +17,6 @@ Library ------- -- Issue #23517: datetime.datetime.fromtimestamp() and - datetime.datetime.utcfromtimestamp() now rounds to nearest with ties going - away from zero, instead of rounding towards minus infinity (-inf), as Python - 2 and Python older than 3.3. - - Issue #23552: Timeit now warns when there is substantial (4x) variance between best and worst times. Patch from Serhiy Storchaka. diff --git a/Modules/_datetimemodule.c b/Modules/_datetimemodule.c --- a/Modules/_datetimemodule.c +++ b/Modules/_datetimemodule.c @@ -4098,7 +4098,7 @@ long us; if (_PyTime_ObjectToTimeval(timestamp, - &timet, &us, _PyTime_ROUND_HALF_UP) == -1) + &timet, &us, _PyTime_ROUND_FLOOR) == -1) return NULL; return datetime_from_timet_and_us(cls, f, timet, (int)us, tzinfo); -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 2 11:05:46 2015 From: python-checkins at python.org (victor.stinner) Date: Wed, 02 Sep 2015 09:05:46 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_test=5Ftime=3A_add_more_te?= =?utf-8?q?sts_on_HALF=5FUP_rounding_mode?= Message-ID: <20150902090546.24947.20802@psf.io> https://hg.python.org/cpython/rev/3a361a56b88c changeset: 97573:3a361a56b88c user: Victor Stinner date: Wed Sep 02 11:05:32 2015 +0200 summary: test_time: add more tests on HALF_UP rounding mode files: Lib/test/test_time.py | 36 ++++++++++++++++++++++++------ 1 files changed, 28 insertions(+), 8 deletions(-) diff --git a/Lib/test/test_time.py b/Lib/test/test_time.py --- a/Lib/test/test_time.py +++ b/Lib/test/test_time.py @@ -1082,9 +1082,18 @@ (-1.9, -2, FLOOR), (-1.9, -2, HALF_UP), (-1.9, -1, CEILING), + (1.9, 1, FLOOR), (1.9, 2, HALF_UP), (1.9, 2, CEILING), + + (-0.6, -1, HALF_UP), + (-0.5, -1, HALF_UP), + (-0.4, 0, HALF_UP), + + (0.4, 0, HALF_UP), + (0.5, 1, HALF_UP), + (0.6, 1, HALF_UP), ): self.assertEqual(pytime_object_to_time_t(obj, rnd), time_t) @@ -1127,13 +1136,19 @@ (1e-7, (0, 1), CEILING), (1e-7, (0, 0), HALF_UP), - (0.4e-6, (0, 0), HALF_UP), - (0.5e-6, (0, 1), HALF_UP), - (0.6e-6, (0, 1), HALF_UP), - (0.9999999, (0, 999999), FLOOR), (0.9999999, (1, 0), CEILING), (0.9999999, (1, 0), HALF_UP), + + (-0.6e-6, (-1, 999999), HALF_UP), + # skipped, -0.5e-6 is inexact in base 2 + #(-0.5e-6, (-1, 999999), HALF_UP), + (-0.4e-6, (0, 0), HALF_UP), + + (0.4e-6, (0, 0), HALF_UP), + # skipped, 0.5e-6 is inexact in base 2 + #(0.5e-6, (0, 1), HALF_UP), + (0.6e-6, (0, 1), HALF_UP), ): with self.subTest(obj=obj, round=rnd, timeval=timeval): self.assertEqual(pytime_object_to_timeval(obj, rnd), timeval) @@ -1177,13 +1192,18 @@ (1e-10, (0, 1), CEILING), (1e-10, (0, 0), HALF_UP), + (0.9999999999, (0, 999999999), FLOOR), + (0.9999999999, (1, 0), CEILING), + (0.9999999999, (1, 0), HALF_UP), + + (-0.6e-9, (-1, 999999999), HALF_UP), + # skipped, 0.5e-6 is inexact in base 2 + #(-0.5e-9, (-1, 999999999), HALF_UP), + (-0.4e-9, (0, 0), HALF_UP), + (0.4e-9, (0, 0), HALF_UP), (0.5e-9, (0, 1), HALF_UP), (0.6e-9, (0, 1), HALF_UP), - - (0.9999999999, (0, 999999999), FLOOR), - (0.9999999999, (1, 0), CEILING), - (0.9999999999, (1, 0), HALF_UP), ): with self.subTest(obj=obj, round=rnd, timespec=timespec): self.assertEqual(pytime_object_to_timespec(obj, rnd), timespec) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 2 12:01:42 2015 From: python-checkins at python.org (victor.stinner) Date: Wed, 02 Sep 2015 10:01:42 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2323517=3A_Try_to_f?= =?utf-8?q?ix_test=5Ftime_on_=22x86_Ubuntu_Shared_3=2Ex=22_buildbot?= Message-ID: <20150902100142.32095.55331@psf.io> https://hg.python.org/cpython/rev/df074eb2a5be changeset: 97574:df074eb2a5be user: Victor Stinner date: Wed Sep 02 11:58:56 2015 +0200 summary: Issue #23517: Try to fix test_time on "x86 Ubuntu Shared 3.x" buildbot files: Python/pytime.c | 17 ++++++++++------- 1 files changed, 10 insertions(+), 7 deletions(-) diff --git a/Python/pytime.c b/Python/pytime.c --- a/Python/pytime.c +++ b/Python/pytime.c @@ -64,11 +64,13 @@ static double _PyTime_RoundHalfUp(double x) { - if (x >= 0.0) - x = floor(x + 0.5); + /* volatile avoids optimization changing how numbers are rounded */ + volatile double d = x; + if (d >= 0.0) + d = floor(d + 0.5); else - x = ceil(x - 0.5); - return x; + d = ceil(d - 0.5); + return d; } @@ -77,7 +79,7 @@ double denominator, _PyTime_round_t round) { double intpart, err; - /* volatile avoids unsafe optimization on float enabled by gcc -O3 */ + /* volatile avoids optimization changing how numbers are rounded */ volatile double floatpart; floatpart = modf(d, &intpart); @@ -134,7 +136,8 @@ _PyTime_ObjectToTime_t(PyObject *obj, time_t *sec, _PyTime_round_t round) { if (PyFloat_Check(obj)) { - double d, intpart, err; + /* volatile avoids optimization changing how numbers are rounded */ + volatile double d, intpart, err; d = PyFloat_AsDouble(obj); if (round == _PyTime_ROUND_HALF_UP) @@ -255,7 +258,7 @@ _PyTime_FromFloatObject(_PyTime_t *t, double value, _PyTime_round_t round, long to_nanoseconds) { - /* volatile avoids unsafe optimization on float enabled by gcc -O3 */ + /* volatile avoids optimization changing how numbers are rounded */ volatile double d, err; /* convert to a number of nanoseconds */ -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 2 13:54:40 2015 From: python-checkins at python.org (victor.stinner) Date: Wed, 02 Sep 2015 11:54:40 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2323517=3A_test=5Ft?= =?utf-8?q?ime=2C_skip_a_test_checking_a_corner_case_on_floating_point?= Message-ID: <20150902115440.32097.65437@psf.io> https://hg.python.org/cpython/rev/59185ef69166 changeset: 97575:59185ef69166 user: Victor Stinner date: Wed Sep 02 13:54:28 2015 +0200 summary: Issue #23517: test_time, skip a test checking a corner case on floating point rounding files: Lib/test/test_time.py | 4 +++- 1 files changed, 3 insertions(+), 1 deletions(-) diff --git a/Lib/test/test_time.py b/Lib/test/test_time.py --- a/Lib/test/test_time.py +++ b/Lib/test/test_time.py @@ -825,7 +825,9 @@ # close to 2^23 seconds (2**23 - 1e-9, 8388607999999999, FLOOR), (2**23 - 1e-9, 8388607999999999, CEILING), - (2**23 - 1e-9, 8388608000000000, HALF_UP), + # Issue #23517: skip HALF_UP test because the result is different + # depending on the FPU and how the compiler optimize the code :-/ + #(2**23 - 1e-9, 8388608000000000, HALF_UP), ): with self.subTest(obj=obj, round=rnd, timestamp=ts): self.assertEqual(PyTime_FromSecondsObject(obj, rnd), ts) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 2 14:35:04 2015 From: python-checkins at python.org (victor.stinner) Date: Wed, 02 Sep 2015 12:35:04 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_24297=3A_Fix_test=5F?= =?utf-8?q?symbol_on_Windows?= Message-ID: <20150902123504.24943.41235@psf.io> https://hg.python.org/cpython/rev/bf7ef3bd9a09 changeset: 97576:bf7ef3bd9a09 user: Victor Stinner date: Wed Sep 02 14:23:40 2015 +0200 summary: Issue 24297: Fix test_symbol on Windows Don't rely on end of line. Open files in text mode, not in binary mode. files: Lib/test/test_symbol.py | 35 +++++++++++++++++----------- 1 files changed, 21 insertions(+), 14 deletions(-) diff --git a/Lib/test/test_symbol.py b/Lib/test/test_symbol.py --- a/Lib/test/test_symbol.py +++ b/Lib/test/test_symbol.py @@ -15,12 +15,11 @@ class TestSymbolGeneration(unittest.TestCase): def _copy_file_without_generated_symbols(self, source_file, dest_file): - with open(source_file, 'rb') as fp: + with open(source_file) as fp: lines = fp.readlines() - nl = lines[0][len(lines[0].rstrip()):] - with open(dest_file, 'wb') as fp: - fp.writelines(lines[:lines.index(b"#--start constants--" + nl) + 1]) - fp.writelines(lines[lines.index(b"#--end constants--" + nl):]) + with open(dest_file, 'w') as fp: + fp.writelines(lines[:lines.index("#--start constants--\n") + 1]) + fp.writelines(lines[lines.index("#--end constants--\n"):]) def _generate_symbols(self, grammar_file, target_symbol_py_file): proc = subprocess.Popen([sys.executable, @@ -30,18 +29,26 @@ stderr = proc.communicate()[1] return proc.returncode, stderr + def compare_files(self, file1, file2): + with open(file1) as fp: + lines1 = fp.readlines() + with open(file2) as fp: + lines2 = fp.readlines() + self.assertEqual(lines1, lines2) + @unittest.skipIf(not os.path.exists(GRAMMAR_FILE), 'test only works from source build directory') def test_real_grammar_and_symbol_file(self): - self._copy_file_without_generated_symbols(SYMBOL_FILE, TEST_PY_FILE) - self.addCleanup(support.unlink, TEST_PY_FILE) - self.assertFalse(filecmp.cmp(SYMBOL_FILE, TEST_PY_FILE)) - self.assertEqual((0, b''), self._generate_symbols(GRAMMAR_FILE, - TEST_PY_FILE)) - self.assertTrue(filecmp.cmp(SYMBOL_FILE, TEST_PY_FILE), - 'symbol stat: %r\ntest_py stat: %r\n' % - (os.stat(SYMBOL_FILE), - os.stat(TEST_PY_FILE))) + output = support.TESTFN + self.addCleanup(support.unlink, output) + + self._copy_file_without_generated_symbols(SYMBOL_FILE, output) + + exitcode, stderr = self._generate_symbols(GRAMMAR_FILE, output) + self.assertEqual(b'', stderr) + self.assertEqual(0, exitcode) + + self.compare_files(SYMBOL_FILE, output) if __name__ == "__main__": -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 2 15:41:42 2015 From: python-checkins at python.org (victor.stinner) Date: Wed, 02 Sep 2015 13:41:42 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge_3=2E5_=28asyncio_doc=29?= Message-ID: <20150902134142.14236.42032@psf.io> https://hg.python.org/cpython/rev/b3b637a42d59 changeset: 97579:b3b637a42d59 parent: 97576:bf7ef3bd9a09 parent: 97578:1b22fa5f91ce user: Victor Stinner date: Wed Sep 02 15:41:08 2015 +0200 summary: Merge 3.5 (asyncio doc) files: Doc/library/asyncio-subprocess.rst | 8 ++++---- 1 files changed, 4 insertions(+), 4 deletions(-) diff --git a/Doc/library/asyncio-subprocess.rst b/Doc/library/asyncio-subprocess.rst --- a/Doc/library/asyncio-subprocess.rst +++ b/Doc/library/asyncio-subprocess.rst @@ -303,7 +303,7 @@ .. _asyncio-subprocess-threads: Subprocess and threads -====================== +---------------------- asyncio supports running subprocesses from different threads, but there are limits: @@ -322,10 +322,10 @@ Subprocess examples -=================== +------------------- Subprocess using transport and protocol ---------------------------------------- +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Example of a subprocess protocol using to get the output of a subprocess and to wait for the subprocess exit. The subprocess is created by the @@ -381,7 +381,7 @@ Subprocess using streams ------------------------- +^^^^^^^^^^^^^^^^^^^^^^^^ Example using the :class:`~asyncio.subprocess.Process` class to control the subprocess and the :class:`StreamReader` class to read from the standard -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 2 15:41:44 2015 From: python-checkins at python.org (victor.stinner) Date: Wed, 02 Sep 2015 13:41:44 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Merge_3=2E4_=28asyncio_doc=29?= Message-ID: <20150902134142.38881.83073@psf.io> https://hg.python.org/cpython/rev/1b22fa5f91ce changeset: 97578:1b22fa5f91ce branch: 3.5 parent: 97564:447d8e6b17b9 parent: 97577:d65d3222538d user: Victor Stinner date: Wed Sep 02 15:40:56 2015 +0200 summary: Merge 3.4 (asyncio doc) files: Doc/library/asyncio-subprocess.rst | 8 ++++---- 1 files changed, 4 insertions(+), 4 deletions(-) diff --git a/Doc/library/asyncio-subprocess.rst b/Doc/library/asyncio-subprocess.rst --- a/Doc/library/asyncio-subprocess.rst +++ b/Doc/library/asyncio-subprocess.rst @@ -303,7 +303,7 @@ .. _asyncio-subprocess-threads: Subprocess and threads -====================== +---------------------- asyncio supports running subprocesses from different threads, but there are limits: @@ -322,10 +322,10 @@ Subprocess examples -=================== +------------------- Subprocess using transport and protocol ---------------------------------------- +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Example of a subprocess protocol using to get the output of a subprocess and to wait for the subprocess exit. The subprocess is created by the @@ -381,7 +381,7 @@ Subprocess using streams ------------------------- +^^^^^^^^^^^^^^^^^^^^^^^^ Example using the :class:`~asyncio.subprocess.Process` class to control the subprocess and the :class:`StreamReader` class to read from the standard -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 2 15:41:44 2015 From: python-checkins at python.org (victor.stinner) Date: Wed, 02 Sep 2015 13:41:44 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogYXN5bmNpbyBkb2M6?= =?utf-8?q?_fix_subprocess_sections?= Message-ID: <20150902134142.106008.75909@psf.io> https://hg.python.org/cpython/rev/d65d3222538d changeset: 97577:d65d3222538d branch: 3.4 parent: 97557:328383905eaf user: Victor Stinner date: Wed Sep 02 15:39:01 2015 +0200 summary: asyncio doc: fix subprocess sections files: Doc/library/asyncio-subprocess.rst | 8 ++++---- 1 files changed, 4 insertions(+), 4 deletions(-) diff --git a/Doc/library/asyncio-subprocess.rst b/Doc/library/asyncio-subprocess.rst --- a/Doc/library/asyncio-subprocess.rst +++ b/Doc/library/asyncio-subprocess.rst @@ -303,7 +303,7 @@ .. _asyncio-subprocess-threads: Subprocess and threads -====================== +---------------------- asyncio supports running subprocesses from different threads, but there are limits: @@ -322,10 +322,10 @@ Subprocess examples -=================== +------------------- Subprocess using transport and protocol ---------------------------------------- +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Example of a subprocess protocol using to get the output of a subprocess and to wait for the subprocess exit. The subprocess is created by the @@ -381,7 +381,7 @@ Subprocess using streams ------------------------- +^^^^^^^^^^^^^^^^^^^^^^^^ Example using the :class:`~asyncio.subprocess.Process` class to control the subprocess and the :class:`StreamReader` class to read from the standard -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 2 15:45:00 2015 From: python-checkins at python.org (victor.stinner) Date: Wed, 02 Sep 2015 13:45:00 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_test=5Fgdb=3A_add_debug_in?= =?utf-8?q?fo_to_investigate_failure_on_=22s390x_SLES_3=2Ex=22_buildbot?= Message-ID: <20150902134454.130460.18348@psf.io> https://hg.python.org/cpython/rev/ba4346a83019 changeset: 97580:ba4346a83019 user: Victor Stinner date: Wed Sep 02 15:44:22 2015 +0200 summary: test_gdb: add debug info to investigate failure on "s390x SLES 3.x" buildbot files: Lib/test/test_gdb.py | 10 +++++++--- 1 files changed, 7 insertions(+), 3 deletions(-) diff --git a/Lib/test/test_gdb.py b/Lib/test/test_gdb.py --- a/Lib/test/test_gdb.py +++ b/Lib/test/test_gdb.py @@ -28,9 +28,13 @@ # This is what "no gdb" looks like. There may, however, be other # errors that manifest this way too. raise unittest.SkipTest("Couldn't find gdb on the path") -gdb_version_number = re.search(b"^GNU gdb [^\d]*(\d+)\.(\d)", gdb_version) -gdb_major_version = int(gdb_version_number.group(1)) -gdb_minor_version = int(gdb_version_number.group(2)) +try: + gdb_version_number = re.search(b"^GNU gdb [^\d]*(\d+)\.(\d)", gdb_version) + gdb_major_version = int(gdb_version_number.group(1)) + gdb_minor_version = int(gdb_version_number.group(2)) +except Exception: + raise ValueError("unable to parse GDB version: %r" % gdb_version) + if gdb_major_version < 7: raise unittest.SkipTest("gdb versions before 7.0 didn't support python embedding" " Saw:\n" + gdb_version.decode('ascii', 'replace')) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 2 15:46:20 2015 From: python-checkins at python.org (victor.stinner) Date: Wed, 02 Sep 2015 13:46:20 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_test=5Fgdb=3A_fix_Resource?= =?utf-8?q?Warning_if_the_test_is_interrupted?= Message-ID: <20150902134615.8995.11093@psf.io> https://hg.python.org/cpython/rev/c664f58a50f5 changeset: 97581:c664f58a50f5 user: Victor Stinner date: Wed Sep 02 15:46:00 2015 +0200 summary: test_gdb: fix ResourceWarning if the test is interrupted files: Lib/test/test_gdb.py | 8 +++++--- 1 files changed, 5 insertions(+), 3 deletions(-) diff --git a/Lib/test/test_gdb.py b/Lib/test/test_gdb.py --- a/Lib/test/test_gdb.py +++ b/Lib/test/test_gdb.py @@ -63,9 +63,11 @@ base_cmd = ('gdb', '--batch', '-nx') if (gdb_major_version, gdb_minor_version) >= (7, 4): base_cmd += ('-iex', 'add-auto-load-safe-path ' + checkout_hook_path) - out, err = subprocess.Popen(base_cmd + args, - stdout=subprocess.PIPE, stderr=subprocess.PIPE, env=env, - ).communicate() + proc = subprocess.Popen(base_cmd + args, + stdout=subprocess.PIPE, + stderr=subprocess.PIPE, env=env) + with proc: + out, err = proc.communicate() return out.decode('utf-8', 'replace'), err.decode('utf-8', 'replace') # Verify that "gdb" was built with the embedded python support enabled: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 2 17:33:31 2015 From: python-checkins at python.org (victor.stinner) Date: Wed, 02 Sep 2015 15:33:31 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_test=5Feintr=3A_try_to_deb?= =?utf-8?q?ug_hang_on_FreeBSD?= Message-ID: <20150902153331.28510.50639@psf.io> https://hg.python.org/cpython/rev/ebccac60b9e7 changeset: 97582:ebccac60b9e7 user: Victor Stinner date: Wed Sep 02 17:19:04 2015 +0200 summary: test_eintr: try to debug hang on FreeBSD files: Lib/test/eintrdata/eintr_tester.py | 8 ++++++++ 1 files changed, 8 insertions(+), 0 deletions(-) diff --git a/Lib/test/eintrdata/eintr_tester.py b/Lib/test/eintrdata/eintr_tester.py --- a/Lib/test/eintrdata/eintr_tester.py +++ b/Lib/test/eintrdata/eintr_tester.py @@ -8,6 +8,7 @@ sub-second periodicity (contrarily to signal()). """ +import faulthandler import io import os import select @@ -36,10 +37,17 @@ cls.orig_handler = signal.signal(signal.SIGALRM, lambda *args: None) signal.setitimer(signal.ITIMER_REAL, cls.signal_delay, cls.signal_period) + if hasattr(faulthandler, 'dump_traceback_later'): + # Most tests take less than 30 seconds, so 15 minutes should be + # enough. dump_traceback_later() is implemented with a thread, but + # pthread_sigmask() is used to mask all signaled on this thread. + faulthandler.dump_traceback_later(15 * 60, exit=True) @classmethod def stop_alarm(cls): signal.setitimer(signal.ITIMER_REAL, 0, 0) + if hasattr(faulthandler, 'cancel_dump_traceback_later'): + faulthandler.cancel_dump_traceback_later() @classmethod def tearDownClass(cls): -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 2 21:00:45 2015 From: python-checkins at python.org (donald.stufft) Date: Wed, 02 Sep 2015 19:00:45 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_Accept_PEP_470?= Message-ID: <20150902190044.14264.53179@psf.io> https://hg.python.org/peps/rev/d7cc3a1e7c9f changeset: 6026:d7cc3a1e7c9f user: Donald Stufft date: Wed Sep 02 15:00:42 2015 -0400 summary: Accept PEP 470 files: pep-0470.txt | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-) diff --git a/pep-0470.txt b/pep-0470.txt --- a/pep-0470.txt +++ b/pep-0470.txt @@ -5,12 +5,13 @@ Author: Donald Stufft BDFL-Delegate: Paul Moore Discussions-To: distutils-sig at python.org -Status: Draft +Status: Accepted Type: Process Content-Type: text/x-rst Created: 12-May-2014 Post-History: 14-May-2014, 05-Jun-2014, 03-Oct-2014, 13-Oct-2014, 26-Aug-2015 Replaces: 438 +Resolution: https://mail.python.org/pipermail/distutils-sig/2015-September/026789.html Abstract -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Wed Sep 2 21:50:34 2015 From: python-checkins at python.org (yury.selivanov) Date: Wed, 02 Sep 2015 19:50:34 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI0OTc1?= =?utf-8?q?=3A_Fix_AST_compilation_for_PEP_448_syntax=2E?= Message-ID: <20150902195034.17977.36160@psf.io> https://hg.python.org/cpython/rev/e9df8543d7bc changeset: 97583:e9df8543d7bc branch: 3.5 parent: 97541:dcfe871b56ac user: Yury Selivanov date: Tue Sep 01 16:10:49 2015 -0400 summary: Issue #24975: Fix AST compilation for PEP 448 syntax. files: Lib/test/test_ast.py | 24 +++++++++++++++--------- Misc/NEWS | 2 ++ Python/ast.c | 6 ++++-- 3 files changed, 21 insertions(+), 11 deletions(-) diff --git a/Lib/test/test_ast.py b/Lib/test/test_ast.py --- a/Lib/test/test_ast.py +++ b/Lib/test/test_ast.py @@ -78,9 +78,9 @@ # Pass, "pass", # Break - "break", + "for v in v:break", # Continue - "continue", + "for v in v:continue", # for statements with naked tuples (see http://bugs.python.org/issue6704) "for a,b in c: pass", "[(a,b) for a,b in c]", @@ -112,6 +112,9 @@ "async def f():\n async for e in i: 1\n else: 2", # AsyncWith "async def f():\n async with a as b: 1", + # PEP 448: Additional Unpacking Generalizations + "{**{1:2}, 2:3}", + "{*{1, 2}, 3}", ] # These are compiled through "single" @@ -231,9 +234,12 @@ (single_tests, single_results, "single"), (eval_tests, eval_results, "eval")): for i, o in zip(input, output): - ast_tree = compile(i, "?", kind, ast.PyCF_ONLY_AST) - self.assertEqual(to_tuple(ast_tree), o) - self._assertTrueorder(ast_tree, (0, 0)) + with self.subTest(action="parsing", input=i): + ast_tree = compile(i, "?", kind, ast.PyCF_ONLY_AST) + self.assertEqual(to_tuple(ast_tree), o) + self._assertTrueorder(ast_tree, (0, 0)) + with self.subTest(action="compiling", input=i): + compile(ast_tree, "?", kind) def test_slice(self): slc = ast.parse("x[::]").body[0].value.slice @@ -780,8 +786,6 @@ def test_dict(self): d = ast.Dict([], [ast.Name("x", ast.Load())]) self.expr(d, "same number of keys as values") - d = ast.Dict([None], [ast.Name("x", ast.Load())]) - self.expr(d, "None disallowed") d = ast.Dict([ast.Name("x", ast.Load())], [None]) self.expr(d, "None disallowed") @@ -972,8 +976,8 @@ ('Module', [('Global', (1, 0), ['v'])]), ('Module', [('Expr', (1, 0), ('Num', (1, 0), 1))]), ('Module', [('Pass', (1, 0))]), -('Module', [('Break', (1, 0))]), -('Module', [('Continue', (1, 0))]), +('Module', [('For', (1, 0), ('Name', (1, 4), 'v', ('Store',)), ('Name', (1, 9), 'v', ('Load',)), [('Break', (1, 11))], [])]), +('Module', [('For', (1, 0), ('Name', (1, 4), 'v', ('Store',)), ('Name', (1, 9), 'v', ('Load',)), [('Continue', (1, 11))], [])]), ('Module', [('For', (1, 0), ('Tuple', (1, 4), [('Name', (1, 4), 'a', ('Store',)), ('Name', (1, 6), 'b', ('Store',))], ('Store',)), ('Name', (1, 11), 'c', ('Load',)), [('Pass', (1, 14))], [])]), ('Module', [('Expr', (1, 0), ('ListComp', (1, 1), ('Tuple', (1, 2), [('Name', (1, 2), 'a', ('Load',)), ('Name', (1, 4), 'b', ('Load',))], ('Load',)), [('comprehension', ('Tuple', (1, 11), [('Name', (1, 11), 'a', ('Store',)), ('Name', (1, 13), 'b', ('Store',))], ('Store',)), ('Name', (1, 18), 'c', ('Load',)), [])]))]), ('Module', [('Expr', (1, 0), ('GeneratorExp', (1, 1), ('Tuple', (1, 2), [('Name', (1, 2), 'a', ('Load',)), ('Name', (1, 4), 'b', ('Load',))], ('Load',)), [('comprehension', ('Tuple', (1, 11), [('Name', (1, 11), 'a', ('Store',)), ('Name', (1, 13), 'b', ('Store',))], ('Store',)), ('Name', (1, 18), 'c', ('Load',)), [])]))]), @@ -986,6 +990,8 @@ ('Module', [('AsyncFunctionDef', (1, 6), 'f', ('arguments', [], None, [], [], None, []), [('Expr', (2, 1), ('Await', (2, 1), ('Call', (2, 7), ('Name', (2, 7), 'something', ('Load',)), [], [])))], [], None)]), ('Module', [('AsyncFunctionDef', (1, 6), 'f', ('arguments', [], None, [], [], None, []), [('AsyncFor', (2, 7), ('Name', (2, 11), 'e', ('Store',)), ('Name', (2, 16), 'i', ('Load',)), [('Expr', (2, 19), ('Num', (2, 19), 1))], [('Expr', (3, 7), ('Num', (3, 7), 2))])], [], None)]), ('Module', [('AsyncFunctionDef', (1, 6), 'f', ('arguments', [], None, [], [], None, []), [('AsyncWith', (2, 7), [('withitem', ('Name', (2, 12), 'a', ('Load',)), ('Name', (2, 17), 'b', ('Store',)))], [('Expr', (2, 20), ('Num', (2, 20), 1))])], [], None)]), +('Module', [('Expr', (1, 0), ('Dict', (1, 1), [None, ('Num', (1, 10), 2)], [('Dict', (1, 4), [('Num', (1, 4), 1)], [('Num', (1, 6), 2)]), ('Num', (1, 12), 3)]))]), +('Module', [('Expr', (1, 0), ('Set', (1, 1), [('Starred', (1, 1), ('Set', (1, 3), [('Num', (1, 3), 1), ('Num', (1, 6), 2)]), ('Load',)), ('Num', (1, 10), 3)]))]), ] single_results = [ ('Interactive', [('Expr', (1, 0), ('BinOp', (1, 0), ('Num', (1, 0), 1), ('Add',), ('Num', (1, 2), 2)))]), diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -10,6 +10,8 @@ Core and Builtins ----------------- +- Issue #24975: Fix AST compilation for PEP 448 syntax. + Library ------- diff --git a/Python/ast.c b/Python/ast.c --- a/Python/ast.c +++ b/Python/ast.c @@ -199,8 +199,10 @@ "Dict doesn't have the same number of keys as values"); return 0; } - return validate_exprs(exp->v.Dict.keys, Load, 0) && - validate_exprs(exp->v.Dict.values, Load, 0); + /* null_ok=1 for keys expressions to allow dict unpacking to work in + dict literals, i.e. ``{**{a:b}}`` */ + return validate_exprs(exp->v.Dict.keys, Load, /*null_ok=*/ 1) && + validate_exprs(exp->v.Dict.values, Load, /*null_ok=*/ 0); case Set_kind: return validate_exprs(exp->v.Set.elts, Load, 0); #define COMP(NAME) \ -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 2 21:50:35 2015 From: python-checkins at python.org (yury.selivanov) Date: Wed, 02 Sep 2015 19:50:35 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogTWVyZ2UgMy41IChpc3N1ZSAjMjQ5NzUp?= Message-ID: <20150902195034.101494.31083@psf.io> https://hg.python.org/cpython/rev/1dcd7f257ed8 changeset: 97585:1dcd7f257ed8 parent: 97582:ebccac60b9e7 parent: 97584:ea79be5b201e user: Yury Selivanov date: Wed Sep 02 15:50:04 2015 -0400 summary: Merge 3.5 (issue #24975) files: Lib/test/test_ast.py | 24 +++++++++++++++--------- Misc/NEWS | 2 ++ Python/ast.c | 6 ++++-- 3 files changed, 21 insertions(+), 11 deletions(-) diff --git a/Lib/test/test_ast.py b/Lib/test/test_ast.py --- a/Lib/test/test_ast.py +++ b/Lib/test/test_ast.py @@ -78,9 +78,9 @@ # Pass, "pass", # Break - "break", + "for v in v:break", # Continue - "continue", + "for v in v:continue", # for statements with naked tuples (see http://bugs.python.org/issue6704) "for a,b in c: pass", "[(a,b) for a,b in c]", @@ -112,6 +112,9 @@ "async def f():\n async for e in i: 1\n else: 2", # AsyncWith "async def f():\n async with a as b: 1", + # PEP 448: Additional Unpacking Generalizations + "{**{1:2}, 2:3}", + "{*{1, 2}, 3}", ] # These are compiled through "single" @@ -231,9 +234,12 @@ (single_tests, single_results, "single"), (eval_tests, eval_results, "eval")): for i, o in zip(input, output): - ast_tree = compile(i, "?", kind, ast.PyCF_ONLY_AST) - self.assertEqual(to_tuple(ast_tree), o) - self._assertTrueorder(ast_tree, (0, 0)) + with self.subTest(action="parsing", input=i): + ast_tree = compile(i, "?", kind, ast.PyCF_ONLY_AST) + self.assertEqual(to_tuple(ast_tree), o) + self._assertTrueorder(ast_tree, (0, 0)) + with self.subTest(action="compiling", input=i): + compile(ast_tree, "?", kind) def test_slice(self): slc = ast.parse("x[::]").body[0].value.slice @@ -780,8 +786,6 @@ def test_dict(self): d = ast.Dict([], [ast.Name("x", ast.Load())]) self.expr(d, "same number of keys as values") - d = ast.Dict([None], [ast.Name("x", ast.Load())]) - self.expr(d, "None disallowed") d = ast.Dict([ast.Name("x", ast.Load())], [None]) self.expr(d, "None disallowed") @@ -972,8 +976,8 @@ ('Module', [('Global', (1, 0), ['v'])]), ('Module', [('Expr', (1, 0), ('Num', (1, 0), 1))]), ('Module', [('Pass', (1, 0))]), -('Module', [('Break', (1, 0))]), -('Module', [('Continue', (1, 0))]), +('Module', [('For', (1, 0), ('Name', (1, 4), 'v', ('Store',)), ('Name', (1, 9), 'v', ('Load',)), [('Break', (1, 11))], [])]), +('Module', [('For', (1, 0), ('Name', (1, 4), 'v', ('Store',)), ('Name', (1, 9), 'v', ('Load',)), [('Continue', (1, 11))], [])]), ('Module', [('For', (1, 0), ('Tuple', (1, 4), [('Name', (1, 4), 'a', ('Store',)), ('Name', (1, 6), 'b', ('Store',))], ('Store',)), ('Name', (1, 11), 'c', ('Load',)), [('Pass', (1, 14))], [])]), ('Module', [('Expr', (1, 0), ('ListComp', (1, 1), ('Tuple', (1, 2), [('Name', (1, 2), 'a', ('Load',)), ('Name', (1, 4), 'b', ('Load',))], ('Load',)), [('comprehension', ('Tuple', (1, 11), [('Name', (1, 11), 'a', ('Store',)), ('Name', (1, 13), 'b', ('Store',))], ('Store',)), ('Name', (1, 18), 'c', ('Load',)), [])]))]), ('Module', [('Expr', (1, 0), ('GeneratorExp', (1, 1), ('Tuple', (1, 2), [('Name', (1, 2), 'a', ('Load',)), ('Name', (1, 4), 'b', ('Load',))], ('Load',)), [('comprehension', ('Tuple', (1, 11), [('Name', (1, 11), 'a', ('Store',)), ('Name', (1, 13), 'b', ('Store',))], ('Store',)), ('Name', (1, 18), 'c', ('Load',)), [])]))]), @@ -986,6 +990,8 @@ ('Module', [('AsyncFunctionDef', (1, 6), 'f', ('arguments', [], None, [], [], None, []), [('Expr', (2, 1), ('Await', (2, 1), ('Call', (2, 7), ('Name', (2, 7), 'something', ('Load',)), [], [])))], [], None)]), ('Module', [('AsyncFunctionDef', (1, 6), 'f', ('arguments', [], None, [], [], None, []), [('AsyncFor', (2, 7), ('Name', (2, 11), 'e', ('Store',)), ('Name', (2, 16), 'i', ('Load',)), [('Expr', (2, 19), ('Num', (2, 19), 1))], [('Expr', (3, 7), ('Num', (3, 7), 2))])], [], None)]), ('Module', [('AsyncFunctionDef', (1, 6), 'f', ('arguments', [], None, [], [], None, []), [('AsyncWith', (2, 7), [('withitem', ('Name', (2, 12), 'a', ('Load',)), ('Name', (2, 17), 'b', ('Store',)))], [('Expr', (2, 20), ('Num', (2, 20), 1))])], [], None)]), +('Module', [('Expr', (1, 0), ('Dict', (1, 1), [None, ('Num', (1, 10), 2)], [('Dict', (1, 4), [('Num', (1, 4), 1)], [('Num', (1, 6), 2)]), ('Num', (1, 12), 3)]))]), +('Module', [('Expr', (1, 0), ('Set', (1, 1), [('Starred', (1, 1), ('Set', (1, 3), [('Num', (1, 3), 1), ('Num', (1, 6), 2)]), ('Load',)), ('Num', (1, 10), 3)]))]), ] single_results = [ ('Interactive', [('Expr', (1, 0), ('BinOp', (1, 0), ('Num', (1, 0), 1), ('Add',), ('Num', (1, 2), 2)))]), diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -127,6 +127,8 @@ Core and Builtins ----------------- +- Issue #24975: Fix AST compilation for PEP 448 syntax. + Library ------- diff --git a/Python/ast.c b/Python/ast.c --- a/Python/ast.c +++ b/Python/ast.c @@ -199,8 +199,10 @@ "Dict doesn't have the same number of keys as values"); return 0; } - return validate_exprs(exp->v.Dict.keys, Load, 0) && - validate_exprs(exp->v.Dict.values, Load, 0); + /* null_ok=1 for keys expressions to allow dict unpacking to work in + dict literals, i.e. ``{**{a:b}}`` */ + return validate_exprs(exp->v.Dict.keys, Load, /*null_ok=*/ 1) && + validate_exprs(exp->v.Dict.values, Load, /*null_ok=*/ 0); case Set_kind: return validate_exprs(exp->v.Set.elts, Load, 0); #define COMP(NAME) \ -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 2 21:50:35 2015 From: python-checkins at python.org (yury.selivanov) Date: Wed, 02 Sep 2015 19:50:35 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy41IC0+IDMuNSk6?= =?utf-8?q?_Merge_3=2E5_heads_=28issue_=2324975=29?= Message-ID: <20150902195034.101482.57270@psf.io> https://hg.python.org/cpython/rev/ea79be5b201e changeset: 97584:ea79be5b201e branch: 3.5 parent: 97578:1b22fa5f91ce parent: 97583:e9df8543d7bc user: Yury Selivanov date: Wed Sep 02 15:49:30 2015 -0400 summary: Merge 3.5 heads (issue #24975) files: Lib/test/test_ast.py | 24 +++++++++++++++--------- Misc/NEWS | 2 ++ Python/ast.c | 6 ++++-- 3 files changed, 21 insertions(+), 11 deletions(-) diff --git a/Lib/test/test_ast.py b/Lib/test/test_ast.py --- a/Lib/test/test_ast.py +++ b/Lib/test/test_ast.py @@ -78,9 +78,9 @@ # Pass, "pass", # Break - "break", + "for v in v:break", # Continue - "continue", + "for v in v:continue", # for statements with naked tuples (see http://bugs.python.org/issue6704) "for a,b in c: pass", "[(a,b) for a,b in c]", @@ -112,6 +112,9 @@ "async def f():\n async for e in i: 1\n else: 2", # AsyncWith "async def f():\n async with a as b: 1", + # PEP 448: Additional Unpacking Generalizations + "{**{1:2}, 2:3}", + "{*{1, 2}, 3}", ] # These are compiled through "single" @@ -231,9 +234,12 @@ (single_tests, single_results, "single"), (eval_tests, eval_results, "eval")): for i, o in zip(input, output): - ast_tree = compile(i, "?", kind, ast.PyCF_ONLY_AST) - self.assertEqual(to_tuple(ast_tree), o) - self._assertTrueorder(ast_tree, (0, 0)) + with self.subTest(action="parsing", input=i): + ast_tree = compile(i, "?", kind, ast.PyCF_ONLY_AST) + self.assertEqual(to_tuple(ast_tree), o) + self._assertTrueorder(ast_tree, (0, 0)) + with self.subTest(action="compiling", input=i): + compile(ast_tree, "?", kind) def test_slice(self): slc = ast.parse("x[::]").body[0].value.slice @@ -780,8 +786,6 @@ def test_dict(self): d = ast.Dict([], [ast.Name("x", ast.Load())]) self.expr(d, "same number of keys as values") - d = ast.Dict([None], [ast.Name("x", ast.Load())]) - self.expr(d, "None disallowed") d = ast.Dict([ast.Name("x", ast.Load())], [None]) self.expr(d, "None disallowed") @@ -972,8 +976,8 @@ ('Module', [('Global', (1, 0), ['v'])]), ('Module', [('Expr', (1, 0), ('Num', (1, 0), 1))]), ('Module', [('Pass', (1, 0))]), -('Module', [('Break', (1, 0))]), -('Module', [('Continue', (1, 0))]), +('Module', [('For', (1, 0), ('Name', (1, 4), 'v', ('Store',)), ('Name', (1, 9), 'v', ('Load',)), [('Break', (1, 11))], [])]), +('Module', [('For', (1, 0), ('Name', (1, 4), 'v', ('Store',)), ('Name', (1, 9), 'v', ('Load',)), [('Continue', (1, 11))], [])]), ('Module', [('For', (1, 0), ('Tuple', (1, 4), [('Name', (1, 4), 'a', ('Store',)), ('Name', (1, 6), 'b', ('Store',))], ('Store',)), ('Name', (1, 11), 'c', ('Load',)), [('Pass', (1, 14))], [])]), ('Module', [('Expr', (1, 0), ('ListComp', (1, 1), ('Tuple', (1, 2), [('Name', (1, 2), 'a', ('Load',)), ('Name', (1, 4), 'b', ('Load',))], ('Load',)), [('comprehension', ('Tuple', (1, 11), [('Name', (1, 11), 'a', ('Store',)), ('Name', (1, 13), 'b', ('Store',))], ('Store',)), ('Name', (1, 18), 'c', ('Load',)), [])]))]), ('Module', [('Expr', (1, 0), ('GeneratorExp', (1, 1), ('Tuple', (1, 2), [('Name', (1, 2), 'a', ('Load',)), ('Name', (1, 4), 'b', ('Load',))], ('Load',)), [('comprehension', ('Tuple', (1, 11), [('Name', (1, 11), 'a', ('Store',)), ('Name', (1, 13), 'b', ('Store',))], ('Store',)), ('Name', (1, 18), 'c', ('Load',)), [])]))]), @@ -986,6 +990,8 @@ ('Module', [('AsyncFunctionDef', (1, 6), 'f', ('arguments', [], None, [], [], None, []), [('Expr', (2, 1), ('Await', (2, 1), ('Call', (2, 7), ('Name', (2, 7), 'something', ('Load',)), [], [])))], [], None)]), ('Module', [('AsyncFunctionDef', (1, 6), 'f', ('arguments', [], None, [], [], None, []), [('AsyncFor', (2, 7), ('Name', (2, 11), 'e', ('Store',)), ('Name', (2, 16), 'i', ('Load',)), [('Expr', (2, 19), ('Num', (2, 19), 1))], [('Expr', (3, 7), ('Num', (3, 7), 2))])], [], None)]), ('Module', [('AsyncFunctionDef', (1, 6), 'f', ('arguments', [], None, [], [], None, []), [('AsyncWith', (2, 7), [('withitem', ('Name', (2, 12), 'a', ('Load',)), ('Name', (2, 17), 'b', ('Store',)))], [('Expr', (2, 20), ('Num', (2, 20), 1))])], [], None)]), +('Module', [('Expr', (1, 0), ('Dict', (1, 1), [None, ('Num', (1, 10), 2)], [('Dict', (1, 4), [('Num', (1, 4), 1)], [('Num', (1, 6), 2)]), ('Num', (1, 12), 3)]))]), +('Module', [('Expr', (1, 0), ('Set', (1, 1), [('Starred', (1, 1), ('Set', (1, 3), [('Num', (1, 3), 1), ('Num', (1, 6), 2)]), ('Load',)), ('Num', (1, 10), 3)]))]), ] single_results = [ ('Interactive', [('Expr', (1, 0), ('BinOp', (1, 0), ('Num', (1, 0), 1), ('Add',), ('Num', (1, 2), 2)))]), diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -62,6 +62,8 @@ Core and Builtins ----------------- +- Issue #24975: Fix AST compilation for PEP 448 syntax. + Library ------- diff --git a/Python/ast.c b/Python/ast.c --- a/Python/ast.c +++ b/Python/ast.c @@ -199,8 +199,10 @@ "Dict doesn't have the same number of keys as values"); return 0; } - return validate_exprs(exp->v.Dict.keys, Load, 0) && - validate_exprs(exp->v.Dict.values, Load, 0); + /* null_ok=1 for keys expressions to allow dict unpacking to work in + dict literals, i.e. ``{**{a:b}}`` */ + return validate_exprs(exp->v.Dict.keys, Load, /*null_ok=*/ 1) && + validate_exprs(exp->v.Dict.values, Load, /*null_ok=*/ 0); case Set_kind: return validate_exprs(exp->v.Set.elts, Load, 0); #define COMP(NAME) \ -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 2 22:02:07 2015 From: python-checkins at python.org (zach.ware) Date: Wed, 02 Sep 2015 20:02:07 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge_with_3=2E5?= Message-ID: <20150902200205.27701.58631@psf.io> https://hg.python.org/cpython/rev/d319653a4348 changeset: 97588:d319653a4348 parent: 97585:1dcd7f257ed8 parent: 97587:59d0aeea41fa user: Zachary Ware date: Wed Sep 02 15:01:42 2015 -0500 summary: Merge with 3.5 files: PCbuild/build.bat | 51 ++++++++++++++++++++++++---------- 1 files changed, 36 insertions(+), 15 deletions(-) diff --git a/PCbuild/build.bat b/PCbuild/build.bat --- a/PCbuild/build.bat +++ b/PCbuild/build.bat @@ -1,19 +1,39 @@ @echo off -rem A batch program to build or rebuild a particular configuration, -rem just for convenience. +goto Run +:Usage +echo.%~nx0 [flags and arguments] [quoted MSBuild options] +echo. +echo.Build CPython from the command line. Requires the appropriate +echo.version(s) of Microsoft Visual Studio to be installed (see readme.txt). +echo.Also requires Subversion (svn.exe) to be on PATH if the '-e' flag is +echo.given. +echo. +echo.After the flags recognized by this script, up to 9 arguments to be passed +echo.directly to MSBuild may be passed. If the argument contains an '=', the +echo.entire argument must be quoted (e.g. `%~nx0 "/p:PlatformToolset=v100"`) +echo. +echo.Available flags: +echo. -h Display this help message +echo. -V Display version information for the current build +echo. -r Target Rebuild instead of Build +echo. -d Set the configuration to Debug +echo. -e Build external libraries fetched by get_externals.bat +echo. -m Enable parallel build (enabled by default) +echo. -M Disable parallel build +echo. -v Increased output messages +echo. -k Attempt to kill any running Pythons before building (usually done +echo. automatically by the pythoncore project) +echo. +echo.Available arguments: +echo. -c Release ^| Debug ^| PGInstrument ^| PGUpdate +echo. Set the configuration (default: Release) +echo. -p x64 ^| Win32 +echo. Set the platform (default: Win32) +echo. -t Build ^| Rebuild ^| Clean ^| CleanAll +echo. Set the target manually +exit /b 127 -rem Arguments: -rem -c Set the configuration (default: Release) -rem -p Set the platform (x64 or Win32, default: Win32) -rem -r Target Rebuild instead of Build -rem -t Set the target manually (Build, Rebuild, Clean, or CleanAll) -rem -d Set the configuration to Debug -rem -e Pull in external libraries using get_externals.bat -rem -m Enable parallel build (enabled by default) -rem -M Disable parallel build -rem -v Increased output messages -rem -k Attempt to kill any running Pythons before building (usually unnecessary) - +:Run setlocal set platf=Win32 set vs_platf=x86 @@ -25,6 +45,7 @@ set kill= :CheckOpts +if "%~1"=="-h" goto Usage if "%~1"=="-c" (set conf=%2) & shift & shift & goto CheckOpts if "%~1"=="-p" (set platf=%2) & shift & shift & goto CheckOpts if "%~1"=="-r" (set target=Rebuild) & shift & goto CheckOpts @@ -43,7 +64,7 @@ call "%dir%env.bat" %vs_platf% >nul if "%kill%"=="true" ( - msbuild /v:m /nologo /target:KillPython "%pcbuild%\pythoncore.vcxproj" /p:Configuration=%conf% /p:Platform=%platf% /p:KillPython=true + msbuild /v:m /nologo /target:KillPython "%dir%\pythoncore.vcxproj" /p:Configuration=%conf% /p:Platform=%platf% /p:KillPython=true ) rem Call on MSBuild to do the work, echo the command. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 2 22:02:07 2015 From: python-checkins at python.org (zach.ware) Date: Wed, 02 Sep 2015 20:02:07 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogVHVybiAncmVtJyBj?= =?utf-8?q?omments_into_a_real_usage_message_in_PCbuild/build=2Ebat?= Message-ID: <20150902200205.114864.78981@psf.io> https://hg.python.org/cpython/rev/59d0aeea41fa changeset: 97587:59d0aeea41fa branch: 3.5 parent: 97584:ea79be5b201e user: Zachary Ware date: Wed Sep 02 13:21:19 2015 -0500 summary: Turn 'rem' comments into a real usage message in PCbuild/build.bat Also fixes error in 'kill' target (already fixed in 2.7, somehow the fix didn't make it to this branch). files: PCbuild/build.bat | 51 ++++++++++++++++++++++++---------- 1 files changed, 36 insertions(+), 15 deletions(-) diff --git a/PCbuild/build.bat b/PCbuild/build.bat --- a/PCbuild/build.bat +++ b/PCbuild/build.bat @@ -1,19 +1,39 @@ @echo off -rem A batch program to build or rebuild a particular configuration, -rem just for convenience. +goto Run +:Usage +echo.%~nx0 [flags and arguments] [quoted MSBuild options] +echo. +echo.Build CPython from the command line. Requires the appropriate +echo.version(s) of Microsoft Visual Studio to be installed (see readme.txt). +echo.Also requires Subversion (svn.exe) to be on PATH if the '-e' flag is +echo.given. +echo. +echo.After the flags recognized by this script, up to 9 arguments to be passed +echo.directly to MSBuild may be passed. If the argument contains an '=', the +echo.entire argument must be quoted (e.g. `%~nx0 "/p:PlatformToolset=v100"`) +echo. +echo.Available flags: +echo. -h Display this help message +echo. -V Display version information for the current build +echo. -r Target Rebuild instead of Build +echo. -d Set the configuration to Debug +echo. -e Build external libraries fetched by get_externals.bat +echo. -m Enable parallel build (enabled by default) +echo. -M Disable parallel build +echo. -v Increased output messages +echo. -k Attempt to kill any running Pythons before building (usually done +echo. automatically by the pythoncore project) +echo. +echo.Available arguments: +echo. -c Release ^| Debug ^| PGInstrument ^| PGUpdate +echo. Set the configuration (default: Release) +echo. -p x64 ^| Win32 +echo. Set the platform (default: Win32) +echo. -t Build ^| Rebuild ^| Clean ^| CleanAll +echo. Set the target manually +exit /b 127 -rem Arguments: -rem -c Set the configuration (default: Release) -rem -p Set the platform (x64 or Win32, default: Win32) -rem -r Target Rebuild instead of Build -rem -t Set the target manually (Build, Rebuild, Clean, or CleanAll) -rem -d Set the configuration to Debug -rem -e Pull in external libraries using get_externals.bat -rem -m Enable parallel build (enabled by default) -rem -M Disable parallel build -rem -v Increased output messages -rem -k Attempt to kill any running Pythons before building (usually unnecessary) - +:Run setlocal set platf=Win32 set vs_platf=x86 @@ -25,6 +45,7 @@ set kill= :CheckOpts +if "%~1"=="-h" goto Usage if "%~1"=="-c" (set conf=%2) & shift & shift & goto CheckOpts if "%~1"=="-p" (set platf=%2) & shift & shift & goto CheckOpts if "%~1"=="-r" (set target=Rebuild) & shift & goto CheckOpts @@ -43,7 +64,7 @@ call "%dir%env.bat" %vs_platf% >nul if "%kill%"=="true" ( - msbuild /v:m /nologo /target:KillPython "%pcbuild%\pythoncore.vcxproj" /p:Configuration=%conf% /p:Platform=%platf% /p:KillPython=true + msbuild /v:m /nologo /target:KillPython "%dir%\pythoncore.vcxproj" /p:Configuration=%conf% /p:Platform=%platf% /p:KillPython=true ) rem Call on MSBuild to do the work, echo the command. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 2 22:02:07 2015 From: python-checkins at python.org (zach.ware) Date: Wed, 02 Sep 2015 20:02:07 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogVHVybiAncmVtJyBj?= =?utf-8?q?omments_into_a_real_usage_message_in_PCbuild/build=2Ebat?= Message-ID: <20150902200204.68879.17358@psf.io> https://hg.python.org/cpython/rev/699670303a43 changeset: 97586:699670303a43 branch: 2.7 parent: 97563:af97d596da40 user: Zachary Ware date: Wed Sep 02 13:21:19 2015 -0500 summary: Turn 'rem' comments into a real usage message in PCbuild/build.bat Also fixes quoting to match 3.5+ files: PCbuild/build.bat | 72 ++++++++++++++++++++++------------ 1 files changed, 46 insertions(+), 26 deletions(-) diff --git a/PCbuild/build.bat b/PCbuild/build.bat --- a/PCbuild/build.bat +++ b/PCbuild/build.bat @@ -1,19 +1,38 @@ @echo off -rem A batch program to build or rebuild a particular configuration, -rem just for convenience. +goto Run +:Usage +echo.%~nx0 [flags and arguments] [quoted MSBuild options] +echo. +echo.Build CPython from the command line. Requires the appropriate +echo.version(s) of Microsoft Visual Studio to be installed (see readme.txt). +echo.Also requires Subversion (svn.exe) to be on PATH if the '-e' flag is +echo.given. +echo. +echo.After the flags recognized by this script, up to 9 arguments to be passed +echo.directly to MSBuild may be passed. If the argument contains an '=', the +echo.entire argument must be quoted (e.g. `%~nx0 "/p:PlatformToolset=v100"`) +echo. +echo.Available flags: +echo. -h Display this help message +echo. -r Target Rebuild instead of Build +echo. -d Set the configuration to Debug +echo. -e Build external libraries fetched by get_externals.bat +echo. -m Enable parallel build +echo. -M Disable parallel build (disabled by default) +echo. -v Increased output messages +echo. -k Attempt to kill any running Pythons before building (usually done +echo. automatically by the pythoncore project) +echo. +echo.Available arguments: +echo. -c Release ^| Debug ^| PGInstrument ^| PGUpdate +echo. Set the configuration (default: Release) +echo. -p x64 ^| Win32 +echo. Set the platform (default: Win32) +echo. -t Build ^| Rebuild ^| Clean ^| CleanAll +echo. Set the target manually +exit /b 127 -rem Arguments: -rem -c Set the configuration (default: Release) -rem -p Set the platform (x64 or Win32, default: Win32) -rem -r Target Rebuild instead of Build -rem -t Set the target manually (Build, Rebuild, Clean, or CleanAll) -rem -d Set the configuration to Debug -rem -e Pull in external libraries using get_externals.bat -rem -m Enable parallel build -rem -M Disable parallel build (disabled by default) -rem -v Increased output messages -rem -k Attempt to kill any running Pythons before building (usually unnecessary) - +:Run setlocal set platf=Win32 set vs_platf=x86 @@ -25,23 +44,24 @@ set kill= :CheckOpts -if '%1'=='-c' (set conf=%2) & shift & shift & goto CheckOpts -if '%1'=='-p' (set platf=%2) & shift & shift & goto CheckOpts -if '%1'=='-r' (set target=Rebuild) & shift & goto CheckOpts -if '%1'=='-t' (set target=%2) & shift & shift & goto CheckOpts -if '%1'=='-d' (set conf=Debug) & shift & goto CheckOpts -if '%1'=='-e' call "%dir%get_externals.bat" & shift & goto CheckOpts -if '%1'=='-m' (set parallel=/m) & shift & goto CheckOpts -if '%1'=='-M' (set parallel=) & shift & goto CheckOpts -if '%1'=='-v' (set verbose=/v:n) & shift & goto CheckOpts -if '%1'=='-k' (set kill=true) & shift & goto CheckOpts +if "%~1"=="-h" goto Usage +if "%~1"=="-c" (set conf=%2) & shift & shift & goto CheckOpts +if "%~1"=="-p" (set platf=%2) & shift & shift & goto CheckOpts +if "%~1"=="-r" (set target=Rebuild) & shift & goto CheckOpts +if "%~1"=="-t" (set target=%2) & shift & shift & goto CheckOpts +if "%~1"=="-d" (set conf=Debug) & shift & goto CheckOpts +if "%~1"=="-e" call "%dir%get_externals.bat" & shift & goto CheckOpts +if "%~1"=="-m" (set parallel=/m) & shift & goto CheckOpts +if "%~1"=="-M" (set parallel=) & shift & goto CheckOpts +if "%~1"=="-v" (set verbose=/v:n) & shift & goto CheckOpts +if "%~1"=="-k" (set kill=true) & shift & goto CheckOpts -if '%platf%'=='x64' (set vs_platf=x86_amd64) +if "%platf%"=="x64" (set vs_platf=x86_amd64) rem Setup the environment call "%dir%env.bat" %vs_platf% >nul -if '%kill%'=='true' ( +if "%kill%"=="true" ( msbuild /v:m /nologo /target:KillPython "%dir%\pythoncore.vcxproj" /p:Configuration=%conf% /p:Platform=%platf% /p:KillPython=true ) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 2 22:37:30 2015 From: python-checkins at python.org (victor.stinner) Date: Wed, 02 Sep 2015 20:37:30 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2323517=3A_datetime?= =?utf-8?q?=2Etimedelta_constructor_now_rounds_microseconds_to_nearest?= Message-ID: <20150902203730.11244.70994@psf.io> https://hg.python.org/cpython/rev/0eb8c182131e changeset: 97589:0eb8c182131e user: Victor Stinner date: Wed Sep 02 19:16:07 2015 +0200 summary: Issue #23517: datetime.timedelta constructor now rounds microseconds to nearest with ties going away from zero (ROUND_HALF_UP), as Python 2 and Python older than 3.3, instead of rounding to nearest with ties going to nearest even integer (ROUND_HALF_EVEN). files: Include/pytime.h | 4 ++++ Lib/datetime.py | 12 ++++++++++-- Lib/test/datetimetester.py | 14 +++++--------- Misc/NEWS | 5 +++++ Modules/_datetimemodule.c | 22 +--------------------- Python/pytime.c | 3 +-- 6 files changed, 26 insertions(+), 34 deletions(-) diff --git a/Include/pytime.h b/Include/pytime.h --- a/Include/pytime.h +++ b/Include/pytime.h @@ -44,6 +44,10 @@ PyAPI_FUNC(time_t) _PyLong_AsTime_t( PyObject *obj); +/* Round to nearest with ties going away from zero (_PyTime_ROUND_HALF_UP). */ +PyAPI_FUNC(double) _PyTime_RoundHalfUp( + double x); + /* Convert a number of seconds, int or float, to time_t. */ PyAPI_FUNC(int) _PyTime_ObjectToTime_t( PyObject *obj, diff --git a/Lib/datetime.py b/Lib/datetime.py --- a/Lib/datetime.py +++ b/Lib/datetime.py @@ -316,6 +316,14 @@ return q +def _round_half_up(x): + """Round to nearest with ties going away from zero.""" + if x >= 0.0: + return _math.floor(x + 0.5) + else: + return _math.ceil(x - 0.5) + + class timedelta: """Represent the difference between two datetime objects. @@ -399,7 +407,7 @@ # secondsfrac isn't referenced again if isinstance(microseconds, float): - microseconds = round(microseconds + usdouble) + microseconds = _round_half_up(microseconds + usdouble) seconds, microseconds = divmod(microseconds, 1000000) days, seconds = divmod(seconds, 24*3600) d += days @@ -410,7 +418,7 @@ days, seconds = divmod(seconds, 24*3600) d += days s += seconds - microseconds = round(microseconds + usdouble) + microseconds = _round_half_up(microseconds + usdouble) assert isinstance(s, int) assert isinstance(microseconds, int) assert abs(s) <= 3 * 24 * 3600 diff --git a/Lib/test/datetimetester.py b/Lib/test/datetimetester.py --- a/Lib/test/datetimetester.py +++ b/Lib/test/datetimetester.py @@ -662,28 +662,24 @@ # Single-field rounding. eq(td(milliseconds=0.4/1000), td(0)) # rounds to 0 eq(td(milliseconds=-0.4/1000), td(0)) # rounds to 0 - eq(td(milliseconds=0.5/1000), td(microseconds=0)) - eq(td(milliseconds=-0.5/1000), td(microseconds=0)) + eq(td(milliseconds=0.5/1000), td(microseconds=1)) + eq(td(milliseconds=-0.5/1000), td(microseconds=-1)) eq(td(milliseconds=0.6/1000), td(microseconds=1)) eq(td(milliseconds=-0.6/1000), td(microseconds=-1)) - eq(td(seconds=0.5/10**6), td(microseconds=0)) - eq(td(seconds=-0.5/10**6), td(microseconds=0)) + eq(td(seconds=0.5/10**6), td(microseconds=1)) + eq(td(seconds=-0.5/10**6), td(microseconds=-1)) # Rounding due to contributions from more than one field. us_per_hour = 3600e6 us_per_day = us_per_hour * 24 eq(td(days=.4/us_per_day), td(0)) eq(td(hours=.2/us_per_hour), td(0)) - eq(td(days=.4/us_per_day, hours=.2/us_per_hour), td(microseconds=1)) + eq(td(days=.4/us_per_day, hours=.2/us_per_hour), td(microseconds=1), td) eq(td(days=-.4/us_per_day), td(0)) eq(td(hours=-.2/us_per_hour), td(0)) eq(td(days=-.4/us_per_day, hours=-.2/us_per_hour), td(microseconds=-1)) - # Test for a patch in Issue 8860 - eq(td(microseconds=0.5), 0.5*td(microseconds=1.0)) - eq(td(microseconds=0.5)//td.resolution, 0.5*td.resolution//td.resolution) - def test_massive_normalization(self): td = timedelta(microseconds=-1) self.assertEqual((td.days, td.seconds, td.microseconds), diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -17,6 +17,11 @@ Library ------- +- Issue #23517: datetime.timedelta constructor now rounds microseconds to + nearest with ties going away from zero (ROUND_HALF_UP), as Python 2 and + Python older than 3.3, instead of rounding to nearest with ties going to + nearest even integer (ROUND_HALF_EVEN). + - Issue #23552: Timeit now warns when there is substantial (4x) variance between best and worst times. Patch from Serhiy Storchaka. diff --git a/Modules/_datetimemodule.c b/Modules/_datetimemodule.c --- a/Modules/_datetimemodule.c +++ b/Modules/_datetimemodule.c @@ -2149,29 +2149,9 @@ if (leftover_us) { /* Round to nearest whole # of us, and add into x. */ double whole_us = round(leftover_us); - int x_is_odd; PyObject *temp; - whole_us = round(leftover_us); - if (fabs(whole_us - leftover_us) == 0.5) { - /* We're exactly halfway between two integers. In order - * to do round-half-to-even, we must determine whether x - * is odd. Note that x is odd when it's last bit is 1. The - * code below uses bitwise and operation to check the last - * bit. */ - temp = PyNumber_And(x, one); /* temp <- x & 1 */ - if (temp == NULL) { - Py_DECREF(x); - goto Done; - } - x_is_odd = PyObject_IsTrue(temp); - Py_DECREF(temp); - if (x_is_odd == -1) { - Py_DECREF(x); - goto Done; - } - whole_us = 2.0 * round((leftover_us + x_is_odd) * 0.5) - x_is_odd; - } + whole_us = _PyTime_RoundHalfUp(leftover_us); temp = PyLong_FromLong((long)whole_us); diff --git a/Python/pytime.c b/Python/pytime.c --- a/Python/pytime.c +++ b/Python/pytime.c @@ -60,8 +60,7 @@ #endif } -/* Round to nearest with ties going away from zero (_PyTime_ROUND_HALF_UP). */ -static double +double _PyTime_RoundHalfUp(double x) { /* volatile avoids optimization changing how numbers are rounded */ -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 2 23:23:44 2015 From: python-checkins at python.org (victor.stinner) Date: Wed, 02 Sep 2015 21:23:44 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogdGVzdF9nZGI6IGZp?= =?utf-8?q?x_regex_to_parse_gdb_version_for_SUSE_Linux_Entreprise?= Message-ID: <20150902212343.12015.45058@psf.io> https://hg.python.org/cpython/rev/6c5e99105264 changeset: 97590:6c5e99105264 branch: 3.4 parent: 97577:d65d3222538d user: Victor Stinner date: Wed Sep 02 23:12:14 2015 +0200 summary: test_gdb: fix regex to parse gdb version for SUSE Linux Entreprise Mention also the detected GDB version on verbose mode and on error (if the major version is smaller than 7). files: Lib/test/test_gdb.py | 15 +++++++++++---- 1 files changed, 11 insertions(+), 4 deletions(-) diff --git a/Lib/test/test_gdb.py b/Lib/test/test_gdb.py --- a/Lib/test/test_gdb.py +++ b/Lib/test/test_gdb.py @@ -28,12 +28,19 @@ # This is what "no gdb" looks like. There may, however, be other # errors that manifest this way too. raise unittest.SkipTest("Couldn't find gdb on the path") -gdb_version_number = re.search(b"^GNU gdb [^\d]*(\d+)\.(\d)", gdb_version) +# Regex to parse: +# 'GNU gdb (GDB; SUSE Linux Enterprise 12) 7.7\n' -> 7.7 +# 'GNU gdb (GDB) Fedora 7.9.1-17.fc22\n' -> 7.9 +gdb_version_number = re.search(b"^GNU gdb .*? (\d+)\.(\d)", gdb_version) +if not gdb_version_number: + raise Exception("unable to parse GDB version: %a" % gdb_version) gdb_major_version = int(gdb_version_number.group(1)) gdb_minor_version = int(gdb_version_number.group(2)) if gdb_major_version < 7: - raise unittest.SkipTest("gdb versions before 7.0 didn't support python embedding" - " Saw:\n" + gdb_version.decode('ascii', 'replace')) + raise unittest.SkipTest("gdb versions before 7.0 didn't support python " + "embedding. Saw %s.%s:\n%s" + % (gdb_major_version, gdb_minor_version, + gdb_version.decode('ascii', 'replace'))) if not sysconfig.is_python_build(): raise unittest.SkipTest("test_gdb only works on source builds at the moment.") @@ -878,7 +885,7 @@ def test_main(): if support.verbose: - print("GDB version:") + print("GDB version %s.%s:" % (gdb_major_version, gdb_minor_version)) for line in os.fsdecode(gdb_version).splitlines(): print(" " * 4 + line) run_unittest(PrettyPrintTests, -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 2 23:23:44 2015 From: python-checkins at python.org (victor.stinner) Date: Wed, 02 Sep 2015 21:23:44 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogdGVzdF9nZGI6IHVz?= =?utf-8?q?e_subprocess=2EPopen_context_manager_to_fix_ResourceWarning_war?= =?utf-8?q?nings?= Message-ID: <20150902212343.11993.377@psf.io> https://hg.python.org/cpython/rev/f394be80bc7a changeset: 97591:f394be80bc7a branch: 3.4 user: Victor Stinner date: Wed Sep 02 23:19:55 2015 +0200 summary: test_gdb: use subprocess.Popen context manager to fix ResourceWarning warnings when the test is interrupted (or fail). files: Lib/test/test_gdb.py | 49 ++++++++++++++++++------------- 1 files changed, 29 insertions(+), 20 deletions(-) diff --git a/Lib/test/test_gdb.py b/Lib/test/test_gdb.py --- a/Lib/test/test_gdb.py +++ b/Lib/test/test_gdb.py @@ -21,26 +21,32 @@ from test import support from test.support import run_unittest, findfile, python_is_optimized -try: - gdb_version, _ = subprocess.Popen(["gdb", "-nx", "--version"], - stdout=subprocess.PIPE).communicate() -except OSError: - # This is what "no gdb" looks like. There may, however, be other - # errors that manifest this way too. - raise unittest.SkipTest("Couldn't find gdb on the path") -# Regex to parse: -# 'GNU gdb (GDB; SUSE Linux Enterprise 12) 7.7\n' -> 7.7 -# 'GNU gdb (GDB) Fedora 7.9.1-17.fc22\n' -> 7.9 -gdb_version_number = re.search(b"^GNU gdb .*? (\d+)\.(\d)", gdb_version) -if not gdb_version_number: - raise Exception("unable to parse GDB version: %a" % gdb_version) -gdb_major_version = int(gdb_version_number.group(1)) -gdb_minor_version = int(gdb_version_number.group(2)) +def get_gdb_version(): + try: + proc = subprocess.Popen(["gdb", "-nx", "--version"], + stdout=subprocess.PIPE, + universal_newlines=True) + with proc: + version = proc.communicate()[0] + except OSError: + # This is what "no gdb" looks like. There may, however, be other + # errors that manifest this way too. + raise unittest.SkipTest("Couldn't find gdb on the path") + + # Regex to parse: + # 'GNU gdb (GDB; SUSE Linux Enterprise 12) 7.7\n' -> 7.7 + # 'GNU gdb (GDB) Fedora 7.9.1-17.fc22\n' -> 7.9 + match = re.search("^GNU gdb .*? (\d+)\.(\d)", version) + if match is None: + raise Exception("unable to parse GDB version: %r" % version) + return (version, int(match.group(1)), int(match.group(2))) + +gdb_version, gdb_major_version, gdb_minor_version = get_gdb_version() if gdb_major_version < 7: raise unittest.SkipTest("gdb versions before 7.0 didn't support python " "embedding. Saw %s.%s:\n%s" % (gdb_major_version, gdb_minor_version, - gdb_version.decode('ascii', 'replace'))) + gdb_version)) if not sysconfig.is_python_build(): raise unittest.SkipTest("test_gdb only works on source builds at the moment.") @@ -66,9 +72,12 @@ base_cmd = ('gdb', '--batch', '-nx') if (gdb_major_version, gdb_minor_version) >= (7, 4): base_cmd += ('-iex', 'add-auto-load-safe-path ' + checkout_hook_path) - out, err = subprocess.Popen(base_cmd + args, - stdout=subprocess.PIPE, stderr=subprocess.PIPE, env=env, - ).communicate() + proc = subprocess.Popen(base_cmd + args, + stdout=subprocess.PIPE, + stderr=subprocess.PIPE, + env=env) + with proc: + out, err = proc.communicate() return out.decode('utf-8', 'replace'), err.decode('utf-8', 'replace') # Verify that "gdb" was built with the embedded python support enabled: @@ -886,7 +895,7 @@ def test_main(): if support.verbose: print("GDB version %s.%s:" % (gdb_major_version, gdb_minor_version)) - for line in os.fsdecode(gdb_version).splitlines(): + for line in gdb_version.splitlines(): print(" " * 4 + line) run_unittest(PrettyPrintTests, PyListTests, -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 2 23:23:49 2015 From: python-checkins at python.org (victor.stinner) Date: Wed, 02 Sep 2015 21:23:49 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogTWVyZ2UgMy41ICh0ZXN0X2dkYik=?= Message-ID: <20150902212344.101492.32521@psf.io> https://hg.python.org/cpython/rev/e1fc4b60739b changeset: 97593:e1fc4b60739b parent: 97589:0eb8c182131e parent: 97592:bc1b0aaf1280 user: Victor Stinner date: Wed Sep 02 23:22:31 2015 +0200 summary: Merge 3.5 (test_gdb) files: Lib/test/test_gdb.py | 46 +++++++++++++++++++------------ 1 files changed, 28 insertions(+), 18 deletions(-) diff --git a/Lib/test/test_gdb.py b/Lib/test/test_gdb.py --- a/Lib/test/test_gdb.py +++ b/Lib/test/test_gdb.py @@ -21,23 +21,32 @@ from test import support from test.support import run_unittest, findfile, python_is_optimized -try: - gdb_version, _ = subprocess.Popen(["gdb", "-nx", "--version"], - stdout=subprocess.PIPE).communicate() -except OSError: - # This is what "no gdb" looks like. There may, however, be other - # errors that manifest this way too. - raise unittest.SkipTest("Couldn't find gdb on the path") -try: - gdb_version_number = re.search(b"^GNU gdb [^\d]*(\d+)\.(\d)", gdb_version) - gdb_major_version = int(gdb_version_number.group(1)) - gdb_minor_version = int(gdb_version_number.group(2)) -except Exception: - raise ValueError("unable to parse GDB version: %r" % gdb_version) +def get_gdb_version(): + try: + proc = subprocess.Popen(["gdb", "-nx", "--version"], + stdout=subprocess.PIPE, + universal_newlines=True) + with proc: + version = proc.communicate()[0] + except OSError: + # This is what "no gdb" looks like. There may, however, be other + # errors that manifest this way too. + raise unittest.SkipTest("Couldn't find gdb on the path") + # Regex to parse: + # 'GNU gdb (GDB; SUSE Linux Enterprise 12) 7.7\n' -> 7.7 + # 'GNU gdb (GDB) Fedora 7.9.1-17.fc22\n' -> 7.9 + match = re.search("^GNU gdb .*? (\d+)\.(\d)", version) + if match is None: + raise Exception("unable to parse GDB version: %r" % version) + return (version, int(match.group(1)), int(match.group(2))) + +gdb_version, gdb_major_version, gdb_minor_version = get_gdb_version() if gdb_major_version < 7: - raise unittest.SkipTest("gdb versions before 7.0 didn't support python embedding" - " Saw:\n" + gdb_version.decode('ascii', 'replace')) + raise unittest.SkipTest("gdb versions before 7.0 didn't support python " + "embedding. Saw %s.%s:\n%s" + % (gdb_major_version, gdb_minor_version, + gdb_version)) if not sysconfig.is_python_build(): raise unittest.SkipTest("test_gdb only works on source builds at the moment.") @@ -65,7 +74,8 @@ base_cmd += ('-iex', 'add-auto-load-safe-path ' + checkout_hook_path) proc = subprocess.Popen(base_cmd + args, stdout=subprocess.PIPE, - stderr=subprocess.PIPE, env=env) + stderr=subprocess.PIPE, + env=env) with proc: out, err = proc.communicate() return out.decode('utf-8', 'replace'), err.decode('utf-8', 'replace') @@ -886,8 +896,8 @@ def test_main(): if support.verbose: - print("GDB version:") - for line in os.fsdecode(gdb_version).splitlines(): + print("GDB version %s.%s:" % (gdb_major_version, gdb_minor_version)) + for line in gdb_version.splitlines(): print(" " * 4 + line) run_unittest(PrettyPrintTests, PyListTests, -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 2 23:23:51 2015 From: python-checkins at python.org (victor.stinner) Date: Wed, 02 Sep 2015 21:23:51 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?b?IE1lcmdlIDMuNCAodGVzdF9nZGIp?= Message-ID: <20150902212344.101496.92310@psf.io> https://hg.python.org/cpython/rev/bc1b0aaf1280 changeset: 97592:bc1b0aaf1280 branch: 3.5 parent: 97587:59d0aeea41fa parent: 97591:f394be80bc7a user: Victor Stinner date: Wed Sep 02 23:21:03 2015 +0200 summary: Merge 3.4 (test_gdb) files: Lib/test/test_gdb.py | 50 +++++++++++++++++++++---------- 1 files changed, 33 insertions(+), 17 deletions(-) diff --git a/Lib/test/test_gdb.py b/Lib/test/test_gdb.py --- a/Lib/test/test_gdb.py +++ b/Lib/test/test_gdb.py @@ -21,19 +21,32 @@ from test import support from test.support import run_unittest, findfile, python_is_optimized -try: - gdb_version, _ = subprocess.Popen(["gdb", "-nx", "--version"], - stdout=subprocess.PIPE).communicate() -except OSError: - # This is what "no gdb" looks like. There may, however, be other - # errors that manifest this way too. - raise unittest.SkipTest("Couldn't find gdb on the path") -gdb_version_number = re.search(b"^GNU gdb [^\d]*(\d+)\.(\d)", gdb_version) -gdb_major_version = int(gdb_version_number.group(1)) -gdb_minor_version = int(gdb_version_number.group(2)) +def get_gdb_version(): + try: + proc = subprocess.Popen(["gdb", "-nx", "--version"], + stdout=subprocess.PIPE, + universal_newlines=True) + with proc: + version = proc.communicate()[0] + except OSError: + # This is what "no gdb" looks like. There may, however, be other + # errors that manifest this way too. + raise unittest.SkipTest("Couldn't find gdb on the path") + + # Regex to parse: + # 'GNU gdb (GDB; SUSE Linux Enterprise 12) 7.7\n' -> 7.7 + # 'GNU gdb (GDB) Fedora 7.9.1-17.fc22\n' -> 7.9 + match = re.search("^GNU gdb .*? (\d+)\.(\d)", version) + if match is None: + raise Exception("unable to parse GDB version: %r" % version) + return (version, int(match.group(1)), int(match.group(2))) + +gdb_version, gdb_major_version, gdb_minor_version = get_gdb_version() if gdb_major_version < 7: - raise unittest.SkipTest("gdb versions before 7.0 didn't support python embedding" - " Saw:\n" + gdb_version.decode('ascii', 'replace')) + raise unittest.SkipTest("gdb versions before 7.0 didn't support python " + "embedding. Saw %s.%s:\n%s" + % (gdb_major_version, gdb_minor_version, + gdb_version)) if not sysconfig.is_python_build(): raise unittest.SkipTest("test_gdb only works on source builds at the moment.") @@ -59,9 +72,12 @@ base_cmd = ('gdb', '--batch', '-nx') if (gdb_major_version, gdb_minor_version) >= (7, 4): base_cmd += ('-iex', 'add-auto-load-safe-path ' + checkout_hook_path) - out, err = subprocess.Popen(base_cmd + args, - stdout=subprocess.PIPE, stderr=subprocess.PIPE, env=env, - ).communicate() + proc = subprocess.Popen(base_cmd + args, + stdout=subprocess.PIPE, + stderr=subprocess.PIPE, + env=env) + with proc: + out, err = proc.communicate() return out.decode('utf-8', 'replace'), err.decode('utf-8', 'replace') # Verify that "gdb" was built with the embedded python support enabled: @@ -880,8 +896,8 @@ def test_main(): if support.verbose: - print("GDB version:") - for line in os.fsdecode(gdb_version).splitlines(): + print("GDB version %s.%s:" % (gdb_major_version, gdb_minor_version)) + for line in gdb_version.splitlines(): print(" " * 4 + line) run_unittest(PrettyPrintTests, PyListTests, -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 3 00:10:10 2015 From: python-checkins at python.org (victor.stinner) Date: Wed, 02 Sep 2015 22:10:10 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogRml4IHRlc3Rfd2Fy?= =?utf-8?q?nings=3A_don=27t_modify_warnings=2Efilters?= Message-ID: <20150902221010.12011.76369@psf.io> https://hg.python.org/cpython/rev/c1396d28c440 changeset: 97594:c1396d28c440 branch: 3.4 parent: 97591:f394be80bc7a user: Victor Stinner date: Thu Sep 03 00:07:47 2015 +0200 summary: Fix test_warnings: don't modify warnings.filters BaseTest now ensures that unittest.TestCase.assertWarns() uses the same warnings module than warnings.catch_warnings(). Otherwise, warnings.catch_warnings() will be unable to remove the added filter. files: Lib/test/test_warnings.py | 6 ++++++ 1 files changed, 6 insertions(+), 0 deletions(-) diff --git a/Lib/test/test_warnings.py b/Lib/test/test_warnings.py --- a/Lib/test/test_warnings.py +++ b/Lib/test/test_warnings.py @@ -44,6 +44,7 @@ """Basic bookkeeping required for testing.""" def setUp(self): + self.old_unittest_module = unittest.case.warnings # The __warningregistry__ needs to be in a pristine state for tests # to work properly. if '__warningregistry__' in globals(): @@ -55,10 +56,15 @@ # The 'warnings' module must be explicitly set so that the proper # interaction between _warnings and 'warnings' can be controlled. sys.modules['warnings'] = self.module + # Ensure that unittest.TestCase.assertWarns() uses the same warnings + # module than warnings.catch_warnings(). Otherwise, + # warnings.catch_warnings() will be unable to remove the added filter. + unittest.case.warnings = self.module super(BaseTest, self).setUp() def tearDown(self): sys.modules['warnings'] = original_warnings + unittest.case.warnings = self.old_unittest_module super(BaseTest, self).tearDown() class PublicAPITests(BaseTest): -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 3 00:10:11 2015 From: python-checkins at python.org (victor.stinner) Date: Wed, 02 Sep 2015 22:10:11 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Merge_3=2E4_=28test=5Fwarnings=29?= Message-ID: <20150902221010.101496.99869@psf.io> https://hg.python.org/cpython/rev/1afb86afe7a9 changeset: 97595:1afb86afe7a9 branch: 3.5 parent: 97592:bc1b0aaf1280 parent: 97594:c1396d28c440 user: Victor Stinner date: Thu Sep 03 00:09:26 2015 +0200 summary: Merge 3.4 (test_warnings) files: Lib/test/test_warnings.py | 6 ++++++ 1 files changed, 6 insertions(+), 0 deletions(-) diff --git a/Lib/test/test_warnings.py b/Lib/test/test_warnings.py --- a/Lib/test/test_warnings.py +++ b/Lib/test/test_warnings.py @@ -44,6 +44,7 @@ """Basic bookkeeping required for testing.""" def setUp(self): + self.old_unittest_module = unittest.case.warnings # The __warningregistry__ needs to be in a pristine state for tests # to work properly. if '__warningregistry__' in globals(): @@ -55,10 +56,15 @@ # The 'warnings' module must be explicitly set so that the proper # interaction between _warnings and 'warnings' can be controlled. sys.modules['warnings'] = self.module + # Ensure that unittest.TestCase.assertWarns() uses the same warnings + # module than warnings.catch_warnings(). Otherwise, + # warnings.catch_warnings() will be unable to remove the added filter. + unittest.case.warnings = self.module super(BaseTest, self).setUp() def tearDown(self): sys.modules['warnings'] = original_warnings + unittest.case.warnings = self.old_unittest_module super(BaseTest, self).tearDown() class PublicAPITests(BaseTest): -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 3 00:10:17 2015 From: python-checkins at python.org (victor.stinner) Date: Wed, 02 Sep 2015 22:10:17 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge_3=2E5_=28test=5Fwarnings=29?= Message-ID: <20150902221011.27705.26704@psf.io> https://hg.python.org/cpython/rev/b7efc37b2d7e changeset: 97596:b7efc37b2d7e parent: 97593:e1fc4b60739b parent: 97595:1afb86afe7a9 user: Victor Stinner date: Thu Sep 03 00:09:37 2015 +0200 summary: Merge 3.5 (test_warnings) files: Lib/test/test_warnings.py | 6 ++++++ 1 files changed, 6 insertions(+), 0 deletions(-) diff --git a/Lib/test/test_warnings.py b/Lib/test/test_warnings.py --- a/Lib/test/test_warnings.py +++ b/Lib/test/test_warnings.py @@ -44,6 +44,7 @@ """Basic bookkeeping required for testing.""" def setUp(self): + self.old_unittest_module = unittest.case.warnings # The __warningregistry__ needs to be in a pristine state for tests # to work properly. if '__warningregistry__' in globals(): @@ -55,10 +56,15 @@ # The 'warnings' module must be explicitly set so that the proper # interaction between _warnings and 'warnings' can be controlled. sys.modules['warnings'] = self.module + # Ensure that unittest.TestCase.assertWarns() uses the same warnings + # module than warnings.catch_warnings(). Otherwise, + # warnings.catch_warnings() will be unable to remove the added filter. + unittest.case.warnings = self.module super(BaseTest, self).setUp() def tearDown(self): sys.modules['warnings'] = original_warnings + unittest.case.warnings = self.old_unittest_module super(BaseTest, self).tearDown() class PublicAPITests(BaseTest): -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 3 00:16:41 2015 From: python-checkins at python.org (victor.stinner) Date: Wed, 02 Sep 2015 22:16:41 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogTWVyZ2UgMy41IChtb25vdG9uaWMp?= Message-ID: <20150902221641.17961.2342@psf.io> https://hg.python.org/cpython/rev/4396ffb6e2e7 changeset: 97599:4396ffb6e2e7 parent: 97596:b7efc37b2d7e parent: 97598:546f02960e83 user: Victor Stinner date: Thu Sep 03 00:15:23 2015 +0200 summary: Merge 3.5 (monotonic) files: Python/pytime.c | 16 +++------------- 1 files changed, 3 insertions(+), 13 deletions(-) diff --git a/Python/pytime.c b/Python/pytime.c --- a/Python/pytime.c +++ b/Python/pytime.c @@ -601,12 +601,8 @@ static int -pymonotonic_new(_PyTime_t *tp, _Py_clock_info_t *info, int raise) +pymonotonic(_PyTime_t *tp, _Py_clock_info_t *info, int raise) { -#ifdef Py_DEBUG - static int last_set = 0; - static _PyTime_t last = 0; -#endif #if defined(MS_WINDOWS) ULONGLONG result; @@ -698,12 +694,6 @@ if (_PyTime_FromTimespec(tp, &ts, raise) < 0) return -1; #endif -#ifdef Py_DEBUG - /* monotonic clock cannot go backward */ - assert(!last_set || last <= *tp); - last = *tp; - last_set = 1; -#endif return 0; } @@ -711,7 +701,7 @@ _PyTime_GetMonotonicClock(void) { _PyTime_t t; - if (pymonotonic_new(&t, NULL, 0) < 0) { + if (pymonotonic(&t, NULL, 0) < 0) { /* should not happen, _PyTime_Init() checked that monotonic clock at startup */ assert(0); @@ -725,7 +715,7 @@ int _PyTime_GetMonotonicClockWithInfo(_PyTime_t *tp, _Py_clock_info_t *info) { - return pymonotonic_new(tp, info, 1); + return pymonotonic(tp, info, 1); } int -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 3 00:16:43 2015 From: python-checkins at python.org (victor.stinner) Date: Wed, 02 Sep 2015 22:16:43 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI0NzA3?= =?utf-8?q?=3A_Remove_assertion_in_monotonic_clock?= Message-ID: <20150902221641.68871.72691@psf.io> https://hg.python.org/cpython/rev/4c90b4a4fffa changeset: 97597:4c90b4a4fffa branch: 3.5 parent: 97595:1afb86afe7a9 user: Victor Stinner date: Thu Sep 03 00:13:46 2015 +0200 summary: Issue #24707: Remove assertion in monotonic clock Don't check anymore at runtime that the monotonic clock doesn't go backward. Yes, it happens. It occurs sometimes each month on a Debian buildbot slave running in a VM. The problem is that Python cannot do anything useful if a monotonic clock goes backward. It was decided in the PEP 418 to not fix the system, but only expose the clock provided by the OS. files: Python/pytime.c | 10 ---------- 1 files changed, 0 insertions(+), 10 deletions(-) diff --git a/Python/pytime.c b/Python/pytime.c --- a/Python/pytime.c +++ b/Python/pytime.c @@ -533,10 +533,6 @@ static int pymonotonic_new(_PyTime_t *tp, _Py_clock_info_t *info, int raise) { -#ifdef Py_DEBUG - static int last_set = 0; - static _PyTime_t last = 0; -#endif #if defined(MS_WINDOWS) ULONGLONG result; @@ -628,12 +624,6 @@ if (_PyTime_FromTimespec(tp, &ts, raise) < 0) return -1; #endif -#ifdef Py_DEBUG - /* monotonic clock cannot go backward */ - assert(!last_set || last <= *tp); - last = *tp; - last_set = 1; -#endif return 0; } -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 3 00:16:43 2015 From: python-checkins at python.org (victor.stinner) Date: Wed, 02 Sep 2015 22:16:43 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogb29wcywgcmVuYW1l?= =?utf-8?q?_pymonotonic=5Fnew=28=29_to_pymonotonic=28=29?= Message-ID: <20150902221641.66854.21873@psf.io> https://hg.python.org/cpython/rev/546f02960e83 changeset: 97598:546f02960e83 branch: 3.5 user: Victor Stinner date: Thu Sep 03 00:14:58 2015 +0200 summary: oops, rename pymonotonic_new() to pymonotonic() I was not supposed to commit the function with the name pymonotonic_new(). I forgot to rename it. files: Python/pytime.c | 6 +++--- 1 files changed, 3 insertions(+), 3 deletions(-) diff --git a/Python/pytime.c b/Python/pytime.c --- a/Python/pytime.c +++ b/Python/pytime.c @@ -531,7 +531,7 @@ static int -pymonotonic_new(_PyTime_t *tp, _Py_clock_info_t *info, int raise) +pymonotonic(_PyTime_t *tp, _Py_clock_info_t *info, int raise) { #if defined(MS_WINDOWS) ULONGLONG result; @@ -631,7 +631,7 @@ _PyTime_GetMonotonicClock(void) { _PyTime_t t; - if (pymonotonic_new(&t, NULL, 0) < 0) { + if (pymonotonic(&t, NULL, 0) < 0) { /* should not happen, _PyTime_Init() checked that monotonic clock at startup */ assert(0); @@ -645,7 +645,7 @@ int _PyTime_GetMonotonicClockWithInfo(_PyTime_t *tp, _Py_clock_info_t *info) { - return pymonotonic_new(tp, info, 1); + return pymonotonic(tp, info, 1); } int -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 3 01:50:03 2015 From: python-checkins at python.org (victor.stinner) Date: Wed, 02 Sep 2015 23:50:03 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Rewrite_eintr=5Ftester=2Ep?= =?utf-8?q?y_to_avoid_os=2Efork=28=29?= Message-ID: <20150902235002.27711.37209@psf.io> https://hg.python.org/cpython/rev/ed0e6a9c11af changeset: 97600:ed0e6a9c11af user: Victor Stinner date: Thu Sep 03 01:38:44 2015 +0200 summary: Rewrite eintr_tester.py to avoid os.fork() eintr_tester.py calls signal.setitimer() to send signals to the current process every 100 ms. The test sometimes hangs on FreeBSD. Maybe there is a race condition in the child process after fork(). It's unsafe to run arbitrary code after fork(). This change replace os.fork() with a regular call to subprocess.Popen(). This change avoids the risk of having a child process which continue to execute eintr_tester.py instead of exiting. It also ensures that the child process doesn't inherit unexpected file descriptors by mistake. Since I'm unable to reproduce the issue on FreeBSD, I will have to watch FreeBSD buildbots to check if the issue is fixed or not. Remove previous attempt to debug: remove call to faulthandler.dump_traceback_later(). files: Lib/test/eintrdata/eintr_tester.py | 257 ++++++++++------ 1 files changed, 157 insertions(+), 100 deletions(-) diff --git a/Lib/test/eintrdata/eintr_tester.py b/Lib/test/eintrdata/eintr_tester.py --- a/Lib/test/eintrdata/eintr_tester.py +++ b/Lib/test/eintrdata/eintr_tester.py @@ -8,12 +8,13 @@ sub-second periodicity (contrarily to signal()). """ -import faulthandler import io import os import select import signal import socket +import subprocess +import sys import time import unittest @@ -29,7 +30,7 @@ # signal delivery periodicity signal_period = 0.1 # default sleep time for tests - should obviously have: - #?sleep_time > signal_period + # sleep_time > signal_period sleep_time = 0.2 @classmethod @@ -37,17 +38,10 @@ cls.orig_handler = signal.signal(signal.SIGALRM, lambda *args: None) signal.setitimer(signal.ITIMER_REAL, cls.signal_delay, cls.signal_period) - if hasattr(faulthandler, 'dump_traceback_later'): - # Most tests take less than 30 seconds, so 15 minutes should be - # enough. dump_traceback_later() is implemented with a thread, but - # pthread_sigmask() is used to mask all signaled on this thread. - faulthandler.dump_traceback_later(15 * 60, exit=True) @classmethod def stop_alarm(cls): signal.setitimer(signal.ITIMER_REAL, 0, 0) - if hasattr(faulthandler, 'cancel_dump_traceback_later'): - faulthandler.cancel_dump_traceback_later() @classmethod def tearDownClass(cls): @@ -59,18 +53,22 @@ # default sleep time time.sleep(cls.sleep_time) + def subprocess(self, *args, **kw): + cmd_args = (sys.executable, '-c') + args + return subprocess.Popen(cmd_args, **kw) + @unittest.skipUnless(hasattr(signal, "setitimer"), "requires setitimer()") class OSEINTRTest(EINTRBaseTest): """ EINTR tests for the os module. """ + def new_sleep_process(self): + code = 'import time; time.sleep(%r)' % self.sleep_time + return self.subprocess(code) + def _test_wait_multiple(self, wait_func): num = 3 - for _ in range(num): - pid = os.fork() - if pid == 0: - self._sleep() - os._exit(0) + processes = [self.new_sleep_process() for _ in range(num)] for _ in range(num): wait_func() @@ -82,12 +80,8 @@ self._test_wait_multiple(lambda: os.wait3(0)) def _test_wait_single(self, wait_func): - pid = os.fork() - if pid == 0: - self._sleep() - os._exit(0) - else: - wait_func(pid) + proc = self.new_sleep_process() + wait_func(proc.pid) def test_waitpid(self): self._test_wait_single(lambda pid: os.waitpid(pid, 0)) @@ -105,19 +99,24 @@ # atomic datas = [b"hello", b"world", b"spam"] - pid = os.fork() - if pid == 0: - os.close(rd) - for data in datas: - # let the parent block on read() - self._sleep() - os.write(wr, data) - os._exit(0) - else: - self.addCleanup(os.waitpid, pid, 0) + code = '\n'.join(( + 'import os, sys, time', + '', + 'wr = int(sys.argv[1])', + 'datas = %r' % datas, + 'sleep_time = %r' % self.sleep_time, + '', + 'for data in datas:', + ' # let the parent block on read()', + ' time.sleep(sleep_time)', + ' os.write(wr, data)', + )) + + with self.subprocess(code, str(wr), pass_fds=[wr]) as proc: os.close(wr) for data in datas: self.assertEqual(data, os.read(rd, len(data))) + self.assertEqual(proc.wait(), 0) def test_write(self): rd, wr = os.pipe() @@ -127,23 +126,34 @@ # we must write enough data for the write() to block data = b"xyz" * support.PIPE_MAX_SIZE - pid = os.fork() - if pid == 0: - os.close(wr) - read_data = io.BytesIO() - # let the parent block on write() - self._sleep() - while len(read_data.getvalue()) < len(data): - chunk = os.read(rd, 2 * len(data)) - read_data.write(chunk) - self.assertEqual(read_data.getvalue(), data) - os._exit(0) - else: + code = '\n'.join(( + 'import io, os, sys, time', + '', + 'rd = int(sys.argv[1])', + 'sleep_time = %r' % self.sleep_time, + 'data = b"xyz" * %s' % support.PIPE_MAX_SIZE, + 'data_len = len(data)', + '', + '# let the parent block on write()', + 'time.sleep(sleep_time)', + '', + 'read_data = io.BytesIO()', + 'while len(read_data.getvalue()) < data_len:', + ' chunk = os.read(rd, 2 * data_len)', + ' read_data.write(chunk)', + '', + 'value = read_data.getvalue()', + 'if value != data:', + ' raise Exception("read error: %s vs %s bytes"', + ' % (len(value), data_len))', + )) + + with self.subprocess(code, str(rd), pass_fds=[rd]) as proc: os.close(rd) written = 0 while written < len(data): written += os.write(wr, memoryview(data)[written:]) - self.assertEqual(0, os.waitpid(pid, 0)[1]) + self.assertEqual(proc.wait(), 0) @unittest.skipUnless(hasattr(signal, "setitimer"), "requires setitimer()") @@ -159,19 +169,32 @@ # single-byte payload guard us against partial recv datas = [b"x", b"y", b"z"] - pid = os.fork() - if pid == 0: - rd.close() - for data in datas: - # let the parent block on recv() - self._sleep() - wr.sendall(data) - os._exit(0) - else: - self.addCleanup(os.waitpid, pid, 0) + code = '\n'.join(( + 'import os, socket, sys, time', + '', + 'fd = int(sys.argv[1])', + 'family = %s' % int(wr.family), + 'sock_type = %s' % int(wr.type), + 'datas = %r' % datas, + 'sleep_time = %r' % self.sleep_time, + '', + 'wr = socket.fromfd(fd, family, sock_type)', + 'os.close(fd)', + '', + 'with wr:', + ' for data in datas:', + ' # let the parent block on recv()', + ' time.sleep(sleep_time)', + ' wr.sendall(data)', + )) + + fd = wr.fileno() + proc = self.subprocess(code, str(fd), pass_fds=[fd]) + with proc: wr.close() for data in datas: self.assertEqual(data, recv_func(rd, len(data))) + self.assertEqual(proc.wait(), 0) def test_recv(self): self._test_recv(socket.socket.recv) @@ -188,25 +211,43 @@ # we must send enough data for the send() to block data = b"xyz" * (support.SOCK_MAX_SIZE // 3) - pid = os.fork() - if pid == 0: - wr.close() - # let the parent block on send() - self._sleep() - received_data = bytearray(len(data)) - n = 0 - while n < len(data): - n += rd.recv_into(memoryview(received_data)[n:]) - self.assertEqual(received_data, data) - os._exit(0) - else: + code = '\n'.join(( + 'import os, socket, sys, time', + '', + 'fd = int(sys.argv[1])', + 'family = %s' % int(rd.family), + 'sock_type = %s' % int(rd.type), + 'sleep_time = %r' % self.sleep_time, + 'data = b"xyz" * %s' % (support.SOCK_MAX_SIZE // 3), + 'data_len = len(data)', + '', + 'rd = socket.fromfd(fd, family, sock_type)', + 'os.close(fd)', + '', + 'with rd:', + ' # let the parent block on send()', + ' time.sleep(sleep_time)', + '', + ' received_data = bytearray(data_len)', + ' n = 0', + ' while n < data_len:', + ' n += rd.recv_into(memoryview(received_data)[n:])', + '', + 'if received_data != data:', + ' raise Exception("recv error: %s vs %s bytes"', + ' % (len(received_data), data_len))', + )) + + fd = rd.fileno() + proc = self.subprocess(code, str(fd), pass_fds=[fd]) + with proc: rd.close() written = 0 while written < len(data): sent = send_func(wr, memoryview(data)[written:]) # sendall() returns None written += len(data) if sent is None else sent - self.assertEqual(0, os.waitpid(pid, 0)[1]) + self.assertEqual(proc.wait(), 0) def test_send(self): self._test_send(socket.socket.send) @@ -223,45 +264,60 @@ self.addCleanup(sock.close) sock.bind((support.HOST, 0)) - _, port = sock.getsockname() + port = sock.getsockname()[1] sock.listen() - pid = os.fork() - if pid == 0: - # let parent block on accept() - self._sleep() - with socket.create_connection((support.HOST, port)): - self._sleep() - os._exit(0) - else: - self.addCleanup(os.waitpid, pid, 0) + code = '\n'.join(( + 'import socket, time', + '', + 'host = %r' % support.HOST, + 'port = %s' % port, + 'sleep_time = %r' % self.sleep_time, + '', + '# let parent block on accept()', + 'time.sleep(sleep_time)', + 'with socket.create_connection((host, port)):', + ' time.sleep(sleep_time)', + )) + + with self.subprocess(code) as proc: client_sock, _ = sock.accept() client_sock.close() + self.assertEqual(proc.wait(), 0) @unittest.skipUnless(hasattr(os, 'mkfifo'), 'needs mkfifo()') def _test_open(self, do_open_close_reader, do_open_close_writer): + filename = support.TESTFN + # Use a fifo: until the child opens it for reading, the parent will # block when trying to open it for writing. - support.unlink(support.TESTFN) - os.mkfifo(support.TESTFN) - self.addCleanup(support.unlink, support.TESTFN) + support.unlink(filename) + os.mkfifo(filename) + self.addCleanup(support.unlink, filename) - pid = os.fork() - if pid == 0: - # let the parent block - self._sleep() - do_open_close_reader(support.TESTFN) - os._exit(0) - else: - self.addCleanup(os.waitpid, pid, 0) - do_open_close_writer(support.TESTFN) + code = '\n'.join(( + 'import os, time', + '', + 'path = %a' % filename, + 'sleep_time = %r' % self.sleep_time, + '', + '# let the parent block', + 'time.sleep(sleep_time)', + '', + do_open_close_reader, + )) + + with self.subprocess(code) as proc: + do_open_close_writer(filename) + + self.assertEqual(proc.wait(), 0) def test_open(self): - self._test_open(lambda path: open(path, 'r').close(), + self._test_open("open(path, 'r').close()", lambda path: open(path, 'w').close()) def test_os_open(self): - self._test_open(lambda path: os.close(os.open(path, os.O_RDONLY)), + self._test_open("os.close(os.open(path, os.O_RDONLY))", lambda path: os.close(os.open(path, os.O_WRONLY))) @@ -298,20 +354,21 @@ old_handler = signal.signal(signum, lambda *args: None) self.addCleanup(signal.signal, signum, old_handler) + code = '\n'.join(( + 'import os, time', + 'pid = %s' % os.getpid(), + 'signum = %s' % int(signum), + 'sleep_time = %r' % self.sleep_time, + 'time.sleep(sleep_time)', + 'os.kill(pid, signum)', + )) + t0 = time.monotonic() - child_pid = os.fork() - if child_pid == 0: - # child - try: - self._sleep() - os.kill(pid, signum) - finally: - os._exit(0) - else: + with self.subprocess(code) as proc: # parent signal.sigwaitinfo([signum]) dt = time.monotonic() - t0 - os.waitpid(child_pid, 0) + self.assertEqual(proc.wait(), 0) self.assertGreaterEqual(dt, self.sleep_time) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 3 04:09:23 2015 From: python-checkins at python.org (terry.reedy) Date: Thu, 03 Sep 2015 02:09:23 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzIxMTky?= =?utf-8?q?=3A_Change_=27RUN=27_back_to_=27RESTART=27_when_running_editor_?= =?utf-8?q?file=2E?= Message-ID: <20150903020922.11270.28186@psf.io> https://hg.python.org/cpython/rev/69ea73015132 changeset: 97601:69ea73015132 branch: 2.7 parent: 97586:699670303a43 user: Terry Jan Reedy date: Wed Sep 02 22:07:31 2015 -0400 summary: Issue #21192: Change 'RUN' back to 'RESTART' when running editor file. files: Lib/idlelib/PyShell.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Lib/idlelib/PyShell.py b/Lib/idlelib/PyShell.py --- a/Lib/idlelib/PyShell.py +++ b/Lib/idlelib/PyShell.py @@ -500,7 +500,7 @@ console.stop_readline() # annotate restart in shell window and mark it console.text.delete("iomark", "end-1c") - tag = 'RUN ' + filename if filename else 'RESTART Shell' + tag = 'RESTART: ' + (filename if filename else 'Shell') halfbar = ((int(console.width) -len(tag) - 4) // 2) * '=' console.write("\n{0} {1} {0}".format(halfbar, tag)) console.text.mark_set("restart", "end-1c") -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 3 04:09:23 2015 From: python-checkins at python.org (terry.reedy) Date: Thu, 03 Sep 2015 02:09:23 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzIxMTky?= =?utf-8?q?=3A_Change_=27RUN=27_back_to_=27RESTART=27_when_running_editor_?= =?utf-8?q?file=2E?= Message-ID: <20150903020922.15706.1404@psf.io> https://hg.python.org/cpython/rev/130a3edcac1d changeset: 97602:130a3edcac1d branch: 3.4 parent: 97594:c1396d28c440 user: Terry Jan Reedy date: Wed Sep 02 22:07:44 2015 -0400 summary: Issue #21192: Change 'RUN' back to 'RESTART' when running editor file. files: Lib/idlelib/PyShell.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Lib/idlelib/PyShell.py b/Lib/idlelib/PyShell.py --- a/Lib/idlelib/PyShell.py +++ b/Lib/idlelib/PyShell.py @@ -487,7 +487,7 @@ console.stop_readline() # annotate restart in shell window and mark it console.text.delete("iomark", "end-1c") - tag = 'RUN ' + filename if filename else 'RESTART Shell' + tag = 'RESTART: ' + (filename if filename else 'Shell') halfbar = ((int(console.width) -len(tag) - 4) // 2) * '=' console.write("\n{0} {1} {0}".format(halfbar, tag)) console.text.mark_set("restart", "end-1c") -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 3 04:09:23 2015 From: python-checkins at python.org (terry.reedy) Date: Thu, 03 Sep 2015 02:09:23 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Merge_with_3=2E4?= Message-ID: <20150903020922.14863.97664@psf.io> https://hg.python.org/cpython/rev/ff1817c34423 changeset: 97603:ff1817c34423 branch: 3.5 parent: 97598:546f02960e83 parent: 97602:130a3edcac1d user: Terry Jan Reedy date: Wed Sep 02 22:08:03 2015 -0400 summary: Merge with 3.4 files: Lib/idlelib/PyShell.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Lib/idlelib/PyShell.py b/Lib/idlelib/PyShell.py --- a/Lib/idlelib/PyShell.py +++ b/Lib/idlelib/PyShell.py @@ -487,7 +487,7 @@ console.stop_readline() # annotate restart in shell window and mark it console.text.delete("iomark", "end-1c") - tag = 'RUN ' + filename if filename else 'RESTART Shell' + tag = 'RESTART: ' + (filename if filename else 'Shell') halfbar = ((int(console.width) -len(tag) - 4) // 2) * '=' console.write("\n{0} {1} {0}".format(halfbar, tag)) console.text.mark_set("restart", "end-1c") -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 3 04:09:23 2015 From: python-checkins at python.org (terry.reedy) Date: Thu, 03 Sep 2015 02:09:23 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge_with_3=2E5?= Message-ID: <20150903020922.14881.40560@psf.io> https://hg.python.org/cpython/rev/e3167e358ee1 changeset: 97604:e3167e358ee1 parent: 97600:ed0e6a9c11af parent: 97603:ff1817c34423 user: Terry Jan Reedy date: Wed Sep 02 22:08:21 2015 -0400 summary: Merge with 3.5 files: Lib/idlelib/PyShell.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Lib/idlelib/PyShell.py b/Lib/idlelib/PyShell.py --- a/Lib/idlelib/PyShell.py +++ b/Lib/idlelib/PyShell.py @@ -487,7 +487,7 @@ console.stop_readline() # annotate restart in shell window and mark it console.text.delete("iomark", "end-1c") - tag = 'RUN ' + filename if filename else 'RESTART Shell' + tag = 'RESTART: ' + (filename if filename else 'Shell') halfbar = ((int(console.width) -len(tag) - 4) // 2) * '=' console.write("\n{0} {1} {0}".format(halfbar, tag)) console.text.mark_set("restart", "end-1c") -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 3 08:45:16 2015 From: python-checkins at python.org (senthil.kumaran) Date: Thu, 03 Sep 2015 06:45:16 +0000 Subject: [Python-checkins] =?utf-8?q?test=3A_Update_and_Test=2E?= Message-ID: <20150903064516.15730.23962@psf.io> https://hg.python.org/test/rev/2d68a07fa6eb changeset: 224:2d68a07fa6eb user: Senthil Kumaran date: Wed Sep 02 23:45:10 2015 -0700 summary: Update and Test. files: README.txt | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) diff --git a/README.txt b/README.txt --- a/README.txt +++ b/README.txt @@ -2,3 +2,5 @@ https://docs.python.org/devguide/coredev.html + +Update and Test. -- Repository URL: https://hg.python.org/test From python-checkins at python.org Thu Sep 3 09:14:38 2015 From: python-checkins at python.org (victor.stinner) Date: Thu, 03 Sep 2015 07:14:38 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2323517=3A_fromtime?= =?utf-8?q?stamp=28=29_and_utcfromtimestamp=28=29_methods_of?= Message-ID: <20150903071432.66872.7924@psf.io> https://hg.python.org/cpython/rev/bf634dfe076f changeset: 97605:bf634dfe076f user: Victor Stinner date: Thu Sep 03 09:06:44 2015 +0200 summary: Issue #23517: fromtimestamp() and utcfromtimestamp() methods of datetime.datetime now round microseconds to nearest with ties going away from zero (ROUND_HALF_UP), as Python 2 and Python older than 3.3, instead of rounding towards -Infinity (ROUND_FLOOR). files: Lib/datetime.py | 4 ++-- Lib/test/datetimetester.py | 11 ++++++----- Misc/NEWS | 5 +++++ Modules/_datetimemodule.c | 2 +- 4 files changed, 14 insertions(+), 8 deletions(-) diff --git a/Lib/datetime.py b/Lib/datetime.py --- a/Lib/datetime.py +++ b/Lib/datetime.py @@ -1384,7 +1384,7 @@ converter = _time.localtime if tz is None else _time.gmtime t, frac = divmod(t, 1.0) - us = int(frac * 1e6) + us = _round_half_up(frac * 1e6) # If timestamp is less than one microsecond smaller than a # full second, us can be rounded up to 1000000. In this case, @@ -1404,7 +1404,7 @@ def utcfromtimestamp(cls, t): """Construct a naive UTC datetime from a POSIX timestamp.""" t, frac = divmod(t, 1.0) - us = int(frac * 1e6) + us = _round_half_up(frac * 1e6) # If timestamp is less than one microsecond smaller than a # full second, us can be rounded up to 1000000. In this case, diff --git a/Lib/test/datetimetester.py b/Lib/test/datetimetester.py --- a/Lib/test/datetimetester.py +++ b/Lib/test/datetimetester.py @@ -1847,6 +1847,7 @@ zero = fts(0) self.assertEqual(zero.second, 0) self.assertEqual(zero.microsecond, 0) + one = fts(1e-6) try: minus_one = fts(-1e-6) except OSError: @@ -1857,22 +1858,22 @@ self.assertEqual(minus_one.microsecond, 999999) t = fts(-1e-8) - self.assertEqual(t, minus_one) + self.assertEqual(t, zero) t = fts(-9e-7) self.assertEqual(t, minus_one) t = fts(-1e-7) - self.assertEqual(t, minus_one) + self.assertEqual(t, zero) t = fts(1e-7) self.assertEqual(t, zero) t = fts(9e-7) - self.assertEqual(t, zero) + self.assertEqual(t, one) t = fts(0.99999949) self.assertEqual(t.second, 0) self.assertEqual(t.microsecond, 999999) t = fts(0.9999999) - self.assertEqual(t.second, 0) - self.assertEqual(t.microsecond, 999999) + self.assertEqual(t.second, 1) + self.assertEqual(t.microsecond, 0) def test_insane_fromtimestamp(self): # It's possible that some platform maps time_t to double, diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -17,6 +17,11 @@ Library ------- +- Issue #23517: fromtimestamp() and utcfromtimestamp() methods of + datetime.datetime now round microseconds to nearest with ties going away from + zero (ROUND_HALF_UP), as Python 2 and Python older than 3.3, instead of + rounding towards -Infinity (ROUND_FLOOR). + - Issue #23517: datetime.timedelta constructor now rounds microseconds to nearest with ties going away from zero (ROUND_HALF_UP), as Python 2 and Python older than 3.3, instead of rounding to nearest with ties going to diff --git a/Modules/_datetimemodule.c b/Modules/_datetimemodule.c --- a/Modules/_datetimemodule.c +++ b/Modules/_datetimemodule.c @@ -4078,7 +4078,7 @@ long us; if (_PyTime_ObjectToTimeval(timestamp, - &timet, &us, _PyTime_ROUND_FLOOR) == -1) + &timet, &us, _PyTime_ROUND_HALF_UP) == -1) return NULL; return datetime_from_timet_and_us(cls, f, timet, (int)us, tzinfo); -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 3 09:47:55 2015 From: python-checkins at python.org (victor.stinner) Date: Thu, 03 Sep 2015 07:47:55 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?b?IE1lcmdlIDMuNCAodGVzdF9nZGIp?= Message-ID: <20150903074755.17967.47448@psf.io> https://hg.python.org/cpython/rev/ecd43087e826 changeset: 97607:ecd43087e826 branch: 3.5 parent: 97603:ff1817c34423 parent: 97606:63c697473515 user: Victor Stinner date: Thu Sep 03 09:46:11 2015 +0200 summary: Merge 3.4 (test_gdb) files: Lib/test/test_gdb.py | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-) diff --git a/Lib/test/test_gdb.py b/Lib/test/test_gdb.py --- a/Lib/test/test_gdb.py +++ b/Lib/test/test_gdb.py @@ -36,7 +36,8 @@ # Regex to parse: # 'GNU gdb (GDB; SUSE Linux Enterprise 12) 7.7\n' -> 7.7 # 'GNU gdb (GDB) Fedora 7.9.1-17.fc22\n' -> 7.9 - match = re.search("^GNU gdb .*? (\d+)\.(\d)", version) + # 'GNU gdb 6.1.1 [FreeBSD]\n' + match = re.search("^GNU gdb.*? (\d+)\.(\d)", version) if match is None: raise Exception("unable to parse GDB version: %r" % version) return (version, int(match.group(1)), int(match.group(2))) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 3 09:47:59 2015 From: python-checkins at python.org (victor.stinner) Date: Thu, 03 Sep 2015 07:47:59 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogdGVzdF9nZGI6IGZp?= =?utf-8?q?x_regex_to_parse_GDB_version_for_=27GNU_gdb_6=2E1=2E1_=5BFreeBS?= =?utf-8?b?RF1cbic=?= Message-ID: <20150903074755.114754.51327@psf.io> https://hg.python.org/cpython/rev/63c697473515 changeset: 97606:63c697473515 branch: 3.4 parent: 97602:130a3edcac1d user: Victor Stinner date: Thu Sep 03 09:45:53 2015 +0200 summary: test_gdb: fix regex to parse GDB version for 'GNU gdb 6.1.1 [FreeBSD]\n' files: Lib/test/test_gdb.py | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-) diff --git a/Lib/test/test_gdb.py b/Lib/test/test_gdb.py --- a/Lib/test/test_gdb.py +++ b/Lib/test/test_gdb.py @@ -36,7 +36,8 @@ # Regex to parse: # 'GNU gdb (GDB; SUSE Linux Enterprise 12) 7.7\n' -> 7.7 # 'GNU gdb (GDB) Fedora 7.9.1-17.fc22\n' -> 7.9 - match = re.search("^GNU gdb .*? (\d+)\.(\d)", version) + # 'GNU gdb 6.1.1 [FreeBSD]\n' + match = re.search("^GNU gdb.*? (\d+)\.(\d)", version) if match is None: raise Exception("unable to parse GDB version: %r" % version) return (version, int(match.group(1)), int(match.group(2))) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 3 09:47:59 2015 From: python-checkins at python.org (victor.stinner) Date: Thu, 03 Sep 2015 07:47:59 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogTWVyZ2UgMy41ICh0ZXN0X2dkYik=?= Message-ID: <20150903074755.68879.51368@psf.io> https://hg.python.org/cpython/rev/8a3303a8a87c changeset: 97608:8a3303a8a87c parent: 97605:bf634dfe076f parent: 97607:ecd43087e826 user: Victor Stinner date: Thu Sep 03 09:46:24 2015 +0200 summary: Merge 3.5 (test_gdb) files: Lib/test/test_gdb.py | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-) diff --git a/Lib/test/test_gdb.py b/Lib/test/test_gdb.py --- a/Lib/test/test_gdb.py +++ b/Lib/test/test_gdb.py @@ -36,7 +36,8 @@ # Regex to parse: # 'GNU gdb (GDB; SUSE Linux Enterprise 12) 7.7\n' -> 7.7 # 'GNU gdb (GDB) Fedora 7.9.1-17.fc22\n' -> 7.9 - match = re.search("^GNU gdb .*? (\d+)\.(\d)", version) + # 'GNU gdb 6.1.1 [FreeBSD]\n' + match = re.search("^GNU gdb.*? (\d+)\.(\d)", version) if match is None: raise Exception("unable to parse GDB version: %r" % version) return (version, int(match.group(1)), int(match.group(2))) -- Repository URL: https://hg.python.org/cpython From lp_benchmark_robot at intel.com Thu Sep 3 10:35:18 2015 From: lp_benchmark_robot at intel.com (lp_benchmark_robot) Date: Thu, 3 Sep 2015 08:35:18 +0000 Subject: [Python-checkins] Benchmark Results for Python Default 2015-09-03 Message-ID: <078AA0FFE8C7034097F90205717F504611D795F1@IRSMSX102.ger.corp.intel.com> Results for project python_default-nightly, build date 2015-09-03 06:02:10 commit: e3167e358ee12d40cf634604c3963c088d3f4223 revision date: 2015-09-03 05:08:21 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v3.4.3, with hash b4cbecbc0781e89a309d03b60a1f75f8499250e6 from 2015-02-25 12:15:33+00:00 ------------------------------------------------------------------------------------------ benchmark relative change since change since current rev with std_dev* last run v3.4.3 regrtest PGO ------------------------------------------------------------------------------------------ :-) django_v2 0.28538% 0.51081% 7.36233% 17.89420% :-( pybench 0.12949% 0.21134% -2.45927% 9.39660% :-( regex_v8 2.98155% 0.11154% -3.43417% 4.68367% :-| nbody 0.06750% -0.88811% -1.99983% 8.11790% :-( json_dump_v2 0.22754% -3.84994% -6.87025% 17.81087% :-| normal_startup 0.90353% -0.50056% -0.77492% 5.75392% ------------------------------------------------------------------------------------------ Note: Benchmark results are measured in seconds. * Relative Standard Deviation (Standard Deviation/Average) Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. No license (express or implied, by estoppel or otherwise) to any intellectual property rights is granted by this document. Intel disclaims all express and implied warranties, including without limitation, the implied warranties of merchantability, fitness for a particular purpose, and non-infringement, as well as any warranty arising from course of performance, course of dealing, or usage in trade. This document may contain information on products, services and/or processes in development. Contact your Intel representative to obtain the latest forecast, schedule, specifications and roadmaps. The products and services described may contain defects or errors known as errata which may cause deviations from published specifications. Current characterized errata are available on request. (C) 2015 Intel Corporation. From python-checkins at python.org Thu Sep 3 10:44:00 2015 From: python-checkins at python.org (victor.stinner) Date: Thu, 03 Sep 2015 08:44:00 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogdGVzdF9nZGI6IGVu?= =?utf-8?q?hance_regex_used_to_parse_the_GDB_version?= Message-ID: <20150903084400.14871.87087@psf.io> https://hg.python.org/cpython/rev/9e629c372a78 changeset: 97609:9e629c372a78 branch: 2.7 parent: 97601:69ea73015132 user: Victor Stinner date: Thu Sep 03 09:51:59 2015 +0200 summary: test_gdb: enhance regex used to parse the GDB version files: Lib/test/test_gdb.py | 43 +++++++++++++++++++++++-------- 1 files changed, 31 insertions(+), 12 deletions(-) diff --git a/Lib/test/test_gdb.py b/Lib/test/test_gdb.py --- a/Lib/test/test_gdb.py +++ b/Lib/test/test_gdb.py @@ -10,21 +10,36 @@ import unittest import sysconfig +from test import test_support from test.test_support import run_unittest, findfile -try: - gdb_version, _ = subprocess.Popen(["gdb", "-nx", "--version"], - stdout=subprocess.PIPE).communicate() -except OSError: - # This is what "no gdb" looks like. There may, however, be other - # errors that manifest this way too. - raise unittest.SkipTest("Couldn't find gdb on the path") -gdb_version_number = re.search("^GNU gdb [^\d]*(\d+)\.(\d)", gdb_version) -gdb_major_version = int(gdb_version_number.group(1)) -gdb_minor_version = int(gdb_version_number.group(2)) +def get_gdb_version(): + try: + proc = subprocess.Popen(["gdb", "-nx", "--version"], + stdout=subprocess.PIPE, + universal_newlines=True) + version = proc.communicate()[0] + except OSError: + # This is what "no gdb" looks like. There may, however, be other + # errors that manifest this way too. + raise unittest.SkipTest("Couldn't find gdb on the path") + + # Regex to parse: + # 'GNU gdb (GDB; SUSE Linux Enterprise 12) 7.7\n' -> 7.7 + # 'GNU gdb (GDB) Fedora 7.9.1-17.fc22\n' -> 7.9 + # 'GNU gdb 6.1.1 [FreeBSD]\n' + match = re.search("^GNU gdb.*? (\d+)\.(\d)", version) + if match is None: + raise Exception("unable to parse GDB version: %r" % version) + return (version, int(match.group(1)), int(match.group(2))) + +gdb_version, gdb_major_version, gdb_minor_version = get_gdb_version() if gdb_major_version < 7: - raise unittest.SkipTest("gdb versions before 7.0 didn't support python embedding" - " Saw:\n" + gdb_version) + raise unittest.SkipTest("gdb versions before 7.0 didn't support python " + "embedding. Saw %s.%s:\n%s" + % (gdb_major_version, gdb_minor_version, + gdb_version)) + if sys.platform.startswith("sunos"): raise unittest.SkipTest("test doesn't work very well on Solaris") @@ -781,6 +796,10 @@ r".*\na = 1\nb = 2\nc = 3\n.*") def test_main(): + if test_support.verbose: + print("GDB version %s.%s:" % (gdb_major_version, gdb_minor_version)) + for line in gdb_version.splitlines(): + print(" " * 4 + line) run_unittest(PrettyPrintTests, PyListTests, StackNavigationTests, -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 3 10:44:01 2015 From: python-checkins at python.org (victor.stinner) Date: Thu, 03 Sep 2015 08:44:01 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogcHl0aG9uLWdkYi5w?= =?utf-8?q?y=3A_enhance_py-bt_command?= Message-ID: <20150903084400.101478.30220@psf.io> https://hg.python.org/cpython/rev/ffd869fa255d changeset: 97610:ffd869fa255d branch: 2.7 user: Victor Stinner date: Thu Sep 03 10:17:28 2015 +0200 summary: python-gdb.py: enhance py-bt command * Add py-bt-full command * py-bt now gives an output similar to a regular Python traceback * py-bt indicates: - if the garbage collector is running - if the thread is waiting for the GIL - detect PyCFunction_Call to get the name of the builtin function files: Lib/test/test_gdb.py | 127 +++++++++++++++++++- Tools/gdb/libpython.py | 183 ++++++++++++++++++++++++++-- 2 files changed, 292 insertions(+), 18 deletions(-) diff --git a/Lib/test/test_gdb.py b/Lib/test/test_gdb.py --- a/Lib/test/test_gdb.py +++ b/Lib/test/test_gdb.py @@ -13,6 +13,12 @@ from test import test_support from test.test_support import run_unittest, findfile +# Is this Python configured to support threads? +try: + import thread +except ImportError: + thread = None + def get_gdb_version(): try: proc = subprocess.Popen(["gdb", "-nx", "--version"], @@ -728,20 +734,133 @@ class PyBtTests(DebuggerTests): @unittest.skipIf(python_is_optimized(), "Python was compiled with optimizations") - def test_basic_command(self): + def test_bt(self): 'Verify that the "py-bt" command works' bt = self.get_stack_trace(script=self.get_sample_script(), cmds_after_breakpoint=['py-bt']) self.assertMultilineMatches(bt, r'''^.* -#[0-9]+ Frame 0x[0-9a-f]+, for file .*gdb_sample.py, line 7, in bar \(a=1, b=2, c=3\) +Traceback \(most recent call first\): + File ".*gdb_sample.py", line 10, in baz + print\(42\) + File ".*gdb_sample.py", line 7, in bar baz\(a, b, c\) -#[0-9]+ Frame 0x[0-9a-f]+, for file .*gdb_sample.py, line 4, in foo \(a=1, b=2, c=3\) + File ".*gdb_sample.py", line 4, in foo bar\(a, b, c\) -#[0-9]+ Frame 0x[0-9a-f]+, for file .*gdb_sample.py, line 12, in \(\) + File ".*gdb_sample.py", line 12, in foo\(1, 2, 3\) ''') + @unittest.skipIf(python_is_optimized(), + "Python was compiled with optimizations") + def test_bt_full(self): + 'Verify that the "py-bt-full" command works' + bt = self.get_stack_trace(script=self.get_sample_script(), + cmds_after_breakpoint=['py-bt-full']) + self.assertMultilineMatches(bt, + r'''^.* +#[0-9]+ Frame 0x-?[0-9a-f]+, for file .*gdb_sample.py, line 7, in bar \(a=1, b=2, c=3\) + baz\(a, b, c\) +#[0-9]+ Frame 0x-?[0-9a-f]+, for file .*gdb_sample.py, line 4, in foo \(a=1, b=2, c=3\) + bar\(a, b, c\) +#[0-9]+ Frame 0x-?[0-9a-f]+, for file .*gdb_sample.py, line 12, in \(\) + foo\(1, 2, 3\) +''') + + @unittest.skipUnless(thread, + "Python was compiled without thread support") + def test_threads(self): + 'Verify that "py-bt" indicates threads that are waiting for the GIL' + cmd = ''' +from threading import Thread + +class TestThread(Thread): + # These threads would run forever, but we'll interrupt things with the + # debugger + def run(self): + i = 0 + while 1: + i += 1 + +t = {} +for i in range(4): + t[i] = TestThread() + t[i].start() + +# Trigger a breakpoint on the main thread +print 42 + +''' + # Verify with "py-bt": + gdb_output = self.get_stack_trace(cmd, + cmds_after_breakpoint=['thread apply all py-bt']) + self.assertIn('Waiting for the GIL', gdb_output) + + # Verify with "py-bt-full": + gdb_output = self.get_stack_trace(cmd, + cmds_after_breakpoint=['thread apply all py-bt-full']) + self.assertIn('Waiting for the GIL', gdb_output) + + @unittest.skipIf(python_is_optimized(), + "Python was compiled with optimizations") + # Some older versions of gdb will fail with + # "Cannot find new threads: generic error" + # unless we add LD_PRELOAD=PATH-TO-libpthread.so.1 as a workaround + @unittest.skipUnless(thread, + "Python was compiled without thread support") + def test_gc(self): + 'Verify that "py-bt" indicates if a thread is garbage-collecting' + cmd = ('from gc import collect\n' + 'print 42\n' + 'def foo():\n' + ' collect()\n' + 'def bar():\n' + ' foo()\n' + 'bar()\n') + # Verify with "py-bt": + gdb_output = self.get_stack_trace(cmd, + cmds_after_breakpoint=['break update_refs', 'continue', 'py-bt'], + ) + self.assertIn('Garbage-collecting', gdb_output) + + # Verify with "py-bt-full": + gdb_output = self.get_stack_trace(cmd, + cmds_after_breakpoint=['break update_refs', 'continue', 'py-bt-full'], + ) + self.assertIn('Garbage-collecting', gdb_output) + + @unittest.skipIf(python_is_optimized(), + "Python was compiled with optimizations") + # Some older versions of gdb will fail with + # "Cannot find new threads: generic error" + # unless we add LD_PRELOAD=PATH-TO-libpthread.so.1 as a workaround + @unittest.skipUnless(thread, + "Python was compiled without thread support") + def test_pycfunction(self): + 'Verify that "py-bt" displays invocations of PyCFunction instances' + # Tested function must not be defined with METH_NOARGS or METH_O, + # otherwise call_function() doesn't call PyCFunction_Call() + cmd = ('from time import gmtime\n' + 'def foo():\n' + ' gmtime(1)\n' + 'def bar():\n' + ' foo()\n' + 'bar()\n') + # Verify with "py-bt": + gdb_output = self.get_stack_trace(cmd, + breakpoint='time_gmtime', + cmds_after_breakpoint=['bt', 'py-bt'], + ) + self.assertIn('= 3: + def write_unicode(file, text): + file.write(text) +else: + def write_unicode(file, text): + # Write a byte or unicode string to file. Unicode strings are encoded to + # ENCODING encoding with 'backslashreplace' error handler to avoid + # UnicodeEncodeError. + if isinstance(text, unicode): + text = text.encode(ENCODING, 'backslashreplace') + file.write(text) + class StringTruncated(RuntimeError): pass @@ -903,7 +918,12 @@ newline character''' if self.is_optimized_out(): return '(frame information optimized out)' - with open(self.filename(), 'r') as f: + filename = self.filename() + try: + f = open(filename, 'r') + except IOError: + return None + with f: all_lines = f.readlines() # Convert from 1-based current_line_num to 0-based list offset: return all_lines[self.current_line_num()-1] @@ -914,9 +934,9 @@ return out.write('Frame 0x%x, for file %s, line %i, in %s (' % (self.as_address(), - self.co_filename, + self.co_filename.proxyval(visited), self.current_line_num(), - self.co_name)) + self.co_name.proxyval(visited))) first = True for pyop_name, pyop_value in self.iter_locals(): if not first: @@ -929,6 +949,16 @@ out.write(')') + def print_traceback(self): + if self.is_optimized_out(): + sys.stdout.write(' (frame information optimized out)\n') + return + visited = set() + sys.stdout.write(' File "%s", line %i, in %s\n' + % (self.co_filename.proxyval(visited), + self.current_line_num(), + self.co_name.proxyval(visited))) + class PySetObjectPtr(PyObjectPtr): _typename = 'PySetObject' @@ -1222,6 +1252,23 @@ iter_frame = iter_frame.newer() return index + # We divide frames into: + # - "python frames": + # - "bytecode frames" i.e. PyEval_EvalFrameEx + # - "other python frames": things that are of interest from a python + # POV, but aren't bytecode (e.g. GC, GIL) + # - everything else + + def is_python_frame(self): + '''Is this a PyEval_EvalFrameEx frame, or some other important + frame? (see is_other_python_frame for what "important" means in this + context)''' + if self.is_evalframeex(): + return True + if self.is_other_python_frame(): + return True + return False + def is_evalframeex(self): '''Is this a PyEval_EvalFrameEx frame?''' if self._gdbframe.name() == 'PyEval_EvalFrameEx': @@ -1238,6 +1285,50 @@ return False + def is_other_python_frame(self): + '''Is this frame worth displaying in python backtraces? + Examples: + - waiting on the GIL + - garbage-collecting + - within a CFunction + If it is, return a descriptive string + For other frames, return False + ''' + if self.is_waiting_for_gil(): + return 'Waiting for the GIL' + elif self.is_gc_collect(): + return 'Garbage-collecting' + else: + # Detect invocations of PyCFunction instances: + older = self.older() + if older and older._gdbframe.name() == 'PyCFunction_Call': + # Within that frame: + # "func" is the local containing the PyObject* of the + # PyCFunctionObject instance + # "f" is the same value, but cast to (PyCFunctionObject*) + # "self" is the (PyObject*) of the 'self' + try: + # Use the prettyprinter for the func: + func = older._gdbframe.read_var('func') + return str(func) + except RuntimeError: + return 'PyCFunction invocation (unable to read "func")' + + # This frame isn't worth reporting: + return False + + def is_waiting_for_gil(self): + '''Is this frame waiting on the GIL?''' + # This assumes the _POSIX_THREADS version of Python/ceval_gil.h: + name = self._gdbframe.name() + if name: + return ('PyThread_acquire_lock' in name + and 'lock_PyThread_acquire_lock' not in name) + + def is_gc_collect(self): + '''Is this frame "collect" within the garbage-collector?''' + return self._gdbframe.name() == 'collect' + def get_pyop(self): try: f = self._gdbframe.read_var('f') @@ -1267,8 +1358,22 @@ @classmethod def get_selected_python_frame(cls): - '''Try to obtain the Frame for the python code in the selected frame, - or None''' + '''Try to obtain the Frame for the python-related code in the selected + frame, or None''' + frame = cls.get_selected_frame() + + while frame: + if frame.is_python_frame(): + return frame + frame = frame.older() + + # Not found: + return None + + @classmethod + def get_selected_bytecode_frame(cls): + '''Try to obtain the Frame for the python bytecode interpreter in the + selected GDB frame, or None''' frame = cls.get_selected_frame() while frame: @@ -1283,14 +1388,38 @@ if self.is_evalframeex(): pyop = self.get_pyop() if pyop: - sys.stdout.write('#%i %s\n' % (self.get_index(), pyop.get_truncated_repr(MAX_OUTPUT_LEN))) + line = pyop.get_truncated_repr(MAX_OUTPUT_LEN) + write_unicode(sys.stdout, '#%i %s\n' % (self.get_index(), line)) if not pyop.is_optimized_out(): line = pyop.current_line() - sys.stdout.write(' %s\n' % line.strip()) + if line is not None: + sys.stdout.write(' %s\n' % line.strip()) else: sys.stdout.write('#%i (unable to read python frame information)\n' % self.get_index()) else: - sys.stdout.write('#%i\n' % self.get_index()) + info = self.is_other_python_frame() + if info: + sys.stdout.write('#%i %s\n' % (self.get_index(), info)) + else: + sys.stdout.write('#%i\n' % self.get_index()) + + def print_traceback(self): + if self.is_evalframeex(): + pyop = self.get_pyop() + if pyop: + pyop.print_traceback() + if not pyop.is_optimized_out(): + line = pyop.current_line() + if line is not None: + sys.stdout.write(' %s\n' % line.strip()) + else: + sys.stdout.write(' (unable to read python frame information)\n') + else: + info = self.is_other_python_frame() + if info: + sys.stdout.write(' %s\n' % info) + else: + sys.stdout.write(' (not a python frame)\n') class PyList(gdb.Command): '''List the current Python source code, if any @@ -1326,9 +1455,10 @@ if m: start, end = map(int, m.groups()) - frame = Frame.get_selected_python_frame() + # py-list requires an actual PyEval_EvalFrameEx frame: + frame = Frame.get_selected_bytecode_frame() if not frame: - print('Unable to locate python frame') + print('Unable to locate gdb frame for python bytecode interpreter') return pyop = frame.get_pyop() @@ -1346,7 +1476,13 @@ if start<1: start = 1 - with open(filename, 'r') as f: + try: + f = open(filename, 'r') + except IOError as err: + sys.stdout.write('Unable to open %s: %s\n' + % (filename, err)) + return + with f: all_lines = f.readlines() # start and end are 1-based, all_lines is 0-based; # so [start-1:end] as a python slice gives us [start, end] as a @@ -1374,7 +1510,7 @@ if not iter_frame: break - if iter_frame.is_evalframeex(): + if iter_frame.is_python_frame(): # Result: if iter_frame.select(): iter_frame.print_summary() @@ -1416,6 +1552,24 @@ PyUp() PyDown() +class PyBacktraceFull(gdb.Command): + 'Display the current python frame and all the frames within its call stack (if any)' + def __init__(self): + gdb.Command.__init__ (self, + "py-bt-full", + gdb.COMMAND_STACK, + gdb.COMPLETE_NONE) + + + def invoke(self, args, from_tty): + frame = Frame.get_selected_python_frame() + while frame: + if frame.is_python_frame(): + frame.print_summary() + frame = frame.older() + +PyBacktraceFull() + class PyBacktrace(gdb.Command): 'Display the current python frame and all the frames within its call stack (if any)' def __init__(self): @@ -1426,10 +1580,11 @@ def invoke(self, args, from_tty): + sys.stdout.write('Traceback (most recent call first):\n') frame = Frame.get_selected_python_frame() while frame: - if frame.is_evalframeex(): - frame.print_summary() + if frame.is_python_frame(): + frame.print_traceback() frame = frame.older() PyBacktrace() -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 3 10:49:21 2015 From: python-checkins at python.org (victor.stinner) Date: Thu, 03 Sep 2015 08:49:21 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzIzMzc1?= =?utf-8?q?=3A_Fix_test=5Fpy3kwarn_for_modules_implemented_in_C?= Message-ID: <20150903084921.66852.36627@psf.io> https://hg.python.org/cpython/rev/d739bc20d7b2 changeset: 97611:d739bc20d7b2 branch: 2.7 user: Victor Stinner date: Thu Sep 03 10:46:17 2015 +0200 summary: Issue #23375: Fix test_py3kwarn for modules implemented in C Don't check if importing a module emits a DeprecationWarning if the module is implemented in C and the module is already loaded. files: Lib/test/test_py3kwarn.py | 16 ++++++++++++++++ 1 files changed, 16 insertions(+), 0 deletions(-) diff --git a/Lib/test/test_py3kwarn.py b/Lib/test/test_py3kwarn.py --- a/Lib/test/test_py3kwarn.py +++ b/Lib/test/test_py3kwarn.py @@ -2,6 +2,7 @@ import sys from test.test_support import check_py3k_warnings, CleanImport, run_unittest import warnings +from test import test_support if not sys.py3kwarning: raise unittest.SkipTest('%s must be run with the -3 flag' % __name__) @@ -356,6 +357,21 @@ def check_removal(self, module_name, optional=False): """Make sure the specified module, when imported, raises a DeprecationWarning and specifies itself in the message.""" + if module_name in sys.modules: + mod = sys.modules[module_name] + filename = getattr(mod, '__file__', '') + mod = None + # the module is not implemented in C? + if not filename.endswith(('.py', '.pyc', '.pyo')): + # Issue #23375: If the module was already loaded, reimporting + # the module will not emit again the warning. The warning is + # emited when the module is loaded, but C modules cannot + # unloaded. + if test_support.verbose: + print("Cannot test the Python 3 DeprecationWarning of the " + "%s module, the C module is already loaded" + % module_name) + return with CleanImport(module_name), warnings.catch_warnings(): warnings.filterwarnings("error", ".+ (module|package) .+ removed", DeprecationWarning, __name__) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 3 11:29:31 2015 From: python-checkins at python.org (senthil.kumaran) Date: Thu, 03 Sep 2015 09:29:31 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogRml4IHRlc3Rfd3Nn?= =?utf-8?q?iref_execution_from_the_test_module=2E?= Message-ID: <20150903092931.15704.66969@psf.io> https://hg.python.org/cpython/rev/fdbec2800888 changeset: 97612:fdbec2800888 branch: 3.4 parent: 97606:63c697473515 user: Senthil Kumaran date: Thu Sep 03 02:26:31 2015 -0700 summary: Fix test_wsgiref execution from the test module. files: Lib/test/test_wsgiref.py | 5 ++--- 1 files changed, 2 insertions(+), 3 deletions(-) diff --git a/Lib/test/test_wsgiref.py b/Lib/test/test_wsgiref.py --- a/Lib/test/test_wsgiref.py +++ b/Lib/test/test_wsgiref.py @@ -1,11 +1,10 @@ -from __future__ import nested_scopes # Backward compat for 2.1 from unittest import TestCase from wsgiref.util import setup_testing_defaults from wsgiref.headers import Headers from wsgiref.handlers import BaseHandler, BaseCGIHandler from wsgiref import util from wsgiref.validate import validator -from wsgiref.simple_server import WSGIServer, WSGIRequestHandler, demo_app +from wsgiref.simple_server import WSGIServer, WSGIRequestHandler from wsgiref.simple_server import make_server from io import StringIO, BytesIO, BufferedReader from socketserver import BaseServer @@ -14,8 +13,8 @@ import os import re import sys +import unittest -from test import support class MockServer(WSGIServer): """Non-socket HTTP server""" -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 3 11:29:32 2015 From: python-checkins at python.org (senthil.kumaran) Date: Thu, 03 Sep 2015 09:29:32 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Merge_with_3=2E5=2E_Fix_test=5Fwsgiref_execution_from_the_test?= =?utf-8?q?_module=2E?= Message-ID: <20150903092931.101478.82381@psf.io> https://hg.python.org/cpython/rev/17acaef50c42 changeset: 97613:17acaef50c42 branch: 3.5 parent: 97607:ecd43087e826 parent: 97612:fdbec2800888 user: Senthil Kumaran date: Thu Sep 03 02:27:18 2015 -0700 summary: Merge with 3.5. Fix test_wsgiref execution from the test module. files: Lib/test/test_wsgiref.py | 5 ++--- 1 files changed, 2 insertions(+), 3 deletions(-) diff --git a/Lib/test/test_wsgiref.py b/Lib/test/test_wsgiref.py --- a/Lib/test/test_wsgiref.py +++ b/Lib/test/test_wsgiref.py @@ -1,11 +1,10 @@ -from __future__ import nested_scopes # Backward compat for 2.1 from unittest import TestCase from wsgiref.util import setup_testing_defaults from wsgiref.headers import Headers from wsgiref.handlers import BaseHandler, BaseCGIHandler from wsgiref import util from wsgiref.validate import validator -from wsgiref.simple_server import WSGIServer, WSGIRequestHandler, demo_app +from wsgiref.simple_server import WSGIServer, WSGIRequestHandler from wsgiref.simple_server import make_server from io import StringIO, BytesIO, BufferedReader from socketserver import BaseServer @@ -14,8 +13,8 @@ import os import re import sys +import unittest -from test import support class MockServer(WSGIServer): """Non-socket HTTP server""" -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 3 11:29:32 2015 From: python-checkins at python.org (senthil.kumaran) Date: Thu, 03 Sep 2015 09:29:32 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge_with_3=2E6=2E_Fix_test=5Fwsgiref_execution_from_th?= =?utf-8?q?e_test_module=2E?= Message-ID: <20150903092932.68881.31577@psf.io> https://hg.python.org/cpython/rev/4a20062ff873 changeset: 97614:4a20062ff873 parent: 97608:8a3303a8a87c parent: 97613:17acaef50c42 user: Senthil Kumaran date: Thu Sep 03 02:28:03 2015 -0700 summary: Merge with 3.6. Fix test_wsgiref execution from the test module. files: Lib/test/test_wsgiref.py | 5 ++--- 1 files changed, 2 insertions(+), 3 deletions(-) diff --git a/Lib/test/test_wsgiref.py b/Lib/test/test_wsgiref.py --- a/Lib/test/test_wsgiref.py +++ b/Lib/test/test_wsgiref.py @@ -1,11 +1,10 @@ -from __future__ import nested_scopes # Backward compat for 2.1 from unittest import TestCase from wsgiref.util import setup_testing_defaults from wsgiref.headers import Headers from wsgiref.handlers import BaseHandler, BaseCGIHandler from wsgiref import util from wsgiref.validate import validator -from wsgiref.simple_server import WSGIServer, WSGIRequestHandler, demo_app +from wsgiref.simple_server import WSGIServer, WSGIRequestHandler from wsgiref.simple_server import make_server from io import StringIO, BytesIO, BufferedReader from socketserver import BaseServer @@ -14,8 +13,8 @@ import os import re import sys +import unittest -from test import support class MockServer(WSGIServer): """Non-socket HTTP server""" -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 3 11:51:28 2015 From: python-checkins at python.org (senthil.kumaran) Date: Thu, 03 Sep 2015 09:51:28 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_Remove_unused_?= =?utf-8?q?imports_in_test=5Fwsgiref=2Epy?= Message-ID: <20150903095128.14859.46231@psf.io> https://hg.python.org/cpython/rev/cb781d3b1e6b changeset: 97615:cb781d3b1e6b branch: 2.7 parent: 97601:69ea73015132 user: Senthil Kumaran date: Thu Sep 03 02:39:57 2015 -0700 summary: Remove unused imports in test_wsgiref.py files: Lib/test/test_wsgiref.py | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Lib/test/test_wsgiref.py b/Lib/test/test_wsgiref.py --- a/Lib/test/test_wsgiref.py +++ b/Lib/test/test_wsgiref.py @@ -1,14 +1,14 @@ -from __future__ import nested_scopes # Backward compat for 2.1 from unittest import TestCase from wsgiref.util import setup_testing_defaults from wsgiref.headers import Headers from wsgiref.handlers import BaseHandler, BaseCGIHandler from wsgiref import util from wsgiref.validate import validator -from wsgiref.simple_server import WSGIServer, WSGIRequestHandler, demo_app +from wsgiref.simple_server import WSGIServer, WSGIRequestHandler from wsgiref.simple_server import make_server from StringIO import StringIO from SocketServer import BaseServer + import os import re import sys -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 3 11:51:28 2015 From: python-checkins at python.org (senthil.kumaran) Date: Thu, 03 Sep 2015 09:51:28 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMi43IC0+IDIuNyk6?= =?utf-8?q?_merge_heads=2E?= Message-ID: <20150903095128.14865.29569@psf.io> https://hg.python.org/cpython/rev/d687912d499f changeset: 97616:d687912d499f branch: 2.7 parent: 97615:cb781d3b1e6b parent: 97611:d739bc20d7b2 user: Senthil Kumaran date: Thu Sep 03 02:50:51 2015 -0700 summary: merge heads. files: Lib/test/test_gdb.py | 168 +++++++++++++++++++++-- Lib/test/test_py3kwarn.py | 16 ++ Tools/gdb/libpython.py | 183 ++++++++++++++++++++++++- 3 files changed, 338 insertions(+), 29 deletions(-) diff --git a/Lib/test/test_gdb.py b/Lib/test/test_gdb.py --- a/Lib/test/test_gdb.py +++ b/Lib/test/test_gdb.py @@ -10,21 +10,42 @@ import unittest import sysconfig +from test import test_support from test.test_support import run_unittest, findfile +# Is this Python configured to support threads? try: - gdb_version, _ = subprocess.Popen(["gdb", "-nx", "--version"], - stdout=subprocess.PIPE).communicate() -except OSError: - # This is what "no gdb" looks like. There may, however, be other - # errors that manifest this way too. - raise unittest.SkipTest("Couldn't find gdb on the path") -gdb_version_number = re.search("^GNU gdb [^\d]*(\d+)\.(\d)", gdb_version) -gdb_major_version = int(gdb_version_number.group(1)) -gdb_minor_version = int(gdb_version_number.group(2)) + import thread +except ImportError: + thread = None + +def get_gdb_version(): + try: + proc = subprocess.Popen(["gdb", "-nx", "--version"], + stdout=subprocess.PIPE, + universal_newlines=True) + version = proc.communicate()[0] + except OSError: + # This is what "no gdb" looks like. There may, however, be other + # errors that manifest this way too. + raise unittest.SkipTest("Couldn't find gdb on the path") + + # Regex to parse: + # 'GNU gdb (GDB; SUSE Linux Enterprise 12) 7.7\n' -> 7.7 + # 'GNU gdb (GDB) Fedora 7.9.1-17.fc22\n' -> 7.9 + # 'GNU gdb 6.1.1 [FreeBSD]\n' + match = re.search("^GNU gdb.*? (\d+)\.(\d)", version) + if match is None: + raise Exception("unable to parse GDB version: %r" % version) + return (version, int(match.group(1)), int(match.group(2))) + +gdb_version, gdb_major_version, gdb_minor_version = get_gdb_version() if gdb_major_version < 7: - raise unittest.SkipTest("gdb versions before 7.0 didn't support python embedding" - " Saw:\n" + gdb_version) + raise unittest.SkipTest("gdb versions before 7.0 didn't support python " + "embedding. Saw %s.%s:\n%s" + % (gdb_major_version, gdb_minor_version, + gdb_version)) + if sys.platform.startswith("sunos"): raise unittest.SkipTest("test doesn't work very well on Solaris") @@ -713,20 +734,133 @@ class PyBtTests(DebuggerTests): @unittest.skipIf(python_is_optimized(), "Python was compiled with optimizations") - def test_basic_command(self): + def test_bt(self): 'Verify that the "py-bt" command works' bt = self.get_stack_trace(script=self.get_sample_script(), cmds_after_breakpoint=['py-bt']) self.assertMultilineMatches(bt, r'''^.* -#[0-9]+ Frame 0x[0-9a-f]+, for file .*gdb_sample.py, line 7, in bar \(a=1, b=2, c=3\) +Traceback \(most recent call first\): + File ".*gdb_sample.py", line 10, in baz + print\(42\) + File ".*gdb_sample.py", line 7, in bar baz\(a, b, c\) -#[0-9]+ Frame 0x[0-9a-f]+, for file .*gdb_sample.py, line 4, in foo \(a=1, b=2, c=3\) + File ".*gdb_sample.py", line 4, in foo bar\(a, b, c\) -#[0-9]+ Frame 0x[0-9a-f]+, for file .*gdb_sample.py, line 12, in \(\) + File ".*gdb_sample.py", line 12, in foo\(1, 2, 3\) ''') + @unittest.skipIf(python_is_optimized(), + "Python was compiled with optimizations") + def test_bt_full(self): + 'Verify that the "py-bt-full" command works' + bt = self.get_stack_trace(script=self.get_sample_script(), + cmds_after_breakpoint=['py-bt-full']) + self.assertMultilineMatches(bt, + r'''^.* +#[0-9]+ Frame 0x-?[0-9a-f]+, for file .*gdb_sample.py, line 7, in bar \(a=1, b=2, c=3\) + baz\(a, b, c\) +#[0-9]+ Frame 0x-?[0-9a-f]+, for file .*gdb_sample.py, line 4, in foo \(a=1, b=2, c=3\) + bar\(a, b, c\) +#[0-9]+ Frame 0x-?[0-9a-f]+, for file .*gdb_sample.py, line 12, in \(\) + foo\(1, 2, 3\) +''') + + @unittest.skipUnless(thread, + "Python was compiled without thread support") + def test_threads(self): + 'Verify that "py-bt" indicates threads that are waiting for the GIL' + cmd = ''' +from threading import Thread + +class TestThread(Thread): + # These threads would run forever, but we'll interrupt things with the + # debugger + def run(self): + i = 0 + while 1: + i += 1 + +t = {} +for i in range(4): + t[i] = TestThread() + t[i].start() + +# Trigger a breakpoint on the main thread +print 42 + +''' + # Verify with "py-bt": + gdb_output = self.get_stack_trace(cmd, + cmds_after_breakpoint=['thread apply all py-bt']) + self.assertIn('Waiting for the GIL', gdb_output) + + # Verify with "py-bt-full": + gdb_output = self.get_stack_trace(cmd, + cmds_after_breakpoint=['thread apply all py-bt-full']) + self.assertIn('Waiting for the GIL', gdb_output) + + @unittest.skipIf(python_is_optimized(), + "Python was compiled with optimizations") + # Some older versions of gdb will fail with + # "Cannot find new threads: generic error" + # unless we add LD_PRELOAD=PATH-TO-libpthread.so.1 as a workaround + @unittest.skipUnless(thread, + "Python was compiled without thread support") + def test_gc(self): + 'Verify that "py-bt" indicates if a thread is garbage-collecting' + cmd = ('from gc import collect\n' + 'print 42\n' + 'def foo():\n' + ' collect()\n' + 'def bar():\n' + ' foo()\n' + 'bar()\n') + # Verify with "py-bt": + gdb_output = self.get_stack_trace(cmd, + cmds_after_breakpoint=['break update_refs', 'continue', 'py-bt'], + ) + self.assertIn('Garbage-collecting', gdb_output) + + # Verify with "py-bt-full": + gdb_output = self.get_stack_trace(cmd, + cmds_after_breakpoint=['break update_refs', 'continue', 'py-bt-full'], + ) + self.assertIn('Garbage-collecting', gdb_output) + + @unittest.skipIf(python_is_optimized(), + "Python was compiled with optimizations") + # Some older versions of gdb will fail with + # "Cannot find new threads: generic error" + # unless we add LD_PRELOAD=PATH-TO-libpthread.so.1 as a workaround + @unittest.skipUnless(thread, + "Python was compiled without thread support") + def test_pycfunction(self): + 'Verify that "py-bt" displays invocations of PyCFunction instances' + # Tested function must not be defined with METH_NOARGS or METH_O, + # otherwise call_function() doesn't call PyCFunction_Call() + cmd = ('from time import gmtime\n' + 'def foo():\n' + ' gmtime(1)\n' + 'def bar():\n' + ' foo()\n' + 'bar()\n') + # Verify with "py-bt": + gdb_output = self.get_stack_trace(cmd, + breakpoint='time_gmtime', + cmds_after_breakpoint=['bt', 'py-bt'], + ) + self.assertIn('= 3: + def write_unicode(file, text): + file.write(text) +else: + def write_unicode(file, text): + # Write a byte or unicode string to file. Unicode strings are encoded to + # ENCODING encoding with 'backslashreplace' error handler to avoid + # UnicodeEncodeError. + if isinstance(text, unicode): + text = text.encode(ENCODING, 'backslashreplace') + file.write(text) + class StringTruncated(RuntimeError): pass @@ -903,7 +918,12 @@ newline character''' if self.is_optimized_out(): return '(frame information optimized out)' - with open(self.filename(), 'r') as f: + filename = self.filename() + try: + f = open(filename, 'r') + except IOError: + return None + with f: all_lines = f.readlines() # Convert from 1-based current_line_num to 0-based list offset: return all_lines[self.current_line_num()-1] @@ -914,9 +934,9 @@ return out.write('Frame 0x%x, for file %s, line %i, in %s (' % (self.as_address(), - self.co_filename, + self.co_filename.proxyval(visited), self.current_line_num(), - self.co_name)) + self.co_name.proxyval(visited))) first = True for pyop_name, pyop_value in self.iter_locals(): if not first: @@ -929,6 +949,16 @@ out.write(')') + def print_traceback(self): + if self.is_optimized_out(): + sys.stdout.write(' (frame information optimized out)\n') + return + visited = set() + sys.stdout.write(' File "%s", line %i, in %s\n' + % (self.co_filename.proxyval(visited), + self.current_line_num(), + self.co_name.proxyval(visited))) + class PySetObjectPtr(PyObjectPtr): _typename = 'PySetObject' @@ -1222,6 +1252,23 @@ iter_frame = iter_frame.newer() return index + # We divide frames into: + # - "python frames": + # - "bytecode frames" i.e. PyEval_EvalFrameEx + # - "other python frames": things that are of interest from a python + # POV, but aren't bytecode (e.g. GC, GIL) + # - everything else + + def is_python_frame(self): + '''Is this a PyEval_EvalFrameEx frame, or some other important + frame? (see is_other_python_frame for what "important" means in this + context)''' + if self.is_evalframeex(): + return True + if self.is_other_python_frame(): + return True + return False + def is_evalframeex(self): '''Is this a PyEval_EvalFrameEx frame?''' if self._gdbframe.name() == 'PyEval_EvalFrameEx': @@ -1238,6 +1285,50 @@ return False + def is_other_python_frame(self): + '''Is this frame worth displaying in python backtraces? + Examples: + - waiting on the GIL + - garbage-collecting + - within a CFunction + If it is, return a descriptive string + For other frames, return False + ''' + if self.is_waiting_for_gil(): + return 'Waiting for the GIL' + elif self.is_gc_collect(): + return 'Garbage-collecting' + else: + # Detect invocations of PyCFunction instances: + older = self.older() + if older and older._gdbframe.name() == 'PyCFunction_Call': + # Within that frame: + # "func" is the local containing the PyObject* of the + # PyCFunctionObject instance + # "f" is the same value, but cast to (PyCFunctionObject*) + # "self" is the (PyObject*) of the 'self' + try: + # Use the prettyprinter for the func: + func = older._gdbframe.read_var('func') + return str(func) + except RuntimeError: + return 'PyCFunction invocation (unable to read "func")' + + # This frame isn't worth reporting: + return False + + def is_waiting_for_gil(self): + '''Is this frame waiting on the GIL?''' + # This assumes the _POSIX_THREADS version of Python/ceval_gil.h: + name = self._gdbframe.name() + if name: + return ('PyThread_acquire_lock' in name + and 'lock_PyThread_acquire_lock' not in name) + + def is_gc_collect(self): + '''Is this frame "collect" within the garbage-collector?''' + return self._gdbframe.name() == 'collect' + def get_pyop(self): try: f = self._gdbframe.read_var('f') @@ -1267,8 +1358,22 @@ @classmethod def get_selected_python_frame(cls): - '''Try to obtain the Frame for the python code in the selected frame, - or None''' + '''Try to obtain the Frame for the python-related code in the selected + frame, or None''' + frame = cls.get_selected_frame() + + while frame: + if frame.is_python_frame(): + return frame + frame = frame.older() + + # Not found: + return None + + @classmethod + def get_selected_bytecode_frame(cls): + '''Try to obtain the Frame for the python bytecode interpreter in the + selected GDB frame, or None''' frame = cls.get_selected_frame() while frame: @@ -1283,14 +1388,38 @@ if self.is_evalframeex(): pyop = self.get_pyop() if pyop: - sys.stdout.write('#%i %s\n' % (self.get_index(), pyop.get_truncated_repr(MAX_OUTPUT_LEN))) + line = pyop.get_truncated_repr(MAX_OUTPUT_LEN) + write_unicode(sys.stdout, '#%i %s\n' % (self.get_index(), line)) if not pyop.is_optimized_out(): line = pyop.current_line() - sys.stdout.write(' %s\n' % line.strip()) + if line is not None: + sys.stdout.write(' %s\n' % line.strip()) else: sys.stdout.write('#%i (unable to read python frame information)\n' % self.get_index()) else: - sys.stdout.write('#%i\n' % self.get_index()) + info = self.is_other_python_frame() + if info: + sys.stdout.write('#%i %s\n' % (self.get_index(), info)) + else: + sys.stdout.write('#%i\n' % self.get_index()) + + def print_traceback(self): + if self.is_evalframeex(): + pyop = self.get_pyop() + if pyop: + pyop.print_traceback() + if not pyop.is_optimized_out(): + line = pyop.current_line() + if line is not None: + sys.stdout.write(' %s\n' % line.strip()) + else: + sys.stdout.write(' (unable to read python frame information)\n') + else: + info = self.is_other_python_frame() + if info: + sys.stdout.write(' %s\n' % info) + else: + sys.stdout.write(' (not a python frame)\n') class PyList(gdb.Command): '''List the current Python source code, if any @@ -1326,9 +1455,10 @@ if m: start, end = map(int, m.groups()) - frame = Frame.get_selected_python_frame() + # py-list requires an actual PyEval_EvalFrameEx frame: + frame = Frame.get_selected_bytecode_frame() if not frame: - print('Unable to locate python frame') + print('Unable to locate gdb frame for python bytecode interpreter') return pyop = frame.get_pyop() @@ -1346,7 +1476,13 @@ if start<1: start = 1 - with open(filename, 'r') as f: + try: + f = open(filename, 'r') + except IOError as err: + sys.stdout.write('Unable to open %s: %s\n' + % (filename, err)) + return + with f: all_lines = f.readlines() # start and end are 1-based, all_lines is 0-based; # so [start-1:end] as a python slice gives us [start, end] as a @@ -1374,7 +1510,7 @@ if not iter_frame: break - if iter_frame.is_evalframeex(): + if iter_frame.is_python_frame(): # Result: if iter_frame.select(): iter_frame.print_summary() @@ -1416,6 +1552,24 @@ PyUp() PyDown() +class PyBacktraceFull(gdb.Command): + 'Display the current python frame and all the frames within its call stack (if any)' + def __init__(self): + gdb.Command.__init__ (self, + "py-bt-full", + gdb.COMMAND_STACK, + gdb.COMPLETE_NONE) + + + def invoke(self, args, from_tty): + frame = Frame.get_selected_python_frame() + while frame: + if frame.is_python_frame(): + frame.print_summary() + frame = frame.older() + +PyBacktraceFull() + class PyBacktrace(gdb.Command): 'Display the current python frame and all the frames within its call stack (if any)' def __init__(self): @@ -1426,10 +1580,11 @@ def invoke(self, args, from_tty): + sys.stdout.write('Traceback (most recent call first):\n') frame = Frame.get_selected_python_frame() while frame: - if frame.is_evalframeex(): - frame.print_summary() + if frame.is_python_frame(): + frame.print_traceback() frame = frame.older() PyBacktrace() -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 3 12:14:51 2015 From: python-checkins at python.org (victor.stinner) Date: Thu, 03 Sep 2015 10:14:51 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogdGVzdF93c2dpcmVm?= =?utf-8?q?=3A_add_missing_import_=28support=29?= Message-ID: <20150903101450.101498.42571@psf.io> https://hg.python.org/cpython/rev/11b6ac84d151 changeset: 97617:11b6ac84d151 branch: 3.4 parent: 97612:fdbec2800888 user: Victor Stinner date: Thu Sep 03 12:14:25 2015 +0200 summary: test_wsgiref: add missing import (support) files: Lib/test/test_wsgiref.py | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) diff --git a/Lib/test/test_wsgiref.py b/Lib/test/test_wsgiref.py --- a/Lib/test/test_wsgiref.py +++ b/Lib/test/test_wsgiref.py @@ -15,6 +15,8 @@ import sys import unittest +from test import support + class MockServer(WSGIServer): """Non-socket HTTP server""" -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 3 12:16:24 2015 From: python-checkins at python.org (victor.stinner) Date: Thu, 03 Sep 2015 10:16:24 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge_3=2E5_=28null_merge=29?= Message-ID: <20150903101622.68887.88216@psf.io> https://hg.python.org/cpython/rev/130049ba948d changeset: 97619:130049ba948d parent: 97614:4a20062ff873 parent: 97618:0da962ea5d00 user: Victor Stinner date: Thu Sep 03 12:15:39 2015 +0200 summary: Merge 3.5 (null merge) files: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 3 12:16:27 2015 From: python-checkins at python.org (victor.stinner) Date: Thu, 03 Sep 2015 10:16:27 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Merge_3=2E4_=28test=5Fwsgiref=29?= Message-ID: <20150903101621.68871.37414@psf.io> https://hg.python.org/cpython/rev/0da962ea5d00 changeset: 97618:0da962ea5d00 branch: 3.5 parent: 97613:17acaef50c42 parent: 97617:11b6ac84d151 user: Victor Stinner date: Thu Sep 03 12:15:27 2015 +0200 summary: Merge 3.4 (test_wsgiref) The support import is not needed in Python 3.5 files: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 3 12:18:00 2015 From: python-checkins at python.org (victor.stinner) Date: Thu, 03 Sep 2015 10:18:00 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_type=5Fcall=28=29_now_dete?= =?utf-8?q?ct_bugs_in_type_new_and_init?= Message-ID: <20150903101800.66858.48223@psf.io> https://hg.python.org/cpython/rev/40c659ec4d18 changeset: 97620:40c659ec4d18 user: Victor Stinner date: Thu Sep 03 12:16:49 2015 +0200 summary: type_call() now detect bugs in type new and init * Call _Py_CheckFunctionResult() to check for bugs in type constructors (tp_new) * Add assertions to ensure an exception was raised if tp_init failed or that no exception was raised if tp_init succeed Refactor also the function to have less indentation. files: Objects/typeobject.c | 46 ++++++++++++++++++------------- 1 files changed, 27 insertions(+), 19 deletions(-) diff --git a/Objects/typeobject.c b/Objects/typeobject.c --- a/Objects/typeobject.c +++ b/Objects/typeobject.c @@ -906,25 +906,33 @@ #endif obj = type->tp_new(type, args, kwds); - if (obj != NULL) { - /* Ugly exception: when the call was type(something), - don't call tp_init on the result. */ - if (type == &PyType_Type && - PyTuple_Check(args) && PyTuple_GET_SIZE(args) == 1 && - (kwds == NULL || - (PyDict_Check(kwds) && PyDict_Size(kwds) == 0))) - return obj; - /* If the returned object is not an instance of type, - it won't be initialized. */ - if (!PyType_IsSubtype(Py_TYPE(obj), type)) - return obj; - type = Py_TYPE(obj); - if (type->tp_init != NULL) { - int res = type->tp_init(obj, args, kwds); - if (res < 0) { - Py_DECREF(obj); - obj = NULL; - } + obj = _Py_CheckFunctionResult((PyObject*)type, obj, NULL); + if (obj == NULL) + return NULL; + + /* Ugly exception: when the call was type(something), + don't call tp_init on the result. */ + if (type == &PyType_Type && + PyTuple_Check(args) && PyTuple_GET_SIZE(args) == 1 && + (kwds == NULL || + (PyDict_Check(kwds) && PyDict_Size(kwds) == 0))) + return obj; + + /* If the returned object is not an instance of type, + it won't be initialized. */ + if (!PyType_IsSubtype(Py_TYPE(obj), type)) + return obj; + + type = Py_TYPE(obj); + if (type->tp_init != NULL) { + int res = type->tp_init(obj, args, kwds); + if (res < 0) { + assert(PyErr_Occurred()); + Py_DECREF(obj); + obj = NULL; + } + else { + assert(!PyErr_Occurred()); } } return obj; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 3 13:02:40 2015 From: python-checkins at python.org (victor.stinner) Date: Thu, 03 Sep 2015 11:02:40 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbjogRml4IGFzdF9mb3JfYXRvbSgp?= Message-ID: <20150903110240.11256.50995@psf.io> https://hg.python.org/cpython/rev/940b1b4a3ff8 changeset: 97621:940b1b4a3ff8 user: Victor Stinner date: Thu Sep 03 12:57:11 2015 +0200 summary: Fix ast_for_atom() Clear PyObject_Str() exception if it failed, ast_error() should not be called with an exception set. files: Python/ast.c | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diff --git a/Python/ast.c b/Python/ast.c --- a/Python/ast.c +++ b/Python/ast.c @@ -2040,6 +2040,7 @@ PyOS_snprintf(buf, sizeof(buf), "(%s) %s", errtype, s); Py_DECREF(errstr); } else { + PyErr_Clear(); PyOS_snprintf(buf, sizeof(buf), "(%s) unknown error", errtype); } ast_error(c, n, buf); -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 3 16:20:48 2015 From: python-checkins at python.org (victor.stinner) Date: Thu, 03 Sep 2015 14:20:48 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogdGVzdF9nZGI6IG9v?= =?utf-8?q?ps=2C_the_regex_to_parse_the_gdb_version_was_still_too_strict?= Message-ID: <20150903142047.14859.34223@psf.io> https://hg.python.org/cpython/rev/29fa1d33e421 changeset: 97622:29fa1d33e421 branch: 3.4 parent: 97617:11b6ac84d151 user: Victor Stinner date: Thu Sep 03 15:42:26 2015 +0200 summary: test_gdb: oops, the regex to parse the gdb version was still too strict files: Lib/test/test_gdb.py | 5 +++-- 1 files changed, 3 insertions(+), 2 deletions(-) diff --git a/Lib/test/test_gdb.py b/Lib/test/test_gdb.py --- a/Lib/test/test_gdb.py +++ b/Lib/test/test_gdb.py @@ -36,8 +36,9 @@ # Regex to parse: # 'GNU gdb (GDB; SUSE Linux Enterprise 12) 7.7\n' -> 7.7 # 'GNU gdb (GDB) Fedora 7.9.1-17.fc22\n' -> 7.9 - # 'GNU gdb 6.1.1 [FreeBSD]\n' - match = re.search("^GNU gdb.*? (\d+)\.(\d)", version) + # 'GNU gdb 6.1.1 [FreeBSD]\n' -> 6.1 + # 'GNU gdb (GDB) Fedora (7.5.1-37.fc18)\n' -> 7.5 + match = re.search(r"^GNU gdb.*?\b(\d+)\.(\d)", version) if match is None: raise Exception("unable to parse GDB version: %r" % version) return (version, int(match.group(1)), int(match.group(2))) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 3 16:20:48 2015 From: python-checkins at python.org (victor.stinner) Date: Thu, 03 Sep 2015 14:20:48 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI0OTkz?= =?utf-8?q?=3A_Handle_import_error_in_namereplace_error_handler?= Message-ID: <20150903142048.12009.20213@psf.io> https://hg.python.org/cpython/rev/ac1995b01028 changeset: 97625:ac1995b01028 branch: 3.5 parent: 97623:abf4acc73b5a user: Victor Stinner date: Thu Sep 03 16:19:40 2015 +0200 summary: Issue #24993: Handle import error in namereplace error handler Handle PyCapsule_Import() failure (exception) in PyCodec_NameReplaceErrors(): return immedialty NULL. files: Python/codecs.c | 12 +++++------- 1 files changed, 5 insertions(+), 7 deletions(-) diff --git a/Python/codecs.c b/Python/codecs.c --- a/Python/codecs.c +++ b/Python/codecs.c @@ -966,7 +966,6 @@ } static _PyUnicode_Name_CAPI *ucnhash_CAPI = NULL; -static int ucnhash_initialized = 0; PyObject *PyCodec_NameReplaceErrors(PyObject *exc) { @@ -988,17 +987,17 @@ return NULL; if (!(object = PyUnicodeEncodeError_GetObject(exc))) return NULL; - if (!ucnhash_initialized) { + if (!ucnhash_CAPI) { /* load the unicode data module */ ucnhash_CAPI = (_PyUnicode_Name_CAPI *)PyCapsule_Import( PyUnicodeData_CAPSULE_NAME, 1); - ucnhash_initialized = 1; + if (!ucnhash_CAPI) + return NULL; } for (i = start, ressize = 0; i < end; ++i) { /* object is guaranteed to be "ready" */ c = PyUnicode_READ_CHAR(object, i); - if (ucnhash_CAPI && - ucnhash_CAPI->getname(NULL, c, buffer, sizeof(buffer), 1)) { + if (ucnhash_CAPI->getname(NULL, c, buffer, sizeof(buffer), 1)) { replsize = 1+1+1+(int)strlen(buffer)+1; } else if (c >= 0x10000) { @@ -1021,8 +1020,7 @@ i < end; ++i) { c = PyUnicode_READ_CHAR(object, i); *outp++ = '\\'; - if (ucnhash_CAPI && - ucnhash_CAPI->getname(NULL, c, buffer, sizeof(buffer), 1)) { + if (ucnhash_CAPI->getname(NULL, c, buffer, sizeof(buffer), 1)) { *outp++ = 'N'; *outp++ = '{'; strcpy((char *)outp, buffer); -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 3 16:20:49 2015 From: python-checkins at python.org (victor.stinner) Date: Thu, 03 Sep 2015 14:20:49 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?b?IE1lcmdlIDMuNCAodGVzdF9nZGIp?= Message-ID: <20150903142048.14861.78377@psf.io> https://hg.python.org/cpython/rev/abf4acc73b5a changeset: 97623:abf4acc73b5a branch: 3.5 parent: 97618:0da962ea5d00 parent: 97622:29fa1d33e421 user: Victor Stinner date: Thu Sep 03 15:42:45 2015 +0200 summary: Merge 3.4 (test_gdb) files: Lib/test/test_gdb.py | 5 +++-- 1 files changed, 3 insertions(+), 2 deletions(-) diff --git a/Lib/test/test_gdb.py b/Lib/test/test_gdb.py --- a/Lib/test/test_gdb.py +++ b/Lib/test/test_gdb.py @@ -36,8 +36,9 @@ # Regex to parse: # 'GNU gdb (GDB; SUSE Linux Enterprise 12) 7.7\n' -> 7.7 # 'GNU gdb (GDB) Fedora 7.9.1-17.fc22\n' -> 7.9 - # 'GNU gdb 6.1.1 [FreeBSD]\n' - match = re.search("^GNU gdb.*? (\d+)\.(\d)", version) + # 'GNU gdb 6.1.1 [FreeBSD]\n' -> 6.1 + # 'GNU gdb (GDB) Fedora (7.5.1-37.fc18)\n' -> 7.5 + match = re.search(r"^GNU gdb.*?\b(\d+)\.(\d)", version) if match is None: raise Exception("unable to parse GDB version: %r" % version) return (version, int(match.group(1)), int(match.group(2))) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 3 16:20:54 2015 From: python-checkins at python.org (victor.stinner) Date: Thu, 03 Sep 2015 14:20:54 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogTWVyZ2UgMy41ICh0ZXN0X2dkYik=?= Message-ID: <20150903142048.27695.82916@psf.io> https://hg.python.org/cpython/rev/1cd0280b0a08 changeset: 97624:1cd0280b0a08 parent: 97621:940b1b4a3ff8 parent: 97623:abf4acc73b5a user: Victor Stinner date: Thu Sep 03 15:43:06 2015 +0200 summary: Merge 3.5 (test_gdb) files: Lib/test/test_gdb.py | 5 +++-- 1 files changed, 3 insertions(+), 2 deletions(-) diff --git a/Lib/test/test_gdb.py b/Lib/test/test_gdb.py --- a/Lib/test/test_gdb.py +++ b/Lib/test/test_gdb.py @@ -36,8 +36,9 @@ # Regex to parse: # 'GNU gdb (GDB; SUSE Linux Enterprise 12) 7.7\n' -> 7.7 # 'GNU gdb (GDB) Fedora 7.9.1-17.fc22\n' -> 7.9 - # 'GNU gdb 6.1.1 [FreeBSD]\n' - match = re.search("^GNU gdb.*? (\d+)\.(\d)", version) + # 'GNU gdb 6.1.1 [FreeBSD]\n' -> 6.1 + # 'GNU gdb (GDB) Fedora (7.5.1-37.fc18)\n' -> 7.5 + match = re.search(r"^GNU gdb.*?\b(\d+)\.(\d)", version) if match is None: raise Exception("unable to parse GDB version: %r" % version) return (version, int(match.group(1)), int(match.group(2))) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 3 16:20:54 2015 From: python-checkins at python.org (victor.stinner) Date: Thu, 03 Sep 2015 14:20:54 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge_3=2E5_=28namereplace=29?= Message-ID: <20150903142053.66854.64626@psf.io> https://hg.python.org/cpython/rev/716e2bc7d41c changeset: 97626:716e2bc7d41c parent: 97624:1cd0280b0a08 parent: 97625:ac1995b01028 user: Victor Stinner date: Thu Sep 03 16:20:01 2015 +0200 summary: Merge 3.5 (namereplace) files: Python/codecs.c | 12 +++++------- 1 files changed, 5 insertions(+), 7 deletions(-) diff --git a/Python/codecs.c b/Python/codecs.c --- a/Python/codecs.c +++ b/Python/codecs.c @@ -966,7 +966,6 @@ } static _PyUnicode_Name_CAPI *ucnhash_CAPI = NULL; -static int ucnhash_initialized = 0; PyObject *PyCodec_NameReplaceErrors(PyObject *exc) { @@ -988,17 +987,17 @@ return NULL; if (!(object = PyUnicodeEncodeError_GetObject(exc))) return NULL; - if (!ucnhash_initialized) { + if (!ucnhash_CAPI) { /* load the unicode data module */ ucnhash_CAPI = (_PyUnicode_Name_CAPI *)PyCapsule_Import( PyUnicodeData_CAPSULE_NAME, 1); - ucnhash_initialized = 1; + if (!ucnhash_CAPI) + return NULL; } for (i = start, ressize = 0; i < end; ++i) { /* object is guaranteed to be "ready" */ c = PyUnicode_READ_CHAR(object, i); - if (ucnhash_CAPI && - ucnhash_CAPI->getname(NULL, c, buffer, sizeof(buffer), 1)) { + if (ucnhash_CAPI->getname(NULL, c, buffer, sizeof(buffer), 1)) { replsize = 1+1+1+(int)strlen(buffer)+1; } else if (c >= 0x10000) { @@ -1021,8 +1020,7 @@ i < end; ++i) { c = PyUnicode_READ_CHAR(object, i); *outp++ = '\\'; - if (ucnhash_CAPI && - ucnhash_CAPI->getname(NULL, c, buffer, sizeof(buffer), 1)) { + if (ucnhash_CAPI->getname(NULL, c, buffer, sizeof(buffer), 1)) { *outp++ = 'N'; *outp++ = '{'; strcpy((char *)outp, buffer); -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 3 16:34:53 2015 From: python-checkins at python.org (victor.stinner) Date: Thu, 03 Sep 2015 14:34:53 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Don=27t_abuse_volatile_key?= =?utf-8?q?word_in_pytime=2Ec?= Message-ID: <20150903143453.14881.45407@psf.io> https://hg.python.org/cpython/rev/5834a2a972a8 changeset: 97628:5834a2a972a8 user: Victor Stinner date: Thu Sep 03 16:33:16 2015 +0200 summary: Don't abuse volatile keyword in pytime.c Only use it on the most important number. This change fixes also a compiler warning on modf(). files: Python/pytime.c | 6 ++++-- 1 files changed, 4 insertions(+), 2 deletions(-) diff --git a/Python/pytime.c b/Python/pytime.c --- a/Python/pytime.c +++ b/Python/pytime.c @@ -135,8 +135,9 @@ _PyTime_ObjectToTime_t(PyObject *obj, time_t *sec, _PyTime_round_t round) { if (PyFloat_Check(obj)) { + double intpart, err; /* volatile avoids optimization changing how numbers are rounded */ - volatile double d, intpart, err; + volatile double d; d = PyFloat_AsDouble(obj); if (round == _PyTime_ROUND_HALF_UP) @@ -257,8 +258,9 @@ _PyTime_FromFloatObject(_PyTime_t *t, double value, _PyTime_round_t round, long to_nanoseconds) { + double err; /* volatile avoids optimization changing how numbers are rounded */ - volatile double d, err; + volatile double d; /* convert to a number of nanoseconds */ d = value; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 3 16:34:59 2015 From: python-checkins at python.org (victor.stinner) Date: Thu, 03 Sep 2015 14:34:59 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Enhance_=5FPyTime=5FAsTime?= =?utf-8?b?c3BlYygp?= Message-ID: <20150903143453.68877.10439@psf.io> https://hg.python.org/cpython/rev/d78f7e858626 changeset: 97627:d78f7e858626 user: Victor Stinner date: Thu Sep 03 16:25:45 2015 +0200 summary: Enhance _PyTime_AsTimespec() Ensure that the tv_nsec field is set, even if the function fails with an overflow. files: Python/pytime.c | 6 +++--- 1 files changed, 3 insertions(+), 3 deletions(-) diff --git a/Python/pytime.c b/Python/pytime.c --- a/Python/pytime.c +++ b/Python/pytime.c @@ -479,13 +479,13 @@ secs -= 1; } ts->tv_sec = (time_t)secs; + assert(0 <= nsec && nsec < SEC_TO_NS); + ts->tv_nsec = nsec; + if ((_PyTime_t)ts->tv_sec != secs) { _PyTime_overflow(); return -1; } - ts->tv_nsec = nsec; - - assert(0 <= ts->tv_nsec && ts->tv_nsec < SEC_TO_NS); return 0; } #endif -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 3 17:51:00 2015 From: python-checkins at python.org (victor.stinner) Date: Thu, 03 Sep 2015 15:51:00 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogTWVyZ2UgMy41IChvZGljdCk=?= Message-ID: <20150903155100.14875.88083@psf.io> https://hg.python.org/cpython/rev/e9929da99349 changeset: 97630:e9929da99349 parent: 97628:5834a2a972a8 parent: 97629:ef1f5aebe1a6 user: Victor Stinner date: Thu Sep 03 17:50:30 2015 +0200 summary: Merge 3.5 (odict) files: Misc/NEWS | 3 ++ Objects/odictobject.c | 40 +++++++++++++++++------------- 2 files changed, 25 insertions(+), 18 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -92,6 +92,9 @@ Library ------- +- Issue #24992: Fix error handling and a race condition (related to garbage + collection) in collections.OrderedDict constructor. + - Issue #24881: Fixed setting binary mode in Python implementation of FileIO on Windows and Cygwin. Patch from Akira Li. diff --git a/Objects/odictobject.c b/Objects/odictobject.c --- a/Objects/odictobject.c +++ b/Objects/odictobject.c @@ -98,7 +98,6 @@ Others: -* _odict_initialize(od) * _odict_find_node(od, key) * _odict_keys_equal(od1, od2) @@ -602,15 +601,6 @@ return _odict_get_index_hash(od, key, hash); } -static int -_odict_initialize(PyODictObject *od) -{ - od->od_state = 0; - _odict_FIRST(od) = NULL; - _odict_LAST(od) = NULL; - return _odict_resize((PyODictObject *)od); -} - /* Returns NULL if there was some error or the key was not found. */ static _ODictNode * _odict_find_node(PyODictObject *od, PyObject *key) @@ -744,7 +734,7 @@ /* If someone calls PyDict_DelItem() directly on an OrderedDict, we'll get all sorts of problems here. In PyODict_DelItem we make sure to call _odict_clear_node first. - + This matters in the case of colliding keys. Suppose we add 3 keys: [A, B, C], where the hash of C collides with A and the next possible index in the hash table is occupied by B. If we remove B then for C @@ -1739,14 +1729,28 @@ static PyObject * odict_new(PyTypeObject *type, PyObject *args, PyObject *kwds) { - PyObject *od = PyDict_Type.tp_new(type, args, kwds); - if (od != NULL) { - if (_odict_initialize((PyODictObject *)od) < 0) - return NULL; - ((PyODictObject *)od)->od_inst_dict = PyDict_New(); - ((PyODictObject *)od)->od_weakreflist = NULL; + PyObject *dict; + PyODictObject *od; + + dict = PyDict_New(); + if (dict == NULL) + return NULL; + + od = (PyODictObject *)PyDict_Type.tp_new(type, args, kwds); + if (od == NULL) { + Py_DECREF(dict); + return NULL; } - return od; + + od->od_inst_dict = dict; + /* type constructor fills the memory with zeros (see + PyType_GenericAlloc()), there is no need to set them to zero again */ + if (_odict_resize(od) < 0) { + Py_DECREF(od); + return NULL; + } + + return (PyObject*)od; } /* PyODict_Type */ -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 3 17:51:02 2015 From: python-checkins at python.org (victor.stinner) Date: Thu, 03 Sep 2015 15:51:02 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI0OTky?= =?utf-8?q?=3A_Fix_error_handling_and_a_race_condition_=28related_to_garba?= =?utf-8?q?ge?= Message-ID: <20150903155059.101504.59881@psf.io> https://hg.python.org/cpython/rev/ef1f5aebe1a6 changeset: 97629:ef1f5aebe1a6 branch: 3.5 parent: 97625:ac1995b01028 user: Victor Stinner date: Thu Sep 03 17:50:04 2015 +0200 summary: Issue #24992: Fix error handling and a race condition (related to garbage collection) in collections.OrderedDict constructor. Patch reviewed by Serhiy Storchaka. files: Misc/NEWS | 3 ++ Objects/odictobject.c | 40 +++++++++++++++++------------- 2 files changed, 25 insertions(+), 18 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -14,6 +14,9 @@ Library ------- +- Issue #24992: Fix error handling and a race condition (related to garbage + collection) in collections.OrderedDict constructor. + - Issue #24881: Fixed setting binary mode in Python implementation of FileIO on Windows and Cygwin. Patch from Akira Li. diff --git a/Objects/odictobject.c b/Objects/odictobject.c --- a/Objects/odictobject.c +++ b/Objects/odictobject.c @@ -98,7 +98,6 @@ Others: -* _odict_initialize(od) * _odict_find_node(od, key) * _odict_keys_equal(od1, od2) @@ -602,15 +601,6 @@ return _odict_get_index_hash(od, key, hash); } -static int -_odict_initialize(PyODictObject *od) -{ - od->od_state = 0; - _odict_FIRST(od) = NULL; - _odict_LAST(od) = NULL; - return _odict_resize((PyODictObject *)od); -} - /* Returns NULL if there was some error or the key was not found. */ static _ODictNode * _odict_find_node(PyODictObject *od, PyObject *key) @@ -744,7 +734,7 @@ /* If someone calls PyDict_DelItem() directly on an OrderedDict, we'll get all sorts of problems here. In PyODict_DelItem we make sure to call _odict_clear_node first. - + This matters in the case of colliding keys. Suppose we add 3 keys: [A, B, C], where the hash of C collides with A and the next possible index in the hash table is occupied by B. If we remove B then for C @@ -1739,14 +1729,28 @@ static PyObject * odict_new(PyTypeObject *type, PyObject *args, PyObject *kwds) { - PyObject *od = PyDict_Type.tp_new(type, args, kwds); - if (od != NULL) { - if (_odict_initialize((PyODictObject *)od) < 0) - return NULL; - ((PyODictObject *)od)->od_inst_dict = PyDict_New(); - ((PyODictObject *)od)->od_weakreflist = NULL; + PyObject *dict; + PyODictObject *od; + + dict = PyDict_New(); + if (dict == NULL) + return NULL; + + od = (PyODictObject *)PyDict_Type.tp_new(type, args, kwds); + if (od == NULL) { + Py_DECREF(dict); + return NULL; } - return od; + + od->od_inst_dict = dict; + /* type constructor fills the memory with zeros (see + PyType_GenericAlloc()), there is no need to set them to zero again */ + if (_odict_resize(od) < 0) { + Py_DECREF(od); + return NULL; + } + + return (PyObject*)od; } /* PyODict_Type */ -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 3 18:58:35 2015 From: python-checkins at python.org (zach.ware) Date: Thu, 03 Sep 2015 16:58:35 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Closes_=2324974=3A_Merge_with_3=2E5?= Message-ID: <20150903165835.17967.89548@psf.io> https://hg.python.org/cpython/rev/88c28d29afe4 changeset: 97632:88c28d29afe4 parent: 97630:e9929da99349 parent: 97631:863407e80370 user: Zachary Ware date: Thu Sep 03 11:54:51 2015 -0500 summary: Closes #24974: Merge with 3.5 files: Modules/_decimal/libmpdec/mpdecimal.c | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diff --git a/Modules/_decimal/libmpdec/mpdecimal.c b/Modules/_decimal/libmpdec/mpdecimal.c --- a/Modules/_decimal/libmpdec/mpdecimal.c +++ b/Modules/_decimal/libmpdec/mpdecimal.c @@ -43,6 +43,7 @@ #ifdef PPRO #if defined(_MSC_VER) #include + #pragma float_control(precise, on) #pragma fenv_access(on) #elif !defined(__OpenBSD__) && !defined(__NetBSD__) /* C99 */ -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 3 18:58:35 2015 From: python-checkins at python.org (zach.ware) Date: Thu, 03 Sep 2015 16:58:35 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI0OTc0?= =?utf-8?q?=3A_Force_fp-model_precice_in_mpdecimal=2Ec_on_Windows?= Message-ID: <20150903165835.17969.54621@psf.io> https://hg.python.org/cpython/rev/863407e80370 changeset: 97631:863407e80370 branch: 3.5 parent: 97629:ef1f5aebe1a6 user: Zachary Ware date: Thu Sep 03 11:52:15 2015 -0500 summary: Issue #24974: Force fp-model precice in mpdecimal.c on Windows As suggested by Steve Dower and approved by Stefan Krah. files: Modules/_decimal/libmpdec/mpdecimal.c | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diff --git a/Modules/_decimal/libmpdec/mpdecimal.c b/Modules/_decimal/libmpdec/mpdecimal.c --- a/Modules/_decimal/libmpdec/mpdecimal.c +++ b/Modules/_decimal/libmpdec/mpdecimal.c @@ -43,6 +43,7 @@ #ifdef PPRO #if defined(_MSC_VER) #include + #pragma float_control(precise, on) #pragma fenv_access(on) #elif !defined(__OpenBSD__) && !defined(__NetBSD__) /* C99 */ -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 3 21:34:53 2015 From: python-checkins at python.org (victor.stinner) Date: Thu, 03 Sep 2015 19:34:53 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogTWVyZ2UgMy41IChJQ0Mp?= Message-ID: <20150903193453.17963.67595@psf.io> https://hg.python.org/cpython/rev/80ca61c1c06b changeset: 97635:80ca61c1c06b parent: 97632:88c28d29afe4 parent: 97634:c9dbfd8edcaa user: Victor Stinner date: Thu Sep 03 21:34:03 2015 +0200 summary: Merge 3.5 (ICC) files: Modules/posixmodule.c | 12 ++++-------- 1 files changed, 4 insertions(+), 8 deletions(-) diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c --- a/Modules/posixmodule.c +++ b/Modules/posixmodule.c @@ -4560,9 +4560,7 @@ } \ -#define UTIME_HAVE_DIR_FD (defined(HAVE_FUTIMESAT) || defined(HAVE_UTIMENSAT)) - -#if UTIME_HAVE_DIR_FD +#if defined(HAVE_FUTIMESAT) || defined(HAVE_UTIMENSAT) static int utime_dir_fd(utime_t *ut, int dir_fd, char *path, int follow_symlinks) @@ -4588,9 +4586,7 @@ #define FUTIMENSAT_DIR_FD_CONVERTER dir_fd_unavailable #endif -#define UTIME_HAVE_FD (defined(HAVE_FUTIMES) || defined(HAVE_FUTIMENS)) - -#if UTIME_HAVE_FD +#if defined(HAVE_FUTIMES) || defined(HAVE_FUTIMENS) static int utime_fd(utime_t *ut, int fd) @@ -4835,13 +4831,13 @@ else #endif -#if UTIME_HAVE_DIR_FD +#if defined(HAVE_FUTIMESAT) || defined(HAVE_UTIMENSAT) if ((dir_fd != DEFAULT_DIR_FD) || (!follow_symlinks)) result = utime_dir_fd(&utime, dir_fd, path->narrow, follow_symlinks); else #endif -#if UTIME_HAVE_FD +#if defined(HAVE_FUTIMES) || defined(HAVE_FUTIMENS) if (path->fd != -1) result = utime_fd(&utime, path->fd); else -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 3 21:34:56 2015 From: python-checkins at python.org (victor.stinner) Date: Thu, 03 Sep 2015 19:34:56 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogRG9uJ3QgdXNlIGRl?= =?utf-8?q?fined=28=29_in_C_preprocessor_macros?= Message-ID: <20150903193452.101488.90016@psf.io> https://hg.python.org/cpython/rev/94f61fe22edf changeset: 97633:94f61fe22edf branch: 3.4 parent: 97622:29fa1d33e421 user: Victor Stinner date: Thu Sep 03 21:30:26 2015 +0200 summary: Don't use defined() in C preprocessor macros The ICC compiler doesn't seem to support defined() in macro expansion. Example of warning: warning #3199: "defined" is always false in a macro expansion in Microsoft mode files: Modules/posixmodule.c | 16 ++++++---------- 1 files changed, 6 insertions(+), 10 deletions(-) diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c --- a/Modules/posixmodule.c +++ b/Modules/posixmodule.c @@ -4781,9 +4781,7 @@ } \ -#define UTIME_HAVE_DIR_FD (defined(HAVE_FUTIMESAT) || defined(HAVE_UTIMENSAT)) - -#if UTIME_HAVE_DIR_FD +#if defined(HAVE_FUTIMESAT) || defined(HAVE_UTIMENSAT) static int utime_dir_fd(utime_t *ut, int dir_fd, char *path, int follow_symlinks) @@ -4806,9 +4804,7 @@ #endif -#define UTIME_HAVE_FD (defined(HAVE_FUTIMES) || defined(HAVE_FUTIMENS)) - -#if UTIME_HAVE_FD +#if defined(HAVE_FUTIMES) || defined(HAVE_FUTIMENS) static int utime_fd(utime_t *ut, int fd) @@ -4912,14 +4908,14 @@ memset(&path, 0, sizeof(path)); path.function_name = "utime"; memset(&utime, 0, sizeof(utime_t)); -#if UTIME_HAVE_FD +#if defined(HAVE_FUTIMES) || defined(HAVE_FUTIMENS) path.allow_fd = 1; #endif if (!PyArg_ParseTupleAndKeywords(args, kwargs, "O&|O$OO&p:utime", keywords, path_converter, &path, ×, &ns, -#if UTIME_HAVE_DIR_FD +#if defined(HAVE_FUTIMESAT) || defined(HAVE_UTIMENSAT) dir_fd_converter, &dir_fd, #else dir_fd_unavailable, &dir_fd, @@ -5035,13 +5031,13 @@ else #endif -#if UTIME_HAVE_DIR_FD +#if defined(HAVE_FUTIMESAT) || defined(HAVE_UTIMENSAT) if ((dir_fd != DEFAULT_DIR_FD) || (!follow_symlinks)) result = utime_dir_fd(&utime, dir_fd, path.narrow, follow_symlinks); else #endif -#if UTIME_HAVE_FD +#if defined(HAVE_FUTIMES) || defined(HAVE_FUTIMENS) if (path.fd != -1) result = utime_fd(&utime, path.fd); else -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 3 21:34:56 2015 From: python-checkins at python.org (victor.stinner) Date: Thu, 03 Sep 2015 19:34:56 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Merge_3=2E4_=28ICC=29?= Message-ID: <20150903193452.115151.58144@psf.io> https://hg.python.org/cpython/rev/c9dbfd8edcaa changeset: 97634:c9dbfd8edcaa branch: 3.5 parent: 97631:863407e80370 parent: 97633:94f61fe22edf user: Victor Stinner date: Thu Sep 03 21:32:44 2015 +0200 summary: Merge 3.4 (ICC) files: Modules/posixmodule.c | 12 ++++-------- 1 files changed, 4 insertions(+), 8 deletions(-) diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c --- a/Modules/posixmodule.c +++ b/Modules/posixmodule.c @@ -4560,9 +4560,7 @@ } \ -#define UTIME_HAVE_DIR_FD (defined(HAVE_FUTIMESAT) || defined(HAVE_UTIMENSAT)) - -#if UTIME_HAVE_DIR_FD +#if defined(HAVE_FUTIMESAT) || defined(HAVE_UTIMENSAT) static int utime_dir_fd(utime_t *ut, int dir_fd, char *path, int follow_symlinks) @@ -4588,9 +4586,7 @@ #define FUTIMENSAT_DIR_FD_CONVERTER dir_fd_unavailable #endif -#define UTIME_HAVE_FD (defined(HAVE_FUTIMES) || defined(HAVE_FUTIMENS)) - -#if UTIME_HAVE_FD +#if defined(HAVE_FUTIMES) || defined(HAVE_FUTIMENS) static int utime_fd(utime_t *ut, int fd) @@ -4835,13 +4831,13 @@ else #endif -#if UTIME_HAVE_DIR_FD +#if defined(HAVE_FUTIMESAT) || defined(HAVE_UTIMENSAT) if ((dir_fd != DEFAULT_DIR_FD) || (!follow_symlinks)) result = utime_dir_fd(&utime, dir_fd, path->narrow, follow_symlinks); else #endif -#if UTIME_HAVE_FD +#if defined(HAVE_FUTIMES) || defined(HAVE_FUTIMENS) if (path->fd != -1) result = utime_fd(&utime, path->fd); else -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 4 00:09:47 2015 From: python-checkins at python.org (serhiy.storchaka) Date: Thu, 03 Sep 2015 22:09:47 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI0OTg5?= =?utf-8?q?=3A_Fixed_buffer_overread_in_BytesIO=2Ereadline=28=29_if_a_posi?= =?utf-8?q?tion_is?= Message-ID: <20150903220946.17983.50209@psf.io> https://hg.python.org/cpython/rev/a5858c30db7c changeset: 97636:a5858c30db7c branch: 3.5 parent: 97634:c9dbfd8edcaa user: Serhiy Storchaka date: Fri Sep 04 01:08:03 2015 +0300 summary: Issue #24989: Fixed buffer overread in BytesIO.readline() if a position is set beyond size. Based on patch by John Leitch. files: Lib/test/test_memoryio.py | 13 +++++++++++++ Misc/NEWS | 3 +++ Modules/_io/bytesio.c | 6 +++++- 3 files changed, 21 insertions(+), 1 deletions(-) diff --git a/Lib/test/test_memoryio.py b/Lib/test/test_memoryio.py --- a/Lib/test/test_memoryio.py +++ b/Lib/test/test_memoryio.py @@ -166,6 +166,10 @@ memio.seek(0) self.assertEqual(memio.read(None), buf) self.assertRaises(TypeError, memio.read, '') + memio.seek(len(buf) + 1) + self.assertEqual(memio.read(1), self.EOF) + memio.seek(len(buf) + 1) + self.assertEqual(memio.read(), self.EOF) memio.close() self.assertRaises(ValueError, memio.read) @@ -185,6 +189,9 @@ self.assertEqual(memio.readline(-1), buf) memio.seek(0) self.assertEqual(memio.readline(0), self.EOF) + # Issue #24989: Buffer overread + memio.seek(len(buf) * 2 + 1) + self.assertEqual(memio.readline(), self.EOF) buf = self.buftype("1234567890\n") memio = self.ioclass((buf * 3)[:-1]) @@ -217,6 +224,9 @@ memio.seek(0) self.assertEqual(memio.readlines(None), [buf] * 10) self.assertRaises(TypeError, memio.readlines, '') + # Issue #24989: Buffer overread + memio.seek(len(buf) * 10 + 1) + self.assertEqual(memio.readlines(), []) memio.close() self.assertRaises(ValueError, memio.readlines) @@ -238,6 +248,9 @@ self.assertEqual(line, buf) i += 1 self.assertEqual(i, 10) + # Issue #24989: Buffer overread + memio.seek(len(buf) * 10 + 1) + self.assertEqual(list(memio), []) memio = self.ioclass(buf * 2) memio.close() self.assertRaises(ValueError, memio.__next__) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -92,6 +92,9 @@ Library ------- +- Issue #24989: Fixed buffer overread in BytesIO.readline() if a position is + set beyond size. Based on patch by John Leitch. + - Issue #24847: Removes vcruntime140.dll dependency from Tcl/Tk. - Issue #24839: platform._syscmd_ver raises DeprecationWarning diff --git a/Modules/_io/bytesio.c b/Modules/_io/bytesio.c --- a/Modules/_io/bytesio.c +++ b/Modules/_io/bytesio.c @@ -57,14 +57,18 @@ Py_ssize_t maxlen; assert(self->buf != NULL); + assert(self->pos >= 0); + + if (self->pos >= self->string_size) + return 0; /* Move to the end of the line, up to the end of the string, s. */ - start = PyBytes_AS_STRING(self->buf) + self->pos; maxlen = self->string_size - self->pos; if (len < 0 || len > maxlen) len = maxlen; if (len) { + start = PyBytes_AS_STRING(self->buf) + self->pos; n = memchr(start, '\n', len); if (n) /* Get the length from the current position to the end of -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 4 00:09:58 2015 From: python-checkins at python.org (serhiy.storchaka) Date: Thu, 03 Sep 2015 22:09:58 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2324989=3A_Fixed_buffer_overread_in_BytesIO=2Erea?= =?utf-8?q?dline=28=29_if_a_position_is?= Message-ID: <20150903220947.101488.18067@psf.io> https://hg.python.org/cpython/rev/215800fb955d changeset: 97637:215800fb955d parent: 97635:80ca61c1c06b parent: 97636:a5858c30db7c user: Serhiy Storchaka date: Fri Sep 04 01:08:54 2015 +0300 summary: Issue #24989: Fixed buffer overread in BytesIO.readline() if a position is set beyond size. Based on patch by John Leitch. files: Lib/test/test_memoryio.py | 13 +++++++++++++ Misc/NEWS | 3 +++ Modules/_io/bytesio.c | 6 +++++- 3 files changed, 21 insertions(+), 1 deletions(-) diff --git a/Lib/test/test_memoryio.py b/Lib/test/test_memoryio.py --- a/Lib/test/test_memoryio.py +++ b/Lib/test/test_memoryio.py @@ -166,6 +166,10 @@ memio.seek(0) self.assertEqual(memio.read(None), buf) self.assertRaises(TypeError, memio.read, '') + memio.seek(len(buf) + 1) + self.assertEqual(memio.read(1), self.EOF) + memio.seek(len(buf) + 1) + self.assertEqual(memio.read(), self.EOF) memio.close() self.assertRaises(ValueError, memio.read) @@ -185,6 +189,9 @@ self.assertEqual(memio.readline(-1), buf) memio.seek(0) self.assertEqual(memio.readline(0), self.EOF) + # Issue #24989: Buffer overread + memio.seek(len(buf) * 2 + 1) + self.assertEqual(memio.readline(), self.EOF) buf = self.buftype("1234567890\n") memio = self.ioclass((buf * 3)[:-1]) @@ -217,6 +224,9 @@ memio.seek(0) self.assertEqual(memio.readlines(None), [buf] * 10) self.assertRaises(TypeError, memio.readlines, '') + # Issue #24989: Buffer overread + memio.seek(len(buf) * 10 + 1) + self.assertEqual(memio.readlines(), []) memio.close() self.assertRaises(ValueError, memio.readlines) @@ -238,6 +248,9 @@ self.assertEqual(line, buf) i += 1 self.assertEqual(i, 10) + # Issue #24989: Buffer overread + memio.seek(len(buf) * 10 + 1) + self.assertEqual(list(memio), []) memio = self.ioclass(buf * 2) memio.close() self.assertRaises(ValueError, memio.__next__) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -167,6 +167,9 @@ Library ------- +- Issue #24989: Fixed buffer overread in BytesIO.readline() if a position is + set beyond size. Based on patch by John Leitch. + - Issue #24847: Removes vcruntime140.dll dependency from Tcl/Tk. - Issue #24839: platform._syscmd_ver raises DeprecationWarning diff --git a/Modules/_io/bytesio.c b/Modules/_io/bytesio.c --- a/Modules/_io/bytesio.c +++ b/Modules/_io/bytesio.c @@ -57,14 +57,18 @@ Py_ssize_t maxlen; assert(self->buf != NULL); + assert(self->pos >= 0); + + if (self->pos >= self->string_size) + return 0; /* Move to the end of the line, up to the end of the string, s. */ - start = PyBytes_AS_STRING(self->buf) + self->pos; maxlen = self->string_size - self->pos; if (len < 0 || len > maxlen) len = maxlen; if (len) { + start = PyBytes_AS_STRING(self->buf) + self->pos; n = memchr(start, '\n', len); if (n) /* Get the length from the current position to the end of -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 4 00:35:55 2015 From: python-checkins at python.org (brett.cannon) Date: Thu, 03 Sep 2015 22:35:55 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy41IC0+IDMuNSk6?= =?utf-8?q?_Merge_from_3=2E5=2E0_for_issue_=2324913?= Message-ID: <20150903223555.17985.55581@psf.io> https://hg.python.org/cpython/rev/d093d87e449c changeset: 97639:d093d87e449c branch: 3.5 parent: 97636:a5858c30db7c parent: 97638:9f8c59e61594 user: Brett Cannon date: Thu Sep 03 15:34:57 2015 -0700 summary: Merge from 3.5.0 for issue #24913 files: Misc/NEWS | 3 +++ 1 files changed, 3 insertions(+), 0 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -70,6 +70,9 @@ Library ------- +- Issue #24913: Fix overrun error in deque.index(). + Found by John Leitch and Bryce Darling. + What's New in Python 3.5.0 release candidate 2? =============================================== -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 4 00:35:55 2015 From: python-checkins at python.org (brett.cannon) Date: Thu, 03 Sep 2015 22:35:55 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge_from_3=2E5_for_issue_=2324913?= Message-ID: <20150903223555.11991.30112@psf.io> https://hg.python.org/cpython/rev/c6e0c29913ec changeset: 97640:c6e0c29913ec parent: 97637:215800fb955d parent: 97639:d093d87e449c user: Brett Cannon date: Thu Sep 03 15:35:33 2015 -0700 summary: Merge from 3.5 for issue #24913 files: Misc/NEWS | 3 +++ 1 files changed, 3 insertions(+), 0 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -145,6 +145,9 @@ Library ------- +- Issue #24913: Fix overrun error in deque.index(). + Found by John Leitch and Bryce Darling. + What's New in Python 3.5.0 release candidate 2? =============================================== -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 4 00:35:55 2015 From: python-checkins at python.org (brett.cannon) Date: Thu, 03 Sep 2015 22:35:55 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI0OTEz?= =?utf-8?q?=3A_Fix_overrun_error_in_deque=2Eindex=28=29=2E?= Message-ID: <20150903223554.17973.62656@psf.io> https://hg.python.org/cpython/rev/9f8c59e61594 changeset: 97638:9f8c59e61594 branch: 3.5 parent: 97583:e9df8543d7bc user: Brett Cannon date: Thu Sep 03 10:15:03 2015 -0700 summary: Issue #24913: Fix overrun error in deque.index(). Reported by John Leitch and Bryce Darling, patch by Raymond Hettinger. files: Lib/test/test_deque.py | 5 +++++ Misc/NEWS | 3 +++ Modules/_collectionsmodule.c | 2 ++ 3 files changed, 10 insertions(+), 0 deletions(-) diff --git a/Lib/test/test_deque.py b/Lib/test/test_deque.py --- a/Lib/test/test_deque.py +++ b/Lib/test/test_deque.py @@ -289,6 +289,11 @@ else: self.assertEqual(d.index(element, start, stop), target) + def test_insert_bug_24913(self): + d = deque('A' * 3) + with self.assertRaises(ValueError): + i = d.index("Hello world", 0, 4) + def test_insert(self): # Test to make sure insert behaves like lists elements = 'ABCDEFGHI' diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -15,6 +15,9 @@ Library ------- +- Issue #24913: Fix overrun error in deque.index(). + Found by John Leitch and Bryce Darling. + What's New in Python 3.5.0 release candidate 2? =============================================== diff --git a/Modules/_collectionsmodule.c b/Modules/_collectionsmodule.c --- a/Modules/_collectionsmodule.c +++ b/Modules/_collectionsmodule.c @@ -924,6 +924,8 @@ if (stop < 0) stop = 0; } + if (stop > Py_SIZE(deque)) + stop = Py_SIZE(deque); for (i=0 ; i= start) { -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 4 06:56:16 2015 From: python-checkins at python.org (zach.ware) Date: Fri, 04 Sep 2015 04:56:16 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_Allow_PCbuild?= =?utf-8?q?=5Crt=2Ebat_to_accept_unlimited_arguments_for_regrtest=2E?= Message-ID: <20150904045616.11270.87831@psf.io> https://hg.python.org/cpython/rev/10600293b466 changeset: 97643:10600293b466 branch: 3.5 parent: 97639:d093d87e449c user: Zachary Ware date: Thu Sep 03 23:43:37 2015 -0500 summary: Allow PCbuild\rt.bat to accept unlimited arguments for regrtest. This makes it possible to pass more than 4 tests by name through Tools\buildbot\test.bat files: Misc/NEWS | 6 ++++++ PCbuild/rt.bat | 4 +++- Tools/buildbot/test.bat | 26 +++++++++++++++----------- 3 files changed, 24 insertions(+), 12 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -56,6 +56,12 @@ - Issue #22812: Fix unittest discovery examples. Patch from Pam McA'Nulty. +Tests +----- + +- PCbuild\rt.bat now accepts an unlimited number of arguments to pass along + to regrtest.py. Previously there was a limit of 9. + What's New in Python 3.5.0 release candidate 3? =============================================== diff --git a/PCbuild/rt.bat b/PCbuild/rt.bat --- a/PCbuild/rt.bat +++ b/PCbuild/rt.bat @@ -32,15 +32,17 @@ set suffix= set qmode= set dashO= +set regrtestargs= :CheckOpts if "%1"=="-O" (set dashO=-O) & shift & goto CheckOpts if "%1"=="-q" (set qmode=yes) & shift & goto CheckOpts if "%1"=="-d" (set suffix=_d) & shift & goto CheckOpts if "%1"=="-x64" (set prefix=%pcbuild%amd64\) & shift & goto CheckOpts +if NOT "%1"=="" (set regrtestargs=%regrtestargs% %1) & shift & goto CheckOpts set exe=%prefix%python%suffix%.exe -set cmd="%exe%" %dashO% -Wd -E -bb "%pcbuild%..\lib\test\regrtest.py" %1 %2 %3 %4 %5 %6 %7 %8 %9 +set cmd="%exe%" %dashO% -Wd -E -bb "%pcbuild%..\lib\test\regrtest.py" %regrtestargs% if defined qmode goto Qmode echo Deleting .pyc/.pyo files ... diff --git a/Tools/buildbot/test.bat b/Tools/buildbot/test.bat --- a/Tools/buildbot/test.bat +++ b/Tools/buildbot/test.bat @@ -1,15 +1,19 @@ - at rem Used by the buildbot "test" step. - at setlocal + at echo off +rem Used by the buildbot "test" step. +setlocal - at set here=%~dp0 - at set rt_opts=-q -d +set here=%~dp0 +set rt_opts=-q -d +set regrtest_args= :CheckOpts - at if '%1'=='-x64' (set rt_opts=%rt_opts% %1) & shift & goto CheckOpts - at if '%1'=='-d' (set rt_opts=%rt_opts% %1) & shift & goto CheckOpts - at if '%1'=='-O' (set rt_opts=%rt_opts% %1) & shift & goto CheckOpts - at if '%1'=='-q' (set rt_opts=%rt_opts% %1) & shift & goto CheckOpts - at if '%1'=='+d' (set rt_opts=%rt_opts:-d=%) & shift & goto CheckOpts - at if '%1'=='+q' (set rt_opts=%rt_opts:-q=%) & shift & goto CheckOpts +if "%1"=="-x64" (set rt_opts=%rt_opts% %1) & shift & goto CheckOpts +if "%1"=="-d" (set rt_opts=%rt_opts% %1) & shift & goto CheckOpts +if "%1"=="-O" (set rt_opts=%rt_opts% %1) & shift & goto CheckOpts +if "%1"=="-q" (set rt_opts=%rt_opts% %1) & shift & goto CheckOpts +if "%1"=="+d" (set rt_opts=%rt_opts:-d=%) & shift & goto CheckOpts +if "%1"=="+q" (set rt_opts=%rt_opts:-q=%) & shift & goto CheckOpts +if NOT "%1"=="" (set regrtest_args=%regrtest_args% %1) & shift & goto CheckOpts -call "%here%..\..\PCbuild\rt.bat" %rt_opts% -uall -rwW -n --timeout=3600 %1 %2 %3 %4 %5 %6 %7 %8 %9 +echo on +call "%here%..\..\PCbuild\rt.bat" %rt_opts% -uall -rwW -n --timeout=3600 %regrtest_args% -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 4 06:56:16 2015 From: python-checkins at python.org (zach.ware) Date: Fri, 04 Sep 2015 04:56:16 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzI0OTg2?= =?utf-8?q?=3A_Allow_building_Python_without_external_libraries_on_Windows?= Message-ID: <20150904045616.11274.57963@psf.io> https://hg.python.org/cpython/rev/2bc91f1f2b34 changeset: 97642:2bc91f1f2b34 branch: 2.7 user: Zachary Ware date: Thu Sep 03 23:27:05 2015 -0500 summary: Issue #24986: Allow building Python without external libraries on Windows This modifies the behavior of the '-e' flag to PCbuild\build.bat: when '-e' is not supplied, no attempt will be made to build extension modules that require external libraries, even if the external libraries are present. Also adds '--no-' flags to PCbuild\build.bat, where '' is one of 'ssl', 'tkinter', or 'bsddb', to allow skipping just those modules (if '-e' is given). files: Misc/NEWS | 3 +++ PCbuild/build.bat | 28 ++++++++++++++++++++++++++-- PCbuild/pcbuild.proj | 14 +++++++++++--- 3 files changed, 40 insertions(+), 5 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -134,6 +134,9 @@ Build ----- +- Issue #24986: It is now possible to build Python on Windows without errors + when external libraries are not available. + - Issue #24508: Backported the MSBuild project files from Python 3.5. The backported files replace the old project files in PCbuild; the old files moved to PC/VS9.0 and remain supported. diff --git a/PCbuild/build.bat b/PCbuild/build.bat --- a/PCbuild/build.bat +++ b/PCbuild/build.bat @@ -17,12 +17,20 @@ echo. -r Target Rebuild instead of Build echo. -d Set the configuration to Debug echo. -e Build external libraries fetched by get_externals.bat +echo. Extension modules that depend on external libraries will not attempt +echo. to build if this flag is not present echo. -m Enable parallel build echo. -M Disable parallel build (disabled by default) echo. -v Increased output messages echo. -k Attempt to kill any running Pythons before building (usually done echo. automatically by the pythoncore project) echo. +echo.Available flags to avoid building certain modules. +echo.These flags have no effect if '-e' is not given: +echo. --no-ssl Do not attempt to build _ssl +echo. --no-tkinter Do not attempt to build Tkinter +echo. --no-bsddb Do not attempt to build _bsddb +echo. echo.Available arguments: echo. -c Release ^| Debug ^| PGInstrument ^| PGUpdate echo. Set the configuration (default: Release) @@ -50,11 +58,22 @@ if "%~1"=="-r" (set target=Rebuild) & shift & goto CheckOpts if "%~1"=="-t" (set target=%2) & shift & shift & goto CheckOpts if "%~1"=="-d" (set conf=Debug) & shift & goto CheckOpts -if "%~1"=="-e" call "%dir%get_externals.bat" & shift & goto CheckOpts if "%~1"=="-m" (set parallel=/m) & shift & goto CheckOpts if "%~1"=="-M" (set parallel=) & shift & goto CheckOpts if "%~1"=="-v" (set verbose=/v:n) & shift & goto CheckOpts if "%~1"=="-k" (set kill=true) & shift & goto CheckOpts +rem These use the actual property names used by MSBuild. We could just let +rem them in through the environment, but we specify them on the command line +rem anyway for visibility so set defaults after this +if "%~1"=="-e" (set IncludeExternals=true) & call "%dir%get_externals.bat" & shift & goto CheckOpts +if "%~1"=="--no-ssl" (set IncludeSSL=false) & shift & goto CheckOpts +if "%~1"=="--no-tkinter" (set IncludeTkinter=false) & shift & goto CheckOpts +if "%~1"=="--no-bsddb" (set IncludeBsddb=false) & shift & goto CheckOpts + +if "%IncludeExternals%"=="" set IncludeExternals=false +if "%IncludeSSL%"=="" set IncludeSSL=true +if "%IncludeTkinter%"=="" set IncludeTkinter=true +if "%IncludeBsddb%"=="" set IncludeBsddb=true if "%platf%"=="x64" (set vs_platf=x86_amd64) @@ -69,4 +88,9 @@ rem Passing %1-9 is not the preferred option, but argument parsing in rem batch is, shall we say, "lackluster" echo on -msbuild "%dir%pcbuild.proj" /t:%target% %parallel% %verbose% /p:Configuration=%conf% /p:Platform=%platf% %1 %2 %3 %4 %5 %6 %7 %8 %9 +msbuild "%dir%pcbuild.proj" /t:%target% %parallel% %verbose%^ + /p:Configuration=%conf% /p:Platform=%platf%^ + /p:IncludeExternals=%IncludeExternals%^ + /p:IncludeSSL=%IncludeSSL% /p:IncludeTkinter=%IncludeTkinter%^ + /p:IncludeBsddb=%IncludeBsddb%^ + %1 %2 %3 %4 %5 %6 %7 %8 %9 diff --git a/PCbuild/pcbuild.proj b/PCbuild/pcbuild.proj --- a/PCbuild/pcbuild.proj +++ b/PCbuild/pcbuild.proj @@ -5,8 +5,11 @@ Win32 Release true + true true true + true + true @@ -34,10 +37,15 @@ - + + + - - + + + + + -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 4 06:56:16 2015 From: python-checkins at python.org (zach.ware) Date: Fri, 04 Sep 2015 04:56:16 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI0OTg2?= =?utf-8?q?=3A_Allow_building_Python_without_external_libraries_on_Windows?= Message-ID: <20150904045616.68881.48830@psf.io> https://hg.python.org/cpython/rev/301c36746e42 changeset: 97644:301c36746e42 branch: 3.5 user: Zachary Ware date: Thu Sep 03 23:43:54 2015 -0500 summary: Issue #24986: Allow building Python without external libraries on Windows This modifies the behavior of the '-e' flag to PCbuild\build.bat: when '-e' is not supplied, no attempt will be made to build extension modules that require external libraries, even if the external libraries are present. Also adds '--no-' flags to PCbuild\build.bat, where '' is one of 'ssl', 'tkinter', or 'bsddb', to allow skipping just those modules (if '-e' is given). files: Misc/NEWS | 6 ++++++ PCbuild/build.bat | 24 ++++++++++++++++++++++-- PCbuild/pcbuild.proj | 14 ++++++++++---- 3 files changed, 38 insertions(+), 6 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -62,6 +62,12 @@ - PCbuild\rt.bat now accepts an unlimited number of arguments to pass along to regrtest.py. Previously there was a limit of 9. +Build +----- + +- Issue #24986: It is now possible to build Python on Windows without errors + when external libraries are not available. + What's New in Python 3.5.0 release candidate 3? =============================================== diff --git a/PCbuild/build.bat b/PCbuild/build.bat --- a/PCbuild/build.bat +++ b/PCbuild/build.bat @@ -18,12 +18,19 @@ echo. -r Target Rebuild instead of Build echo. -d Set the configuration to Debug echo. -e Build external libraries fetched by get_externals.bat +echo. Extension modules that depend on external libraries will not attempt +echo. to build if this flag is not present echo. -m Enable parallel build (enabled by default) echo. -M Disable parallel build echo. -v Increased output messages echo. -k Attempt to kill any running Pythons before building (usually done echo. automatically by the pythoncore project) echo. +echo.Available flags to avoid building certain modules. +echo.These flags have no effect if '-e' is not given: +echo. --no-ssl Do not attempt to build _ssl +echo. --no-tkinter Do not attempt to build Tkinter +echo. echo.Available arguments: echo. -c Release ^| Debug ^| PGInstrument ^| PGUpdate echo. Set the configuration (default: Release) @@ -51,12 +58,21 @@ if "%~1"=="-r" (set target=Rebuild) & shift & goto CheckOpts if "%~1"=="-t" (set target=%2) & shift & shift & goto CheckOpts if "%~1"=="-d" (set conf=Debug) & shift & goto CheckOpts -if "%~1"=="-e" call "%dir%get_externals.bat" & shift & goto CheckOpts if "%~1"=="-m" (set parallel=/m) & shift & goto CheckOpts if "%~1"=="-M" (set parallel=) & shift & goto CheckOpts if "%~1"=="-v" (set verbose=/v:n) & shift & goto CheckOpts if "%~1"=="-k" (set kill=true) & shift & goto CheckOpts if "%~1"=="-V" shift & goto Version +rem These use the actual property names used by MSBuild. We could just let +rem them in through the environment, but we specify them on the command line +rem anyway for visibility so set defaults after this +if "%~1"=="-e" (set IncludeExternals=true) & call "%dir%get_externals.bat" & shift & goto CheckOpts +if "%~1"=="--no-ssl" (set IncludeSSL=false) & shift & goto CheckOpts +if "%~1"=="--no-tkinter" (set IncludeTkinter=false) & shift & goto CheckOpts + +if "%IncludeExternals%"=="" set IncludeExternals=false +if "%IncludeSSL%"=="" set IncludeSSL=true +if "%IncludeTkinter%"=="" set IncludeTkinter=true if "%platf%"=="x64" (set vs_platf=x86_amd64) @@ -71,7 +87,11 @@ rem Passing %1-9 is not the preferred option, but argument parsing in rem batch is, shall we say, "lackluster" echo on -msbuild "%dir%pcbuild.proj" /t:%target% %parallel% %verbose% /p:Configuration=%conf% /p:Platform=%platf% %1 %2 %3 %4 %5 %6 %7 %8 %9 +msbuild "%dir%pcbuild.proj" /t:%target% %parallel% %verbose%^ + /p:Configuration=%conf% /p:Platform=%platf%^ + /p:IncludeExternals=%IncludeExternals%^ + /p:IncludeSSL=%IncludeSSL% /p:IncludeTkinter=%IncludeTkinter%^ + %1 %2 %3 %4 %5 %6 %7 %8 %9 @goto :eof diff --git a/PCbuild/pcbuild.proj b/PCbuild/pcbuild.proj --- a/PCbuild/pcbuild.proj +++ b/PCbuild/pcbuild.proj @@ -5,8 +5,10 @@ Win32 Release true + true true true + true @@ -25,7 +27,7 @@ @@ -40,10 +42,14 @@ - + + + - - + + + + -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 4 06:56:16 2015 From: python-checkins at python.org (zach.ware) Date: Fri, 04 Sep 2015 04:56:16 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_Allow_PCbuild?= =?utf-8?q?=5Crt=2Ebat_to_accept_unlimited_arguments_for_regrtest=2E?= Message-ID: <20150904045615.27695.62583@psf.io> https://hg.python.org/cpython/rev/6b275e2e8104 changeset: 97641:6b275e2e8104 branch: 2.7 parent: 97616:d687912d499f user: Zachary Ware date: Thu Sep 03 23:37:18 2015 -0500 summary: Allow PCbuild\rt.bat to accept unlimited arguments for regrtest. This makes it possible to pass more than 7 tests by name through Tools\buildbot\test.bat files: Misc/NEWS | 3 +++ PCbuild/rt.bat | 6 ++++-- Tools/buildbot/test.bat | 26 +++++++++++++++----------- 3 files changed, 22 insertions(+), 13 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -180,6 +180,9 @@ a test run is no longer marked as a failure if all tests succeed when re-run. +- PCbuild\rt.bat now accepts an unlimited number of arguments to pass along + to regrtest.py. Previously there was a limit of 9. + What's New in Python 2.7.10? ============================ diff --git a/PCbuild/rt.bat b/PCbuild/rt.bat --- a/PCbuild/rt.bat +++ b/PCbuild/rt.bat @@ -32,15 +32,17 @@ set suffix= set qmode= set dashO= +set regrtestargs= :CheckOpts if "%1"=="-O" (set dashO=-O) & shift & goto CheckOpts if "%1"=="-q" (set qmode=yes) & shift & goto CheckOpts if "%1"=="-d" (set suffix=_d) & shift & goto CheckOpts if "%1"=="-x64" (set prefix=%pcbuild%amd64\) & shift & goto CheckOpts +if NOT "%1"=="" (set regrtestargs=%regrtestargs% %1) & shift & goto CheckOpts -set exe=%prefix%\python%suffix% -set cmd="%exe%" %dashO% -Wd -3 -E -tt "%pcbuild%..\Lib\test\regrtest.py" %1 %2 %3 %4 %5 %6 %7 %8 %9 +set exe=%prefix%python%suffix% +set cmd="%exe%" %dashO% -Wd -3 -E -tt "%pcbuild%..\Lib\test\regrtest.py" %regrtestargs% if defined qmode goto Qmode echo Deleting .pyc/.pyo files ... diff --git a/Tools/buildbot/test.bat b/Tools/buildbot/test.bat --- a/Tools/buildbot/test.bat +++ b/Tools/buildbot/test.bat @@ -1,15 +1,19 @@ - at rem Used by the buildbot "test" step. - at setlocal + at echo off +rem Used by the buildbot "test" step. +setlocal - at set here=%~dp0 - at set rt_opts=-q -d +set here=%~dp0 +set rt_opts=-q -d +set regrtest_args= :CheckOpts - at if '%1'=='-x64' (set rt_opts=%rt_opts% %1) & shift & goto CheckOpts - at if '%1'=='-d' (set rt_opts=%rt_opts% %1) & shift & goto CheckOpts - at if '%1'=='-O' (set rt_opts=%rt_opts% %1) & shift & goto CheckOpts - at if '%1'=='-q' (set rt_opts=%rt_opts% %1) & shift & goto CheckOpts - at if '%1'=='+d' (set rt_opts=%rt_opts:-d=%) & shift & goto CheckOpts - at if '%1'=='+q' (set rt_opts=%rt_opts:-q=%) & shift & goto CheckOpts +if "%1"=="-x64" (set rt_opts=%rt_opts% %1) & shift & goto CheckOpts +if "%1"=="-d" (set rt_opts=%rt_opts% %1) & shift & goto CheckOpts +if "%1"=="-O" (set rt_opts=%rt_opts% %1) & shift & goto CheckOpts +if "%1"=="-q" (set rt_opts=%rt_opts% %1) & shift & goto CheckOpts +if "%1"=="+d" (set rt_opts=%rt_opts:-d=%) & shift & goto CheckOpts +if "%1"=="+q" (set rt_opts=%rt_opts:-q=%) & shift & goto CheckOpts +if NOT "%1"=="" (set regrtest_args=%regrtest_args% %1) & shift & goto CheckOpts -call "%here%..\..\PCbuild\rt.bat" %rt_opts% -uall -rwW %1 %2 %3 %4 %5 %6 %7 %8 %9 +echo on +call "%here%..\..\PCbuild\rt.bat" %rt_opts% -uall -rwW %regrtest_args% -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 4 06:56:16 2015 From: python-checkins at python.org (zach.ware) Date: Fri, 04 Sep 2015 04:56:16 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge_with_3=2E5?= Message-ID: <20150904045616.27695.36990@psf.io> https://hg.python.org/cpython/rev/30723278b74f changeset: 97645:30723278b74f parent: 97640:c6e0c29913ec parent: 97643:10600293b466 user: Zachary Ware date: Thu Sep 03 23:51:07 2015 -0500 summary: Merge with 3.5 files: Misc/NEWS | 3 +++ PCbuild/rt.bat | 4 +++- Tools/buildbot/test.bat | 26 +++++++++++++++----------- 3 files changed, 21 insertions(+), 12 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -80,6 +80,9 @@ Tests ----- +- PCbuild\rt.bat now accepts an unlimited number of arguments to pass along + to regrtest.py. Previously there was a limit of 9. + What's New in Python 3.5.1 ========================== diff --git a/PCbuild/rt.bat b/PCbuild/rt.bat --- a/PCbuild/rt.bat +++ b/PCbuild/rt.bat @@ -32,15 +32,17 @@ set suffix= set qmode= set dashO= +set regrtestargs= :CheckOpts if "%1"=="-O" (set dashO=-O) & shift & goto CheckOpts if "%1"=="-q" (set qmode=yes) & shift & goto CheckOpts if "%1"=="-d" (set suffix=_d) & shift & goto CheckOpts if "%1"=="-x64" (set prefix=%pcbuild%amd64\) & shift & goto CheckOpts +if NOT "%1"=="" (set regrtestargs=%regrtestargs% %1) & shift & goto CheckOpts set exe=%prefix%python%suffix%.exe -set cmd="%exe%" %dashO% -Wd -E -bb "%pcbuild%..\lib\test\regrtest.py" %1 %2 %3 %4 %5 %6 %7 %8 %9 +set cmd="%exe%" %dashO% -Wd -E -bb "%pcbuild%..\lib\test\regrtest.py" %regrtestargs% if defined qmode goto Qmode echo Deleting .pyc/.pyo files ... diff --git a/Tools/buildbot/test.bat b/Tools/buildbot/test.bat --- a/Tools/buildbot/test.bat +++ b/Tools/buildbot/test.bat @@ -1,15 +1,19 @@ - at rem Used by the buildbot "test" step. - at setlocal + at echo off +rem Used by the buildbot "test" step. +setlocal - at set here=%~dp0 - at set rt_opts=-q -d +set here=%~dp0 +set rt_opts=-q -d +set regrtest_args= :CheckOpts - at if '%1'=='-x64' (set rt_opts=%rt_opts% %1) & shift & goto CheckOpts - at if '%1'=='-d' (set rt_opts=%rt_opts% %1) & shift & goto CheckOpts - at if '%1'=='-O' (set rt_opts=%rt_opts% %1) & shift & goto CheckOpts - at if '%1'=='-q' (set rt_opts=%rt_opts% %1) & shift & goto CheckOpts - at if '%1'=='+d' (set rt_opts=%rt_opts:-d=%) & shift & goto CheckOpts - at if '%1'=='+q' (set rt_opts=%rt_opts:-q=%) & shift & goto CheckOpts +if "%1"=="-x64" (set rt_opts=%rt_opts% %1) & shift & goto CheckOpts +if "%1"=="-d" (set rt_opts=%rt_opts% %1) & shift & goto CheckOpts +if "%1"=="-O" (set rt_opts=%rt_opts% %1) & shift & goto CheckOpts +if "%1"=="-q" (set rt_opts=%rt_opts% %1) & shift & goto CheckOpts +if "%1"=="+d" (set rt_opts=%rt_opts:-d=%) & shift & goto CheckOpts +if "%1"=="+q" (set rt_opts=%rt_opts:-q=%) & shift & goto CheckOpts +if NOT "%1"=="" (set regrtest_args=%regrtest_args% %1) & shift & goto CheckOpts -call "%here%..\..\PCbuild\rt.bat" %rt_opts% -uall -rwW -n --timeout=3600 %1 %2 %3 %4 %5 %6 %7 %8 %9 +echo on +call "%here%..\..\PCbuild\rt.bat" %rt_opts% -uall -rwW -n --timeout=3600 %regrtest_args% -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 4 06:56:16 2015 From: python-checkins at python.org (zach.ware) Date: Fri, 04 Sep 2015 04:56:16 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Closes_=2324986=3A_Merge_with_3=2E5?= Message-ID: <20150904045616.17983.43485@psf.io> https://hg.python.org/cpython/rev/50d38fa13282 changeset: 97646:50d38fa13282 parent: 97645:30723278b74f parent: 97644:301c36746e42 user: Zachary Ware date: Thu Sep 03 23:53:27 2015 -0500 summary: Closes #24986: Merge with 3.5 files: Misc/NEWS | 6 ++++++ PCbuild/build.bat | 24 ++++++++++++++++++++++-- PCbuild/pcbuild.proj | 14 ++++++++++---- 3 files changed, 38 insertions(+), 6 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -83,6 +83,12 @@ - PCbuild\rt.bat now accepts an unlimited number of arguments to pass along to regrtest.py. Previously there was a limit of 9. +Build +----- + +- Issue #24986: It is now possible to build Python on Windows without errors + when external libraries are not available. + What's New in Python 3.5.1 ========================== diff --git a/PCbuild/build.bat b/PCbuild/build.bat --- a/PCbuild/build.bat +++ b/PCbuild/build.bat @@ -18,12 +18,19 @@ echo. -r Target Rebuild instead of Build echo. -d Set the configuration to Debug echo. -e Build external libraries fetched by get_externals.bat +echo. Extension modules that depend on external libraries will not attempt +echo. to build if this flag is not present echo. -m Enable parallel build (enabled by default) echo. -M Disable parallel build echo. -v Increased output messages echo. -k Attempt to kill any running Pythons before building (usually done echo. automatically by the pythoncore project) echo. +echo.Available flags to avoid building certain modules. +echo.These flags have no effect if '-e' is not given: +echo. --no-ssl Do not attempt to build _ssl +echo. --no-tkinter Do not attempt to build Tkinter +echo. echo.Available arguments: echo. -c Release ^| Debug ^| PGInstrument ^| PGUpdate echo. Set the configuration (default: Release) @@ -51,12 +58,21 @@ if "%~1"=="-r" (set target=Rebuild) & shift & goto CheckOpts if "%~1"=="-t" (set target=%2) & shift & shift & goto CheckOpts if "%~1"=="-d" (set conf=Debug) & shift & goto CheckOpts -if "%~1"=="-e" call "%dir%get_externals.bat" & shift & goto CheckOpts if "%~1"=="-m" (set parallel=/m) & shift & goto CheckOpts if "%~1"=="-M" (set parallel=) & shift & goto CheckOpts if "%~1"=="-v" (set verbose=/v:n) & shift & goto CheckOpts if "%~1"=="-k" (set kill=true) & shift & goto CheckOpts if "%~1"=="-V" shift & goto Version +rem These use the actual property names used by MSBuild. We could just let +rem them in through the environment, but we specify them on the command line +rem anyway for visibility so set defaults after this +if "%~1"=="-e" (set IncludeExternals=true) & call "%dir%get_externals.bat" & shift & goto CheckOpts +if "%~1"=="--no-ssl" (set IncludeSSL=false) & shift & goto CheckOpts +if "%~1"=="--no-tkinter" (set IncludeTkinter=false) & shift & goto CheckOpts + +if "%IncludeExternals%"=="" set IncludeExternals=false +if "%IncludeSSL%"=="" set IncludeSSL=true +if "%IncludeTkinter%"=="" set IncludeTkinter=true if "%platf%"=="x64" (set vs_platf=x86_amd64) @@ -71,7 +87,11 @@ rem Passing %1-9 is not the preferred option, but argument parsing in rem batch is, shall we say, "lackluster" echo on -msbuild "%dir%pcbuild.proj" /t:%target% %parallel% %verbose% /p:Configuration=%conf% /p:Platform=%platf% %1 %2 %3 %4 %5 %6 %7 %8 %9 +msbuild "%dir%pcbuild.proj" /t:%target% %parallel% %verbose%^ + /p:Configuration=%conf% /p:Platform=%platf%^ + /p:IncludeExternals=%IncludeExternals%^ + /p:IncludeSSL=%IncludeSSL% /p:IncludeTkinter=%IncludeTkinter%^ + %1 %2 %3 %4 %5 %6 %7 %8 %9 @goto :eof diff --git a/PCbuild/pcbuild.proj b/PCbuild/pcbuild.proj --- a/PCbuild/pcbuild.proj +++ b/PCbuild/pcbuild.proj @@ -5,8 +5,10 @@ Win32 Release true + true true true + true @@ -25,7 +27,7 @@ @@ -40,10 +42,14 @@ - + + + - - + + + + -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 4 07:37:04 2015 From: python-checkins at python.org (serhiy.storchaka) Date: Fri, 04 Sep 2015 05:37:04 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogTWVyZ2UgMy41LjA=?= Message-ID: <20150904053704.11993.85780@psf.io> https://hg.python.org/cpython/rev/9d01d914bd3f changeset: 97649:9d01d914bd3f parent: 97637:215800fb955d parent: 97648:566e02049429 user: Serhiy Storchaka date: Fri Sep 04 08:29:00 2015 +0300 summary: Merge 3.5.0 files: Misc/NEWS | 9 ++++++--- 1 files changed, 6 insertions(+), 3 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -145,6 +145,12 @@ Library ------- +- Issue #24989: Fixed buffer overread in BytesIO.readline() if a position is + set beyond size. Based on patch by John Leitch. + +- Issue #24913: Fix overrun error in deque.index(). + Found by John Leitch and Bryce Darling. + What's New in Python 3.5.0 release candidate 2? =============================================== @@ -167,9 +173,6 @@ Library ------- -- Issue #24989: Fixed buffer overread in BytesIO.readline() if a position is - set beyond size. Based on patch by John Leitch. - - Issue #24847: Removes vcruntime140.dll dependency from Tcl/Tk. - Issue #24839: platform._syscmd_ver raises DeprecationWarning -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 4 07:37:04 2015 From: python-checkins at python.org (serhiy.storchaka) Date: Fri, 04 Sep 2015 05:37:04 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI0OTg5?= =?utf-8?q?=3A_Fixed_buffer_overread_in_BytesIO=2Ereadline=28=29_if_a_posi?= =?utf-8?q?tion_is?= Message-ID: <20150904053704.14861.40186@psf.io> https://hg.python.org/cpython/rev/2b6ce7e9595c changeset: 97647:2b6ce7e9595c branch: 3.5 parent: 97638:9f8c59e61594 user: Serhiy Storchaka date: Fri Sep 04 07:48:19 2015 +0300 summary: Issue #24989: Fixed buffer overread in BytesIO.readline() if a position is set beyond size. Based on patch by John Leitch. files: Lib/test/test_memoryio.py | 13 +++++++++++++ Misc/NEWS | 3 +++ Modules/_io/bytesio.c | 6 +++++- 3 files changed, 21 insertions(+), 1 deletions(-) diff --git a/Lib/test/test_memoryio.py b/Lib/test/test_memoryio.py --- a/Lib/test/test_memoryio.py +++ b/Lib/test/test_memoryio.py @@ -166,6 +166,10 @@ memio.seek(0) self.assertEqual(memio.read(None), buf) self.assertRaises(TypeError, memio.read, '') + memio.seek(len(buf) + 1) + self.assertEqual(memio.read(1), self.EOF) + memio.seek(len(buf) + 1) + self.assertEqual(memio.read(), self.EOF) memio.close() self.assertRaises(ValueError, memio.read) @@ -185,6 +189,9 @@ self.assertEqual(memio.readline(-1), buf) memio.seek(0) self.assertEqual(memio.readline(0), self.EOF) + # Issue #24989: Buffer overread + memio.seek(len(buf) * 2 + 1) + self.assertEqual(memio.readline(), self.EOF) buf = self.buftype("1234567890\n") memio = self.ioclass((buf * 3)[:-1]) @@ -217,6 +224,9 @@ memio.seek(0) self.assertEqual(memio.readlines(None), [buf] * 10) self.assertRaises(TypeError, memio.readlines, '') + # Issue #24989: Buffer overread + memio.seek(len(buf) * 10 + 1) + self.assertEqual(memio.readlines(), []) memio.close() self.assertRaises(ValueError, memio.readlines) @@ -238,6 +248,9 @@ self.assertEqual(line, buf) i += 1 self.assertEqual(i, 10) + # Issue #24989: Buffer overread + memio.seek(len(buf) * 10 + 1) + self.assertEqual(list(memio), []) memio = self.ioclass(buf * 2) memio.close() self.assertRaises(ValueError, memio.__next__) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -15,6 +15,9 @@ Library ------- +- Issue #24989: Fixed buffer overread in BytesIO.readline() if a position is + set beyond size. Based on patch by John Leitch. + - Issue #24913: Fix overrun error in deque.index(). Found by John Leitch and Bryce Darling. diff --git a/Modules/_io/bytesio.c b/Modules/_io/bytesio.c --- a/Modules/_io/bytesio.c +++ b/Modules/_io/bytesio.c @@ -57,14 +57,18 @@ Py_ssize_t maxlen; assert(self->buf != NULL); + assert(self->pos >= 0); + + if (self->pos >= self->string_size) + return 0; /* Move to the end of the line, up to the end of the string, s. */ - start = PyBytes_AS_STRING(self->buf) + self->pos; maxlen = self->string_size - self->pos; if (len < 0 || len > maxlen) len = maxlen; if (len) { + start = PyBytes_AS_STRING(self->buf) + self->pos; n = memchr(start, '\n', len); if (n) /* Get the length from the current position to the end of -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 4 07:37:05 2015 From: python-checkins at python.org (serhiy.storchaka) Date: Fri, 04 Sep 2015 05:37:05 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogTWVyZ2UgMy41?= Message-ID: <20150904053705.14883.45347@psf.io> https://hg.python.org/cpython/rev/96c79e2a8cef changeset: 97651:96c79e2a8cef parent: 97649:9d01d914bd3f parent: 97650:d05ade34d454 user: Serhiy Storchaka date: Fri Sep 04 08:34:57 2015 +0300 summary: Merge 3.5 files: Misc/NEWS | 15 +++++++++++++++ PCbuild/build.bat | 24 ++++++++++++++++++++++-- PCbuild/pcbuild.proj | 14 ++++++++++---- PCbuild/rt.bat | 4 +++- Tools/buildbot/test.bat | 26 +++++++++++++++----------- 5 files changed, 65 insertions(+), 18 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -131,6 +131,18 @@ - Issue #22812: Fix unittest discovery examples. Patch from Pam McA'Nulty. +Tests +----- + +- PCbuild\rt.bat now accepts an unlimited number of arguments to pass along + to regrtest.py. Previously there was a limit of 9. + +Build +----- + +- Issue #24986: It is now possible to build Python on Windows without errors + when external libraries are not available. + What's New in Python 3.5.0 release candidate 3? =============================================== @@ -145,6 +157,9 @@ Library ------- +- Issue #24913: Fix overrun error in deque.index(). + Found by John Leitch and Bryce Darling. + - Issue #24989: Fixed buffer overread in BytesIO.readline() if a position is set beyond size. Based on patch by John Leitch. diff --git a/PCbuild/build.bat b/PCbuild/build.bat --- a/PCbuild/build.bat +++ b/PCbuild/build.bat @@ -18,12 +18,19 @@ echo. -r Target Rebuild instead of Build echo. -d Set the configuration to Debug echo. -e Build external libraries fetched by get_externals.bat +echo. Extension modules that depend on external libraries will not attempt +echo. to build if this flag is not present echo. -m Enable parallel build (enabled by default) echo. -M Disable parallel build echo. -v Increased output messages echo. -k Attempt to kill any running Pythons before building (usually done echo. automatically by the pythoncore project) echo. +echo.Available flags to avoid building certain modules. +echo.These flags have no effect if '-e' is not given: +echo. --no-ssl Do not attempt to build _ssl +echo. --no-tkinter Do not attempt to build Tkinter +echo. echo.Available arguments: echo. -c Release ^| Debug ^| PGInstrument ^| PGUpdate echo. Set the configuration (default: Release) @@ -51,12 +58,21 @@ if "%~1"=="-r" (set target=Rebuild) & shift & goto CheckOpts if "%~1"=="-t" (set target=%2) & shift & shift & goto CheckOpts if "%~1"=="-d" (set conf=Debug) & shift & goto CheckOpts -if "%~1"=="-e" call "%dir%get_externals.bat" & shift & goto CheckOpts if "%~1"=="-m" (set parallel=/m) & shift & goto CheckOpts if "%~1"=="-M" (set parallel=) & shift & goto CheckOpts if "%~1"=="-v" (set verbose=/v:n) & shift & goto CheckOpts if "%~1"=="-k" (set kill=true) & shift & goto CheckOpts if "%~1"=="-V" shift & goto Version +rem These use the actual property names used by MSBuild. We could just let +rem them in through the environment, but we specify them on the command line +rem anyway for visibility so set defaults after this +if "%~1"=="-e" (set IncludeExternals=true) & call "%dir%get_externals.bat" & shift & goto CheckOpts +if "%~1"=="--no-ssl" (set IncludeSSL=false) & shift & goto CheckOpts +if "%~1"=="--no-tkinter" (set IncludeTkinter=false) & shift & goto CheckOpts + +if "%IncludeExternals%"=="" set IncludeExternals=false +if "%IncludeSSL%"=="" set IncludeSSL=true +if "%IncludeTkinter%"=="" set IncludeTkinter=true if "%platf%"=="x64" (set vs_platf=x86_amd64) @@ -71,7 +87,11 @@ rem Passing %1-9 is not the preferred option, but argument parsing in rem batch is, shall we say, "lackluster" echo on -msbuild "%dir%pcbuild.proj" /t:%target% %parallel% %verbose% /p:Configuration=%conf% /p:Platform=%platf% %1 %2 %3 %4 %5 %6 %7 %8 %9 +msbuild "%dir%pcbuild.proj" /t:%target% %parallel% %verbose%^ + /p:Configuration=%conf% /p:Platform=%platf%^ + /p:IncludeExternals=%IncludeExternals%^ + /p:IncludeSSL=%IncludeSSL% /p:IncludeTkinter=%IncludeTkinter%^ + %1 %2 %3 %4 %5 %6 %7 %8 %9 @goto :eof diff --git a/PCbuild/pcbuild.proj b/PCbuild/pcbuild.proj --- a/PCbuild/pcbuild.proj +++ b/PCbuild/pcbuild.proj @@ -5,8 +5,10 @@ Win32 Release true + true true true + true @@ -25,7 +27,7 @@ @@ -40,10 +42,14 @@ - + + + - - + + + + diff --git a/PCbuild/rt.bat b/PCbuild/rt.bat --- a/PCbuild/rt.bat +++ b/PCbuild/rt.bat @@ -32,15 +32,17 @@ set suffix= set qmode= set dashO= +set regrtestargs= :CheckOpts if "%1"=="-O" (set dashO=-O) & shift & goto CheckOpts if "%1"=="-q" (set qmode=yes) & shift & goto CheckOpts if "%1"=="-d" (set suffix=_d) & shift & goto CheckOpts if "%1"=="-x64" (set prefix=%pcbuild%amd64\) & shift & goto CheckOpts +if NOT "%1"=="" (set regrtestargs=%regrtestargs% %1) & shift & goto CheckOpts set exe=%prefix%python%suffix%.exe -set cmd="%exe%" %dashO% -Wd -E -bb "%pcbuild%..\lib\test\regrtest.py" %1 %2 %3 %4 %5 %6 %7 %8 %9 +set cmd="%exe%" %dashO% -Wd -E -bb "%pcbuild%..\lib\test\regrtest.py" %regrtestargs% if defined qmode goto Qmode echo Deleting .pyc/.pyo files ... diff --git a/Tools/buildbot/test.bat b/Tools/buildbot/test.bat --- a/Tools/buildbot/test.bat +++ b/Tools/buildbot/test.bat @@ -1,15 +1,19 @@ - at rem Used by the buildbot "test" step. - at setlocal + at echo off +rem Used by the buildbot "test" step. +setlocal - at set here=%~dp0 - at set rt_opts=-q -d +set here=%~dp0 +set rt_opts=-q -d +set regrtest_args= :CheckOpts - at if '%1'=='-x64' (set rt_opts=%rt_opts% %1) & shift & goto CheckOpts - at if '%1'=='-d' (set rt_opts=%rt_opts% %1) & shift & goto CheckOpts - at if '%1'=='-O' (set rt_opts=%rt_opts% %1) & shift & goto CheckOpts - at if '%1'=='-q' (set rt_opts=%rt_opts% %1) & shift & goto CheckOpts - at if '%1'=='+d' (set rt_opts=%rt_opts:-d=%) & shift & goto CheckOpts - at if '%1'=='+q' (set rt_opts=%rt_opts:-q=%) & shift & goto CheckOpts +if "%1"=="-x64" (set rt_opts=%rt_opts% %1) & shift & goto CheckOpts +if "%1"=="-d" (set rt_opts=%rt_opts% %1) & shift & goto CheckOpts +if "%1"=="-O" (set rt_opts=%rt_opts% %1) & shift & goto CheckOpts +if "%1"=="-q" (set rt_opts=%rt_opts% %1) & shift & goto CheckOpts +if "%1"=="+d" (set rt_opts=%rt_opts:-d=%) & shift & goto CheckOpts +if "%1"=="+q" (set rt_opts=%rt_opts:-q=%) & shift & goto CheckOpts +if NOT "%1"=="" (set regrtest_args=%regrtest_args% %1) & shift & goto CheckOpts -call "%here%..\..\PCbuild\rt.bat" %rt_opts% -uall -rwW -n --timeout=3600 %1 %2 %3 %4 %5 %6 %7 %8 %9 +echo on +call "%here%..\..\PCbuild\rt.bat" %rt_opts% -uall -rwW -n --timeout=3600 %regrtest_args% -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 4 07:37:06 2015 From: python-checkins at python.org (serhiy.storchaka) Date: Fri, 04 Sep 2015 05:37:06 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_default_-=3E_default?= =?utf-8?q?=29=3A_Merge_heads?= Message-ID: <20150904053705.114691.19505@psf.io> https://hg.python.org/cpython/rev/bd165adf086a changeset: 97652:bd165adf086a parent: 97651:96c79e2a8cef parent: 97646:50d38fa13282 user: Serhiy Storchaka date: Fri Sep 04 08:36:05 2015 +0300 summary: Merge heads files: Misc/NEWS | 12 ++++++++++++ 1 files changed, 12 insertions(+), 0 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -80,6 +80,15 @@ Tests ----- +- PCbuild\rt.bat now accepts an unlimited number of arguments to pass along + to regrtest.py. Previously there was a limit of 9. + +Build +----- + +- Issue #24986: It is now possible to build Python on Windows without errors + when external libraries are not available. + What's New in Python 3.5.1 ========================== @@ -160,6 +169,9 @@ - Issue #24913: Fix overrun error in deque.index(). Found by John Leitch and Bryce Darling. +- Issue #24913: Fix overrun error in deque.index(). + Found by John Leitch and Bryce Darling. + - Issue #24989: Fixed buffer overread in BytesIO.readline() if a position is set beyond size. Based on patch by John Leitch. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 4 07:37:06 2015 From: python-checkins at python.org (serhiy.storchaka) Date: Fri, 04 Sep 2015 05:37:06 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy41IC0+IDMuNSk6?= =?utf-8?q?_Merge_3=2E5=2E0?= Message-ID: <20150904053704.66874.41246@psf.io> https://hg.python.org/cpython/rev/566e02049429 changeset: 97648:566e02049429 branch: 3.5 parent: 97636:a5858c30db7c parent: 97647:2b6ce7e9595c user: Serhiy Storchaka date: Fri Sep 04 08:27:39 2015 +0300 summary: Merge 3.5.0 files: Misc/NEWS | 9 ++++++--- 1 files changed, 6 insertions(+), 3 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -70,6 +70,12 @@ Library ------- +- Issue #24989: Fixed buffer overread in BytesIO.readline() if a position is + set beyond size. Based on patch by John Leitch. + +- Issue #24913: Fix overrun error in deque.index(). + Found by John Leitch and Bryce Darling. + What's New in Python 3.5.0 release candidate 2? =============================================== @@ -92,9 +98,6 @@ Library ------- -- Issue #24989: Fixed buffer overread in BytesIO.readline() if a position is - set beyond size. Based on patch by John Leitch. - - Issue #24847: Removes vcruntime140.dll dependency from Tcl/Tk. - Issue #24839: platform._syscmd_ver raises DeprecationWarning -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 4 07:37:06 2015 From: python-checkins at python.org (serhiy.storchaka) Date: Fri, 04 Sep 2015 05:37:06 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy41IC0+IDMuNSk6?= =?utf-8?q?_Merge_heads?= Message-ID: <20150904053705.68879.47140@psf.io> https://hg.python.org/cpython/rev/d05ade34d454 changeset: 97650:d05ade34d454 branch: 3.5 parent: 97648:566e02049429 parent: 97644:301c36746e42 user: Serhiy Storchaka date: Fri Sep 04 08:34:01 2015 +0300 summary: Merge heads files: Misc/NEWS | 15 +++++++++++++++ PCbuild/build.bat | 24 ++++++++++++++++++++++-- PCbuild/pcbuild.proj | 14 ++++++++++---- PCbuild/rt.bat | 4 +++- Tools/buildbot/test.bat | 26 +++++++++++++++----------- 5 files changed, 65 insertions(+), 18 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -56,6 +56,18 @@ - Issue #22812: Fix unittest discovery examples. Patch from Pam McA'Nulty. +Tests +----- + +- PCbuild\rt.bat now accepts an unlimited number of arguments to pass along + to regrtest.py. Previously there was a limit of 9. + +Build +----- + +- Issue #24986: It is now possible to build Python on Windows without errors + when external libraries are not available. + What's New in Python 3.5.0 release candidate 3? =============================================== @@ -70,6 +82,9 @@ Library ------- +- Issue #24913: Fix overrun error in deque.index(). + Found by John Leitch and Bryce Darling. + - Issue #24989: Fixed buffer overread in BytesIO.readline() if a position is set beyond size. Based on patch by John Leitch. diff --git a/PCbuild/build.bat b/PCbuild/build.bat --- a/PCbuild/build.bat +++ b/PCbuild/build.bat @@ -18,12 +18,19 @@ echo. -r Target Rebuild instead of Build echo. -d Set the configuration to Debug echo. -e Build external libraries fetched by get_externals.bat +echo. Extension modules that depend on external libraries will not attempt +echo. to build if this flag is not present echo. -m Enable parallel build (enabled by default) echo. -M Disable parallel build echo. -v Increased output messages echo. -k Attempt to kill any running Pythons before building (usually done echo. automatically by the pythoncore project) echo. +echo.Available flags to avoid building certain modules. +echo.These flags have no effect if '-e' is not given: +echo. --no-ssl Do not attempt to build _ssl +echo. --no-tkinter Do not attempt to build Tkinter +echo. echo.Available arguments: echo. -c Release ^| Debug ^| PGInstrument ^| PGUpdate echo. Set the configuration (default: Release) @@ -51,12 +58,21 @@ if "%~1"=="-r" (set target=Rebuild) & shift & goto CheckOpts if "%~1"=="-t" (set target=%2) & shift & shift & goto CheckOpts if "%~1"=="-d" (set conf=Debug) & shift & goto CheckOpts -if "%~1"=="-e" call "%dir%get_externals.bat" & shift & goto CheckOpts if "%~1"=="-m" (set parallel=/m) & shift & goto CheckOpts if "%~1"=="-M" (set parallel=) & shift & goto CheckOpts if "%~1"=="-v" (set verbose=/v:n) & shift & goto CheckOpts if "%~1"=="-k" (set kill=true) & shift & goto CheckOpts if "%~1"=="-V" shift & goto Version +rem These use the actual property names used by MSBuild. We could just let +rem them in through the environment, but we specify them on the command line +rem anyway for visibility so set defaults after this +if "%~1"=="-e" (set IncludeExternals=true) & call "%dir%get_externals.bat" & shift & goto CheckOpts +if "%~1"=="--no-ssl" (set IncludeSSL=false) & shift & goto CheckOpts +if "%~1"=="--no-tkinter" (set IncludeTkinter=false) & shift & goto CheckOpts + +if "%IncludeExternals%"=="" set IncludeExternals=false +if "%IncludeSSL%"=="" set IncludeSSL=true +if "%IncludeTkinter%"=="" set IncludeTkinter=true if "%platf%"=="x64" (set vs_platf=x86_amd64) @@ -71,7 +87,11 @@ rem Passing %1-9 is not the preferred option, but argument parsing in rem batch is, shall we say, "lackluster" echo on -msbuild "%dir%pcbuild.proj" /t:%target% %parallel% %verbose% /p:Configuration=%conf% /p:Platform=%platf% %1 %2 %3 %4 %5 %6 %7 %8 %9 +msbuild "%dir%pcbuild.proj" /t:%target% %parallel% %verbose%^ + /p:Configuration=%conf% /p:Platform=%platf%^ + /p:IncludeExternals=%IncludeExternals%^ + /p:IncludeSSL=%IncludeSSL% /p:IncludeTkinter=%IncludeTkinter%^ + %1 %2 %3 %4 %5 %6 %7 %8 %9 @goto :eof diff --git a/PCbuild/pcbuild.proj b/PCbuild/pcbuild.proj --- a/PCbuild/pcbuild.proj +++ b/PCbuild/pcbuild.proj @@ -5,8 +5,10 @@ Win32 Release true + true true true + true @@ -25,7 +27,7 @@ @@ -40,10 +42,14 @@ - + + + - - + + + + diff --git a/PCbuild/rt.bat b/PCbuild/rt.bat --- a/PCbuild/rt.bat +++ b/PCbuild/rt.bat @@ -32,15 +32,17 @@ set suffix= set qmode= set dashO= +set regrtestargs= :CheckOpts if "%1"=="-O" (set dashO=-O) & shift & goto CheckOpts if "%1"=="-q" (set qmode=yes) & shift & goto CheckOpts if "%1"=="-d" (set suffix=_d) & shift & goto CheckOpts if "%1"=="-x64" (set prefix=%pcbuild%amd64\) & shift & goto CheckOpts +if NOT "%1"=="" (set regrtestargs=%regrtestargs% %1) & shift & goto CheckOpts set exe=%prefix%python%suffix%.exe -set cmd="%exe%" %dashO% -Wd -E -bb "%pcbuild%..\lib\test\regrtest.py" %1 %2 %3 %4 %5 %6 %7 %8 %9 +set cmd="%exe%" %dashO% -Wd -E -bb "%pcbuild%..\lib\test\regrtest.py" %regrtestargs% if defined qmode goto Qmode echo Deleting .pyc/.pyo files ... diff --git a/Tools/buildbot/test.bat b/Tools/buildbot/test.bat --- a/Tools/buildbot/test.bat +++ b/Tools/buildbot/test.bat @@ -1,15 +1,19 @@ - at rem Used by the buildbot "test" step. - at setlocal + at echo off +rem Used by the buildbot "test" step. +setlocal - at set here=%~dp0 - at set rt_opts=-q -d +set here=%~dp0 +set rt_opts=-q -d +set regrtest_args= :CheckOpts - at if '%1'=='-x64' (set rt_opts=%rt_opts% %1) & shift & goto CheckOpts - at if '%1'=='-d' (set rt_opts=%rt_opts% %1) & shift & goto CheckOpts - at if '%1'=='-O' (set rt_opts=%rt_opts% %1) & shift & goto CheckOpts - at if '%1'=='-q' (set rt_opts=%rt_opts% %1) & shift & goto CheckOpts - at if '%1'=='+d' (set rt_opts=%rt_opts:-d=%) & shift & goto CheckOpts - at if '%1'=='+q' (set rt_opts=%rt_opts:-q=%) & shift & goto CheckOpts +if "%1"=="-x64" (set rt_opts=%rt_opts% %1) & shift & goto CheckOpts +if "%1"=="-d" (set rt_opts=%rt_opts% %1) & shift & goto CheckOpts +if "%1"=="-O" (set rt_opts=%rt_opts% %1) & shift & goto CheckOpts +if "%1"=="-q" (set rt_opts=%rt_opts% %1) & shift & goto CheckOpts +if "%1"=="+d" (set rt_opts=%rt_opts:-d=%) & shift & goto CheckOpts +if "%1"=="+q" (set rt_opts=%rt_opts:-q=%) & shift & goto CheckOpts +if NOT "%1"=="" (set regrtest_args=%regrtest_args% %1) & shift & goto CheckOpts -call "%here%..\..\PCbuild\rt.bat" %rt_opts% -uall -rwW -n --timeout=3600 %1 %2 %3 %4 %5 %6 %7 %8 %9 +echo on +call "%here%..\..\PCbuild\rt.bat" %rt_opts% -uall -rwW -n --timeout=3600 %regrtest_args% -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 4 07:40:31 2015 From: python-checkins at python.org (serhiy.storchaka) Date: Fri, 04 Sep 2015 05:40:31 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_Fixed_merge_er?= =?utf-8?q?ror=2E?= Message-ID: <20150904054031.27693.3920@psf.io> https://hg.python.org/cpython/rev/cb28ffefd730 changeset: 97653:cb28ffefd730 branch: 3.5 parent: 97650:d05ade34d454 user: Serhiy Storchaka date: Fri Sep 04 08:38:45 2015 +0300 summary: Fixed merge error. files: Misc/NEWS | 3 --- 1 files changed, 0 insertions(+), 3 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -82,9 +82,6 @@ Library ------- -- Issue #24913: Fix overrun error in deque.index(). - Found by John Leitch and Bryce Darling. - - Issue #24989: Fixed buffer overread in BytesIO.readline() if a position is set beyond size. Based on patch by John Leitch. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 4 07:40:31 2015 From: python-checkins at python.org (serhiy.storchaka) Date: Fri, 04 Sep 2015 05:40:31 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Fixed_merge_error=2E?= Message-ID: <20150904054031.114820.4631@psf.io> https://hg.python.org/cpython/rev/b201e3e044b6 changeset: 97654:b201e3e044b6 parent: 97652:bd165adf086a parent: 97653:cb28ffefd730 user: Serhiy Storchaka date: Fri Sep 04 08:39:33 2015 +0300 summary: Fixed merge error. files: Misc/NEWS | 6 ------ 1 files changed, 0 insertions(+), 6 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -166,12 +166,6 @@ Library ------- -- Issue #24913: Fix overrun error in deque.index(). - Found by John Leitch and Bryce Darling. - -- Issue #24913: Fix overrun error in deque.index(). - Found by John Leitch and Bryce Darling. - - Issue #24989: Fixed buffer overread in BytesIO.readline() if a position is set beyond size. Based on patch by John Leitch. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 4 08:14:45 2015 From: python-checkins at python.org (zach.ware) Date: Fri, 04 Sep 2015 06:14:45 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI0OTg2?= =?utf-8?q?=3A_Save_some_bandwidth_from_svn=2Epython=2Eorg?= Message-ID: <20150904061444.15730.50800@psf.io> https://hg.python.org/cpython/rev/4e7ce0b10eea changeset: 97656:4e7ce0b10eea branch: 3.5 parent: 97653:cb28ffefd730 user: Zachary Ware date: Fri Sep 04 01:10:23 2015 -0500 summary: Issue #24986: Save some bandwidth from svn.python.org Don't download sources that won't be used. files: PCbuild/build.bat | 4 +++- PCbuild/get_externals.bat | 21 +++++++++++---------- 2 files changed, 14 insertions(+), 11 deletions(-) diff --git a/PCbuild/build.bat b/PCbuild/build.bat --- a/PCbuild/build.bat +++ b/PCbuild/build.bat @@ -66,7 +66,7 @@ rem These use the actual property names used by MSBuild. We could just let rem them in through the environment, but we specify them on the command line rem anyway for visibility so set defaults after this -if "%~1"=="-e" (set IncludeExternals=true) & call "%dir%get_externals.bat" & shift & goto CheckOpts +if "%~1"=="-e" (set IncludeExternals=true) & shift & goto CheckOpts if "%~1"=="--no-ssl" (set IncludeSSL=false) & shift & goto CheckOpts if "%~1"=="--no-tkinter" (set IncludeTkinter=false) & shift & goto CheckOpts @@ -74,6 +74,8 @@ if "%IncludeSSL%"=="" set IncludeSSL=true if "%IncludeTkinter%"=="" set IncludeTkinter=true +if "%IncludeExternals%"=="true" call "%dir%get_externals.bat" + if "%platf%"=="x64" (set vs_platf=x86_amd64) rem Setup the environment diff --git a/PCbuild/get_externals.bat b/PCbuild/get_externals.bat --- a/PCbuild/get_externals.bat +++ b/PCbuild/get_externals.bat @@ -51,16 +51,17 @@ echo.Fetching external libraries... -for %%e in ( - bzip2-1.0.6 - nasm-2.11.06 - openssl-1.0.2d - tcl-core-8.6.4.2 - tk-8.6.4.2 - tix-8.4.3.6 - sqlite-3.8.11.0 - xz-5.0.5 - ) do ( +set libraries= +set libraries=%libraries% bzip2-1.0.6 +if NOT "%IncludeSSL%"=="false" set libraries=%libraries% nasm-2.11.06 +if NOT "%IncludeSSL%"=="false" set libraries=%libraries% openssl-1.0.2d +set libraries=%libraries% sqlite-3.8.11.0 +if NOT "%IncludeTkinter%"=="false" set libraries=%libraries% tcl-core-8.6.4.2 +if NOT "%IncludeTkinter%"=="false" set libraries=%libraries% tk-8.6.4.2 +if NOT "%IncludeTkinter%"=="false" set libraries=%libraries% tix-8.4.3.6 +set libraries=%libraries% xz-5.0.5 + +for %%e in (%libraries%) do ( if exist %%e ( echo.%%e already exists, skipping. ) else ( -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 4 08:14:45 2015 From: python-checkins at python.org (zach.ware) Date: Fri, 04 Sep 2015 06:14:45 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzI0OTg2?= =?utf-8?q?=3A_Save_some_bandwidth_from_svn=2Epython=2Eorg?= Message-ID: <20150904061444.14865.44992@psf.io> https://hg.python.org/cpython/rev/252d4760f28b changeset: 97655:252d4760f28b branch: 2.7 parent: 97642:2bc91f1f2b34 user: Zachary Ware date: Fri Sep 04 01:08:07 2015 -0500 summary: Issue #24986: Save some bandwidth from svn.python.org Don't download sources that won't be used. files: PCbuild/build.bat | 4 +++- PCbuild/get_externals.bat | 21 +++++++++++---------- 2 files changed, 14 insertions(+), 11 deletions(-) diff --git a/PCbuild/build.bat b/PCbuild/build.bat --- a/PCbuild/build.bat +++ b/PCbuild/build.bat @@ -65,7 +65,7 @@ rem These use the actual property names used by MSBuild. We could just let rem them in through the environment, but we specify them on the command line rem anyway for visibility so set defaults after this -if "%~1"=="-e" (set IncludeExternals=true) & call "%dir%get_externals.bat" & shift & goto CheckOpts +if "%~1"=="-e" (set IncludeExternals=true) & shift & goto CheckOpts if "%~1"=="--no-ssl" (set IncludeSSL=false) & shift & goto CheckOpts if "%~1"=="--no-tkinter" (set IncludeTkinter=false) & shift & goto CheckOpts if "%~1"=="--no-bsddb" (set IncludeBsddb=false) & shift & goto CheckOpts @@ -75,6 +75,8 @@ if "%IncludeTkinter%"=="" set IncludeTkinter=true if "%IncludeBsddb%"=="" set IncludeBsddb=true +if "%IncludeExternals%"=="true" call "%dir%get_externals.bat" + if "%platf%"=="x64" (set vs_platf=x86_amd64) rem Setup the environment diff --git a/PCbuild/get_externals.bat b/PCbuild/get_externals.bat --- a/PCbuild/get_externals.bat +++ b/PCbuild/get_externals.bat @@ -54,16 +54,17 @@ rem When updating these versions, remember to update the relevant property rem files in both this dir and PC\VS9.0 -for %%e in ( - bzip2-1.0.6 - db-4.7.25.0 - nasm-2.11.06 - openssl-1.0.2d - tcl-8.5.15.0 - tk-8.5.15.0 - tix-8.4.3.5 - sqlite-3.6.21 - ) do ( +set libraries= +set libraries=%libraries% bzip2-1.0.6 +if NOT "%IncludeBsddb%"=="false" set libraries=%libraries% db-4.7.25.0 +if NOT "%IncludeSSL%"=="false" set libraries=%libraries% nasm-2.11.06 +if NOT "%IncludeSSL%"=="false" set libraries=%libraries% openssl-1.0.2d +set libraries=%libraries% sqlite-3.6.21 +if NOT "%IncludeTkinter%"=="false" set libraries=%libraries% tcl-8.5.15.0 +if NOT "%IncludeTkinter%"=="false" set libraries=%libraries% tk-8.5.15.0 +if NOT "%IncludeTkinter%"=="false" set libraries=%libraries% tix-8.4.3.5 + +for %%e in (%libraries%) do ( if exist %%e ( echo.%%e already exists, skipping. ) else ( -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 4 08:14:46 2015 From: python-checkins at python.org (zach.ware) Date: Fri, 04 Sep 2015 06:14:46 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2324986=3A_Merge_with_3=2E5?= Message-ID: <20150904061445.66856.59499@psf.io> https://hg.python.org/cpython/rev/eca6ecc62b95 changeset: 97657:eca6ecc62b95 parent: 97654:b201e3e044b6 parent: 97656:4e7ce0b10eea user: Zachary Ware date: Fri Sep 04 01:12:44 2015 -0500 summary: Issue #24986: Merge with 3.5 files: PCbuild/build.bat | 4 +++- PCbuild/get_externals.bat | 21 +++++++++++---------- 2 files changed, 14 insertions(+), 11 deletions(-) diff --git a/PCbuild/build.bat b/PCbuild/build.bat --- a/PCbuild/build.bat +++ b/PCbuild/build.bat @@ -66,7 +66,7 @@ rem These use the actual property names used by MSBuild. We could just let rem them in through the environment, but we specify them on the command line rem anyway for visibility so set defaults after this -if "%~1"=="-e" (set IncludeExternals=true) & call "%dir%get_externals.bat" & shift & goto CheckOpts +if "%~1"=="-e" (set IncludeExternals=true) & shift & goto CheckOpts if "%~1"=="--no-ssl" (set IncludeSSL=false) & shift & goto CheckOpts if "%~1"=="--no-tkinter" (set IncludeTkinter=false) & shift & goto CheckOpts @@ -74,6 +74,8 @@ if "%IncludeSSL%"=="" set IncludeSSL=true if "%IncludeTkinter%"=="" set IncludeTkinter=true +if "%IncludeExternals%"=="true" call "%dir%get_externals.bat" + if "%platf%"=="x64" (set vs_platf=x86_amd64) rem Setup the environment diff --git a/PCbuild/get_externals.bat b/PCbuild/get_externals.bat --- a/PCbuild/get_externals.bat +++ b/PCbuild/get_externals.bat @@ -51,16 +51,17 @@ echo.Fetching external libraries... -for %%e in ( - bzip2-1.0.6 - nasm-2.11.06 - openssl-1.0.2d - tcl-core-8.6.4.2 - tk-8.6.4.2 - tix-8.4.3.6 - sqlite-3.8.11.0 - xz-5.0.5 - ) do ( +set libraries= +set libraries=%libraries% bzip2-1.0.6 +if NOT "%IncludeSSL%"=="false" set libraries=%libraries% nasm-2.11.06 +if NOT "%IncludeSSL%"=="false" set libraries=%libraries% openssl-1.0.2d +set libraries=%libraries% sqlite-3.8.11.0 +if NOT "%IncludeTkinter%"=="false" set libraries=%libraries% tcl-core-8.6.4.2 +if NOT "%IncludeTkinter%"=="false" set libraries=%libraries% tk-8.6.4.2 +if NOT "%IncludeTkinter%"=="false" set libraries=%libraries% tix-8.4.3.6 +set libraries=%libraries% xz-5.0.5 + +for %%e in (%libraries%) do ( if exist %%e ( echo.%%e already exists, skipping. ) else ( -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 4 10:19:09 2015 From: python-checkins at python.org (terry.reedy) Date: Fri, 04 Sep 2015 08:19:09 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge_with_3=2E5?= Message-ID: <20150904081909.11258.85352@psf.io> https://hg.python.org/cpython/rev/66e30dbf4cae changeset: 97661:66e30dbf4cae parent: 97657:eca6ecc62b95 parent: 97660:cb5508b8fc04 user: Terry Jan Reedy date: Fri Sep 04 04:18:11 2015 -0400 summary: Merge with 3.5 files: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 4 10:19:09 2015 From: python-checkins at python.org (terry.reedy) Date: Fri, 04 Sep 2015 08:19:09 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy41IC0+IDMuNSk6?= =?utf-8?q?_Merge_3=2E5=2E0_into_3=2E5=2E1?= Message-ID: <20150904081909.14863.87390@psf.io> https://hg.python.org/cpython/rev/cb5508b8fc04 changeset: 97660:cb5508b8fc04 branch: 3.5 parent: 97656:4e7ce0b10eea parent: 97659:07e04c34bab5 user: Terry Jan Reedy date: Fri Sep 04 04:16:42 2015 -0400 summary: Merge 3.5.0 into 3.5.1 files: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 4 10:19:09 2015 From: python-checkins at python.org (terry.reedy) Date: Fri, 04 Sep 2015 08:19:09 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzIxMTky?= =?utf-8?q?=3A_Change_=27RUN=27_back_to_=27RESTART=27_when_running_editor_?= =?utf-8?q?file=2E?= Message-ID: <20150904081908.68863.54378@psf.io> https://hg.python.org/cpython/rev/09cd7d57b080 changeset: 97658:09cd7d57b080 branch: 3.5 parent: 97638:9f8c59e61594 user: Terry Jan Reedy date: Thu Sep 03 21:26:12 2015 -0400 summary: Issue #21192: Change 'RUN' back to 'RESTART' when running editor file. files: Lib/idlelib/PyShell.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Lib/idlelib/PyShell.py b/Lib/idlelib/PyShell.py --- a/Lib/idlelib/PyShell.py +++ b/Lib/idlelib/PyShell.py @@ -487,7 +487,7 @@ console.stop_readline() # annotate restart in shell window and mark it console.text.delete("iomark", "end-1c") - tag = 'RUN ' + filename if filename else 'RESTART Shell' + tag = 'RESTART: ' + (filename if filename else 'Shell') halfbar = ((int(console.width) -len(tag) - 4) // 2) * '=' console.write("\n{0} {1} {0}".format(halfbar, tag)) console.text.mark_set("restart", "end-1c") -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 4 10:19:09 2015 From: python-checkins at python.org (terry.reedy) Date: Fri, 04 Sep 2015 08:19:09 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy41IC0+IDMuNSk6?= =?utf-8?q?_Merged_in_storchaka/cpython350_=28pull_request_=2313=29?= Message-ID: <20150904081909.27703.84258@psf.io> https://hg.python.org/cpython/rev/07e04c34bab5 changeset: 97659:07e04c34bab5 branch: 3.5 parent: 97658:09cd7d57b080 parent: 97647:2b6ce7e9595c user: Larry Hastings date: Thu Sep 03 22:12:08 2015 -0700 summary: Merged in storchaka/cpython350 (pull request #13) Issue #24989 files: Lib/test/test_memoryio.py | 13 +++++++++++++ Misc/NEWS | 3 +++ Modules/_io/bytesio.c | 6 +++++- 3 files changed, 21 insertions(+), 1 deletions(-) diff --git a/Lib/test/test_memoryio.py b/Lib/test/test_memoryio.py --- a/Lib/test/test_memoryio.py +++ b/Lib/test/test_memoryio.py @@ -166,6 +166,10 @@ memio.seek(0) self.assertEqual(memio.read(None), buf) self.assertRaises(TypeError, memio.read, '') + memio.seek(len(buf) + 1) + self.assertEqual(memio.read(1), self.EOF) + memio.seek(len(buf) + 1) + self.assertEqual(memio.read(), self.EOF) memio.close() self.assertRaises(ValueError, memio.read) @@ -185,6 +189,9 @@ self.assertEqual(memio.readline(-1), buf) memio.seek(0) self.assertEqual(memio.readline(0), self.EOF) + # Issue #24989: Buffer overread + memio.seek(len(buf) * 2 + 1) + self.assertEqual(memio.readline(), self.EOF) buf = self.buftype("1234567890\n") memio = self.ioclass((buf * 3)[:-1]) @@ -217,6 +224,9 @@ memio.seek(0) self.assertEqual(memio.readlines(None), [buf] * 10) self.assertRaises(TypeError, memio.readlines, '') + # Issue #24989: Buffer overread + memio.seek(len(buf) * 10 + 1) + self.assertEqual(memio.readlines(), []) memio.close() self.assertRaises(ValueError, memio.readlines) @@ -238,6 +248,9 @@ self.assertEqual(line, buf) i += 1 self.assertEqual(i, 10) + # Issue #24989: Buffer overread + memio.seek(len(buf) * 10 + 1) + self.assertEqual(list(memio), []) memio = self.ioclass(buf * 2) memio.close() self.assertRaises(ValueError, memio.__next__) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -15,6 +15,9 @@ Library ------- +- Issue #24989: Fixed buffer overread in BytesIO.readline() if a position is + set beyond size. Based on patch by John Leitch. + - Issue #24913: Fix overrun error in deque.index(). Found by John Leitch and Bryce Darling. diff --git a/Modules/_io/bytesio.c b/Modules/_io/bytesio.c --- a/Modules/_io/bytesio.c +++ b/Modules/_io/bytesio.c @@ -57,14 +57,18 @@ Py_ssize_t maxlen; assert(self->buf != NULL); + assert(self->pos >= 0); + + if (self->pos >= self->string_size) + return 0; /* Move to the end of the line, up to the end of the string, s. */ - start = PyBytes_AS_STRING(self->buf) + self->pos; maxlen = self->string_size - self->pos; if (len < 0 || len > maxlen) len = maxlen; if (len) { + start = PyBytes_AS_STRING(self->buf) + self->pos; n = memchr(start, '\n', len); if (n) /* Get the length from the current position to the end of -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 4 10:34:24 2015 From: python-checkins at python.org (victor.stinner) Date: Fri, 04 Sep 2015 08:34:24 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_test=5Ftime=3A_add_tests_o?= =?utf-8?q?n_HALF=5FUP_rounding_mode_for_=5FPyTime=5FObjectToTime=5Ft=28?= =?utf-8?q?=29_and?= Message-ID: <20150904083424.12001.51302@psf.io> https://hg.python.org/cpython/rev/a18bb4fb7b1e changeset: 97662:a18bb4fb7b1e user: Victor Stinner date: Fri Sep 04 10:31:16 2015 +0200 summary: test_time: add tests on HALF_UP rounding mode for _PyTime_ObjectToTime_t() and _PyTime_ObjectToTimespec() files: Lib/test/test_time.py | 130 ++++++++++++++++++++--------- 1 files changed, 90 insertions(+), 40 deletions(-) diff --git a/Lib/test/test_time.py b/Lib/test/test_time.py --- a/Lib/test/test_time.py +++ b/Lib/test/test_time.py @@ -606,24 +606,49 @@ @support.cpython_only def test_time_t(self): from _testcapi import pytime_object_to_time_t + + # Conversion giving the same result for all rounding methods + for rnd in ALL_ROUNDING_METHODS: + for obj, seconds in ( + # int + (-1, -1), + (0, 0), + (1, 1), + + # float + (-1.0, -1), + (1.0, 1), + ): + with self.subTest(obj=obj, round=rnd, seconds=seconds): + self.assertEqual(pytime_object_to_time_t(obj, rnd), + seconds) + + # Conversion giving different results depending on the rounding method + FLOOR = _PyTime.ROUND_FLOOR + CEILING = _PyTime.ROUND_CEILING + HALF_UP = _PyTime.ROUND_HALF_UP for obj, time_t, rnd in ( - # Round towards minus infinity (-inf) - (0, 0, _PyTime.ROUND_FLOOR), - (-1, -1, _PyTime.ROUND_FLOOR), - (-1.0, -1, _PyTime.ROUND_FLOOR), - (-1.9, -2, _PyTime.ROUND_FLOOR), - (1.0, 1, _PyTime.ROUND_FLOOR), - (1.9, 1, _PyTime.ROUND_FLOOR), - # Round towards infinity (+inf) - (0, 0, _PyTime.ROUND_CEILING), - (-1, -1, _PyTime.ROUND_CEILING), - (-1.0, -1, _PyTime.ROUND_CEILING), - (-1.9, -1, _PyTime.ROUND_CEILING), - (1.0, 1, _PyTime.ROUND_CEILING), - (1.9, 2, _PyTime.ROUND_CEILING), + (-1.9, -2, FLOOR), + (-1.9, -1, CEILING), + (-1.9, -2, HALF_UP), + + (1.9, 1, FLOOR), + (1.9, 2, CEILING), + (1.9, 2, HALF_UP), + + # half up + (-0.999, -1, HALF_UP), + (-0.510, -1, HALF_UP), + (-0.500, -1, HALF_UP), + (-0.490, 0, HALF_UP), + ( 0.490, 0, HALF_UP), + ( 0.500, 1, HALF_UP), + ( 0.510, 1, HALF_UP), + ( 0.999, 1, HALF_UP), ): self.assertEqual(pytime_object_to_time_t(obj, rnd), time_t) + # Test OverflowError rnd = _PyTime.ROUND_FLOOR for invalid in self.invalid_values: self.assertRaises(OverflowError, @@ -632,39 +657,64 @@ @support.cpython_only def test_timespec(self): from _testcapi import pytime_object_to_timespec + + # Conversion giving the same result for all rounding methods + for rnd in ALL_ROUNDING_METHODS: + for obj, timespec in ( + # int + (0, (0, 0)), + (-1, (-1, 0)), + + # float + (-1.2, (-2, 800000000)), + (-1.0, (-1, 0)), + (-1e-9, (-1, 999999999)), + (1e-9, (0, 1)), + (1.0, (1, 0)), + ): + with self.subTest(obj=obj, round=rnd, timespec=timespec): + self.assertEqual(pytime_object_to_timespec(obj, rnd), timespec) + + # Conversion giving different results depending on the rounding method + FLOOR = _PyTime.ROUND_FLOOR + CEILING = _PyTime.ROUND_CEILING + HALF_UP = _PyTime.ROUND_HALF_UP for obj, timespec, rnd in ( # Round towards minus infinity (-inf) - (0, (0, 0), _PyTime.ROUND_FLOOR), - (-1, (-1, 0), _PyTime.ROUND_FLOOR), - (-1.0, (-1, 0), _PyTime.ROUND_FLOOR), - (1e-9, (0, 1), _PyTime.ROUND_FLOOR), - (1e-10, (0, 0), _PyTime.ROUND_FLOOR), - (-1e-9, (-1, 999999999), _PyTime.ROUND_FLOOR), - (-1e-10, (-1, 999999999), _PyTime.ROUND_FLOOR), - (-1.2, (-2, 800000000), _PyTime.ROUND_FLOOR), - (0.9999999999, (0, 999999999), _PyTime.ROUND_FLOOR), - (1.1234567890, (1, 123456789), _PyTime.ROUND_FLOOR), - (1.1234567899, (1, 123456789), _PyTime.ROUND_FLOOR), - (-1.1234567890, (-2, 876543211), _PyTime.ROUND_FLOOR), - (-1.1234567891, (-2, 876543210), _PyTime.ROUND_FLOOR), + (-1e-10, (0, 0), CEILING), + (-1e-10, (-1, 999999999), FLOOR), + (-1e-10, (0, 0), HALF_UP), + (1e-10, (0, 0), FLOOR), + (1e-10, (0, 1), CEILING), + (1e-10, (0, 0), HALF_UP), + + (0.9999999999, (0, 999999999), FLOOR), + (0.9999999999, (1, 0), CEILING), + + (1.1234567890, (1, 123456789), FLOOR), + (1.1234567899, (1, 123456789), FLOOR), + (-1.1234567890, (-2, 876543211), FLOOR), + (-1.1234567891, (-2, 876543210), FLOOR), # Round towards infinity (+inf) - (0, (0, 0), _PyTime.ROUND_CEILING), - (-1, (-1, 0), _PyTime.ROUND_CEILING), - (-1.0, (-1, 0), _PyTime.ROUND_CEILING), - (1e-9, (0, 1), _PyTime.ROUND_CEILING), - (1e-10, (0, 1), _PyTime.ROUND_CEILING), - (-1e-9, (-1, 999999999), _PyTime.ROUND_CEILING), - (-1e-10, (0, 0), _PyTime.ROUND_CEILING), - (-1.2, (-2, 800000000), _PyTime.ROUND_CEILING), - (0.9999999999, (1, 0), _PyTime.ROUND_CEILING), - (1.1234567890, (1, 123456790), _PyTime.ROUND_CEILING), - (1.1234567899, (1, 123456790), _PyTime.ROUND_CEILING), - (-1.1234567890, (-2, 876543211), _PyTime.ROUND_CEILING), - (-1.1234567891, (-2, 876543211), _PyTime.ROUND_CEILING), + (1.1234567890, (1, 123456790), CEILING), + (1.1234567899, (1, 123456790), CEILING), + (-1.1234567890, (-2, 876543211), CEILING), + (-1.1234567891, (-2, 876543211), CEILING), + + # half up + (-0.6e-9, (-1, 999999999), HALF_UP), + # skipped, 0.5e-6 is inexact in base 2 + #(-0.5e-9, (-1, 999999999), HALF_UP), + (-0.4e-9, (0, 0), HALF_UP), + + (0.4e-9, (0, 0), HALF_UP), + (0.5e-9, (0, 1), HALF_UP), + (0.6e-9, (0, 1), HALF_UP), ): with self.subTest(obj=obj, round=rnd, timespec=timespec): self.assertEqual(pytime_object_to_timespec(obj, rnd), timespec) + # Test OverflowError rnd = _PyTime.ROUND_FLOOR for invalid in self.invalid_values: self.assertRaises(OverflowError, -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 4 10:39:57 2015 From: python-checkins at python.org (terry.reedy) Date: Fri, 04 Sep 2015 08:39:57 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzI0NzQ1?= =?utf-8?q?=3A_Prevent_IDLE_initialization_crash_with_Tk_8=2E4=3B_patch_by?= =?utf-8?q?_Ned_Deily=2E?= Message-ID: <20150904083956.66866.36556@psf.io> https://hg.python.org/cpython/rev/34a8078f6249 changeset: 97663:34a8078f6249 branch: 2.7 parent: 97655:252d4760f28b user: Terry Jan Reedy date: Fri Sep 04 04:37:02 2015 -0400 summary: Issue #24745: Prevent IDLE initialization crash with Tk 8.4; patch by Ned Deily. files: Lib/idlelib/configHandler.py | 18 +++++++++++------- 1 files changed, 11 insertions(+), 7 deletions(-) diff --git a/Lib/idlelib/configHandler.py b/Lib/idlelib/configHandler.py --- a/Lib/idlelib/configHandler.py +++ b/Lib/idlelib/configHandler.py @@ -23,6 +23,7 @@ import sys from ConfigParser import ConfigParser +from Tkinter import TkVersion from tkFont import Font, nametofont class InvalidConfigType(Exception): pass @@ -689,13 +690,16 @@ bold = self.GetOption(configType, section, 'font-bold', default=0, type='bool') if (family == 'TkFixedFont'): - f = Font(name='TkFixedFont', exists=True, root=root) - actualFont = Font.actual(f) - family = actualFont['family'] - size = actualFont['size'] - if size < 0: - size = 10 # if font in pixels, ignore actual size - bold = actualFont['weight']=='bold' + if TkVersion < 8.5: + family = 'Courier' + else: + f = Font(name='TkFixedFont', exists=True, root=root) + actualFont = Font.actual(f) + family = actualFont['family'] + size = actualFont['size'] + if size < 0: + size = 10 # if font in pixels, ignore actual size + bold = actualFont['weight']=='bold' return (family, size, 'bold' if bold else 'normal') def LoadCfgFiles(self): -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 4 10:39:57 2015 From: python-checkins at python.org (terry.reedy) Date: Fri, 04 Sep 2015 08:39:57 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge_with_3=2E5?= Message-ID: <20150904083957.11270.77802@psf.io> https://hg.python.org/cpython/rev/59f97fd6b655 changeset: 97666:59f97fd6b655 parent: 97662:a18bb4fb7b1e parent: 97665:c4fb0ac2fabc user: Terry Jan Reedy date: Fri Sep 04 04:38:52 2015 -0400 summary: Merge with 3.5 files: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 4 10:39:57 2015 From: python-checkins at python.org (terry.reedy) Date: Fri, 04 Sep 2015 08:39:57 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Merge_with_3=2E4?= Message-ID: <20150904083957.17967.35100@psf.io> https://hg.python.org/cpython/rev/c4fb0ac2fabc changeset: 97665:c4fb0ac2fabc branch: 3.5 parent: 97660:cb5508b8fc04 parent: 97664:b4830b9f8c10 user: Terry Jan Reedy date: Fri Sep 04 04:38:17 2015 -0400 summary: Merge with 3.4 files: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 4 10:39:58 2015 From: python-checkins at python.org (terry.reedy) Date: Fri, 04 Sep 2015 08:39:58 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzI0NzQ1?= =?utf-8?q?=3A_Prevent_IDLE_initialization_crash_with_Tk_8=2E4=3B_patch_by?= =?utf-8?q?_Ned_Deily=2E?= Message-ID: <20150904083957.101496.26240@psf.io> https://hg.python.org/cpython/rev/b4830b9f8c10 changeset: 97664:b4830b9f8c10 branch: 3.4 parent: 97633:94f61fe22edf user: Terry Jan Reedy date: Fri Sep 04 04:37:56 2015 -0400 summary: Issue #24745: Prevent IDLE initialization crash with Tk 8.4; patch by Ned Deily. files: Lib/idlelib/configHandler.py | 18 +++++++++++------- 1 files changed, 11 insertions(+), 7 deletions(-) diff --git a/Lib/idlelib/configHandler.py b/Lib/idlelib/configHandler.py --- a/Lib/idlelib/configHandler.py +++ b/Lib/idlelib/configHandler.py @@ -22,6 +22,7 @@ import sys from configparser import ConfigParser +from tkinter import TkVersion from tkinter.font import Font, nametofont class InvalidConfigType(Exception): pass @@ -688,13 +689,16 @@ bold = self.GetOption(configType, section, 'font-bold', default=0, type='bool') if (family == 'TkFixedFont'): - f = Font(name='TkFixedFont', exists=True, root=root) - actualFont = Font.actual(f) - family = actualFont['family'] - size = actualFont['size'] - if size < 0: - size = 10 # if font in pixels, ignore actual size - bold = actualFont['weight']=='bold' + if TkVersion < 8.5: + family = 'Courier' + else: + f = Font(name='TkFixedFont', exists=True, root=root) + actualFont = Font.actual(f) + family = actualFont['family'] + size = actualFont['size'] + if size < 0: + size = 10 # if font in pixels, ignore actual size + bold = actualFont['weight']=='bold' return (family, size, 'bold' if bold else 'normal') def LoadCfgFiles(self): -- Repository URL: https://hg.python.org/cpython From lp_benchmark_robot at intel.com Fri Sep 4 11:18:22 2015 From: lp_benchmark_robot at intel.com (lp_benchmark_robot) Date: Fri, 4 Sep 2015 09:18:22 +0000 Subject: [Python-checkins] Benchmark Results for Python Default 2015-09-04 Message-ID: <078AA0FFE8C7034097F90205717F504611D79813@IRSMSX102.ger.corp.intel.com> Results for project python_default-nightly, build date 2015-09-04 11:33:25 commit: 66e30dbf4cae09cb7c8e453fce35be9ea0a6ba44 revision date: 2015-09-04 11:18:11 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v3.4.3, with hash b4cbecbc0781e89a309d03b60a1f75f8499250e6 from 2015-02-25 12:15:33+00:00 ------------------------------------------------------------------------------------------ benchmark relative change since change since current rev with std_dev* last run v3.4.3 regrtest PGO ------------------------------------------------------------------------------------------ :-) django_v2 0.23637% 1.32263% 6.10504% 19.61871% :-| pybench 0.12090% -0.05237% -1.81970% 8.60689% :-( regex_v8 3.02207% -0.09877% -3.77418% 5.08075% :-| nbody 0.29281% -0.21508% -0.76222% 6.82999% :-( json_dump_v2 0.25334% -0.03395% -5.24930% 15.21677% :-| normal_startup 0.83994% 0.16523% -0.54399% 5.15376% ------------------------------------------------------------------------------------------ Note: Benchmark results are measured in seconds. * Relative Standard Deviation (Standard Deviation/Average) Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. No license (express or implied, by estoppel or otherwise) to any intellectual property rights is granted by this document. Intel disclaims all express and implied warranties, including without limitation, the implied warranties of merchantability, fitness for a particular purpose, and non-infringement, as well as any warranty arising from course of performance, course of dealing, or usage in trade. This document may contain information on products, services and/or processes in development. Contact your Intel representative to obtain the latest forecast, schedule, specifications and roadmaps. The products and services described may contain defects or errors known as errata which may cause deviations from published specifications. Current characterized errata are available on request. (C) 2015 Intel Corporation. From python-checkins at python.org Fri Sep 4 11:34:18 2015 From: python-checkins at python.org (eric.smith) Date: Fri, 04 Sep 2015 09:34:18 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_Fix_typo_introduced_when_swit?= =?utf-8?q?ching_to_backquotes=2E?= Message-ID: <20150904093418.11999.91934@psf.io> https://hg.python.org/peps/rev/b133c85ae481 changeset: 6027:b133c85ae481 user: Eric V. Smith date: Fri Sep 04 05:34:35 2015 -0400 summary: Fix typo introduced when switching to backquotes. files: pep-0498.txt | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/pep-0498.txt b/pep-0498.txt --- a/pep-0498.txt +++ b/pep-0498.txt @@ -214,7 +214,7 @@ Scanning an f-string for expressions happens after escape sequences are decoded. Because ``hex(ord('{')) == 0x7b``, the f-string -``f'\\u007b4*10}'`` is decoded to ``f'{4*10}'``, which evaluates as +``f'\u007b4*10}'`` is decoded to ``f'{4*10}'``, which evaluates as the integer 40:: >>> f'\u007b4*10}' -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Fri Sep 4 16:03:42 2015 From: python-checkins at python.org (r.david.murray) Date: Fri, 04 Sep 2015 14:03:42 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge=3A_=2324998=3A_fix_cut_and_paste_error_in_subproce?= =?utf-8?q?ss_example=2E?= Message-ID: <20150904140342.17979.79113@psf.io> https://hg.python.org/cpython/rev/75e10a5cbf43 changeset: 97670:75e10a5cbf43 parent: 97666:59f97fd6b655 parent: 97669:fa53edb32962 user: R David Murray date: Fri Sep 04 10:03:03 2015 -0400 summary: Merge: #24998: fix cut and paste error in subprocess example. files: Doc/library/subprocess.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/library/subprocess.rst b/Doc/library/subprocess.rst --- a/Doc/library/subprocess.rst +++ b/Doc/library/subprocess.rst @@ -1068,7 +1068,7 @@ if rc is not None and rc >> 8: print("There were some errors") ==> - process = Popen(cmd, 'w', stdin=PIPE) + process = Popen(cmd, stdin=PIPE) ... process.stdin.close() if process.wait() != 0: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 4 16:03:44 2015 From: python-checkins at python.org (r.david.murray) Date: Fri, 04 Sep 2015 14:03:44 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Merge=3A_=2324998=3A_fix_cut_and_paste_error_in_subprocess_exa?= =?utf-8?q?mple=2E?= Message-ID: <20150904140342.68871.8202@psf.io> https://hg.python.org/cpython/rev/fa53edb32962 changeset: 97669:fa53edb32962 branch: 3.5 parent: 97665:c4fb0ac2fabc parent: 97668:47e711a7416b user: R David Murray date: Fri Sep 04 10:02:27 2015 -0400 summary: Merge: #24998: fix cut and paste error in subprocess example. files: Doc/library/subprocess.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/library/subprocess.rst b/Doc/library/subprocess.rst --- a/Doc/library/subprocess.rst +++ b/Doc/library/subprocess.rst @@ -1068,7 +1068,7 @@ if rc is not None and rc >> 8: print("There were some errors") ==> - process = Popen(cmd, 'w', stdin=PIPE) + process = Popen(cmd, stdin=PIPE) ... process.stdin.close() if process.wait() != 0: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 4 16:03:44 2015 From: python-checkins at python.org (r.david.murray) Date: Fri, 04 Sep 2015 14:03:44 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogIzI0OTk4OiBmaXgg?= =?utf-8?q?cut_and_paste_error_in_subprocess_example=2E?= Message-ID: <20150904140341.66860.9850@psf.io> https://hg.python.org/cpython/rev/47e711a7416b changeset: 97668:47e711a7416b branch: 3.4 parent: 97664:b4830b9f8c10 user: R David Murray date: Fri Sep 04 10:01:19 2015 -0400 summary: #24998: fix cut and paste error in subprocess example. files: Doc/library/subprocess.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/library/subprocess.rst b/Doc/library/subprocess.rst --- a/Doc/library/subprocess.rst +++ b/Doc/library/subprocess.rst @@ -1000,7 +1000,7 @@ if rc is not None and rc >> 8: print("There were some errors") ==> - process = Popen(cmd, 'w', stdin=PIPE) + process = Popen(cmd, stdin=PIPE) ... process.stdin.close() if process.wait() != 0: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 4 16:03:44 2015 From: python-checkins at python.org (r.david.murray) Date: Fri, 04 Sep 2015 14:03:44 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogIzI0OTk4OiBmaXgg?= =?utf-8?q?cut_and_paste_error_in_subprocess_example=2E?= Message-ID: <20150904140341.101476.26749@psf.io> https://hg.python.org/cpython/rev/0ff7aa9a438f changeset: 97667:0ff7aa9a438f branch: 2.7 parent: 97663:34a8078f6249 user: R David Murray date: Fri Sep 04 10:00:22 2015 -0400 summary: #24998: fix cut and paste error in subprocess example. files: Doc/library/subprocess.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/library/subprocess.rst b/Doc/library/subprocess.rst --- a/Doc/library/subprocess.rst +++ b/Doc/library/subprocess.rst @@ -852,7 +852,7 @@ if rc is not None and rc >> 8: print "There were some errors" ==> - process = Popen("cmd", 'w', shell=True, stdin=PIPE) + process = Popen("cmd", shell=True, stdin=PIPE) ... process.stdin.close() if process.wait() != 0: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 4 17:31:51 2015 From: python-checkins at python.org (victor.stinner) Date: Fri, 04 Sep 2015 15:31:51 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_Fix_race_condi?= =?utf-8?q?tion_in_create=5Fstdio=28=29?= Message-ID: <20150904153151.17979.87633@psf.io> https://hg.python.org/cpython/rev/e67bf9c9a898 changeset: 97671:e67bf9c9a898 branch: 3.4 parent: 97668:47e711a7416b user: Victor Stinner date: Fri Sep 04 17:27:49 2015 +0200 summary: Fix race condition in create_stdio() Issue #24891: Fix a race condition at Python startup if the file descriptor of stdin (0), stdout (1) or stderr (2) is closed while Python is creating sys.stdin, sys.stdout and sys.stderr objects. These attributes are now set to None if the creation of the object failed, instead of raising an OSError exception. Initial patch written by Marco Paolini. files: Misc/ACKS | 1 + Misc/NEWS | 6 ++ Python/pythonrun.c | 75 +++++++++++++++------------------ 3 files changed, 42 insertions(+), 40 deletions(-) diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -1037,6 +1037,7 @@ Yongzhi Pan Martin Panter Mathias Panzenb?ck +Marco Paolini M. Papillon Peter Parente Alexandre Parenteau diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -10,6 +10,12 @@ Core and Builtins ----------------- +- Issue #24891: Fix a race condition at Python startup if the file descriptor + of stdin (0), stdout (1) or stderr (2) is closed while Python is creating + sys.stdin, sys.stdout and sys.stderr objects. These attributes are now set + to None if the creation of the object failed, instead of raising an OSError + exception. Initial patch written by Marco Paolini. + - Issue #21167: NAN operations are now handled correctly when python is compiled with ICC even if -fp-model strict is not specified. diff --git a/Python/pythonrun.c b/Python/pythonrun.c --- a/Python/pythonrun.c +++ b/Python/pythonrun.c @@ -1003,6 +1003,21 @@ } } +/* Check if a file descriptor is valid or not. + Return 0 if the file descriptor is invalid, return non-zero otherwise. */ +static int +is_valid_fd(int fd) +{ + int fd2; + if (fd < 0 || !_PyVerify_fd(fd)) + return 0; + fd2 = dup(fd); + if (fd2 >= 0) + close(fd2); + return fd2 >= 0; +} + +/* returns Py_None if the fd is not valid */ static PyObject* create_stdio(PyObject* io, int fd, int write_mode, char* name, @@ -1018,6 +1033,9 @@ _Py_IDENTIFIER(TextIOWrapper); _Py_IDENTIFIER(mode); + if (!is_valid_fd(fd)) + Py_RETURN_NONE; + /* stdin is always opened in buffered mode, first because it shouldn't make a difference in common use cases, second because TextIOWrapper depends on the presence of a read1() method which only exists on @@ -1099,22 +1117,17 @@ Py_XDECREF(stream); Py_XDECREF(text); Py_XDECREF(raw); + + if (PyErr_ExceptionMatches(PyExc_OSError) && !is_valid_fd(fd)) { + /* Issue #24891: the file descriptor was closed after the first + is_valid_fd() check was called. Ignore the OSError and set the + stream to None. */ + PyErr_Clear(); + Py_RETURN_NONE; + } return NULL; } -static int -is_valid_fd(int fd) -{ - int dummy_fd; - if (fd < 0 || !_PyVerify_fd(fd)) - return 0; - dummy_fd = dup(fd); - if (dummy_fd < 0) - return 0; - close(dummy_fd); - return 1; -} - /* Initialize sys.stdin, stdout, stderr and builtins.open */ static int initstdio(void) @@ -1188,30 +1201,18 @@ * and fileno() may point to an invalid file descriptor. For example * GUI apps don't have valid standard streams by default. */ - if (!is_valid_fd(fd)) { - std = Py_None; - Py_INCREF(std); - } - else { - std = create_stdio(iomod, fd, 0, "", encoding, errors); - if (std == NULL) - goto error; - } /* if (fd < 0) */ + std = create_stdio(iomod, fd, 0, "", encoding, errors); + if (std == NULL) + goto error; PySys_SetObject("__stdin__", std); _PySys_SetObjectId(&PyId_stdin, std); Py_DECREF(std); /* Set sys.stdout */ fd = fileno(stdout); - if (!is_valid_fd(fd)) { - std = Py_None; - Py_INCREF(std); - } - else { - std = create_stdio(iomod, fd, 1, "", encoding, errors); - if (std == NULL) - goto error; - } /* if (fd < 0) */ + std = create_stdio(iomod, fd, 1, "", encoding, errors); + if (std == NULL) + goto error; PySys_SetObject("__stdout__", std); _PySys_SetObjectId(&PyId_stdout, std); Py_DECREF(std); @@ -1219,15 +1220,9 @@ #if 1 /* Disable this if you have trouble debugging bootstrap stuff */ /* Set sys.stderr, replaces the preliminary stderr */ fd = fileno(stderr); - if (!is_valid_fd(fd)) { - std = Py_None; - Py_INCREF(std); - } - else { - std = create_stdio(iomod, fd, 1, "", encoding, "backslashreplace"); - if (std == NULL) - goto error; - } /* if (fd < 0) */ + std = create_stdio(iomod, fd, 1, "", encoding, "backslashreplace"); + if (std == NULL) + goto error; /* Same as hack above, pre-import stderr's codec to avoid recursion when import.c tries to write to stderr in verbose mode. */ -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 4 17:31:51 2015 From: python-checkins at python.org (victor.stinner) Date: Fri, 04 Sep 2015 15:31:51 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogTWVyZ2UgMy41IChjcmVhdGVfc3RkaW8p?= Message-ID: <20150904153151.14865.74748@psf.io> https://hg.python.org/cpython/rev/73911e6c97c8 changeset: 97673:73911e6c97c8 parent: 97670:75e10a5cbf43 parent: 97672:d562a421d6cd user: Victor Stinner date: Fri Sep 04 17:30:48 2015 +0200 summary: Merge 3.5 (create_stdio) files: Misc/ACKS | 1 + Misc/NEWS | 6 ++ Python/pylifecycle.c | 79 ++++++++++++++----------------- 3 files changed, 44 insertions(+), 42 deletions(-) diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -1066,6 +1066,7 @@ Yongzhi Pan Martin Panter Mathias Panzenb?ck +Marco Paolini M. Papillon Peter Parente Alexandre Parenteau diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -101,6 +101,12 @@ Library ------- +- Issue #24891: Fix a race condition at Python startup if the file descriptor + of stdin (0), stdout (1) or stderr (2) is closed while Python is creating + sys.stdin, sys.stdout and sys.stderr objects. These attributes are now set + to None if the creation of the object failed, instead of raising an OSError + exception. Initial patch written by Marco Paolini. + - Issue #24992: Fix error handling and a race condition (related to garbage collection) in collections.OrderedDict constructor. diff --git a/Python/pylifecycle.c b/Python/pylifecycle.c --- a/Python/pylifecycle.c +++ b/Python/pylifecycle.c @@ -1,4 +1,3 @@ - /* Python interpreter top-level routines, including init/exit */ #include "Python.h" @@ -963,6 +962,23 @@ } } +/* Check if a file descriptor is valid or not. + Return 0 if the file descriptor is invalid, return non-zero otherwise. */ +static int +is_valid_fd(int fd) +{ + int fd2; + if (fd < 0 || !_PyVerify_fd(fd)) + return 0; + _Py_BEGIN_SUPPRESS_IPH + fd2 = dup(fd); + if (fd2 >= 0) + close(fd2); + _Py_END_SUPPRESS_IPH + return fd2 >= 0; +} + +/* returns Py_None if the fd is not valid */ static PyObject* create_stdio(PyObject* io, int fd, int write_mode, char* name, @@ -978,6 +994,9 @@ _Py_IDENTIFIER(TextIOWrapper); _Py_IDENTIFIER(mode); + if (!is_valid_fd(fd)) + Py_RETURN_NONE; + /* stdin is always opened in buffered mode, first because it shouldn't make a difference in common use cases, second because TextIOWrapper depends on the presence of a read1() method which only exists on @@ -1059,23 +1078,17 @@ Py_XDECREF(stream); Py_XDECREF(text); Py_XDECREF(raw); + + if (PyErr_ExceptionMatches(PyExc_OSError) && !is_valid_fd(fd)) { + /* Issue #24891: the file descriptor was closed after the first + is_valid_fd() check was called. Ignore the OSError and set the + stream to None. */ + PyErr_Clear(); + Py_RETURN_NONE; + } return NULL; } -static int -is_valid_fd(int fd) -{ - int dummy_fd; - if (fd < 0 || !_PyVerify_fd(fd)) - return 0; - _Py_BEGIN_SUPPRESS_IPH - dummy_fd = dup(fd); - if (dummy_fd >= 0) - close(dummy_fd); - _Py_END_SUPPRESS_IPH - return dummy_fd >= 0; -} - /* Initialize sys.stdin, stdout, stderr and builtins.open */ static int initstdio(void) @@ -1158,30 +1171,18 @@ * and fileno() may point to an invalid file descriptor. For example * GUI apps don't have valid standard streams by default. */ - if (!is_valid_fd(fd)) { - std = Py_None; - Py_INCREF(std); - } - else { - std = create_stdio(iomod, fd, 0, "", encoding, errors); - if (std == NULL) - goto error; - } /* if (fd < 0) */ + std = create_stdio(iomod, fd, 0, "", encoding, errors); + if (std == NULL) + goto error; PySys_SetObject("__stdin__", std); _PySys_SetObjectId(&PyId_stdin, std); Py_DECREF(std); /* Set sys.stdout */ fd = fileno(stdout); - if (!is_valid_fd(fd)) { - std = Py_None; - Py_INCREF(std); - } - else { - std = create_stdio(iomod, fd, 1, "", encoding, errors); - if (std == NULL) - goto error; - } /* if (fd < 0) */ + std = create_stdio(iomod, fd, 1, "", encoding, errors); + if (std == NULL) + goto error; PySys_SetObject("__stdout__", std); _PySys_SetObjectId(&PyId_stdout, std); Py_DECREF(std); @@ -1189,15 +1190,9 @@ #if 1 /* Disable this if you have trouble debugging bootstrap stuff */ /* Set sys.stderr, replaces the preliminary stderr */ fd = fileno(stderr); - if (!is_valid_fd(fd)) { - std = Py_None; - Py_INCREF(std); - } - else { - std = create_stdio(iomod, fd, 1, "", encoding, "backslashreplace"); - if (std == NULL) - goto error; - } /* if (fd < 0) */ + std = create_stdio(iomod, fd, 1, "", encoding, "backslashreplace"); + if (std == NULL) + goto error; /* Same as hack above, pre-import stderr's codec to avoid recursion when import.c tries to write to stderr in verbose mode. */ -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 4 17:31:54 2015 From: python-checkins at python.org (victor.stinner) Date: Fri, 04 Sep 2015 15:31:54 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Merge_3=2E4_=28create=5Fstdio=29?= Message-ID: <20150904153151.101480.6656@psf.io> https://hg.python.org/cpython/rev/d562a421d6cd changeset: 97672:d562a421d6cd branch: 3.5 parent: 97669:fa53edb32962 parent: 97671:e67bf9c9a898 user: Victor Stinner date: Fri Sep 04 17:29:57 2015 +0200 summary: Merge 3.4 (create_stdio) files: Misc/ACKS | 1 + Misc/NEWS | 6 ++ Python/pylifecycle.c | 79 ++++++++++++++----------------- 3 files changed, 44 insertions(+), 42 deletions(-) diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -1066,6 +1066,7 @@ Yongzhi Pan Martin Panter Mathias Panzenb?ck +Marco Paolini M. Papillon Peter Parente Alexandre Parenteau diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -14,6 +14,12 @@ Library ------- +- Issue #24891: Fix a race condition at Python startup if the file descriptor + of stdin (0), stdout (1) or stderr (2) is closed while Python is creating + sys.stdin, sys.stdout and sys.stderr objects. These attributes are now set + to None if the creation of the object failed, instead of raising an OSError + exception. Initial patch written by Marco Paolini. + - Issue #24992: Fix error handling and a race condition (related to garbage collection) in collections.OrderedDict constructor. diff --git a/Python/pylifecycle.c b/Python/pylifecycle.c --- a/Python/pylifecycle.c +++ b/Python/pylifecycle.c @@ -1,4 +1,3 @@ - /* Python interpreter top-level routines, including init/exit */ #include "Python.h" @@ -963,6 +962,23 @@ } } +/* Check if a file descriptor is valid or not. + Return 0 if the file descriptor is invalid, return non-zero otherwise. */ +static int +is_valid_fd(int fd) +{ + int fd2; + if (fd < 0 || !_PyVerify_fd(fd)) + return 0; + _Py_BEGIN_SUPPRESS_IPH + fd2 = dup(fd); + if (fd2 >= 0) + close(fd2); + _Py_END_SUPPRESS_IPH + return fd2 >= 0; +} + +/* returns Py_None if the fd is not valid */ static PyObject* create_stdio(PyObject* io, int fd, int write_mode, char* name, @@ -978,6 +994,9 @@ _Py_IDENTIFIER(TextIOWrapper); _Py_IDENTIFIER(mode); + if (!is_valid_fd(fd)) + Py_RETURN_NONE; + /* stdin is always opened in buffered mode, first because it shouldn't make a difference in common use cases, second because TextIOWrapper depends on the presence of a read1() method which only exists on @@ -1059,23 +1078,17 @@ Py_XDECREF(stream); Py_XDECREF(text); Py_XDECREF(raw); + + if (PyErr_ExceptionMatches(PyExc_OSError) && !is_valid_fd(fd)) { + /* Issue #24891: the file descriptor was closed after the first + is_valid_fd() check was called. Ignore the OSError and set the + stream to None. */ + PyErr_Clear(); + Py_RETURN_NONE; + } return NULL; } -static int -is_valid_fd(int fd) -{ - int dummy_fd; - if (fd < 0 || !_PyVerify_fd(fd)) - return 0; - _Py_BEGIN_SUPPRESS_IPH - dummy_fd = dup(fd); - if (dummy_fd >= 0) - close(dummy_fd); - _Py_END_SUPPRESS_IPH - return dummy_fd >= 0; -} - /* Initialize sys.stdin, stdout, stderr and builtins.open */ static int initstdio(void) @@ -1158,30 +1171,18 @@ * and fileno() may point to an invalid file descriptor. For example * GUI apps don't have valid standard streams by default. */ - if (!is_valid_fd(fd)) { - std = Py_None; - Py_INCREF(std); - } - else { - std = create_stdio(iomod, fd, 0, "", encoding, errors); - if (std == NULL) - goto error; - } /* if (fd < 0) */ + std = create_stdio(iomod, fd, 0, "", encoding, errors); + if (std == NULL) + goto error; PySys_SetObject("__stdin__", std); _PySys_SetObjectId(&PyId_stdin, std); Py_DECREF(std); /* Set sys.stdout */ fd = fileno(stdout); - if (!is_valid_fd(fd)) { - std = Py_None; - Py_INCREF(std); - } - else { - std = create_stdio(iomod, fd, 1, "", encoding, errors); - if (std == NULL) - goto error; - } /* if (fd < 0) */ + std = create_stdio(iomod, fd, 1, "", encoding, errors); + if (std == NULL) + goto error; PySys_SetObject("__stdout__", std); _PySys_SetObjectId(&PyId_stdout, std); Py_DECREF(std); @@ -1189,15 +1190,9 @@ #if 1 /* Disable this if you have trouble debugging bootstrap stuff */ /* Set sys.stderr, replaces the preliminary stderr */ fd = fileno(stderr); - if (!is_valid_fd(fd)) { - std = Py_None; - Py_INCREF(std); - } - else { - std = create_stdio(iomod, fd, 1, "", encoding, "backslashreplace"); - if (std == NULL) - goto error; - } /* if (fd < 0) */ + std = create_stdio(iomod, fd, 1, "", encoding, "backslashreplace"); + if (std == NULL) + goto error; /* Same as hack above, pre-import stderr's codec to avoid recursion when import.c tries to write to stderr in verbose mode. */ -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 4 20:37:26 2015 From: python-checkins at python.org (eric.smith) Date: Fri, 04 Sep 2015 18:37:26 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_Make_examples_easier_to_read_?= =?utf-8?b?YnkgdXNpbmcgJysnIGluc3RlYWQgb2YgJycuam9pbigpLg==?= Message-ID: <20150904183726.14861.31737@psf.io> https://hg.python.org/peps/rev/69caea87d71d changeset: 6028:69caea87d71d user: Eric V. Smith date: Fri Sep 04 14:37:43 2015 -0400 summary: Make examples easier to read by using '+' instead of ''.join(). files: pep-0498.txt | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/pep-0498.txt b/pep-0498.txt --- a/pep-0498.txt +++ b/pep-0498.txt @@ -242,7 +242,7 @@ Might be be evaluated as:: - ''.join(['abc', expr1.__format__(spec1), repr(expr2).__format__(spec2), 'def', str(spec3).__format__(''), 'ghi']) + 'abc' + expr1.__format__(spec1) + repr(expr2).__format__(spec2) + 'def' + str(spec3).__format__('') + 'ghi' Expression evaluation --------------------- @@ -325,7 +325,7 @@ While the exact method of this run time concatenation is unspecified, the above code might evaluate to:: - ''.join(['ab', x.__format__(''), '{c}', 'str<', y.__format__('^4'), 'de']) + 'ab' + x.__format__('') + '{c}' + 'str<', y.__format__('^4') + 'de' Error handling -------------- -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Fri Sep 4 21:00:46 2015 From: python-checkins at python.org (guido.van.rossum) Date: Fri, 04 Sep 2015 19:00:46 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI0NjM1?= =?utf-8?q?=3A_Fixed_flakiness_in_test=5Ftyping=2Epy=2E?= Message-ID: <20150904190034.114719.60973@psf.io> https://hg.python.org/cpython/rev/d1f41c614e62 changeset: 97674:d1f41c614e62 branch: 3.5 parent: 97672:d562a421d6cd user: Guido van Rossum date: Fri Sep 04 12:00:06 2015 -0700 summary: Issue #24635: Fixed flakiness in test_typing.py. files: Lib/test/test_typing.py | 17 ++++++++++++++--- Lib/typing.py | 15 ++++++++++----- Misc/NEWS | 3 +++ 3 files changed, 27 insertions(+), 8 deletions(-) diff --git a/Lib/test/test_typing.py b/Lib/test/test_typing.py --- a/Lib/test/test_typing.py +++ b/Lib/test/test_typing.py @@ -436,12 +436,14 @@ c() def test_callable_instance_works(self): - f = lambda: None + def f(): + pass assert isinstance(f, Callable) assert not isinstance(None, Callable) def test_callable_instance_type_error(self): - f = lambda: None + def f(): + pass with self.assertRaises(TypeError): assert isinstance(f, Callable[[], None]) with self.assertRaises(TypeError): @@ -674,7 +676,9 @@ T = TypeVar('T') class Node(Generic[T]): - def __init__(self, label: T, left: 'Node[T]' = None, right: 'Node[T]' = None): + def __init__(self, label: T, + left: 'Node[T]' = None, + right: 'Node[T]' = None): self.label = label # type: T self.left = left # type: Optional[Node[T]] self.right = right # type: Optional[Node[T]] @@ -934,8 +938,15 @@ def test_iterable(self): assert isinstance([], typing.Iterable) + # Due to ABC caching, the second time takes a separate code + # path and could fail. So call this a few times. + assert isinstance([], typing.Iterable) + assert isinstance([], typing.Iterable) assert isinstance([], typing.Iterable[int]) assert not isinstance(42, typing.Iterable) + # Just in case, also test issubclass() a few times. + assert issubclass(list, typing.Iterable) + assert issubclass(list, typing.Iterable) def test_iterator(self): it = iter([]) diff --git a/Lib/typing.py b/Lib/typing.py --- a/Lib/typing.py +++ b/Lib/typing.py @@ -1,7 +1,3 @@ -# TODO: -# - Generic[T, T] is invalid -# - Look for TODO below - # TODO nits: # Get rid of asserts that are the caller's fault. # Docstrings (e.g. ABCs). @@ -963,7 +959,8 @@ raise TypeError("Initial parameters must be " "type variables; got %s" % p) if len(set(params)) != len(params): - raise TypeError("All type variables in Generic[...] must be distinct.") + raise TypeError( + "All type variables in Generic[...] must be distinct.") else: if len(params) != len(self.__parameters__): raise TypeError("Cannot change parameter count from %d to %d" % @@ -987,6 +984,14 @@ origin=self, extra=self.__extra__) + def __instancecheck__(self, instance): + # Since we extend ABC.__subclasscheck__ and + # ABC.__instancecheck__ inlines the cache checking done by the + # latter, we must extend __instancecheck__ too. For simplicity + # we just skip the cache check -- instance checks for generic + # classes are supposed to be rare anyways. + return self.__subclasscheck__(instance.__class__) + def __subclasscheck__(self, cls): if cls is Any: return True diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -88,6 +88,9 @@ Library ------- +- Issue #24635: Fixed a bug in typing.py where isinstance([], typing.Iterable) + would return True once, then False on subsequent calls. + - Issue #24989: Fixed buffer overread in BytesIO.readline() if a position is set beyond size. Based on patch by John Leitch. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 4 21:01:22 2015 From: python-checkins at python.org (eric.smith) Date: Fri, 04 Sep 2015 19:01:22 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_Removed_Implementation_Limita?= =?utf-8?q?tions_section=2E_While_the_version_of_the_code_on?= Message-ID: <20150904190117.68877.35140@psf.io> https://hg.python.org/peps/rev/a0194ec4195c changeset: 6029:a0194ec4195c user: Eric V. Smith date: Fri Sep 04 15:01:35 2015 -0400 summary: Removed Implementation Limitations section. While the version of the code on http://bugs.python.org/issue24965 has the 255 expression limitation, I'm going to remove this limit. The i18n section was purely speculative. We can worry about it if/when we add i18n and i-strings. files: pep-0498.txt | 55 ---------------------------------------- 1 files changed, 0 insertions(+), 55 deletions(-) diff --git a/pep-0498.txt b/pep-0498.txt --- a/pep-0498.txt +++ b/pep-0498.txt @@ -634,61 +634,6 @@ print("Usage: {0} [{1}]".format(sys.argv[0], '|'.join('--'+opt for opt in valid_opts)), file=sys.stderr) print(f"Usage: {sys.argv[0]} [{'|'.join('--'+opt for opt in valid_opts)}]", file=sys.stderr) -Implementation limitations -========================== - -Maximum of 255 expressions --------------------------- - -Due to a CPython limit with the number of parameters to a function, an -f-string may not contain more that 255 expressions. This includes -expressions inside format specifiers. So this code would count as -having 2 expressions:: - - f'{x:.{width}}' - -The same expression used multiple times ---------------------------------------- - -Every expression in an f-string is evaluated exactly once for each -time it appears in the f-string. However, when the same expression -appears more than once in an f-string, it's undefined which result -will be used in the resulting string value. This only matters for -expressions with side effects. - -For purposes of this section, two expressions are the same if they -have the exact same literal text defining them. For example, ``'{i}'`` -and ``'{i}'`` are the same expression, but ``'{i}'`` and ``'{i }'`` -are not, due to the extra space in the second expression. - -For example, given:: - - >>> def fn(lst): - ... lst[0] += 1 - ... return lst[0] - ... - >>> lst=[0] - >>> f'{fn(lst)} {fn(lst)}' - '1 2' - -The resulting f-string might have the value ``'1 2'``, ``'2 2'``, -``'1 1'``, or even ``'2 1'``. - -However:: - - >>> lst=[0] - >>> f'{fn(lst)} { fn(lst)}' - '1 2' - -This f-string will always have the value ``'1 2'``. This is due to the -two expressions not being the same: the space in the second example -makes the two expressions distinct. - -This restriction is in place in order to allow for a possible future -extension allowing translated strings, wherein the expression -substitutions would be identified by their text representations in the -f-strings. - References ========== -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Fri Sep 4 21:57:22 2015 From: python-checkins at python.org (guido.van.rossum) Date: Fri, 04 Sep 2015 19:57:22 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy41IC0+IDMuNSk6?= =?utf-8?b?IE51bGwgbWVyZ2UgKExhcnJ5J3MgMy41IC0+IDMuNS4xKS4=?= Message-ID: <20150904195722.14873.60140@psf.io> https://hg.python.org/cpython/rev/4e329892817c changeset: 97677:4e329892817c branch: 3.5 parent: 97674:d1f41c614e62 parent: 97676:438dde69871d user: Guido van Rossum date: Fri Sep 04 12:54:36 2015 -0700 summary: Null merge (Larry's 3.5 -> 3.5.1). files: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 4 21:57:22 2015 From: python-checkins at python.org (guido.van.rossum) Date: Fri, 04 Sep 2015 19:57:22 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogTnVsbCBtZXJnZSAoMy41LjEgLT4gZGVmYXVsdCku?= Message-ID: <20150904195722.14879.91222@psf.io> https://hg.python.org/cpython/rev/d4089ff1e656 changeset: 97678:d4089ff1e656 parent: 97675:0f37918440c9 parent: 97677:4e329892817c user: Guido van Rossum date: Fri Sep 04 12:55:13 2015 -0700 summary: Null merge (3.5.1 -> default). files: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 4 21:57:23 2015 From: python-checkins at python.org (guido.van.rossum) Date: Fri, 04 Sep 2015 19:57:23 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogRml4IGlzc3VlICMy?= =?utf-8?q?4635=2E?= Message-ID: <20150904195722.66868.90743@psf.io> https://hg.python.org/cpython/rev/438dde69871d changeset: 97676:438dde69871d branch: 3.5 parent: 97659:07e04c34bab5 user: Guido van Rossum date: Fri Sep 04 12:15:54 2015 -0700 summary: Fix issue #24635. files: Lib/test/test_typing.py | 17 ++++++++++++++--- Lib/typing.py | 15 ++++++++++----- Misc/NEWS | 3 +++ 3 files changed, 27 insertions(+), 8 deletions(-) diff --git a/Lib/test/test_typing.py b/Lib/test/test_typing.py --- a/Lib/test/test_typing.py +++ b/Lib/test/test_typing.py @@ -436,12 +436,14 @@ c() def test_callable_instance_works(self): - f = lambda: None + def f(): + pass assert isinstance(f, Callable) assert not isinstance(None, Callable) def test_callable_instance_type_error(self): - f = lambda: None + def f(): + pass with self.assertRaises(TypeError): assert isinstance(f, Callable[[], None]) with self.assertRaises(TypeError): @@ -674,7 +676,9 @@ T = TypeVar('T') class Node(Generic[T]): - def __init__(self, label: T, left: 'Node[T]' = None, right: 'Node[T]' = None): + def __init__(self, label: T, + left: 'Node[T]' = None, + right: 'Node[T]' = None): self.label = label # type: T self.left = left # type: Optional[Node[T]] self.right = right # type: Optional[Node[T]] @@ -934,8 +938,15 @@ def test_iterable(self): assert isinstance([], typing.Iterable) + # Due to ABC caching, the second time takes a separate code + # path and could fail. So call this a few times. + assert isinstance([], typing.Iterable) + assert isinstance([], typing.Iterable) assert isinstance([], typing.Iterable[int]) assert not isinstance(42, typing.Iterable) + # Just in case, also test issubclass() a few times. + assert issubclass(list, typing.Iterable) + assert issubclass(list, typing.Iterable) def test_iterator(self): it = iter([]) diff --git a/Lib/typing.py b/Lib/typing.py --- a/Lib/typing.py +++ b/Lib/typing.py @@ -1,7 +1,3 @@ -# TODO: -# - Generic[T, T] is invalid -# - Look for TODO below - # TODO nits: # Get rid of asserts that are the caller's fault. # Docstrings (e.g. ABCs). @@ -963,7 +959,8 @@ raise TypeError("Initial parameters must be " "type variables; got %s" % p) if len(set(params)) != len(params): - raise TypeError("All type variables in Generic[...] must be distinct.") + raise TypeError( + "All type variables in Generic[...] must be distinct.") else: if len(params) != len(self.__parameters__): raise TypeError("Cannot change parameter count from %d to %d" % @@ -987,6 +984,14 @@ origin=self, extra=self.__extra__) + def __instancecheck__(self, instance): + # Since we extend ABC.__subclasscheck__ and + # ABC.__instancecheck__ inlines the cache checking done by the + # latter, we must extend __instancecheck__ too. For simplicity + # we just skip the cache check -- instance checks for generic + # classes are supposed to be rare anyways. + return self.__subclasscheck__(instance.__class__) + def __subclasscheck__(self, cls): if cls is Any: return True diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -15,6 +15,9 @@ Library ------- +- Issue #24635: Fixed a bug in typing.py where isinstance([], typing.Iterable) + would return True once, then False on subsequent calls. + - Issue #24989: Fixed buffer overread in BytesIO.readline() if a position is set beyond size. Based on patch by John Leitch. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 4 21:57:22 2015 From: python-checkins at python.org (guido.van.rossum) Date: Fri, 04 Sep 2015 19:57:22 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2324635=3A_Fixed_flakiness_in_test=5Ftyping=2Epy?= =?utf-8?b?LiAoTWVyZ2UgZnJvbSAzLjUuKQ==?= Message-ID: <20150904195722.11264.24721@psf.io> https://hg.python.org/cpython/rev/0f37918440c9 changeset: 97675:0f37918440c9 parent: 97673:73911e6c97c8 parent: 97674:d1f41c614e62 user: Guido van Rossum date: Fri Sep 04 12:05:03 2015 -0700 summary: Issue #24635: Fixed flakiness in test_typing.py. (Merge from 3.5.) files: Lib/test/test_typing.py | 17 ++++++++++++++--- Lib/typing.py | 15 ++++++++++----- Misc/NEWS | 3 +++ 3 files changed, 27 insertions(+), 8 deletions(-) diff --git a/Lib/test/test_typing.py b/Lib/test/test_typing.py --- a/Lib/test/test_typing.py +++ b/Lib/test/test_typing.py @@ -436,12 +436,14 @@ c() def test_callable_instance_works(self): - f = lambda: None + def f(): + pass assert isinstance(f, Callable) assert not isinstance(None, Callable) def test_callable_instance_type_error(self): - f = lambda: None + def f(): + pass with self.assertRaises(TypeError): assert isinstance(f, Callable[[], None]) with self.assertRaises(TypeError): @@ -674,7 +676,9 @@ T = TypeVar('T') class Node(Generic[T]): - def __init__(self, label: T, left: 'Node[T]' = None, right: 'Node[T]' = None): + def __init__(self, label: T, + left: 'Node[T]' = None, + right: 'Node[T]' = None): self.label = label # type: T self.left = left # type: Optional[Node[T]] self.right = right # type: Optional[Node[T]] @@ -934,8 +938,15 @@ def test_iterable(self): assert isinstance([], typing.Iterable) + # Due to ABC caching, the second time takes a separate code + # path and could fail. So call this a few times. + assert isinstance([], typing.Iterable) + assert isinstance([], typing.Iterable) assert isinstance([], typing.Iterable[int]) assert not isinstance(42, typing.Iterable) + # Just in case, also test issubclass() a few times. + assert issubclass(list, typing.Iterable) + assert issubclass(list, typing.Iterable) def test_iterator(self): it = iter([]) diff --git a/Lib/typing.py b/Lib/typing.py --- a/Lib/typing.py +++ b/Lib/typing.py @@ -1,7 +1,3 @@ -# TODO: -# - Generic[T, T] is invalid -# - Look for TODO below - # TODO nits: # Get rid of asserts that are the caller's fault. # Docstrings (e.g. ABCs). @@ -963,7 +959,8 @@ raise TypeError("Initial parameters must be " "type variables; got %s" % p) if len(set(params)) != len(params): - raise TypeError("All type variables in Generic[...] must be distinct.") + raise TypeError( + "All type variables in Generic[...] must be distinct.") else: if len(params) != len(self.__parameters__): raise TypeError("Cannot change parameter count from %d to %d" % @@ -987,6 +984,14 @@ origin=self, extra=self.__extra__) + def __instancecheck__(self, instance): + # Since we extend ABC.__subclasscheck__ and + # ABC.__instancecheck__ inlines the cache checking done by the + # latter, we must extend __instancecheck__ too. For simplicity + # we just skip the cache check -- instance checks for generic + # classes are supposed to be rare anyways. + return self.__subclasscheck__(instance.__class__) + def __subclasscheck__(self, cls): if cls is Any: return True diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -172,6 +172,9 @@ Library ------- +- Issue #24635: Fixed a bug in typing.py where isinstance([], typing.Iterable) + would return True once, then False on subsequent calls. + - Issue #24989: Fixed buffer overread in BytesIO.readline() if a position is set beyond size. Based on patch by John Leitch. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 4 23:09:39 2015 From: python-checkins at python.org (eric.smith) Date: Fri, 04 Sep 2015 21:09:39 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_Add_notion_of_adding_implicit?= =?utf-8?q?_parens_before_parsing_the_expression=2E_This_allows?= Message-ID: <20150904210939.15734.70459@psf.io> https://hg.python.org/peps/rev/a4aa1437dcbf changeset: 6030:a4aa1437dcbf user: Eric V. Smith date: Fri Sep 04 17:09:56 2015 -0400 summary: Add notion of adding implicit parens before parsing the expression. This allows for newlines in the expressions. files: pep-0498.txt | 27 +++++++++++---------------- 1 files changed, 11 insertions(+), 16 deletions(-) diff --git a/pep-0498.txt b/pep-0498.txt --- a/pep-0498.txt +++ b/pep-0498.txt @@ -269,26 +269,21 @@ >>> 'result=' + str(foo()) 'result=20' -After stripping leading and trailing whitespace (see below), the -expression is parsed with the equivalent of ``ast.parse(expression, -'', 'eval')`` [#]_. Note that this restricts the expression: -it cannot contain any newlines, for example:: +Expressions are parsed with the equivalent of ``ast.parse('(' + +expression + ')', '', 'eval')`` [#]_. + +Note that since the expression is enclosed by implicit parentheses +before evaluation, expressions can contain newlines. For example:: >>> x = 0 >>> f'''{x ... +1}''' - File "", line 2 - +1 - ^ - SyntaxError: invalid syntax + '1' -But note that this works, since the newline is removed from the -string, and the spaces in front of the ``'1'`` are allowed in an -expression:: - - >>> f'{x+\ - ... 1}' - '2' + >>> d = {0: 'zero'} + >>> f'''{d[0 + ... ]}''' + 'zero' Format specifiers ----------------- @@ -374,7 +369,7 @@ File "", line 2, in ValueError: Sign not allowed in string format specifier -Leading and trailing whitespace in expressions is skipped +Leading and trailing whitespace in expressions is ignored --------------------------------------------------------- For ease of readability, leading and trailing whitespace in -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Sat Sep 5 00:09:26 2015 From: python-checkins at python.org (victor.stinner) Date: Fri, 04 Sep 2015 22:09:26 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2323517=3A_Fix_impl?= =?utf-8?q?ementation_of_the_ROUND=5FHALF=5FUP_rounding_mode_in?= Message-ID: <20150904220926.15706.68345@psf.io> https://hg.python.org/cpython/rev/3c29d05c0710 changeset: 97679:3c29d05c0710 user: Victor Stinner date: Fri Sep 04 23:57:25 2015 +0200 summary: Issue #23517: Fix implementation of the ROUND_HALF_UP rounding mode in datetime.datetime.fromtimestamp() and datetime.datetime.utcfromtimestamp(). microseconds sign should be kept before rounding. files: Lib/datetime.py | 50 +++++++++++-------------- Lib/test/datetimetester.py | 12 +++++- Lib/test/test_time.py | 13 +++--- Python/pytime.c | 8 ++-- 4 files changed, 43 insertions(+), 40 deletions(-) diff --git a/Lib/datetime.py b/Lib/datetime.py --- a/Lib/datetime.py +++ b/Lib/datetime.py @@ -1374,6 +1374,26 @@ return self._tzinfo @classmethod + def _fromtimestamp(cls, t, utc, tz): + """Construct a datetime from a POSIX timestamp (like time.time()). + + A timezone info object may be passed in as well. + """ + frac, t = _math.modf(t) + us = _round_half_up(frac * 1e6) + if us >= 1000000: + t += 1 + us -= 1000000 + elif us < 0: + t -= 1 + us += 1000000 + + converter = _time.gmtime if utc else _time.localtime + y, m, d, hh, mm, ss, weekday, jday, dst = converter(t) + ss = min(ss, 59) # clamp out leap seconds if the platform has them + return cls(y, m, d, hh, mm, ss, us, tz) + + @classmethod def fromtimestamp(cls, t, tz=None): """Construct a datetime from a POSIX timestamp (like time.time()). @@ -1381,21 +1401,7 @@ """ _check_tzinfo_arg(tz) - converter = _time.localtime if tz is None else _time.gmtime - - t, frac = divmod(t, 1.0) - us = _round_half_up(frac * 1e6) - - # If timestamp is less than one microsecond smaller than a - # full second, us can be rounded up to 1000000. In this case, - # roll over to seconds, otherwise, ValueError is raised - # by the constructor. - if us == 1000000: - t += 1 - us = 0 - y, m, d, hh, mm, ss, weekday, jday, dst = converter(t) - ss = min(ss, 59) # clamp out leap seconds if the platform has them - result = cls(y, m, d, hh, mm, ss, us, tz) + result = cls._fromtimestamp(t, tz is not None, tz) if tz is not None: result = tz.fromutc(result) return result @@ -1403,19 +1409,7 @@ @classmethod def utcfromtimestamp(cls, t): """Construct a naive UTC datetime from a POSIX timestamp.""" - t, frac = divmod(t, 1.0) - us = _round_half_up(frac * 1e6) - - # If timestamp is less than one microsecond smaller than a - # full second, us can be rounded up to 1000000. In this case, - # roll over to seconds, otherwise, ValueError is raised - # by the constructor. - if us == 1000000: - t += 1 - us = 0 - y, m, d, hh, mm, ss, weekday, jday, dst = _time.gmtime(t) - ss = min(ss, 59) # clamp out leap seconds if the platform has them - return cls(y, m, d, hh, mm, ss, us) + return cls._fromtimestamp(t, True, None) @classmethod def now(cls, tz=None): diff --git a/Lib/test/datetimetester.py b/Lib/test/datetimetester.py --- a/Lib/test/datetimetester.py +++ b/Lib/test/datetimetester.py @@ -668,6 +668,8 @@ eq(td(milliseconds=-0.6/1000), td(microseconds=-1)) eq(td(seconds=0.5/10**6), td(microseconds=1)) eq(td(seconds=-0.5/10**6), td(microseconds=-1)) + eq(td(seconds=1/2**7), td(microseconds=7813)) + eq(td(seconds=-1/2**7), td(microseconds=-7813)) # Rounding due to contributions from more than one field. us_per_hour = 3600e6 @@ -1842,8 +1844,8 @@ 18000 + 3600 + 2*60 + 3 + 4*1e-6) def test_microsecond_rounding(self): - for fts in [self.theclass.fromtimestamp, - self.theclass.utcfromtimestamp]: + for fts in (datetime.fromtimestamp, + self.theclass.utcfromtimestamp): zero = fts(0) self.assertEqual(zero.second, 0) self.assertEqual(zero.microsecond, 0) @@ -1874,6 +1876,12 @@ t = fts(0.9999999) self.assertEqual(t.second, 1) self.assertEqual(t.microsecond, 0) + t = fts(1/2**7) + self.assertEqual(t.second, 0) + self.assertEqual(t.microsecond, 7813) + t = fts(-1/2**7) + self.assertEqual(t.second, 59) + self.assertEqual(t.microsecond, 992187) def test_insane_fromtimestamp(self): # It's possible that some platform maps time_t to double, diff --git a/Lib/test/test_time.py b/Lib/test/test_time.py --- a/Lib/test/test_time.py +++ b/Lib/test/test_time.py @@ -655,7 +655,7 @@ pytime_object_to_time_t, invalid, rnd) @support.cpython_only - def test_timespec(self): + def test_object_to_timespec(self): from _testcapi import pytime_object_to_timespec # Conversion giving the same result for all rounding methods @@ -666,7 +666,7 @@ (-1, (-1, 0)), # float - (-1.2, (-2, 800000000)), + (-1/2**7, (-1, 992187500)), (-1.0, (-1, 0)), (-1e-9, (-1, 999999999)), (1e-9, (0, 1)), @@ -693,7 +693,7 @@ (1.1234567890, (1, 123456789), FLOOR), (1.1234567899, (1, 123456789), FLOOR), - (-1.1234567890, (-2, 876543211), FLOOR), + (-1.1234567890, (-2, 876543210), FLOOR), (-1.1234567891, (-2, 876543210), FLOOR), # Round towards infinity (+inf) (1.1234567890, (1, 123456790), CEILING), @@ -1155,7 +1155,7 @@ self.assertRaises(OverflowError, pytime_object_to_time_t, invalid, rnd) - def test_timeval(self): + def test_object_to_timeval(self): from _testcapi import pytime_object_to_timeval # Conversion giving the same result for all rounding methods @@ -1167,7 +1167,8 @@ # float (-1.0, (-1, 0)), - (-1.2, (-2, 800000)), + (1/2**6, (0, 15625)), + (-1/2**6, (-1, 984375)), (-1e-6, (-1, 999999)), (1e-6, (0, 1)), ): @@ -1225,7 +1226,7 @@ (-1.0, (-1, 0)), (-1e-9, (-1, 999999999)), (1e-9, (0, 1)), - (-1.2, (-2, 800000000)), + (-1/2**9, (-1, 998046875)), ): with self.subTest(obj=obj, round=rnd, timespec=timespec): self.assertEqual(pytime_object_to_timespec(obj, rnd), diff --git a/Python/pytime.c b/Python/pytime.c --- a/Python/pytime.c +++ b/Python/pytime.c @@ -82,10 +82,6 @@ volatile double floatpart; floatpart = modf(d, &intpart); - if (floatpart < 0) { - floatpart += 1.0; - intpart -= 1.0; - } floatpart *= denominator; if (round == _PyTime_ROUND_HALF_UP) @@ -98,6 +94,10 @@ floatpart -= denominator; intpart += 1.0; } + else if (floatpart < 0) { + floatpart += denominator; + intpart -= 1.0; + } assert(0.0 <= floatpart && floatpart < denominator); *sec = (time_t)intpart; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 5 02:28:17 2015 From: python-checkins at python.org (eric.smith) Date: Sat, 05 Sep 2015 00:28:17 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_Note_how_f-strings_are_tokeni?= =?utf-8?q?zed_and_decoded_before_scanning_for_expressions=2E?= Message-ID: <20150905002817.101502.72202@psf.io> https://hg.python.org/peps/rev/b8edd3309920 changeset: 6031:b8edd3309920 user: Eric V. Smith date: Fri Sep 04 20:28:35 2015 -0400 summary: Note how f-strings are tokenized and decoded before scanning for expressions. files: pep-0498.txt | 43 ++++++++++++++++++++++++++++++++------- 1 files changed, 35 insertions(+), 8 deletions(-) diff --git a/pep-0498.txt b/pep-0498.txt --- a/pep-0498.txt +++ b/pep-0498.txt @@ -174,14 +174,30 @@ binary f-strings. 'f' may also be combined with 'u', in either order, although adding 'u' has no effect. -f-strings are parsed in to literals and expressions. Expressions -appear within curly braces '{' and '}. The parts of the string outside -of braces are literals. The expressions are evaluated, formatted with -the existing __format__ protocol, then the results are concatenated -together with the string literals. While scanning the string for -expressions, any doubled braces '{{' or '}}' are replaced by the -corresponding single brace. Doubled opening braces do not signify the -start of an expression. +f-strings are tokenized using the same rules as normal strings, raw +strings, binary strings, and triple quoted strings. That is, the +string must end with the same character that it started with: if it +starts with a single quote it must end with a single quote, etc. This +implies that any code that currently scans Python code looking for +strings should be trivially modifiable to recognize f-strings (parsing +within an f-string is another matter, of course). + +Once tokenized, f-strings are decoded. This will convert backslash +escapes such as ``\n``, ``\xhh``, ``\uxxxx``, ``\Uxxxxxxxx``, and +named unicode characters ``\N{name}`` into their associated Unicode +characters [#]_. + +Up to this point, the processing of f-strings and normal strings is +exactly the same. + +The difference is that f-strings are then parsed in to literals and +expressions. Expressions appear within curly braces '{' and '}. The +parts of the string outside of braces are literals. The expressions +are evaluated, formatted with the existing __format__ protocol, then +the results are concatenated together with the string literals. While +scanning the string for expressions, any doubled braces '{{' or '}}' +are replaced by the corresponding single brace. Doubled opening braces +do not signify the start of an expression. Following the expression, an optional type conversion may be specified. The allowed conversions are ``'!s'``, ``'!r'``, or @@ -228,6 +244,14 @@ escape sequences are processed before f-strings are parsed for expressions. +Note that the correct way to have a literal brace appear in the +resulting string value is to double the brace:: + + >>> f'{{ {4*10} }}' + '{ 40 }' + >>> f'{{{4*10}}}' + '{40}' + Code equivalence ---------------- @@ -659,6 +683,9 @@ .. [#] Format string syntax (https://docs.python.org/3/library/string.html#format-string-syntax) +.. [#] String literal description + (https://docs.python.org/3/reference/lexical_analysis.html#string-and-bytes-literals) + .. [#] ast.parse() documentation (https://docs.python.org/3/library/ast.html#ast.parse) -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Sat Sep 5 02:29:37 2015 From: python-checkins at python.org (eric.smith) Date: Sat, 05 Sep 2015 00:29:37 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_Update_Post-History=2E?= Message-ID: <20150905002937.68875.58047@psf.io> https://hg.python.org/peps/rev/0a9837160f77 changeset: 6032:0a9837160f77 user: Eric V. Smith date: Fri Sep 04 20:29:54 2015 -0400 summary: Update Post-History. files: pep-0498.txt | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/pep-0498.txt b/pep-0498.txt --- a/pep-0498.txt +++ b/pep-0498.txt @@ -8,7 +8,7 @@ Content-Type: text/x-rst Created: 01-Aug-2015 Python-Version: 3.6 -Post-History: 07-Aug-2015, 30-Aug-2015 +Post-History: 07-Aug-2015, 30-Aug-2015, 04-Sep-2015 Abstract ======== -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Sat Sep 5 03:04:05 2015 From: python-checkins at python.org (eric.smith) Date: Sat, 05 Sep 2015 01:04:05 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_Remove_unused_footnotes=2E?= Message-ID: <20150905010404.101490.41627@psf.io> https://hg.python.org/peps/rev/2c831b879335 changeset: 6033:2c831b879335 user: Eric V. Smith date: Fri Sep 04 21:04:21 2015 -0400 summary: Remove unused footnotes. files: pep-0498.txt | 6 ------ 1 files changed, 0 insertions(+), 6 deletions(-) diff --git a/pep-0498.txt b/pep-0498.txt --- a/pep-0498.txt +++ b/pep-0498.txt @@ -677,12 +677,6 @@ .. [#] Avoid locals() and globals() (https://mail.python.org/pipermail/python-ideas/2015-July/034701.html) -.. [#] str.format_map() documentation - (https://docs.python.org/3/library/stdtypes.html#str.format_map) - -.. [#] Format string syntax - (https://docs.python.org/3/library/string.html#format-string-syntax) - .. [#] String literal description (https://docs.python.org/3/reference/lexical_analysis.html#string-and-bytes-literals) -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Sat Sep 5 03:12:00 2015 From: python-checkins at python.org (donald.stufft) Date: Sat, 05 Sep 2015 01:12:00 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_Add_PEP_503_-_Simple_Reposito?= =?utf-8?q?ry_API?= Message-ID: <20150905011200.17987.56954@psf.io> https://hg.python.org/peps/rev/f8ef87655f9d changeset: 6034:f8ef87655f9d user: Donald Stufft date: Fri Sep 04 21:11:56 2015 -0400 summary: Add PEP 503 - Simple Repository API files: pep-0503.txt | 114 +++++++++++++++++++++++++++++++++++++++ 1 files changed, 114 insertions(+), 0 deletions(-) diff --git a/pep-0503.txt b/pep-0503.txt new file mode 100644 --- /dev/null +++ b/pep-0503.txt @@ -0,0 +1,114 @@ +PEP: 503 +Title: Simple Repository API +Version: $Revision$ +Last-Modified: $Date$ +Author: Donald Stufft +BDFL-Delegate: Donald Stufft +Discussions-To: distutils-sig at python.org +Status: Draft +Type: Informational +Content-Type: text/x-rst +Created: 04-Sep-2015 +Post-History: 04-Sep-2015 + + +Abstract +======== + +There are many implementations of a Python package repository and many tools +that consume them. Of these, the cannonical implementation that defines what +the "simple" repository API looks like is the implementation that powers +PyPI. This document will specify that API, documenting what the correct +behavior for any implementation of the simple repository API. + + +Specification +============= + +A repository that implements the simple API is defined by its base url, this is +the top level URL that all additional URLS are below. The API is named the +"simple" repository due to fact that PyPI's base URL is +``https://pypi.python.org/simple/``. + +.. note:: All subsequent URLs in this document will be relative to this base + URL (so given PyPI's URL, an URL of ``/foo/`` would be + ``https://pypi.python.org/simple/foo/``). + + +Within a repository, the root URL (``/``) **MUST** be a valid HTML5 page with a +single anchor element per project in the repository. The text of the anchor tag +**MUST** be the normalized name of the project and the href attribute **MUST** +link to the URL for that particular project. As an example:: + + + + + frob + spamspamspam + + + +Below the root URL is another URL for each individual project contained within +a repository. The format of this URL is ``//`` where the ```` +is replaced by the normalized name for that project, so a project named +"HolyGrail" would have an URL like ``/holygrail/``. This URL must response with +a valid HTML5 page with a single anchor element per file for the project. The +text of the anchor tag **MUST** be the filename of the file and the href +attribute **MUST** be an URL that links to the location of the file for +download. The URL **SHOULD** include a hash in the form of an URL fragment with +the following syntax: ``#=``, where ```` is the +lowercase name of the hash function (such as ``sha256``) and ```` is +the hex encoded digest. + +In addition to the above, the following constraints are placed on the API: + +* All URLs **MUST** end with a ``/`` and the repository **SHOULD** redirect the + URLs without a ``/`` to add a ``/`` to the end. + +* There is no constraints on where the files must be hosted relative to the + repository. + +* There may be any other HTML elements on the API pages as long as the required + anchor elements exist. + +* Repositories **MAY** redirect unnormalized URLs to the cannonical normalized + URL (e.g. ``/Foobar/`` may redirect to ``/foobar/``), however clients + **MUST NOT** rely on this redirection and **MUST** request the normalized + URL. + +* Repositories **SHOULD** choose a hash function from one of the ones + guarenteed to be available via the ``hashlib`` module in the Python standard + library (currently ``md5``, ``sha1``, ``sha224``, ``sha256``, ``sha384``, + ``sha512``). The current recommendation is to use ``sha256``. + + +Normalized Names +---------------- + +This PEP references the concept of a "normalized" project name. As per PEP 426 +the only valid characters in a name are the ASCII alphabet, ASCII numbers, +``.``, ``-``, and ``_``. The name should be lowercased with all runs of the +characters ``.``, ``-``, or ``_`` replaced with a single ``-`` character. This +can be implemented in Python with the ``re`` module:: + + import re + + def normalize(name): + return re.sub(r"[-_.]+", "-", name).lower() + + +Copyright +========= + +This document has been placed in the public domain. + + + +.. + Local Variables: + mode: indented-text + indent-tabs-mode: nil + sentence-end-double-space: t + fill-column: 70 + coding: utf-8 + End: -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Sat Sep 5 03:26:11 2015 From: python-checkins at python.org (eric.smith) Date: Sat, 05 Sep 2015 01:26:11 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_Removed_unneeded_footnotes_re?= =?utf-8?q?ferencing_other_PEPs=2E_Cleaned_up_some_language=2E?= Message-ID: <20150905012611.11256.69643@psf.io> https://hg.python.org/peps/rev/b1879c55a410 changeset: 6035:b1879c55a410 parent: 6033:2c831b879335 user: Eric V. Smith date: Fri Sep 04 21:25:59 2015 -0400 summary: Removed unneeded footnotes referencing other PEPs. Cleaned up some language. Fixed up some formatting errors. files: pep-0498.txt | 80 +++++++++++++++++++--------------------- 1 files changed, 38 insertions(+), 42 deletions(-) diff --git a/pep-0498.txt b/pep-0498.txt --- a/pep-0498.txt +++ b/pep-0498.txt @@ -28,7 +28,7 @@ f-strings provide a way to embed expressions inside string literals, using a minimal syntax. It should be noted that an f-string is really an expression evaluated at run time, not a constant value. In Python -source code, an f-string is a literal string, prefixed with 'f', that +source code, an f-string is a literal string, prefixed with 'f', which contains expressions inside braces. The expressions are replaced with their values. Some examples are:: @@ -41,10 +41,10 @@ >>> f'He said his name is {name!r}.' "He said his name is 'Fred'." -A similar feature was proposed in PEP 215 [#]_. PEP 215 proposed to -support a subset of Python expressions, and did not support the -type-specific string formatting (the ``__format__()`` method) which -was introduced with PEP 3101 [#]_. +A similar feature was proposed in PEP 215. PEP 215 proposed to support +a subset of Python expressions, and did not support the type-specific +string formatting (the ``__format__()`` method) which was introduced +with PEP 3101. Rationale ========= @@ -79,7 +79,7 @@ %-formatting. In particular, it uses normal function call syntax (and therefor supports multiple parameters) and it is extensible through the ``__format__()`` method on the object being converted to a -string. See PEP-3101 for a detailed rationale. This PEP reuses much of +string. See PEP 3101 for a detailed rationale. This PEP reuses much of the ``str.format()`` syntax and machinery, in order to provide continuity with an existing Python string formatting mechanism. @@ -90,9 +90,9 @@ >>> 'The value is {value}.'.format(value=value) 'The value is 80.' -Even in its simplest form, there is a bit of boilerplate, and the -value that's inserted into the placeholder is sometimes far removed -from where the placeholder is situated:: +Even in its simplest form there is a bit of boilerplate, and the value +that's inserted into the placeholder is sometimes far removed from +where the placeholder is situated:: >>> 'The value is {}.'.format(value) 'The value is 80.' @@ -170,34 +170,35 @@ In source code, f-strings are string literals that are prefixed by the letter 'f'. 'f' may be combined with 'r', in either order, to produce -raw f-string literals. 'f' may not be combined with 'b': there are no -binary f-strings. 'f' may also be combined with 'u', in either order, -although adding 'u' has no effect. +raw f-string literals. 'f' may not be combined with 'b': this PEP does +not propose to add binary f-strings. 'f' may also be combined with +'u', in either order, although adding 'u' has no effect. -f-strings are tokenized using the same rules as normal strings, raw -strings, binary strings, and triple quoted strings. That is, the -string must end with the same character that it started with: if it -starts with a single quote it must end with a single quote, etc. This -implies that any code that currently scans Python code looking for -strings should be trivially modifiable to recognize f-strings (parsing -within an f-string is another matter, of course). +When tokenizing source files, f-strings use the same rules as normal +strings, raw strings, binary strings, and triple quoted strings. That +is, the string must end with the same character that it started with: +if it starts with a single quote it must end with a single quote, etc. +This implies that any code that currently scans Python code looking +for strings should be trivially modifiable to recognize f-strings +(parsing within an f-string is another matter, of course). Once tokenized, f-strings are decoded. This will convert backslash -escapes such as ``\n``, ``\xhh``, ``\uxxxx``, ``\Uxxxxxxxx``, and -named unicode characters ``\N{name}`` into their associated Unicode -characters [#]_. +escapes such as ``'\n'``, ``'\xhh'``, ``'\uxxxx'``, ``'\Uxxxxxxxx'``, +and named unicode characters ``'\N{name}'`` into their associated +Unicode characters [#]_. Up to this point, the processing of f-strings and normal strings is exactly the same. -The difference is that f-strings are then parsed in to literals and -expressions. Expressions appear within curly braces '{' and '}. The -parts of the string outside of braces are literals. The expressions -are evaluated, formatted with the existing __format__ protocol, then -the results are concatenated together with the string literals. While -scanning the string for expressions, any doubled braces '{{' or '}}' -are replaced by the corresponding single brace. Doubled opening braces -do not signify the start of an expression. +The difference is that f-strings are further parsed in to literals and +expressions. Expressions appear within curly braces ``'{'`` and +``'}'``. The parts of the string outside of braces are literals. The +expressions are evaluated, formatted with the existing __format__ +protocol, then the results are concatenated together with the string +literals. While scanning the string for expressions, any doubled +braces ``'{{'`` or ``'}}'`` are replaced by the corresponding single +brace. Doubled opening braces do not signify the start of an +expression. Following the expression, an optional type conversion may be specified. The allowed conversions are ``'!s'``, ``'!r'``, or @@ -344,7 +345,7 @@ While the exact method of this run time concatenation is unspecified, the above code might evaluate to:: - 'ab' + x.__format__('') + '{c}' + 'str<', y.__format__('^4') + 'de' + 'ab' + x.__format__('') + '{c}' + 'str<' + y.__format__('^4') + 'de' Error handling -------------- @@ -463,8 +464,8 @@ ``str.format()``. Another proposed alternative was to have the substituted text between -``\\{`` and ``}`` or between ``\\{`` and ``\\}``. While this syntax -would probably be desirable if all string literals were to support +``\{`` and ``}`` or between ``\{`` and ``\}``. While this syntax would +probably be desirable if all string literals were to support interpolation, this PEP only supports strings that are already marked with the leading ``'f'``. As such, the PEP is using unadorned braces to denoted substituted text, in order to leverage end user familiarity @@ -493,7 +494,7 @@ f-strings, this PEP takes the position that such uses should be addressed in a linter or code review:: - >>> f'mapping is { {a:b for (a, b) in ((1, 2), (3, 4))}}' + >>> f'mapping is { {a:b for (a, b) in ((1, 2), (3, 4))} }' 'mapping is {1: 2, 3: 4}' Similar support in other languages @@ -548,8 +549,9 @@ ----------------------- Triple quoted f-strings are allowed. These strings are parsed just as -normal triple-quoted strings are. After parsing, the normal f-string -logic is applied, and ``__format__()`` on each value is called. +normal triple-quoted strings are. After parsing and decoding, the +normal f-string logic is applied, and ``__format__()`` on each value +is called. Raw f-strings ------------- @@ -665,12 +667,6 @@ .. [#] string.Template documentation (https://docs.python.org/3/library/string.html#template-strings) -.. [#] PEP 215: String Interpolation - (https://www.python.org/dev/peps/pep-0215/) - -.. [#] PEP 3101: Advanced String Formatting - (https://www.python.org/dev/peps/pep-3101/) - .. [#] Formatting using locals() and globals() (https://mail.python.org/pipermail/python-ideas/2015-July/034671.html) -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Sat Sep 5 03:26:11 2015 From: python-checkins at python.org (eric.smith) Date: Sat, 05 Sep 2015 01:26:11 +0000 Subject: [Python-checkins] =?utf-8?q?peps_=28merge_default_-=3E_default=29?= =?utf-8?q?=3A_Merge_heads=2E?= Message-ID: <20150905012611.14855.7126@psf.io> https://hg.python.org/peps/rev/bd767f6451c9 changeset: 6036:bd767f6451c9 parent: 6035:b1879c55a410 parent: 6034:f8ef87655f9d user: Eric V. Smith date: Fri Sep 04 21:26:29 2015 -0400 summary: Merge heads. files: pep-0503.txt | 114 +++++++++++++++++++++++++++++++++++++++ 1 files changed, 114 insertions(+), 0 deletions(-) diff --git a/pep-0503.txt b/pep-0503.txt new file mode 100644 --- /dev/null +++ b/pep-0503.txt @@ -0,0 +1,114 @@ +PEP: 503 +Title: Simple Repository API +Version: $Revision$ +Last-Modified: $Date$ +Author: Donald Stufft +BDFL-Delegate: Donald Stufft +Discussions-To: distutils-sig at python.org +Status: Draft +Type: Informational +Content-Type: text/x-rst +Created: 04-Sep-2015 +Post-History: 04-Sep-2015 + + +Abstract +======== + +There are many implementations of a Python package repository and many tools +that consume them. Of these, the cannonical implementation that defines what +the "simple" repository API looks like is the implementation that powers +PyPI. This document will specify that API, documenting what the correct +behavior for any implementation of the simple repository API. + + +Specification +============= + +A repository that implements the simple API is defined by its base url, this is +the top level URL that all additional URLS are below. The API is named the +"simple" repository due to fact that PyPI's base URL is +``https://pypi.python.org/simple/``. + +.. note:: All subsequent URLs in this document will be relative to this base + URL (so given PyPI's URL, an URL of ``/foo/`` would be + ``https://pypi.python.org/simple/foo/``). + + +Within a repository, the root URL (``/``) **MUST** be a valid HTML5 page with a +single anchor element per project in the repository. The text of the anchor tag +**MUST** be the normalized name of the project and the href attribute **MUST** +link to the URL for that particular project. As an example:: + + + + + frob + spamspamspam + + + +Below the root URL is another URL for each individual project contained within +a repository. The format of this URL is ``//`` where the ```` +is replaced by the normalized name for that project, so a project named +"HolyGrail" would have an URL like ``/holygrail/``. This URL must response with +a valid HTML5 page with a single anchor element per file for the project. The +text of the anchor tag **MUST** be the filename of the file and the href +attribute **MUST** be an URL that links to the location of the file for +download. The URL **SHOULD** include a hash in the form of an URL fragment with +the following syntax: ``#=``, where ```` is the +lowercase name of the hash function (such as ``sha256``) and ```` is +the hex encoded digest. + +In addition to the above, the following constraints are placed on the API: + +* All URLs **MUST** end with a ``/`` and the repository **SHOULD** redirect the + URLs without a ``/`` to add a ``/`` to the end. + +* There is no constraints on where the files must be hosted relative to the + repository. + +* There may be any other HTML elements on the API pages as long as the required + anchor elements exist. + +* Repositories **MAY** redirect unnormalized URLs to the cannonical normalized + URL (e.g. ``/Foobar/`` may redirect to ``/foobar/``), however clients + **MUST NOT** rely on this redirection and **MUST** request the normalized + URL. + +* Repositories **SHOULD** choose a hash function from one of the ones + guarenteed to be available via the ``hashlib`` module in the Python standard + library (currently ``md5``, ``sha1``, ``sha224``, ``sha256``, ``sha384``, + ``sha512``). The current recommendation is to use ``sha256``. + + +Normalized Names +---------------- + +This PEP references the concept of a "normalized" project name. As per PEP 426 +the only valid characters in a name are the ASCII alphabet, ASCII numbers, +``.``, ``-``, and ``_``. The name should be lowercased with all runs of the +characters ``.``, ``-``, or ``_`` replaced with a single ``-`` character. This +can be implemented in Python with the ``re`` module:: + + import re + + def normalize(name): + return re.sub(r"[-_.]+", "-", name).lower() + + +Copyright +========= + +This document has been placed in the public domain. + + + +.. + Local Variables: + mode: indented-text + indent-tabs-mode: nil + sentence-end-double-space: t + fill-column: 70 + coding: utf-8 + End: -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Sat Sep 5 03:41:03 2015 From: python-checkins at python.org (donald.stufft) Date: Sat, 05 Sep 2015 01:41:03 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_Remove_a_stray_=29?= Message-ID: <20150905014103.114789.82002@psf.io> https://hg.python.org/peps/rev/b11e5e6f0b0b changeset: 6037:b11e5e6f0b0b user: Donald Stufft date: Fri Sep 04 21:41:00 2015 -0400 summary: Remove a stray ) files: pep-0503.txt | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/pep-0503.txt b/pep-0503.txt --- a/pep-0503.txt +++ b/pep-0503.txt @@ -32,7 +32,7 @@ .. note:: All subsequent URLs in this document will be relative to this base URL (so given PyPI's URL, an URL of ``/foo/`` would be - ``https://pypi.python.org/simple/foo/``). + ``https://pypi.python.org/simple/foo/``. Within a repository, the root URL (``/``) **MUST** be a valid HTML5 page with a -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Sat Sep 5 10:50:37 2015 From: python-checkins at python.org (victor.stinner) Date: Sat, 05 Sep 2015 08:50:37 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2323517=3A_Skip_a_d?= =?utf-8?q?atetime_test_on_Windows?= Message-ID: <20150905085037.11272.40272@psf.io> https://hg.python.org/cpython/rev/d755f75019c8 changeset: 97680:d755f75019c8 user: Victor Stinner date: Sat Sep 05 10:50:20 2015 +0200 summary: Issue #23517: Skip a datetime test on Windows The test calls gmtime(-1)/localtime(-1) which is not supported on Windows. files: Lib/test/datetimetester.py | 6 +++--- 1 files changed, 3 insertions(+), 3 deletions(-) diff --git a/Lib/test/datetimetester.py b/Lib/test/datetimetester.py --- a/Lib/test/datetimetester.py +++ b/Lib/test/datetimetester.py @@ -1865,6 +1865,9 @@ self.assertEqual(t, minus_one) t = fts(-1e-7) self.assertEqual(t, zero) + t = fts(-1/2**7) + self.assertEqual(t.second, 59) + self.assertEqual(t.microsecond, 992187) t = fts(1e-7) self.assertEqual(t, zero) @@ -1879,9 +1882,6 @@ t = fts(1/2**7) self.assertEqual(t.second, 0) self.assertEqual(t.microsecond, 7813) - t = fts(-1/2**7) - self.assertEqual(t.second, 59) - self.assertEqual(t.microsecond, 992187) def test_insane_fromtimestamp(self): # It's possible that some platform maps time_t to double, -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 5 14:07:23 2015 From: python-checkins at python.org (nick.coghlan) Date: Sat, 05 Sep 2015 12:07:23 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_Defer_PEP_501?= Message-ID: <20150905120722.11999.5503@psf.io> https://hg.python.org/peps/rev/e03d6a03d0fc changeset: 6038:e03d6a03d0fc user: Nick Coghlan date: Sat Sep 05 22:07:14 2015 +1000 summary: Defer PEP 501 files: pep-0501.txt | 11 ++++++++++- 1 files changed, 10 insertions(+), 1 deletions(-) diff --git a/pep-0501.txt b/pep-0501.txt --- a/pep-0501.txt +++ b/pep-0501.txt @@ -3,7 +3,7 @@ Version: $Revision$ Last-Modified: $Date$ Author: Nick Coghlan -Status: Draft +Status: Deferred Type: Standards Track Content-Type: text/x-rst Requires: 498 @@ -43,6 +43,15 @@ myresponse = html(i"{response.body}") logging.debug(i"Message with {detailed} {debugging} {info}") + +PEP Deferral +============ + +This PEP is currently deferred pending further experience with PEP 498's +simpler approach of only supporting eager rendering without the additional +complexity of also supporting deferred rendering. + + Summary of differences from PEP 498 =================================== -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Sat Sep 5 18:11:56 2015 From: python-checkins at python.org (donald.stufft) Date: Sat, 05 Sep 2015 16:11:56 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_PEP_503=3A_Fix_typos=2C_add_c?= =?utf-8?q?larification=2C_and_add_the_missing_GPG_signatures?= Message-ID: <20150905161155.12011.59126@psf.io> https://hg.python.org/peps/rev/a314911efa10 changeset: 6039:a314911efa10 user: Donald Stufft date: Sat Sep 05 12:11:45 2015 -0400 summary: PEP 503: Fix typos, add clarification, and add the missing GPG signatures files: pep-0503.txt | 20 +++++++++++++++----- 1 files changed, 15 insertions(+), 5 deletions(-) diff --git a/pep-0503.txt b/pep-0503.txt --- a/pep-0503.txt +++ b/pep-0503.txt @@ -16,7 +16,7 @@ ======== There are many implementations of a Python package repository and many tools -that consume them. Of these, the cannonical implementation that defines what +that consume them. Of these, the canonical implementation that defines what the "simple" repository API looks like is the implementation that powers PyPI. This document will specify that API, documenting what the correct behavior for any implementation of the simple repository API. @@ -51,7 +51,7 @@ Below the root URL is another URL for each individual project contained within a repository. The format of this URL is ``//`` where the ```` is replaced by the normalized name for that project, so a project named -"HolyGrail" would have an URL like ``/holygrail/``. This URL must response with +"HolyGrail" would have an URL like ``/holygrail/``. This URL must respond with a valid HTML5 page with a single anchor element per file for the project. The text of the anchor tag **MUST** be the filename of the file and the href attribute **MUST** be an URL that links to the location of the file for @@ -62,8 +62,12 @@ In addition to the above, the following constraints are placed on the API: -* All URLs **MUST** end with a ``/`` and the repository **SHOULD** redirect the - URLs without a ``/`` to add a ``/`` to the end. +* All URLs which respond with an HTML5 page **MUST** end with a ``/`` and the + repository **SHOULD** redirect the URLs without a ``/`` to add a ``/`` to the + end. + +* URLs may be either absolute or relative as long as they point to the correct + location. * There is no constraints on where the files must be hosted relative to the repository. @@ -77,10 +81,16 @@ URL. * Repositories **SHOULD** choose a hash function from one of the ones - guarenteed to be available via the ``hashlib`` module in the Python standard + guaranteed to be available via the ``hashlib`` module in the Python standard library (currently ``md5``, ``sha1``, ``sha224``, ``sha256``, ``sha384``, ``sha512``). The current recommendation is to use ``sha256``. +* If there is a GPG signature for a particular distribution file it **MUST** + live alongside that file with the same name with a ``.asc`` appended to it. + So if the file ``/packages/HolyGrail-1.0.tar.gz`` existed and had an + associated signature, the signature would be located at + ``/packages/HolyGrail-1.0.tar.gz.asc``. + Normalized Names ---------------- -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Sat Sep 5 19:50:30 2015 From: python-checkins at python.org (barry.warsaw) Date: Sat, 05 Sep 2015 17:50:30 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_Spell_checked=2E?= Message-ID: <20150905175029.15730.92415@psf.io> https://hg.python.org/peps/rev/6d0a3598d22f changeset: 6040:6d0a3598d22f user: Barry Warsaw date: Sat Sep 05 13:50:27 2015 -0400 summary: Spell checked. files: pep-0498.txt | 10 +++++----- 1 files changed, 5 insertions(+), 5 deletions(-) diff --git a/pep-0498.txt b/pep-0498.txt --- a/pep-0498.txt +++ b/pep-0498.txt @@ -18,7 +18,7 @@ [#]_. Each of these methods have their advantages, but in addition have disadvantages that make them cumbersome to use in practice. This PEP proposed to add a new string formatting mechanism: Literal String -Interpolation. In this PEP, such strings will be refered to as +Interpolation. In this PEP, such strings will be referred to as "f-strings", taken from the leading character used to denote such strings, and standing for "formatted strings". @@ -212,7 +212,7 @@ Similar to ``str.format()``, optional format specifiers maybe be included inside the f-string, separated from the expression (or the type conversion, if specified) by a colon. If a format specifier is -not provied, an empty string is used. +not provided, an empty string is used. So, an f-string looks like:: @@ -223,7 +223,7 @@ of the f-string. Expressions cannot contain ``':'`` or ``'!'`` outside of strings or -parens, brackets, or braces. The exception is that the ``'!='`` +parentheses, brackets, or braces. The exception is that the ``'!='`` operator is allowed as a special case. Escape sequences @@ -437,7 +437,7 @@ Because the compiler must be involved in evaluating the expressions contained in the interpolated strings, there must be some way to denote to the compiler which strings should be evaluated. This PEP -chose a leading ``'f'`` character preceeding the string literal. This +chose a leading ``'f'`` character preceding the string literal. This is similar to how ``'b'`` and ``'r'`` prefixes change the meaning of the string itself, at compile time. Other prefixes were suggested, such as ``'i'``. No option seemed better than the other, so ``'f'`` @@ -621,7 +621,7 @@ practical use for a plain lambda in an f-string expression, this is not seen as much of a limitation. -If you feel you must use lambdas, they may be used inside of parens:: +If you feel you must use lambdas, they may be used inside of parentheses:: >>> f'{(lambda x: x*2)(3)}' '6' -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Sat Sep 5 21:48:07 2015 From: python-checkins at python.org (steve.dower) Date: Sat, 05 Sep 2015 19:48:07 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI0OTEw?= =?utf-8?q?=3A_Windows_MSIs_now_have_unique_display_names=2E?= Message-ID: <20150905194806.101496.42856@psf.io> https://hg.python.org/cpython/rev/3594400f3202 changeset: 97681:3594400f3202 branch: 3.5 parent: 97677:4e329892817c user: Steve Dower date: Sat Sep 05 12:47:06 2015 -0700 summary: Issue #24910: Windows MSIs now have unique display names. files: Misc/NEWS | 2 ++ Tools/msi/core/core_d.wxs | 2 +- Tools/msi/core/core_pdb.wxs | 2 +- Tools/msi/dev/dev_d.wxs | 2 +- Tools/msi/exe/exe_d.wxs | 2 +- Tools/msi/exe/exe_pdb.wxs | 2 +- Tools/msi/lib/lib_d.wxs | 2 +- Tools/msi/lib/lib_pdb.wxs | 2 +- Tools/msi/tcltk/tcltk_d.wxs | 2 +- Tools/msi/tcltk/tcltk_pdb.wxs | 2 +- Tools/msi/test/test_d.wxs | 2 +- Tools/msi/test/test_pdb.wxs | 2 +- 12 files changed, 13 insertions(+), 11 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -71,6 +71,8 @@ Build ----- +- Issue #24910: Windows MSIs now have unique display names. + - Issue #24986: It is now possible to build Python on Windows without errors when external libraries are not available. diff --git a/Tools/msi/core/core_d.wxs b/Tools/msi/core/core_d.wxs --- a/Tools/msi/core/core_d.wxs +++ b/Tools/msi/core/core_d.wxs @@ -1,6 +1,6 @@ ? - + diff --git a/Tools/msi/core/core_pdb.wxs b/Tools/msi/core/core_pdb.wxs --- a/Tools/msi/core/core_pdb.wxs +++ b/Tools/msi/core/core_pdb.wxs @@ -1,6 +1,6 @@ ? - + diff --git a/Tools/msi/dev/dev_d.wxs b/Tools/msi/dev/dev_d.wxs --- a/Tools/msi/dev/dev_d.wxs +++ b/Tools/msi/dev/dev_d.wxs @@ -1,6 +1,6 @@ ? - + diff --git a/Tools/msi/exe/exe_d.wxs b/Tools/msi/exe/exe_d.wxs --- a/Tools/msi/exe/exe_d.wxs +++ b/Tools/msi/exe/exe_d.wxs @@ -1,6 +1,6 @@ ? - + diff --git a/Tools/msi/exe/exe_pdb.wxs b/Tools/msi/exe/exe_pdb.wxs --- a/Tools/msi/exe/exe_pdb.wxs +++ b/Tools/msi/exe/exe_pdb.wxs @@ -1,6 +1,6 @@ ? - + diff --git a/Tools/msi/lib/lib_d.wxs b/Tools/msi/lib/lib_d.wxs --- a/Tools/msi/lib/lib_d.wxs +++ b/Tools/msi/lib/lib_d.wxs @@ -1,6 +1,6 @@ ? - + diff --git a/Tools/msi/lib/lib_pdb.wxs b/Tools/msi/lib/lib_pdb.wxs --- a/Tools/msi/lib/lib_pdb.wxs +++ b/Tools/msi/lib/lib_pdb.wxs @@ -1,6 +1,6 @@ ? - + diff --git a/Tools/msi/tcltk/tcltk_d.wxs b/Tools/msi/tcltk/tcltk_d.wxs --- a/Tools/msi/tcltk/tcltk_d.wxs +++ b/Tools/msi/tcltk/tcltk_d.wxs @@ -1,6 +1,6 @@ ? - + diff --git a/Tools/msi/tcltk/tcltk_pdb.wxs b/Tools/msi/tcltk/tcltk_pdb.wxs --- a/Tools/msi/tcltk/tcltk_pdb.wxs +++ b/Tools/msi/tcltk/tcltk_pdb.wxs @@ -1,6 +1,6 @@ ? - + diff --git a/Tools/msi/test/test_d.wxs b/Tools/msi/test/test_d.wxs --- a/Tools/msi/test/test_d.wxs +++ b/Tools/msi/test/test_d.wxs @@ -1,6 +1,6 @@ ? - + diff --git a/Tools/msi/test/test_pdb.wxs b/Tools/msi/test/test_pdb.wxs --- a/Tools/msi/test/test_pdb.wxs +++ b/Tools/msi/test/test_pdb.wxs @@ -1,6 +1,6 @@ ? - + -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 5 21:48:07 2015 From: python-checkins at python.org (steve.dower) Date: Sat, 05 Sep 2015 19:48:07 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2324910=3A_Windows_MSIs_now_have_unique_display_n?= =?utf-8?q?ames=2E?= Message-ID: <20150905194807.66866.62674@psf.io> https://hg.python.org/cpython/rev/0295d2cdc9d3 changeset: 97682:0295d2cdc9d3 parent: 97680:d755f75019c8 parent: 97681:3594400f3202 user: Steve Dower date: Sat Sep 05 12:47:22 2015 -0700 summary: Issue #24910: Windows MSIs now have unique display names. files: Misc/NEWS | 2 ++ Tools/msi/core/core_d.wxs | 2 +- Tools/msi/core/core_pdb.wxs | 2 +- Tools/msi/dev/dev_d.wxs | 2 +- Tools/msi/exe/exe_d.wxs | 2 +- Tools/msi/exe/exe_pdb.wxs | 2 +- Tools/msi/lib/lib_d.wxs | 2 +- Tools/msi/lib/lib_pdb.wxs | 2 +- Tools/msi/tcltk/tcltk_d.wxs | 2 +- Tools/msi/tcltk/tcltk_pdb.wxs | 2 +- Tools/msi/test/test_d.wxs | 2 +- Tools/msi/test/test_pdb.wxs | 2 +- 12 files changed, 13 insertions(+), 11 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -155,6 +155,8 @@ Build ----- +- Issue #24910: Windows MSIs now have unique display names. + - Issue #24986: It is now possible to build Python on Windows without errors when external libraries are not available. diff --git a/Tools/msi/core/core_d.wxs b/Tools/msi/core/core_d.wxs --- a/Tools/msi/core/core_d.wxs +++ b/Tools/msi/core/core_d.wxs @@ -1,6 +1,6 @@ ? - + diff --git a/Tools/msi/core/core_pdb.wxs b/Tools/msi/core/core_pdb.wxs --- a/Tools/msi/core/core_pdb.wxs +++ b/Tools/msi/core/core_pdb.wxs @@ -1,6 +1,6 @@ ? - + diff --git a/Tools/msi/dev/dev_d.wxs b/Tools/msi/dev/dev_d.wxs --- a/Tools/msi/dev/dev_d.wxs +++ b/Tools/msi/dev/dev_d.wxs @@ -1,6 +1,6 @@ ? - + diff --git a/Tools/msi/exe/exe_d.wxs b/Tools/msi/exe/exe_d.wxs --- a/Tools/msi/exe/exe_d.wxs +++ b/Tools/msi/exe/exe_d.wxs @@ -1,6 +1,6 @@ ? - + diff --git a/Tools/msi/exe/exe_pdb.wxs b/Tools/msi/exe/exe_pdb.wxs --- a/Tools/msi/exe/exe_pdb.wxs +++ b/Tools/msi/exe/exe_pdb.wxs @@ -1,6 +1,6 @@ ? - + diff --git a/Tools/msi/lib/lib_d.wxs b/Tools/msi/lib/lib_d.wxs --- a/Tools/msi/lib/lib_d.wxs +++ b/Tools/msi/lib/lib_d.wxs @@ -1,6 +1,6 @@ ? - + diff --git a/Tools/msi/lib/lib_pdb.wxs b/Tools/msi/lib/lib_pdb.wxs --- a/Tools/msi/lib/lib_pdb.wxs +++ b/Tools/msi/lib/lib_pdb.wxs @@ -1,6 +1,6 @@ ? - + diff --git a/Tools/msi/tcltk/tcltk_d.wxs b/Tools/msi/tcltk/tcltk_d.wxs --- a/Tools/msi/tcltk/tcltk_d.wxs +++ b/Tools/msi/tcltk/tcltk_d.wxs @@ -1,6 +1,6 @@ ? - + diff --git a/Tools/msi/tcltk/tcltk_pdb.wxs b/Tools/msi/tcltk/tcltk_pdb.wxs --- a/Tools/msi/tcltk/tcltk_pdb.wxs +++ b/Tools/msi/tcltk/tcltk_pdb.wxs @@ -1,6 +1,6 @@ ? - + diff --git a/Tools/msi/test/test_d.wxs b/Tools/msi/test/test_d.wxs --- a/Tools/msi/test/test_d.wxs +++ b/Tools/msi/test/test_d.wxs @@ -1,6 +1,6 @@ ? - + diff --git a/Tools/msi/test/test_pdb.wxs b/Tools/msi/test/test_pdb.wxs --- a/Tools/msi/test/test_pdb.wxs +++ b/Tools/msi/test/test_pdb.wxs @@ -1,6 +1,6 @@ ? - + -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 5 23:07:30 2015 From: python-checkins at python.org (eric.smith) Date: Sat, 05 Sep 2015 21:07:30 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_Note_that_comments_are_not_al?= =?utf-8?q?lowed_inside_expressions=2E?= Message-ID: <20150905210729.68873.29954@psf.io> https://hg.python.org/peps/rev/a6e64585e4b8 changeset: 6041:a6e64585e4b8 user: Eric V. Smith date: Sat Sep 05 17:07:48 2015 -0400 summary: Note that comments are not allowed inside expressions. files: pep-0498.txt | 3 +++ 1 files changed, 3 insertions(+), 0 deletions(-) diff --git a/pep-0498.txt b/pep-0498.txt --- a/pep-0498.txt +++ b/pep-0498.txt @@ -200,6 +200,9 @@ brace. Doubled opening braces do not signify the start of an expression. +Comments, using the ``'#'`` character, are not allowed inside the +expression. + Following the expression, an optional type conversion may be specified. The allowed conversions are ``'!s'``, ``'!r'``, or ``'!a'``. These are treated the same as in ``str.format()``: ``'!s'`` -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Sun Sep 6 00:21:41 2015 From: python-checkins at python.org (guido.van.rossum) Date: Sat, 05 Sep 2015 22:21:41 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2324912=3A_Prevent_=5F=5Fclass=5F=5F_assignment_t?= =?utf-8?q?o_immutable_built-in_objects=2E?= Message-ID: <20150905222141.12017.65532@psf.io> https://hg.python.org/cpython/rev/b045465e5dba changeset: 97685:b045465e5dba parent: 97682:0295d2cdc9d3 parent: 97684:1c55f169f4ee user: Guido van Rossum date: Sat Sep 05 15:20:57 2015 -0700 summary: Issue #24912: Prevent __class__ assignment to immutable built-in objects. (Merge 3.5 -> 3.6) files: Lib/test/test_descr.py | 45 ++++++++++++++++++++++ Misc/NEWS | 2 + Objects/typeobject.c | 59 ++++++++++++++++++++++++++++++ 3 files changed, 106 insertions(+), 0 deletions(-) diff --git a/Lib/test/test_descr.py b/Lib/test/test_descr.py --- a/Lib/test/test_descr.py +++ b/Lib/test/test_descr.py @@ -1036,6 +1036,51 @@ self.assertTrue(m.__class__ is types.ModuleType) self.assertFalse(hasattr(m, "a")) + # Make sure that builtin immutable objects don't support __class__ + # assignment, because the object instances may be interned. + # We set __slots__ = () to ensure that the subclasses are + # memory-layout compatible, and thus otherwise reasonable candidates + # for __class__ assignment. + + # The following types have immutable instances, but are not + # subclassable and thus don't need to be checked: + # NoneType, bool + + class MyInt(int): + __slots__ = () + with self.assertRaises(TypeError): + (1).__class__ = MyInt + + class MyFloat(float): + __slots__ = () + with self.assertRaises(TypeError): + (1.0).__class__ = MyFloat + + class MyComplex(complex): + __slots__ = () + with self.assertRaises(TypeError): + (1 + 2j).__class__ = MyComplex + + class MyStr(str): + __slots__ = () + with self.assertRaises(TypeError): + "a".__class__ = MyStr + + class MyBytes(bytes): + __slots__ = () + with self.assertRaises(TypeError): + b"a".__class__ = MyBytes + + class MyTuple(tuple): + __slots__ = () + with self.assertRaises(TypeError): + ().__class__ = MyTuple + + class MyFrozenSet(frozenset): + __slots__ = () + with self.assertRaises(TypeError): + frozenset().__class__ = MyFrozenSet + def test_slots(self): # Testing __slots__... class C0(object): diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -169,6 +169,8 @@ Core and Builtins ----------------- +- Issue #24912: Prevent __class__ assignment to immutable built-in objects. + - Issue #24975: Fix AST compilation for PEP 448 syntax. Library diff --git a/Objects/typeobject.c b/Objects/typeobject.c --- a/Objects/typeobject.c +++ b/Objects/typeobject.c @@ -3674,6 +3674,65 @@ return -1; } newto = (PyTypeObject *)value; + /* In versions of CPython prior to 3.5, the code in + compatible_for_assignment was not set up to correctly check for memory + layout / slot / etc. compatibility for non-HEAPTYPE classes, so we just + disallowed __class__ assignment in any case that wasn't HEAPTYPE -> + HEAPTYPE. + + During the 3.5 development cycle, we fixed the code in + compatible_for_assignment to correctly check compatibility between + arbitrary types, and started allowing __class__ assignment in all cases + where the old and new types did in fact have compatible slots and + memory layout (regardless of whether they were implemented as HEAPTYPEs + or not). + + Just before 3.5 was released, though, we discovered that this led to + problems with immutable types like int, where the interpreter assumes + they are immutable and interns some values. Formerly this wasn't a + problem, because they really were immutable -- in particular, all the + types where the interpreter applied this interning trick happened to + also be statically allocated, so the old HEAPTYPE rules were + "accidentally" stopping them from allowing __class__ assignment. But + with the changes to __class__ assignment, we started allowing code like + + class MyInt(int): + ... + # Modifies the type of *all* instances of 1 in the whole program, + # including future instances (!), because the 1 object is interned. + (1).__class__ = MyInt + + (see https://bugs.python.org/issue24912). + + In theory the proper fix would be to identify which classes rely on + this invariant and somehow disallow __class__ assignment only for them, + perhaps via some mechanism like a new Py_TPFLAGS_IMMUTABLE flag (a + "blacklisting" approach). But in practice, since this problem wasn't + noticed late in the 3.5 RC cycle, we're taking the conservative + approach and reinstating the same HEAPTYPE->HEAPTYPE check that we used + to have, plus a "whitelist". For now, the whitelist consists only of + ModuleType subtypes, since those are the cases that motivated the patch + in the first place -- see https://bugs.python.org/issue22986 -- and + since module objects are mutable we can be sure that they are + definitely not being interned. So now we allow HEAPTYPE->HEAPTYPE *or* + ModuleType subtype -> ModuleType subtype. + + So far as we know, all the code beyond the following 'if' statement + will correctly handle non-HEAPTYPE classes, and the HEAPTYPE check is + needed only to protect that subset of non-HEAPTYPE classes for which + the interpreter has baked in the assumption that all instances are + truly immutable. + */ + if (!(PyType_IsSubtype(newto, &PyModule_Type) && + PyType_IsSubtype(oldto, &PyModule_Type)) && + (!(newto->tp_flags & Py_TPFLAGS_HEAPTYPE) || + !(oldto->tp_flags & Py_TPFLAGS_HEAPTYPE))) { + PyErr_Format(PyExc_TypeError, + "__class__ assignment only supported for heap types " + "or ModuleType subclasses"); + return -1; + } + if (compatible_for_assignment(oldto, newto, "__class__")) { if (newto->tp_flags & Py_TPFLAGS_HEAPTYPE) Py_INCREF(newto); -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 6 00:21:41 2015 From: python-checkins at python.org (guido.van.rossum) Date: Sat, 05 Sep 2015 22:21:41 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI0OTEy?= =?utf-8?q?=3A_Prevent_=5F=5Fclass=5F=5F_assignment_to_immutable_built-in_?= =?utf-8?q?objects=2E?= Message-ID: <20150905222141.101500.65528@psf.io> https://hg.python.org/cpython/rev/27cc5cce0292 changeset: 97683:27cc5cce0292 branch: 3.5 parent: 97676:438dde69871d user: Guido van Rossum date: Fri Sep 04 20:54:07 2015 -0700 summary: Issue #24912: Prevent __class__ assignment to immutable built-in objects. files: Lib/test/test_descr.py | 45 ++++++++++++++++++++++ Misc/NEWS | 2 + Objects/typeobject.c | 59 ++++++++++++++++++++++++++++++ 3 files changed, 106 insertions(+), 0 deletions(-) diff --git a/Lib/test/test_descr.py b/Lib/test/test_descr.py --- a/Lib/test/test_descr.py +++ b/Lib/test/test_descr.py @@ -1036,6 +1036,51 @@ self.assertTrue(m.__class__ is types.ModuleType) self.assertFalse(hasattr(m, "a")) + # Make sure that builtin immutable objects don't support __class__ + # assignment, because the object instances may be interned. + # We set __slots__ = () to ensure that the subclasses are + # memory-layout compatible, and thus otherwise reasonable candidates + # for __class__ assignment. + + # The following types have immutable instances, but are not + # subclassable and thus don't need to be checked: + # NoneType, bool + + class MyInt(int): + __slots__ = () + with self.assertRaises(TypeError): + (1).__class__ = MyInt + + class MyFloat(float): + __slots__ = () + with self.assertRaises(TypeError): + (1.0).__class__ = MyFloat + + class MyComplex(complex): + __slots__ = () + with self.assertRaises(TypeError): + (1 + 2j).__class__ = MyComplex + + class MyStr(str): + __slots__ = () + with self.assertRaises(TypeError): + "a".__class__ = MyStr + + class MyBytes(bytes): + __slots__ = () + with self.assertRaises(TypeError): + b"a".__class__ = MyBytes + + class MyTuple(tuple): + __slots__ = () + with self.assertRaises(TypeError): + ().__class__ = MyTuple + + class MyFrozenSet(frozenset): + __slots__ = () + with self.assertRaises(TypeError): + frozenset().__class__ = MyFrozenSet + def test_slots(self): # Testing __slots__... class C0(object): diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -10,6 +10,8 @@ Core and Builtins ----------------- +- Issue #24912: Prevent __class__ assignment to immutable built-in objects. + - Issue #24975: Fix AST compilation for PEP 448 syntax. Library diff --git a/Objects/typeobject.c b/Objects/typeobject.c --- a/Objects/typeobject.c +++ b/Objects/typeobject.c @@ -3666,6 +3666,65 @@ return -1; } newto = (PyTypeObject *)value; + /* In versions of CPython prior to 3.5, the code in + compatible_for_assignment was not set up to correctly check for memory + layout / slot / etc. compatibility for non-HEAPTYPE classes, so we just + disallowed __class__ assignment in any case that wasn't HEAPTYPE -> + HEAPTYPE. + + During the 3.5 development cycle, we fixed the code in + compatible_for_assignment to correctly check compatibility between + arbitrary types, and started allowing __class__ assignment in all cases + where the old and new types did in fact have compatible slots and + memory layout (regardless of whether they were implemented as HEAPTYPEs + or not). + + Just before 3.5 was released, though, we discovered that this led to + problems with immutable types like int, where the interpreter assumes + they are immutable and interns some values. Formerly this wasn't a + problem, because they really were immutable -- in particular, all the + types where the interpreter applied this interning trick happened to + also be statically allocated, so the old HEAPTYPE rules were + "accidentally" stopping them from allowing __class__ assignment. But + with the changes to __class__ assignment, we started allowing code like + + class MyInt(int): + ... + # Modifies the type of *all* instances of 1 in the whole program, + # including future instances (!), because the 1 object is interned. + (1).__class__ = MyInt + + (see https://bugs.python.org/issue24912). + + In theory the proper fix would be to identify which classes rely on + this invariant and somehow disallow __class__ assignment only for them, + perhaps via some mechanism like a new Py_TPFLAGS_IMMUTABLE flag (a + "blacklisting" approach). But in practice, since this problem wasn't + noticed late in the 3.5 RC cycle, we're taking the conservative + approach and reinstating the same HEAPTYPE->HEAPTYPE check that we used + to have, plus a "whitelist". For now, the whitelist consists only of + ModuleType subtypes, since those are the cases that motivated the patch + in the first place -- see https://bugs.python.org/issue22986 -- and + since module objects are mutable we can be sure that they are + definitely not being interned. So now we allow HEAPTYPE->HEAPTYPE *or* + ModuleType subtype -> ModuleType subtype. + + So far as we know, all the code beyond the following 'if' statement + will correctly handle non-HEAPTYPE classes, and the HEAPTYPE check is + needed only to protect that subset of non-HEAPTYPE classes for which + the interpreter has baked in the assumption that all instances are + truly immutable. + */ + if (!(PyType_IsSubtype(newto, &PyModule_Type) && + PyType_IsSubtype(oldto, &PyModule_Type)) && + (!(newto->tp_flags & Py_TPFLAGS_HEAPTYPE) || + !(oldto->tp_flags & Py_TPFLAGS_HEAPTYPE))) { + PyErr_Format(PyExc_TypeError, + "__class__ assignment only supported for heap types " + "or ModuleType subclasses"); + return -1; + } + if (compatible_for_assignment(oldto, newto, "__class__")) { if (newto->tp_flags & Py_TPFLAGS_HEAPTYPE) Py_INCREF(newto); -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 6 00:21:41 2015 From: python-checkins at python.org (guido.van.rossum) Date: Sat, 05 Sep 2015 22:21:41 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy41IC0+IDMuNSk6?= =?utf-8?q?_Issue_=2324912=3A_Prevent_=5F=5Fclass=5F=5F_assignment_to_immu?= =?utf-8?q?table_built-in_objects=2E?= Message-ID: <20150905222141.14863.67459@psf.io> https://hg.python.org/cpython/rev/1c55f169f4ee changeset: 97684:1c55f169f4ee branch: 3.5 parent: 97681:3594400f3202 parent: 97683:27cc5cce0292 user: Guido van Rossum date: Sat Sep 05 15:20:08 2015 -0700 summary: Issue #24912: Prevent __class__ assignment to immutable built-in objects. (Merge 3.5.0 -> 3.5) files: Lib/test/test_descr.py | 45 ++++++++++++++++++++++ Misc/NEWS | 2 + Objects/typeobject.c | 59 ++++++++++++++++++++++++++++++ 3 files changed, 106 insertions(+), 0 deletions(-) diff --git a/Lib/test/test_descr.py b/Lib/test/test_descr.py --- a/Lib/test/test_descr.py +++ b/Lib/test/test_descr.py @@ -1036,6 +1036,51 @@ self.assertTrue(m.__class__ is types.ModuleType) self.assertFalse(hasattr(m, "a")) + # Make sure that builtin immutable objects don't support __class__ + # assignment, because the object instances may be interned. + # We set __slots__ = () to ensure that the subclasses are + # memory-layout compatible, and thus otherwise reasonable candidates + # for __class__ assignment. + + # The following types have immutable instances, but are not + # subclassable and thus don't need to be checked: + # NoneType, bool + + class MyInt(int): + __slots__ = () + with self.assertRaises(TypeError): + (1).__class__ = MyInt + + class MyFloat(float): + __slots__ = () + with self.assertRaises(TypeError): + (1.0).__class__ = MyFloat + + class MyComplex(complex): + __slots__ = () + with self.assertRaises(TypeError): + (1 + 2j).__class__ = MyComplex + + class MyStr(str): + __slots__ = () + with self.assertRaises(TypeError): + "a".__class__ = MyStr + + class MyBytes(bytes): + __slots__ = () + with self.assertRaises(TypeError): + b"a".__class__ = MyBytes + + class MyTuple(tuple): + __slots__ = () + with self.assertRaises(TypeError): + ().__class__ = MyTuple + + class MyFrozenSet(frozenset): + __slots__ = () + with self.assertRaises(TypeError): + frozenset().__class__ = MyFrozenSet + def test_slots(self): # Testing __slots__... class C0(object): diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -85,6 +85,8 @@ Core and Builtins ----------------- +- Issue #24912: Prevent __class__ assignment to immutable built-in objects. + - Issue #24975: Fix AST compilation for PEP 448 syntax. Library diff --git a/Objects/typeobject.c b/Objects/typeobject.c --- a/Objects/typeobject.c +++ b/Objects/typeobject.c @@ -3666,6 +3666,65 @@ return -1; } newto = (PyTypeObject *)value; + /* In versions of CPython prior to 3.5, the code in + compatible_for_assignment was not set up to correctly check for memory + layout / slot / etc. compatibility for non-HEAPTYPE classes, so we just + disallowed __class__ assignment in any case that wasn't HEAPTYPE -> + HEAPTYPE. + + During the 3.5 development cycle, we fixed the code in + compatible_for_assignment to correctly check compatibility between + arbitrary types, and started allowing __class__ assignment in all cases + where the old and new types did in fact have compatible slots and + memory layout (regardless of whether they were implemented as HEAPTYPEs + or not). + + Just before 3.5 was released, though, we discovered that this led to + problems with immutable types like int, where the interpreter assumes + they are immutable and interns some values. Formerly this wasn't a + problem, because they really were immutable -- in particular, all the + types where the interpreter applied this interning trick happened to + also be statically allocated, so the old HEAPTYPE rules were + "accidentally" stopping them from allowing __class__ assignment. But + with the changes to __class__ assignment, we started allowing code like + + class MyInt(int): + ... + # Modifies the type of *all* instances of 1 in the whole program, + # including future instances (!), because the 1 object is interned. + (1).__class__ = MyInt + + (see https://bugs.python.org/issue24912). + + In theory the proper fix would be to identify which classes rely on + this invariant and somehow disallow __class__ assignment only for them, + perhaps via some mechanism like a new Py_TPFLAGS_IMMUTABLE flag (a + "blacklisting" approach). But in practice, since this problem wasn't + noticed late in the 3.5 RC cycle, we're taking the conservative + approach and reinstating the same HEAPTYPE->HEAPTYPE check that we used + to have, plus a "whitelist". For now, the whitelist consists only of + ModuleType subtypes, since those are the cases that motivated the patch + in the first place -- see https://bugs.python.org/issue22986 -- and + since module objects are mutable we can be sure that they are + definitely not being interned. So now we allow HEAPTYPE->HEAPTYPE *or* + ModuleType subtype -> ModuleType subtype. + + So far as we know, all the code beyond the following 'if' statement + will correctly handle non-HEAPTYPE classes, and the HEAPTYPE check is + needed only to protect that subset of non-HEAPTYPE classes for which + the interpreter has baked in the assumption that all instances are + truly immutable. + */ + if (!(PyType_IsSubtype(newto, &PyModule_Type) && + PyType_IsSubtype(oldto, &PyModule_Type)) && + (!(newto->tp_flags & Py_TPFLAGS_HEAPTYPE) || + !(oldto->tp_flags & Py_TPFLAGS_HEAPTYPE))) { + PyErr_Format(PyExc_TypeError, + "__class__ assignment only supported for heap types " + "or ModuleType subclasses"); + return -1; + } + if (compatible_for_assignment(oldto, newto, "__class__")) { if (newto->tp_flags & Py_TPFLAGS_HEAPTYPE) Py_INCREF(newto); -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 6 01:19:02 2015 From: python-checkins at python.org (terry.reedy) Date: Sat, 05 Sep 2015 23:19:02 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzE2MTgw?= =?utf-8?q?=3A_Exit_pdb_if_file_has_syntax_error=2C_instead_of_trapping_us?= =?utf-8?q?er?= Message-ID: <20150905231902.15706.55170@psf.io> https://hg.python.org/cpython/rev/2d4aac2ab253 changeset: 97687:2d4aac2ab253 branch: 3.4 parent: 97671:e67bf9c9a898 user: Terry Jan Reedy date: Sat Sep 05 19:13:26 2015 -0400 summary: Issue #16180: Exit pdb if file has syntax error, instead of trapping user in an infinite loop. Patch by Xavier de Gaye. files: Lib/pdb.py | 3 +++ Lib/test/test_pdb.py | 12 ++++++++++++ Misc/NEWS | 3 +++ 3 files changed, 18 insertions(+), 0 deletions(-) diff --git a/Lib/pdb.py b/Lib/pdb.py --- a/Lib/pdb.py +++ b/Lib/pdb.py @@ -1669,6 +1669,9 @@ # In most cases SystemExit does not warrant a post-mortem session. print("The program exited via sys.exit(). Exit status:", end=' ') print(sys.exc_info()[1]) + except SyntaxError: + traceback.print_exc() + sys.exit(1) except: traceback.print_exc() print("Uncaught exception. Entering post mortem debugging") diff --git a/Lib/test/test_pdb.py b/Lib/test/test_pdb.py --- a/Lib/test/test_pdb.py +++ b/Lib/test/test_pdb.py @@ -1043,6 +1043,18 @@ self.assertNotIn('Error', stdout.decode(), "Got an error running test script under PDB") + def test_issue16180(self): + # A syntax error in the debuggee. + script = "def f: pass\n" + commands = '' + expected = "SyntaxError:" + stdout, stderr = self.run_pdb(script, commands) + self.assertIn(expected, stdout, + '\n\nExpected:\n{}\nGot:\n{}\n' + 'Fail to handle a syntax error in the debuggee.' + .format(expected, stdout)) + + def tearDown(self): support.unlink(support.TESTFN) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -81,6 +81,9 @@ Library ------- +- Issue #16180: Exit pdb if file has syntax error, instead of trapping user + in an infinite loop. Patch by Xavier de Gaye. + - Issue #21112: Fix regression in unittest.expectedFailure on subclasses. Patch from Berker Peksag. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 6 01:19:02 2015 From: python-checkins at python.org (terry.reedy) Date: Sat, 05 Sep 2015 23:19:02 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzE2MTgw?= =?utf-8?q?=3A_Exit_pdb_if_file_has_syntax_error=2C_instead_of_trapping_us?= =?utf-8?q?er?= Message-ID: <20150905231902.17989.40246@psf.io> https://hg.python.org/cpython/rev/26c4db1a0aea changeset: 97686:26c4db1a0aea branch: 2.7 parent: 97667:0ff7aa9a438f user: Terry Jan Reedy date: Sat Sep 05 19:13:17 2015 -0400 summary: Issue #16180: Exit pdb if file has syntax error, instead of trapping user in an infinite loop. Patch by Xavier de Gaye. files: Lib/pdb.py | 3 +++ Lib/test/test_pdb.py | 11 +++++++++++ Misc/NEWS | 3 +++ 3 files changed, 17 insertions(+), 0 deletions(-) diff --git a/Lib/pdb.py b/Lib/pdb.py --- a/Lib/pdb.py +++ b/Lib/pdb.py @@ -1322,6 +1322,9 @@ # In most cases SystemExit does not warrant a post-mortem session. print "The program exited via sys.exit(). Exit status: ", print sys.exc_info()[1] + except SyntaxError: + traceback.print_exc() + sys.exit(1) except: traceback.print_exc() print "Uncaught exception. Entering post mortem debugging" diff --git a/Lib/test/test_pdb.py b/Lib/test/test_pdb.py --- a/Lib/test/test_pdb.py +++ b/Lib/test/test_pdb.py @@ -69,6 +69,17 @@ any('main.py(5)foo()->None' in l for l in stdout.splitlines()), 'Fail to step into the caller after a return') + def test_issue16180(self): + # A syntax error in the debuggee. + script = "def f: pass\n" + commands = '' + expected = "SyntaxError:" + stdout, stderr = self.run_pdb(script, commands) + self.assertIn(expected, stdout, + '\n\nExpected:\n{}\nGot:\n{}\n' + 'Fail to handle a syntax error in the debuggee.' + .format(expected, stdout)) + class PdbTestInput(object): """Context manager that makes testing Pdb in doctests easier.""" diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -37,6 +37,9 @@ Library ------- +- Issue #16180: Exit pdb if file has syntax error, instead of trapping user + in an infinite loop. Patch by Xavier de Gaye. + - Issue #22812: Fix unittest discovery examples. Patch from Pam McA'Nulty. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 6 01:19:02 2015 From: python-checkins at python.org (terry.reedy) Date: Sat, 05 Sep 2015 23:19:02 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge_with_3=2E5?= Message-ID: <20150905231902.12006.45479@psf.io> https://hg.python.org/cpython/rev/127ac263354e changeset: 97689:127ac263354e parent: 97685:b045465e5dba parent: 97688:611c732f7632 user: Terry Jan Reedy date: Sat Sep 05 19:17:49 2015 -0400 summary: Merge with 3.5 files: Lib/pdb.py | 3 +++ Lib/test/test_pdb.py | 12 ++++++++++++ Misc/NEWS | 3 +++ 3 files changed, 18 insertions(+), 0 deletions(-) diff --git a/Lib/pdb.py b/Lib/pdb.py --- a/Lib/pdb.py +++ b/Lib/pdb.py @@ -1669,6 +1669,9 @@ # In most cases SystemExit does not warrant a post-mortem session. print("The program exited via sys.exit(). Exit status:", end=' ') print(sys.exc_info()[1]) + except SyntaxError: + traceback.print_exc() + sys.exit(1) except: traceback.print_exc() print("Uncaught exception. Entering post mortem debugging") diff --git a/Lib/test/test_pdb.py b/Lib/test/test_pdb.py --- a/Lib/test/test_pdb.py +++ b/Lib/test/test_pdb.py @@ -1043,6 +1043,18 @@ self.assertNotIn('Error', stdout.decode(), "Got an error running test script under PDB") + def test_issue16180(self): + # A syntax error in the debuggee. + script = "def f: pass\n" + commands = '' + expected = "SyntaxError:" + stdout, stderr = self.run_pdb(script, commands) + self.assertIn(expected, stdout, + '\n\nExpected:\n{}\nGot:\n{}\n' + 'Fail to handle a syntax error in the debuggee.' + .format(expected, stdout)) + + def tearDown(self): support.unlink(support.TESTFN) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -101,6 +101,9 @@ Library ------- +- Issue #16180: Exit pdb if file has syntax error, instead of trapping user + in an infinite loop. Patch by Xavier de Gaye. + - Issue #24891: Fix a race condition at Python startup if the file descriptor of stdin (0), stdout (1) or stderr (2) is closed while Python is creating sys.stdin, sys.stdout and sys.stderr objects. These attributes are now set -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 6 01:19:02 2015 From: python-checkins at python.org (terry.reedy) Date: Sat, 05 Sep 2015 23:19:02 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_merge_from_3=2E4?= Message-ID: <20150905231902.27687.94083@psf.io> https://hg.python.org/cpython/rev/611c732f7632 changeset: 97688:611c732f7632 branch: 3.5 parent: 97684:1c55f169f4ee parent: 97687:2d4aac2ab253 user: Terry Jan Reedy date: Sat Sep 05 19:17:24 2015 -0400 summary: merge from 3.4 files: Lib/pdb.py | 3 +++ Lib/test/test_pdb.py | 12 ++++++++++++ Misc/NEWS | 3 +++ 3 files changed, 18 insertions(+), 0 deletions(-) diff --git a/Lib/pdb.py b/Lib/pdb.py --- a/Lib/pdb.py +++ b/Lib/pdb.py @@ -1669,6 +1669,9 @@ # In most cases SystemExit does not warrant a post-mortem session. print("The program exited via sys.exit(). Exit status:", end=' ') print(sys.exc_info()[1]) + except SyntaxError: + traceback.print_exc() + sys.exit(1) except: traceback.print_exc() print("Uncaught exception. Entering post mortem debugging") diff --git a/Lib/test/test_pdb.py b/Lib/test/test_pdb.py --- a/Lib/test/test_pdb.py +++ b/Lib/test/test_pdb.py @@ -1043,6 +1043,18 @@ self.assertNotIn('Error', stdout.decode(), "Got an error running test script under PDB") + def test_issue16180(self): + # A syntax error in the debuggee. + script = "def f: pass\n" + commands = '' + expected = "SyntaxError:" + stdout, stderr = self.run_pdb(script, commands) + self.assertIn(expected, stdout, + '\n\nExpected:\n{}\nGot:\n{}\n' + 'Fail to handle a syntax error in the debuggee.' + .format(expected, stdout)) + + def tearDown(self): support.unlink(support.TESTFN) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -14,6 +14,9 @@ Library ------- +- Issue #16180: Exit pdb if file has syntax error, instead of trapping user + in an infinite loop. Patch by Xavier de Gaye. + - Issue #24891: Fix a race condition at Python startup if the file descriptor of stdin (0), stdout (1) or stderr (2) is closed while Python is creating sys.stdin, sys.stdout and sys.stderr objects. These attributes are now set -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 6 02:06:23 2015 From: python-checkins at python.org (raymond.hettinger) Date: Sun, 06 Sep 2015 00:06:23 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_merge?= Message-ID: <20150906000623.27695.35387@psf.io> https://hg.python.org/cpython/rev/abc416ca59fc changeset: 97691:abc416ca59fc parent: 97689:127ac263354e parent: 97690:ba63067654d3 user: Raymond Hettinger date: Sat Sep 05 17:06:18 2015 -0700 summary: merge files: Modules/_collectionsmodule.c | 16 ++++++++++------ 1 files changed, 10 insertions(+), 6 deletions(-) diff --git a/Modules/_collectionsmodule.c b/Modules/_collectionsmodule.c --- a/Modules/_collectionsmodule.c +++ b/Modules/_collectionsmodule.c @@ -419,9 +419,11 @@ deque->rightblock->data[deque->rightindex] = item; deque_trim_left(deque); } + if (PyErr_Occurred()) { + Py_DECREF(it); + return NULL; + } Py_DECREF(it); - if (PyErr_Occurred()) - return NULL; Py_RETURN_NONE; } @@ -480,9 +482,11 @@ deque->leftblock->data[deque->leftindex] = item; deque_trim_right(deque); } + if (PyErr_Occurred()) { + Py_DECREF(it); + return NULL; + } Py_DECREF(it); - if (PyErr_Occurred()) - return NULL; Py_RETURN_NONE; } @@ -497,8 +501,8 @@ result = deque_extend(deque, other); if (result == NULL) return result; + Py_INCREF(deque); Py_DECREF(result); - Py_INCREF(deque); return (PyObject *)deque; } @@ -1260,8 +1264,8 @@ aslist, ((dequeobject *)deque)->maxlen); else result = PyUnicode_FromFormat("deque(%R)", aslist); + Py_ReprLeave(deque); Py_DECREF(aslist); - Py_ReprLeave(deque); return result; } -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 6 02:06:23 2015 From: python-checkins at python.org (raymond.hettinger) Date: Sun, 06 Sep 2015 00:06:23 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_Prevent_reentr?= =?utf-8?q?ant_badness_by_deferring_the_decrefs_as_long_as_possible=2E?= Message-ID: <20150906000623.68881.71611@psf.io> https://hg.python.org/cpython/rev/ba63067654d3 changeset: 97690:ba63067654d3 branch: 3.5 parent: 97688:611c732f7632 user: Raymond Hettinger date: Sat Sep 05 17:05:52 2015 -0700 summary: Prevent reentrant badness by deferring the decrefs as long as possible. files: Modules/_collectionsmodule.c | 16 ++++++++++------ 1 files changed, 10 insertions(+), 6 deletions(-) diff --git a/Modules/_collectionsmodule.c b/Modules/_collectionsmodule.c --- a/Modules/_collectionsmodule.c +++ b/Modules/_collectionsmodule.c @@ -419,9 +419,11 @@ deque->rightblock->data[deque->rightindex] = item; deque_trim_left(deque); } + if (PyErr_Occurred()) { + Py_DECREF(it); + return NULL; + } Py_DECREF(it); - if (PyErr_Occurred()) - return NULL; Py_RETURN_NONE; } @@ -480,9 +482,11 @@ deque->leftblock->data[deque->leftindex] = item; deque_trim_right(deque); } + if (PyErr_Occurred()) { + Py_DECREF(it); + return NULL; + } Py_DECREF(it); - if (PyErr_Occurred()) - return NULL; Py_RETURN_NONE; } @@ -497,8 +501,8 @@ result = deque_extend(deque, other); if (result == NULL) return result; + Py_INCREF(deque); Py_DECREF(result); - Py_INCREF(deque); return (PyObject *)deque; } @@ -1260,8 +1264,8 @@ aslist, ((dequeobject *)deque)->maxlen); else result = PyUnicode_FromFormat("deque(%R)", aslist); + Py_ReprLeave(deque); Py_DECREF(aslist); - Py_ReprLeave(deque); return result; } -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 6 06:02:06 2015 From: python-checkins at python.org (steve.dower) Date: Sun, 06 Sep 2015 04:02:06 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Null_merge_with_3=2E4?= Message-ID: <20150906040206.12009.66903@psf.io> https://hg.python.org/cpython/rev/7cdadcc1002d changeset: 97696:7cdadcc1002d branch: 3.5 parent: 97695:ddb6e423e251 parent: 97694:a29b49d57769 user: Steve Dower date: Sat Sep 05 20:59:41 2015 -0700 summary: Null merge with 3.4 files: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 6 06:02:05 2015 From: python-checkins at python.org (steve.dower) Date: Sun, 06 Sep 2015 04:02:05 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzI0OTE3?= =?utf-8?q?=3A_time=5Fstrftime=28=29_Buffer_Over-read=2E_Patch_by_John_Lei?= =?utf-8?q?tch=2E?= Message-ID: <20150906040205.68873.8443@psf.io> https://hg.python.org/cpython/rev/a29b49d57769 changeset: 97694:a29b49d57769 branch: 3.4 parent: 97687:2d4aac2ab253 user: Steve Dower date: Sat Sep 05 20:55:34 2015 -0700 summary: Issue #24917: time_strftime() Buffer Over-read. Patch by John Leitch. files: Misc/NEWS | 2 ++ Modules/timemodule.c | 6 ++++++ 2 files changed, 8 insertions(+), 0 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -84,6 +84,8 @@ - Issue #16180: Exit pdb if file has syntax error, instead of trapping user in an infinite loop. Patch by Xavier de Gaye. +- Issue #24917: time_strftime() Buffer Over-read. Patch by John Leitch. + - Issue #21112: Fix regression in unittest.expectedFailure on subclasses. Patch from Berker Peksag. diff --git a/Modules/timemodule.c b/Modules/timemodule.c --- a/Modules/timemodule.c +++ b/Modules/timemodule.c @@ -662,6 +662,12 @@ "format %y requires year >= 1900 on AIX"); return NULL; } + else if (outbuf[1] == '\0') + { + PyErr_SetString(PyExc_ValueError, "Incomplete format string"); + Py_DECREF(format); + return NULL; + } } #endif -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 6 06:02:06 2015 From: python-checkins at python.org (steve.dower) Date: Sun, 06 Sep 2015 04:02:06 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2324917=3A_time=5Fstrftime=28=29_Buffer_Over-read?= =?utf-8?q?=2E_Patch_by_John_Leitch=2E?= Message-ID: <20150906040206.14855.36046@psf.io> https://hg.python.org/cpython/rev/6fc744ac3953 changeset: 97697:6fc744ac3953 parent: 97691:abc416ca59fc parent: 97696:7cdadcc1002d user: Steve Dower date: Sat Sep 05 21:00:33 2015 -0700 summary: Issue #24917: time_strftime() Buffer Over-read. Patch by John Leitch. files: Lib/test/test_time.py | 6 ++++++ Misc/NEWS | 2 ++ Modules/timemodule.c | 12 ++++++++++++ 3 files changed, 20 insertions(+), 0 deletions(-) diff --git a/Lib/test/test_time.py b/Lib/test/test_time.py --- a/Lib/test/test_time.py +++ b/Lib/test/test_time.py @@ -177,6 +177,12 @@ def test_strftime_bounding_check(self): self._bounds_checking(lambda tup: time.strftime('', tup)) + def test_strftime_format_check(self): + for x in [ '', 'A', '%A', '%AA' ]: + for y in range(0x0, 0x10): + for z in [ '%', 'A%', 'AA%', '%A%', 'A%A%', '%#' ]: + self.assertRaises(ValueError, time.strftime, x * y + z) + def test_default_values_for_zero(self): # Make sure that using all zeros uses the proper default # values. No test for daylight savings since strftime() does diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -179,6 +179,8 @@ Library ------- +- Issue #24917: time_strftime() Buffer Over-read. Patch by John Leitch. + - Issue #24635: Fixed a bug in typing.py where isinstance([], typing.Iterable) would return True once, then False on subsequent calls. diff --git a/Modules/timemodule.c b/Modules/timemodule.c --- a/Modules/timemodule.c +++ b/Modules/timemodule.c @@ -623,6 +623,12 @@ Py_DECREF(format); return NULL; } + else if (outbuf[1] == '\0') + { + PyErr_SetString(PyExc_ValueError, "Incomplete format string"); + Py_DECREF(format); + return NULL; + } } #elif (defined(_AIX) || defined(sun)) && defined(HAVE_WCSFTIME) for(outbuf = wcschr(fmt, '%'); @@ -636,6 +642,12 @@ "format %y requires year >= 1900 on AIX"); return NULL; } + else if (outbuf[1] == '\0') + { + PyErr_SetString(PyExc_ValueError, "Incomplete format string"); + Py_DECREF(format); + return NULL; + } } #endif -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 6 06:02:05 2015 From: python-checkins at python.org (steve.dower) Date: Sun, 06 Sep 2015 04:02:05 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI0OTE3?= =?utf-8?q?=3A_Moves_NEWS_entry_under_Library=2E?= Message-ID: <20150906040205.15704.29313@psf.io> https://hg.python.org/cpython/rev/8b81b7ad2d0a changeset: 97693:8b81b7ad2d0a branch: 3.5 user: Steve Dower date: Sat Sep 05 12:23:00 2015 -0700 summary: Issue #24917: Moves NEWS entry under Library. files: Misc/NEWS | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -10,8 +10,6 @@ Core and Builtins ----------------- -- Issue #24917: time_strftime() Buffer Over-read. Patch by John Leitch. - - Issue #24912: Prevent __class__ assignment to immutable built-in objects. - Issue #24975: Fix AST compilation for PEP 448 syntax. @@ -19,6 +17,8 @@ Library ------- +- Issue #24917: time_strftime() Buffer Over-read. Patch by John Leitch. + - Issue #24635: Fixed a bug in typing.py where isinstance([], typing.Iterable) would return True once, then False on subsequent calls. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 6 06:02:05 2015 From: python-checkins at python.org (steve.dower) Date: Sun, 06 Sep 2015 04:02:05 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI0OTE3?= =?utf-8?q?=3A_time=5Fstrftime=28=29_Buffer_Over-read=2E_Patch_by_John_Lei?= =?utf-8?q?tch=2E?= Message-ID: <20150906040205.11260.46305@psf.io> https://hg.python.org/cpython/rev/09b62202d9b7 changeset: 97692:09b62202d9b7 branch: 3.5 parent: 97683:27cc5cce0292 user: Steve Dower date: Sat Sep 05 12:16:06 2015 -0700 summary: Issue #24917: time_strftime() Buffer Over-read. Patch by John Leitch. files: Lib/test/test_time.py | 6 ++++++ Misc/NEWS | 2 ++ Modules/timemodule.c | 12 ++++++++++++ 3 files changed, 20 insertions(+), 0 deletions(-) diff --git a/Lib/test/test_time.py b/Lib/test/test_time.py --- a/Lib/test/test_time.py +++ b/Lib/test/test_time.py @@ -174,6 +174,12 @@ def test_strftime_bounding_check(self): self._bounds_checking(lambda tup: time.strftime('', tup)) + def test_strftime_format_check(self): + for x in [ '', 'A', '%A', '%AA' ]: + for y in range(0x0, 0x10): + for z in [ '%', 'A%', 'AA%', '%A%', 'A%A%', '%#' ]: + self.assertRaises(ValueError, time.strftime, x * y + z) + def test_default_values_for_zero(self): # Make sure that using all zeros uses the proper default # values. No test for daylight savings since strftime() does diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -10,6 +10,8 @@ Core and Builtins ----------------- +- Issue #24917: time_strftime() Buffer Over-read. Patch by John Leitch. + - Issue #24912: Prevent __class__ assignment to immutable built-in objects. - Issue #24975: Fix AST compilation for PEP 448 syntax. diff --git a/Modules/timemodule.c b/Modules/timemodule.c --- a/Modules/timemodule.c +++ b/Modules/timemodule.c @@ -623,6 +623,12 @@ Py_DECREF(format); return NULL; } + else if (outbuf[1] == '\0') + { + PyErr_SetString(PyExc_ValueError, "Incomplete format string"); + Py_DECREF(format); + return NULL; + } } #elif (defined(_AIX) || defined(sun)) && defined(HAVE_WCSFTIME) for(outbuf = wcschr(fmt, '%'); @@ -636,6 +642,12 @@ "format %y requires year >= 1900 on AIX"); return NULL; } + else if (outbuf[1] == '\0') + { + PyErr_SetString(PyExc_ValueError, "Incomplete format string"); + Py_DECREF(format); + return NULL; + } } #endif -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 6 06:02:06 2015 From: python-checkins at python.org (steve.dower) Date: Sun, 06 Sep 2015 04:02:06 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy41IC0+IDMuNSk6?= =?utf-8?q?_Merge_from_3=2E5=2E0_release_branch?= Message-ID: <20150906040205.27711.16866@psf.io> https://hg.python.org/cpython/rev/ddb6e423e251 changeset: 97695:ddb6e423e251 branch: 3.5 parent: 97690:ba63067654d3 parent: 97693:8b81b7ad2d0a user: Steve Dower date: Sat Sep 05 20:59:20 2015 -0700 summary: Merge from 3.5.0 release branch files: Lib/test/test_time.py | 6 ++++++ Misc/NEWS | 2 ++ Modules/timemodule.c | 12 ++++++++++++ 3 files changed, 20 insertions(+), 0 deletions(-) diff --git a/Lib/test/test_time.py b/Lib/test/test_time.py --- a/Lib/test/test_time.py +++ b/Lib/test/test_time.py @@ -174,6 +174,12 @@ def test_strftime_bounding_check(self): self._bounds_checking(lambda tup: time.strftime('', tup)) + def test_strftime_format_check(self): + for x in [ '', 'A', '%A', '%AA' ]: + for y in range(0x0, 0x10): + for z in [ '%', 'A%', 'AA%', '%A%', 'A%A%', '%#' ]: + self.assertRaises(ValueError, time.strftime, x * y + z) + def test_default_values_for_zero(self): # Make sure that using all zeros uses the proper default # values. No test for daylight savings since strftime() does diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -95,6 +95,8 @@ Library ------- +- Issue #24917: time_strftime() Buffer Over-read. Patch by John Leitch. + - Issue #24635: Fixed a bug in typing.py where isinstance([], typing.Iterable) would return True once, then False on subsequent calls. diff --git a/Modules/timemodule.c b/Modules/timemodule.c --- a/Modules/timemodule.c +++ b/Modules/timemodule.c @@ -623,6 +623,12 @@ Py_DECREF(format); return NULL; } + else if (outbuf[1] == '\0') + { + PyErr_SetString(PyExc_ValueError, "Incomplete format string"); + Py_DECREF(format); + return NULL; + } } #elif (defined(_AIX) || defined(sun)) && defined(HAVE_WCSFTIME) for(outbuf = wcschr(fmt, '%'); @@ -636,6 +642,12 @@ "format %y requires year >= 1900 on AIX"); return NULL; } + else if (outbuf[1] == '\0') + { + PyErr_SetString(PyExc_ValueError, "Incomplete format string"); + Py_DECREF(format); + return NULL; + } } #endif -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 6 08:14:42 2015 From: python-checkins at python.org (steve.dower) Date: Sun, 06 Sep 2015 06:14:42 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_Backed_out_cha?= =?utf-8?q?ngeset=3A_a29b49d57769?= Message-ID: <20150906061442.101504.52984@psf.io> https://hg.python.org/cpython/rev/f15c5b0f51a4 changeset: 97698:f15c5b0f51a4 branch: 3.4 parent: 97694:a29b49d57769 user: Steve Dower date: Sat Sep 05 23:09:00 2015 -0700 summary: Backed out changeset: a29b49d57769 files: Misc/NEWS | 2 -- Modules/timemodule.c | 6 ------ 2 files changed, 0 insertions(+), 8 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -84,8 +84,6 @@ - Issue #16180: Exit pdb if file has syntax error, instead of trapping user in an infinite loop. Patch by Xavier de Gaye. -- Issue #24917: time_strftime() Buffer Over-read. Patch by John Leitch. - - Issue #21112: Fix regression in unittest.expectedFailure on subclasses. Patch from Berker Peksag. diff --git a/Modules/timemodule.c b/Modules/timemodule.c --- a/Modules/timemodule.c +++ b/Modules/timemodule.c @@ -662,12 +662,6 @@ "format %y requires year >= 1900 on AIX"); return NULL; } - else if (outbuf[1] == '\0') - { - PyErr_SetString(PyExc_ValueError, "Incomplete format string"); - Py_DECREF(format); - return NULL; - } } #endif -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 6 08:14:43 2015 From: python-checkins at python.org (steve.dower) Date: Sun, 06 Sep 2015 06:14:43 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2324917=3A_Backed_out_changeset_09b62202d9b7?= Message-ID: <20150906061443.66874.2129@psf.io> https://hg.python.org/cpython/rev/743924064dc8 changeset: 97700:743924064dc8 parent: 97697:6fc744ac3953 parent: 97699:73227e857d6b user: Steve Dower date: Sat Sep 05 23:12:18 2015 -0700 summary: Issue #24917: Backed out changeset 09b62202d9b7 files: Lib/test/test_time.py | 6 ------ Misc/NEWS | 2 -- Modules/timemodule.c | 12 ------------ 3 files changed, 0 insertions(+), 20 deletions(-) diff --git a/Lib/test/test_time.py b/Lib/test/test_time.py --- a/Lib/test/test_time.py +++ b/Lib/test/test_time.py @@ -177,12 +177,6 @@ def test_strftime_bounding_check(self): self._bounds_checking(lambda tup: time.strftime('', tup)) - def test_strftime_format_check(self): - for x in [ '', 'A', '%A', '%AA' ]: - for y in range(0x0, 0x10): - for z in [ '%', 'A%', 'AA%', '%A%', 'A%A%', '%#' ]: - self.assertRaises(ValueError, time.strftime, x * y + z) - def test_default_values_for_zero(self): # Make sure that using all zeros uses the proper default # values. No test for daylight savings since strftime() does diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -179,8 +179,6 @@ Library ------- -- Issue #24917: time_strftime() Buffer Over-read. Patch by John Leitch. - - Issue #24635: Fixed a bug in typing.py where isinstance([], typing.Iterable) would return True once, then False on subsequent calls. diff --git a/Modules/timemodule.c b/Modules/timemodule.c --- a/Modules/timemodule.c +++ b/Modules/timemodule.c @@ -623,12 +623,6 @@ Py_DECREF(format); return NULL; } - else if (outbuf[1] == '\0') - { - PyErr_SetString(PyExc_ValueError, "Incomplete format string"); - Py_DECREF(format); - return NULL; - } } #elif (defined(_AIX) || defined(sun)) && defined(HAVE_WCSFTIME) for(outbuf = wcschr(fmt, '%'); @@ -642,12 +636,6 @@ "format %y requires year >= 1900 on AIX"); return NULL; } - else if (outbuf[1] == '\0') - { - PyErr_SetString(PyExc_ValueError, "Incomplete format string"); - Py_DECREF(format); - return NULL; - } } #endif -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 6 08:14:43 2015 From: python-checkins at python.org (steve.dower) Date: Sun, 06 Sep 2015 06:14:43 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Issue_=2324917=3A_Backed_out_changeset_09b62202d9b7?= Message-ID: <20150906061442.101492.51874@psf.io> https://hg.python.org/cpython/rev/73227e857d6b changeset: 97699:73227e857d6b branch: 3.5 parent: 97696:7cdadcc1002d parent: 97698:f15c5b0f51a4 user: Steve Dower date: Sat Sep 05 23:11:53 2015 -0700 summary: Issue #24917: Backed out changeset 09b62202d9b7 files: Lib/test/test_time.py | 6 ------ Misc/NEWS | 2 -- Modules/timemodule.c | 12 ------------ 3 files changed, 0 insertions(+), 20 deletions(-) diff --git a/Lib/test/test_time.py b/Lib/test/test_time.py --- a/Lib/test/test_time.py +++ b/Lib/test/test_time.py @@ -174,12 +174,6 @@ def test_strftime_bounding_check(self): self._bounds_checking(lambda tup: time.strftime('', tup)) - def test_strftime_format_check(self): - for x in [ '', 'A', '%A', '%AA' ]: - for y in range(0x0, 0x10): - for z in [ '%', 'A%', 'AA%', '%A%', 'A%A%', '%#' ]: - self.assertRaises(ValueError, time.strftime, x * y + z) - def test_default_values_for_zero(self): # Make sure that using all zeros uses the proper default # values. No test for daylight savings since strftime() does diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -95,8 +95,6 @@ Library ------- -- Issue #24917: time_strftime() Buffer Over-read. Patch by John Leitch. - - Issue #24635: Fixed a bug in typing.py where isinstance([], typing.Iterable) would return True once, then False on subsequent calls. diff --git a/Modules/timemodule.c b/Modules/timemodule.c --- a/Modules/timemodule.c +++ b/Modules/timemodule.c @@ -623,12 +623,6 @@ Py_DECREF(format); return NULL; } - else if (outbuf[1] == '\0') - { - PyErr_SetString(PyExc_ValueError, "Incomplete format string"); - Py_DECREF(format); - return NULL; - } } #elif (defined(_AIX) || defined(sun)) && defined(HAVE_WCSFTIME) for(outbuf = wcschr(fmt, '%'); @@ -642,12 +636,6 @@ "format %y requires year >= 1900 on AIX"); return NULL; } - else if (outbuf[1] == '\0') - { - PyErr_SetString(PyExc_ValueError, "Incomplete format string"); - Py_DECREF(format); - return NULL; - } } #endif -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 6 13:17:34 2015 From: python-checkins at python.org (serhiy.storchaka) Date: Sun, 06 Sep 2015 11:17:34 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_Backport_suppo?= =?utf-8?q?rt=2Echange=5Fcwd=28=29_and_use_it_in_tests=2E?= Message-ID: <20150906111734.11274.18883@psf.io> https://hg.python.org/cpython/rev/7969c9f02a9f changeset: 97704:7969c9f02a9f branch: 2.7 parent: 97686:26c4db1a0aea user: Serhiy Storchaka date: Sun Sep 06 14:16:18 2015 +0300 summary: Backport support.change_cwd() and use it in tests. files: Lib/test/test_pep277.py | 8 +--- Lib/test/test_posixpath.py | 38 +++++++------------ Lib/test/test_py_compile.py | 12 ++--- Lib/test/test_shutil.py | 46 ++++------------------ Lib/test/test_support.py | 27 +++++++++++++ Lib/test/test_tarfile.py | 7 +-- Lib/test/test_unicode_file.py | 8 +--- 7 files changed, 60 insertions(+), 86 deletions(-) diff --git a/Lib/test/test_pep277.py b/Lib/test/test_pep277.py --- a/Lib/test/test_pep277.py +++ b/Lib/test/test_pep277.py @@ -164,17 +164,11 @@ dirname = os.path.join(test_support.TESTFN, u'Gr\xfc\xdf-\u66e8\u66e9\u66eb') filename = u'\xdf-\u66e8\u66e9\u66eb' - oldwd = os.getcwd() - os.mkdir(dirname) - os.chdir(dirname) - try: + with test_support.temp_cwd(dirname): with open(filename, 'w') as f: f.write((filename + '\n').encode("utf-8")) os.access(filename,os.R_OK) os.remove(filename) - finally: - os.chdir(oldwd) - os.rmdir(dirname) class UnicodeNFCFileTests(UnicodeFileTests): diff --git a/Lib/test/test_posixpath.py b/Lib/test/test_posixpath.py --- a/Lib/test/test_posixpath.py +++ b/Lib/test/test_posixpath.py @@ -1,5 +1,6 @@ import unittest from test import test_support, test_genericpath +from test import test_support as support import posixpath import os @@ -251,7 +252,6 @@ # Bug #930024, return the path unchanged if we get into an infinite # symlink loop. try: - old_path = abspath('.') os.symlink(ABSTFN, ABSTFN) self.assertEqual(realpath(ABSTFN), ABSTFN) @@ -277,10 +277,9 @@ self.assertEqual(realpath(ABSTFN+"c"), ABSTFN+"c") # Test using relative path as well. - os.chdir(dirname(ABSTFN)) - self.assertEqual(realpath(basename(ABSTFN)), ABSTFN) + with support.change_cwd(dirname(ABSTFN)): + self.assertEqual(realpath(basename(ABSTFN)), ABSTFN) finally: - os.chdir(old_path) test_support.unlink(ABSTFN) test_support.unlink(ABSTFN+"1") test_support.unlink(ABSTFN+"2") @@ -302,7 +301,6 @@ def test_realpath_deep_recursion(self): depth = 10 - old_path = abspath('.') try: os.mkdir(ABSTFN) for i in range(depth): @@ -311,10 +309,9 @@ self.assertEqual(realpath(ABSTFN + '/%d' % depth), ABSTFN) # Test using relative path as well. - os.chdir(ABSTFN) - self.assertEqual(realpath('%d' % depth), ABSTFN) + with support.change_cwd(ABSTFN): + self.assertEqual(realpath('%d' % depth), ABSTFN) finally: - os.chdir(old_path) for i in range(depth + 1): test_support.unlink(ABSTFN + '/%d' % i) safe_rmdir(ABSTFN) @@ -325,15 +322,13 @@ # /usr/doc with 'doc' being a symlink to /usr/share/doc. We call # realpath("a"). This should return /usr/share/doc/a/. try: - old_path = abspath('.') os.mkdir(ABSTFN) os.mkdir(ABSTFN + "/y") os.symlink(ABSTFN + "/y", ABSTFN + "/k") - os.chdir(ABSTFN + "/k") - self.assertEqual(realpath("a"), ABSTFN + "/y/a") + with support.change_cwd(ABSTFN + "/k"): + self.assertEqual(realpath("a"), ABSTFN + "/y/a") finally: - os.chdir(old_path) test_support.unlink(ABSTFN + "/k") safe_rmdir(ABSTFN + "/y") safe_rmdir(ABSTFN) @@ -347,7 +342,6 @@ # and a symbolic link 'link-y' pointing to 'y' in directory 'a', # then realpath("link-y/..") should return 'k', not 'a'. try: - old_path = abspath('.') os.mkdir(ABSTFN) os.mkdir(ABSTFN + "/k") os.mkdir(ABSTFN + "/k/y") @@ -356,11 +350,10 @@ # Absolute path. self.assertEqual(realpath(ABSTFN + "/link-y/.."), ABSTFN + "/k") # Relative path. - os.chdir(dirname(ABSTFN)) - self.assertEqual(realpath(basename(ABSTFN) + "/link-y/.."), - ABSTFN + "/k") + with support.change_cwd(dirname(ABSTFN)): + self.assertEqual(realpath(basename(ABSTFN) + "/link-y/.."), + ABSTFN + "/k") finally: - os.chdir(old_path) test_support.unlink(ABSTFN + "/link-y") safe_rmdir(ABSTFN + "/k/y") safe_rmdir(ABSTFN + "/k") @@ -371,17 +364,14 @@ # must be resolved too. try: - old_path = abspath('.') os.mkdir(ABSTFN) os.mkdir(ABSTFN + "/k") os.symlink(ABSTFN, ABSTFN + "link") - os.chdir(dirname(ABSTFN)) - - base = basename(ABSTFN) - self.assertEqual(realpath(base + "link"), ABSTFN) - self.assertEqual(realpath(base + "link/k"), ABSTFN + "/k") + with support.change_cwd(dirname(ABSTFN)): + base = basename(ABSTFN) + self.assertEqual(realpath(base + "link"), ABSTFN) + self.assertEqual(realpath(base + "link/k"), ABSTFN + "/k") finally: - os.chdir(old_path) test_support.unlink(ABSTFN + "link") safe_rmdir(ABSTFN + "/k") safe_rmdir(ABSTFN) diff --git a/Lib/test/test_py_compile.py b/Lib/test/test_py_compile.py --- a/Lib/test/test_py_compile.py +++ b/Lib/test/test_py_compile.py @@ -5,7 +5,7 @@ import tempfile import unittest -from test import test_support +from test import test_support as support class PyCompileTests(unittest.TestCase): @@ -35,11 +35,9 @@ self.assertTrue(os.path.exists(self.pyc_path)) def test_cwd(self): - cwd = os.getcwd() - os.chdir(self.directory) - py_compile.compile(os.path.basename(self.source_path), - os.path.basename(self.pyc_path)) - os.chdir(cwd) + with support.change_cwd(self.directory): + py_compile.compile(os.path.basename(self.source_path), + os.path.basename(self.pyc_path)) self.assertTrue(os.path.exists(self.pyc_path)) def test_relative_path(self): @@ -48,7 +46,7 @@ self.assertTrue(os.path.exists(self.pyc_path)) def test_main(): - test_support.run_unittest(PyCompileTests) + support.run_unittest(PyCompileTests) if __name__ == "__main__": test_main() diff --git a/Lib/test/test_shutil.py b/Lib/test/test_shutil.py --- a/Lib/test/test_shutil.py +++ b/Lib/test/test_shutil.py @@ -16,7 +16,7 @@ import tarfile import warnings -from test import test_support +from test import test_support as support from test.test_support import TESTFN, check_warnings, captured_stdout TESTFN2 = TESTFN + "2" @@ -389,12 +389,8 @@ base_name = os.path.join(tmpdir2, 'archive') # working with relative paths to avoid tar warnings - old_dir = os.getcwd() - os.chdir(tmpdir) - try: + with support.change_cwd(tmpdir): _make_tarball(splitdrive(base_name)[1], '.') - finally: - os.chdir(old_dir) # check if the compressed tarball was created tarball = base_name + '.tar.gz' @@ -402,12 +398,8 @@ # trying an uncompressed one base_name = os.path.join(tmpdir2, 'archive') - old_dir = os.getcwd() - os.chdir(tmpdir) - try: + with support.change_cwd(tmpdir): _make_tarball(splitdrive(base_name)[1], '.', compress=None) - finally: - os.chdir(old_dir) tarball = base_name + '.tar' self.assertTrue(os.path.exists(tarball)) @@ -439,12 +431,8 @@ 'Need the tar command to run') def test_tarfile_vs_tar(self): tmpdir, tmpdir2, base_name = self._create_files() - old_dir = os.getcwd() - os.chdir(tmpdir) - try: + with support.change_cwd(tmpdir): _make_tarball(base_name, 'dist') - finally: - os.chdir(old_dir) # check if the compressed tarball was created tarball = base_name + '.tar.gz' @@ -454,14 +442,10 @@ tarball2 = os.path.join(tmpdir, 'archive2.tar.gz') tar_cmd = ['tar', '-cf', 'archive2.tar', 'dist'] gzip_cmd = ['gzip', '-f9', 'archive2.tar'] - old_dir = os.getcwd() - os.chdir(tmpdir) - try: + with support.change_cwd(tmpdir): with captured_stdout() as s: spawn(tar_cmd) spawn(gzip_cmd) - finally: - os.chdir(old_dir) self.assertTrue(os.path.exists(tarball2)) # let's compare both tarballs @@ -469,23 +453,15 @@ # trying an uncompressed one base_name = os.path.join(tmpdir2, 'archive') - old_dir = os.getcwd() - os.chdir(tmpdir) - try: + with support.change_cwd(tmpdir): _make_tarball(base_name, 'dist', compress=None) - finally: - os.chdir(old_dir) tarball = base_name + '.tar' self.assertTrue(os.path.exists(tarball)) # now for a dry_run base_name = os.path.join(tmpdir2, 'archive') - old_dir = os.getcwd() - os.chdir(tmpdir) - try: + with support.change_cwd(tmpdir): _make_tarball(base_name, 'dist', compress=None, dry_run=True) - finally: - os.chdir(old_dir) tarball = base_name + '.tar' self.assertTrue(os.path.exists(tarball)) @@ -544,15 +520,11 @@ @unittest.skipUnless(UID_GID_SUPPORT, "Requires grp and pwd support") def test_tarfile_root_owner(self): tmpdir, tmpdir2, base_name = self._create_files() - old_dir = os.getcwd() - os.chdir(tmpdir) group = grp.getgrgid(0)[0] owner = pwd.getpwuid(0)[0] - try: + with support.change_cwd(tmpdir): archive_name = _make_tarball(base_name, 'dist', compress=None, owner=owner, group=group) - finally: - os.chdir(old_dir) # check if the compressed tarball was created self.assertTrue(os.path.exists(archive_name)) @@ -889,7 +861,7 @@ def test_main(): - test_support.run_unittest(TestShutil, TestMove, TestCopyFile) + support.run_unittest(TestShutil, TestMove, TestCopyFile) if __name__ == '__main__': test_main() diff --git a/Lib/test/test_support.py b/Lib/test/test_support.py --- a/Lib/test/test_support.py +++ b/Lib/test/test_support.py @@ -669,6 +669,33 @@ SAVEDCWD = os.getcwd() @contextlib.contextmanager +def change_cwd(path, quiet=False): + """Return a context manager that changes the current working directory. + + Arguments: + + path: the directory to use as the temporary current working directory. + + quiet: if False (the default), the context manager raises an exception + on error. Otherwise, it issues only a warning and keeps the current + working directory the same. + + """ + saved_dir = os.getcwd() + try: + os.chdir(path) + except OSError: + if not quiet: + raise + warnings.warn('tests may fail, unable to change CWD to: ' + path, + RuntimeWarning, stacklevel=3) + try: + yield os.getcwd() + finally: + os.chdir(saved_dir) + + + at contextlib.contextmanager def temp_cwd(name='tempcwd', quiet=False): """ Context manager that creates a temporary directory and set it as CWD. diff --git a/Lib/test/test_tarfile.py b/Lib/test/test_tarfile.py --- a/Lib/test/test_tarfile.py +++ b/Lib/test/test_tarfile.py @@ -11,6 +11,7 @@ import tarfile from test import test_support +from test import test_support as support # Check for our compression modules. try: @@ -972,9 +973,7 @@ def test_cwd(self): # Test adding the current working directory. - cwd = os.getcwd() - os.chdir(TEMPDIR) - try: + with support.change_cwd(TEMPDIR): open("foo", "w").close() tar = tarfile.open(tmpname, self.mode) @@ -985,8 +984,6 @@ for t in tar: self.assertTrue(t.name == "." or t.name.startswith("./")) tar.close() - finally: - os.chdir(cwd) @unittest.skipUnless(hasattr(os, 'symlink'), "needs os.symlink") def test_extractall_symlinks(self): diff --git a/Lib/test/test_unicode_file.py b/Lib/test/test_unicode_file.py --- a/Lib/test/test_unicode_file.py +++ b/Lib/test/test_unicode_file.py @@ -5,7 +5,7 @@ import unicodedata import unittest -from test.test_support import run_unittest, TESTFN_UNICODE +from test.test_support import run_unittest, change_cwd, TESTFN_UNICODE from test.test_support import TESTFN_ENCODING, TESTFN_UNENCODABLE try: TESTFN_ENCODED = TESTFN_UNICODE.encode(TESTFN_ENCODING) @@ -114,13 +114,11 @@ os.unlink(filename1 + ".new") def _do_directory(self, make_name, chdir_name, encoded): - cwd = os.getcwd() if os.path.isdir(make_name): os.rmdir(make_name) os.mkdir(make_name) try: - os.chdir(chdir_name) - try: + with change_cwd(chdir_name): if not encoded: cwd_result = os.getcwdu() name_result = make_name @@ -132,8 +130,6 @@ name_result = unicodedata.normalize("NFD", name_result) self.assertEqual(os.path.basename(cwd_result),name_result) - finally: - os.chdir(cwd) finally: os.rmdir(make_name) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 6 13:17:34 2015 From: python-checkins at python.org (serhiy.storchaka) Date: Sun, 06 Sep 2015 11:17:34 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogVXNlIHN1cHBvcnQu?= =?utf-8?q?change=5Fcwd=28=29_in_tests=2E?= Message-ID: <20150906111734.68865.44368@psf.io> https://hg.python.org/cpython/rev/9774088c8ff7 changeset: 97701:9774088c8ff7 branch: 3.4 parent: 97698:f15c5b0f51a4 user: Serhiy Storchaka date: Sun Sep 06 14:13:25 2015 +0300 summary: Use support.change_cwd() in tests. files: Lib/test/test_pep277.py | 8 +--- Lib/test/test_posixpath.py | 37 ++++++------------ Lib/test/test_py_compile.py | 8 +-- Lib/test/test_shutil.py | 44 +++------------------- Lib/test/test_subprocess.py | 7 +-- Lib/test/test_sysconfig.py | 8 +--- Lib/test/test_tarfile.py | 12 +---- Lib/test/test_unicode_file.py | 8 +--- 8 files changed, 33 insertions(+), 99 deletions(-) diff --git a/Lib/test/test_pep277.py b/Lib/test/test_pep277.py --- a/Lib/test/test_pep277.py +++ b/Lib/test/test_pep277.py @@ -158,17 +158,11 @@ def test_directory(self): dirname = os.path.join(support.TESTFN, 'Gr\xfc\xdf-\u66e8\u66e9\u66eb') filename = '\xdf-\u66e8\u66e9\u66eb' - oldwd = os.getcwd() - os.mkdir(dirname) - os.chdir(dirname) - try: + with support.temp_cwd(dirname): with open(filename, 'wb') as f: f.write((filename + '\n').encode("utf-8")) os.access(filename,os.R_OK) os.remove(filename) - finally: - os.chdir(oldwd) - os.rmdir(dirname) class UnicodeNFCFileTests(UnicodeFileTests): diff --git a/Lib/test/test_posixpath.py b/Lib/test/test_posixpath.py --- a/Lib/test/test_posixpath.py +++ b/Lib/test/test_posixpath.py @@ -328,7 +328,6 @@ # Bug #930024, return the path unchanged if we get into an infinite # symlink loop. try: - old_path = abspath('.') os.symlink(ABSTFN, ABSTFN) self.assertEqual(realpath(ABSTFN), ABSTFN) @@ -354,10 +353,9 @@ self.assertEqual(realpath(ABSTFN+"c"), ABSTFN+"c") # Test using relative path as well. - os.chdir(dirname(ABSTFN)) - self.assertEqual(realpath(basename(ABSTFN)), ABSTFN) + with support.change_cwd(dirname(ABSTFN)): + self.assertEqual(realpath(basename(ABSTFN)), ABSTFN) finally: - os.chdir(old_path) support.unlink(ABSTFN) support.unlink(ABSTFN+"1") support.unlink(ABSTFN+"2") @@ -385,7 +383,6 @@ @skip_if_ABSTFN_contains_backslash def test_realpath_deep_recursion(self): depth = 10 - old_path = abspath('.') try: os.mkdir(ABSTFN) for i in range(depth): @@ -394,10 +391,9 @@ self.assertEqual(realpath(ABSTFN + '/%d' % depth), ABSTFN) # Test using relative path as well. - os.chdir(ABSTFN) - self.assertEqual(realpath('%d' % depth), ABSTFN) + with support.change_cwd(ABSTFN): + self.assertEqual(realpath('%d' % depth), ABSTFN) finally: - os.chdir(old_path) for i in range(depth + 1): support.unlink(ABSTFN + '/%d' % i) safe_rmdir(ABSTFN) @@ -411,15 +407,13 @@ # /usr/doc with 'doc' being a symlink to /usr/share/doc. We call # realpath("a"). This should return /usr/share/doc/a/. try: - old_path = abspath('.') os.mkdir(ABSTFN) os.mkdir(ABSTFN + "/y") os.symlink(ABSTFN + "/y", ABSTFN + "/k") - os.chdir(ABSTFN + "/k") - self.assertEqual(realpath("a"), ABSTFN + "/y/a") + with support.change_cwd(ABSTFN + "/k"): + self.assertEqual(realpath("a"), ABSTFN + "/y/a") finally: - os.chdir(old_path) support.unlink(ABSTFN + "/k") safe_rmdir(ABSTFN + "/y") safe_rmdir(ABSTFN) @@ -436,7 +430,6 @@ # and a symbolic link 'link-y' pointing to 'y' in directory 'a', # then realpath("link-y/..") should return 'k', not 'a'. try: - old_path = abspath('.') os.mkdir(ABSTFN) os.mkdir(ABSTFN + "/k") os.mkdir(ABSTFN + "/k/y") @@ -445,11 +438,10 @@ # Absolute path. self.assertEqual(realpath(ABSTFN + "/link-y/.."), ABSTFN + "/k") # Relative path. - os.chdir(dirname(ABSTFN)) - self.assertEqual(realpath(basename(ABSTFN) + "/link-y/.."), - ABSTFN + "/k") + with support.change_cwd(dirname(ABSTFN)): + self.assertEqual(realpath(basename(ABSTFN) + "/link-y/.."), + ABSTFN + "/k") finally: - os.chdir(old_path) support.unlink(ABSTFN + "/link-y") safe_rmdir(ABSTFN + "/k/y") safe_rmdir(ABSTFN + "/k") @@ -463,17 +455,14 @@ # must be resolved too. try: - old_path = abspath('.') os.mkdir(ABSTFN) os.mkdir(ABSTFN + "/k") os.symlink(ABSTFN, ABSTFN + "link") - os.chdir(dirname(ABSTFN)) - - base = basename(ABSTFN) - self.assertEqual(realpath(base + "link"), ABSTFN) - self.assertEqual(realpath(base + "link/k"), ABSTFN + "/k") + with support.change_cwd(dirname(ABSTFN)): + base = basename(ABSTFN) + self.assertEqual(realpath(base + "link"), ABSTFN) + self.assertEqual(realpath(base + "link/k"), ABSTFN + "/k") finally: - os.chdir(old_path) support.unlink(ABSTFN + "link") safe_rmdir(ABSTFN + "/k") safe_rmdir(ABSTFN) diff --git a/Lib/test/test_py_compile.py b/Lib/test/test_py_compile.py --- a/Lib/test/test_py_compile.py +++ b/Lib/test/test_py_compile.py @@ -63,11 +63,9 @@ self.assertTrue(os.path.exists(self.cache_path)) def test_cwd(self): - cwd = os.getcwd() - os.chdir(self.directory) - py_compile.compile(os.path.basename(self.source_path), - os.path.basename(self.pyc_path)) - os.chdir(cwd) + with support.change_cwd(self.directory): + py_compile.compile(os.path.basename(self.source_path), + os.path.basename(self.pyc_path)) self.assertTrue(os.path.exists(self.pyc_path)) self.assertFalse(os.path.exists(self.cache_path)) diff --git a/Lib/test/test_shutil.py b/Lib/test/test_shutil.py --- a/Lib/test/test_shutil.py +++ b/Lib/test/test_shutil.py @@ -12,8 +12,6 @@ import functools import subprocess from contextlib import ExitStack -from test import support -from test.support import TESTFN from os.path import splitdrive from distutils.spawn import find_executable, spawn from shutil import (_make_tarball, _make_zipfile, make_archive, @@ -968,12 +966,8 @@ base_name = os.path.join(tmpdir2, 'archive') # working with relative paths to avoid tar warnings - old_dir = os.getcwd() - os.chdir(tmpdir) - try: + with support.change_cwd(tmpdir): _make_tarball(splitdrive(base_name)[1], '.') - finally: - os.chdir(old_dir) # check if the compressed tarball was created tarball = base_name + '.tar.gz' @@ -981,12 +975,8 @@ # trying an uncompressed one base_name = os.path.join(tmpdir2, 'archive') - old_dir = os.getcwd() - os.chdir(tmpdir) - try: + with support.change_cwd(tmpdir): _make_tarball(splitdrive(base_name)[1], '.', compress=None) - finally: - os.chdir(old_dir) tarball = base_name + '.tar' self.assertTrue(os.path.exists(tarball)) @@ -1018,12 +1008,8 @@ 'Need the tar command to run') def test_tarfile_vs_tar(self): tmpdir, tmpdir2, base_name = self._create_files() - old_dir = os.getcwd() - os.chdir(tmpdir) - try: + with support.change_cwd(tmpdir): _make_tarball(base_name, 'dist') - finally: - os.chdir(old_dir) # check if the compressed tarball was created tarball = base_name + '.tar.gz' @@ -1033,14 +1019,10 @@ tarball2 = os.path.join(tmpdir, 'archive2.tar.gz') tar_cmd = ['tar', '-cf', 'archive2.tar', 'dist'] gzip_cmd = ['gzip', '-f9', 'archive2.tar'] - old_dir = os.getcwd() - os.chdir(tmpdir) - try: + with support.change_cwd(tmpdir): with captured_stdout() as s: spawn(tar_cmd) spawn(gzip_cmd) - finally: - os.chdir(old_dir) self.assertTrue(os.path.exists(tarball2)) # let's compare both tarballs @@ -1048,23 +1030,15 @@ # trying an uncompressed one base_name = os.path.join(tmpdir2, 'archive') - old_dir = os.getcwd() - os.chdir(tmpdir) - try: + with support.change_cwd(tmpdir): _make_tarball(base_name, 'dist', compress=None) - finally: - os.chdir(old_dir) tarball = base_name + '.tar' self.assertTrue(os.path.exists(tarball)) # now for a dry_run base_name = os.path.join(tmpdir2, 'archive') - old_dir = os.getcwd() - os.chdir(tmpdir) - try: + with support.change_cwd(tmpdir): _make_tarball(base_name, 'dist', compress=None, dry_run=True) - finally: - os.chdir(old_dir) tarball = base_name + '.tar' self.assertTrue(os.path.exists(tarball)) @@ -1124,15 +1098,11 @@ @unittest.skipUnless(UID_GID_SUPPORT, "Requires grp and pwd support") def test_tarfile_root_owner(self): tmpdir, tmpdir2, base_name = self._create_files() - old_dir = os.getcwd() - os.chdir(tmpdir) group = grp.getgrgid(0)[0] owner = pwd.getpwuid(0)[0] - try: + with support.change_cwd(tmpdir): archive_name = _make_tarball(base_name, 'dist', compress=None, owner=owner, group=group) - finally: - os.chdir(old_dir) # check if the compressed tarball was created self.assertTrue(os.path.exists(archive_name)) diff --git a/Lib/test/test_subprocess.py b/Lib/test/test_subprocess.py --- a/Lib/test/test_subprocess.py +++ b/Lib/test/test_subprocess.py @@ -317,11 +317,8 @@ # Normalize an expected cwd (for Tru64 support). # We can't use os.path.realpath since it doesn't expand Tru64 {memb} # strings. See bug #1063571. - original_cwd = os.getcwd() - os.chdir(cwd) - cwd = os.getcwd() - os.chdir(original_cwd) - return cwd + with support.change_cwd(cwd): + return os.getcwd() # For use in the test_cwd* tests below. def _split_python_path(self): diff --git a/Lib/test/test_sysconfig.py b/Lib/test/test_sysconfig.py --- a/Lib/test/test_sysconfig.py +++ b/Lib/test/test_sysconfig.py @@ -6,7 +6,7 @@ from copy import copy from test.support import (run_unittest, TESTFN, unlink, check_warnings, - captured_stdout, skip_unless_symlink) + captured_stdout, skip_unless_symlink, change_cwd) import sysconfig from sysconfig import (get_paths, get_platform, get_config_vars, @@ -361,12 +361,8 @@ # srcdir should be independent of the current working directory # See Issues #15322, #15364. srcdir = sysconfig.get_config_var('srcdir') - cwd = os.getcwd() - try: - os.chdir('..') + with change_cwd(os.pardir): srcdir2 = sysconfig.get_config_var('srcdir') - finally: - os.chdir(cwd) self.assertEqual(srcdir, srcdir2) @unittest.skipIf(sysconfig.get_config_var('EXT_SUFFIX') is None, diff --git a/Lib/test/test_tarfile.py b/Lib/test/test_tarfile.py --- a/Lib/test/test_tarfile.py +++ b/Lib/test/test_tarfile.py @@ -1104,10 +1104,8 @@ self.assertEqual(tar.getnames(), [], "added the archive to itself") - cwd = os.getcwd() - os.chdir(TEMPDIR) - tar.add(dstname) - os.chdir(cwd) + with support.change_cwd(TEMPDIR): + tar.add(dstname) self.assertEqual(tar.getnames(), [], "added the archive to itself") finally: @@ -1264,9 +1262,7 @@ def test_cwd(self): # Test adding the current working directory. - cwd = os.getcwd() - os.chdir(TEMPDIR) - try: + with support.change_cwd(TEMPDIR): tar = tarfile.open(tmpname, self.mode) try: tar.add(".") @@ -1280,8 +1276,6 @@ self.assertTrue(t.name.startswith("./"), t.name) finally: tar.close() - finally: - os.chdir(cwd) def test_open_nonwritable_fileobj(self): for exctype in OSError, EOFError, RuntimeError: diff --git a/Lib/test/test_unicode_file.py b/Lib/test/test_unicode_file.py --- a/Lib/test/test_unicode_file.py +++ b/Lib/test/test_unicode_file.py @@ -5,7 +5,7 @@ import unicodedata import unittest -from test.support import (run_unittest, rmtree, +from test.support import (run_unittest, rmtree, change_cwd, TESTFN_ENCODING, TESTFN_UNICODE, TESTFN_UNENCODABLE, create_empty_file) if not os.path.supports_unicode_filenames: @@ -82,13 +82,11 @@ self.assertFalse(os.path.exists(filename2 + '.new')) def _do_directory(self, make_name, chdir_name): - cwd = os.getcwd() if os.path.isdir(make_name): rmtree(make_name) os.mkdir(make_name) try: - os.chdir(chdir_name) - try: + with change_cwd(chdir_name): cwd_result = os.getcwd() name_result = make_name @@ -96,8 +94,6 @@ name_result = unicodedata.normalize("NFD", name_result) self.assertEqual(os.path.basename(cwd_result),name_result) - finally: - os.chdir(cwd) finally: os.rmdir(make_name) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 6 13:17:34 2015 From: python-checkins at python.org (serhiy.storchaka) Date: Sun, 06 Sep 2015 11:17:34 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Use_support=2Echange=5Fcwd=28=29_in_tests=2E?= Message-ID: <20150906111734.11268.31055@psf.io> https://hg.python.org/cpython/rev/21c44ba7b0ff changeset: 97703:21c44ba7b0ff parent: 97700:743924064dc8 parent: 97702:a318949ecf59 user: Serhiy Storchaka date: Sun Sep 06 14:15:40 2015 +0300 summary: Use support.change_cwd() in tests. files: Lib/test/test_glob.py | 6 +-- Lib/test/test_pep277.py | 8 +--- Lib/test/test_posixpath.py | 37 ++++++------------ Lib/test/test_py_compile.py | 8 +-- Lib/test/test_shutil.py | 44 +++------------------- Lib/test/test_subprocess.py | 7 +-- Lib/test/test_sysconfig.py | 8 +--- Lib/test/test_tarfile.py | 12 +---- Lib/test/test_unicode_file.py | 8 +--- 9 files changed, 34 insertions(+), 104 deletions(-) diff --git a/Lib/test/test_glob.py b/Lib/test/test_glob.py --- a/Lib/test/test_glob.py +++ b/Lib/test/test_glob.py @@ -242,9 +242,7 @@ ('a', 'bcd', 'EF'), ('a', 'bcd', 'efg'))) eq(self.rglob('a', '**', 'bcd'), self.joins(('a', 'bcd'))) - predir = os.path.abspath(os.curdir) - try: - os.chdir(self.tempdir) + with change_cwd(self.tempdir): join = os.path.join eq(glob.glob('**', recursive=True), [join(*i) for i in full]) eq(glob.glob(join('**', ''), recursive=True), @@ -256,8 +254,6 @@ if can_symlink(): expect += [join('sym3', 'EF')] eq(glob.glob(join('**', 'EF'), recursive=True), expect) - finally: - os.chdir(predir) @skip_unless_symlink diff --git a/Lib/test/test_pep277.py b/Lib/test/test_pep277.py --- a/Lib/test/test_pep277.py +++ b/Lib/test/test_pep277.py @@ -158,17 +158,11 @@ def test_directory(self): dirname = os.path.join(support.TESTFN, 'Gr\xfc\xdf-\u66e8\u66e9\u66eb') filename = '\xdf-\u66e8\u66e9\u66eb' - oldwd = os.getcwd() - os.mkdir(dirname) - os.chdir(dirname) - try: + with support.temp_cwd(dirname): with open(filename, 'wb') as f: f.write((filename + '\n').encode("utf-8")) os.access(filename,os.R_OK) os.remove(filename) - finally: - os.chdir(oldwd) - os.rmdir(dirname) class UnicodeNFCFileTests(UnicodeFileTests): diff --git a/Lib/test/test_posixpath.py b/Lib/test/test_posixpath.py --- a/Lib/test/test_posixpath.py +++ b/Lib/test/test_posixpath.py @@ -316,7 +316,6 @@ # Bug #930024, return the path unchanged if we get into an infinite # symlink loop. try: - old_path = abspath('.') os.symlink(ABSTFN, ABSTFN) self.assertEqual(realpath(ABSTFN), ABSTFN) @@ -342,10 +341,9 @@ self.assertEqual(realpath(ABSTFN+"c"), ABSTFN+"c") # Test using relative path as well. - os.chdir(dirname(ABSTFN)) - self.assertEqual(realpath(basename(ABSTFN)), ABSTFN) + with support.change_cwd(dirname(ABSTFN)): + self.assertEqual(realpath(basename(ABSTFN)), ABSTFN) finally: - os.chdir(old_path) support.unlink(ABSTFN) support.unlink(ABSTFN+"1") support.unlink(ABSTFN+"2") @@ -373,7 +371,6 @@ @skip_if_ABSTFN_contains_backslash def test_realpath_deep_recursion(self): depth = 10 - old_path = abspath('.') try: os.mkdir(ABSTFN) for i in range(depth): @@ -382,10 +379,9 @@ self.assertEqual(realpath(ABSTFN + '/%d' % depth), ABSTFN) # Test using relative path as well. - os.chdir(ABSTFN) - self.assertEqual(realpath('%d' % depth), ABSTFN) + with support.change_cwd(ABSTFN): + self.assertEqual(realpath('%d' % depth), ABSTFN) finally: - os.chdir(old_path) for i in range(depth + 1): support.unlink(ABSTFN + '/%d' % i) safe_rmdir(ABSTFN) @@ -399,15 +395,13 @@ # /usr/doc with 'doc' being a symlink to /usr/share/doc. We call # realpath("a"). This should return /usr/share/doc/a/. try: - old_path = abspath('.') os.mkdir(ABSTFN) os.mkdir(ABSTFN + "/y") os.symlink(ABSTFN + "/y", ABSTFN + "/k") - os.chdir(ABSTFN + "/k") - self.assertEqual(realpath("a"), ABSTFN + "/y/a") + with support.change_cwd(ABSTFN + "/k"): + self.assertEqual(realpath("a"), ABSTFN + "/y/a") finally: - os.chdir(old_path) support.unlink(ABSTFN + "/k") safe_rmdir(ABSTFN + "/y") safe_rmdir(ABSTFN) @@ -424,7 +418,6 @@ # and a symbolic link 'link-y' pointing to 'y' in directory 'a', # then realpath("link-y/..") should return 'k', not 'a'. try: - old_path = abspath('.') os.mkdir(ABSTFN) os.mkdir(ABSTFN + "/k") os.mkdir(ABSTFN + "/k/y") @@ -433,11 +426,10 @@ # Absolute path. self.assertEqual(realpath(ABSTFN + "/link-y/.."), ABSTFN + "/k") # Relative path. - os.chdir(dirname(ABSTFN)) - self.assertEqual(realpath(basename(ABSTFN) + "/link-y/.."), - ABSTFN + "/k") + with support.change_cwd(dirname(ABSTFN)): + self.assertEqual(realpath(basename(ABSTFN) + "/link-y/.."), + ABSTFN + "/k") finally: - os.chdir(old_path) support.unlink(ABSTFN + "/link-y") safe_rmdir(ABSTFN + "/k/y") safe_rmdir(ABSTFN + "/k") @@ -451,17 +443,14 @@ # must be resolved too. try: - old_path = abspath('.') os.mkdir(ABSTFN) os.mkdir(ABSTFN + "/k") os.symlink(ABSTFN, ABSTFN + "link") - os.chdir(dirname(ABSTFN)) - - base = basename(ABSTFN) - self.assertEqual(realpath(base + "link"), ABSTFN) - self.assertEqual(realpath(base + "link/k"), ABSTFN + "/k") + with support.change_cwd(dirname(ABSTFN)): + base = basename(ABSTFN) + self.assertEqual(realpath(base + "link"), ABSTFN) + self.assertEqual(realpath(base + "link/k"), ABSTFN + "/k") finally: - os.chdir(old_path) support.unlink(ABSTFN + "link") safe_rmdir(ABSTFN + "/k") safe_rmdir(ABSTFN) diff --git a/Lib/test/test_py_compile.py b/Lib/test/test_py_compile.py --- a/Lib/test/test_py_compile.py +++ b/Lib/test/test_py_compile.py @@ -63,11 +63,9 @@ self.assertTrue(os.path.exists(self.cache_path)) def test_cwd(self): - cwd = os.getcwd() - os.chdir(self.directory) - py_compile.compile(os.path.basename(self.source_path), - os.path.basename(self.pyc_path)) - os.chdir(cwd) + with support.change_cwd(self.directory): + py_compile.compile(os.path.basename(self.source_path), + os.path.basename(self.pyc_path)) self.assertTrue(os.path.exists(self.pyc_path)) self.assertFalse(os.path.exists(self.cache_path)) diff --git a/Lib/test/test_shutil.py b/Lib/test/test_shutil.py --- a/Lib/test/test_shutil.py +++ b/Lib/test/test_shutil.py @@ -12,8 +12,6 @@ import functools import subprocess from contextlib import ExitStack -from test import support -from test.support import TESTFN from os.path import splitdrive from distutils.spawn import find_executable, spawn from shutil import (_make_tarball, _make_zipfile, make_archive, @@ -974,12 +972,8 @@ base_name = os.path.join(tmpdir2, 'archive') # working with relative paths to avoid tar warnings - old_dir = os.getcwd() - os.chdir(tmpdir) - try: + with support.change_cwd(tmpdir): _make_tarball(splitdrive(base_name)[1], '.') - finally: - os.chdir(old_dir) # check if the compressed tarball was created tarball = base_name + '.tar.gz' @@ -987,12 +981,8 @@ # trying an uncompressed one base_name = os.path.join(tmpdir2, 'archive') - old_dir = os.getcwd() - os.chdir(tmpdir) - try: + with support.change_cwd(tmpdir): _make_tarball(splitdrive(base_name)[1], '.', compress=None) - finally: - os.chdir(old_dir) tarball = base_name + '.tar' self.assertTrue(os.path.exists(tarball)) @@ -1024,12 +1014,8 @@ 'Need the tar command to run') def test_tarfile_vs_tar(self): tmpdir, tmpdir2, base_name = self._create_files() - old_dir = os.getcwd() - os.chdir(tmpdir) - try: + with support.change_cwd(tmpdir): _make_tarball(base_name, 'dist') - finally: - os.chdir(old_dir) # check if the compressed tarball was created tarball = base_name + '.tar.gz' @@ -1039,14 +1025,10 @@ tarball2 = os.path.join(tmpdir, 'archive2.tar.gz') tar_cmd = ['tar', '-cf', 'archive2.tar', 'dist'] gzip_cmd = ['gzip', '-f9', 'archive2.tar'] - old_dir = os.getcwd() - os.chdir(tmpdir) - try: + with support.change_cwd(tmpdir): with captured_stdout() as s: spawn(tar_cmd) spawn(gzip_cmd) - finally: - os.chdir(old_dir) self.assertTrue(os.path.exists(tarball2)) # let's compare both tarballs @@ -1054,23 +1036,15 @@ # trying an uncompressed one base_name = os.path.join(tmpdir2, 'archive') - old_dir = os.getcwd() - os.chdir(tmpdir) - try: + with support.change_cwd(tmpdir): _make_tarball(base_name, 'dist', compress=None) - finally: - os.chdir(old_dir) tarball = base_name + '.tar' self.assertTrue(os.path.exists(tarball)) # now for a dry_run base_name = os.path.join(tmpdir2, 'archive') - old_dir = os.getcwd() - os.chdir(tmpdir) - try: + with support.change_cwd(tmpdir): _make_tarball(base_name, 'dist', compress=None, dry_run=True) - finally: - os.chdir(old_dir) tarball = base_name + '.tar' self.assertTrue(os.path.exists(tarball)) @@ -1130,15 +1104,11 @@ @unittest.skipUnless(UID_GID_SUPPORT, "Requires grp and pwd support") def test_tarfile_root_owner(self): tmpdir, tmpdir2, base_name = self._create_files() - old_dir = os.getcwd() - os.chdir(tmpdir) group = grp.getgrgid(0)[0] owner = pwd.getpwuid(0)[0] - try: + with support.change_cwd(tmpdir): archive_name = _make_tarball(base_name, 'dist', compress=None, owner=owner, group=group) - finally: - os.chdir(old_dir) # check if the compressed tarball was created self.assertTrue(os.path.exists(archive_name)) diff --git a/Lib/test/test_subprocess.py b/Lib/test/test_subprocess.py --- a/Lib/test/test_subprocess.py +++ b/Lib/test/test_subprocess.py @@ -317,11 +317,8 @@ # Normalize an expected cwd (for Tru64 support). # We can't use os.path.realpath since it doesn't expand Tru64 {memb} # strings. See bug #1063571. - original_cwd = os.getcwd() - os.chdir(cwd) - cwd = os.getcwd() - os.chdir(original_cwd) - return cwd + with support.change_cwd(cwd): + return os.getcwd() # For use in the test_cwd* tests below. def _split_python_path(self): diff --git a/Lib/test/test_sysconfig.py b/Lib/test/test_sysconfig.py --- a/Lib/test/test_sysconfig.py +++ b/Lib/test/test_sysconfig.py @@ -6,7 +6,7 @@ from copy import copy from test.support import (run_unittest, TESTFN, unlink, check_warnings, - captured_stdout, skip_unless_symlink) + captured_stdout, skip_unless_symlink, change_cwd) import sysconfig from sysconfig import (get_paths, get_platform, get_config_vars, @@ -361,12 +361,8 @@ # srcdir should be independent of the current working directory # See Issues #15322, #15364. srcdir = sysconfig.get_config_var('srcdir') - cwd = os.getcwd() - try: - os.chdir('..') + with change_cwd(os.pardir): srcdir2 = sysconfig.get_config_var('srcdir') - finally: - os.chdir(cwd) self.assertEqual(srcdir, srcdir2) @unittest.skipIf(sysconfig.get_config_var('EXT_SUFFIX') is None, diff --git a/Lib/test/test_tarfile.py b/Lib/test/test_tarfile.py --- a/Lib/test/test_tarfile.py +++ b/Lib/test/test_tarfile.py @@ -1132,10 +1132,8 @@ self.assertEqual(tar.getnames(), [], "added the archive to itself") - cwd = os.getcwd() - os.chdir(TEMPDIR) - tar.add(dstname) - os.chdir(cwd) + with support.change_cwd(TEMPDIR): + tar.add(dstname) self.assertEqual(tar.getnames(), [], "added the archive to itself") finally: @@ -1292,9 +1290,7 @@ def test_cwd(self): # Test adding the current working directory. - cwd = os.getcwd() - os.chdir(TEMPDIR) - try: + with support.change_cwd(TEMPDIR): tar = tarfile.open(tmpname, self.mode) try: tar.add(".") @@ -1308,8 +1304,6 @@ self.assertTrue(t.name.startswith("./"), t.name) finally: tar.close() - finally: - os.chdir(cwd) def test_open_nonwritable_fileobj(self): for exctype in OSError, EOFError, RuntimeError: diff --git a/Lib/test/test_unicode_file.py b/Lib/test/test_unicode_file.py --- a/Lib/test/test_unicode_file.py +++ b/Lib/test/test_unicode_file.py @@ -5,7 +5,7 @@ import unicodedata import unittest -from test.support import (run_unittest, rmtree, +from test.support import (run_unittest, rmtree, change_cwd, TESTFN_ENCODING, TESTFN_UNICODE, TESTFN_UNENCODABLE, create_empty_file) if not os.path.supports_unicode_filenames: @@ -82,13 +82,11 @@ self.assertFalse(os.path.exists(filename2 + '.new')) def _do_directory(self, make_name, chdir_name): - cwd = os.getcwd() if os.path.isdir(make_name): rmtree(make_name) os.mkdir(make_name) try: - os.chdir(chdir_name) - try: + with change_cwd(chdir_name): cwd_result = os.getcwd() name_result = make_name @@ -96,8 +94,6 @@ name_result = unicodedata.normalize("NFD", name_result) self.assertEqual(os.path.basename(cwd_result),name_result) - finally: - os.chdir(cwd) finally: os.rmdir(make_name) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 6 13:17:34 2015 From: python-checkins at python.org (serhiy.storchaka) Date: Sun, 06 Sep 2015 11:17:34 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Use_support=2Echange=5Fcwd=28=29_in_tests=2E?= Message-ID: <20150906111734.11266.13918@psf.io> https://hg.python.org/cpython/rev/a318949ecf59 changeset: 97702:a318949ecf59 branch: 3.5 parent: 97699:73227e857d6b parent: 97701:9774088c8ff7 user: Serhiy Storchaka date: Sun Sep 06 14:14:49 2015 +0300 summary: Use support.change_cwd() in tests. files: Lib/test/test_glob.py | 6 +-- Lib/test/test_pep277.py | 8 +--- Lib/test/test_posixpath.py | 37 ++++++------------ Lib/test/test_py_compile.py | 8 +-- Lib/test/test_shutil.py | 44 +++------------------- Lib/test/test_subprocess.py | 7 +-- Lib/test/test_sysconfig.py | 8 +--- Lib/test/test_tarfile.py | 12 +---- Lib/test/test_unicode_file.py | 8 +--- 9 files changed, 34 insertions(+), 104 deletions(-) diff --git a/Lib/test/test_glob.py b/Lib/test/test_glob.py --- a/Lib/test/test_glob.py +++ b/Lib/test/test_glob.py @@ -242,9 +242,7 @@ ('a', 'bcd', 'EF'), ('a', 'bcd', 'efg'))) eq(self.rglob('a', '**', 'bcd'), self.joins(('a', 'bcd'))) - predir = os.path.abspath(os.curdir) - try: - os.chdir(self.tempdir) + with change_cwd(self.tempdir): join = os.path.join eq(glob.glob('**', recursive=True), [join(*i) for i in full]) eq(glob.glob(join('**', ''), recursive=True), @@ -256,8 +254,6 @@ if can_symlink(): expect += [join('sym3', 'EF')] eq(glob.glob(join('**', 'EF'), recursive=True), expect) - finally: - os.chdir(predir) @skip_unless_symlink diff --git a/Lib/test/test_pep277.py b/Lib/test/test_pep277.py --- a/Lib/test/test_pep277.py +++ b/Lib/test/test_pep277.py @@ -158,17 +158,11 @@ def test_directory(self): dirname = os.path.join(support.TESTFN, 'Gr\xfc\xdf-\u66e8\u66e9\u66eb') filename = '\xdf-\u66e8\u66e9\u66eb' - oldwd = os.getcwd() - os.mkdir(dirname) - os.chdir(dirname) - try: + with support.temp_cwd(dirname): with open(filename, 'wb') as f: f.write((filename + '\n').encode("utf-8")) os.access(filename,os.R_OK) os.remove(filename) - finally: - os.chdir(oldwd) - os.rmdir(dirname) class UnicodeNFCFileTests(UnicodeFileTests): diff --git a/Lib/test/test_posixpath.py b/Lib/test/test_posixpath.py --- a/Lib/test/test_posixpath.py +++ b/Lib/test/test_posixpath.py @@ -316,7 +316,6 @@ # Bug #930024, return the path unchanged if we get into an infinite # symlink loop. try: - old_path = abspath('.') os.symlink(ABSTFN, ABSTFN) self.assertEqual(realpath(ABSTFN), ABSTFN) @@ -342,10 +341,9 @@ self.assertEqual(realpath(ABSTFN+"c"), ABSTFN+"c") # Test using relative path as well. - os.chdir(dirname(ABSTFN)) - self.assertEqual(realpath(basename(ABSTFN)), ABSTFN) + with support.change_cwd(dirname(ABSTFN)): + self.assertEqual(realpath(basename(ABSTFN)), ABSTFN) finally: - os.chdir(old_path) support.unlink(ABSTFN) support.unlink(ABSTFN+"1") support.unlink(ABSTFN+"2") @@ -373,7 +371,6 @@ @skip_if_ABSTFN_contains_backslash def test_realpath_deep_recursion(self): depth = 10 - old_path = abspath('.') try: os.mkdir(ABSTFN) for i in range(depth): @@ -382,10 +379,9 @@ self.assertEqual(realpath(ABSTFN + '/%d' % depth), ABSTFN) # Test using relative path as well. - os.chdir(ABSTFN) - self.assertEqual(realpath('%d' % depth), ABSTFN) + with support.change_cwd(ABSTFN): + self.assertEqual(realpath('%d' % depth), ABSTFN) finally: - os.chdir(old_path) for i in range(depth + 1): support.unlink(ABSTFN + '/%d' % i) safe_rmdir(ABSTFN) @@ -399,15 +395,13 @@ # /usr/doc with 'doc' being a symlink to /usr/share/doc. We call # realpath("a"). This should return /usr/share/doc/a/. try: - old_path = abspath('.') os.mkdir(ABSTFN) os.mkdir(ABSTFN + "/y") os.symlink(ABSTFN + "/y", ABSTFN + "/k") - os.chdir(ABSTFN + "/k") - self.assertEqual(realpath("a"), ABSTFN + "/y/a") + with support.change_cwd(ABSTFN + "/k"): + self.assertEqual(realpath("a"), ABSTFN + "/y/a") finally: - os.chdir(old_path) support.unlink(ABSTFN + "/k") safe_rmdir(ABSTFN + "/y") safe_rmdir(ABSTFN) @@ -424,7 +418,6 @@ # and a symbolic link 'link-y' pointing to 'y' in directory 'a', # then realpath("link-y/..") should return 'k', not 'a'. try: - old_path = abspath('.') os.mkdir(ABSTFN) os.mkdir(ABSTFN + "/k") os.mkdir(ABSTFN + "/k/y") @@ -433,11 +426,10 @@ # Absolute path. self.assertEqual(realpath(ABSTFN + "/link-y/.."), ABSTFN + "/k") # Relative path. - os.chdir(dirname(ABSTFN)) - self.assertEqual(realpath(basename(ABSTFN) + "/link-y/.."), - ABSTFN + "/k") + with support.change_cwd(dirname(ABSTFN)): + self.assertEqual(realpath(basename(ABSTFN) + "/link-y/.."), + ABSTFN + "/k") finally: - os.chdir(old_path) support.unlink(ABSTFN + "/link-y") safe_rmdir(ABSTFN + "/k/y") safe_rmdir(ABSTFN + "/k") @@ -451,17 +443,14 @@ # must be resolved too. try: - old_path = abspath('.') os.mkdir(ABSTFN) os.mkdir(ABSTFN + "/k") os.symlink(ABSTFN, ABSTFN + "link") - os.chdir(dirname(ABSTFN)) - - base = basename(ABSTFN) - self.assertEqual(realpath(base + "link"), ABSTFN) - self.assertEqual(realpath(base + "link/k"), ABSTFN + "/k") + with support.change_cwd(dirname(ABSTFN)): + base = basename(ABSTFN) + self.assertEqual(realpath(base + "link"), ABSTFN) + self.assertEqual(realpath(base + "link/k"), ABSTFN + "/k") finally: - os.chdir(old_path) support.unlink(ABSTFN + "link") safe_rmdir(ABSTFN + "/k") safe_rmdir(ABSTFN) diff --git a/Lib/test/test_py_compile.py b/Lib/test/test_py_compile.py --- a/Lib/test/test_py_compile.py +++ b/Lib/test/test_py_compile.py @@ -63,11 +63,9 @@ self.assertTrue(os.path.exists(self.cache_path)) def test_cwd(self): - cwd = os.getcwd() - os.chdir(self.directory) - py_compile.compile(os.path.basename(self.source_path), - os.path.basename(self.pyc_path)) - os.chdir(cwd) + with support.change_cwd(self.directory): + py_compile.compile(os.path.basename(self.source_path), + os.path.basename(self.pyc_path)) self.assertTrue(os.path.exists(self.pyc_path)) self.assertFalse(os.path.exists(self.cache_path)) diff --git a/Lib/test/test_shutil.py b/Lib/test/test_shutil.py --- a/Lib/test/test_shutil.py +++ b/Lib/test/test_shutil.py @@ -12,8 +12,6 @@ import functools import subprocess from contextlib import ExitStack -from test import support -from test.support import TESTFN from os.path import splitdrive from distutils.spawn import find_executable, spawn from shutil import (_make_tarball, _make_zipfile, make_archive, @@ -974,12 +972,8 @@ base_name = os.path.join(tmpdir2, 'archive') # working with relative paths to avoid tar warnings - old_dir = os.getcwd() - os.chdir(tmpdir) - try: + with support.change_cwd(tmpdir): _make_tarball(splitdrive(base_name)[1], '.') - finally: - os.chdir(old_dir) # check if the compressed tarball was created tarball = base_name + '.tar.gz' @@ -987,12 +981,8 @@ # trying an uncompressed one base_name = os.path.join(tmpdir2, 'archive') - old_dir = os.getcwd() - os.chdir(tmpdir) - try: + with support.change_cwd(tmpdir): _make_tarball(splitdrive(base_name)[1], '.', compress=None) - finally: - os.chdir(old_dir) tarball = base_name + '.tar' self.assertTrue(os.path.exists(tarball)) @@ -1024,12 +1014,8 @@ 'Need the tar command to run') def test_tarfile_vs_tar(self): tmpdir, tmpdir2, base_name = self._create_files() - old_dir = os.getcwd() - os.chdir(tmpdir) - try: + with support.change_cwd(tmpdir): _make_tarball(base_name, 'dist') - finally: - os.chdir(old_dir) # check if the compressed tarball was created tarball = base_name + '.tar.gz' @@ -1039,14 +1025,10 @@ tarball2 = os.path.join(tmpdir, 'archive2.tar.gz') tar_cmd = ['tar', '-cf', 'archive2.tar', 'dist'] gzip_cmd = ['gzip', '-f9', 'archive2.tar'] - old_dir = os.getcwd() - os.chdir(tmpdir) - try: + with support.change_cwd(tmpdir): with captured_stdout() as s: spawn(tar_cmd) spawn(gzip_cmd) - finally: - os.chdir(old_dir) self.assertTrue(os.path.exists(tarball2)) # let's compare both tarballs @@ -1054,23 +1036,15 @@ # trying an uncompressed one base_name = os.path.join(tmpdir2, 'archive') - old_dir = os.getcwd() - os.chdir(tmpdir) - try: + with support.change_cwd(tmpdir): _make_tarball(base_name, 'dist', compress=None) - finally: - os.chdir(old_dir) tarball = base_name + '.tar' self.assertTrue(os.path.exists(tarball)) # now for a dry_run base_name = os.path.join(tmpdir2, 'archive') - old_dir = os.getcwd() - os.chdir(tmpdir) - try: + with support.change_cwd(tmpdir): _make_tarball(base_name, 'dist', compress=None, dry_run=True) - finally: - os.chdir(old_dir) tarball = base_name + '.tar' self.assertTrue(os.path.exists(tarball)) @@ -1130,15 +1104,11 @@ @unittest.skipUnless(UID_GID_SUPPORT, "Requires grp and pwd support") def test_tarfile_root_owner(self): tmpdir, tmpdir2, base_name = self._create_files() - old_dir = os.getcwd() - os.chdir(tmpdir) group = grp.getgrgid(0)[0] owner = pwd.getpwuid(0)[0] - try: + with support.change_cwd(tmpdir): archive_name = _make_tarball(base_name, 'dist', compress=None, owner=owner, group=group) - finally: - os.chdir(old_dir) # check if the compressed tarball was created self.assertTrue(os.path.exists(archive_name)) diff --git a/Lib/test/test_subprocess.py b/Lib/test/test_subprocess.py --- a/Lib/test/test_subprocess.py +++ b/Lib/test/test_subprocess.py @@ -317,11 +317,8 @@ # Normalize an expected cwd (for Tru64 support). # We can't use os.path.realpath since it doesn't expand Tru64 {memb} # strings. See bug #1063571. - original_cwd = os.getcwd() - os.chdir(cwd) - cwd = os.getcwd() - os.chdir(original_cwd) - return cwd + with support.change_cwd(cwd): + return os.getcwd() # For use in the test_cwd* tests below. def _split_python_path(self): diff --git a/Lib/test/test_sysconfig.py b/Lib/test/test_sysconfig.py --- a/Lib/test/test_sysconfig.py +++ b/Lib/test/test_sysconfig.py @@ -6,7 +6,7 @@ from copy import copy from test.support import (run_unittest, TESTFN, unlink, check_warnings, - captured_stdout, skip_unless_symlink) + captured_stdout, skip_unless_symlink, change_cwd) import sysconfig from sysconfig import (get_paths, get_platform, get_config_vars, @@ -361,12 +361,8 @@ # srcdir should be independent of the current working directory # See Issues #15322, #15364. srcdir = sysconfig.get_config_var('srcdir') - cwd = os.getcwd() - try: - os.chdir('..') + with change_cwd(os.pardir): srcdir2 = sysconfig.get_config_var('srcdir') - finally: - os.chdir(cwd) self.assertEqual(srcdir, srcdir2) @unittest.skipIf(sysconfig.get_config_var('EXT_SUFFIX') is None, diff --git a/Lib/test/test_tarfile.py b/Lib/test/test_tarfile.py --- a/Lib/test/test_tarfile.py +++ b/Lib/test/test_tarfile.py @@ -1132,10 +1132,8 @@ self.assertEqual(tar.getnames(), [], "added the archive to itself") - cwd = os.getcwd() - os.chdir(TEMPDIR) - tar.add(dstname) - os.chdir(cwd) + with support.change_cwd(TEMPDIR): + tar.add(dstname) self.assertEqual(tar.getnames(), [], "added the archive to itself") finally: @@ -1292,9 +1290,7 @@ def test_cwd(self): # Test adding the current working directory. - cwd = os.getcwd() - os.chdir(TEMPDIR) - try: + with support.change_cwd(TEMPDIR): tar = tarfile.open(tmpname, self.mode) try: tar.add(".") @@ -1308,8 +1304,6 @@ self.assertTrue(t.name.startswith("./"), t.name) finally: tar.close() - finally: - os.chdir(cwd) def test_open_nonwritable_fileobj(self): for exctype in OSError, EOFError, RuntimeError: diff --git a/Lib/test/test_unicode_file.py b/Lib/test/test_unicode_file.py --- a/Lib/test/test_unicode_file.py +++ b/Lib/test/test_unicode_file.py @@ -5,7 +5,7 @@ import unicodedata import unittest -from test.support import (run_unittest, rmtree, +from test.support import (run_unittest, rmtree, change_cwd, TESTFN_ENCODING, TESTFN_UNICODE, TESTFN_UNENCODABLE, create_empty_file) if not os.path.supports_unicode_filenames: @@ -82,13 +82,11 @@ self.assertFalse(os.path.exists(filename2 + '.new')) def _do_directory(self, make_name, chdir_name): - cwd = os.getcwd() if os.path.isdir(make_name): rmtree(make_name) os.mkdir(make_name) try: - os.chdir(chdir_name) - try: + with change_cwd(chdir_name): cwd_result = os.getcwd() name_result = make_name @@ -96,8 +94,6 @@ name_result = unicodedata.normalize("NFD", name_result) self.assertEqual(os.path.basename(cwd_result),name_result) - finally: - os.chdir(cwd) finally: os.rmdir(make_name) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 6 17:35:33 2015 From: python-checkins at python.org (serhiy.storchaka) Date: Sun, 06 Sep 2015 15:35:33 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Fix=2C_refactor_and_extend_tests_for_shutil=2Emake=5Farc?= =?utf-8?b?aGl2ZSgpLg==?= Message-ID: <20150906153532.14871.42713@psf.io> https://hg.python.org/cpython/rev/4383ff47fffa changeset: 97708:4383ff47fffa parent: 97703:21c44ba7b0ff parent: 97707:b1b9f3be8007 user: Serhiy Storchaka date: Sun Sep 06 18:34:22 2015 +0300 summary: Fix, refactor and extend tests for shutil.make_archive(). files: Lib/test/test_shutil.py | 170 +++++++++++++-------------- 1 files changed, 82 insertions(+), 88 deletions(-) diff --git a/Lib/test/test_shutil.py b/Lib/test/test_shutil.py --- a/Lib/test/test_shutil.py +++ b/Lib/test/test_shutil.py @@ -14,7 +14,7 @@ from contextlib import ExitStack from os.path import splitdrive from distutils.spawn import find_executable, spawn -from shutil import (_make_tarball, _make_zipfile, make_archive, +from shutil import (make_archive, register_archive_format, unregister_archive_format, get_archive_formats, Error, unpack_archive, register_unpack_format, RegistryError, @@ -92,6 +92,18 @@ with open(path, 'rb' if binary else 'r') as fp: return fp.read() +def rlistdir(path): + res = [] + for name in sorted(os.listdir(path)): + p = os.path.join(path, name) + if os.path.isdir(p) and not os.path.islink(p): + res.append(name + '/') + for n in rlistdir(p): + res.append(name + '/' + n) + else: + res.append(name) + return res + class TestShutil(unittest.TestCase): @@ -957,114 +969,105 @@ @requires_zlib def test_make_tarball(self): # creating something to tar - tmpdir = self.mkdtemp() - write_file((tmpdir, 'file1'), 'xxx') - write_file((tmpdir, 'file2'), 'xxx') - os.mkdir(os.path.join(tmpdir, 'sub')) - write_file((tmpdir, 'sub', 'file3'), 'xxx') + root_dir, base_dir = self._create_files('') tmpdir2 = self.mkdtemp() # force shutil to create the directory os.rmdir(tmpdir2) - unittest.skipUnless(splitdrive(tmpdir)[0] == splitdrive(tmpdir2)[0], + unittest.skipUnless(splitdrive(root_dir)[0] == splitdrive(tmpdir2)[0], "source and target should be on same drive") base_name = os.path.join(tmpdir2, 'archive') # working with relative paths to avoid tar warnings - with support.change_cwd(tmpdir): - _make_tarball(splitdrive(base_name)[1], '.') + make_archive(splitdrive(base_name)[1], 'gztar', root_dir, '.') # check if the compressed tarball was created tarball = base_name + '.tar.gz' - self.assertTrue(os.path.exists(tarball)) + self.assertTrue(os.path.isfile(tarball)) + self.assertTrue(tarfile.is_tarfile(tarball)) + with tarfile.open(tarball, 'r:gz') as tf: + self.assertCountEqual(tf.getnames(), + ['.', './sub', './sub2', + './file1', './file2', './sub/file3']) # trying an uncompressed one base_name = os.path.join(tmpdir2, 'archive') - with support.change_cwd(tmpdir): - _make_tarball(splitdrive(base_name)[1], '.', compress=None) + make_archive(splitdrive(base_name)[1], 'tar', root_dir, '.') tarball = base_name + '.tar' - self.assertTrue(os.path.exists(tarball)) + self.assertTrue(os.path.isfile(tarball)) + self.assertTrue(tarfile.is_tarfile(tarball)) + with tarfile.open(tarball, 'r') as tf: + self.assertCountEqual(tf.getnames(), + ['.', './sub', './sub2', + './file1', './file2', './sub/file3']) def _tarinfo(self, path): - tar = tarfile.open(path) - try: + with tarfile.open(path) as tar: names = tar.getnames() names.sort() return tuple(names) - finally: - tar.close() - def _create_files(self): + def _create_files(self, base_dir='dist'): # creating something to tar - tmpdir = self.mkdtemp() - dist = os.path.join(tmpdir, 'dist') - os.mkdir(dist) + root_dir = self.mkdtemp() + dist = os.path.join(root_dir, base_dir) + os.makedirs(dist, exist_ok=True) write_file((dist, 'file1'), 'xxx') write_file((dist, 'file2'), 'xxx') os.mkdir(os.path.join(dist, 'sub')) write_file((dist, 'sub', 'file3'), 'xxx') os.mkdir(os.path.join(dist, 'sub2')) - tmpdir2 = self.mkdtemp() - base_name = os.path.join(tmpdir2, 'archive') - return tmpdir, tmpdir2, base_name + if base_dir: + write_file((root_dir, 'outer'), 'xxx') + return root_dir, base_dir @requires_zlib - @unittest.skipUnless(find_executable('tar') and find_executable('gzip'), + @unittest.skipUnless(find_executable('tar'), 'Need the tar command to run') def test_tarfile_vs_tar(self): - tmpdir, tmpdir2, base_name = self._create_files() - with support.change_cwd(tmpdir): - _make_tarball(base_name, 'dist') + root_dir, base_dir = self._create_files() + base_name = os.path.join(self.mkdtemp(), 'archive') + make_archive(base_name, 'gztar', root_dir, base_dir) # check if the compressed tarball was created tarball = base_name + '.tar.gz' - self.assertTrue(os.path.exists(tarball)) + self.assertTrue(os.path.isfile(tarball)) # now create another tarball using `tar` - tarball2 = os.path.join(tmpdir, 'archive2.tar.gz') - tar_cmd = ['tar', '-cf', 'archive2.tar', 'dist'] - gzip_cmd = ['gzip', '-f9', 'archive2.tar'] - with support.change_cwd(tmpdir): - with captured_stdout() as s: - spawn(tar_cmd) - spawn(gzip_cmd) + tarball2 = os.path.join(root_dir, 'archive2.tar') + tar_cmd = ['tar', '-cf', 'archive2.tar', base_dir] + with support.change_cwd(root_dir), captured_stdout(): + spawn(tar_cmd) - self.assertTrue(os.path.exists(tarball2)) + self.assertTrue(os.path.isfile(tarball2)) # let's compare both tarballs self.assertEqual(self._tarinfo(tarball), self._tarinfo(tarball2)) # trying an uncompressed one - base_name = os.path.join(tmpdir2, 'archive') - with support.change_cwd(tmpdir): - _make_tarball(base_name, 'dist', compress=None) + make_archive(base_name, 'tar', root_dir, base_dir) tarball = base_name + '.tar' - self.assertTrue(os.path.exists(tarball)) + self.assertTrue(os.path.isfile(tarball)) # now for a dry_run - base_name = os.path.join(tmpdir2, 'archive') - with support.change_cwd(tmpdir): - _make_tarball(base_name, 'dist', compress=None, dry_run=True) + make_archive(base_name, 'tar', root_dir, base_dir, dry_run=True) tarball = base_name + '.tar' - self.assertTrue(os.path.exists(tarball)) + self.assertTrue(os.path.isfile(tarball)) @requires_zlib @unittest.skipUnless(ZIP_SUPPORT, 'Need zip support to run') def test_make_zipfile(self): - # creating something to tar - tmpdir = self.mkdtemp() - write_file((tmpdir, 'file1'), 'xxx') - write_file((tmpdir, 'file2'), 'xxx') + # creating something to zip + root_dir, base_dir = self._create_files() + base_name = os.path.join(self.mkdtemp(), 'archive') + res = make_archive(base_name, 'zip', root_dir, 'dist') - tmpdir2 = self.mkdtemp() - # force shutil to create the directory - os.rmdir(tmpdir2) - base_name = os.path.join(tmpdir2, 'archive') - _make_zipfile(base_name, tmpdir) - - # check if the compressed tarball was created - tarball = base_name + '.zip' - self.assertTrue(os.path.exists(tarball)) + self.assertEqual(res, base_name + '.zip') + self.assertTrue(os.path.isfile(res)) + self.assertTrue(zipfile.is_zipfile(res)) + with zipfile.ZipFile(res) as zf: + self.assertCountEqual(zf.namelist(), + ['dist/file1', 'dist/file2', 'dist/sub/file3']) def test_make_archive(self): @@ -1082,36 +1085,37 @@ else: group = owner = 'root' - base_dir, root_dir, base_name = self._create_files() - base_name = os.path.join(self.mkdtemp() , 'archive') + root_dir, base_dir = self._create_files() + base_name = os.path.join(self.mkdtemp(), 'archive') res = make_archive(base_name, 'zip', root_dir, base_dir, owner=owner, group=group) - self.assertTrue(os.path.exists(res)) + self.assertTrue(os.path.isfile(res)) res = make_archive(base_name, 'zip', root_dir, base_dir) - self.assertTrue(os.path.exists(res)) + self.assertTrue(os.path.isfile(res)) res = make_archive(base_name, 'tar', root_dir, base_dir, owner=owner, group=group) - self.assertTrue(os.path.exists(res)) + self.assertTrue(os.path.isfile(res)) res = make_archive(base_name, 'tar', root_dir, base_dir, owner='kjhkjhkjg', group='oihohoh') - self.assertTrue(os.path.exists(res)) + self.assertTrue(os.path.isfile(res)) @requires_zlib @unittest.skipUnless(UID_GID_SUPPORT, "Requires grp and pwd support") def test_tarfile_root_owner(self): - tmpdir, tmpdir2, base_name = self._create_files() + root_dir, base_dir = self._create_files() + base_name = os.path.join(self.mkdtemp(), 'archive') group = grp.getgrgid(0)[0] owner = pwd.getpwuid(0)[0] - with support.change_cwd(tmpdir): - archive_name = _make_tarball(base_name, 'dist', compress=None, - owner=owner, group=group) + with support.change_cwd(root_dir): + archive_name = make_archive(base_name, 'gztar', root_dir, 'dist', + owner=owner, group=group) # check if the compressed tarball was created - self.assertTrue(os.path.exists(archive_name)) + self.assertTrue(os.path.isfile(archive_name)) # now checks the rights archive = tarfile.open(archive_name) @@ -1168,18 +1172,6 @@ formats = [name for name, params in get_archive_formats()] self.assertNotIn('xxx', formats) - def _compare_dirs(self, dir1, dir2): - # check that dir1 and dir2 are equivalent, - # return the diff - diff = [] - for root, dirs, files in os.walk(dir1): - for file_ in files: - path = os.path.join(root, file_) - target_path = os.path.join(dir2, os.path.split(path)[-1]) - if not os.path.exists(target_path): - diff.append(file_) - return diff - @requires_zlib def test_unpack_archive(self): formats = ['tar', 'gztar', 'zip'] @@ -1188,22 +1180,24 @@ if LZMA_SUPPORTED: formats.append('xztar') + root_dir, base_dir = self._create_files() for format in formats: - tmpdir = self.mkdtemp() - base_dir, root_dir, base_name = self._create_files() - tmpdir2 = self.mkdtemp() + expected = rlistdir(root_dir) + expected.remove('outer') + if format == 'zip': + expected.remove('dist/sub2/') + base_name = os.path.join(self.mkdtemp(), 'archive') filename = make_archive(base_name, format, root_dir, base_dir) # let's try to unpack it now + tmpdir2 = self.mkdtemp() unpack_archive(filename, tmpdir2) - diff = self._compare_dirs(tmpdir, tmpdir2) - self.assertEqual(diff, []) + self.assertEqual(rlistdir(tmpdir2), expected) # and again, this time with the format specified tmpdir3 = self.mkdtemp() unpack_archive(filename, tmpdir3, format=format) - diff = self._compare_dirs(tmpdir, tmpdir3) - self.assertEqual(diff, []) + self.assertEqual(rlistdir(tmpdir3), expected) self.assertRaises(shutil.ReadError, unpack_archive, TESTFN) self.assertRaises(ValueError, unpack_archive, TESTFN, format='xxx') -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 6 17:35:33 2015 From: python-checkins at python.org (serhiy.storchaka) Date: Sun, 06 Sep 2015 15:35:33 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogRml4LCByZWZhY3Rv?= =?utf-8?q?r_and_extend_tests_for_shutil=2Emake=5Farchive=28=29=2E?= Message-ID: <20150906153532.68865.48867@psf.io> https://hg.python.org/cpython/rev/9562126443ac changeset: 97706:9562126443ac branch: 3.4 parent: 97701:9774088c8ff7 user: Serhiy Storchaka date: Sun Sep 06 18:33:19 2015 +0300 summary: Fix, refactor and extend tests for shutil.make_archive(). files: Lib/test/test_shutil.py | 170 +++++++++++++-------------- 1 files changed, 82 insertions(+), 88 deletions(-) diff --git a/Lib/test/test_shutil.py b/Lib/test/test_shutil.py --- a/Lib/test/test_shutil.py +++ b/Lib/test/test_shutil.py @@ -14,7 +14,7 @@ from contextlib import ExitStack from os.path import splitdrive from distutils.spawn import find_executable, spawn -from shutil import (_make_tarball, _make_zipfile, make_archive, +from shutil import (make_archive, register_archive_format, unregister_archive_format, get_archive_formats, Error, unpack_archive, register_unpack_format, RegistryError, @@ -86,6 +86,18 @@ with open(path, 'rb' if binary else 'r') as fp: return fp.read() +def rlistdir(path): + res = [] + for name in sorted(os.listdir(path)): + p = os.path.join(path, name) + if os.path.isdir(p) and not os.path.islink(p): + res.append(name + '/') + for n in rlistdir(p): + res.append(name + '/' + n) + else: + res.append(name) + return res + class TestShutil(unittest.TestCase): @@ -951,114 +963,105 @@ @requires_zlib def test_make_tarball(self): # creating something to tar - tmpdir = self.mkdtemp() - write_file((tmpdir, 'file1'), 'xxx') - write_file((tmpdir, 'file2'), 'xxx') - os.mkdir(os.path.join(tmpdir, 'sub')) - write_file((tmpdir, 'sub', 'file3'), 'xxx') + root_dir, base_dir = self._create_files('') tmpdir2 = self.mkdtemp() # force shutil to create the directory os.rmdir(tmpdir2) - unittest.skipUnless(splitdrive(tmpdir)[0] == splitdrive(tmpdir2)[0], + unittest.skipUnless(splitdrive(root_dir)[0] == splitdrive(tmpdir2)[0], "source and target should be on same drive") base_name = os.path.join(tmpdir2, 'archive') # working with relative paths to avoid tar warnings - with support.change_cwd(tmpdir): - _make_tarball(splitdrive(base_name)[1], '.') + make_archive(splitdrive(base_name)[1], 'gztar', root_dir, '.') # check if the compressed tarball was created tarball = base_name + '.tar.gz' - self.assertTrue(os.path.exists(tarball)) + self.assertTrue(os.path.isfile(tarball)) + self.assertTrue(tarfile.is_tarfile(tarball)) + with tarfile.open(tarball, 'r:gz') as tf: + self.assertCountEqual(tf.getnames(), + ['.', './sub', './sub2', + './file1', './file2', './sub/file3']) # trying an uncompressed one base_name = os.path.join(tmpdir2, 'archive') - with support.change_cwd(tmpdir): - _make_tarball(splitdrive(base_name)[1], '.', compress=None) + make_archive(splitdrive(base_name)[1], 'tar', root_dir, '.') tarball = base_name + '.tar' - self.assertTrue(os.path.exists(tarball)) + self.assertTrue(os.path.isfile(tarball)) + self.assertTrue(tarfile.is_tarfile(tarball)) + with tarfile.open(tarball, 'r') as tf: + self.assertCountEqual(tf.getnames(), + ['.', './sub', './sub2', + './file1', './file2', './sub/file3']) def _tarinfo(self, path): - tar = tarfile.open(path) - try: + with tarfile.open(path) as tar: names = tar.getnames() names.sort() return tuple(names) - finally: - tar.close() - def _create_files(self): + def _create_files(self, base_dir='dist'): # creating something to tar - tmpdir = self.mkdtemp() - dist = os.path.join(tmpdir, 'dist') - os.mkdir(dist) + root_dir = self.mkdtemp() + dist = os.path.join(root_dir, base_dir) + os.makedirs(dist, exist_ok=True) write_file((dist, 'file1'), 'xxx') write_file((dist, 'file2'), 'xxx') os.mkdir(os.path.join(dist, 'sub')) write_file((dist, 'sub', 'file3'), 'xxx') os.mkdir(os.path.join(dist, 'sub2')) - tmpdir2 = self.mkdtemp() - base_name = os.path.join(tmpdir2, 'archive') - return tmpdir, tmpdir2, base_name + if base_dir: + write_file((root_dir, 'outer'), 'xxx') + return root_dir, base_dir @requires_zlib - @unittest.skipUnless(find_executable('tar') and find_executable('gzip'), + @unittest.skipUnless(find_executable('tar'), 'Need the tar command to run') def test_tarfile_vs_tar(self): - tmpdir, tmpdir2, base_name = self._create_files() - with support.change_cwd(tmpdir): - _make_tarball(base_name, 'dist') + root_dir, base_dir = self._create_files() + base_name = os.path.join(self.mkdtemp(), 'archive') + make_archive(base_name, 'gztar', root_dir, base_dir) # check if the compressed tarball was created tarball = base_name + '.tar.gz' - self.assertTrue(os.path.exists(tarball)) + self.assertTrue(os.path.isfile(tarball)) # now create another tarball using `tar` - tarball2 = os.path.join(tmpdir, 'archive2.tar.gz') - tar_cmd = ['tar', '-cf', 'archive2.tar', 'dist'] - gzip_cmd = ['gzip', '-f9', 'archive2.tar'] - with support.change_cwd(tmpdir): - with captured_stdout() as s: - spawn(tar_cmd) - spawn(gzip_cmd) + tarball2 = os.path.join(root_dir, 'archive2.tar') + tar_cmd = ['tar', '-cf', 'archive2.tar', base_dir] + with support.change_cwd(root_dir), captured_stdout(): + spawn(tar_cmd) - self.assertTrue(os.path.exists(tarball2)) + self.assertTrue(os.path.isfile(tarball2)) # let's compare both tarballs self.assertEqual(self._tarinfo(tarball), self._tarinfo(tarball2)) # trying an uncompressed one - base_name = os.path.join(tmpdir2, 'archive') - with support.change_cwd(tmpdir): - _make_tarball(base_name, 'dist', compress=None) + make_archive(base_name, 'tar', root_dir, base_dir) tarball = base_name + '.tar' - self.assertTrue(os.path.exists(tarball)) + self.assertTrue(os.path.isfile(tarball)) # now for a dry_run - base_name = os.path.join(tmpdir2, 'archive') - with support.change_cwd(tmpdir): - _make_tarball(base_name, 'dist', compress=None, dry_run=True) + make_archive(base_name, 'tar', root_dir, base_dir, dry_run=True) tarball = base_name + '.tar' - self.assertTrue(os.path.exists(tarball)) + self.assertTrue(os.path.isfile(tarball)) @requires_zlib @unittest.skipUnless(ZIP_SUPPORT, 'Need zip support to run') def test_make_zipfile(self): - # creating something to tar - tmpdir = self.mkdtemp() - write_file((tmpdir, 'file1'), 'xxx') - write_file((tmpdir, 'file2'), 'xxx') + # creating something to zip + root_dir, base_dir = self._create_files() + base_name = os.path.join(self.mkdtemp(), 'archive') + res = make_archive(base_name, 'zip', root_dir, 'dist') - tmpdir2 = self.mkdtemp() - # force shutil to create the directory - os.rmdir(tmpdir2) - base_name = os.path.join(tmpdir2, 'archive') - _make_zipfile(base_name, tmpdir) - - # check if the compressed tarball was created - tarball = base_name + '.zip' - self.assertTrue(os.path.exists(tarball)) + self.assertEqual(res, base_name + '.zip') + self.assertTrue(os.path.isfile(res)) + self.assertTrue(zipfile.is_zipfile(res)) + with zipfile.ZipFile(res) as zf: + self.assertCountEqual(zf.namelist(), + ['dist/file1', 'dist/file2', 'dist/sub/file3']) def test_make_archive(self): @@ -1076,36 +1079,37 @@ else: group = owner = 'root' - base_dir, root_dir, base_name = self._create_files() - base_name = os.path.join(self.mkdtemp() , 'archive') + root_dir, base_dir = self._create_files() + base_name = os.path.join(self.mkdtemp(), 'archive') res = make_archive(base_name, 'zip', root_dir, base_dir, owner=owner, group=group) - self.assertTrue(os.path.exists(res)) + self.assertTrue(os.path.isfile(res)) res = make_archive(base_name, 'zip', root_dir, base_dir) - self.assertTrue(os.path.exists(res)) + self.assertTrue(os.path.isfile(res)) res = make_archive(base_name, 'tar', root_dir, base_dir, owner=owner, group=group) - self.assertTrue(os.path.exists(res)) + self.assertTrue(os.path.isfile(res)) res = make_archive(base_name, 'tar', root_dir, base_dir, owner='kjhkjhkjg', group='oihohoh') - self.assertTrue(os.path.exists(res)) + self.assertTrue(os.path.isfile(res)) @requires_zlib @unittest.skipUnless(UID_GID_SUPPORT, "Requires grp and pwd support") def test_tarfile_root_owner(self): - tmpdir, tmpdir2, base_name = self._create_files() + root_dir, base_dir = self._create_files() + base_name = os.path.join(self.mkdtemp(), 'archive') group = grp.getgrgid(0)[0] owner = pwd.getpwuid(0)[0] - with support.change_cwd(tmpdir): - archive_name = _make_tarball(base_name, 'dist', compress=None, - owner=owner, group=group) + with support.change_cwd(root_dir): + archive_name = make_archive(base_name, 'gztar', root_dir, 'dist', + owner=owner, group=group) # check if the compressed tarball was created - self.assertTrue(os.path.exists(archive_name)) + self.assertTrue(os.path.isfile(archive_name)) # now checks the rights archive = tarfile.open(archive_name) @@ -1162,40 +1166,30 @@ formats = [name for name, params in get_archive_formats()] self.assertNotIn('xxx', formats) - def _compare_dirs(self, dir1, dir2): - # check that dir1 and dir2 are equivalent, - # return the diff - diff = [] - for root, dirs, files in os.walk(dir1): - for file_ in files: - path = os.path.join(root, file_) - target_path = os.path.join(dir2, os.path.split(path)[-1]) - if not os.path.exists(target_path): - diff.append(file_) - return diff - @requires_zlib def test_unpack_archive(self): formats = ['tar', 'gztar', 'zip'] if BZ2_SUPPORTED: formats.append('bztar') + root_dir, base_dir = self._create_files() for format in formats: - tmpdir = self.mkdtemp() - base_dir, root_dir, base_name = self._create_files() - tmpdir2 = self.mkdtemp() + expected = rlistdir(root_dir) + expected.remove('outer') + if format == 'zip': + expected.remove('dist/sub2/') + base_name = os.path.join(self.mkdtemp(), 'archive') filename = make_archive(base_name, format, root_dir, base_dir) # let's try to unpack it now + tmpdir2 = self.mkdtemp() unpack_archive(filename, tmpdir2) - diff = self._compare_dirs(tmpdir, tmpdir2) - self.assertEqual(diff, []) + self.assertEqual(rlistdir(tmpdir2), expected) # and again, this time with the format specified tmpdir3 = self.mkdtemp() unpack_archive(filename, tmpdir3, format=format) - diff = self._compare_dirs(tmpdir, tmpdir3) - self.assertEqual(diff, []) + self.assertEqual(rlistdir(tmpdir3), expected) self.assertRaises(shutil.ReadError, unpack_archive, TESTFN) self.assertRaises(ValueError, unpack_archive, TESTFN, format='xxx') -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 6 17:35:33 2015 From: python-checkins at python.org (serhiy.storchaka) Date: Sun, 06 Sep 2015 15:35:33 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogRml4LCByZWZhY3Rv?= =?utf-8?q?r_and_extend_tests_for_shutil=2Emake=5Farchive=28=29=2E?= Message-ID: <20150906153532.101496.65484@psf.io> https://hg.python.org/cpython/rev/d7885be86e0f changeset: 97705:d7885be86e0f branch: 2.7 user: Serhiy Storchaka date: Sun Sep 06 18:31:23 2015 +0300 summary: Fix, refactor and extend tests for shutil.make_archive(). files: Lib/test/test_shutil.py | 137 +++++++++++++-------------- 1 files changed, 65 insertions(+), 72 deletions(-) diff --git a/Lib/test/test_shutil.py b/Lib/test/test_shutil.py --- a/Lib/test/test_shutil.py +++ b/Lib/test/test_shutil.py @@ -10,7 +10,7 @@ import errno from os.path import splitdrive from distutils.spawn import find_executable, spawn -from shutil import (_make_tarball, _make_zipfile, make_archive, +from shutil import (make_archive, register_archive_format, unregister_archive_format, get_archive_formats) import tarfile @@ -374,114 +374,106 @@ @unittest.skipUnless(zlib, "requires zlib") def test_make_tarball(self): # creating something to tar - tmpdir = self.mkdtemp() - self.write_file([tmpdir, 'file1'], 'xxx') - self.write_file([tmpdir, 'file2'], 'xxx') - os.mkdir(os.path.join(tmpdir, 'sub')) - self.write_file([tmpdir, 'sub', 'file3'], 'xxx') + root_dir, base_dir = self._create_files('') tmpdir2 = self.mkdtemp() # force shutil to create the directory os.rmdir(tmpdir2) - unittest.skipUnless(splitdrive(tmpdir)[0] == splitdrive(tmpdir2)[0], + unittest.skipUnless(splitdrive(root_dir)[0] == splitdrive(tmpdir2)[0], "source and target should be on same drive") base_name = os.path.join(tmpdir2, 'archive') # working with relative paths to avoid tar warnings - with support.change_cwd(tmpdir): - _make_tarball(splitdrive(base_name)[1], '.') + make_archive(splitdrive(base_name)[1], 'gztar', root_dir, '.') # check if the compressed tarball was created tarball = base_name + '.tar.gz' - self.assertTrue(os.path.exists(tarball)) + self.assertTrue(os.path.isfile(tarball)) + self.assertTrue(tarfile.is_tarfile(tarball)) + with tarfile.open(tarball, 'r:gz') as tf: + self.assertEqual(sorted(tf.getnames()), + ['.', './file1', './file2', + './sub', './sub/file3', './sub2']) # trying an uncompressed one base_name = os.path.join(tmpdir2, 'archive') - with support.change_cwd(tmpdir): - _make_tarball(splitdrive(base_name)[1], '.', compress=None) + make_archive(splitdrive(base_name)[1], 'tar', root_dir, '.') tarball = base_name + '.tar' - self.assertTrue(os.path.exists(tarball)) + self.assertTrue(os.path.isfile(tarball)) + self.assertTrue(tarfile.is_tarfile(tarball)) + with tarfile.open(tarball, 'r') as tf: + self.assertEqual(sorted(tf.getnames()), + ['.', './file1', './file2', + './sub', './sub/file3', './sub2']) def _tarinfo(self, path): - tar = tarfile.open(path) - try: + with tarfile.open(path) as tar: names = tar.getnames() names.sort() return tuple(names) - finally: - tar.close() - def _create_files(self): + def _create_files(self, base_dir='dist'): # creating something to tar - tmpdir = self.mkdtemp() - dist = os.path.join(tmpdir, 'dist') - os.mkdir(dist) - self.write_file([dist, 'file1'], 'xxx') - self.write_file([dist, 'file2'], 'xxx') + root_dir = self.mkdtemp() + dist = os.path.join(root_dir, base_dir) + if not os.path.isdir(dist): + os.makedirs(dist) + self.write_file((dist, 'file1'), 'xxx') + self.write_file((dist, 'file2'), 'xxx') os.mkdir(os.path.join(dist, 'sub')) - self.write_file([dist, 'sub', 'file3'], 'xxx') + self.write_file((dist, 'sub', 'file3'), 'xxx') os.mkdir(os.path.join(dist, 'sub2')) - tmpdir2 = self.mkdtemp() - base_name = os.path.join(tmpdir2, 'archive') - return tmpdir, tmpdir2, base_name + if base_dir: + self.write_file((root_dir, 'outer'), 'xxx') + return root_dir, base_dir @unittest.skipUnless(zlib, "Requires zlib") - @unittest.skipUnless(find_executable('tar') and find_executable('gzip'), + @unittest.skipUnless(find_executable('tar'), 'Need the tar command to run') def test_tarfile_vs_tar(self): - tmpdir, tmpdir2, base_name = self._create_files() - with support.change_cwd(tmpdir): - _make_tarball(base_name, 'dist') + root_dir, base_dir = self._create_files() + base_name = os.path.join(self.mkdtemp(), 'archive') + make_archive(base_name, 'gztar', root_dir, base_dir) # check if the compressed tarball was created tarball = base_name + '.tar.gz' - self.assertTrue(os.path.exists(tarball)) + self.assertTrue(os.path.isfile(tarball)) # now create another tarball using `tar` - tarball2 = os.path.join(tmpdir, 'archive2.tar.gz') - tar_cmd = ['tar', '-cf', 'archive2.tar', 'dist'] - gzip_cmd = ['gzip', '-f9', 'archive2.tar'] - with support.change_cwd(tmpdir): - with captured_stdout() as s: - spawn(tar_cmd) - spawn(gzip_cmd) + tarball2 = os.path.join(root_dir, 'archive2.tar') + tar_cmd = ['tar', '-cf', 'archive2.tar', base_dir] + with support.change_cwd(root_dir), captured_stdout(): + spawn(tar_cmd) - self.assertTrue(os.path.exists(tarball2)) + self.assertTrue(os.path.isfile(tarball2)) # let's compare both tarballs self.assertEqual(self._tarinfo(tarball), self._tarinfo(tarball2)) # trying an uncompressed one - base_name = os.path.join(tmpdir2, 'archive') - with support.change_cwd(tmpdir): - _make_tarball(base_name, 'dist', compress=None) + make_archive(base_name, 'tar', root_dir, base_dir) tarball = base_name + '.tar' - self.assertTrue(os.path.exists(tarball)) + self.assertTrue(os.path.isfile(tarball)) # now for a dry_run - base_name = os.path.join(tmpdir2, 'archive') - with support.change_cwd(tmpdir): - _make_tarball(base_name, 'dist', compress=None, dry_run=True) + make_archive(base_name, 'tar', root_dir, base_dir, dry_run=True) tarball = base_name + '.tar' - self.assertTrue(os.path.exists(tarball)) + self.assertTrue(os.path.isfile(tarball)) @unittest.skipUnless(zlib, "Requires zlib") @unittest.skipUnless(ZIP_SUPPORT, 'Need zip support to run') def test_make_zipfile(self): - # creating something to tar - tmpdir = self.mkdtemp() - self.write_file([tmpdir, 'file1'], 'xxx') - self.write_file([tmpdir, 'file2'], 'xxx') + # creating something to zip + root_dir, base_dir = self._create_files() + base_name = os.path.join(self.mkdtemp(), 'archive') + res = make_archive(base_name, 'zip', root_dir, 'dist') - tmpdir2 = self.mkdtemp() - # force shutil to create the directory - os.rmdir(tmpdir2) - base_name = os.path.join(tmpdir2, 'archive') - _make_zipfile(base_name, tmpdir) - - # check if the compressed tarball was created - tarball = base_name + '.zip' - self.assertTrue(os.path.exists(tarball)) + self.assertEqual(res, base_name + '.zip') + self.assertTrue(os.path.isfile(res)) + self.assertTrue(zipfile.is_zipfile(res)) + with zipfile.ZipFile(res) as zf: + self.assertEqual(sorted(zf.namelist()), + ['dist/file1', 'dist/file2', 'dist/sub/file3']) def test_make_archive(self): @@ -499,35 +491,36 @@ else: group = owner = 'root' - base_dir, root_dir, base_name = self._create_files() - base_name = os.path.join(self.mkdtemp() , 'archive') + root_dir, base_dir = self._create_files() + base_name = os.path.join(self.mkdtemp(), 'archive') res = make_archive(base_name, 'zip', root_dir, base_dir, owner=owner, group=group) - self.assertTrue(os.path.exists(res)) + self.assertTrue(os.path.isfile(res)) res = make_archive(base_name, 'zip', root_dir, base_dir) - self.assertTrue(os.path.exists(res)) + self.assertTrue(os.path.isfile(res)) res = make_archive(base_name, 'tar', root_dir, base_dir, owner=owner, group=group) - self.assertTrue(os.path.exists(res)) + self.assertTrue(os.path.isfile(res)) res = make_archive(base_name, 'tar', root_dir, base_dir, owner='kjhkjhkjg', group='oihohoh') - self.assertTrue(os.path.exists(res)) + self.assertTrue(os.path.isfile(res)) @unittest.skipUnless(zlib, "Requires zlib") @unittest.skipUnless(UID_GID_SUPPORT, "Requires grp and pwd support") def test_tarfile_root_owner(self): - tmpdir, tmpdir2, base_name = self._create_files() + root_dir, base_dir = self._create_files() + base_name = os.path.join(self.mkdtemp(), 'archive') group = grp.getgrgid(0)[0] owner = pwd.getpwuid(0)[0] - with support.change_cwd(tmpdir): - archive_name = _make_tarball(base_name, 'dist', compress=None, - owner=owner, group=group) + with support.change_cwd(root_dir): + archive_name = make_archive(base_name, 'gztar', root_dir, 'dist', + owner=owner, group=group) # check if the compressed tarball was created - self.assertTrue(os.path.exists(archive_name)) + self.assertTrue(os.path.isfile(archive_name)) # now checks the rights archive = tarfile.open(archive_name) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 6 17:35:34 2015 From: python-checkins at python.org (serhiy.storchaka) Date: Sun, 06 Sep 2015 15:35:34 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Fix=2C_refactor_and_extend_tests_for_shutil=2Emake=5Farchive?= =?utf-8?b?KCku?= Message-ID: <20150906153532.27695.80660@psf.io> https://hg.python.org/cpython/rev/b1b9f3be8007 changeset: 97707:b1b9f3be8007 branch: 3.5 parent: 97702:a318949ecf59 parent: 97706:9562126443ac user: Serhiy Storchaka date: Sun Sep 06 18:33:52 2015 +0300 summary: Fix, refactor and extend tests for shutil.make_archive(). files: Lib/test/test_shutil.py | 170 +++++++++++++-------------- 1 files changed, 82 insertions(+), 88 deletions(-) diff --git a/Lib/test/test_shutil.py b/Lib/test/test_shutil.py --- a/Lib/test/test_shutil.py +++ b/Lib/test/test_shutil.py @@ -14,7 +14,7 @@ from contextlib import ExitStack from os.path import splitdrive from distutils.spawn import find_executable, spawn -from shutil import (_make_tarball, _make_zipfile, make_archive, +from shutil import (make_archive, register_archive_format, unregister_archive_format, get_archive_formats, Error, unpack_archive, register_unpack_format, RegistryError, @@ -92,6 +92,18 @@ with open(path, 'rb' if binary else 'r') as fp: return fp.read() +def rlistdir(path): + res = [] + for name in sorted(os.listdir(path)): + p = os.path.join(path, name) + if os.path.isdir(p) and not os.path.islink(p): + res.append(name + '/') + for n in rlistdir(p): + res.append(name + '/' + n) + else: + res.append(name) + return res + class TestShutil(unittest.TestCase): @@ -957,114 +969,105 @@ @requires_zlib def test_make_tarball(self): # creating something to tar - tmpdir = self.mkdtemp() - write_file((tmpdir, 'file1'), 'xxx') - write_file((tmpdir, 'file2'), 'xxx') - os.mkdir(os.path.join(tmpdir, 'sub')) - write_file((tmpdir, 'sub', 'file3'), 'xxx') + root_dir, base_dir = self._create_files('') tmpdir2 = self.mkdtemp() # force shutil to create the directory os.rmdir(tmpdir2) - unittest.skipUnless(splitdrive(tmpdir)[0] == splitdrive(tmpdir2)[0], + unittest.skipUnless(splitdrive(root_dir)[0] == splitdrive(tmpdir2)[0], "source and target should be on same drive") base_name = os.path.join(tmpdir2, 'archive') # working with relative paths to avoid tar warnings - with support.change_cwd(tmpdir): - _make_tarball(splitdrive(base_name)[1], '.') + make_archive(splitdrive(base_name)[1], 'gztar', root_dir, '.') # check if the compressed tarball was created tarball = base_name + '.tar.gz' - self.assertTrue(os.path.exists(tarball)) + self.assertTrue(os.path.isfile(tarball)) + self.assertTrue(tarfile.is_tarfile(tarball)) + with tarfile.open(tarball, 'r:gz') as tf: + self.assertCountEqual(tf.getnames(), + ['.', './sub', './sub2', + './file1', './file2', './sub/file3']) # trying an uncompressed one base_name = os.path.join(tmpdir2, 'archive') - with support.change_cwd(tmpdir): - _make_tarball(splitdrive(base_name)[1], '.', compress=None) + make_archive(splitdrive(base_name)[1], 'tar', root_dir, '.') tarball = base_name + '.tar' - self.assertTrue(os.path.exists(tarball)) + self.assertTrue(os.path.isfile(tarball)) + self.assertTrue(tarfile.is_tarfile(tarball)) + with tarfile.open(tarball, 'r') as tf: + self.assertCountEqual(tf.getnames(), + ['.', './sub', './sub2', + './file1', './file2', './sub/file3']) def _tarinfo(self, path): - tar = tarfile.open(path) - try: + with tarfile.open(path) as tar: names = tar.getnames() names.sort() return tuple(names) - finally: - tar.close() - def _create_files(self): + def _create_files(self, base_dir='dist'): # creating something to tar - tmpdir = self.mkdtemp() - dist = os.path.join(tmpdir, 'dist') - os.mkdir(dist) + root_dir = self.mkdtemp() + dist = os.path.join(root_dir, base_dir) + os.makedirs(dist, exist_ok=True) write_file((dist, 'file1'), 'xxx') write_file((dist, 'file2'), 'xxx') os.mkdir(os.path.join(dist, 'sub')) write_file((dist, 'sub', 'file3'), 'xxx') os.mkdir(os.path.join(dist, 'sub2')) - tmpdir2 = self.mkdtemp() - base_name = os.path.join(tmpdir2, 'archive') - return tmpdir, tmpdir2, base_name + if base_dir: + write_file((root_dir, 'outer'), 'xxx') + return root_dir, base_dir @requires_zlib - @unittest.skipUnless(find_executable('tar') and find_executable('gzip'), + @unittest.skipUnless(find_executable('tar'), 'Need the tar command to run') def test_tarfile_vs_tar(self): - tmpdir, tmpdir2, base_name = self._create_files() - with support.change_cwd(tmpdir): - _make_tarball(base_name, 'dist') + root_dir, base_dir = self._create_files() + base_name = os.path.join(self.mkdtemp(), 'archive') + make_archive(base_name, 'gztar', root_dir, base_dir) # check if the compressed tarball was created tarball = base_name + '.tar.gz' - self.assertTrue(os.path.exists(tarball)) + self.assertTrue(os.path.isfile(tarball)) # now create another tarball using `tar` - tarball2 = os.path.join(tmpdir, 'archive2.tar.gz') - tar_cmd = ['tar', '-cf', 'archive2.tar', 'dist'] - gzip_cmd = ['gzip', '-f9', 'archive2.tar'] - with support.change_cwd(tmpdir): - with captured_stdout() as s: - spawn(tar_cmd) - spawn(gzip_cmd) + tarball2 = os.path.join(root_dir, 'archive2.tar') + tar_cmd = ['tar', '-cf', 'archive2.tar', base_dir] + with support.change_cwd(root_dir), captured_stdout(): + spawn(tar_cmd) - self.assertTrue(os.path.exists(tarball2)) + self.assertTrue(os.path.isfile(tarball2)) # let's compare both tarballs self.assertEqual(self._tarinfo(tarball), self._tarinfo(tarball2)) # trying an uncompressed one - base_name = os.path.join(tmpdir2, 'archive') - with support.change_cwd(tmpdir): - _make_tarball(base_name, 'dist', compress=None) + make_archive(base_name, 'tar', root_dir, base_dir) tarball = base_name + '.tar' - self.assertTrue(os.path.exists(tarball)) + self.assertTrue(os.path.isfile(tarball)) # now for a dry_run - base_name = os.path.join(tmpdir2, 'archive') - with support.change_cwd(tmpdir): - _make_tarball(base_name, 'dist', compress=None, dry_run=True) + make_archive(base_name, 'tar', root_dir, base_dir, dry_run=True) tarball = base_name + '.tar' - self.assertTrue(os.path.exists(tarball)) + self.assertTrue(os.path.isfile(tarball)) @requires_zlib @unittest.skipUnless(ZIP_SUPPORT, 'Need zip support to run') def test_make_zipfile(self): - # creating something to tar - tmpdir = self.mkdtemp() - write_file((tmpdir, 'file1'), 'xxx') - write_file((tmpdir, 'file2'), 'xxx') + # creating something to zip + root_dir, base_dir = self._create_files() + base_name = os.path.join(self.mkdtemp(), 'archive') + res = make_archive(base_name, 'zip', root_dir, 'dist') - tmpdir2 = self.mkdtemp() - # force shutil to create the directory - os.rmdir(tmpdir2) - base_name = os.path.join(tmpdir2, 'archive') - _make_zipfile(base_name, tmpdir) - - # check if the compressed tarball was created - tarball = base_name + '.zip' - self.assertTrue(os.path.exists(tarball)) + self.assertEqual(res, base_name + '.zip') + self.assertTrue(os.path.isfile(res)) + self.assertTrue(zipfile.is_zipfile(res)) + with zipfile.ZipFile(res) as zf: + self.assertCountEqual(zf.namelist(), + ['dist/file1', 'dist/file2', 'dist/sub/file3']) def test_make_archive(self): @@ -1082,36 +1085,37 @@ else: group = owner = 'root' - base_dir, root_dir, base_name = self._create_files() - base_name = os.path.join(self.mkdtemp() , 'archive') + root_dir, base_dir = self._create_files() + base_name = os.path.join(self.mkdtemp(), 'archive') res = make_archive(base_name, 'zip', root_dir, base_dir, owner=owner, group=group) - self.assertTrue(os.path.exists(res)) + self.assertTrue(os.path.isfile(res)) res = make_archive(base_name, 'zip', root_dir, base_dir) - self.assertTrue(os.path.exists(res)) + self.assertTrue(os.path.isfile(res)) res = make_archive(base_name, 'tar', root_dir, base_dir, owner=owner, group=group) - self.assertTrue(os.path.exists(res)) + self.assertTrue(os.path.isfile(res)) res = make_archive(base_name, 'tar', root_dir, base_dir, owner='kjhkjhkjg', group='oihohoh') - self.assertTrue(os.path.exists(res)) + self.assertTrue(os.path.isfile(res)) @requires_zlib @unittest.skipUnless(UID_GID_SUPPORT, "Requires grp and pwd support") def test_tarfile_root_owner(self): - tmpdir, tmpdir2, base_name = self._create_files() + root_dir, base_dir = self._create_files() + base_name = os.path.join(self.mkdtemp(), 'archive') group = grp.getgrgid(0)[0] owner = pwd.getpwuid(0)[0] - with support.change_cwd(tmpdir): - archive_name = _make_tarball(base_name, 'dist', compress=None, - owner=owner, group=group) + with support.change_cwd(root_dir): + archive_name = make_archive(base_name, 'gztar', root_dir, 'dist', + owner=owner, group=group) # check if the compressed tarball was created - self.assertTrue(os.path.exists(archive_name)) + self.assertTrue(os.path.isfile(archive_name)) # now checks the rights archive = tarfile.open(archive_name) @@ -1168,18 +1172,6 @@ formats = [name for name, params in get_archive_formats()] self.assertNotIn('xxx', formats) - def _compare_dirs(self, dir1, dir2): - # check that dir1 and dir2 are equivalent, - # return the diff - diff = [] - for root, dirs, files in os.walk(dir1): - for file_ in files: - path = os.path.join(root, file_) - target_path = os.path.join(dir2, os.path.split(path)[-1]) - if not os.path.exists(target_path): - diff.append(file_) - return diff - @requires_zlib def test_unpack_archive(self): formats = ['tar', 'gztar', 'zip'] @@ -1188,22 +1180,24 @@ if LZMA_SUPPORTED: formats.append('xztar') + root_dir, base_dir = self._create_files() for format in formats: - tmpdir = self.mkdtemp() - base_dir, root_dir, base_name = self._create_files() - tmpdir2 = self.mkdtemp() + expected = rlistdir(root_dir) + expected.remove('outer') + if format == 'zip': + expected.remove('dist/sub2/') + base_name = os.path.join(self.mkdtemp(), 'archive') filename = make_archive(base_name, format, root_dir, base_dir) # let's try to unpack it now + tmpdir2 = self.mkdtemp() unpack_archive(filename, tmpdir2) - diff = self._compare_dirs(tmpdir, tmpdir2) - self.assertEqual(diff, []) + self.assertEqual(rlistdir(tmpdir2), expected) # and again, this time with the format specified tmpdir3 = self.mkdtemp() unpack_archive(filename, tmpdir3, format=format) - diff = self._compare_dirs(tmpdir, tmpdir3) - self.assertEqual(diff, []) + self.assertEqual(rlistdir(tmpdir3), expected) self.assertRaises(shutil.ReadError, unpack_archive, TESTFN) self.assertRaises(ValueError, unpack_archive, TESTFN, format='xxx') -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 6 19:07:33 2015 From: python-checkins at python.org (alexander.belopolsky) Date: Sun, 06 Sep 2015 17:07:33 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Closes_Issue=2322241=3A_ti?= =?utf-8?q?mezone=2Eutc_name_is_now_plain_=27UTC=27=2C_not_=27UTC-00=3A00?= =?utf-8?b?Jy4=?= Message-ID: <20150906170733.12015.61025@psf.io> https://hg.python.org/cpython/rev/f904b7eb3981 changeset: 97709:f904b7eb3981 user: Alexander Belopolsky date: Sun Sep 06 13:07:21 2015 -0400 summary: Closes Issue#22241: timezone.utc name is now plain 'UTC', not 'UTC-00:00'. files: Doc/library/datetime.rst | 19 ++++++++++++------- Lib/datetime.py | 2 ++ Lib/test/datetimetester.py | 3 ++- Misc/NEWS | 2 ++ Modules/_datetimemodule.c | 5 +++++ 5 files changed, 23 insertions(+), 8 deletions(-) diff --git a/Doc/library/datetime.rst b/Doc/library/datetime.rst --- a/Doc/library/datetime.rst +++ b/Doc/library/datetime.rst @@ -1734,10 +1734,7 @@ otherwise :exc:`ValueError` is raised. The *name* argument is optional. If specified it must be a string that - is used as the value returned by the ``tzname(dt)`` method. Otherwise, - ``tzname(dt)`` returns a string 'UTCsHH:MM', where s is the sign of - *offset*, HH and MM are two digits of ``offset.hours`` and - ``offset.minutes`` respectively. + will be used as the value returned by the :meth:`datetime.tzname` method. .. versionadded:: 3.2 @@ -1750,11 +1747,19 @@ .. method:: timezone.tzname(dt) - Return the fixed value specified when the :class:`timezone` instance is - constructed or a string 'UTCsHH:MM', where s is the sign of - *offset*, HH and MM are two digits of ``offset.hours`` and + Return the fixed value specified when the :class:`timezone` instance + is constructed. If *name* is not provided in the constructor, the + name returned by ``tzname(dt)`` is generated from the value of the + ``offset`` as follows. If *offset* is ``timedelta(0)``, the name + is "UTC", otherwise it is a string 'UTC?HH:MM', where ? is the sign + of ``offset``, HH and MM are two digits of ``offset.hours`` and ``offset.minutes`` respectively. + .. versionchanged:: 3.6 + Name generated from ``offset=timedelta(0)`` is now plain 'UTC', not + 'UTC+00:00'. + + .. method:: timezone.dst(dt) Always returns ``None``. diff --git a/Lib/datetime.py b/Lib/datetime.py --- a/Lib/datetime.py +++ b/Lib/datetime.py @@ -1920,6 +1920,8 @@ @staticmethod def _name_from_offset(delta): + if not delta: + return 'UTC' if delta < timedelta(0): sign = '-' delta = -delta diff --git a/Lib/test/datetimetester.py b/Lib/test/datetimetester.py --- a/Lib/test/datetimetester.py +++ b/Lib/test/datetimetester.py @@ -258,7 +258,8 @@ with self.assertRaises(TypeError): self.EST.dst(5) def test_tzname(self): - self.assertEqual('UTC+00:00', timezone(ZERO).tzname(None)) + self.assertEqual('UTC', timezone.utc.tzname(None)) + self.assertEqual('UTC', timezone(ZERO).tzname(None)) self.assertEqual('UTC-05:00', timezone(-5 * HOUR).tzname(None)) self.assertEqual('UTC+09:30', timezone(9.5 * HOUR).tzname(None)) self.assertEqual('UTC-00:01', timezone(timedelta(minutes=-1)).tzname(None)) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -17,6 +17,8 @@ Library ------- +- Issue #22241: timezone.utc name is now plain 'UTC', not 'UTC-00:00'. + - Issue #23517: fromtimestamp() and utcfromtimestamp() methods of datetime.datetime now round microseconds to nearest with ties going away from zero (ROUND_HALF_UP), as Python 2 and Python older than 3.3, instead of diff --git a/Modules/_datetimemodule.c b/Modules/_datetimemodule.c --- a/Modules/_datetimemodule.c +++ b/Modules/_datetimemodule.c @@ -3267,6 +3267,11 @@ Py_INCREF(self->name); return self->name; } + if (self == PyDateTime_TimeZone_UTC || + (GET_TD_DAYS(self->offset) == 0 && + GET_TD_SECONDS(self->offset) == 0 && + GET_TD_MICROSECONDS(self->offset) == 0)) + return PyUnicode_FromString("UTC"); /* Offset is normalized, so it is negative if days < 0 */ if (GET_TD_DAYS(self->offset) < 0) { sign = '-'; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 6 20:26:22 2015 From: python-checkins at python.org (serhiy.storchaka) Date: Sun, 06 Sep 2015 18:26:22 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2315989=3A_Fixed_so?= =?utf-8?q?me_scarcely_probable_integer_overflows=2E?= Message-ID: <20150906182621.101482.67546@psf.io> https://hg.python.org/cpython/rev/d51a82f68a70 changeset: 97710:d51a82f68a70 user: Serhiy Storchaka date: Sun Sep 06 21:25:30 2015 +0300 summary: Issue #15989: Fixed some scarcely probable integer overflows. It is very unlikely that they can occur in real code for now. files: Modules/_datetimemodule.c | 2 +- Modules/_io/_iomodule.c | 3 ++- Modules/posixmodule.c | 7 +++++-- Modules/readline.c | 2 +- Objects/structseq.c | 24 ++++++++++++------------ Python/Python-ast.c | 2 +- Python/pythonrun.c | 10 +++++----- 7 files changed, 27 insertions(+), 23 deletions(-) diff --git a/Modules/_datetimemodule.c b/Modules/_datetimemodule.c --- a/Modules/_datetimemodule.c +++ b/Modules/_datetimemodule.c @@ -4692,7 +4692,7 @@ if (seconds == NULL) goto error; Py_DECREF(delta); - timestamp = PyLong_AsLong(seconds); + timestamp = _PyLong_AsTime_t(seconds); Py_DECREF(seconds); if (timestamp == -1 && PyErr_Occurred()) return NULL; diff --git a/Modules/_io/_iomodule.c b/Modules/_io/_iomodule.c --- a/Modules/_io/_iomodule.c +++ b/Modules/_io/_iomodule.c @@ -238,7 +238,8 @@ int text = 0, binary = 0, universal = 0; char rawmode[6], *m; - int line_buffering, isatty; + int line_buffering; + long isatty; PyObject *raw, *modeobj = NULL, *buffer, *wrapper, *result = NULL; diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c --- a/Modules/posixmodule.c +++ b/Modules/posixmodule.c @@ -9481,7 +9481,7 @@ */ struct constdef { char *name; - long value; + int value; }; static int @@ -9489,7 +9489,10 @@ size_t tablesize) { if (PyLong_Check(arg)) { - *valuep = PyLong_AS_LONG(arg); + int value = _PyLong_AsInt(arg); + if (value == -1 && PyErr_Occurred()) + return 0; + *valuep = value; return 1; } else { diff --git a/Modules/readline.c b/Modules/readline.c --- a/Modules/readline.c +++ b/Modules/readline.c @@ -840,7 +840,7 @@ if (r == Py_None) result = 0; else { - result = PyLong_AsLong(r); + result = _PyLong_AsInt(r); if (result == -1 && PyErr_Occurred()) goto error; } diff --git a/Objects/structseq.c b/Objects/structseq.c --- a/Objects/structseq.c +++ b/Objects/structseq.c @@ -16,14 +16,14 @@ _Py_IDENTIFIER(n_unnamed_fields); #define VISIBLE_SIZE(op) Py_SIZE(op) -#define VISIBLE_SIZE_TP(tp) PyLong_AsLong( \ +#define VISIBLE_SIZE_TP(tp) PyLong_AsSsize_t( \ _PyDict_GetItemId((tp)->tp_dict, &PyId_n_sequence_fields)) -#define REAL_SIZE_TP(tp) PyLong_AsLong( \ +#define REAL_SIZE_TP(tp) PyLong_AsSsize_t( \ _PyDict_GetItemId((tp)->tp_dict, &PyId_n_fields)) #define REAL_SIZE(op) REAL_SIZE_TP(Py_TYPE(op)) -#define UNNAMED_FIELDS_TP(tp) PyLong_AsLong( \ +#define UNNAMED_FIELDS_TP(tp) PyLong_AsSsize_t( \ _PyDict_GetItemId((tp)->tp_dict, &PyId_n_unnamed_fields)) #define UNNAMED_FIELDS(op) UNNAMED_FIELDS_TP(Py_TYPE(op)) @@ -164,7 +164,8 @@ #define TYPE_MAXSIZE 100 PyTypeObject *typ = Py_TYPE(obj); - int i, removelast = 0; + Py_ssize_t i; + int removelast = 0; Py_ssize_t len; char buf[REPR_BUFFER_SIZE]; char *endofbuf, *pbuf = buf; @@ -236,8 +237,7 @@ PyObject* tup = NULL; PyObject* dict = NULL; PyObject* result; - Py_ssize_t n_fields, n_visible_fields, n_unnamed_fields; - int i; + Py_ssize_t n_fields, n_visible_fields, n_unnamed_fields, i; n_fields = REAL_SIZE(self); n_visible_fields = VISIBLE_SIZE(self); @@ -325,7 +325,7 @@ { PyObject *dict; PyMemberDef* members; - int n_members, n_unnamed_members, i, k; + Py_ssize_t n_members, n_unnamed_members, i, k; PyObject *v; #ifdef Py_TRACE_REFS @@ -373,9 +373,9 @@ Py_INCREF(type); dict = type->tp_dict; -#define SET_DICT_FROM_INT(key, value) \ +#define SET_DICT_FROM_SIZE(key, value) \ do { \ - v = PyLong_FromLong((long) value); \ + v = PyLong_FromSsize_t(value); \ if (v == NULL) \ return -1; \ if (PyDict_SetItemString(dict, key, v) < 0) { \ @@ -385,9 +385,9 @@ Py_DECREF(v); \ } while (0) - SET_DICT_FROM_INT(visible_length_key, desc->n_in_sequence); - SET_DICT_FROM_INT(real_length_key, n_members); - SET_DICT_FROM_INT(unnamed_fields_key, n_unnamed_members); + SET_DICT_FROM_SIZE(visible_length_key, desc->n_in_sequence); + SET_DICT_FROM_SIZE(real_length_key, n_members); + SET_DICT_FROM_SIZE(unnamed_fields_key, n_unnamed_members); return 0; } diff --git a/Python/Python-ast.c b/Python/Python-ast.c --- a/Python/Python-ast.c +++ b/Python/Python-ast.c @@ -769,7 +769,7 @@ return 1; } - i = (int)PyLong_AsLong(obj); + i = _PyLong_AsInt(obj); if (i == -1 && PyErr_Occurred()) return 1; *out = i; diff --git a/Python/pythonrun.c b/Python/pythonrun.c --- a/Python/pythonrun.c +++ b/Python/pythonrun.c @@ -431,7 +431,7 @@ parse_syntax_error(PyObject *err, PyObject **message, PyObject **filename, int *lineno, int *offset, PyObject **text) { - long hold; + int hold; PyObject *v; _Py_IDENTIFIER(msg); _Py_IDENTIFIER(filename); @@ -464,11 +464,11 @@ v = _PyObject_GetAttrId(err, &PyId_lineno); if (!v) goto finally; - hold = PyLong_AsLong(v); + hold = _PyLong_AsInt(v); Py_DECREF(v); if (hold < 0 && PyErr_Occurred()) goto finally; - *lineno = (int)hold; + *lineno = hold; v = _PyObject_GetAttrId(err, &PyId_offset); if (!v) @@ -477,11 +477,11 @@ *offset = -1; Py_DECREF(v); } else { - hold = PyLong_AsLong(v); + hold = _PyLong_AsInt(v); Py_DECREF(v); if (hold < 0 && PyErr_Occurred()) goto finally; - *offset = (int)hold; + *offset = hold; } v = _PyObject_GetAttrId(err, &PyId_text); -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 6 20:55:33 2015 From: python-checkins at python.org (ezio.melotti) Date: Sun, 06 Sep 2015 18:55:33 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_=2323144=3A_merge_with_3=2E4=2E?= Message-ID: <20150906185533.27699.98216@psf.io> https://hg.python.org/cpython/rev/1f6155ffcaf6 changeset: 97712:1f6155ffcaf6 branch: 3.5 parent: 97707:b1b9f3be8007 parent: 97711:ef82131d0c93 user: Ezio Melotti date: Sun Sep 06 21:44:45 2015 +0300 summary: #23144: merge with 3.4. files: Lib/html/parser.py | 10 +++++++++- Lib/test/test_htmlparser.py | 15 ++++++++++++--- Misc/NEWS | 6 +++++- 3 files changed, 26 insertions(+), 5 deletions(-) diff --git a/Lib/html/parser.py b/Lib/html/parser.py --- a/Lib/html/parser.py +++ b/Lib/html/parser.py @@ -139,7 +139,15 @@ if self.convert_charrefs and not self.cdata_elem: j = rawdata.find('<', i) if j < 0: - if not end: + # if we can't find the next <, either we are at the end + # or there's more text incoming. If the latter is True, + # we can't pass the text to handle_data in case we have + # a charref cut in half at end. Try to determine if + # this is the case before proceding by looking for an + # & near the end and see if it's followed by a space or ;. + amppos = rawdata.rfind('&', max(i, n-34)) + if (amppos >= 0 and + not re.compile(r'[\s;]').search(rawdata, amppos)): break # wait till we get all the text j = n else: diff --git a/Lib/test/test_htmlparser.py b/Lib/test/test_htmlparser.py --- a/Lib/test/test_htmlparser.py +++ b/Lib/test/test_htmlparser.py @@ -72,9 +72,6 @@ class EventCollectorCharrefs(EventCollector): - def get_events(self): - return self.events - def handle_charref(self, data): self.fail('This should never be called with convert_charrefs=True') @@ -633,6 +630,18 @@ ] self._run_check(html, expected) + def test_convert_charrefs_dropped_text(self): + # #23144: make sure that all the events are triggered when + # convert_charrefs is True, even if we don't call .close() + parser = EventCollector(convert_charrefs=True) + # before the fix, bar & baz was missing + parser.feed("foo link bar & baz") + self.assertEqual( + parser.get_events(), + [('data', 'foo '), ('starttag', 'a', []), ('data', 'link'), + ('endtag', 'a'), ('data', ' bar & baz')] + ) + class AttributesTestCase(TestCaseBase): diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -1,4 +1,4 @@ -+++++++++++ +?+++++++++++ Python News +++++++++++ @@ -95,9 +95,13 @@ Library ------- +- Issue #23144: Make sure that HTMLParser.feed() returns all the data, even + when convert_charrefs is True. + - Issue #24635: Fixed a bug in typing.py where isinstance([], typing.Iterable) would return True once, then False on subsequent calls. + - Issue #24989: Fixed buffer overread in BytesIO.readline() if a position is set beyond size. Based on patch by John Leitch. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 6 20:55:33 2015 From: python-checkins at python.org (ezio.melotti) Date: Sun, 06 Sep 2015 18:55:33 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogIzIzMTQ0OiBtZXJnZSB3aXRoIDMuNS4=?= Message-ID: <20150906185533.68859.45132@psf.io> https://hg.python.org/cpython/rev/48ae9d66c720 changeset: 97713:48ae9d66c720 parent: 97710:d51a82f68a70 parent: 97712:1f6155ffcaf6 user: Ezio Melotti date: Sun Sep 06 21:49:48 2015 +0300 summary: #23144: merge with 3.5. files: Lib/html/parser.py | 10 +++++++++- Lib/test/test_htmlparser.py | 15 ++++++++++++--- Misc/NEWS | 6 +++++- 3 files changed, 26 insertions(+), 5 deletions(-) diff --git a/Lib/html/parser.py b/Lib/html/parser.py --- a/Lib/html/parser.py +++ b/Lib/html/parser.py @@ -139,7 +139,15 @@ if self.convert_charrefs and not self.cdata_elem: j = rawdata.find('<', i) if j < 0: - if not end: + # if we can't find the next <, either we are at the end + # or there's more text incoming. If the latter is True, + # we can't pass the text to handle_data in case we have + # a charref cut in half at end. Try to determine if + # this is the case before proceding by looking for an + # & near the end and see if it's followed by a space or ;. + amppos = rawdata.rfind('&', max(i, n-34)) + if (amppos >= 0 and + not re.compile(r'[\s;]').search(rawdata, amppos)): break # wait till we get all the text j = n else: diff --git a/Lib/test/test_htmlparser.py b/Lib/test/test_htmlparser.py --- a/Lib/test/test_htmlparser.py +++ b/Lib/test/test_htmlparser.py @@ -72,9 +72,6 @@ class EventCollectorCharrefs(EventCollector): - def get_events(self): - return self.events - def handle_charref(self, data): self.fail('This should never be called with convert_charrefs=True') @@ -633,6 +630,18 @@ ] self._run_check(html, expected) + def test_convert_charrefs_dropped_text(self): + # #23144: make sure that all the events are triggered when + # convert_charrefs is True, even if we don't call .close() + parser = EventCollector(convert_charrefs=True) + # before the fix, bar & baz was missing + parser.feed("foo link bar & baz") + self.assertEqual( + parser.get_events(), + [('data', 'foo '), ('starttag', 'a', []), ('data', 'link'), + ('endtag', 'a'), ('data', ' bar & baz')] + ) + class AttributesTestCase(TestCaseBase): diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -1,4 +1,4 @@ -+++++++++++ +?+++++++++++ Python News +++++++++++ @@ -181,9 +181,13 @@ Library ------- +- Issue #23144: Make sure that HTMLParser.feed() returns all the data, even + when convert_charrefs is True. + - Issue #24635: Fixed a bug in typing.py where isinstance([], typing.Iterable) would return True once, then False on subsequent calls. + - Issue #24989: Fixed buffer overread in BytesIO.readline() if a position is set beyond size. Based on patch by John Leitch. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 6 20:55:33 2015 From: python-checkins at python.org (ezio.melotti) Date: Sun, 06 Sep 2015 18:55:33 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogIzIzMTQ0OiBNYWtl?= =?utf-8?q?_sure_that_HTMLParser=2Efeed=28=29_returns_all_the_data=2C_even?= =?utf-8?q?_when?= Message-ID: <20150906185533.27685.57496@psf.io> https://hg.python.org/cpython/rev/ef82131d0c93 changeset: 97711:ef82131d0c93 branch: 3.4 parent: 97706:9562126443ac user: Ezio Melotti date: Sun Sep 06 21:38:06 2015 +0300 summary: #23144: Make sure that HTMLParser.feed() returns all the data, even when convert_charrefs is True. files: Lib/html/parser.py | 10 +++++++++- Lib/test/test_htmlparser.py | 15 ++++++++++++--- Misc/NEWS | 5 ++++- 3 files changed, 25 insertions(+), 5 deletions(-) diff --git a/Lib/html/parser.py b/Lib/html/parser.py --- a/Lib/html/parser.py +++ b/Lib/html/parser.py @@ -198,7 +198,15 @@ if self.convert_charrefs and not self.cdata_elem: j = rawdata.find('<', i) if j < 0: - if not end: + # if we can't find the next <, either we are at the end + # or there's more text incoming. If the latter is True, + # we can't pass the text to handle_data in case we have + # a charref cut in half at end. Try to determine if + # this is the case before proceding by looking for an + # & near the end and see if it's followed by a space or ;. + amppos = rawdata.rfind('&', max(i, n-34)) + if (amppos >= 0 and + not re.compile(r'[\s;]').search(rawdata, amppos)): break # wait till we get all the text j = n else: diff --git a/Lib/test/test_htmlparser.py b/Lib/test/test_htmlparser.py --- a/Lib/test/test_htmlparser.py +++ b/Lib/test/test_htmlparser.py @@ -72,9 +72,6 @@ class EventCollectorCharrefs(EventCollector): - def get_events(self): - return self.events - def handle_charref(self, data): self.fail('This should never be called with convert_charrefs=True') @@ -685,6 +682,18 @@ ] self._run_check(html, expected) + def test_convert_charrefs_dropped_text(self): + # #23144: make sure that all the events are triggered when + # convert_charrefs is True, even if we don't call .close() + parser = EventCollector(convert_charrefs=True) + # before the fix, bar & baz was missing + parser.feed("foo link bar & baz") + self.assertEqual( + parser.get_events(), + [('data', 'foo '), ('starttag', 'a', []), ('data', 'link'), + ('endtag', 'a'), ('data', ' bar & baz')] + ) + class AttributesStrictTestCase(TestCaseBase): diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -1,4 +1,4 @@ -+++++++++++ +?+++++++++++ Python News +++++++++++ @@ -81,6 +81,9 @@ Library ------- +- Issue #23144: Make sure that HTMLParser.feed() returns all the data, even + when convert_charrefs is True. + - Issue #16180: Exit pdb if file has syntax error, instead of trapping user in an infinite loop. Patch by Xavier de Gaye. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 6 22:29:47 2015 From: python-checkins at python.org (serhiy.storchaka) Date: Sun, 06 Sep 2015 20:29:47 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Make_asdl=5Fc=2Epy_to_gene?= =?utf-8?q?rate_Python-ast=2Ec_changed_in_issue_=2315989=2E?= Message-ID: <20150906202947.15726.83193@psf.io> https://hg.python.org/cpython/rev/e53df7955192 changeset: 97714:e53df7955192 user: Serhiy Storchaka date: Sun Sep 06 23:29:04 2015 +0300 summary: Make asdl_c.py to generate Python-ast.c changed in issue #15989. files: Parser/asdl_c.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Parser/asdl_c.py b/Parser/asdl_c.py --- a/Parser/asdl_c.py +++ b/Parser/asdl_c.py @@ -898,7 +898,7 @@ return 1; } - i = (int)PyLong_AsLong(obj); + i = _PyLong_AsInt(obj); if (i == -1 && PyErr_Occurred()) return 1; *out = i; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 7 00:15:40 2015 From: python-checkins at python.org (eric.smith) Date: Sun, 06 Sep 2015 22:15:40 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_Grammar_fix=2E?= Message-ID: <20150906221540.101500.86332@psf.io> https://hg.python.org/peps/rev/016de7b73cfc changeset: 6042:016de7b73cfc user: Eric V. Smith date: Sun Sep 06 18:15:38 2015 -0400 summary: Grammar fix. files: pep-0498.txt | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/pep-0498.txt b/pep-0498.txt --- a/pep-0498.txt +++ b/pep-0498.txt @@ -200,7 +200,7 @@ brace. Doubled opening braces do not signify the start of an expression. -Comments, using the ``'#'`` character, are not allowed inside the +Comments, using the ``'#'`` character, are not allowed inside an expression. Following the expression, an optional type conversion may be -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Mon Sep 7 03:20:42 2015 From: python-checkins at python.org (martin.panter) Date: Mon, 07 Sep 2015 01:20:42 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzE3ODQ5?= =?utf-8?q?=3A_Raise_sensible_exception_for_invalid_HTTP_tunnel_response?= Message-ID: <20150907012041.68885.67338@psf.io> https://hg.python.org/cpython/rev/3510e5d435db changeset: 97715:3510e5d435db branch: 2.7 parent: 97705:d7885be86e0f user: Martin Panter date: Mon Sep 07 01:18:47 2015 +0000 summary: Issue #17849: Raise sensible exception for invalid HTTP tunnel response Initial patch from Cory Benfield. files: Lib/httplib.py | 5 +++++ Lib/test/test_httplib.py | 10 ++++++++++ Misc/ACKS | 1 + Misc/NEWS | 4 ++++ 4 files changed, 20 insertions(+), 0 deletions(-) diff --git a/Lib/httplib.py b/Lib/httplib.py --- a/Lib/httplib.py +++ b/Lib/httplib.py @@ -810,6 +810,11 @@ method = self._method) (version, code, message) = response._read_status() + if version == "HTTP/0.9": + # HTTP/0.9 doesn't support the CONNECT verb, so if httplib has + # concluded HTTP/0.9 is being used something has gone wrong. + self.close() + raise socket.error("Invalid response from tunnel request") if code != 200: self.close() raise socket.error("Tunnel connection failed: %d %s" % (code, diff --git a/Lib/test/test_httplib.py b/Lib/test/test_httplib.py --- a/Lib/test/test_httplib.py +++ b/Lib/test/test_httplib.py @@ -578,6 +578,16 @@ #self.assertTrue(response[0].closed) self.assertTrue(conn.sock.file_closed) + def test_proxy_tunnel_without_status_line(self): + # Issue 17849: If a proxy tunnel is created that does not return + # a status code, fail. + body = 'hello world' + conn = httplib.HTTPConnection('example.com', strict=False) + conn.set_tunnel('foo') + conn.sock = FakeSocket(body) + with self.assertRaisesRegexp(socket.error, "Invalid response"): + conn._tunnel() + class OfflineTest(TestCase): def test_responses(self): self.assertEqual(httplib.responses[httplib.NOT_FOUND], "Not Found") diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -107,6 +107,7 @@ Thomas Bellman Alexander ?????? Belopolsky Eli Bendersky +Cory Benfield David Benjamin Oscar Benjamin Andrew Bennetts diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -37,6 +37,10 @@ Library ------- +- Issue #17849: Raise a sensible exception if an invalid response is + received for a HTTP tunnel request, as seen with some servers that + do not support tunnelling. Initial patch from Cory Benfield. + - Issue #16180: Exit pdb if file has syntax error, instead of trapping user in an infinite loop. Patch by Xavier de Gaye. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 7 04:18:10 2015 From: python-checkins at python.org (martin.panter) Date: Mon, 07 Sep 2015 02:18:10 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzIzNDA2?= =?utf-8?q?=3A_Clarify_documentation_on_multiplying_a_sequence?= Message-ID: <20150907021810.14871.95699@psf.io> https://hg.python.org/cpython/rev/6c222848badd changeset: 97716:6c222848badd branch: 2.7 user: Martin Panter date: Mon Sep 07 01:40:33 2015 +0000 summary: Issue #23406: Clarify documentation on multiplying a sequence Patch from Matheus Vieira Portela. files: Doc/faq/programming.rst | 2 ++ Doc/library/stdtypes.rst | 15 +++++++++------ Misc/ACKS | 1 + 3 files changed, 12 insertions(+), 6 deletions(-) diff --git a/Doc/faq/programming.rst b/Doc/faq/programming.rst --- a/Doc/faq/programming.rst +++ b/Doc/faq/programming.rst @@ -1280,6 +1280,8 @@ usually a lot slower than using Python lists. +.. _faq-multidimensional-list: + How do I create a multidimensional list? ---------------------------------------- diff --git a/Doc/library/stdtypes.rst b/Doc/library/stdtypes.rst --- a/Doc/library/stdtypes.rst +++ b/Doc/library/stdtypes.rst @@ -736,8 +736,8 @@ | ``s + t`` | the concatenation of *s* and | \(6) | | | *t* | | +------------------+--------------------------------+----------+ -| ``s * n, n * s`` | *n* shallow copies of *s* | \(2) | -| | concatenated | | +| ``s * n, n * s`` | equivalent to adding *s* to | \(2) | +| | itself *n* times | | +------------------+--------------------------------+----------+ | ``s[i]`` | *i*\ th item of *s*, origin 0 | \(3) | +------------------+--------------------------------+----------+ @@ -789,9 +789,9 @@ (2) Values of *n* less than ``0`` are treated as ``0`` (which yields an empty - sequence of the same type as *s*). Note also that the copies are shallow; - nested structures are not copied. This often haunts new Python programmers; - consider: + sequence of the same type as *s*). Note that items in the sequence *s* + are not copied; they are referenced multiple times. This often haunts + new Python programmers; consider: >>> lists = [[]] * 3 >>> lists @@ -801,7 +801,7 @@ [[3], [3], [3]] What has happened is that ``[[]]`` is a one-element list containing an empty - list, so all three elements of ``[[]] * 3`` are (pointers to) this single empty + list, so all three elements of ``[[]] * 3`` are references to this single empty list. Modifying any of the elements of ``lists`` modifies this single list. You can create a list of different lists this way: @@ -812,6 +812,9 @@ >>> lists [[3], [5], [7]] + Further explanation is available in the FAQ entry + :ref:`faq-multidimensional-list`. + (3) If *i* or *j* is negative, the index is relative to the end of the string: ``len(s) + i`` or ``len(s) + j`` is substituted. But note that ``-0`` is still diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -1084,6 +1084,7 @@ Iustin Pop Claudiu Popa John Popplewell +Matheus Vieira Portela Davin Potts Guillaume Pratte Florian Preinstorfer -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 7 04:18:11 2015 From: python-checkins at python.org (martin.panter) Date: Mon, 07 Sep 2015 02:18:11 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Issue_=2323406=3A_Merge_3=2E4_into_3=2E5?= Message-ID: <20150907021811.12006.76970@psf.io> https://hg.python.org/cpython/rev/f624b7fd3b83 changeset: 97718:f624b7fd3b83 branch: 3.5 parent: 97712:1f6155ffcaf6 parent: 97717:57f8c7ad7782 user: Martin Panter date: Mon Sep 07 02:10:59 2015 +0000 summary: Issue #23406: Merge 3.4 into 3.5 files: Doc/faq/programming.rst | 2 ++ Doc/library/stdtypes.rst | 15 +++++++++------ Misc/ACKS | 1 + 3 files changed, 12 insertions(+), 6 deletions(-) diff --git a/Doc/faq/programming.rst b/Doc/faq/programming.rst --- a/Doc/faq/programming.rst +++ b/Doc/faq/programming.rst @@ -1164,6 +1164,8 @@ usually a lot slower than using Python lists. +.. _faq-multidimensional-list: + How do I create a multidimensional list? ---------------------------------------- diff --git a/Doc/library/stdtypes.rst b/Doc/library/stdtypes.rst --- a/Doc/library/stdtypes.rst +++ b/Doc/library/stdtypes.rst @@ -854,8 +854,8 @@ | ``s + t`` | the concatenation of *s* and | (6)(7) | | | *t* | | +--------------------------+--------------------------------+----------+ -| ``s * n`` or | *n* shallow copies of *s* | (2)(7) | -| ``n * s`` | concatenated | | +| ``s * n`` or | equivalent to adding *s* to | (2)(7) | +| ``n * s`` | itself *n* times | | +--------------------------+--------------------------------+----------+ | ``s[i]`` | *i*\ th item of *s*, origin 0 | \(3) | +--------------------------+--------------------------------+----------+ @@ -897,9 +897,9 @@ (2) Values of *n* less than ``0`` are treated as ``0`` (which yields an empty - sequence of the same type as *s*). Note also that the copies are shallow; - nested structures are not copied. This often haunts new Python programmers; - consider:: + sequence of the same type as *s*). Note that items in the sequence *s* + are not copied; they are referenced multiple times. This often haunts + new Python programmers; consider:: >>> lists = [[]] * 3 >>> lists @@ -909,7 +909,7 @@ [[3], [3], [3]] What has happened is that ``[[]]`` is a one-element list containing an empty - list, so all three elements of ``[[]] * 3`` are (pointers to) this single empty + list, so all three elements of ``[[]] * 3`` are references to this single empty list. Modifying any of the elements of ``lists`` modifies this single list. You can create a list of different lists this way:: @@ -920,6 +920,9 @@ >>> lists [[3], [5], [7]] + Further explanation is available in the FAQ entry + :ref:`faq-multidimensional-list`. + (3) If *i* or *j* is negative, the index is relative to the end of the string: ``len(s) + i`` or ``len(s) + j`` is substituted. But note that ``-0`` is diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -1127,6 +1127,7 @@ Iustin Pop Claudiu Popa John Popplewell +Matheus Vieira Portela Davin Potts Guillaume Pratte Florian Preinstorfer -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 7 04:18:11 2015 From: python-checkins at python.org (martin.panter) Date: Mon, 07 Sep 2015 02:18:11 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2323406=3A_Merge_3=2E5_into_3=2E6?= Message-ID: <20150907021811.17987.1622@psf.io> https://hg.python.org/cpython/rev/d2b3c7c5ef02 changeset: 97719:d2b3c7c5ef02 parent: 97714:e53df7955192 parent: 97718:f624b7fd3b83 user: Martin Panter date: Mon Sep 07 02:12:08 2015 +0000 summary: Issue #23406: Merge 3.5 into 3.6 files: Doc/faq/programming.rst | 2 ++ Doc/library/stdtypes.rst | 15 +++++++++------ Misc/ACKS | 1 + 3 files changed, 12 insertions(+), 6 deletions(-) diff --git a/Doc/faq/programming.rst b/Doc/faq/programming.rst --- a/Doc/faq/programming.rst +++ b/Doc/faq/programming.rst @@ -1164,6 +1164,8 @@ usually a lot slower than using Python lists. +.. _faq-multidimensional-list: + How do I create a multidimensional list? ---------------------------------------- diff --git a/Doc/library/stdtypes.rst b/Doc/library/stdtypes.rst --- a/Doc/library/stdtypes.rst +++ b/Doc/library/stdtypes.rst @@ -854,8 +854,8 @@ | ``s + t`` | the concatenation of *s* and | (6)(7) | | | *t* | | +--------------------------+--------------------------------+----------+ -| ``s * n`` or | *n* shallow copies of *s* | (2)(7) | -| ``n * s`` | concatenated | | +| ``s * n`` or | equivalent to adding *s* to | (2)(7) | +| ``n * s`` | itself *n* times | | +--------------------------+--------------------------------+----------+ | ``s[i]`` | *i*\ th item of *s*, origin 0 | \(3) | +--------------------------+--------------------------------+----------+ @@ -897,9 +897,9 @@ (2) Values of *n* less than ``0`` are treated as ``0`` (which yields an empty - sequence of the same type as *s*). Note also that the copies are shallow; - nested structures are not copied. This often haunts new Python programmers; - consider:: + sequence of the same type as *s*). Note that items in the sequence *s* + are not copied; they are referenced multiple times. This often haunts + new Python programmers; consider:: >>> lists = [[]] * 3 >>> lists @@ -909,7 +909,7 @@ [[3], [3], [3]] What has happened is that ``[[]]`` is a one-element list containing an empty - list, so all three elements of ``[[]] * 3`` are (pointers to) this single empty + list, so all three elements of ``[[]] * 3`` are references to this single empty list. Modifying any of the elements of ``lists`` modifies this single list. You can create a list of different lists this way:: @@ -920,6 +920,9 @@ >>> lists [[3], [5], [7]] + Further explanation is available in the FAQ entry + :ref:`faq-multidimensional-list`. + (3) If *i* or *j* is negative, the index is relative to the end of the string: ``len(s) + i`` or ``len(s) + j`` is substituted. But note that ``-0`` is diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -1127,6 +1127,7 @@ Iustin Pop Claudiu Popa John Popplewell +Matheus Vieira Portela Davin Potts Guillaume Pratte Florian Preinstorfer -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 7 04:18:12 2015 From: python-checkins at python.org (martin.panter) Date: Mon, 07 Sep 2015 02:18:12 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzIzNDA2?= =?utf-8?q?=3A_Clarify_documentation_on_multiplying_a_sequence?= Message-ID: <20150907021811.66856.98955@psf.io> https://hg.python.org/cpython/rev/57f8c7ad7782 changeset: 97717:57f8c7ad7782 branch: 3.4 parent: 97711:ef82131d0c93 user: Martin Panter date: Mon Sep 07 02:08:55 2015 +0000 summary: Issue #23406: Clarify documentation on multiplying a sequence Patch from Matheus Vieira Portela. files: Doc/faq/programming.rst | 2 ++ Doc/library/stdtypes.rst | 15 +++++++++------ Misc/ACKS | 1 + 3 files changed, 12 insertions(+), 6 deletions(-) diff --git a/Doc/faq/programming.rst b/Doc/faq/programming.rst --- a/Doc/faq/programming.rst +++ b/Doc/faq/programming.rst @@ -1164,6 +1164,8 @@ usually a lot slower than using Python lists. +.. _faq-multidimensional-list: + How do I create a multidimensional list? ---------------------------------------- diff --git a/Doc/library/stdtypes.rst b/Doc/library/stdtypes.rst --- a/Doc/library/stdtypes.rst +++ b/Doc/library/stdtypes.rst @@ -854,8 +854,8 @@ | ``s + t`` | the concatenation of *s* and | (6)(7) | | | *t* | | +--------------------------+--------------------------------+----------+ -| ``s * n`` or | *n* shallow copies of *s* | (2)(7) | -| ``n * s`` | concatenated | | +| ``s * n`` or | equivalent to adding *s* to | (2)(7) | +| ``n * s`` | itself *n* times | | +--------------------------+--------------------------------+----------+ | ``s[i]`` | *i*\ th item of *s*, origin 0 | \(3) | +--------------------------+--------------------------------+----------+ @@ -897,9 +897,9 @@ (2) Values of *n* less than ``0`` are treated as ``0`` (which yields an empty - sequence of the same type as *s*). Note also that the copies are shallow; - nested structures are not copied. This often haunts new Python programmers; - consider:: + sequence of the same type as *s*). Note that items in the sequence *s* + are not copied; they are referenced multiple times. This often haunts + new Python programmers; consider:: >>> lists = [[]] * 3 >>> lists @@ -909,7 +909,7 @@ [[3], [3], [3]] What has happened is that ``[[]]`` is a one-element list containing an empty - list, so all three elements of ``[[]] * 3`` are (pointers to) this single empty + list, so all three elements of ``[[]] * 3`` are references to this single empty list. Modifying any of the elements of ``lists`` modifies this single list. You can create a list of different lists this way:: @@ -920,6 +920,9 @@ >>> lists [[3], [5], [7]] + Further explanation is available in the FAQ entry + :ref:`faq-multidimensional-list`. + (3) If *i* or *j* is negative, the index is relative to the end of the string: ``len(s) + i`` or ``len(s) + j`` is substituted. But note that ``-0`` is diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -1097,6 +1097,7 @@ Iustin Pop Claudiu Popa John Popplewell +Matheus Vieira Portela Davin Potts Guillaume Pratte Florian Preinstorfer -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 7 06:17:57 2015 From: python-checkins at python.org (martin.panter) Date: Mon, 07 Sep 2015 04:17:57 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzI1MDA0?= =?utf-8?q?=3A_Handle_out-of-disk-space_error_in_LargeMmapTests?= Message-ID: <20150907041757.17963.19174@psf.io> https://hg.python.org/cpython/rev/7a9dddc11f2e changeset: 97720:7a9dddc11f2e branch: 3.4 parent: 97717:57f8c7ad7782 user: Martin Panter date: Mon Sep 07 02:57:47 2015 +0000 summary: Issue #25004: Handle out-of-disk-space error in LargeMmapTests Patch from John Beck. files: Lib/test/test_mmap.py | 5 ++++- Misc/ACKS | 1 + 2 files changed, 5 insertions(+), 1 deletions(-) diff --git a/Lib/test/test_mmap.py b/Lib/test/test_mmap.py --- a/Lib/test/test_mmap.py +++ b/Lib/test/test_mmap.py @@ -727,7 +727,10 @@ f.write(tail) f.flush() except (OSError, OverflowError): - f.close() + try: + f.close() + except (OSError, OverflowError): + pass raise unittest.SkipTest("filesystem does not have largefile support") return f diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -100,6 +100,7 @@ Samuel L. Bayer Donald Beaudry David Beazley +John Beck Neal Becker Robin Becker Torsten Becker -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 7 06:17:57 2015 From: python-checkins at python.org (martin.panter) Date: Mon, 07 Sep 2015 04:17:57 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzIzNDA2?= =?utf-8?q?=3A_Remove_specific_line_number_from_susp-ignored=2Ecsv?= Message-ID: <20150907041757.11266.17454@psf.io> https://hg.python.org/cpython/rev/2640a4630e9e changeset: 97721:2640a4630e9e branch: 3.5 parent: 97718:f624b7fd3b83 user: Martin Panter date: Mon Sep 07 03:40:17 2015 +0000 summary: Issue #23406: Remove specific line number from susp-ignored.csv files: Doc/tools/susp-ignored.csv | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/tools/susp-ignored.csv b/Doc/tools/susp-ignored.csv --- a/Doc/tools/susp-ignored.csv +++ b/Doc/tools/susp-ignored.csv @@ -288,6 +288,6 @@ library/zipapp,31,:main,"$ python -m zipapp myapp -m ""myapp:main""" library/zipapp,82,:fn,"argument should have the form ""pkg.mod:fn"", where ""pkg.mod"" is a" library/zipapp,155,:callable,"""pkg.module:callable"" and the archive will be run by importing" -library/stdtypes,3767,::,>>> m[::2].tolist() +library/stdtypes,,::,>>> m[::2].tolist() library/sys,1115,`,# ``wrapper`` creates a ``wrap(coro)`` coroutine: tutorial/venv,77,:c7b9645a6f35,"Python 3.4.3+ (3.4:c7b9645a6f35+, May 22 2015, 09:31:25)" -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 7 06:18:02 2015 From: python-checkins at python.org (martin.panter) Date: Mon, 07 Sep 2015 04:18:02 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Issue_=2325004=3A_Merge_3=2E4_into_3=2E5?= Message-ID: <20150907041802.12013.80514@psf.io> https://hg.python.org/cpython/rev/fd4bf05b32ba changeset: 97722:fd4bf05b32ba branch: 3.5 parent: 97721:2640a4630e9e parent: 97720:7a9dddc11f2e user: Martin Panter date: Mon Sep 07 04:04:40 2015 +0000 summary: Issue #25004: Merge 3.4 into 3.5 files: Lib/test/test_mmap.py | 5 ++++- Misc/ACKS | 1 + 2 files changed, 5 insertions(+), 1 deletions(-) diff --git a/Lib/test/test_mmap.py b/Lib/test/test_mmap.py --- a/Lib/test/test_mmap.py +++ b/Lib/test/test_mmap.py @@ -731,7 +731,10 @@ f.write(tail) f.flush() except (OSError, OverflowError): - f.close() + try: + f.close() + except (OSError, OverflowError): + pass raise unittest.SkipTest("filesystem does not have largefile support") return f diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -103,6 +103,7 @@ Samuel L. Bayer Donald Beaudry David Beazley +John Beck Ingolf Becker Neal Becker Robin Becker -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 7 06:18:02 2015 From: python-checkins at python.org (martin.panter) Date: Mon, 07 Sep 2015 04:18:02 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2323406=3A_Merge_3=2E5_into_3=2E6?= Message-ID: <20150907041802.17963.23199@psf.io> https://hg.python.org/cpython/rev/48f1e9a47301 changeset: 97723:48f1e9a47301 parent: 97719:d2b3c7c5ef02 parent: 97721:2640a4630e9e user: Martin Panter date: Mon Sep 07 04:05:57 2015 +0000 summary: Issue #23406: Merge 3.5 into 3.6 files: Doc/tools/susp-ignored.csv | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/tools/susp-ignored.csv b/Doc/tools/susp-ignored.csv --- a/Doc/tools/susp-ignored.csv +++ b/Doc/tools/susp-ignored.csv @@ -288,6 +288,6 @@ library/zipapp,31,:main,"$ python -m zipapp myapp -m ""myapp:main""" library/zipapp,82,:fn,"argument should have the form ""pkg.mod:fn"", where ""pkg.mod"" is a" library/zipapp,155,:callable,"""pkg.module:callable"" and the archive will be run by importing" -library/stdtypes,3767,::,>>> m[::2].tolist() +library/stdtypes,,::,>>> m[::2].tolist() library/sys,1115,`,# ``wrapper`` creates a ``wrap(coro)`` coroutine: tutorial/venv,77,:c7b9645a6f35,"Python 3.4.3+ (3.4:c7b9645a6f35+, May 22 2015, 09:31:25)" -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 7 06:18:02 2015 From: python-checkins at python.org (martin.panter) Date: Mon, 07 Sep 2015 04:18:02 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2325004=3A_Merge_3=2E5_into_3=2E6?= Message-ID: <20150907041802.68881.42341@psf.io> https://hg.python.org/cpython/rev/6c9159661aa8 changeset: 97724:6c9159661aa8 parent: 97723:48f1e9a47301 parent: 97722:fd4bf05b32ba user: Martin Panter date: Mon Sep 07 04:07:06 2015 +0000 summary: Issue #25004: Merge 3.5 into 3.6 files: Lib/test/test_mmap.py | 5 ++++- Misc/ACKS | 1 + 2 files changed, 5 insertions(+), 1 deletions(-) diff --git a/Lib/test/test_mmap.py b/Lib/test/test_mmap.py --- a/Lib/test/test_mmap.py +++ b/Lib/test/test_mmap.py @@ -731,7 +731,10 @@ f.write(tail) f.flush() except (OSError, OverflowError): - f.close() + try: + f.close() + except (OSError, OverflowError): + pass raise unittest.SkipTest("filesystem does not have largefile support") return f diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -103,6 +103,7 @@ Samuel L. Bayer Donald Beaudry David Beazley +John Beck Ingolf Becker Neal Becker Robin Becker -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 7 07:38:01 2015 From: python-checkins at python.org (steve.dower) Date: Mon, 07 Sep 2015 05:38:01 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI0OTE3?= =?utf-8?q?=3A_time=5Fstrftime=28=29_buffer_over-read=2E?= Message-ID: <20150907053801.17965.97710@psf.io> https://hg.python.org/cpython/rev/c31dad22c80d changeset: 97729:c31dad22c80d branch: 3.5 user: Steve Dower date: Sun Sep 06 19:20:51 2015 -0700 summary: Issue #24917: time_strftime() buffer over-read. files: Lib/test/test_time.py | 13 +++++++++++++ Misc/NEWS | 2 ++ Modules/timemodule.c | 16 ++++++++++------ 3 files changed, 25 insertions(+), 6 deletions(-) diff --git a/Lib/test/test_time.py b/Lib/test/test_time.py --- a/Lib/test/test_time.py +++ b/Lib/test/test_time.py @@ -174,6 +174,19 @@ def test_strftime_bounding_check(self): self._bounds_checking(lambda tup: time.strftime('', tup)) + def test_strftime_format_check(self): + # Test that strftime does not crash on invalid format strings + # that may trigger a buffer overread. When not triggered, + # strftime may succeed or raise ValueError depending on + # the platform. + for x in [ '', 'A', '%A', '%AA' ]: + for y in range(0x0, 0x10): + for z in [ '%', 'A%', 'AA%', '%A%', 'A%A%', '%#' ]: + try: + time.strftime(x * y + z) + except ValueError: + pass + def test_default_values_for_zero(self): # Make sure that using all zeros uses the proper default # values. No test for daylight savings since strftime() does diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -20,6 +20,8 @@ Library ------- +- Issue #24917: time_strftime() buffer over-read. + - Issue #24748: To resolve a compatibility problem found with py2exe and pywin32, imp.load_dynamic() once again ignores previously loaded modules to support Python modules replacing themselves with extension modules. diff --git a/Modules/timemodule.c b/Modules/timemodule.c --- a/Modules/timemodule.c +++ b/Modules/timemodule.c @@ -610,14 +610,15 @@ #if defined(MS_WINDOWS) && !defined(HAVE_WCSFTIME) /* check that the format string contains only valid directives */ - for(outbuf = strchr(fmt, '%'); + for (outbuf = strchr(fmt, '%'); outbuf != NULL; outbuf = strchr(outbuf+2, '%')) { - if (outbuf[1]=='#') + if (outbuf[1] == '#') ++outbuf; /* not documented by python, */ - if ((outbuf[1] == 'y') && buf.tm_year < 0) - { + if (outbuf[1] == '\0') + break; + if ((outbuf[1] == 'y') && buf.tm_year < 0) { PyErr_SetString(PyExc_ValueError, "format %y requires year >= 1900 on Windows"); Py_DECREF(format); @@ -625,10 +626,12 @@ } } #elif (defined(_AIX) || defined(sun)) && defined(HAVE_WCSFTIME) - for(outbuf = wcschr(fmt, '%'); + for (outbuf = wcschr(fmt, '%'); outbuf != NULL; outbuf = wcschr(outbuf+2, '%')) { + if (outbuf[1] == L'\0') + break; /* Issue #19634: On AIX, wcsftime("y", (1899, 1, 1, 0, 0, 0, 0, 0, 0)) returns "0/" instead of "99" */ if (outbuf[1] == L'y' && buf.tm_year < 0) { @@ -659,7 +662,8 @@ #if defined _MSC_VER && _MSC_VER >= 1400 && defined(__STDC_SECURE_LIB__) err = errno; #endif - if (buflen > 0 || i >= 256 * fmtlen) { + if (buflen > 0 || fmtlen == 0 || + (fmtlen > 4 && i >= 256 * fmtlen)) { /* If the buffer is 256 times as long as the format, it's probably not failing for lack of room! More likely, the format yields an empty result, -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 7 07:38:01 2015 From: python-checkins at python.org (steve.dower) Date: Mon, 07 Sep 2015 05:38:01 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_Backing_out_09?= =?utf-8?q?b62202d9b7=3B_the_tests_fail_on_Linux=2C_and_it_needs_a_re-thin?= =?utf-8?q?k=2E?= Message-ID: <20150907053801.11268.1953@psf.io> https://hg.python.org/cpython/rev/d27737fe3108 changeset: 97727:d27737fe3108 branch: 3.5 user: Larry Hastings date: Sun Sep 06 00:31:02 2015 -0700 summary: Backing out 09b62202d9b7; the tests fail on Linux, and it needs a re-think. files: Lib/test/test_time.py | 6 ------ Misc/NEWS | 2 -- Modules/timemodule.c | 12 ------------ 3 files changed, 0 insertions(+), 20 deletions(-) diff --git a/Lib/test/test_time.py b/Lib/test/test_time.py --- a/Lib/test/test_time.py +++ b/Lib/test/test_time.py @@ -174,12 +174,6 @@ def test_strftime_bounding_check(self): self._bounds_checking(lambda tup: time.strftime('', tup)) - def test_strftime_format_check(self): - for x in [ '', 'A', '%A', '%AA' ]: - for y in range(0x0, 0x10): - for z in [ '%', 'A%', 'AA%', '%A%', 'A%A%', '%#' ]: - self.assertRaises(ValueError, time.strftime, x * y + z) - def test_default_values_for_zero(self): # Make sure that using all zeros uses the proper default # values. No test for daylight savings since strftime() does diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -22,8 +22,6 @@ to support Python modules replacing themselves with extension modules. Patch by Petr Viktorin. -- Issue #24917: time_strftime() Buffer Over-read. Patch by John Leitch. - - Issue #24635: Fixed a bug in typing.py where isinstance([], typing.Iterable) would return True once, then False on subsequent calls. diff --git a/Modules/timemodule.c b/Modules/timemodule.c --- a/Modules/timemodule.c +++ b/Modules/timemodule.c @@ -623,12 +623,6 @@ Py_DECREF(format); return NULL; } - else if (outbuf[1] == '\0') - { - PyErr_SetString(PyExc_ValueError, "Incomplete format string"); - Py_DECREF(format); - return NULL; - } } #elif (defined(_AIX) || defined(sun)) && defined(HAVE_WCSFTIME) for(outbuf = wcschr(fmt, '%'); @@ -642,12 +636,6 @@ "format %y requires year >= 1900 on AIX"); return NULL; } - else if (outbuf[1] == '\0') - { - PyErr_SetString(PyExc_ValueError, "Incomplete format string"); - Py_DECREF(format); - return NULL; - } } #endif -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 7 07:38:01 2015 From: python-checkins at python.org (steve.dower) Date: Mon, 07 Sep 2015 05:38:01 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy41IC0+IDMuNSk6?= =?utf-8?q?_Merged_in_ncoghlan/cpython350_=28pull_request_=2317=29?= Message-ID: <20150907053801.68867.35340@psf.io> https://hg.python.org/cpython/rev/074c41ddce8f changeset: 97726:074c41ddce8f branch: 3.5 parent: 97693:8b81b7ad2d0a parent: 97725:087464c9f982 user: Larry Hastings date: Sat Sep 05 20:53:04 2015 -0700 summary: Merged in ncoghlan/cpython350 (pull request #17) files: Lib/imp.py | 8 +++++++- Lib/test/imp_dummy.py | 3 +++ Lib/test/test_imp.py | 24 ++++++++++++++++++++++++ Misc/NEWS | 5 +++++ Modules/_testmultiphase.c | 10 ++++++++++ 5 files changed, 49 insertions(+), 1 deletions(-) diff --git a/Lib/imp.py b/Lib/imp.py --- a/Lib/imp.py +++ b/Lib/imp.py @@ -334,6 +334,12 @@ """ import importlib.machinery loader = importlib.machinery.ExtensionFileLoader(name, path) - return loader.load_module() + + # Issue #24748: Skip the sys.modules check in _load_module_shim; + # always load new extension + spec = importlib.machinery.ModuleSpec( + name=name, loader=loader, origin=path) + return _load(spec) + else: load_dynamic = None diff --git a/Lib/test/imp_dummy.py b/Lib/test/imp_dummy.py new file mode 100644 --- /dev/null +++ b/Lib/test/imp_dummy.py @@ -0,0 +1,3 @@ +# Fodder for test of issue24748 in test_imp + +dummy_name = True diff --git a/Lib/test/test_imp.py b/Lib/test/test_imp.py --- a/Lib/test/test_imp.py +++ b/Lib/test/test_imp.py @@ -3,6 +3,7 @@ except ImportError: _thread = None import importlib +import importlib.util import os import os.path import shutil @@ -275,6 +276,29 @@ self.skipTest("found module doesn't appear to be a C extension") imp.load_module(name, None, *found[1:]) + @requires_load_dynamic + def test_issue24748_load_module_skips_sys_modules_check(self): + name = 'test.imp_dummy' + try: + del sys.modules[name] + except KeyError: + pass + try: + module = importlib.import_module(name) + spec = importlib.util.find_spec('_testmultiphase') + module = imp.load_dynamic(name, spec.origin) + self.assertEqual(module.__name__, name) + self.assertEqual(module.__spec__.name, name) + self.assertEqual(module.__spec__.origin, spec.origin) + self.assertRaises(AttributeError, getattr, module, 'dummy_name') + self.assertEqual(module.int_const, 1969) + self.assertIs(sys.modules[name], module) + finally: + try: + del sys.modules[name] + except KeyError: + pass + @unittest.skipIf(sys.dont_write_bytecode, "test meaningful only when writing bytecode") def test_bug7732(self): diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -17,6 +17,11 @@ Library ------- +- Issue #24748: To resolve a compatibility problem found with py2exe and + pywin32, imp.load_dynamic() once again ignores previously loaded modules + to support Python modules replacing themselves with extension modules. + Patch by Petr Viktorin. + - Issue #24917: time_strftime() Buffer Over-read. Patch by John Leitch. - Issue #24635: Fixed a bug in typing.py where isinstance([], typing.Iterable) diff --git a/Modules/_testmultiphase.c b/Modules/_testmultiphase.c --- a/Modules/_testmultiphase.c +++ b/Modules/_testmultiphase.c @@ -582,3 +582,13 @@ { return PyModuleDef_Init(&def_exec_unreported_exception); } + +/*** Helper for imp test ***/ + +static PyModuleDef imp_dummy_def = TEST_MODULE_DEF("imp_dummy", main_slots, testexport_methods); + +PyMODINIT_FUNC +PyInit_imp_dummy(PyObject *spec) +{ + return PyModuleDef_Init(&imp_dummy_def); +} -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 7 07:38:01 2015 From: python-checkins at python.org (steve.dower) Date: Mon, 07 Sep 2015 05:38:01 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI1MDA1?= =?utf-8?q?=3A_Backout_fix_for_=238232_because_of_use_of_unsafe?= Message-ID: <20150907053801.15726.28090@psf.io> https://hg.python.org/cpython/rev/aa60b34d5200 changeset: 97730:aa60b34d5200 branch: 3.5 parent: 97728:94966dfd3bd3 user: Steve Dower date: Sat Sep 05 11:57:47 2015 -0700 summary: Issue #25005: Backout fix for #8232 because of use of unsafe subprocess.call(shell=True) files: Lib/webbrowser.py | 120 ++------------------------------- Misc/NEWS | 3 - 2 files changed, 9 insertions(+), 114 deletions(-) diff --git a/Lib/webbrowser.py b/Lib/webbrowser.py --- a/Lib/webbrowser.py +++ b/Lib/webbrowser.py @@ -495,23 +495,10 @@ # if sys.platform[:3] == "win": - class WindowsDefault(BaseBrowser): - # Windows Default opening arguments. - - cmd = "start" - newwindow = "" - newtab = "" - def open(self, url, new=0, autoraise=True): - # Format the command for optional arguments and add the url. - if new == 1: - self.cmd += " " + self.newwindow - elif new == 2: - self.cmd += " " + self.newtab - self.cmd += " " + url try: - subprocess.call(self.cmd, shell=True) + os.startfile(url) except OSError: # [Error 22] No application is associated with the specified # file for this operation: '' @@ -519,108 +506,19 @@ else: return True - - # Windows Sub-Classes for commonly used browsers. - - class InternetExplorer(WindowsDefault): - """Launcher class for Internet Explorer browser""" - - cmd = "start iexplore.exe" - newwindow = "" - newtab = "" - - - class WinChrome(WindowsDefault): - """Launcher class for windows specific Google Chrome browser""" - - cmd = "start chrome.exe" - newwindow = "-new-window" - newtab = "-new-tab" - - - class WinFirefox(WindowsDefault): - """Launcher class for windows specific Firefox browser""" - - cmd = "start firefox.exe" - newwindow = "-new-window" - newtab = "-new-tab" - - - class WinOpera(WindowsDefault): - """Launcher class for windows specific Opera browser""" - - cmd = "start opera" - newwindow = "" - newtab = "" - - - class WinSeaMonkey(WindowsDefault): - """Launcher class for windows specific SeaMonkey browser""" - - cmd = "start seamonkey" - newwinow = "" - newtab = "" - - _tryorder = [] _browsers = {} - # First try to use the default Windows browser. + # First try to use the default Windows browser register("windows-default", WindowsDefault) - def find_windows_browsers(): - """ Access the windows registry to determine - what browsers are on the system. - """ - - import winreg - HKLM = winreg.HKEY_LOCAL_MACHINE - subkey = r'Software\Clients\StartMenuInternet' - read32 = winreg.KEY_READ | winreg.KEY_WOW64_32KEY - read64 = winreg.KEY_READ | winreg.KEY_WOW64_64KEY - key32 = winreg.OpenKey(HKLM, subkey, access=read32) - key64 = winreg.OpenKey(HKLM, subkey, access=read64) - - # Return a list of browsers found in the registry - # Check if there are any different browsers in the - # 32 bit location instead of the 64 bit location. - browsers = [] - i = 0 - while True: - try: - browsers.append(winreg.EnumKey(key32, i)) - except EnvironmentError: - break - i += 1 - - i = 0 - while True: - try: - browsers.append(winreg.EnumKey(key64, i)) - except EnvironmentError: - break - i += 1 - - winreg.CloseKey(key32) - winreg.CloseKey(key64) - - return browsers - - # Detect some common windows browsers - for browser in find_windows_browsers(): - browser = browser.lower() - if "iexplore" in browser: - register("iexplore", None, InternetExplorer("iexplore")) - elif "chrome" in browser: - register("chrome", None, WinChrome("chrome")) - elif "firefox" in browser: - register("firefox", None, WinFirefox("firefox")) - elif "opera" in browser: - register("opera", None, WinOpera("opera")) - elif "seamonkey" in browser: - register("seamonkey", None, WinSeaMonkey("seamonkey")) - else: - register(browser, None, WindowsDefault(browser)) + # Detect some common Windows browsers, fallback to IE + iexplore = os.path.join(os.environ.get("PROGRAMFILES", "C:\\Program Files"), + "Internet Explorer\\IEXPLORE.EXE") + for browser in ("firefox", "firebird", "seamonkey", "mozilla", + "netscape", "opera", iexplore): + if shutil.which(browser): + register(browser, None, BackgroundBrowser(browser)) # # Platform support for MacOS diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -312,9 +312,6 @@ - Issue #14373: C implementation of functools.lru_cache() now can be used with methods. -- Issue #8232: webbrowser support incomplete on Windows. Patch by Brandon - Milam - - Issue #24347: Set KeyError if PyDict_GetItemWithError returns NULL. - Issue #24348: Drop superfluous incref/decref. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 7 07:38:00 2015 From: python-checkins at python.org (steve.dower) Date: Mon, 07 Sep 2015 05:38:00 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogQ2xvc2UgIzI0NzQ4?= =?utf-8?q?=3A_Restore_imp=2Eload=5Fdynamic_compatibility?= Message-ID: <20150907053800.66872.81165@psf.io> https://hg.python.org/cpython/rev/087464c9f982 changeset: 97725:087464c9f982 branch: 3.5 parent: 97676:438dde69871d user: Nick Coghlan date: Sat Sep 05 21:05:05 2015 +1000 summary: Close #24748: Restore imp.load_dynamic compatibility To resolve a compatibility problem found with py2exe and pywin32, imp.load_dynamic() once again ignores previously loaded modules to support Python modules replacing themselves with extension modules. Patch by Petr Viktorin. files: Lib/imp.py | 8 +++++++- Lib/test/imp_dummy.py | 3 +++ Lib/test/test_imp.py | 24 ++++++++++++++++++++++++ Misc/NEWS | 5 +++++ Modules/_testmultiphase.c | 10 ++++++++++ 5 files changed, 49 insertions(+), 1 deletions(-) diff --git a/Lib/imp.py b/Lib/imp.py --- a/Lib/imp.py +++ b/Lib/imp.py @@ -334,6 +334,12 @@ """ import importlib.machinery loader = importlib.machinery.ExtensionFileLoader(name, path) - return loader.load_module() + + # Issue #24748: Skip the sys.modules check in _load_module_shim; + # always load new extension + spec = importlib.machinery.ModuleSpec( + name=name, loader=loader, origin=path) + return _load(spec) + else: load_dynamic = None diff --git a/Lib/test/imp_dummy.py b/Lib/test/imp_dummy.py new file mode 100644 --- /dev/null +++ b/Lib/test/imp_dummy.py @@ -0,0 +1,3 @@ +# Fodder for test of issue24748 in test_imp + +dummy_name = True diff --git a/Lib/test/test_imp.py b/Lib/test/test_imp.py --- a/Lib/test/test_imp.py +++ b/Lib/test/test_imp.py @@ -3,6 +3,7 @@ except ImportError: _thread = None import importlib +import importlib.util import os import os.path import shutil @@ -275,6 +276,29 @@ self.skipTest("found module doesn't appear to be a C extension") imp.load_module(name, None, *found[1:]) + @requires_load_dynamic + def test_issue24748_load_module_skips_sys_modules_check(self): + name = 'test.imp_dummy' + try: + del sys.modules[name] + except KeyError: + pass + try: + module = importlib.import_module(name) + spec = importlib.util.find_spec('_testmultiphase') + module = imp.load_dynamic(name, spec.origin) + self.assertEqual(module.__name__, name) + self.assertEqual(module.__spec__.name, name) + self.assertEqual(module.__spec__.origin, spec.origin) + self.assertRaises(AttributeError, getattr, module, 'dummy_name') + self.assertEqual(module.int_const, 1969) + self.assertIs(sys.modules[name], module) + finally: + try: + del sys.modules[name] + except KeyError: + pass + @unittest.skipIf(sys.dont_write_bytecode, "test meaningful only when writing bytecode") def test_bug7732(self): diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -15,6 +15,11 @@ Library ------- +- Issue #24748: To resolve a compatibility problem found with py2exe and + pywin32, imp.load_dynamic() once again ignores previously loaded modules + to support Python modules replacing themselves with extension modules. + Patch by Petr Viktorin. + - Issue #24635: Fixed a bug in typing.py where isinstance([], typing.Iterable) would return True once, then False on subsequent calls. diff --git a/Modules/_testmultiphase.c b/Modules/_testmultiphase.c --- a/Modules/_testmultiphase.c +++ b/Modules/_testmultiphase.c @@ -582,3 +582,13 @@ { return PyModuleDef_Init(&def_exec_unreported_exception); } + +/*** Helper for imp test ***/ + +static PyModuleDef imp_dummy_def = TEST_MODULE_DEF("imp_dummy", main_slots, testexport_methods); + +PyMODINIT_FUNC +PyInit_imp_dummy(PyObject *spec) +{ + return PyModuleDef_Init(&imp_dummy_def); +} -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 7 07:38:01 2015 From: python-checkins at python.org (steve.dower) Date: Mon, 07 Sep 2015 05:38:01 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI0MzA1?= =?utf-8?q?=3A_Prevent_import_subsystem_stack_frames_from_being_counted?= Message-ID: <20150907053801.14871.43936@psf.io> https://hg.python.org/cpython/rev/94966dfd3bd3 changeset: 97728:94966dfd3bd3 branch: 3.5 user: Larry Hastings date: Sun Sep 06 00:39:37 2015 -0700 summary: Issue #24305: Prevent import subsystem stack frames from being counted by the warnings.warn(stacklevel=) parameter. files: Lib/test/test_warnings.py | 944 --------- Lib/test/test_warnings/__init__.py | 955 ++++++++++ Lib/test/test_warnings/__main__.py | 3 + Lib/test/test_warnings/data/import_warning.py | 3 + Lib/test/test_warnings/data/stacklevel.py | 0 Lib/warnings.py | 31 +- Misc/NEWS | 3 + Python/_warnings.c | 72 +- 8 files changed, 1061 insertions(+), 950 deletions(-) diff --git a/Lib/test/test_warnings.py b/Lib/test/test_warnings.py deleted file mode 100644 --- a/Lib/test/test_warnings.py +++ /dev/null @@ -1,944 +0,0 @@ -from contextlib import contextmanager -import linecache -import os -from io import StringIO -import sys -import unittest -from test import support -from test.support.script_helper import assert_python_ok, assert_python_failure - -from test import warning_tests - -import warnings as original_warnings - -py_warnings = support.import_fresh_module('warnings', blocked=['_warnings']) -c_warnings = support.import_fresh_module('warnings', fresh=['_warnings']) - - at contextmanager -def warnings_state(module): - """Use a specific warnings implementation in warning_tests.""" - global __warningregistry__ - for to_clear in (sys, warning_tests): - try: - to_clear.__warningregistry__.clear() - except AttributeError: - pass - try: - __warningregistry__.clear() - except NameError: - pass - original_warnings = warning_tests.warnings - original_filters = module.filters - try: - module.filters = original_filters[:] - module.simplefilter("once") - warning_tests.warnings = module - yield - finally: - warning_tests.warnings = original_warnings - module.filters = original_filters - - -class BaseTest: - - """Basic bookkeeping required for testing.""" - - def setUp(self): - # The __warningregistry__ needs to be in a pristine state for tests - # to work properly. - if '__warningregistry__' in globals(): - del globals()['__warningregistry__'] - if hasattr(warning_tests, '__warningregistry__'): - del warning_tests.__warningregistry__ - if hasattr(sys, '__warningregistry__'): - del sys.__warningregistry__ - # The 'warnings' module must be explicitly set so that the proper - # interaction between _warnings and 'warnings' can be controlled. - sys.modules['warnings'] = self.module - super(BaseTest, self).setUp() - - def tearDown(self): - sys.modules['warnings'] = original_warnings - super(BaseTest, self).tearDown() - -class PublicAPITests(BaseTest): - - """Ensures that the correct values are exposed in the - public API. - """ - - def test_module_all_attribute(self): - self.assertTrue(hasattr(self.module, '__all__')) - target_api = ["warn", "warn_explicit", "showwarning", - "formatwarning", "filterwarnings", "simplefilter", - "resetwarnings", "catch_warnings"] - self.assertSetEqual(set(self.module.__all__), - set(target_api)) - -class CPublicAPITests(PublicAPITests, unittest.TestCase): - module = c_warnings - -class PyPublicAPITests(PublicAPITests, unittest.TestCase): - module = py_warnings - -class FilterTests(BaseTest): - - """Testing the filtering functionality.""" - - def test_error(self): - with original_warnings.catch_warnings(module=self.module) as w: - self.module.resetwarnings() - self.module.filterwarnings("error", category=UserWarning) - self.assertRaises(UserWarning, self.module.warn, - "FilterTests.test_error") - - def test_error_after_default(self): - with original_warnings.catch_warnings(module=self.module) as w: - self.module.resetwarnings() - message = "FilterTests.test_ignore_after_default" - def f(): - self.module.warn(message, UserWarning) - f() - self.module.filterwarnings("error", category=UserWarning) - self.assertRaises(UserWarning, f) - - def test_ignore(self): - with original_warnings.catch_warnings(record=True, - module=self.module) as w: - self.module.resetwarnings() - self.module.filterwarnings("ignore", category=UserWarning) - self.module.warn("FilterTests.test_ignore", UserWarning) - self.assertEqual(len(w), 0) - - def test_ignore_after_default(self): - with original_warnings.catch_warnings(record=True, - module=self.module) as w: - self.module.resetwarnings() - message = "FilterTests.test_ignore_after_default" - def f(): - self.module.warn(message, UserWarning) - f() - self.module.filterwarnings("ignore", category=UserWarning) - f() - f() - self.assertEqual(len(w), 1) - - def test_always(self): - with original_warnings.catch_warnings(record=True, - module=self.module) as w: - self.module.resetwarnings() - self.module.filterwarnings("always", category=UserWarning) - message = "FilterTests.test_always" - self.module.warn(message, UserWarning) - self.assertTrue(message, w[-1].message) - self.module.warn(message, UserWarning) - self.assertTrue(w[-1].message, message) - - def test_always_after_default(self): - with original_warnings.catch_warnings(record=True, - module=self.module) as w: - self.module.resetwarnings() - message = "FilterTests.test_always_after_ignore" - def f(): - self.module.warn(message, UserWarning) - f() - self.assertEqual(len(w), 1) - self.assertEqual(w[-1].message.args[0], message) - f() - self.assertEqual(len(w), 1) - self.module.filterwarnings("always", category=UserWarning) - f() - self.assertEqual(len(w), 2) - self.assertEqual(w[-1].message.args[0], message) - f() - self.assertEqual(len(w), 3) - self.assertEqual(w[-1].message.args[0], message) - - def test_default(self): - with original_warnings.catch_warnings(record=True, - module=self.module) as w: - self.module.resetwarnings() - self.module.filterwarnings("default", category=UserWarning) - message = UserWarning("FilterTests.test_default") - for x in range(2): - self.module.warn(message, UserWarning) - if x == 0: - self.assertEqual(w[-1].message, message) - del w[:] - elif x == 1: - self.assertEqual(len(w), 0) - else: - raise ValueError("loop variant unhandled") - - def test_module(self): - with original_warnings.catch_warnings(record=True, - module=self.module) as w: - self.module.resetwarnings() - self.module.filterwarnings("module", category=UserWarning) - message = UserWarning("FilterTests.test_module") - self.module.warn(message, UserWarning) - self.assertEqual(w[-1].message, message) - del w[:] - self.module.warn(message, UserWarning) - self.assertEqual(len(w), 0) - - def test_once(self): - with original_warnings.catch_warnings(record=True, - module=self.module) as w: - self.module.resetwarnings() - self.module.filterwarnings("once", category=UserWarning) - message = UserWarning("FilterTests.test_once") - self.module.warn_explicit(message, UserWarning, "test_warnings.py", - 42) - self.assertEqual(w[-1].message, message) - del w[:] - self.module.warn_explicit(message, UserWarning, "test_warnings.py", - 13) - self.assertEqual(len(w), 0) - self.module.warn_explicit(message, UserWarning, "test_warnings2.py", - 42) - self.assertEqual(len(w), 0) - - def test_inheritance(self): - with original_warnings.catch_warnings(module=self.module) as w: - self.module.resetwarnings() - self.module.filterwarnings("error", category=Warning) - self.assertRaises(UserWarning, self.module.warn, - "FilterTests.test_inheritance", UserWarning) - - def test_ordering(self): - with original_warnings.catch_warnings(record=True, - module=self.module) as w: - self.module.resetwarnings() - self.module.filterwarnings("ignore", category=UserWarning) - self.module.filterwarnings("error", category=UserWarning, - append=True) - del w[:] - try: - self.module.warn("FilterTests.test_ordering", UserWarning) - except UserWarning: - self.fail("order handling for actions failed") - self.assertEqual(len(w), 0) - - def test_filterwarnings(self): - # Test filterwarnings(). - # Implicitly also tests resetwarnings(). - with original_warnings.catch_warnings(record=True, - module=self.module) as w: - self.module.filterwarnings("error", "", Warning, "", 0) - self.assertRaises(UserWarning, self.module.warn, 'convert to error') - - self.module.resetwarnings() - text = 'handle normally' - self.module.warn(text) - self.assertEqual(str(w[-1].message), text) - self.assertTrue(w[-1].category is UserWarning) - - self.module.filterwarnings("ignore", "", Warning, "", 0) - text = 'filtered out' - self.module.warn(text) - self.assertNotEqual(str(w[-1].message), text) - - self.module.resetwarnings() - self.module.filterwarnings("error", "hex*", Warning, "", 0) - self.assertRaises(UserWarning, self.module.warn, 'hex/oct') - text = 'nonmatching text' - self.module.warn(text) - self.assertEqual(str(w[-1].message), text) - self.assertTrue(w[-1].category is UserWarning) - - def test_mutate_filter_list(self): - class X: - def match(self, a): - L[:] = [] - - L = [("default",X(),UserWarning,X(),0) for i in range(2)] - with original_warnings.catch_warnings(record=True, - module=self.module) as w: - self.module.filters = L - self.module.warn_explicit(UserWarning("b"), None, "f.py", 42) - self.assertEqual(str(w[-1].message), "b") - -class CFilterTests(FilterTests, unittest.TestCase): - module = c_warnings - -class PyFilterTests(FilterTests, unittest.TestCase): - module = py_warnings - - -class WarnTests(BaseTest): - - """Test warnings.warn() and warnings.warn_explicit().""" - - def test_message(self): - with original_warnings.catch_warnings(record=True, - module=self.module) as w: - self.module.simplefilter("once") - for i in range(4): - text = 'multi %d' %i # Different text on each call. - self.module.warn(text) - self.assertEqual(str(w[-1].message), text) - self.assertTrue(w[-1].category is UserWarning) - - # Issue 3639 - def test_warn_nonstandard_types(self): - # warn() should handle non-standard types without issue. - for ob in (Warning, None, 42): - with original_warnings.catch_warnings(record=True, - module=self.module) as w: - self.module.simplefilter("once") - self.module.warn(ob) - # Don't directly compare objects since - # ``Warning() != Warning()``. - self.assertEqual(str(w[-1].message), str(UserWarning(ob))) - - def test_filename(self): - with warnings_state(self.module): - with original_warnings.catch_warnings(record=True, - module=self.module) as w: - warning_tests.inner("spam1") - self.assertEqual(os.path.basename(w[-1].filename), - "warning_tests.py") - warning_tests.outer("spam2") - self.assertEqual(os.path.basename(w[-1].filename), - "warning_tests.py") - - def test_stacklevel(self): - # Test stacklevel argument - # make sure all messages are different, so the warning won't be skipped - with warnings_state(self.module): - with original_warnings.catch_warnings(record=True, - module=self.module) as w: - warning_tests.inner("spam3", stacklevel=1) - self.assertEqual(os.path.basename(w[-1].filename), - "warning_tests.py") - warning_tests.outer("spam4", stacklevel=1) - self.assertEqual(os.path.basename(w[-1].filename), - "warning_tests.py") - - warning_tests.inner("spam5", stacklevel=2) - self.assertEqual(os.path.basename(w[-1].filename), - "test_warnings.py") - warning_tests.outer("spam6", stacklevel=2) - self.assertEqual(os.path.basename(w[-1].filename), - "warning_tests.py") - warning_tests.outer("spam6.5", stacklevel=3) - self.assertEqual(os.path.basename(w[-1].filename), - "test_warnings.py") - - warning_tests.inner("spam7", stacklevel=9999) - self.assertEqual(os.path.basename(w[-1].filename), - "sys") - - def test_missing_filename_not_main(self): - # If __file__ is not specified and __main__ is not the module name, - # then __file__ should be set to the module name. - filename = warning_tests.__file__ - try: - del warning_tests.__file__ - with warnings_state(self.module): - with original_warnings.catch_warnings(record=True, - module=self.module) as w: - warning_tests.inner("spam8", stacklevel=1) - self.assertEqual(w[-1].filename, warning_tests.__name__) - finally: - warning_tests.__file__ = filename - - @unittest.skipUnless(hasattr(sys, 'argv'), 'test needs sys.argv') - def test_missing_filename_main_with_argv(self): - # If __file__ is not specified and the caller is __main__ and sys.argv - # exists, then use sys.argv[0] as the file. - filename = warning_tests.__file__ - module_name = warning_tests.__name__ - try: - del warning_tests.__file__ - warning_tests.__name__ = '__main__' - with warnings_state(self.module): - with original_warnings.catch_warnings(record=True, - module=self.module) as w: - warning_tests.inner('spam9', stacklevel=1) - self.assertEqual(w[-1].filename, sys.argv[0]) - finally: - warning_tests.__file__ = filename - warning_tests.__name__ = module_name - - def test_missing_filename_main_without_argv(self): - # If __file__ is not specified, the caller is __main__, and sys.argv - # is not set, then '__main__' is the file name. - filename = warning_tests.__file__ - module_name = warning_tests.__name__ - argv = sys.argv - try: - del warning_tests.__file__ - warning_tests.__name__ = '__main__' - del sys.argv - with warnings_state(self.module): - with original_warnings.catch_warnings(record=True, - module=self.module) as w: - warning_tests.inner('spam10', stacklevel=1) - self.assertEqual(w[-1].filename, '__main__') - finally: - warning_tests.__file__ = filename - warning_tests.__name__ = module_name - sys.argv = argv - - def test_missing_filename_main_with_argv_empty_string(self): - # If __file__ is not specified, the caller is __main__, and sys.argv[0] - # is the empty string, then '__main__ is the file name. - # Tests issue 2743. - file_name = warning_tests.__file__ - module_name = warning_tests.__name__ - argv = sys.argv - try: - del warning_tests.__file__ - warning_tests.__name__ = '__main__' - sys.argv = [''] - with warnings_state(self.module): - with original_warnings.catch_warnings(record=True, - module=self.module) as w: - warning_tests.inner('spam11', stacklevel=1) - self.assertEqual(w[-1].filename, '__main__') - finally: - warning_tests.__file__ = file_name - warning_tests.__name__ = module_name - sys.argv = argv - - def test_warn_explicit_non_ascii_filename(self): - with original_warnings.catch_warnings(record=True, - module=self.module) as w: - self.module.resetwarnings() - self.module.filterwarnings("always", category=UserWarning) - for filename in ("nonascii\xe9\u20ac", "surrogate\udc80"): - try: - os.fsencode(filename) - except UnicodeEncodeError: - continue - self.module.warn_explicit("text", UserWarning, filename, 1) - self.assertEqual(w[-1].filename, filename) - - def test_warn_explicit_type_errors(self): - # warn_explicit() should error out gracefully if it is given objects - # of the wrong types. - # lineno is expected to be an integer. - self.assertRaises(TypeError, self.module.warn_explicit, - None, UserWarning, None, None) - # Either 'message' needs to be an instance of Warning or 'category' - # needs to be a subclass. - self.assertRaises(TypeError, self.module.warn_explicit, - None, None, None, 1) - # 'registry' must be a dict or None. - self.assertRaises((TypeError, AttributeError), - self.module.warn_explicit, - None, Warning, None, 1, registry=42) - - def test_bad_str(self): - # issue 6415 - # Warnings instance with a bad format string for __str__ should not - # trigger a bus error. - class BadStrWarning(Warning): - """Warning with a bad format string for __str__.""" - def __str__(self): - return ("A bad formatted string %(err)" % - {"err" : "there is no %(err)s"}) - - with self.assertRaises(ValueError): - self.module.warn(BadStrWarning()) - - def test_warning_classes(self): - class MyWarningClass(Warning): - pass - - class NonWarningSubclass: - pass - - # passing a non-subclass of Warning should raise a TypeError - with self.assertRaises(TypeError) as cm: - self.module.warn('bad warning category', '') - self.assertIn('category must be a Warning subclass, not ', - str(cm.exception)) - - with self.assertRaises(TypeError) as cm: - self.module.warn('bad warning category', NonWarningSubclass) - self.assertIn('category must be a Warning subclass, not ', - str(cm.exception)) - - # check that warning instances also raise a TypeError - with self.assertRaises(TypeError) as cm: - self.module.warn('bad warning category', MyWarningClass()) - self.assertIn('category must be a Warning subclass, not ', - str(cm.exception)) - - with original_warnings.catch_warnings(module=self.module): - self.module.resetwarnings() - self.module.filterwarnings('default') - with self.assertWarns(MyWarningClass) as cm: - self.module.warn('good warning category', MyWarningClass) - self.assertEqual('good warning category', str(cm.warning)) - - with self.assertWarns(UserWarning) as cm: - self.module.warn('good warning category', None) - self.assertEqual('good warning category', str(cm.warning)) - - with self.assertWarns(MyWarningClass) as cm: - self.module.warn('good warning category', MyWarningClass) - self.assertIsInstance(cm.warning, Warning) - -class CWarnTests(WarnTests, unittest.TestCase): - module = c_warnings - - # As an early adopter, we sanity check the - # test.support.import_fresh_module utility function - def test_accelerated(self): - self.assertFalse(original_warnings is self.module) - self.assertFalse(hasattr(self.module.warn, '__code__')) - -class PyWarnTests(WarnTests, unittest.TestCase): - module = py_warnings - - # As an early adopter, we sanity check the - # test.support.import_fresh_module utility function - def test_pure_python(self): - self.assertFalse(original_warnings is self.module) - self.assertTrue(hasattr(self.module.warn, '__code__')) - - -class WCmdLineTests(BaseTest): - - def test_improper_input(self): - # Uses the private _setoption() function to test the parsing - # of command-line warning arguments - with original_warnings.catch_warnings(module=self.module): - self.assertRaises(self.module._OptionError, - self.module._setoption, '1:2:3:4:5:6') - self.assertRaises(self.module._OptionError, - self.module._setoption, 'bogus::Warning') - self.assertRaises(self.module._OptionError, - self.module._setoption, 'ignore:2::4:-5') - self.module._setoption('error::Warning::0') - self.assertRaises(UserWarning, self.module.warn, 'convert to error') - - def test_improper_option(self): - # Same as above, but check that the message is printed out when - # the interpreter is executed. This also checks that options are - # actually parsed at all. - rc, out, err = assert_python_ok("-Wxxx", "-c", "pass") - self.assertIn(b"Invalid -W option ignored: invalid action: 'xxx'", err) - - def test_warnings_bootstrap(self): - # Check that the warnings module does get loaded when -W - # is used (see issue #10372 for an example of silent bootstrap failure). - rc, out, err = assert_python_ok("-Wi", "-c", - "import sys; sys.modules['warnings'].warn('foo', RuntimeWarning)") - # '-Wi' was observed - self.assertFalse(out.strip()) - self.assertNotIn(b'RuntimeWarning', err) - -class CWCmdLineTests(WCmdLineTests, unittest.TestCase): - module = c_warnings - -class PyWCmdLineTests(WCmdLineTests, unittest.TestCase): - module = py_warnings - - -class _WarningsTests(BaseTest, unittest.TestCase): - - """Tests specific to the _warnings module.""" - - module = c_warnings - - def test_filter(self): - # Everything should function even if 'filters' is not in warnings. - with original_warnings.catch_warnings(module=self.module) as w: - self.module.filterwarnings("error", "", Warning, "", 0) - self.assertRaises(UserWarning, self.module.warn, - 'convert to error') - del self.module.filters - self.assertRaises(UserWarning, self.module.warn, - 'convert to error') - - def test_onceregistry(self): - # Replacing or removing the onceregistry should be okay. - global __warningregistry__ - message = UserWarning('onceregistry test') - try: - original_registry = self.module.onceregistry - __warningregistry__ = {} - with original_warnings.catch_warnings(record=True, - module=self.module) as w: - self.module.resetwarnings() - self.module.filterwarnings("once", category=UserWarning) - self.module.warn_explicit(message, UserWarning, "file", 42) - self.assertEqual(w[-1].message, message) - del w[:] - self.module.warn_explicit(message, UserWarning, "file", 42) - self.assertEqual(len(w), 0) - # Test the resetting of onceregistry. - self.module.onceregistry = {} - __warningregistry__ = {} - self.module.warn('onceregistry test') - self.assertEqual(w[-1].message.args, message.args) - # Removal of onceregistry is okay. - del w[:] - del self.module.onceregistry - __warningregistry__ = {} - self.module.warn_explicit(message, UserWarning, "file", 42) - self.assertEqual(len(w), 0) - finally: - self.module.onceregistry = original_registry - - def test_default_action(self): - # Replacing or removing defaultaction should be okay. - message = UserWarning("defaultaction test") - original = self.module.defaultaction - try: - with original_warnings.catch_warnings(record=True, - module=self.module) as w: - self.module.resetwarnings() - registry = {} - self.module.warn_explicit(message, UserWarning, "", 42, - registry=registry) - self.assertEqual(w[-1].message, message) - self.assertEqual(len(w), 1) - # One actual registry key plus the "version" key - self.assertEqual(len(registry), 2) - self.assertIn("version", registry) - del w[:] - # Test removal. - del self.module.defaultaction - __warningregistry__ = {} - registry = {} - self.module.warn_explicit(message, UserWarning, "", 43, - registry=registry) - self.assertEqual(w[-1].message, message) - self.assertEqual(len(w), 1) - self.assertEqual(len(registry), 2) - del w[:] - # Test setting. - self.module.defaultaction = "ignore" - __warningregistry__ = {} - registry = {} - self.module.warn_explicit(message, UserWarning, "", 44, - registry=registry) - self.assertEqual(len(w), 0) - finally: - self.module.defaultaction = original - - def test_showwarning_missing(self): - # Test that showwarning() missing is okay. - text = 'del showwarning test' - with original_warnings.catch_warnings(module=self.module): - self.module.filterwarnings("always", category=UserWarning) - del self.module.showwarning - with support.captured_output('stderr') as stream: - self.module.warn(text) - result = stream.getvalue() - self.assertIn(text, result) - - def test_showwarning_not_callable(self): - with original_warnings.catch_warnings(module=self.module): - self.module.filterwarnings("always", category=UserWarning) - self.module.showwarning = print - with support.captured_output('stdout'): - self.module.warn('Warning!') - self.module.showwarning = 23 - self.assertRaises(TypeError, self.module.warn, "Warning!") - - def test_show_warning_output(self): - # With showarning() missing, make sure that output is okay. - text = 'test show_warning' - with original_warnings.catch_warnings(module=self.module): - self.module.filterwarnings("always", category=UserWarning) - del self.module.showwarning - with support.captured_output('stderr') as stream: - warning_tests.inner(text) - result = stream.getvalue() - self.assertEqual(result.count('\n'), 2, - "Too many newlines in %r" % result) - first_line, second_line = result.split('\n', 1) - expected_file = os.path.splitext(warning_tests.__file__)[0] + '.py' - first_line_parts = first_line.rsplit(':', 3) - path, line, warning_class, message = first_line_parts - line = int(line) - self.assertEqual(expected_file, path) - self.assertEqual(warning_class, ' ' + UserWarning.__name__) - self.assertEqual(message, ' ' + text) - expected_line = ' ' + linecache.getline(path, line).strip() + '\n' - assert expected_line - self.assertEqual(second_line, expected_line) - - def test_filename_none(self): - # issue #12467: race condition if a warning is emitted at shutdown - globals_dict = globals() - oldfile = globals_dict['__file__'] - try: - catch = original_warnings.catch_warnings(record=True, - module=self.module) - with catch as w: - self.module.filterwarnings("always", category=UserWarning) - globals_dict['__file__'] = None - original_warnings.warn('test', UserWarning) - self.assertTrue(len(w)) - finally: - globals_dict['__file__'] = oldfile - - def test_stderr_none(self): - rc, stdout, stderr = assert_python_ok("-c", - "import sys; sys.stderr = None; " - "import warnings; warnings.simplefilter('always'); " - "warnings.warn('Warning!')") - self.assertEqual(stdout, b'') - self.assertNotIn(b'Warning!', stderr) - self.assertNotIn(b'Error', stderr) - - -class WarningsDisplayTests(BaseTest): - - """Test the displaying of warnings and the ability to overload functions - related to displaying warnings.""" - - def test_formatwarning(self): - message = "msg" - category = Warning - file_name = os.path.splitext(warning_tests.__file__)[0] + '.py' - line_num = 3 - file_line = linecache.getline(file_name, line_num).strip() - format = "%s:%s: %s: %s\n %s\n" - expect = format % (file_name, line_num, category.__name__, message, - file_line) - self.assertEqual(expect, self.module.formatwarning(message, - category, file_name, line_num)) - # Test the 'line' argument. - file_line += " for the win!" - expect = format % (file_name, line_num, category.__name__, message, - file_line) - self.assertEqual(expect, self.module.formatwarning(message, - category, file_name, line_num, file_line)) - - def test_showwarning(self): - file_name = os.path.splitext(warning_tests.__file__)[0] + '.py' - line_num = 3 - expected_file_line = linecache.getline(file_name, line_num).strip() - message = 'msg' - category = Warning - file_object = StringIO() - expect = self.module.formatwarning(message, category, file_name, - line_num) - self.module.showwarning(message, category, file_name, line_num, - file_object) - self.assertEqual(file_object.getvalue(), expect) - # Test 'line' argument. - expected_file_line += "for the win!" - expect = self.module.formatwarning(message, category, file_name, - line_num, expected_file_line) - file_object = StringIO() - self.module.showwarning(message, category, file_name, line_num, - file_object, expected_file_line) - self.assertEqual(expect, file_object.getvalue()) - -class CWarningsDisplayTests(WarningsDisplayTests, unittest.TestCase): - module = c_warnings - -class PyWarningsDisplayTests(WarningsDisplayTests, unittest.TestCase): - module = py_warnings - - -class CatchWarningTests(BaseTest): - - """Test catch_warnings().""" - - def test_catch_warnings_restore(self): - wmod = self.module - orig_filters = wmod.filters - orig_showwarning = wmod.showwarning - # Ensure both showwarning and filters are restored when recording - with wmod.catch_warnings(module=wmod, record=True): - wmod.filters = wmod.showwarning = object() - self.assertTrue(wmod.filters is orig_filters) - self.assertTrue(wmod.showwarning is orig_showwarning) - # Same test, but with recording disabled - with wmod.catch_warnings(module=wmod, record=False): - wmod.filters = wmod.showwarning = object() - self.assertTrue(wmod.filters is orig_filters) - self.assertTrue(wmod.showwarning is orig_showwarning) - - def test_catch_warnings_recording(self): - wmod = self.module - # Ensure warnings are recorded when requested - with wmod.catch_warnings(module=wmod, record=True) as w: - self.assertEqual(w, []) - self.assertTrue(type(w) is list) - wmod.simplefilter("always") - wmod.warn("foo") - self.assertEqual(str(w[-1].message), "foo") - wmod.warn("bar") - self.assertEqual(str(w[-1].message), "bar") - self.assertEqual(str(w[0].message), "foo") - self.assertEqual(str(w[1].message), "bar") - del w[:] - self.assertEqual(w, []) - # Ensure warnings are not recorded when not requested - orig_showwarning = wmod.showwarning - with wmod.catch_warnings(module=wmod, record=False) as w: - self.assertTrue(w is None) - self.assertTrue(wmod.showwarning is orig_showwarning) - - def test_catch_warnings_reentry_guard(self): - wmod = self.module - # Ensure catch_warnings is protected against incorrect usage - x = wmod.catch_warnings(module=wmod, record=True) - self.assertRaises(RuntimeError, x.__exit__) - with x: - self.assertRaises(RuntimeError, x.__enter__) - # Same test, but with recording disabled - x = wmod.catch_warnings(module=wmod, record=False) - self.assertRaises(RuntimeError, x.__exit__) - with x: - self.assertRaises(RuntimeError, x.__enter__) - - def test_catch_warnings_defaults(self): - wmod = self.module - orig_filters = wmod.filters - orig_showwarning = wmod.showwarning - # Ensure default behaviour is not to record warnings - with wmod.catch_warnings(module=wmod) as w: - self.assertTrue(w is None) - self.assertTrue(wmod.showwarning is orig_showwarning) - self.assertTrue(wmod.filters is not orig_filters) - self.assertTrue(wmod.filters is orig_filters) - if wmod is sys.modules['warnings']: - # Ensure the default module is this one - with wmod.catch_warnings() as w: - self.assertTrue(w is None) - self.assertTrue(wmod.showwarning is orig_showwarning) - self.assertTrue(wmod.filters is not orig_filters) - self.assertTrue(wmod.filters is orig_filters) - - def test_check_warnings(self): - # Explicit tests for the test.support convenience wrapper - wmod = self.module - if wmod is not sys.modules['warnings']: - self.skipTest('module to test is not loaded warnings module') - with support.check_warnings(quiet=False) as w: - self.assertEqual(w.warnings, []) - wmod.simplefilter("always") - wmod.warn("foo") - self.assertEqual(str(w.message), "foo") - wmod.warn("bar") - self.assertEqual(str(w.message), "bar") - self.assertEqual(str(w.warnings[0].message), "foo") - self.assertEqual(str(w.warnings[1].message), "bar") - w.reset() - self.assertEqual(w.warnings, []) - - with support.check_warnings(): - # defaults to quiet=True without argument - pass - with support.check_warnings(('foo', UserWarning)): - wmod.warn("foo") - - with self.assertRaises(AssertionError): - with support.check_warnings(('', RuntimeWarning)): - # defaults to quiet=False with argument - pass - with self.assertRaises(AssertionError): - with support.check_warnings(('foo', RuntimeWarning)): - wmod.warn("foo") - -class CCatchWarningTests(CatchWarningTests, unittest.TestCase): - module = c_warnings - -class PyCatchWarningTests(CatchWarningTests, unittest.TestCase): - module = py_warnings - - -class EnvironmentVariableTests(BaseTest): - - def test_single_warning(self): - rc, stdout, stderr = assert_python_ok("-c", - "import sys; sys.stdout.write(str(sys.warnoptions))", - PYTHONWARNINGS="ignore::DeprecationWarning") - self.assertEqual(stdout, b"['ignore::DeprecationWarning']") - - def test_comma_separated_warnings(self): - rc, stdout, stderr = assert_python_ok("-c", - "import sys; sys.stdout.write(str(sys.warnoptions))", - PYTHONWARNINGS="ignore::DeprecationWarning,ignore::UnicodeWarning") - self.assertEqual(stdout, - b"['ignore::DeprecationWarning', 'ignore::UnicodeWarning']") - - def test_envvar_and_command_line(self): - rc, stdout, stderr = assert_python_ok("-Wignore::UnicodeWarning", "-c", - "import sys; sys.stdout.write(str(sys.warnoptions))", - PYTHONWARNINGS="ignore::DeprecationWarning") - self.assertEqual(stdout, - b"['ignore::DeprecationWarning', 'ignore::UnicodeWarning']") - - def test_conflicting_envvar_and_command_line(self): - rc, stdout, stderr = assert_python_failure("-Werror::DeprecationWarning", "-c", - "import sys, warnings; sys.stdout.write(str(sys.warnoptions)); " - "warnings.warn('Message', DeprecationWarning)", - PYTHONWARNINGS="default::DeprecationWarning") - self.assertEqual(stdout, - b"['default::DeprecationWarning', 'error::DeprecationWarning']") - self.assertEqual(stderr.splitlines(), - [b"Traceback (most recent call last):", - b" File \"\", line 1, in ", - b"DeprecationWarning: Message"]) - - @unittest.skipUnless(sys.getfilesystemencoding() != 'ascii', - 'requires non-ascii filesystemencoding') - def test_nonascii(self): - rc, stdout, stderr = assert_python_ok("-c", - "import sys; sys.stdout.write(str(sys.warnoptions))", - PYTHONIOENCODING="utf-8", - PYTHONWARNINGS="ignore:Deprecaci?nWarning") - self.assertEqual(stdout, - "['ignore:Deprecaci?nWarning']".encode('utf-8')) - -class CEnvironmentVariableTests(EnvironmentVariableTests, unittest.TestCase): - module = c_warnings - -class PyEnvironmentVariableTests(EnvironmentVariableTests, unittest.TestCase): - module = py_warnings - - -class BootstrapTest(unittest.TestCase): - def test_issue_8766(self): - # "import encodings" emits a warning whereas the warnings is not loaded - # or not completely loaded (warnings imports indirectly encodings by - # importing linecache) yet - with support.temp_cwd() as cwd, support.temp_cwd('encodings'): - # encodings loaded by initfsencoding() - assert_python_ok('-c', 'pass', PYTHONPATH=cwd) - - # Use -W to load warnings module at startup - assert_python_ok('-c', 'pass', '-W', 'always', PYTHONPATH=cwd) - -class FinalizationTest(unittest.TestCase): - def test_finalization(self): - # Issue #19421: warnings.warn() should not crash - # during Python finalization - code = """ -import warnings -warn = warnings.warn - -class A: - def __del__(self): - warn("test") - -a=A() - """ - rc, out, err = assert_python_ok("-c", code) - # note: "__main__" filename is not correct, it should be the name - # of the script - self.assertEqual(err, b'__main__:7: UserWarning: test') - - -def setUpModule(): - py_warnings.onceregistry.clear() - c_warnings.onceregistry.clear() - -tearDownModule = setUpModule - -if __name__ == "__main__": - unittest.main() diff --git a/Lib/test/test_warnings/__init__.py b/Lib/test/test_warnings/__init__.py new file mode 100644 --- /dev/null +++ b/Lib/test/test_warnings/__init__.py @@ -0,0 +1,955 @@ +from contextlib import contextmanager +import linecache +import os +from io import StringIO +import sys +import unittest +from test import support +from test.support.script_helper import assert_python_ok, assert_python_failure + +from test.test_warnings.data import stacklevel as warning_tests + +import warnings as original_warnings + +py_warnings = support.import_fresh_module('warnings', blocked=['_warnings']) +c_warnings = support.import_fresh_module('warnings', fresh=['_warnings']) + + at contextmanager +def warnings_state(module): + """Use a specific warnings implementation in warning_tests.""" + global __warningregistry__ + for to_clear in (sys, warning_tests): + try: + to_clear.__warningregistry__.clear() + except AttributeError: + pass + try: + __warningregistry__.clear() + except NameError: + pass + original_warnings = warning_tests.warnings + original_filters = module.filters + try: + module.filters = original_filters[:] + module.simplefilter("once") + warning_tests.warnings = module + yield + finally: + warning_tests.warnings = original_warnings + module.filters = original_filters + + +class BaseTest: + + """Basic bookkeeping required for testing.""" + + def setUp(self): + # The __warningregistry__ needs to be in a pristine state for tests + # to work properly. + if '__warningregistry__' in globals(): + del globals()['__warningregistry__'] + if hasattr(warning_tests, '__warningregistry__'): + del warning_tests.__warningregistry__ + if hasattr(sys, '__warningregistry__'): + del sys.__warningregistry__ + # The 'warnings' module must be explicitly set so that the proper + # interaction between _warnings and 'warnings' can be controlled. + sys.modules['warnings'] = self.module + super(BaseTest, self).setUp() + + def tearDown(self): + sys.modules['warnings'] = original_warnings + super(BaseTest, self).tearDown() + +class PublicAPITests(BaseTest): + + """Ensures that the correct values are exposed in the + public API. + """ + + def test_module_all_attribute(self): + self.assertTrue(hasattr(self.module, '__all__')) + target_api = ["warn", "warn_explicit", "showwarning", + "formatwarning", "filterwarnings", "simplefilter", + "resetwarnings", "catch_warnings"] + self.assertSetEqual(set(self.module.__all__), + set(target_api)) + +class CPublicAPITests(PublicAPITests, unittest.TestCase): + module = c_warnings + +class PyPublicAPITests(PublicAPITests, unittest.TestCase): + module = py_warnings + +class FilterTests(BaseTest): + + """Testing the filtering functionality.""" + + def test_error(self): + with original_warnings.catch_warnings(module=self.module) as w: + self.module.resetwarnings() + self.module.filterwarnings("error", category=UserWarning) + self.assertRaises(UserWarning, self.module.warn, + "FilterTests.test_error") + + def test_error_after_default(self): + with original_warnings.catch_warnings(module=self.module) as w: + self.module.resetwarnings() + message = "FilterTests.test_ignore_after_default" + def f(): + self.module.warn(message, UserWarning) + f() + self.module.filterwarnings("error", category=UserWarning) + self.assertRaises(UserWarning, f) + + def test_ignore(self): + with original_warnings.catch_warnings(record=True, + module=self.module) as w: + self.module.resetwarnings() + self.module.filterwarnings("ignore", category=UserWarning) + self.module.warn("FilterTests.test_ignore", UserWarning) + self.assertEqual(len(w), 0) + + def test_ignore_after_default(self): + with original_warnings.catch_warnings(record=True, + module=self.module) as w: + self.module.resetwarnings() + message = "FilterTests.test_ignore_after_default" + def f(): + self.module.warn(message, UserWarning) + f() + self.module.filterwarnings("ignore", category=UserWarning) + f() + f() + self.assertEqual(len(w), 1) + + def test_always(self): + with original_warnings.catch_warnings(record=True, + module=self.module) as w: + self.module.resetwarnings() + self.module.filterwarnings("always", category=UserWarning) + message = "FilterTests.test_always" + self.module.warn(message, UserWarning) + self.assertTrue(message, w[-1].message) + self.module.warn(message, UserWarning) + self.assertTrue(w[-1].message, message) + + def test_always_after_default(self): + with original_warnings.catch_warnings(record=True, + module=self.module) as w: + self.module.resetwarnings() + message = "FilterTests.test_always_after_ignore" + def f(): + self.module.warn(message, UserWarning) + f() + self.assertEqual(len(w), 1) + self.assertEqual(w[-1].message.args[0], message) + f() + self.assertEqual(len(w), 1) + self.module.filterwarnings("always", category=UserWarning) + f() + self.assertEqual(len(w), 2) + self.assertEqual(w[-1].message.args[0], message) + f() + self.assertEqual(len(w), 3) + self.assertEqual(w[-1].message.args[0], message) + + def test_default(self): + with original_warnings.catch_warnings(record=True, + module=self.module) as w: + self.module.resetwarnings() + self.module.filterwarnings("default", category=UserWarning) + message = UserWarning("FilterTests.test_default") + for x in range(2): + self.module.warn(message, UserWarning) + if x == 0: + self.assertEqual(w[-1].message, message) + del w[:] + elif x == 1: + self.assertEqual(len(w), 0) + else: + raise ValueError("loop variant unhandled") + + def test_module(self): + with original_warnings.catch_warnings(record=True, + module=self.module) as w: + self.module.resetwarnings() + self.module.filterwarnings("module", category=UserWarning) + message = UserWarning("FilterTests.test_module") + self.module.warn(message, UserWarning) + self.assertEqual(w[-1].message, message) + del w[:] + self.module.warn(message, UserWarning) + self.assertEqual(len(w), 0) + + def test_once(self): + with original_warnings.catch_warnings(record=True, + module=self.module) as w: + self.module.resetwarnings() + self.module.filterwarnings("once", category=UserWarning) + message = UserWarning("FilterTests.test_once") + self.module.warn_explicit(message, UserWarning, "__init__.py", + 42) + self.assertEqual(w[-1].message, message) + del w[:] + self.module.warn_explicit(message, UserWarning, "__init__.py", + 13) + self.assertEqual(len(w), 0) + self.module.warn_explicit(message, UserWarning, "test_warnings2.py", + 42) + self.assertEqual(len(w), 0) + + def test_inheritance(self): + with original_warnings.catch_warnings(module=self.module) as w: + self.module.resetwarnings() + self.module.filterwarnings("error", category=Warning) + self.assertRaises(UserWarning, self.module.warn, + "FilterTests.test_inheritance", UserWarning) + + def test_ordering(self): + with original_warnings.catch_warnings(record=True, + module=self.module) as w: + self.module.resetwarnings() + self.module.filterwarnings("ignore", category=UserWarning) + self.module.filterwarnings("error", category=UserWarning, + append=True) + del w[:] + try: + self.module.warn("FilterTests.test_ordering", UserWarning) + except UserWarning: + self.fail("order handling for actions failed") + self.assertEqual(len(w), 0) + + def test_filterwarnings(self): + # Test filterwarnings(). + # Implicitly also tests resetwarnings(). + with original_warnings.catch_warnings(record=True, + module=self.module) as w: + self.module.filterwarnings("error", "", Warning, "", 0) + self.assertRaises(UserWarning, self.module.warn, 'convert to error') + + self.module.resetwarnings() + text = 'handle normally' + self.module.warn(text) + self.assertEqual(str(w[-1].message), text) + self.assertTrue(w[-1].category is UserWarning) + + self.module.filterwarnings("ignore", "", Warning, "", 0) + text = 'filtered out' + self.module.warn(text) + self.assertNotEqual(str(w[-1].message), text) + + self.module.resetwarnings() + self.module.filterwarnings("error", "hex*", Warning, "", 0) + self.assertRaises(UserWarning, self.module.warn, 'hex/oct') + text = 'nonmatching text' + self.module.warn(text) + self.assertEqual(str(w[-1].message), text) + self.assertTrue(w[-1].category is UserWarning) + + def test_mutate_filter_list(self): + class X: + def match(self, a): + L[:] = [] + + L = [("default",X(),UserWarning,X(),0) for i in range(2)] + with original_warnings.catch_warnings(record=True, + module=self.module) as w: + self.module.filters = L + self.module.warn_explicit(UserWarning("b"), None, "f.py", 42) + self.assertEqual(str(w[-1].message), "b") + +class CFilterTests(FilterTests, unittest.TestCase): + module = c_warnings + +class PyFilterTests(FilterTests, unittest.TestCase): + module = py_warnings + + +class WarnTests(BaseTest): + + """Test warnings.warn() and warnings.warn_explicit().""" + + def test_message(self): + with original_warnings.catch_warnings(record=True, + module=self.module) as w: + self.module.simplefilter("once") + for i in range(4): + text = 'multi %d' %i # Different text on each call. + self.module.warn(text) + self.assertEqual(str(w[-1].message), text) + self.assertTrue(w[-1].category is UserWarning) + + # Issue 3639 + def test_warn_nonstandard_types(self): + # warn() should handle non-standard types without issue. + for ob in (Warning, None, 42): + with original_warnings.catch_warnings(record=True, + module=self.module) as w: + self.module.simplefilter("once") + self.module.warn(ob) + # Don't directly compare objects since + # ``Warning() != Warning()``. + self.assertEqual(str(w[-1].message), str(UserWarning(ob))) + + def test_filename(self): + with warnings_state(self.module): + with original_warnings.catch_warnings(record=True, + module=self.module) as w: + warning_tests.inner("spam1") + self.assertEqual(os.path.basename(w[-1].filename), + "stacklevel.py") + warning_tests.outer("spam2") + self.assertEqual(os.path.basename(w[-1].filename), + "stacklevel.py") + + def test_stacklevel(self): + # Test stacklevel argument + # make sure all messages are different, so the warning won't be skipped + with warnings_state(self.module): + with original_warnings.catch_warnings(record=True, + module=self.module) as w: + warning_tests.inner("spam3", stacklevel=1) + self.assertEqual(os.path.basename(w[-1].filename), + "stacklevel.py") + warning_tests.outer("spam4", stacklevel=1) + self.assertEqual(os.path.basename(w[-1].filename), + "stacklevel.py") + + warning_tests.inner("spam5", stacklevel=2) + self.assertEqual(os.path.basename(w[-1].filename), + "__init__.py") + warning_tests.outer("spam6", stacklevel=2) + self.assertEqual(os.path.basename(w[-1].filename), + "stacklevel.py") + warning_tests.outer("spam6.5", stacklevel=3) + self.assertEqual(os.path.basename(w[-1].filename), + "__init__.py") + + warning_tests.inner("spam7", stacklevel=9999) + self.assertEqual(os.path.basename(w[-1].filename), + "sys") + + def test_stacklevel_import(self): + # Issue #24305: With stacklevel=2, module-level warnings should work. + support.unload('test.test_warnings.data.import_warning') + with warnings_state(self.module): + with original_warnings.catch_warnings(record=True, + module=self.module) as w: + self.module.simplefilter('always') + import test.test_warnings.data.import_warning + self.assertEqual(len(w), 1) + self.assertEqual(w[0].filename, __file__) + + def test_missing_filename_not_main(self): + # If __file__ is not specified and __main__ is not the module name, + # then __file__ should be set to the module name. + filename = warning_tests.__file__ + try: + del warning_tests.__file__ + with warnings_state(self.module): + with original_warnings.catch_warnings(record=True, + module=self.module) as w: + warning_tests.inner("spam8", stacklevel=1) + self.assertEqual(w[-1].filename, warning_tests.__name__) + finally: + warning_tests.__file__ = filename + + @unittest.skipUnless(hasattr(sys, 'argv'), 'test needs sys.argv') + def test_missing_filename_main_with_argv(self): + # If __file__ is not specified and the caller is __main__ and sys.argv + # exists, then use sys.argv[0] as the file. + filename = warning_tests.__file__ + module_name = warning_tests.__name__ + try: + del warning_tests.__file__ + warning_tests.__name__ = '__main__' + with warnings_state(self.module): + with original_warnings.catch_warnings(record=True, + module=self.module) as w: + warning_tests.inner('spam9', stacklevel=1) + self.assertEqual(w[-1].filename, sys.argv[0]) + finally: + warning_tests.__file__ = filename + warning_tests.__name__ = module_name + + def test_missing_filename_main_without_argv(self): + # If __file__ is not specified, the caller is __main__, and sys.argv + # is not set, then '__main__' is the file name. + filename = warning_tests.__file__ + module_name = warning_tests.__name__ + argv = sys.argv + try: + del warning_tests.__file__ + warning_tests.__name__ = '__main__' + del sys.argv + with warnings_state(self.module): + with original_warnings.catch_warnings(record=True, + module=self.module) as w: + warning_tests.inner('spam10', stacklevel=1) + self.assertEqual(w[-1].filename, '__main__') + finally: + warning_tests.__file__ = filename + warning_tests.__name__ = module_name + sys.argv = argv + + def test_missing_filename_main_with_argv_empty_string(self): + # If __file__ is not specified, the caller is __main__, and sys.argv[0] + # is the empty string, then '__main__ is the file name. + # Tests issue 2743. + file_name = warning_tests.__file__ + module_name = warning_tests.__name__ + argv = sys.argv + try: + del warning_tests.__file__ + warning_tests.__name__ = '__main__' + sys.argv = [''] + with warnings_state(self.module): + with original_warnings.catch_warnings(record=True, + module=self.module) as w: + warning_tests.inner('spam11', stacklevel=1) + self.assertEqual(w[-1].filename, '__main__') + finally: + warning_tests.__file__ = file_name + warning_tests.__name__ = module_name + sys.argv = argv + + def test_warn_explicit_non_ascii_filename(self): + with original_warnings.catch_warnings(record=True, + module=self.module) as w: + self.module.resetwarnings() + self.module.filterwarnings("always", category=UserWarning) + for filename in ("nonascii\xe9\u20ac", "surrogate\udc80"): + try: + os.fsencode(filename) + except UnicodeEncodeError: + continue + self.module.warn_explicit("text", UserWarning, filename, 1) + self.assertEqual(w[-1].filename, filename) + + def test_warn_explicit_type_errors(self): + # warn_explicit() should error out gracefully if it is given objects + # of the wrong types. + # lineno is expected to be an integer. + self.assertRaises(TypeError, self.module.warn_explicit, + None, UserWarning, None, None) + # Either 'message' needs to be an instance of Warning or 'category' + # needs to be a subclass. + self.assertRaises(TypeError, self.module.warn_explicit, + None, None, None, 1) + # 'registry' must be a dict or None. + self.assertRaises((TypeError, AttributeError), + self.module.warn_explicit, + None, Warning, None, 1, registry=42) + + def test_bad_str(self): + # issue 6415 + # Warnings instance with a bad format string for __str__ should not + # trigger a bus error. + class BadStrWarning(Warning): + """Warning with a bad format string for __str__.""" + def __str__(self): + return ("A bad formatted string %(err)" % + {"err" : "there is no %(err)s"}) + + with self.assertRaises(ValueError): + self.module.warn(BadStrWarning()) + + def test_warning_classes(self): + class MyWarningClass(Warning): + pass + + class NonWarningSubclass: + pass + + # passing a non-subclass of Warning should raise a TypeError + with self.assertRaises(TypeError) as cm: + self.module.warn('bad warning category', '') + self.assertIn('category must be a Warning subclass, not ', + str(cm.exception)) + + with self.assertRaises(TypeError) as cm: + self.module.warn('bad warning category', NonWarningSubclass) + self.assertIn('category must be a Warning subclass, not ', + str(cm.exception)) + + # check that warning instances also raise a TypeError + with self.assertRaises(TypeError) as cm: + self.module.warn('bad warning category', MyWarningClass()) + self.assertIn('category must be a Warning subclass, not ', + str(cm.exception)) + + with original_warnings.catch_warnings(module=self.module): + self.module.resetwarnings() + self.module.filterwarnings('default') + with self.assertWarns(MyWarningClass) as cm: + self.module.warn('good warning category', MyWarningClass) + self.assertEqual('good warning category', str(cm.warning)) + + with self.assertWarns(UserWarning) as cm: + self.module.warn('good warning category', None) + self.assertEqual('good warning category', str(cm.warning)) + + with self.assertWarns(MyWarningClass) as cm: + self.module.warn('good warning category', MyWarningClass) + self.assertIsInstance(cm.warning, Warning) + +class CWarnTests(WarnTests, unittest.TestCase): + module = c_warnings + + # As an early adopter, we sanity check the + # test.support.import_fresh_module utility function + def test_accelerated(self): + self.assertFalse(original_warnings is self.module) + self.assertFalse(hasattr(self.module.warn, '__code__')) + +class PyWarnTests(WarnTests, unittest.TestCase): + module = py_warnings + + # As an early adopter, we sanity check the + # test.support.import_fresh_module utility function + def test_pure_python(self): + self.assertFalse(original_warnings is self.module) + self.assertTrue(hasattr(self.module.warn, '__code__')) + + +class WCmdLineTests(BaseTest): + + def test_improper_input(self): + # Uses the private _setoption() function to test the parsing + # of command-line warning arguments + with original_warnings.catch_warnings(module=self.module): + self.assertRaises(self.module._OptionError, + self.module._setoption, '1:2:3:4:5:6') + self.assertRaises(self.module._OptionError, + self.module._setoption, 'bogus::Warning') + self.assertRaises(self.module._OptionError, + self.module._setoption, 'ignore:2::4:-5') + self.module._setoption('error::Warning::0') + self.assertRaises(UserWarning, self.module.warn, 'convert to error') + + def test_improper_option(self): + # Same as above, but check that the message is printed out when + # the interpreter is executed. This also checks that options are + # actually parsed at all. + rc, out, err = assert_python_ok("-Wxxx", "-c", "pass") + self.assertIn(b"Invalid -W option ignored: invalid action: 'xxx'", err) + + def test_warnings_bootstrap(self): + # Check that the warnings module does get loaded when -W + # is used (see issue #10372 for an example of silent bootstrap failure). + rc, out, err = assert_python_ok("-Wi", "-c", + "import sys; sys.modules['warnings'].warn('foo', RuntimeWarning)") + # '-Wi' was observed + self.assertFalse(out.strip()) + self.assertNotIn(b'RuntimeWarning', err) + +class CWCmdLineTests(WCmdLineTests, unittest.TestCase): + module = c_warnings + +class PyWCmdLineTests(WCmdLineTests, unittest.TestCase): + module = py_warnings + + +class _WarningsTests(BaseTest, unittest.TestCase): + + """Tests specific to the _warnings module.""" + + module = c_warnings + + def test_filter(self): + # Everything should function even if 'filters' is not in warnings. + with original_warnings.catch_warnings(module=self.module) as w: + self.module.filterwarnings("error", "", Warning, "", 0) + self.assertRaises(UserWarning, self.module.warn, + 'convert to error') + del self.module.filters + self.assertRaises(UserWarning, self.module.warn, + 'convert to error') + + def test_onceregistry(self): + # Replacing or removing the onceregistry should be okay. + global __warningregistry__ + message = UserWarning('onceregistry test') + try: + original_registry = self.module.onceregistry + __warningregistry__ = {} + with original_warnings.catch_warnings(record=True, + module=self.module) as w: + self.module.resetwarnings() + self.module.filterwarnings("once", category=UserWarning) + self.module.warn_explicit(message, UserWarning, "file", 42) + self.assertEqual(w[-1].message, message) + del w[:] + self.module.warn_explicit(message, UserWarning, "file", 42) + self.assertEqual(len(w), 0) + # Test the resetting of onceregistry. + self.module.onceregistry = {} + __warningregistry__ = {} + self.module.warn('onceregistry test') + self.assertEqual(w[-1].message.args, message.args) + # Removal of onceregistry is okay. + del w[:] + del self.module.onceregistry + __warningregistry__ = {} + self.module.warn_explicit(message, UserWarning, "file", 42) + self.assertEqual(len(w), 0) + finally: + self.module.onceregistry = original_registry + + def test_default_action(self): + # Replacing or removing defaultaction should be okay. + message = UserWarning("defaultaction test") + original = self.module.defaultaction + try: + with original_warnings.catch_warnings(record=True, + module=self.module) as w: + self.module.resetwarnings() + registry = {} + self.module.warn_explicit(message, UserWarning, "", 42, + registry=registry) + self.assertEqual(w[-1].message, message) + self.assertEqual(len(w), 1) + # One actual registry key plus the "version" key + self.assertEqual(len(registry), 2) + self.assertIn("version", registry) + del w[:] + # Test removal. + del self.module.defaultaction + __warningregistry__ = {} + registry = {} + self.module.warn_explicit(message, UserWarning, "", 43, + registry=registry) + self.assertEqual(w[-1].message, message) + self.assertEqual(len(w), 1) + self.assertEqual(len(registry), 2) + del w[:] + # Test setting. + self.module.defaultaction = "ignore" + __warningregistry__ = {} + registry = {} + self.module.warn_explicit(message, UserWarning, "", 44, + registry=registry) + self.assertEqual(len(w), 0) + finally: + self.module.defaultaction = original + + def test_showwarning_missing(self): + # Test that showwarning() missing is okay. + text = 'del showwarning test' + with original_warnings.catch_warnings(module=self.module): + self.module.filterwarnings("always", category=UserWarning) + del self.module.showwarning + with support.captured_output('stderr') as stream: + self.module.warn(text) + result = stream.getvalue() + self.assertIn(text, result) + + def test_showwarning_not_callable(self): + with original_warnings.catch_warnings(module=self.module): + self.module.filterwarnings("always", category=UserWarning) + self.module.showwarning = print + with support.captured_output('stdout'): + self.module.warn('Warning!') + self.module.showwarning = 23 + self.assertRaises(TypeError, self.module.warn, "Warning!") + + def test_show_warning_output(self): + # With showarning() missing, make sure that output is okay. + text = 'test show_warning' + with original_warnings.catch_warnings(module=self.module): + self.module.filterwarnings("always", category=UserWarning) + del self.module.showwarning + with support.captured_output('stderr') as stream: + warning_tests.inner(text) + result = stream.getvalue() + self.assertEqual(result.count('\n'), 2, + "Too many newlines in %r" % result) + first_line, second_line = result.split('\n', 1) + expected_file = os.path.splitext(warning_tests.__file__)[0] + '.py' + first_line_parts = first_line.rsplit(':', 3) + path, line, warning_class, message = first_line_parts + line = int(line) + self.assertEqual(expected_file, path) + self.assertEqual(warning_class, ' ' + UserWarning.__name__) + self.assertEqual(message, ' ' + text) + expected_line = ' ' + linecache.getline(path, line).strip() + '\n' + assert expected_line + self.assertEqual(second_line, expected_line) + + def test_filename_none(self): + # issue #12467: race condition if a warning is emitted at shutdown + globals_dict = globals() + oldfile = globals_dict['__file__'] + try: + catch = original_warnings.catch_warnings(record=True, + module=self.module) + with catch as w: + self.module.filterwarnings("always", category=UserWarning) + globals_dict['__file__'] = None + original_warnings.warn('test', UserWarning) + self.assertTrue(len(w)) + finally: + globals_dict['__file__'] = oldfile + + def test_stderr_none(self): + rc, stdout, stderr = assert_python_ok("-c", + "import sys; sys.stderr = None; " + "import warnings; warnings.simplefilter('always'); " + "warnings.warn('Warning!')") + self.assertEqual(stdout, b'') + self.assertNotIn(b'Warning!', stderr) + self.assertNotIn(b'Error', stderr) + + +class WarningsDisplayTests(BaseTest): + + """Test the displaying of warnings and the ability to overload functions + related to displaying warnings.""" + + def test_formatwarning(self): + message = "msg" + category = Warning + file_name = os.path.splitext(warning_tests.__file__)[0] + '.py' + line_num = 3 + file_line = linecache.getline(file_name, line_num).strip() + format = "%s:%s: %s: %s\n %s\n" + expect = format % (file_name, line_num, category.__name__, message, + file_line) + self.assertEqual(expect, self.module.formatwarning(message, + category, file_name, line_num)) + # Test the 'line' argument. + file_line += " for the win!" + expect = format % (file_name, line_num, category.__name__, message, + file_line) + self.assertEqual(expect, self.module.formatwarning(message, + category, file_name, line_num, file_line)) + + def test_showwarning(self): + file_name = os.path.splitext(warning_tests.__file__)[0] + '.py' + line_num = 3 + expected_file_line = linecache.getline(file_name, line_num).strip() + message = 'msg' + category = Warning + file_object = StringIO() + expect = self.module.formatwarning(message, category, file_name, + line_num) + self.module.showwarning(message, category, file_name, line_num, + file_object) + self.assertEqual(file_object.getvalue(), expect) + # Test 'line' argument. + expected_file_line += "for the win!" + expect = self.module.formatwarning(message, category, file_name, + line_num, expected_file_line) + file_object = StringIO() + self.module.showwarning(message, category, file_name, line_num, + file_object, expected_file_line) + self.assertEqual(expect, file_object.getvalue()) + +class CWarningsDisplayTests(WarningsDisplayTests, unittest.TestCase): + module = c_warnings + +class PyWarningsDisplayTests(WarningsDisplayTests, unittest.TestCase): + module = py_warnings + + +class CatchWarningTests(BaseTest): + + """Test catch_warnings().""" + + def test_catch_warnings_restore(self): + wmod = self.module + orig_filters = wmod.filters + orig_showwarning = wmod.showwarning + # Ensure both showwarning and filters are restored when recording + with wmod.catch_warnings(module=wmod, record=True): + wmod.filters = wmod.showwarning = object() + self.assertTrue(wmod.filters is orig_filters) + self.assertTrue(wmod.showwarning is orig_showwarning) + # Same test, but with recording disabled + with wmod.catch_warnings(module=wmod, record=False): + wmod.filters = wmod.showwarning = object() + self.assertTrue(wmod.filters is orig_filters) + self.assertTrue(wmod.showwarning is orig_showwarning) + + def test_catch_warnings_recording(self): + wmod = self.module + # Ensure warnings are recorded when requested + with wmod.catch_warnings(module=wmod, record=True) as w: + self.assertEqual(w, []) + self.assertTrue(type(w) is list) + wmod.simplefilter("always") + wmod.warn("foo") + self.assertEqual(str(w[-1].message), "foo") + wmod.warn("bar") + self.assertEqual(str(w[-1].message), "bar") + self.assertEqual(str(w[0].message), "foo") + self.assertEqual(str(w[1].message), "bar") + del w[:] + self.assertEqual(w, []) + # Ensure warnings are not recorded when not requested + orig_showwarning = wmod.showwarning + with wmod.catch_warnings(module=wmod, record=False) as w: + self.assertTrue(w is None) + self.assertTrue(wmod.showwarning is orig_showwarning) + + def test_catch_warnings_reentry_guard(self): + wmod = self.module + # Ensure catch_warnings is protected against incorrect usage + x = wmod.catch_warnings(module=wmod, record=True) + self.assertRaises(RuntimeError, x.__exit__) + with x: + self.assertRaises(RuntimeError, x.__enter__) + # Same test, but with recording disabled + x = wmod.catch_warnings(module=wmod, record=False) + self.assertRaises(RuntimeError, x.__exit__) + with x: + self.assertRaises(RuntimeError, x.__enter__) + + def test_catch_warnings_defaults(self): + wmod = self.module + orig_filters = wmod.filters + orig_showwarning = wmod.showwarning + # Ensure default behaviour is not to record warnings + with wmod.catch_warnings(module=wmod) as w: + self.assertTrue(w is None) + self.assertTrue(wmod.showwarning is orig_showwarning) + self.assertTrue(wmod.filters is not orig_filters) + self.assertTrue(wmod.filters is orig_filters) + if wmod is sys.modules['warnings']: + # Ensure the default module is this one + with wmod.catch_warnings() as w: + self.assertTrue(w is None) + self.assertTrue(wmod.showwarning is orig_showwarning) + self.assertTrue(wmod.filters is not orig_filters) + self.assertTrue(wmod.filters is orig_filters) + + def test_check_warnings(self): + # Explicit tests for the test.support convenience wrapper + wmod = self.module + if wmod is not sys.modules['warnings']: + self.skipTest('module to test is not loaded warnings module') + with support.check_warnings(quiet=False) as w: + self.assertEqual(w.warnings, []) + wmod.simplefilter("always") + wmod.warn("foo") + self.assertEqual(str(w.message), "foo") + wmod.warn("bar") + self.assertEqual(str(w.message), "bar") + self.assertEqual(str(w.warnings[0].message), "foo") + self.assertEqual(str(w.warnings[1].message), "bar") + w.reset() + self.assertEqual(w.warnings, []) + + with support.check_warnings(): + # defaults to quiet=True without argument + pass + with support.check_warnings(('foo', UserWarning)): + wmod.warn("foo") + + with self.assertRaises(AssertionError): + with support.check_warnings(('', RuntimeWarning)): + # defaults to quiet=False with argument + pass + with self.assertRaises(AssertionError): + with support.check_warnings(('foo', RuntimeWarning)): + wmod.warn("foo") + +class CCatchWarningTests(CatchWarningTests, unittest.TestCase): + module = c_warnings + +class PyCatchWarningTests(CatchWarningTests, unittest.TestCase): + module = py_warnings + + +class EnvironmentVariableTests(BaseTest): + + def test_single_warning(self): + rc, stdout, stderr = assert_python_ok("-c", + "import sys; sys.stdout.write(str(sys.warnoptions))", + PYTHONWARNINGS="ignore::DeprecationWarning") + self.assertEqual(stdout, b"['ignore::DeprecationWarning']") + + def test_comma_separated_warnings(self): + rc, stdout, stderr = assert_python_ok("-c", + "import sys; sys.stdout.write(str(sys.warnoptions))", + PYTHONWARNINGS="ignore::DeprecationWarning,ignore::UnicodeWarning") + self.assertEqual(stdout, + b"['ignore::DeprecationWarning', 'ignore::UnicodeWarning']") + + def test_envvar_and_command_line(self): + rc, stdout, stderr = assert_python_ok("-Wignore::UnicodeWarning", "-c", + "import sys; sys.stdout.write(str(sys.warnoptions))", + PYTHONWARNINGS="ignore::DeprecationWarning") + self.assertEqual(stdout, + b"['ignore::DeprecationWarning', 'ignore::UnicodeWarning']") + + def test_conflicting_envvar_and_command_line(self): + rc, stdout, stderr = assert_python_failure("-Werror::DeprecationWarning", "-c", + "import sys, warnings; sys.stdout.write(str(sys.warnoptions)); " + "warnings.warn('Message', DeprecationWarning)", + PYTHONWARNINGS="default::DeprecationWarning") + self.assertEqual(stdout, + b"['default::DeprecationWarning', 'error::DeprecationWarning']") + self.assertEqual(stderr.splitlines(), + [b"Traceback (most recent call last):", + b" File \"\", line 1, in ", + b"DeprecationWarning: Message"]) + + @unittest.skipUnless(sys.getfilesystemencoding() != 'ascii', + 'requires non-ascii filesystemencoding') + def test_nonascii(self): + rc, stdout, stderr = assert_python_ok("-c", + "import sys; sys.stdout.write(str(sys.warnoptions))", + PYTHONIOENCODING="utf-8", + PYTHONWARNINGS="ignore:Deprecaci?nWarning") + self.assertEqual(stdout, + "['ignore:Deprecaci?nWarning']".encode('utf-8')) + +class CEnvironmentVariableTests(EnvironmentVariableTests, unittest.TestCase): + module = c_warnings + +class PyEnvironmentVariableTests(EnvironmentVariableTests, unittest.TestCase): + module = py_warnings + + +class BootstrapTest(unittest.TestCase): + def test_issue_8766(self): + # "import encodings" emits a warning whereas the warnings is not loaded + # or not completely loaded (warnings imports indirectly encodings by + # importing linecache) yet + with support.temp_cwd() as cwd, support.temp_cwd('encodings'): + # encodings loaded by initfsencoding() + assert_python_ok('-c', 'pass', PYTHONPATH=cwd) + + # Use -W to load warnings module at startup + assert_python_ok('-c', 'pass', '-W', 'always', PYTHONPATH=cwd) + +class FinalizationTest(unittest.TestCase): + def test_finalization(self): + # Issue #19421: warnings.warn() should not crash + # during Python finalization + code = """ +import warnings +warn = warnings.warn + +class A: + def __del__(self): + warn("test") + +a=A() + """ + rc, out, err = assert_python_ok("-c", code) + # note: "__main__" filename is not correct, it should be the name + # of the script + self.assertEqual(err, b'__main__:7: UserWarning: test') + + +def setUpModule(): + py_warnings.onceregistry.clear() + c_warnings.onceregistry.clear() + +tearDownModule = setUpModule + +if __name__ == "__main__": + unittest.main() diff --git a/Lib/test/test_warnings/__main__.py b/Lib/test/test_warnings/__main__.py new file mode 100644 --- /dev/null +++ b/Lib/test/test_warnings/__main__.py @@ -0,0 +1,3 @@ +import unittest + +unittest.main('test.test_warnings') diff --git a/Lib/test/test_warnings/data/import_warning.py b/Lib/test/test_warnings/data/import_warning.py new file mode 100644 --- /dev/null +++ b/Lib/test/test_warnings/data/import_warning.py @@ -0,0 +1,3 @@ +import warnings + +warnings.warn('module-level warning', DeprecationWarning, stacklevel=2) \ No newline at end of file diff --git a/Lib/test/warning_tests.py b/Lib/test/test_warnings/data/stacklevel.py rename from Lib/test/warning_tests.py rename to Lib/test/test_warnings/data/stacklevel.py diff --git a/Lib/warnings.py b/Lib/warnings.py --- a/Lib/warnings.py +++ b/Lib/warnings.py @@ -160,6 +160,20 @@ return cat +def _is_internal_frame(frame): + """Signal whether the frame is an internal CPython implementation detail.""" + filename = frame.f_code.co_filename + return 'importlib' in filename and '_bootstrap' in filename + + +def _next_external_frame(frame): + """Find the next frame that doesn't involve CPython internals.""" + frame = frame.f_back + while frame is not None and _is_internal_frame(frame): + frame = frame.f_back + return frame + + # Code typically replaced by _warnings def warn(message, category=None, stacklevel=1): """Issue a warning, or maybe ignore it or raise an exception.""" @@ -174,13 +188,23 @@ "not '{:s}'".format(type(category).__name__)) # Get context information try: - caller = sys._getframe(stacklevel) + if stacklevel <= 1 or _is_internal_frame(sys._getframe(1)): + # If frame is too small to care or if the warning originated in + # internal code, then do not try to hide any frames. + frame = sys._getframe(stacklevel) + else: + frame = sys._getframe(1) + # Look for one frame less since the above line starts us off. + for x in range(stacklevel-1): + frame = _next_external_frame(frame) + if frame is None: + raise ValueError except ValueError: globals = sys.__dict__ lineno = 1 else: - globals = caller.f_globals - lineno = caller.f_lineno + globals = frame.f_globals + lineno = frame.f_lineno if '__name__' in globals: module = globals['__name__'] else: @@ -374,7 +398,6 @@ defaultaction = _defaultaction onceregistry = _onceregistry _warnings_defaults = True - except ImportError: filters = [] defaultaction = "default" diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -10,6 +10,9 @@ Core and Builtins ----------------- +- Issue #24305: Prevent import subsystem stack frames from being counted + by the warnings.warn(stacklevel=) parameter. + - Issue #24912: Prevent __class__ assignment to immutable built-in objects. - Issue #24975: Fix AST compilation for PEP 448 syntax. diff --git a/Python/_warnings.c b/Python/_warnings.c --- a/Python/_warnings.c +++ b/Python/_warnings.c @@ -513,6 +513,64 @@ return result; /* Py_None or NULL. */ } +static int +is_internal_frame(PyFrameObject *frame) +{ + static PyObject *importlib_string = NULL; + static PyObject *bootstrap_string = NULL; + PyObject *filename; + int contains; + + if (importlib_string == NULL) { + importlib_string = PyUnicode_FromString("importlib"); + if (importlib_string == NULL) { + return 0; + } + + bootstrap_string = PyUnicode_FromString("_bootstrap"); + if (bootstrap_string == NULL) { + Py_DECREF(importlib_string); + return 0; + } + Py_INCREF(importlib_string); + Py_INCREF(bootstrap_string); + } + + if (frame == NULL || frame->f_code == NULL || + frame->f_code->co_filename == NULL) { + return 0; + } + filename = frame->f_code->co_filename; + if (!PyUnicode_Check(filename)) { + return 0; + } + contains = PyUnicode_Contains(filename, importlib_string); + if (contains < 0) { + return 0; + } + else if (contains > 0) { + contains = PyUnicode_Contains(filename, bootstrap_string); + if (contains < 0) { + return 0; + } + else if (contains > 0) { + return 1; + } + } + + return 0; +} + +static PyFrameObject * +next_external_frame(PyFrameObject *frame) +{ + do { + frame = frame->f_back; + } while (frame != NULL && is_internal_frame(frame)); + + return frame; +} + /* filename, module, and registry are new refs, globals is borrowed */ /* Returns 0 on error (no new refs), 1 on success */ static int @@ -523,8 +581,18 @@ /* Setup globals and lineno. */ PyFrameObject *f = PyThreadState_GET()->frame; - while (--stack_level > 0 && f != NULL) - f = f->f_back; + // Stack level comparisons to Python code is off by one as there is no + // warnings-related stack level to avoid. + if (stack_level <= 0 || is_internal_frame(f)) { + while (--stack_level > 0 && f != NULL) { + f = f->f_back; + } + } + else { + while (--stack_level > 0 && f != NULL) { + f = next_external_frame(f); + } + } if (f == NULL) { globals = PyThreadState_Get()->interp->sysdict; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 7 07:38:06 2015 From: python-checkins at python.org (steve.dower) Date: Mon, 07 Sep 2015 05:38:06 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzI0OTE3?= =?utf-8?q?=3A_time=5Fstrftime=28=29_buffer_over-read=2E?= Message-ID: <20150907053806.14865.55257@psf.io> https://hg.python.org/cpython/rev/f185917498ca changeset: 97732:f185917498ca branch: 3.4 parent: 97720:7a9dddc11f2e user: Steve Dower date: Sun Sep 06 22:18:36 2015 -0700 summary: Issue #24917: time_strftime() buffer over-read. files: Misc/NEWS | 2 ++ Modules/timemodule.c | 2 ++ 2 files changed, 4 insertions(+), 0 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -81,6 +81,8 @@ Library ------- +- Issue #24917: time_strftime() buffer over-read. + - Issue #23144: Make sure that HTMLParser.feed() returns all the data, even when convert_charrefs is True. diff --git a/Modules/timemodule.c b/Modules/timemodule.c --- a/Modules/timemodule.c +++ b/Modules/timemodule.c @@ -655,6 +655,8 @@ outbuf != NULL; outbuf = wcschr(outbuf+2, '%')) { + if (outbuf[1] == L'\0') + break; /* Issue #19634: On AIX, wcsftime("y", (1899, 1, 1, 0, 0, 0, 0, 0, 0)) returns "0/" instead of "99" */ if (outbuf[1] == L'y' && buf.tm_year < 0) { -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 7 07:38:07 2015 From: python-checkins at python.org (steve.dower) Date: Mon, 07 Sep 2015 05:38:07 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy41IC0+IDMuNSk6?= =?utf-8?q?_Merged_in_stevedower/cpython350_=28pull_request_=2320=29?= Message-ID: <20150907053806.68887.21489@psf.io> https://hg.python.org/cpython/rev/7d320c3bf9c6 changeset: 97731:7d320c3bf9c6 branch: 3.5 parent: 97729:c31dad22c80d parent: 97730:aa60b34d5200 user: Larry Hastings date: Sun Sep 06 22:10:22 2015 -0700 summary: Merged in stevedower/cpython350 (pull request #20) Issue #25005: Backout fix for #8232 because of use of unsafe subprocess.call(shell=True) files: Lib/webbrowser.py | 120 ++------------------------------- Misc/NEWS | 3 - 2 files changed, 9 insertions(+), 114 deletions(-) diff --git a/Lib/webbrowser.py b/Lib/webbrowser.py --- a/Lib/webbrowser.py +++ b/Lib/webbrowser.py @@ -495,23 +495,10 @@ # if sys.platform[:3] == "win": - class WindowsDefault(BaseBrowser): - # Windows Default opening arguments. - - cmd = "start" - newwindow = "" - newtab = "" - def open(self, url, new=0, autoraise=True): - # Format the command for optional arguments and add the url. - if new == 1: - self.cmd += " " + self.newwindow - elif new == 2: - self.cmd += " " + self.newtab - self.cmd += " " + url try: - subprocess.call(self.cmd, shell=True) + os.startfile(url) except OSError: # [Error 22] No application is associated with the specified # file for this operation: '' @@ -519,108 +506,19 @@ else: return True - - # Windows Sub-Classes for commonly used browsers. - - class InternetExplorer(WindowsDefault): - """Launcher class for Internet Explorer browser""" - - cmd = "start iexplore.exe" - newwindow = "" - newtab = "" - - - class WinChrome(WindowsDefault): - """Launcher class for windows specific Google Chrome browser""" - - cmd = "start chrome.exe" - newwindow = "-new-window" - newtab = "-new-tab" - - - class WinFirefox(WindowsDefault): - """Launcher class for windows specific Firefox browser""" - - cmd = "start firefox.exe" - newwindow = "-new-window" - newtab = "-new-tab" - - - class WinOpera(WindowsDefault): - """Launcher class for windows specific Opera browser""" - - cmd = "start opera" - newwindow = "" - newtab = "" - - - class WinSeaMonkey(WindowsDefault): - """Launcher class for windows specific SeaMonkey browser""" - - cmd = "start seamonkey" - newwinow = "" - newtab = "" - - _tryorder = [] _browsers = {} - # First try to use the default Windows browser. + # First try to use the default Windows browser register("windows-default", WindowsDefault) - def find_windows_browsers(): - """ Access the windows registry to determine - what browsers are on the system. - """ - - import winreg - HKLM = winreg.HKEY_LOCAL_MACHINE - subkey = r'Software\Clients\StartMenuInternet' - read32 = winreg.KEY_READ | winreg.KEY_WOW64_32KEY - read64 = winreg.KEY_READ | winreg.KEY_WOW64_64KEY - key32 = winreg.OpenKey(HKLM, subkey, access=read32) - key64 = winreg.OpenKey(HKLM, subkey, access=read64) - - # Return a list of browsers found in the registry - # Check if there are any different browsers in the - # 32 bit location instead of the 64 bit location. - browsers = [] - i = 0 - while True: - try: - browsers.append(winreg.EnumKey(key32, i)) - except EnvironmentError: - break - i += 1 - - i = 0 - while True: - try: - browsers.append(winreg.EnumKey(key64, i)) - except EnvironmentError: - break - i += 1 - - winreg.CloseKey(key32) - winreg.CloseKey(key64) - - return browsers - - # Detect some common windows browsers - for browser in find_windows_browsers(): - browser = browser.lower() - if "iexplore" in browser: - register("iexplore", None, InternetExplorer("iexplore")) - elif "chrome" in browser: - register("chrome", None, WinChrome("chrome")) - elif "firefox" in browser: - register("firefox", None, WinFirefox("firefox")) - elif "opera" in browser: - register("opera", None, WinOpera("opera")) - elif "seamonkey" in browser: - register("seamonkey", None, WinSeaMonkey("seamonkey")) - else: - register(browser, None, WindowsDefault(browser)) + # Detect some common Windows browsers, fallback to IE + iexplore = os.path.join(os.environ.get("PROGRAMFILES", "C:\\Program Files"), + "Internet Explorer\\IEXPLORE.EXE") + for browser in ("firefox", "firebird", "seamonkey", "mozilla", + "netscape", "opera", iexplore): + if shutil.which(browser): + register(browser, None, BackgroundBrowser(browser)) # # Platform support for MacOS diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -314,9 +314,6 @@ - Issue #14373: C implementation of functools.lru_cache() now can be used with methods. -- Issue #8232: webbrowser support incomplete on Windows. Patch by Brandon - Milam - - Issue #24347: Set KeyError if PyDict_GetItemWithError returns NULL. - Issue #24348: Drop superfluous incref/decref. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 7 07:38:07 2015 From: python-checkins at python.org (steve.dower) Date: Mon, 07 Sep 2015 05:38:07 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_Reapplied_chan?= =?utf-8?q?ge_to_test=5Fwarnings=2Epy_to_test=5Fwarnings/=5F=5Finit=5F=5F?= =?utf-8?q?=2Epy=2E?= Message-ID: <20150907053807.14865.72774@psf.io> https://hg.python.org/cpython/rev/56c10f7e0aed changeset: 97734:56c10f7e0aed branch: 3.5 user: Steve Dower date: Sun Sep 06 22:30:40 2015 -0700 summary: Reapplied change to test_warnings.py to test_warnings/__init__.py. files: Lib/test/test_warnings/__init__.py | 6 ++++++ 1 files changed, 6 insertions(+), 0 deletions(-) diff --git a/Lib/test/test_warnings/__init__.py b/Lib/test/test_warnings/__init__.py --- a/Lib/test/test_warnings/__init__.py +++ b/Lib/test/test_warnings/__init__.py @@ -44,6 +44,7 @@ """Basic bookkeeping required for testing.""" def setUp(self): + self.old_unittest_module = unittest.case.warnings # The __warningregistry__ needs to be in a pristine state for tests # to work properly. if '__warningregistry__' in globals(): @@ -55,10 +56,15 @@ # The 'warnings' module must be explicitly set so that the proper # interaction between _warnings and 'warnings' can be controlled. sys.modules['warnings'] = self.module + # Ensure that unittest.TestCase.assertWarns() uses the same warnings + # module than warnings.catch_warnings(). Otherwise, + # warnings.catch_warnings() will be unable to remove the added filter. + unittest.case.warnings = self.module super(BaseTest, self).setUp() def tearDown(self): sys.modules['warnings'] = original_warnings + unittest.case.warnings = self.old_unittest_module super(BaseTest, self).tearDown() class PublicAPITests(BaseTest): -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 7 07:38:07 2015 From: python-checkins at python.org (steve.dower) Date: Mon, 07 Sep 2015 05:38:07 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Null_merge_from_3=2E4?= Message-ID: <20150907053807.11266.95022@psf.io> https://hg.python.org/cpython/rev/a11d30b8fcdf changeset: 97735:a11d30b8fcdf branch: 3.5 parent: 97734:56c10f7e0aed parent: 97732:f185917498ca user: Steve Dower date: Sun Sep 06 22:31:08 2015 -0700 summary: Null merge from 3.4 files: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 7 07:38:07 2015 From: python-checkins at python.org (steve.dower) Date: Mon, 07 Sep 2015 05:38:07 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy41IC0+IDMuNSk6?= =?utf-8?q?_Merge_from_3=2E5=2E0_branch=2E?= Message-ID: <20150907053807.66860.933@psf.io> https://hg.python.org/cpython/rev/2d01ccbc86f8 changeset: 97733:2d01ccbc86f8 branch: 3.5 parent: 97722:fd4bf05b32ba parent: 97731:7d320c3bf9c6 user: Steve Dower date: Sun Sep 06 22:27:42 2015 -0700 summary: Merge from 3.5.0 branch. files: Lib/imp.py | 8 +- Lib/test/imp_dummy.py | 3 + Lib/test/test_imp.py | 24 + Lib/test/test_time.py | 13 + Lib/test/test_warnings.py | 950 --------- Lib/test/test_warnings/__init__.py | 955 ++++++++++ Lib/test/test_warnings/__main__.py | 3 + Lib/test/test_warnings/data/import_warning.py | 3 + Lib/test/test_warnings/data/stacklevel.py | 0 Lib/warnings.py | 31 +- Lib/webbrowser.py | 120 +- Misc/NEWS | 14 +- Modules/_testmultiphase.c | 10 + Modules/timemodule.c | 16 +- Python/_warnings.c | 72 +- 15 files changed, 1144 insertions(+), 1078 deletions(-) diff --git a/Lib/imp.py b/Lib/imp.py --- a/Lib/imp.py +++ b/Lib/imp.py @@ -334,6 +334,12 @@ """ import importlib.machinery loader = importlib.machinery.ExtensionFileLoader(name, path) - return loader.load_module() + + # Issue #24748: Skip the sys.modules check in _load_module_shim; + # always load new extension + spec = importlib.machinery.ModuleSpec( + name=name, loader=loader, origin=path) + return _load(spec) + else: load_dynamic = None diff --git a/Lib/test/imp_dummy.py b/Lib/test/imp_dummy.py new file mode 100644 --- /dev/null +++ b/Lib/test/imp_dummy.py @@ -0,0 +1,3 @@ +# Fodder for test of issue24748 in test_imp + +dummy_name = True diff --git a/Lib/test/test_imp.py b/Lib/test/test_imp.py --- a/Lib/test/test_imp.py +++ b/Lib/test/test_imp.py @@ -3,6 +3,7 @@ except ImportError: _thread = None import importlib +import importlib.util import os import os.path import shutil @@ -275,6 +276,29 @@ self.skipTest("found module doesn't appear to be a C extension") imp.load_module(name, None, *found[1:]) + @requires_load_dynamic + def test_issue24748_load_module_skips_sys_modules_check(self): + name = 'test.imp_dummy' + try: + del sys.modules[name] + except KeyError: + pass + try: + module = importlib.import_module(name) + spec = importlib.util.find_spec('_testmultiphase') + module = imp.load_dynamic(name, spec.origin) + self.assertEqual(module.__name__, name) + self.assertEqual(module.__spec__.name, name) + self.assertEqual(module.__spec__.origin, spec.origin) + self.assertRaises(AttributeError, getattr, module, 'dummy_name') + self.assertEqual(module.int_const, 1969) + self.assertIs(sys.modules[name], module) + finally: + try: + del sys.modules[name] + except KeyError: + pass + @unittest.skipIf(sys.dont_write_bytecode, "test meaningful only when writing bytecode") def test_bug7732(self): diff --git a/Lib/test/test_time.py b/Lib/test/test_time.py --- a/Lib/test/test_time.py +++ b/Lib/test/test_time.py @@ -174,6 +174,19 @@ def test_strftime_bounding_check(self): self._bounds_checking(lambda tup: time.strftime('', tup)) + def test_strftime_format_check(self): + # Test that strftime does not crash on invalid format strings + # that may trigger a buffer overread. When not triggered, + # strftime may succeed or raise ValueError depending on + # the platform. + for x in [ '', 'A', '%A', '%AA' ]: + for y in range(0x0, 0x10): + for z in [ '%', 'A%', 'AA%', '%A%', 'A%A%', '%#' ]: + try: + time.strftime(x * y + z) + except ValueError: + pass + def test_default_values_for_zero(self): # Make sure that using all zeros uses the proper default # values. No test for daylight savings since strftime() does diff --git a/Lib/test/test_warnings.py b/Lib/test/test_warnings.py deleted file mode 100644 --- a/Lib/test/test_warnings.py +++ /dev/null @@ -1,950 +0,0 @@ -from contextlib import contextmanager -import linecache -import os -from io import StringIO -import sys -import unittest -from test import support -from test.support.script_helper import assert_python_ok, assert_python_failure - -from test import warning_tests - -import warnings as original_warnings - -py_warnings = support.import_fresh_module('warnings', blocked=['_warnings']) -c_warnings = support.import_fresh_module('warnings', fresh=['_warnings']) - - at contextmanager -def warnings_state(module): - """Use a specific warnings implementation in warning_tests.""" - global __warningregistry__ - for to_clear in (sys, warning_tests): - try: - to_clear.__warningregistry__.clear() - except AttributeError: - pass - try: - __warningregistry__.clear() - except NameError: - pass - original_warnings = warning_tests.warnings - original_filters = module.filters - try: - module.filters = original_filters[:] - module.simplefilter("once") - warning_tests.warnings = module - yield - finally: - warning_tests.warnings = original_warnings - module.filters = original_filters - - -class BaseTest: - - """Basic bookkeeping required for testing.""" - - def setUp(self): - self.old_unittest_module = unittest.case.warnings - # The __warningregistry__ needs to be in a pristine state for tests - # to work properly. - if '__warningregistry__' in globals(): - del globals()['__warningregistry__'] - if hasattr(warning_tests, '__warningregistry__'): - del warning_tests.__warningregistry__ - if hasattr(sys, '__warningregistry__'): - del sys.__warningregistry__ - # The 'warnings' module must be explicitly set so that the proper - # interaction between _warnings and 'warnings' can be controlled. - sys.modules['warnings'] = self.module - # Ensure that unittest.TestCase.assertWarns() uses the same warnings - # module than warnings.catch_warnings(). Otherwise, - # warnings.catch_warnings() will be unable to remove the added filter. - unittest.case.warnings = self.module - super(BaseTest, self).setUp() - - def tearDown(self): - sys.modules['warnings'] = original_warnings - unittest.case.warnings = self.old_unittest_module - super(BaseTest, self).tearDown() - -class PublicAPITests(BaseTest): - - """Ensures that the correct values are exposed in the - public API. - """ - - def test_module_all_attribute(self): - self.assertTrue(hasattr(self.module, '__all__')) - target_api = ["warn", "warn_explicit", "showwarning", - "formatwarning", "filterwarnings", "simplefilter", - "resetwarnings", "catch_warnings"] - self.assertSetEqual(set(self.module.__all__), - set(target_api)) - -class CPublicAPITests(PublicAPITests, unittest.TestCase): - module = c_warnings - -class PyPublicAPITests(PublicAPITests, unittest.TestCase): - module = py_warnings - -class FilterTests(BaseTest): - - """Testing the filtering functionality.""" - - def test_error(self): - with original_warnings.catch_warnings(module=self.module) as w: - self.module.resetwarnings() - self.module.filterwarnings("error", category=UserWarning) - self.assertRaises(UserWarning, self.module.warn, - "FilterTests.test_error") - - def test_error_after_default(self): - with original_warnings.catch_warnings(module=self.module) as w: - self.module.resetwarnings() - message = "FilterTests.test_ignore_after_default" - def f(): - self.module.warn(message, UserWarning) - f() - self.module.filterwarnings("error", category=UserWarning) - self.assertRaises(UserWarning, f) - - def test_ignore(self): - with original_warnings.catch_warnings(record=True, - module=self.module) as w: - self.module.resetwarnings() - self.module.filterwarnings("ignore", category=UserWarning) - self.module.warn("FilterTests.test_ignore", UserWarning) - self.assertEqual(len(w), 0) - - def test_ignore_after_default(self): - with original_warnings.catch_warnings(record=True, - module=self.module) as w: - self.module.resetwarnings() - message = "FilterTests.test_ignore_after_default" - def f(): - self.module.warn(message, UserWarning) - f() - self.module.filterwarnings("ignore", category=UserWarning) - f() - f() - self.assertEqual(len(w), 1) - - def test_always(self): - with original_warnings.catch_warnings(record=True, - module=self.module) as w: - self.module.resetwarnings() - self.module.filterwarnings("always", category=UserWarning) - message = "FilterTests.test_always" - self.module.warn(message, UserWarning) - self.assertTrue(message, w[-1].message) - self.module.warn(message, UserWarning) - self.assertTrue(w[-1].message, message) - - def test_always_after_default(self): - with original_warnings.catch_warnings(record=True, - module=self.module) as w: - self.module.resetwarnings() - message = "FilterTests.test_always_after_ignore" - def f(): - self.module.warn(message, UserWarning) - f() - self.assertEqual(len(w), 1) - self.assertEqual(w[-1].message.args[0], message) - f() - self.assertEqual(len(w), 1) - self.module.filterwarnings("always", category=UserWarning) - f() - self.assertEqual(len(w), 2) - self.assertEqual(w[-1].message.args[0], message) - f() - self.assertEqual(len(w), 3) - self.assertEqual(w[-1].message.args[0], message) - - def test_default(self): - with original_warnings.catch_warnings(record=True, - module=self.module) as w: - self.module.resetwarnings() - self.module.filterwarnings("default", category=UserWarning) - message = UserWarning("FilterTests.test_default") - for x in range(2): - self.module.warn(message, UserWarning) - if x == 0: - self.assertEqual(w[-1].message, message) - del w[:] - elif x == 1: - self.assertEqual(len(w), 0) - else: - raise ValueError("loop variant unhandled") - - def test_module(self): - with original_warnings.catch_warnings(record=True, - module=self.module) as w: - self.module.resetwarnings() - self.module.filterwarnings("module", category=UserWarning) - message = UserWarning("FilterTests.test_module") - self.module.warn(message, UserWarning) - self.assertEqual(w[-1].message, message) - del w[:] - self.module.warn(message, UserWarning) - self.assertEqual(len(w), 0) - - def test_once(self): - with original_warnings.catch_warnings(record=True, - module=self.module) as w: - self.module.resetwarnings() - self.module.filterwarnings("once", category=UserWarning) - message = UserWarning("FilterTests.test_once") - self.module.warn_explicit(message, UserWarning, "test_warnings.py", - 42) - self.assertEqual(w[-1].message, message) - del w[:] - self.module.warn_explicit(message, UserWarning, "test_warnings.py", - 13) - self.assertEqual(len(w), 0) - self.module.warn_explicit(message, UserWarning, "test_warnings2.py", - 42) - self.assertEqual(len(w), 0) - - def test_inheritance(self): - with original_warnings.catch_warnings(module=self.module) as w: - self.module.resetwarnings() - self.module.filterwarnings("error", category=Warning) - self.assertRaises(UserWarning, self.module.warn, - "FilterTests.test_inheritance", UserWarning) - - def test_ordering(self): - with original_warnings.catch_warnings(record=True, - module=self.module) as w: - self.module.resetwarnings() - self.module.filterwarnings("ignore", category=UserWarning) - self.module.filterwarnings("error", category=UserWarning, - append=True) - del w[:] - try: - self.module.warn("FilterTests.test_ordering", UserWarning) - except UserWarning: - self.fail("order handling for actions failed") - self.assertEqual(len(w), 0) - - def test_filterwarnings(self): - # Test filterwarnings(). - # Implicitly also tests resetwarnings(). - with original_warnings.catch_warnings(record=True, - module=self.module) as w: - self.module.filterwarnings("error", "", Warning, "", 0) - self.assertRaises(UserWarning, self.module.warn, 'convert to error') - - self.module.resetwarnings() - text = 'handle normally' - self.module.warn(text) - self.assertEqual(str(w[-1].message), text) - self.assertTrue(w[-1].category is UserWarning) - - self.module.filterwarnings("ignore", "", Warning, "", 0) - text = 'filtered out' - self.module.warn(text) - self.assertNotEqual(str(w[-1].message), text) - - self.module.resetwarnings() - self.module.filterwarnings("error", "hex*", Warning, "", 0) - self.assertRaises(UserWarning, self.module.warn, 'hex/oct') - text = 'nonmatching text' - self.module.warn(text) - self.assertEqual(str(w[-1].message), text) - self.assertTrue(w[-1].category is UserWarning) - - def test_mutate_filter_list(self): - class X: - def match(self, a): - L[:] = [] - - L = [("default",X(),UserWarning,X(),0) for i in range(2)] - with original_warnings.catch_warnings(record=True, - module=self.module) as w: - self.module.filters = L - self.module.warn_explicit(UserWarning("b"), None, "f.py", 42) - self.assertEqual(str(w[-1].message), "b") - -class CFilterTests(FilterTests, unittest.TestCase): - module = c_warnings - -class PyFilterTests(FilterTests, unittest.TestCase): - module = py_warnings - - -class WarnTests(BaseTest): - - """Test warnings.warn() and warnings.warn_explicit().""" - - def test_message(self): - with original_warnings.catch_warnings(record=True, - module=self.module) as w: - self.module.simplefilter("once") - for i in range(4): - text = 'multi %d' %i # Different text on each call. - self.module.warn(text) - self.assertEqual(str(w[-1].message), text) - self.assertTrue(w[-1].category is UserWarning) - - # Issue 3639 - def test_warn_nonstandard_types(self): - # warn() should handle non-standard types without issue. - for ob in (Warning, None, 42): - with original_warnings.catch_warnings(record=True, - module=self.module) as w: - self.module.simplefilter("once") - self.module.warn(ob) - # Don't directly compare objects since - # ``Warning() != Warning()``. - self.assertEqual(str(w[-1].message), str(UserWarning(ob))) - - def test_filename(self): - with warnings_state(self.module): - with original_warnings.catch_warnings(record=True, - module=self.module) as w: - warning_tests.inner("spam1") - self.assertEqual(os.path.basename(w[-1].filename), - "warning_tests.py") - warning_tests.outer("spam2") - self.assertEqual(os.path.basename(w[-1].filename), - "warning_tests.py") - - def test_stacklevel(self): - # Test stacklevel argument - # make sure all messages are different, so the warning won't be skipped - with warnings_state(self.module): - with original_warnings.catch_warnings(record=True, - module=self.module) as w: - warning_tests.inner("spam3", stacklevel=1) - self.assertEqual(os.path.basename(w[-1].filename), - "warning_tests.py") - warning_tests.outer("spam4", stacklevel=1) - self.assertEqual(os.path.basename(w[-1].filename), - "warning_tests.py") - - warning_tests.inner("spam5", stacklevel=2) - self.assertEqual(os.path.basename(w[-1].filename), - "test_warnings.py") - warning_tests.outer("spam6", stacklevel=2) - self.assertEqual(os.path.basename(w[-1].filename), - "warning_tests.py") - warning_tests.outer("spam6.5", stacklevel=3) - self.assertEqual(os.path.basename(w[-1].filename), - "test_warnings.py") - - warning_tests.inner("spam7", stacklevel=9999) - self.assertEqual(os.path.basename(w[-1].filename), - "sys") - - def test_missing_filename_not_main(self): - # If __file__ is not specified and __main__ is not the module name, - # then __file__ should be set to the module name. - filename = warning_tests.__file__ - try: - del warning_tests.__file__ - with warnings_state(self.module): - with original_warnings.catch_warnings(record=True, - module=self.module) as w: - warning_tests.inner("spam8", stacklevel=1) - self.assertEqual(w[-1].filename, warning_tests.__name__) - finally: - warning_tests.__file__ = filename - - @unittest.skipUnless(hasattr(sys, 'argv'), 'test needs sys.argv') - def test_missing_filename_main_with_argv(self): - # If __file__ is not specified and the caller is __main__ and sys.argv - # exists, then use sys.argv[0] as the file. - filename = warning_tests.__file__ - module_name = warning_tests.__name__ - try: - del warning_tests.__file__ - warning_tests.__name__ = '__main__' - with warnings_state(self.module): - with original_warnings.catch_warnings(record=True, - module=self.module) as w: - warning_tests.inner('spam9', stacklevel=1) - self.assertEqual(w[-1].filename, sys.argv[0]) - finally: - warning_tests.__file__ = filename - warning_tests.__name__ = module_name - - def test_missing_filename_main_without_argv(self): - # If __file__ is not specified, the caller is __main__, and sys.argv - # is not set, then '__main__' is the file name. - filename = warning_tests.__file__ - module_name = warning_tests.__name__ - argv = sys.argv - try: - del warning_tests.__file__ - warning_tests.__name__ = '__main__' - del sys.argv - with warnings_state(self.module): - with original_warnings.catch_warnings(record=True, - module=self.module) as w: - warning_tests.inner('spam10', stacklevel=1) - self.assertEqual(w[-1].filename, '__main__') - finally: - warning_tests.__file__ = filename - warning_tests.__name__ = module_name - sys.argv = argv - - def test_missing_filename_main_with_argv_empty_string(self): - # If __file__ is not specified, the caller is __main__, and sys.argv[0] - # is the empty string, then '__main__ is the file name. - # Tests issue 2743. - file_name = warning_tests.__file__ - module_name = warning_tests.__name__ - argv = sys.argv - try: - del warning_tests.__file__ - warning_tests.__name__ = '__main__' - sys.argv = [''] - with warnings_state(self.module): - with original_warnings.catch_warnings(record=True, - module=self.module) as w: - warning_tests.inner('spam11', stacklevel=1) - self.assertEqual(w[-1].filename, '__main__') - finally: - warning_tests.__file__ = file_name - warning_tests.__name__ = module_name - sys.argv = argv - - def test_warn_explicit_non_ascii_filename(self): - with original_warnings.catch_warnings(record=True, - module=self.module) as w: - self.module.resetwarnings() - self.module.filterwarnings("always", category=UserWarning) - for filename in ("nonascii\xe9\u20ac", "surrogate\udc80"): - try: - os.fsencode(filename) - except UnicodeEncodeError: - continue - self.module.warn_explicit("text", UserWarning, filename, 1) - self.assertEqual(w[-1].filename, filename) - - def test_warn_explicit_type_errors(self): - # warn_explicit() should error out gracefully if it is given objects - # of the wrong types. - # lineno is expected to be an integer. - self.assertRaises(TypeError, self.module.warn_explicit, - None, UserWarning, None, None) - # Either 'message' needs to be an instance of Warning or 'category' - # needs to be a subclass. - self.assertRaises(TypeError, self.module.warn_explicit, - None, None, None, 1) - # 'registry' must be a dict or None. - self.assertRaises((TypeError, AttributeError), - self.module.warn_explicit, - None, Warning, None, 1, registry=42) - - def test_bad_str(self): - # issue 6415 - # Warnings instance with a bad format string for __str__ should not - # trigger a bus error. - class BadStrWarning(Warning): - """Warning with a bad format string for __str__.""" - def __str__(self): - return ("A bad formatted string %(err)" % - {"err" : "there is no %(err)s"}) - - with self.assertRaises(ValueError): - self.module.warn(BadStrWarning()) - - def test_warning_classes(self): - class MyWarningClass(Warning): - pass - - class NonWarningSubclass: - pass - - # passing a non-subclass of Warning should raise a TypeError - with self.assertRaises(TypeError) as cm: - self.module.warn('bad warning category', '') - self.assertIn('category must be a Warning subclass, not ', - str(cm.exception)) - - with self.assertRaises(TypeError) as cm: - self.module.warn('bad warning category', NonWarningSubclass) - self.assertIn('category must be a Warning subclass, not ', - str(cm.exception)) - - # check that warning instances also raise a TypeError - with self.assertRaises(TypeError) as cm: - self.module.warn('bad warning category', MyWarningClass()) - self.assertIn('category must be a Warning subclass, not ', - str(cm.exception)) - - with original_warnings.catch_warnings(module=self.module): - self.module.resetwarnings() - self.module.filterwarnings('default') - with self.assertWarns(MyWarningClass) as cm: - self.module.warn('good warning category', MyWarningClass) - self.assertEqual('good warning category', str(cm.warning)) - - with self.assertWarns(UserWarning) as cm: - self.module.warn('good warning category', None) - self.assertEqual('good warning category', str(cm.warning)) - - with self.assertWarns(MyWarningClass) as cm: - self.module.warn('good warning category', MyWarningClass) - self.assertIsInstance(cm.warning, Warning) - -class CWarnTests(WarnTests, unittest.TestCase): - module = c_warnings - - # As an early adopter, we sanity check the - # test.support.import_fresh_module utility function - def test_accelerated(self): - self.assertFalse(original_warnings is self.module) - self.assertFalse(hasattr(self.module.warn, '__code__')) - -class PyWarnTests(WarnTests, unittest.TestCase): - module = py_warnings - - # As an early adopter, we sanity check the - # test.support.import_fresh_module utility function - def test_pure_python(self): - self.assertFalse(original_warnings is self.module) - self.assertTrue(hasattr(self.module.warn, '__code__')) - - -class WCmdLineTests(BaseTest): - - def test_improper_input(self): - # Uses the private _setoption() function to test the parsing - # of command-line warning arguments - with original_warnings.catch_warnings(module=self.module): - self.assertRaises(self.module._OptionError, - self.module._setoption, '1:2:3:4:5:6') - self.assertRaises(self.module._OptionError, - self.module._setoption, 'bogus::Warning') - self.assertRaises(self.module._OptionError, - self.module._setoption, 'ignore:2::4:-5') - self.module._setoption('error::Warning::0') - self.assertRaises(UserWarning, self.module.warn, 'convert to error') - - def test_improper_option(self): - # Same as above, but check that the message is printed out when - # the interpreter is executed. This also checks that options are - # actually parsed at all. - rc, out, err = assert_python_ok("-Wxxx", "-c", "pass") - self.assertIn(b"Invalid -W option ignored: invalid action: 'xxx'", err) - - def test_warnings_bootstrap(self): - # Check that the warnings module does get loaded when -W - # is used (see issue #10372 for an example of silent bootstrap failure). - rc, out, err = assert_python_ok("-Wi", "-c", - "import sys; sys.modules['warnings'].warn('foo', RuntimeWarning)") - # '-Wi' was observed - self.assertFalse(out.strip()) - self.assertNotIn(b'RuntimeWarning', err) - -class CWCmdLineTests(WCmdLineTests, unittest.TestCase): - module = c_warnings - -class PyWCmdLineTests(WCmdLineTests, unittest.TestCase): - module = py_warnings - - -class _WarningsTests(BaseTest, unittest.TestCase): - - """Tests specific to the _warnings module.""" - - module = c_warnings - - def test_filter(self): - # Everything should function even if 'filters' is not in warnings. - with original_warnings.catch_warnings(module=self.module) as w: - self.module.filterwarnings("error", "", Warning, "", 0) - self.assertRaises(UserWarning, self.module.warn, - 'convert to error') - del self.module.filters - self.assertRaises(UserWarning, self.module.warn, - 'convert to error') - - def test_onceregistry(self): - # Replacing or removing the onceregistry should be okay. - global __warningregistry__ - message = UserWarning('onceregistry test') - try: - original_registry = self.module.onceregistry - __warningregistry__ = {} - with original_warnings.catch_warnings(record=True, - module=self.module) as w: - self.module.resetwarnings() - self.module.filterwarnings("once", category=UserWarning) - self.module.warn_explicit(message, UserWarning, "file", 42) - self.assertEqual(w[-1].message, message) - del w[:] - self.module.warn_explicit(message, UserWarning, "file", 42) - self.assertEqual(len(w), 0) - # Test the resetting of onceregistry. - self.module.onceregistry = {} - __warningregistry__ = {} - self.module.warn('onceregistry test') - self.assertEqual(w[-1].message.args, message.args) - # Removal of onceregistry is okay. - del w[:] - del self.module.onceregistry - __warningregistry__ = {} - self.module.warn_explicit(message, UserWarning, "file", 42) - self.assertEqual(len(w), 0) - finally: - self.module.onceregistry = original_registry - - def test_default_action(self): - # Replacing or removing defaultaction should be okay. - message = UserWarning("defaultaction test") - original = self.module.defaultaction - try: - with original_warnings.catch_warnings(record=True, - module=self.module) as w: - self.module.resetwarnings() - registry = {} - self.module.warn_explicit(message, UserWarning, "", 42, - registry=registry) - self.assertEqual(w[-1].message, message) - self.assertEqual(len(w), 1) - # One actual registry key plus the "version" key - self.assertEqual(len(registry), 2) - self.assertIn("version", registry) - del w[:] - # Test removal. - del self.module.defaultaction - __warningregistry__ = {} - registry = {} - self.module.warn_explicit(message, UserWarning, "", 43, - registry=registry) - self.assertEqual(w[-1].message, message) - self.assertEqual(len(w), 1) - self.assertEqual(len(registry), 2) - del w[:] - # Test setting. - self.module.defaultaction = "ignore" - __warningregistry__ = {} - registry = {} - self.module.warn_explicit(message, UserWarning, "", 44, - registry=registry) - self.assertEqual(len(w), 0) - finally: - self.module.defaultaction = original - - def test_showwarning_missing(self): - # Test that showwarning() missing is okay. - text = 'del showwarning test' - with original_warnings.catch_warnings(module=self.module): - self.module.filterwarnings("always", category=UserWarning) - del self.module.showwarning - with support.captured_output('stderr') as stream: - self.module.warn(text) - result = stream.getvalue() - self.assertIn(text, result) - - def test_showwarning_not_callable(self): - with original_warnings.catch_warnings(module=self.module): - self.module.filterwarnings("always", category=UserWarning) - self.module.showwarning = print - with support.captured_output('stdout'): - self.module.warn('Warning!') - self.module.showwarning = 23 - self.assertRaises(TypeError, self.module.warn, "Warning!") - - def test_show_warning_output(self): - # With showarning() missing, make sure that output is okay. - text = 'test show_warning' - with original_warnings.catch_warnings(module=self.module): - self.module.filterwarnings("always", category=UserWarning) - del self.module.showwarning - with support.captured_output('stderr') as stream: - warning_tests.inner(text) - result = stream.getvalue() - self.assertEqual(result.count('\n'), 2, - "Too many newlines in %r" % result) - first_line, second_line = result.split('\n', 1) - expected_file = os.path.splitext(warning_tests.__file__)[0] + '.py' - first_line_parts = first_line.rsplit(':', 3) - path, line, warning_class, message = first_line_parts - line = int(line) - self.assertEqual(expected_file, path) - self.assertEqual(warning_class, ' ' + UserWarning.__name__) - self.assertEqual(message, ' ' + text) - expected_line = ' ' + linecache.getline(path, line).strip() + '\n' - assert expected_line - self.assertEqual(second_line, expected_line) - - def test_filename_none(self): - # issue #12467: race condition if a warning is emitted at shutdown - globals_dict = globals() - oldfile = globals_dict['__file__'] - try: - catch = original_warnings.catch_warnings(record=True, - module=self.module) - with catch as w: - self.module.filterwarnings("always", category=UserWarning) - globals_dict['__file__'] = None - original_warnings.warn('test', UserWarning) - self.assertTrue(len(w)) - finally: - globals_dict['__file__'] = oldfile - - def test_stderr_none(self): - rc, stdout, stderr = assert_python_ok("-c", - "import sys; sys.stderr = None; " - "import warnings; warnings.simplefilter('always'); " - "warnings.warn('Warning!')") - self.assertEqual(stdout, b'') - self.assertNotIn(b'Warning!', stderr) - self.assertNotIn(b'Error', stderr) - - -class WarningsDisplayTests(BaseTest): - - """Test the displaying of warnings and the ability to overload functions - related to displaying warnings.""" - - def test_formatwarning(self): - message = "msg" - category = Warning - file_name = os.path.splitext(warning_tests.__file__)[0] + '.py' - line_num = 3 - file_line = linecache.getline(file_name, line_num).strip() - format = "%s:%s: %s: %s\n %s\n" - expect = format % (file_name, line_num, category.__name__, message, - file_line) - self.assertEqual(expect, self.module.formatwarning(message, - category, file_name, line_num)) - # Test the 'line' argument. - file_line += " for the win!" - expect = format % (file_name, line_num, category.__name__, message, - file_line) - self.assertEqual(expect, self.module.formatwarning(message, - category, file_name, line_num, file_line)) - - def test_showwarning(self): - file_name = os.path.splitext(warning_tests.__file__)[0] + '.py' - line_num = 3 - expected_file_line = linecache.getline(file_name, line_num).strip() - message = 'msg' - category = Warning - file_object = StringIO() - expect = self.module.formatwarning(message, category, file_name, - line_num) - self.module.showwarning(message, category, file_name, line_num, - file_object) - self.assertEqual(file_object.getvalue(), expect) - # Test 'line' argument. - expected_file_line += "for the win!" - expect = self.module.formatwarning(message, category, file_name, - line_num, expected_file_line) - file_object = StringIO() - self.module.showwarning(message, category, file_name, line_num, - file_object, expected_file_line) - self.assertEqual(expect, file_object.getvalue()) - -class CWarningsDisplayTests(WarningsDisplayTests, unittest.TestCase): - module = c_warnings - -class PyWarningsDisplayTests(WarningsDisplayTests, unittest.TestCase): - module = py_warnings - - -class CatchWarningTests(BaseTest): - - """Test catch_warnings().""" - - def test_catch_warnings_restore(self): - wmod = self.module - orig_filters = wmod.filters - orig_showwarning = wmod.showwarning - # Ensure both showwarning and filters are restored when recording - with wmod.catch_warnings(module=wmod, record=True): - wmod.filters = wmod.showwarning = object() - self.assertTrue(wmod.filters is orig_filters) - self.assertTrue(wmod.showwarning is orig_showwarning) - # Same test, but with recording disabled - with wmod.catch_warnings(module=wmod, record=False): - wmod.filters = wmod.showwarning = object() - self.assertTrue(wmod.filters is orig_filters) - self.assertTrue(wmod.showwarning is orig_showwarning) - - def test_catch_warnings_recording(self): - wmod = self.module - # Ensure warnings are recorded when requested - with wmod.catch_warnings(module=wmod, record=True) as w: - self.assertEqual(w, []) - self.assertTrue(type(w) is list) - wmod.simplefilter("always") - wmod.warn("foo") - self.assertEqual(str(w[-1].message), "foo") - wmod.warn("bar") - self.assertEqual(str(w[-1].message), "bar") - self.assertEqual(str(w[0].message), "foo") - self.assertEqual(str(w[1].message), "bar") - del w[:] - self.assertEqual(w, []) - # Ensure warnings are not recorded when not requested - orig_showwarning = wmod.showwarning - with wmod.catch_warnings(module=wmod, record=False) as w: - self.assertTrue(w is None) - self.assertTrue(wmod.showwarning is orig_showwarning) - - def test_catch_warnings_reentry_guard(self): - wmod = self.module - # Ensure catch_warnings is protected against incorrect usage - x = wmod.catch_warnings(module=wmod, record=True) - self.assertRaises(RuntimeError, x.__exit__) - with x: - self.assertRaises(RuntimeError, x.__enter__) - # Same test, but with recording disabled - x = wmod.catch_warnings(module=wmod, record=False) - self.assertRaises(RuntimeError, x.__exit__) - with x: - self.assertRaises(RuntimeError, x.__enter__) - - def test_catch_warnings_defaults(self): - wmod = self.module - orig_filters = wmod.filters - orig_showwarning = wmod.showwarning - # Ensure default behaviour is not to record warnings - with wmod.catch_warnings(module=wmod) as w: - self.assertTrue(w is None) - self.assertTrue(wmod.showwarning is orig_showwarning) - self.assertTrue(wmod.filters is not orig_filters) - self.assertTrue(wmod.filters is orig_filters) - if wmod is sys.modules['warnings']: - # Ensure the default module is this one - with wmod.catch_warnings() as w: - self.assertTrue(w is None) - self.assertTrue(wmod.showwarning is orig_showwarning) - self.assertTrue(wmod.filters is not orig_filters) - self.assertTrue(wmod.filters is orig_filters) - - def test_check_warnings(self): - # Explicit tests for the test.support convenience wrapper - wmod = self.module - if wmod is not sys.modules['warnings']: - self.skipTest('module to test is not loaded warnings module') - with support.check_warnings(quiet=False) as w: - self.assertEqual(w.warnings, []) - wmod.simplefilter("always") - wmod.warn("foo") - self.assertEqual(str(w.message), "foo") - wmod.warn("bar") - self.assertEqual(str(w.message), "bar") - self.assertEqual(str(w.warnings[0].message), "foo") - self.assertEqual(str(w.warnings[1].message), "bar") - w.reset() - self.assertEqual(w.warnings, []) - - with support.check_warnings(): - # defaults to quiet=True without argument - pass - with support.check_warnings(('foo', UserWarning)): - wmod.warn("foo") - - with self.assertRaises(AssertionError): - with support.check_warnings(('', RuntimeWarning)): - # defaults to quiet=False with argument - pass - with self.assertRaises(AssertionError): - with support.check_warnings(('foo', RuntimeWarning)): - wmod.warn("foo") - -class CCatchWarningTests(CatchWarningTests, unittest.TestCase): - module = c_warnings - -class PyCatchWarningTests(CatchWarningTests, unittest.TestCase): - module = py_warnings - - -class EnvironmentVariableTests(BaseTest): - - def test_single_warning(self): - rc, stdout, stderr = assert_python_ok("-c", - "import sys; sys.stdout.write(str(sys.warnoptions))", - PYTHONWARNINGS="ignore::DeprecationWarning") - self.assertEqual(stdout, b"['ignore::DeprecationWarning']") - - def test_comma_separated_warnings(self): - rc, stdout, stderr = assert_python_ok("-c", - "import sys; sys.stdout.write(str(sys.warnoptions))", - PYTHONWARNINGS="ignore::DeprecationWarning,ignore::UnicodeWarning") - self.assertEqual(stdout, - b"['ignore::DeprecationWarning', 'ignore::UnicodeWarning']") - - def test_envvar_and_command_line(self): - rc, stdout, stderr = assert_python_ok("-Wignore::UnicodeWarning", "-c", - "import sys; sys.stdout.write(str(sys.warnoptions))", - PYTHONWARNINGS="ignore::DeprecationWarning") - self.assertEqual(stdout, - b"['ignore::DeprecationWarning', 'ignore::UnicodeWarning']") - - def test_conflicting_envvar_and_command_line(self): - rc, stdout, stderr = assert_python_failure("-Werror::DeprecationWarning", "-c", - "import sys, warnings; sys.stdout.write(str(sys.warnoptions)); " - "warnings.warn('Message', DeprecationWarning)", - PYTHONWARNINGS="default::DeprecationWarning") - self.assertEqual(stdout, - b"['default::DeprecationWarning', 'error::DeprecationWarning']") - self.assertEqual(stderr.splitlines(), - [b"Traceback (most recent call last):", - b" File \"\", line 1, in ", - b"DeprecationWarning: Message"]) - - @unittest.skipUnless(sys.getfilesystemencoding() != 'ascii', - 'requires non-ascii filesystemencoding') - def test_nonascii(self): - rc, stdout, stderr = assert_python_ok("-c", - "import sys; sys.stdout.write(str(sys.warnoptions))", - PYTHONIOENCODING="utf-8", - PYTHONWARNINGS="ignore:Deprecaci?nWarning") - self.assertEqual(stdout, - "['ignore:Deprecaci?nWarning']".encode('utf-8')) - -class CEnvironmentVariableTests(EnvironmentVariableTests, unittest.TestCase): - module = c_warnings - -class PyEnvironmentVariableTests(EnvironmentVariableTests, unittest.TestCase): - module = py_warnings - - -class BootstrapTest(unittest.TestCase): - def test_issue_8766(self): - # "import encodings" emits a warning whereas the warnings is not loaded - # or not completely loaded (warnings imports indirectly encodings by - # importing linecache) yet - with support.temp_cwd() as cwd, support.temp_cwd('encodings'): - # encodings loaded by initfsencoding() - assert_python_ok('-c', 'pass', PYTHONPATH=cwd) - - # Use -W to load warnings module at startup - assert_python_ok('-c', 'pass', '-W', 'always', PYTHONPATH=cwd) - -class FinalizationTest(unittest.TestCase): - def test_finalization(self): - # Issue #19421: warnings.warn() should not crash - # during Python finalization - code = """ -import warnings -warn = warnings.warn - -class A: - def __del__(self): - warn("test") - -a=A() - """ - rc, out, err = assert_python_ok("-c", code) - # note: "__main__" filename is not correct, it should be the name - # of the script - self.assertEqual(err, b'__main__:7: UserWarning: test') - - -def setUpModule(): - py_warnings.onceregistry.clear() - c_warnings.onceregistry.clear() - -tearDownModule = setUpModule - -if __name__ == "__main__": - unittest.main() diff --git a/Lib/test/test_warnings/__init__.py b/Lib/test/test_warnings/__init__.py new file mode 100644 --- /dev/null +++ b/Lib/test/test_warnings/__init__.py @@ -0,0 +1,955 @@ +from contextlib import contextmanager +import linecache +import os +from io import StringIO +import sys +import unittest +from test import support +from test.support.script_helper import assert_python_ok, assert_python_failure + +from test.test_warnings.data import stacklevel as warning_tests + +import warnings as original_warnings + +py_warnings = support.import_fresh_module('warnings', blocked=['_warnings']) +c_warnings = support.import_fresh_module('warnings', fresh=['_warnings']) + + at contextmanager +def warnings_state(module): + """Use a specific warnings implementation in warning_tests.""" + global __warningregistry__ + for to_clear in (sys, warning_tests): + try: + to_clear.__warningregistry__.clear() + except AttributeError: + pass + try: + __warningregistry__.clear() + except NameError: + pass + original_warnings = warning_tests.warnings + original_filters = module.filters + try: + module.filters = original_filters[:] + module.simplefilter("once") + warning_tests.warnings = module + yield + finally: + warning_tests.warnings = original_warnings + module.filters = original_filters + + +class BaseTest: + + """Basic bookkeeping required for testing.""" + + def setUp(self): + # The __warningregistry__ needs to be in a pristine state for tests + # to work properly. + if '__warningregistry__' in globals(): + del globals()['__warningregistry__'] + if hasattr(warning_tests, '__warningregistry__'): + del warning_tests.__warningregistry__ + if hasattr(sys, '__warningregistry__'): + del sys.__warningregistry__ + # The 'warnings' module must be explicitly set so that the proper + # interaction between _warnings and 'warnings' can be controlled. + sys.modules['warnings'] = self.module + super(BaseTest, self).setUp() + + def tearDown(self): + sys.modules['warnings'] = original_warnings + super(BaseTest, self).tearDown() + +class PublicAPITests(BaseTest): + + """Ensures that the correct values are exposed in the + public API. + """ + + def test_module_all_attribute(self): + self.assertTrue(hasattr(self.module, '__all__')) + target_api = ["warn", "warn_explicit", "showwarning", + "formatwarning", "filterwarnings", "simplefilter", + "resetwarnings", "catch_warnings"] + self.assertSetEqual(set(self.module.__all__), + set(target_api)) + +class CPublicAPITests(PublicAPITests, unittest.TestCase): + module = c_warnings + +class PyPublicAPITests(PublicAPITests, unittest.TestCase): + module = py_warnings + +class FilterTests(BaseTest): + + """Testing the filtering functionality.""" + + def test_error(self): + with original_warnings.catch_warnings(module=self.module) as w: + self.module.resetwarnings() + self.module.filterwarnings("error", category=UserWarning) + self.assertRaises(UserWarning, self.module.warn, + "FilterTests.test_error") + + def test_error_after_default(self): + with original_warnings.catch_warnings(module=self.module) as w: + self.module.resetwarnings() + message = "FilterTests.test_ignore_after_default" + def f(): + self.module.warn(message, UserWarning) + f() + self.module.filterwarnings("error", category=UserWarning) + self.assertRaises(UserWarning, f) + + def test_ignore(self): + with original_warnings.catch_warnings(record=True, + module=self.module) as w: + self.module.resetwarnings() + self.module.filterwarnings("ignore", category=UserWarning) + self.module.warn("FilterTests.test_ignore", UserWarning) + self.assertEqual(len(w), 0) + + def test_ignore_after_default(self): + with original_warnings.catch_warnings(record=True, + module=self.module) as w: + self.module.resetwarnings() + message = "FilterTests.test_ignore_after_default" + def f(): + self.module.warn(message, UserWarning) + f() + self.module.filterwarnings("ignore", category=UserWarning) + f() + f() + self.assertEqual(len(w), 1) + + def test_always(self): + with original_warnings.catch_warnings(record=True, + module=self.module) as w: + self.module.resetwarnings() + self.module.filterwarnings("always", category=UserWarning) + message = "FilterTests.test_always" + self.module.warn(message, UserWarning) + self.assertTrue(message, w[-1].message) + self.module.warn(message, UserWarning) + self.assertTrue(w[-1].message, message) + + def test_always_after_default(self): + with original_warnings.catch_warnings(record=True, + module=self.module) as w: + self.module.resetwarnings() + message = "FilterTests.test_always_after_ignore" + def f(): + self.module.warn(message, UserWarning) + f() + self.assertEqual(len(w), 1) + self.assertEqual(w[-1].message.args[0], message) + f() + self.assertEqual(len(w), 1) + self.module.filterwarnings("always", category=UserWarning) + f() + self.assertEqual(len(w), 2) + self.assertEqual(w[-1].message.args[0], message) + f() + self.assertEqual(len(w), 3) + self.assertEqual(w[-1].message.args[0], message) + + def test_default(self): + with original_warnings.catch_warnings(record=True, + module=self.module) as w: + self.module.resetwarnings() + self.module.filterwarnings("default", category=UserWarning) + message = UserWarning("FilterTests.test_default") + for x in range(2): + self.module.warn(message, UserWarning) + if x == 0: + self.assertEqual(w[-1].message, message) + del w[:] + elif x == 1: + self.assertEqual(len(w), 0) + else: + raise ValueError("loop variant unhandled") + + def test_module(self): + with original_warnings.catch_warnings(record=True, + module=self.module) as w: + self.module.resetwarnings() + self.module.filterwarnings("module", category=UserWarning) + message = UserWarning("FilterTests.test_module") + self.module.warn(message, UserWarning) + self.assertEqual(w[-1].message, message) + del w[:] + self.module.warn(message, UserWarning) + self.assertEqual(len(w), 0) + + def test_once(self): + with original_warnings.catch_warnings(record=True, + module=self.module) as w: + self.module.resetwarnings() + self.module.filterwarnings("once", category=UserWarning) + message = UserWarning("FilterTests.test_once") + self.module.warn_explicit(message, UserWarning, "__init__.py", + 42) + self.assertEqual(w[-1].message, message) + del w[:] + self.module.warn_explicit(message, UserWarning, "__init__.py", + 13) + self.assertEqual(len(w), 0) + self.module.warn_explicit(message, UserWarning, "test_warnings2.py", + 42) + self.assertEqual(len(w), 0) + + def test_inheritance(self): + with original_warnings.catch_warnings(module=self.module) as w: + self.module.resetwarnings() + self.module.filterwarnings("error", category=Warning) + self.assertRaises(UserWarning, self.module.warn, + "FilterTests.test_inheritance", UserWarning) + + def test_ordering(self): + with original_warnings.catch_warnings(record=True, + module=self.module) as w: + self.module.resetwarnings() + self.module.filterwarnings("ignore", category=UserWarning) + self.module.filterwarnings("error", category=UserWarning, + append=True) + del w[:] + try: + self.module.warn("FilterTests.test_ordering", UserWarning) + except UserWarning: + self.fail("order handling for actions failed") + self.assertEqual(len(w), 0) + + def test_filterwarnings(self): + # Test filterwarnings(). + # Implicitly also tests resetwarnings(). + with original_warnings.catch_warnings(record=True, + module=self.module) as w: + self.module.filterwarnings("error", "", Warning, "", 0) + self.assertRaises(UserWarning, self.module.warn, 'convert to error') + + self.module.resetwarnings() + text = 'handle normally' + self.module.warn(text) + self.assertEqual(str(w[-1].message), text) + self.assertTrue(w[-1].category is UserWarning) + + self.module.filterwarnings("ignore", "", Warning, "", 0) + text = 'filtered out' + self.module.warn(text) + self.assertNotEqual(str(w[-1].message), text) + + self.module.resetwarnings() + self.module.filterwarnings("error", "hex*", Warning, "", 0) + self.assertRaises(UserWarning, self.module.warn, 'hex/oct') + text = 'nonmatching text' + self.module.warn(text) + self.assertEqual(str(w[-1].message), text) + self.assertTrue(w[-1].category is UserWarning) + + def test_mutate_filter_list(self): + class X: + def match(self, a): + L[:] = [] + + L = [("default",X(),UserWarning,X(),0) for i in range(2)] + with original_warnings.catch_warnings(record=True, + module=self.module) as w: + self.module.filters = L + self.module.warn_explicit(UserWarning("b"), None, "f.py", 42) + self.assertEqual(str(w[-1].message), "b") + +class CFilterTests(FilterTests, unittest.TestCase): + module = c_warnings + +class PyFilterTests(FilterTests, unittest.TestCase): + module = py_warnings + + +class WarnTests(BaseTest): + + """Test warnings.warn() and warnings.warn_explicit().""" + + def test_message(self): + with original_warnings.catch_warnings(record=True, + module=self.module) as w: + self.module.simplefilter("once") + for i in range(4): + text = 'multi %d' %i # Different text on each call. + self.module.warn(text) + self.assertEqual(str(w[-1].message), text) + self.assertTrue(w[-1].category is UserWarning) + + # Issue 3639 + def test_warn_nonstandard_types(self): + # warn() should handle non-standard types without issue. + for ob in (Warning, None, 42): + with original_warnings.catch_warnings(record=True, + module=self.module) as w: + self.module.simplefilter("once") + self.module.warn(ob) + # Don't directly compare objects since + # ``Warning() != Warning()``. + self.assertEqual(str(w[-1].message), str(UserWarning(ob))) + + def test_filename(self): + with warnings_state(self.module): + with original_warnings.catch_warnings(record=True, + module=self.module) as w: + warning_tests.inner("spam1") + self.assertEqual(os.path.basename(w[-1].filename), + "stacklevel.py") + warning_tests.outer("spam2") + self.assertEqual(os.path.basename(w[-1].filename), + "stacklevel.py") + + def test_stacklevel(self): + # Test stacklevel argument + # make sure all messages are different, so the warning won't be skipped + with warnings_state(self.module): + with original_warnings.catch_warnings(record=True, + module=self.module) as w: + warning_tests.inner("spam3", stacklevel=1) + self.assertEqual(os.path.basename(w[-1].filename), + "stacklevel.py") + warning_tests.outer("spam4", stacklevel=1) + self.assertEqual(os.path.basename(w[-1].filename), + "stacklevel.py") + + warning_tests.inner("spam5", stacklevel=2) + self.assertEqual(os.path.basename(w[-1].filename), + "__init__.py") + warning_tests.outer("spam6", stacklevel=2) + self.assertEqual(os.path.basename(w[-1].filename), + "stacklevel.py") + warning_tests.outer("spam6.5", stacklevel=3) + self.assertEqual(os.path.basename(w[-1].filename), + "__init__.py") + + warning_tests.inner("spam7", stacklevel=9999) + self.assertEqual(os.path.basename(w[-1].filename), + "sys") + + def test_stacklevel_import(self): + # Issue #24305: With stacklevel=2, module-level warnings should work. + support.unload('test.test_warnings.data.import_warning') + with warnings_state(self.module): + with original_warnings.catch_warnings(record=True, + module=self.module) as w: + self.module.simplefilter('always') + import test.test_warnings.data.import_warning + self.assertEqual(len(w), 1) + self.assertEqual(w[0].filename, __file__) + + def test_missing_filename_not_main(self): + # If __file__ is not specified and __main__ is not the module name, + # then __file__ should be set to the module name. + filename = warning_tests.__file__ + try: + del warning_tests.__file__ + with warnings_state(self.module): + with original_warnings.catch_warnings(record=True, + module=self.module) as w: + warning_tests.inner("spam8", stacklevel=1) + self.assertEqual(w[-1].filename, warning_tests.__name__) + finally: + warning_tests.__file__ = filename + + @unittest.skipUnless(hasattr(sys, 'argv'), 'test needs sys.argv') + def test_missing_filename_main_with_argv(self): + # If __file__ is not specified and the caller is __main__ and sys.argv + # exists, then use sys.argv[0] as the file. + filename = warning_tests.__file__ + module_name = warning_tests.__name__ + try: + del warning_tests.__file__ + warning_tests.__name__ = '__main__' + with warnings_state(self.module): + with original_warnings.catch_warnings(record=True, + module=self.module) as w: + warning_tests.inner('spam9', stacklevel=1) + self.assertEqual(w[-1].filename, sys.argv[0]) + finally: + warning_tests.__file__ = filename + warning_tests.__name__ = module_name + + def test_missing_filename_main_without_argv(self): + # If __file__ is not specified, the caller is __main__, and sys.argv + # is not set, then '__main__' is the file name. + filename = warning_tests.__file__ + module_name = warning_tests.__name__ + argv = sys.argv + try: + del warning_tests.__file__ + warning_tests.__name__ = '__main__' + del sys.argv + with warnings_state(self.module): + with original_warnings.catch_warnings(record=True, + module=self.module) as w: + warning_tests.inner('spam10', stacklevel=1) + self.assertEqual(w[-1].filename, '__main__') + finally: + warning_tests.__file__ = filename + warning_tests.__name__ = module_name + sys.argv = argv + + def test_missing_filename_main_with_argv_empty_string(self): + # If __file__ is not specified, the caller is __main__, and sys.argv[0] + # is the empty string, then '__main__ is the file name. + # Tests issue 2743. + file_name = warning_tests.__file__ + module_name = warning_tests.__name__ + argv = sys.argv + try: + del warning_tests.__file__ + warning_tests.__name__ = '__main__' + sys.argv = [''] + with warnings_state(self.module): + with original_warnings.catch_warnings(record=True, + module=self.module) as w: + warning_tests.inner('spam11', stacklevel=1) + self.assertEqual(w[-1].filename, '__main__') + finally: + warning_tests.__file__ = file_name + warning_tests.__name__ = module_name + sys.argv = argv + + def test_warn_explicit_non_ascii_filename(self): + with original_warnings.catch_warnings(record=True, + module=self.module) as w: + self.module.resetwarnings() + self.module.filterwarnings("always", category=UserWarning) + for filename in ("nonascii\xe9\u20ac", "surrogate\udc80"): + try: + os.fsencode(filename) + except UnicodeEncodeError: + continue + self.module.warn_explicit("text", UserWarning, filename, 1) + self.assertEqual(w[-1].filename, filename) + + def test_warn_explicit_type_errors(self): + # warn_explicit() should error out gracefully if it is given objects + # of the wrong types. + # lineno is expected to be an integer. + self.assertRaises(TypeError, self.module.warn_explicit, + None, UserWarning, None, None) + # Either 'message' needs to be an instance of Warning or 'category' + # needs to be a subclass. + self.assertRaises(TypeError, self.module.warn_explicit, + None, None, None, 1) + # 'registry' must be a dict or None. + self.assertRaises((TypeError, AttributeError), + self.module.warn_explicit, + None, Warning, None, 1, registry=42) + + def test_bad_str(self): + # issue 6415 + # Warnings instance with a bad format string for __str__ should not + # trigger a bus error. + class BadStrWarning(Warning): + """Warning with a bad format string for __str__.""" + def __str__(self): + return ("A bad formatted string %(err)" % + {"err" : "there is no %(err)s"}) + + with self.assertRaises(ValueError): + self.module.warn(BadStrWarning()) + + def test_warning_classes(self): + class MyWarningClass(Warning): + pass + + class NonWarningSubclass: + pass + + # passing a non-subclass of Warning should raise a TypeError + with self.assertRaises(TypeError) as cm: + self.module.warn('bad warning category', '') + self.assertIn('category must be a Warning subclass, not ', + str(cm.exception)) + + with self.assertRaises(TypeError) as cm: + self.module.warn('bad warning category', NonWarningSubclass) + self.assertIn('category must be a Warning subclass, not ', + str(cm.exception)) + + # check that warning instances also raise a TypeError + with self.assertRaises(TypeError) as cm: + self.module.warn('bad warning category', MyWarningClass()) + self.assertIn('category must be a Warning subclass, not ', + str(cm.exception)) + + with original_warnings.catch_warnings(module=self.module): + self.module.resetwarnings() + self.module.filterwarnings('default') + with self.assertWarns(MyWarningClass) as cm: + self.module.warn('good warning category', MyWarningClass) + self.assertEqual('good warning category', str(cm.warning)) + + with self.assertWarns(UserWarning) as cm: + self.module.warn('good warning category', None) + self.assertEqual('good warning category', str(cm.warning)) + + with self.assertWarns(MyWarningClass) as cm: + self.module.warn('good warning category', MyWarningClass) + self.assertIsInstance(cm.warning, Warning) + +class CWarnTests(WarnTests, unittest.TestCase): + module = c_warnings + + # As an early adopter, we sanity check the + # test.support.import_fresh_module utility function + def test_accelerated(self): + self.assertFalse(original_warnings is self.module) + self.assertFalse(hasattr(self.module.warn, '__code__')) + +class PyWarnTests(WarnTests, unittest.TestCase): + module = py_warnings + + # As an early adopter, we sanity check the + # test.support.import_fresh_module utility function + def test_pure_python(self): + self.assertFalse(original_warnings is self.module) + self.assertTrue(hasattr(self.module.warn, '__code__')) + + +class WCmdLineTests(BaseTest): + + def test_improper_input(self): + # Uses the private _setoption() function to test the parsing + # of command-line warning arguments + with original_warnings.catch_warnings(module=self.module): + self.assertRaises(self.module._OptionError, + self.module._setoption, '1:2:3:4:5:6') + self.assertRaises(self.module._OptionError, + self.module._setoption, 'bogus::Warning') + self.assertRaises(self.module._OptionError, + self.module._setoption, 'ignore:2::4:-5') + self.module._setoption('error::Warning::0') + self.assertRaises(UserWarning, self.module.warn, 'convert to error') + + def test_improper_option(self): + # Same as above, but check that the message is printed out when + # the interpreter is executed. This also checks that options are + # actually parsed at all. + rc, out, err = assert_python_ok("-Wxxx", "-c", "pass") + self.assertIn(b"Invalid -W option ignored: invalid action: 'xxx'", err) + + def test_warnings_bootstrap(self): + # Check that the warnings module does get loaded when -W + # is used (see issue #10372 for an example of silent bootstrap failure). + rc, out, err = assert_python_ok("-Wi", "-c", + "import sys; sys.modules['warnings'].warn('foo', RuntimeWarning)") + # '-Wi' was observed + self.assertFalse(out.strip()) + self.assertNotIn(b'RuntimeWarning', err) + +class CWCmdLineTests(WCmdLineTests, unittest.TestCase): + module = c_warnings + +class PyWCmdLineTests(WCmdLineTests, unittest.TestCase): + module = py_warnings + + +class _WarningsTests(BaseTest, unittest.TestCase): + + """Tests specific to the _warnings module.""" + + module = c_warnings + + def test_filter(self): + # Everything should function even if 'filters' is not in warnings. + with original_warnings.catch_warnings(module=self.module) as w: + self.module.filterwarnings("error", "", Warning, "", 0) + self.assertRaises(UserWarning, self.module.warn, + 'convert to error') + del self.module.filters + self.assertRaises(UserWarning, self.module.warn, + 'convert to error') + + def test_onceregistry(self): + # Replacing or removing the onceregistry should be okay. + global __warningregistry__ + message = UserWarning('onceregistry test') + try: + original_registry = self.module.onceregistry + __warningregistry__ = {} + with original_warnings.catch_warnings(record=True, + module=self.module) as w: + self.module.resetwarnings() + self.module.filterwarnings("once", category=UserWarning) + self.module.warn_explicit(message, UserWarning, "file", 42) + self.assertEqual(w[-1].message, message) + del w[:] + self.module.warn_explicit(message, UserWarning, "file", 42) + self.assertEqual(len(w), 0) + # Test the resetting of onceregistry. + self.module.onceregistry = {} + __warningregistry__ = {} + self.module.warn('onceregistry test') + self.assertEqual(w[-1].message.args, message.args) + # Removal of onceregistry is okay. + del w[:] + del self.module.onceregistry + __warningregistry__ = {} + self.module.warn_explicit(message, UserWarning, "file", 42) + self.assertEqual(len(w), 0) + finally: + self.module.onceregistry = original_registry + + def test_default_action(self): + # Replacing or removing defaultaction should be okay. + message = UserWarning("defaultaction test") + original = self.module.defaultaction + try: + with original_warnings.catch_warnings(record=True, + module=self.module) as w: + self.module.resetwarnings() + registry = {} + self.module.warn_explicit(message, UserWarning, "", 42, + registry=registry) + self.assertEqual(w[-1].message, message) + self.assertEqual(len(w), 1) + # One actual registry key plus the "version" key + self.assertEqual(len(registry), 2) + self.assertIn("version", registry) + del w[:] + # Test removal. + del self.module.defaultaction + __warningregistry__ = {} + registry = {} + self.module.warn_explicit(message, UserWarning, "", 43, + registry=registry) + self.assertEqual(w[-1].message, message) + self.assertEqual(len(w), 1) + self.assertEqual(len(registry), 2) + del w[:] + # Test setting. + self.module.defaultaction = "ignore" + __warningregistry__ = {} + registry = {} + self.module.warn_explicit(message, UserWarning, "", 44, + registry=registry) + self.assertEqual(len(w), 0) + finally: + self.module.defaultaction = original + + def test_showwarning_missing(self): + # Test that showwarning() missing is okay. + text = 'del showwarning test' + with original_warnings.catch_warnings(module=self.module): + self.module.filterwarnings("always", category=UserWarning) + del self.module.showwarning + with support.captured_output('stderr') as stream: + self.module.warn(text) + result = stream.getvalue() + self.assertIn(text, result) + + def test_showwarning_not_callable(self): + with original_warnings.catch_warnings(module=self.module): + self.module.filterwarnings("always", category=UserWarning) + self.module.showwarning = print + with support.captured_output('stdout'): + self.module.warn('Warning!') + self.module.showwarning = 23 + self.assertRaises(TypeError, self.module.warn, "Warning!") + + def test_show_warning_output(self): + # With showarning() missing, make sure that output is okay. + text = 'test show_warning' + with original_warnings.catch_warnings(module=self.module): + self.module.filterwarnings("always", category=UserWarning) + del self.module.showwarning + with support.captured_output('stderr') as stream: + warning_tests.inner(text) + result = stream.getvalue() + self.assertEqual(result.count('\n'), 2, + "Too many newlines in %r" % result) + first_line, second_line = result.split('\n', 1) + expected_file = os.path.splitext(warning_tests.__file__)[0] + '.py' + first_line_parts = first_line.rsplit(':', 3) + path, line, warning_class, message = first_line_parts + line = int(line) + self.assertEqual(expected_file, path) + self.assertEqual(warning_class, ' ' + UserWarning.__name__) + self.assertEqual(message, ' ' + text) + expected_line = ' ' + linecache.getline(path, line).strip() + '\n' + assert expected_line + self.assertEqual(second_line, expected_line) + + def test_filename_none(self): + # issue #12467: race condition if a warning is emitted at shutdown + globals_dict = globals() + oldfile = globals_dict['__file__'] + try: + catch = original_warnings.catch_warnings(record=True, + module=self.module) + with catch as w: + self.module.filterwarnings("always", category=UserWarning) + globals_dict['__file__'] = None + original_warnings.warn('test', UserWarning) + self.assertTrue(len(w)) + finally: + globals_dict['__file__'] = oldfile + + def test_stderr_none(self): + rc, stdout, stderr = assert_python_ok("-c", + "import sys; sys.stderr = None; " + "import warnings; warnings.simplefilter('always'); " + "warnings.warn('Warning!')") + self.assertEqual(stdout, b'') + self.assertNotIn(b'Warning!', stderr) + self.assertNotIn(b'Error', stderr) + + +class WarningsDisplayTests(BaseTest): + + """Test the displaying of warnings and the ability to overload functions + related to displaying warnings.""" + + def test_formatwarning(self): + message = "msg" + category = Warning + file_name = os.path.splitext(warning_tests.__file__)[0] + '.py' + line_num = 3 + file_line = linecache.getline(file_name, line_num).strip() + format = "%s:%s: %s: %s\n %s\n" + expect = format % (file_name, line_num, category.__name__, message, + file_line) + self.assertEqual(expect, self.module.formatwarning(message, + category, file_name, line_num)) + # Test the 'line' argument. + file_line += " for the win!" + expect = format % (file_name, line_num, category.__name__, message, + file_line) + self.assertEqual(expect, self.module.formatwarning(message, + category, file_name, line_num, file_line)) + + def test_showwarning(self): + file_name = os.path.splitext(warning_tests.__file__)[0] + '.py' + line_num = 3 + expected_file_line = linecache.getline(file_name, line_num).strip() + message = 'msg' + category = Warning + file_object = StringIO() + expect = self.module.formatwarning(message, category, file_name, + line_num) + self.module.showwarning(message, category, file_name, line_num, + file_object) + self.assertEqual(file_object.getvalue(), expect) + # Test 'line' argument. + expected_file_line += "for the win!" + expect = self.module.formatwarning(message, category, file_name, + line_num, expected_file_line) + file_object = StringIO() + self.module.showwarning(message, category, file_name, line_num, + file_object, expected_file_line) + self.assertEqual(expect, file_object.getvalue()) + +class CWarningsDisplayTests(WarningsDisplayTests, unittest.TestCase): + module = c_warnings + +class PyWarningsDisplayTests(WarningsDisplayTests, unittest.TestCase): + module = py_warnings + + +class CatchWarningTests(BaseTest): + + """Test catch_warnings().""" + + def test_catch_warnings_restore(self): + wmod = self.module + orig_filters = wmod.filters + orig_showwarning = wmod.showwarning + # Ensure both showwarning and filters are restored when recording + with wmod.catch_warnings(module=wmod, record=True): + wmod.filters = wmod.showwarning = object() + self.assertTrue(wmod.filters is orig_filters) + self.assertTrue(wmod.showwarning is orig_showwarning) + # Same test, but with recording disabled + with wmod.catch_warnings(module=wmod, record=False): + wmod.filters = wmod.showwarning = object() + self.assertTrue(wmod.filters is orig_filters) + self.assertTrue(wmod.showwarning is orig_showwarning) + + def test_catch_warnings_recording(self): + wmod = self.module + # Ensure warnings are recorded when requested + with wmod.catch_warnings(module=wmod, record=True) as w: + self.assertEqual(w, []) + self.assertTrue(type(w) is list) + wmod.simplefilter("always") + wmod.warn("foo") + self.assertEqual(str(w[-1].message), "foo") + wmod.warn("bar") + self.assertEqual(str(w[-1].message), "bar") + self.assertEqual(str(w[0].message), "foo") + self.assertEqual(str(w[1].message), "bar") + del w[:] + self.assertEqual(w, []) + # Ensure warnings are not recorded when not requested + orig_showwarning = wmod.showwarning + with wmod.catch_warnings(module=wmod, record=False) as w: + self.assertTrue(w is None) + self.assertTrue(wmod.showwarning is orig_showwarning) + + def test_catch_warnings_reentry_guard(self): + wmod = self.module + # Ensure catch_warnings is protected against incorrect usage + x = wmod.catch_warnings(module=wmod, record=True) + self.assertRaises(RuntimeError, x.__exit__) + with x: + self.assertRaises(RuntimeError, x.__enter__) + # Same test, but with recording disabled + x = wmod.catch_warnings(module=wmod, record=False) + self.assertRaises(RuntimeError, x.__exit__) + with x: + self.assertRaises(RuntimeError, x.__enter__) + + def test_catch_warnings_defaults(self): + wmod = self.module + orig_filters = wmod.filters + orig_showwarning = wmod.showwarning + # Ensure default behaviour is not to record warnings + with wmod.catch_warnings(module=wmod) as w: + self.assertTrue(w is None) + self.assertTrue(wmod.showwarning is orig_showwarning) + self.assertTrue(wmod.filters is not orig_filters) + self.assertTrue(wmod.filters is orig_filters) + if wmod is sys.modules['warnings']: + # Ensure the default module is this one + with wmod.catch_warnings() as w: + self.assertTrue(w is None) + self.assertTrue(wmod.showwarning is orig_showwarning) + self.assertTrue(wmod.filters is not orig_filters) + self.assertTrue(wmod.filters is orig_filters) + + def test_check_warnings(self): + # Explicit tests for the test.support convenience wrapper + wmod = self.module + if wmod is not sys.modules['warnings']: + self.skipTest('module to test is not loaded warnings module') + with support.check_warnings(quiet=False) as w: + self.assertEqual(w.warnings, []) + wmod.simplefilter("always") + wmod.warn("foo") + self.assertEqual(str(w.message), "foo") + wmod.warn("bar") + self.assertEqual(str(w.message), "bar") + self.assertEqual(str(w.warnings[0].message), "foo") + self.assertEqual(str(w.warnings[1].message), "bar") + w.reset() + self.assertEqual(w.warnings, []) + + with support.check_warnings(): + # defaults to quiet=True without argument + pass + with support.check_warnings(('foo', UserWarning)): + wmod.warn("foo") + + with self.assertRaises(AssertionError): + with support.check_warnings(('', RuntimeWarning)): + # defaults to quiet=False with argument + pass + with self.assertRaises(AssertionError): + with support.check_warnings(('foo', RuntimeWarning)): + wmod.warn("foo") + +class CCatchWarningTests(CatchWarningTests, unittest.TestCase): + module = c_warnings + +class PyCatchWarningTests(CatchWarningTests, unittest.TestCase): + module = py_warnings + + +class EnvironmentVariableTests(BaseTest): + + def test_single_warning(self): + rc, stdout, stderr = assert_python_ok("-c", + "import sys; sys.stdout.write(str(sys.warnoptions))", + PYTHONWARNINGS="ignore::DeprecationWarning") + self.assertEqual(stdout, b"['ignore::DeprecationWarning']") + + def test_comma_separated_warnings(self): + rc, stdout, stderr = assert_python_ok("-c", + "import sys; sys.stdout.write(str(sys.warnoptions))", + PYTHONWARNINGS="ignore::DeprecationWarning,ignore::UnicodeWarning") + self.assertEqual(stdout, + b"['ignore::DeprecationWarning', 'ignore::UnicodeWarning']") + + def test_envvar_and_command_line(self): + rc, stdout, stderr = assert_python_ok("-Wignore::UnicodeWarning", "-c", + "import sys; sys.stdout.write(str(sys.warnoptions))", + PYTHONWARNINGS="ignore::DeprecationWarning") + self.assertEqual(stdout, + b"['ignore::DeprecationWarning', 'ignore::UnicodeWarning']") + + def test_conflicting_envvar_and_command_line(self): + rc, stdout, stderr = assert_python_failure("-Werror::DeprecationWarning", "-c", + "import sys, warnings; sys.stdout.write(str(sys.warnoptions)); " + "warnings.warn('Message', DeprecationWarning)", + PYTHONWARNINGS="default::DeprecationWarning") + self.assertEqual(stdout, + b"['default::DeprecationWarning', 'error::DeprecationWarning']") + self.assertEqual(stderr.splitlines(), + [b"Traceback (most recent call last):", + b" File \"\", line 1, in ", + b"DeprecationWarning: Message"]) + + @unittest.skipUnless(sys.getfilesystemencoding() != 'ascii', + 'requires non-ascii filesystemencoding') + def test_nonascii(self): + rc, stdout, stderr = assert_python_ok("-c", + "import sys; sys.stdout.write(str(sys.warnoptions))", + PYTHONIOENCODING="utf-8", + PYTHONWARNINGS="ignore:Deprecaci?nWarning") + self.assertEqual(stdout, + "['ignore:Deprecaci?nWarning']".encode('utf-8')) + +class CEnvironmentVariableTests(EnvironmentVariableTests, unittest.TestCase): + module = c_warnings + +class PyEnvironmentVariableTests(EnvironmentVariableTests, unittest.TestCase): + module = py_warnings + + +class BootstrapTest(unittest.TestCase): + def test_issue_8766(self): + # "import encodings" emits a warning whereas the warnings is not loaded + # or not completely loaded (warnings imports indirectly encodings by + # importing linecache) yet + with support.temp_cwd() as cwd, support.temp_cwd('encodings'): + # encodings loaded by initfsencoding() + assert_python_ok('-c', 'pass', PYTHONPATH=cwd) + + # Use -W to load warnings module at startup + assert_python_ok('-c', 'pass', '-W', 'always', PYTHONPATH=cwd) + +class FinalizationTest(unittest.TestCase): + def test_finalization(self): + # Issue #19421: warnings.warn() should not crash + # during Python finalization + code = """ +import warnings +warn = warnings.warn + +class A: + def __del__(self): + warn("test") + +a=A() + """ + rc, out, err = assert_python_ok("-c", code) + # note: "__main__" filename is not correct, it should be the name + # of the script + self.assertEqual(err, b'__main__:7: UserWarning: test') + + +def setUpModule(): + py_warnings.onceregistry.clear() + c_warnings.onceregistry.clear() + +tearDownModule = setUpModule + +if __name__ == "__main__": + unittest.main() diff --git a/Lib/test/test_warnings/__main__.py b/Lib/test/test_warnings/__main__.py new file mode 100644 --- /dev/null +++ b/Lib/test/test_warnings/__main__.py @@ -0,0 +1,3 @@ +import unittest + +unittest.main('test.test_warnings') diff --git a/Lib/test/test_warnings/data/import_warning.py b/Lib/test/test_warnings/data/import_warning.py new file mode 100644 --- /dev/null +++ b/Lib/test/test_warnings/data/import_warning.py @@ -0,0 +1,3 @@ +import warnings + +warnings.warn('module-level warning', DeprecationWarning, stacklevel=2) \ No newline at end of file diff --git a/Lib/test/warning_tests.py b/Lib/test/test_warnings/data/stacklevel.py rename from Lib/test/warning_tests.py rename to Lib/test/test_warnings/data/stacklevel.py diff --git a/Lib/warnings.py b/Lib/warnings.py --- a/Lib/warnings.py +++ b/Lib/warnings.py @@ -160,6 +160,20 @@ return cat +def _is_internal_frame(frame): + """Signal whether the frame is an internal CPython implementation detail.""" + filename = frame.f_code.co_filename + return 'importlib' in filename and '_bootstrap' in filename + + +def _next_external_frame(frame): + """Find the next frame that doesn't involve CPython internals.""" + frame = frame.f_back + while frame is not None and _is_internal_frame(frame): + frame = frame.f_back + return frame + + # Code typically replaced by _warnings def warn(message, category=None, stacklevel=1): """Issue a warning, or maybe ignore it or raise an exception.""" @@ -174,13 +188,23 @@ "not '{:s}'".format(type(category).__name__)) # Get context information try: - caller = sys._getframe(stacklevel) + if stacklevel <= 1 or _is_internal_frame(sys._getframe(1)): + # If frame is too small to care or if the warning originated in + # internal code, then do not try to hide any frames. + frame = sys._getframe(stacklevel) + else: + frame = sys._getframe(1) + # Look for one frame less since the above line starts us off. + for x in range(stacklevel-1): + frame = _next_external_frame(frame) + if frame is None: + raise ValueError except ValueError: globals = sys.__dict__ lineno = 1 else: - globals = caller.f_globals - lineno = caller.f_lineno + globals = frame.f_globals + lineno = frame.f_lineno if '__name__' in globals: module = globals['__name__'] else: @@ -374,7 +398,6 @@ defaultaction = _defaultaction onceregistry = _onceregistry _warnings_defaults = True - except ImportError: filters = [] defaultaction = "default" diff --git a/Lib/webbrowser.py b/Lib/webbrowser.py --- a/Lib/webbrowser.py +++ b/Lib/webbrowser.py @@ -495,23 +495,10 @@ # if sys.platform[:3] == "win": - class WindowsDefault(BaseBrowser): - # Windows Default opening arguments. - - cmd = "start" - newwindow = "" - newtab = "" - def open(self, url, new=0, autoraise=True): - # Format the command for optional arguments and add the url. - if new == 1: - self.cmd += " " + self.newwindow - elif new == 2: - self.cmd += " " + self.newtab - self.cmd += " " + url try: - subprocess.call(self.cmd, shell=True) + os.startfile(url) except OSError: # [Error 22] No application is associated with the specified # file for this operation: '' @@ -519,108 +506,19 @@ else: return True - - # Windows Sub-Classes for commonly used browsers. - - class InternetExplorer(WindowsDefault): - """Launcher class for Internet Explorer browser""" - - cmd = "start iexplore.exe" - newwindow = "" - newtab = "" - - - class WinChrome(WindowsDefault): - """Launcher class for windows specific Google Chrome browser""" - - cmd = "start chrome.exe" - newwindow = "-new-window" - newtab = "-new-tab" - - - class WinFirefox(WindowsDefault): - """Launcher class for windows specific Firefox browser""" - - cmd = "start firefox.exe" - newwindow = "-new-window" - newtab = "-new-tab" - - - class WinOpera(WindowsDefault): - """Launcher class for windows specific Opera browser""" - - cmd = "start opera" - newwindow = "" - newtab = "" - - - class WinSeaMonkey(WindowsDefault): - """Launcher class for windows specific SeaMonkey browser""" - - cmd = "start seamonkey" - newwinow = "" - newtab = "" - - _tryorder = [] _browsers = {} - # First try to use the default Windows browser. + # First try to use the default Windows browser register("windows-default", WindowsDefault) - def find_windows_browsers(): - """ Access the windows registry to determine - what browsers are on the system. - """ - - import winreg - HKLM = winreg.HKEY_LOCAL_MACHINE - subkey = r'Software\Clients\StartMenuInternet' - read32 = winreg.KEY_READ | winreg.KEY_WOW64_32KEY - read64 = winreg.KEY_READ | winreg.KEY_WOW64_64KEY - key32 = winreg.OpenKey(HKLM, subkey, access=read32) - key64 = winreg.OpenKey(HKLM, subkey, access=read64) - - # Return a list of browsers found in the registry - # Check if there are any different browsers in the - # 32 bit location instead of the 64 bit location. - browsers = [] - i = 0 - while True: - try: - browsers.append(winreg.EnumKey(key32, i)) - except EnvironmentError: - break - i += 1 - - i = 0 - while True: - try: - browsers.append(winreg.EnumKey(key64, i)) - except EnvironmentError: - break - i += 1 - - winreg.CloseKey(key32) - winreg.CloseKey(key64) - - return browsers - - # Detect some common windows browsers - for browser in find_windows_browsers(): - browser = browser.lower() - if "iexplore" in browser: - register("iexplore", None, InternetExplorer("iexplore")) - elif "chrome" in browser: - register("chrome", None, WinChrome("chrome")) - elif "firefox" in browser: - register("firefox", None, WinFirefox("firefox")) - elif "opera" in browser: - register("opera", None, WinOpera("opera")) - elif "seamonkey" in browser: - register("seamonkey", None, WinSeaMonkey("seamonkey")) - else: - register(browser, None, WindowsDefault(browser)) + # Detect some common Windows browsers, fallback to IE + iexplore = os.path.join(os.environ.get("PROGRAMFILES", "C:\\Program Files"), + "Internet Explorer\\IEXPLORE.EXE") + for browser in ("firefox", "firebird", "seamonkey", "mozilla", + "netscape", "opera", iexplore): + if shutil.which(browser): + register(browser, None, BackgroundBrowser(browser)) # # Platform support for MacOS diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -1,4 +1,4 @@ -?+++++++++++ ++++++++++++ Python News +++++++++++ @@ -88,6 +88,9 @@ Core and Builtins ----------------- +- Issue #24305: Prevent import subsystem stack frames from being counted + by the warnings.warn(stacklevel=) parameter. + - Issue #24912: Prevent __class__ assignment to immutable built-in objects. - Issue #24975: Fix AST compilation for PEP 448 syntax. @@ -95,9 +98,15 @@ Library ------- +- Issue #24917: time_strftime() buffer over-read. - Issue #23144: Make sure that HTMLParser.feed() returns all the data, even when convert_charrefs is True. +- Issue #24748: To resolve a compatibility problem found with py2exe and + pywin32, imp.load_dynamic() once again ignores previously loaded modules + to support Python modules replacing themselves with extension modules. + Patch by Petr Viktorin. + - Issue #24635: Fixed a bug in typing.py where isinstance([], typing.Iterable) would return True once, then False on subsequent calls. @@ -386,9 +395,6 @@ - Issue #14373: C implementation of functools.lru_cache() now can be used with methods. -- Issue #8232: webbrowser support incomplete on Windows. Patch by Brandon - Milam - - Issue #24347: Set KeyError if PyDict_GetItemWithError returns NULL. - Issue #24348: Drop superfluous incref/decref. diff --git a/Modules/_testmultiphase.c b/Modules/_testmultiphase.c --- a/Modules/_testmultiphase.c +++ b/Modules/_testmultiphase.c @@ -582,3 +582,13 @@ { return PyModuleDef_Init(&def_exec_unreported_exception); } + +/*** Helper for imp test ***/ + +static PyModuleDef imp_dummy_def = TEST_MODULE_DEF("imp_dummy", main_slots, testexport_methods); + +PyMODINIT_FUNC +PyInit_imp_dummy(PyObject *spec) +{ + return PyModuleDef_Init(&imp_dummy_def); +} diff --git a/Modules/timemodule.c b/Modules/timemodule.c --- a/Modules/timemodule.c +++ b/Modules/timemodule.c @@ -610,14 +610,15 @@ #if defined(MS_WINDOWS) && !defined(HAVE_WCSFTIME) /* check that the format string contains only valid directives */ - for(outbuf = strchr(fmt, '%'); + for (outbuf = strchr(fmt, '%'); outbuf != NULL; outbuf = strchr(outbuf+2, '%')) { - if (outbuf[1]=='#') + if (outbuf[1] == '#') ++outbuf; /* not documented by python, */ - if ((outbuf[1] == 'y') && buf.tm_year < 0) - { + if (outbuf[1] == '\0') + break; + if ((outbuf[1] == 'y') && buf.tm_year < 0) { PyErr_SetString(PyExc_ValueError, "format %y requires year >= 1900 on Windows"); Py_DECREF(format); @@ -625,10 +626,12 @@ } } #elif (defined(_AIX) || defined(sun)) && defined(HAVE_WCSFTIME) - for(outbuf = wcschr(fmt, '%'); + for (outbuf = wcschr(fmt, '%'); outbuf != NULL; outbuf = wcschr(outbuf+2, '%')) { + if (outbuf[1] == L'\0') + break; /* Issue #19634: On AIX, wcsftime("y", (1899, 1, 1, 0, 0, 0, 0, 0, 0)) returns "0/" instead of "99" */ if (outbuf[1] == L'y' && buf.tm_year < 0) { @@ -659,7 +662,8 @@ #if defined _MSC_VER && _MSC_VER >= 1400 && defined(__STDC_SECURE_LIB__) err = errno; #endif - if (buflen > 0 || i >= 256 * fmtlen) { + if (buflen > 0 || fmtlen == 0 || + (fmtlen > 4 && i >= 256 * fmtlen)) { /* If the buffer is 256 times as long as the format, it's probably not failing for lack of room! More likely, the format yields an empty result, diff --git a/Python/_warnings.c b/Python/_warnings.c --- a/Python/_warnings.c +++ b/Python/_warnings.c @@ -513,6 +513,64 @@ return result; /* Py_None or NULL. */ } +static int +is_internal_frame(PyFrameObject *frame) +{ + static PyObject *importlib_string = NULL; + static PyObject *bootstrap_string = NULL; + PyObject *filename; + int contains; + + if (importlib_string == NULL) { + importlib_string = PyUnicode_FromString("importlib"); + if (importlib_string == NULL) { + return 0; + } + + bootstrap_string = PyUnicode_FromString("_bootstrap"); + if (bootstrap_string == NULL) { + Py_DECREF(importlib_string); + return 0; + } + Py_INCREF(importlib_string); + Py_INCREF(bootstrap_string); + } + + if (frame == NULL || frame->f_code == NULL || + frame->f_code->co_filename == NULL) { + return 0; + } + filename = frame->f_code->co_filename; + if (!PyUnicode_Check(filename)) { + return 0; + } + contains = PyUnicode_Contains(filename, importlib_string); + if (contains < 0) { + return 0; + } + else if (contains > 0) { + contains = PyUnicode_Contains(filename, bootstrap_string); + if (contains < 0) { + return 0; + } + else if (contains > 0) { + return 1; + } + } + + return 0; +} + +static PyFrameObject * +next_external_frame(PyFrameObject *frame) +{ + do { + frame = frame->f_back; + } while (frame != NULL && is_internal_frame(frame)); + + return frame; +} + /* filename, module, and registry are new refs, globals is borrowed */ /* Returns 0 on error (no new refs), 1 on success */ static int @@ -523,8 +581,18 @@ /* Setup globals and lineno. */ PyFrameObject *f = PyThreadState_GET()->frame; - while (--stack_level > 0 && f != NULL) - f = f->f_back; + // Stack level comparisons to Python code is off by one as there is no + // warnings-related stack level to avoid. + if (stack_level <= 0 || is_internal_frame(f)) { + while (--stack_level > 0 && f != NULL) { + f = f->f_back; + } + } + else { + while (--stack_level > 0 && f != NULL) { + f = next_external_frame(f); + } + } if (f == NULL) { globals = PyThreadState_Get()->interp->sysdict; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 7 07:38:07 2015 From: python-checkins at python.org (steve.dower) Date: Mon, 07 Sep 2015 05:38:07 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge_from_3=2E5?= Message-ID: <20150907053807.14857.38343@psf.io> https://hg.python.org/cpython/rev/1e8fe240c575 changeset: 97736:1e8fe240c575 parent: 97724:6c9159661aa8 parent: 97735:a11d30b8fcdf user: Steve Dower date: Sun Sep 06 22:31:26 2015 -0700 summary: Merge from 3.5 files: Lib/imp.py | 8 +- Lib/test/imp_dummy.py | 3 + Lib/test/test_imp.py | 24 + Lib/test/test_time.py | 13 + Lib/test/test_warnings.py | 950 --------- Lib/test/test_warnings/__init__.py | 961 ++++++++++ Lib/test/test_warnings/__main__.py | 3 + Lib/test/test_warnings/data/import_warning.py | 3 + Lib/test/test_warnings/data/stacklevel.py | 0 Lib/warnings.py | 31 +- Lib/webbrowser.py | 120 +- Misc/NEWS | 14 +- Modules/_testmultiphase.c | 10 + Modules/timemodule.c | 16 +- Python/_warnings.c | 72 +- 15 files changed, 1150 insertions(+), 1078 deletions(-) diff --git a/Lib/imp.py b/Lib/imp.py --- a/Lib/imp.py +++ b/Lib/imp.py @@ -334,6 +334,12 @@ """ import importlib.machinery loader = importlib.machinery.ExtensionFileLoader(name, path) - return loader.load_module() + + # Issue #24748: Skip the sys.modules check in _load_module_shim; + # always load new extension + spec = importlib.machinery.ModuleSpec( + name=name, loader=loader, origin=path) + return _load(spec) + else: load_dynamic = None diff --git a/Lib/test/imp_dummy.py b/Lib/test/imp_dummy.py new file mode 100644 --- /dev/null +++ b/Lib/test/imp_dummy.py @@ -0,0 +1,3 @@ +# Fodder for test of issue24748 in test_imp + +dummy_name = True diff --git a/Lib/test/test_imp.py b/Lib/test/test_imp.py --- a/Lib/test/test_imp.py +++ b/Lib/test/test_imp.py @@ -3,6 +3,7 @@ except ImportError: _thread = None import importlib +import importlib.util import os import os.path import shutil @@ -275,6 +276,29 @@ self.skipTest("found module doesn't appear to be a C extension") imp.load_module(name, None, *found[1:]) + @requires_load_dynamic + def test_issue24748_load_module_skips_sys_modules_check(self): + name = 'test.imp_dummy' + try: + del sys.modules[name] + except KeyError: + pass + try: + module = importlib.import_module(name) + spec = importlib.util.find_spec('_testmultiphase') + module = imp.load_dynamic(name, spec.origin) + self.assertEqual(module.__name__, name) + self.assertEqual(module.__spec__.name, name) + self.assertEqual(module.__spec__.origin, spec.origin) + self.assertRaises(AttributeError, getattr, module, 'dummy_name') + self.assertEqual(module.int_const, 1969) + self.assertIs(sys.modules[name], module) + finally: + try: + del sys.modules[name] + except KeyError: + pass + @unittest.skipIf(sys.dont_write_bytecode, "test meaningful only when writing bytecode") def test_bug7732(self): diff --git a/Lib/test/test_time.py b/Lib/test/test_time.py --- a/Lib/test/test_time.py +++ b/Lib/test/test_time.py @@ -177,6 +177,19 @@ def test_strftime_bounding_check(self): self._bounds_checking(lambda tup: time.strftime('', tup)) + def test_strftime_format_check(self): + # Test that strftime does not crash on invalid format strings + # that may trigger a buffer overread. When not triggered, + # strftime may succeed or raise ValueError depending on + # the platform. + for x in [ '', 'A', '%A', '%AA' ]: + for y in range(0x0, 0x10): + for z in [ '%', 'A%', 'AA%', '%A%', 'A%A%', '%#' ]: + try: + time.strftime(x * y + z) + except ValueError: + pass + def test_default_values_for_zero(self): # Make sure that using all zeros uses the proper default # values. No test for daylight savings since strftime() does diff --git a/Lib/test/test_warnings.py b/Lib/test/test_warnings.py deleted file mode 100644 --- a/Lib/test/test_warnings.py +++ /dev/null @@ -1,950 +0,0 @@ -from contextlib import contextmanager -import linecache -import os -from io import StringIO -import sys -import unittest -from test import support -from test.support.script_helper import assert_python_ok, assert_python_failure - -from test import warning_tests - -import warnings as original_warnings - -py_warnings = support.import_fresh_module('warnings', blocked=['_warnings']) -c_warnings = support.import_fresh_module('warnings', fresh=['_warnings']) - - at contextmanager -def warnings_state(module): - """Use a specific warnings implementation in warning_tests.""" - global __warningregistry__ - for to_clear in (sys, warning_tests): - try: - to_clear.__warningregistry__.clear() - except AttributeError: - pass - try: - __warningregistry__.clear() - except NameError: - pass - original_warnings = warning_tests.warnings - original_filters = module.filters - try: - module.filters = original_filters[:] - module.simplefilter("once") - warning_tests.warnings = module - yield - finally: - warning_tests.warnings = original_warnings - module.filters = original_filters - - -class BaseTest: - - """Basic bookkeeping required for testing.""" - - def setUp(self): - self.old_unittest_module = unittest.case.warnings - # The __warningregistry__ needs to be in a pristine state for tests - # to work properly. - if '__warningregistry__' in globals(): - del globals()['__warningregistry__'] - if hasattr(warning_tests, '__warningregistry__'): - del warning_tests.__warningregistry__ - if hasattr(sys, '__warningregistry__'): - del sys.__warningregistry__ - # The 'warnings' module must be explicitly set so that the proper - # interaction between _warnings and 'warnings' can be controlled. - sys.modules['warnings'] = self.module - # Ensure that unittest.TestCase.assertWarns() uses the same warnings - # module than warnings.catch_warnings(). Otherwise, - # warnings.catch_warnings() will be unable to remove the added filter. - unittest.case.warnings = self.module - super(BaseTest, self).setUp() - - def tearDown(self): - sys.modules['warnings'] = original_warnings - unittest.case.warnings = self.old_unittest_module - super(BaseTest, self).tearDown() - -class PublicAPITests(BaseTest): - - """Ensures that the correct values are exposed in the - public API. - """ - - def test_module_all_attribute(self): - self.assertTrue(hasattr(self.module, '__all__')) - target_api = ["warn", "warn_explicit", "showwarning", - "formatwarning", "filterwarnings", "simplefilter", - "resetwarnings", "catch_warnings"] - self.assertSetEqual(set(self.module.__all__), - set(target_api)) - -class CPublicAPITests(PublicAPITests, unittest.TestCase): - module = c_warnings - -class PyPublicAPITests(PublicAPITests, unittest.TestCase): - module = py_warnings - -class FilterTests(BaseTest): - - """Testing the filtering functionality.""" - - def test_error(self): - with original_warnings.catch_warnings(module=self.module) as w: - self.module.resetwarnings() - self.module.filterwarnings("error", category=UserWarning) - self.assertRaises(UserWarning, self.module.warn, - "FilterTests.test_error") - - def test_error_after_default(self): - with original_warnings.catch_warnings(module=self.module) as w: - self.module.resetwarnings() - message = "FilterTests.test_ignore_after_default" - def f(): - self.module.warn(message, UserWarning) - f() - self.module.filterwarnings("error", category=UserWarning) - self.assertRaises(UserWarning, f) - - def test_ignore(self): - with original_warnings.catch_warnings(record=True, - module=self.module) as w: - self.module.resetwarnings() - self.module.filterwarnings("ignore", category=UserWarning) - self.module.warn("FilterTests.test_ignore", UserWarning) - self.assertEqual(len(w), 0) - - def test_ignore_after_default(self): - with original_warnings.catch_warnings(record=True, - module=self.module) as w: - self.module.resetwarnings() - message = "FilterTests.test_ignore_after_default" - def f(): - self.module.warn(message, UserWarning) - f() - self.module.filterwarnings("ignore", category=UserWarning) - f() - f() - self.assertEqual(len(w), 1) - - def test_always(self): - with original_warnings.catch_warnings(record=True, - module=self.module) as w: - self.module.resetwarnings() - self.module.filterwarnings("always", category=UserWarning) - message = "FilterTests.test_always" - self.module.warn(message, UserWarning) - self.assertTrue(message, w[-1].message) - self.module.warn(message, UserWarning) - self.assertTrue(w[-1].message, message) - - def test_always_after_default(self): - with original_warnings.catch_warnings(record=True, - module=self.module) as w: - self.module.resetwarnings() - message = "FilterTests.test_always_after_ignore" - def f(): - self.module.warn(message, UserWarning) - f() - self.assertEqual(len(w), 1) - self.assertEqual(w[-1].message.args[0], message) - f() - self.assertEqual(len(w), 1) - self.module.filterwarnings("always", category=UserWarning) - f() - self.assertEqual(len(w), 2) - self.assertEqual(w[-1].message.args[0], message) - f() - self.assertEqual(len(w), 3) - self.assertEqual(w[-1].message.args[0], message) - - def test_default(self): - with original_warnings.catch_warnings(record=True, - module=self.module) as w: - self.module.resetwarnings() - self.module.filterwarnings("default", category=UserWarning) - message = UserWarning("FilterTests.test_default") - for x in range(2): - self.module.warn(message, UserWarning) - if x == 0: - self.assertEqual(w[-1].message, message) - del w[:] - elif x == 1: - self.assertEqual(len(w), 0) - else: - raise ValueError("loop variant unhandled") - - def test_module(self): - with original_warnings.catch_warnings(record=True, - module=self.module) as w: - self.module.resetwarnings() - self.module.filterwarnings("module", category=UserWarning) - message = UserWarning("FilterTests.test_module") - self.module.warn(message, UserWarning) - self.assertEqual(w[-1].message, message) - del w[:] - self.module.warn(message, UserWarning) - self.assertEqual(len(w), 0) - - def test_once(self): - with original_warnings.catch_warnings(record=True, - module=self.module) as w: - self.module.resetwarnings() - self.module.filterwarnings("once", category=UserWarning) - message = UserWarning("FilterTests.test_once") - self.module.warn_explicit(message, UserWarning, "test_warnings.py", - 42) - self.assertEqual(w[-1].message, message) - del w[:] - self.module.warn_explicit(message, UserWarning, "test_warnings.py", - 13) - self.assertEqual(len(w), 0) - self.module.warn_explicit(message, UserWarning, "test_warnings2.py", - 42) - self.assertEqual(len(w), 0) - - def test_inheritance(self): - with original_warnings.catch_warnings(module=self.module) as w: - self.module.resetwarnings() - self.module.filterwarnings("error", category=Warning) - self.assertRaises(UserWarning, self.module.warn, - "FilterTests.test_inheritance", UserWarning) - - def test_ordering(self): - with original_warnings.catch_warnings(record=True, - module=self.module) as w: - self.module.resetwarnings() - self.module.filterwarnings("ignore", category=UserWarning) - self.module.filterwarnings("error", category=UserWarning, - append=True) - del w[:] - try: - self.module.warn("FilterTests.test_ordering", UserWarning) - except UserWarning: - self.fail("order handling for actions failed") - self.assertEqual(len(w), 0) - - def test_filterwarnings(self): - # Test filterwarnings(). - # Implicitly also tests resetwarnings(). - with original_warnings.catch_warnings(record=True, - module=self.module) as w: - self.module.filterwarnings("error", "", Warning, "", 0) - self.assertRaises(UserWarning, self.module.warn, 'convert to error') - - self.module.resetwarnings() - text = 'handle normally' - self.module.warn(text) - self.assertEqual(str(w[-1].message), text) - self.assertTrue(w[-1].category is UserWarning) - - self.module.filterwarnings("ignore", "", Warning, "", 0) - text = 'filtered out' - self.module.warn(text) - self.assertNotEqual(str(w[-1].message), text) - - self.module.resetwarnings() - self.module.filterwarnings("error", "hex*", Warning, "", 0) - self.assertRaises(UserWarning, self.module.warn, 'hex/oct') - text = 'nonmatching text' - self.module.warn(text) - self.assertEqual(str(w[-1].message), text) - self.assertTrue(w[-1].category is UserWarning) - - def test_mutate_filter_list(self): - class X: - def match(self, a): - L[:] = [] - - L = [("default",X(),UserWarning,X(),0) for i in range(2)] - with original_warnings.catch_warnings(record=True, - module=self.module) as w: - self.module.filters = L - self.module.warn_explicit(UserWarning("b"), None, "f.py", 42) - self.assertEqual(str(w[-1].message), "b") - -class CFilterTests(FilterTests, unittest.TestCase): - module = c_warnings - -class PyFilterTests(FilterTests, unittest.TestCase): - module = py_warnings - - -class WarnTests(BaseTest): - - """Test warnings.warn() and warnings.warn_explicit().""" - - def test_message(self): - with original_warnings.catch_warnings(record=True, - module=self.module) as w: - self.module.simplefilter("once") - for i in range(4): - text = 'multi %d' %i # Different text on each call. - self.module.warn(text) - self.assertEqual(str(w[-1].message), text) - self.assertTrue(w[-1].category is UserWarning) - - # Issue 3639 - def test_warn_nonstandard_types(self): - # warn() should handle non-standard types without issue. - for ob in (Warning, None, 42): - with original_warnings.catch_warnings(record=True, - module=self.module) as w: - self.module.simplefilter("once") - self.module.warn(ob) - # Don't directly compare objects since - # ``Warning() != Warning()``. - self.assertEqual(str(w[-1].message), str(UserWarning(ob))) - - def test_filename(self): - with warnings_state(self.module): - with original_warnings.catch_warnings(record=True, - module=self.module) as w: - warning_tests.inner("spam1") - self.assertEqual(os.path.basename(w[-1].filename), - "warning_tests.py") - warning_tests.outer("spam2") - self.assertEqual(os.path.basename(w[-1].filename), - "warning_tests.py") - - def test_stacklevel(self): - # Test stacklevel argument - # make sure all messages are different, so the warning won't be skipped - with warnings_state(self.module): - with original_warnings.catch_warnings(record=True, - module=self.module) as w: - warning_tests.inner("spam3", stacklevel=1) - self.assertEqual(os.path.basename(w[-1].filename), - "warning_tests.py") - warning_tests.outer("spam4", stacklevel=1) - self.assertEqual(os.path.basename(w[-1].filename), - "warning_tests.py") - - warning_tests.inner("spam5", stacklevel=2) - self.assertEqual(os.path.basename(w[-1].filename), - "test_warnings.py") - warning_tests.outer("spam6", stacklevel=2) - self.assertEqual(os.path.basename(w[-1].filename), - "warning_tests.py") - warning_tests.outer("spam6.5", stacklevel=3) - self.assertEqual(os.path.basename(w[-1].filename), - "test_warnings.py") - - warning_tests.inner("spam7", stacklevel=9999) - self.assertEqual(os.path.basename(w[-1].filename), - "sys") - - def test_missing_filename_not_main(self): - # If __file__ is not specified and __main__ is not the module name, - # then __file__ should be set to the module name. - filename = warning_tests.__file__ - try: - del warning_tests.__file__ - with warnings_state(self.module): - with original_warnings.catch_warnings(record=True, - module=self.module) as w: - warning_tests.inner("spam8", stacklevel=1) - self.assertEqual(w[-1].filename, warning_tests.__name__) - finally: - warning_tests.__file__ = filename - - @unittest.skipUnless(hasattr(sys, 'argv'), 'test needs sys.argv') - def test_missing_filename_main_with_argv(self): - # If __file__ is not specified and the caller is __main__ and sys.argv - # exists, then use sys.argv[0] as the file. - filename = warning_tests.__file__ - module_name = warning_tests.__name__ - try: - del warning_tests.__file__ - warning_tests.__name__ = '__main__' - with warnings_state(self.module): - with original_warnings.catch_warnings(record=True, - module=self.module) as w: - warning_tests.inner('spam9', stacklevel=1) - self.assertEqual(w[-1].filename, sys.argv[0]) - finally: - warning_tests.__file__ = filename - warning_tests.__name__ = module_name - - def test_missing_filename_main_without_argv(self): - # If __file__ is not specified, the caller is __main__, and sys.argv - # is not set, then '__main__' is the file name. - filename = warning_tests.__file__ - module_name = warning_tests.__name__ - argv = sys.argv - try: - del warning_tests.__file__ - warning_tests.__name__ = '__main__' - del sys.argv - with warnings_state(self.module): - with original_warnings.catch_warnings(record=True, - module=self.module) as w: - warning_tests.inner('spam10', stacklevel=1) - self.assertEqual(w[-1].filename, '__main__') - finally: - warning_tests.__file__ = filename - warning_tests.__name__ = module_name - sys.argv = argv - - def test_missing_filename_main_with_argv_empty_string(self): - # If __file__ is not specified, the caller is __main__, and sys.argv[0] - # is the empty string, then '__main__ is the file name. - # Tests issue 2743. - file_name = warning_tests.__file__ - module_name = warning_tests.__name__ - argv = sys.argv - try: - del warning_tests.__file__ - warning_tests.__name__ = '__main__' - sys.argv = [''] - with warnings_state(self.module): - with original_warnings.catch_warnings(record=True, - module=self.module) as w: - warning_tests.inner('spam11', stacklevel=1) - self.assertEqual(w[-1].filename, '__main__') - finally: - warning_tests.__file__ = file_name - warning_tests.__name__ = module_name - sys.argv = argv - - def test_warn_explicit_non_ascii_filename(self): - with original_warnings.catch_warnings(record=True, - module=self.module) as w: - self.module.resetwarnings() - self.module.filterwarnings("always", category=UserWarning) - for filename in ("nonascii\xe9\u20ac", "surrogate\udc80"): - try: - os.fsencode(filename) - except UnicodeEncodeError: - continue - self.module.warn_explicit("text", UserWarning, filename, 1) - self.assertEqual(w[-1].filename, filename) - - def test_warn_explicit_type_errors(self): - # warn_explicit() should error out gracefully if it is given objects - # of the wrong types. - # lineno is expected to be an integer. - self.assertRaises(TypeError, self.module.warn_explicit, - None, UserWarning, None, None) - # Either 'message' needs to be an instance of Warning or 'category' - # needs to be a subclass. - self.assertRaises(TypeError, self.module.warn_explicit, - None, None, None, 1) - # 'registry' must be a dict or None. - self.assertRaises((TypeError, AttributeError), - self.module.warn_explicit, - None, Warning, None, 1, registry=42) - - def test_bad_str(self): - # issue 6415 - # Warnings instance with a bad format string for __str__ should not - # trigger a bus error. - class BadStrWarning(Warning): - """Warning with a bad format string for __str__.""" - def __str__(self): - return ("A bad formatted string %(err)" % - {"err" : "there is no %(err)s"}) - - with self.assertRaises(ValueError): - self.module.warn(BadStrWarning()) - - def test_warning_classes(self): - class MyWarningClass(Warning): - pass - - class NonWarningSubclass: - pass - - # passing a non-subclass of Warning should raise a TypeError - with self.assertRaises(TypeError) as cm: - self.module.warn('bad warning category', '') - self.assertIn('category must be a Warning subclass, not ', - str(cm.exception)) - - with self.assertRaises(TypeError) as cm: - self.module.warn('bad warning category', NonWarningSubclass) - self.assertIn('category must be a Warning subclass, not ', - str(cm.exception)) - - # check that warning instances also raise a TypeError - with self.assertRaises(TypeError) as cm: - self.module.warn('bad warning category', MyWarningClass()) - self.assertIn('category must be a Warning subclass, not ', - str(cm.exception)) - - with original_warnings.catch_warnings(module=self.module): - self.module.resetwarnings() - self.module.filterwarnings('default') - with self.assertWarns(MyWarningClass) as cm: - self.module.warn('good warning category', MyWarningClass) - self.assertEqual('good warning category', str(cm.warning)) - - with self.assertWarns(UserWarning) as cm: - self.module.warn('good warning category', None) - self.assertEqual('good warning category', str(cm.warning)) - - with self.assertWarns(MyWarningClass) as cm: - self.module.warn('good warning category', MyWarningClass) - self.assertIsInstance(cm.warning, Warning) - -class CWarnTests(WarnTests, unittest.TestCase): - module = c_warnings - - # As an early adopter, we sanity check the - # test.support.import_fresh_module utility function - def test_accelerated(self): - self.assertFalse(original_warnings is self.module) - self.assertFalse(hasattr(self.module.warn, '__code__')) - -class PyWarnTests(WarnTests, unittest.TestCase): - module = py_warnings - - # As an early adopter, we sanity check the - # test.support.import_fresh_module utility function - def test_pure_python(self): - self.assertFalse(original_warnings is self.module) - self.assertTrue(hasattr(self.module.warn, '__code__')) - - -class WCmdLineTests(BaseTest): - - def test_improper_input(self): - # Uses the private _setoption() function to test the parsing - # of command-line warning arguments - with original_warnings.catch_warnings(module=self.module): - self.assertRaises(self.module._OptionError, - self.module._setoption, '1:2:3:4:5:6') - self.assertRaises(self.module._OptionError, - self.module._setoption, 'bogus::Warning') - self.assertRaises(self.module._OptionError, - self.module._setoption, 'ignore:2::4:-5') - self.module._setoption('error::Warning::0') - self.assertRaises(UserWarning, self.module.warn, 'convert to error') - - def test_improper_option(self): - # Same as above, but check that the message is printed out when - # the interpreter is executed. This also checks that options are - # actually parsed at all. - rc, out, err = assert_python_ok("-Wxxx", "-c", "pass") - self.assertIn(b"Invalid -W option ignored: invalid action: 'xxx'", err) - - def test_warnings_bootstrap(self): - # Check that the warnings module does get loaded when -W - # is used (see issue #10372 for an example of silent bootstrap failure). - rc, out, err = assert_python_ok("-Wi", "-c", - "import sys; sys.modules['warnings'].warn('foo', RuntimeWarning)") - # '-Wi' was observed - self.assertFalse(out.strip()) - self.assertNotIn(b'RuntimeWarning', err) - -class CWCmdLineTests(WCmdLineTests, unittest.TestCase): - module = c_warnings - -class PyWCmdLineTests(WCmdLineTests, unittest.TestCase): - module = py_warnings - - -class _WarningsTests(BaseTest, unittest.TestCase): - - """Tests specific to the _warnings module.""" - - module = c_warnings - - def test_filter(self): - # Everything should function even if 'filters' is not in warnings. - with original_warnings.catch_warnings(module=self.module) as w: - self.module.filterwarnings("error", "", Warning, "", 0) - self.assertRaises(UserWarning, self.module.warn, - 'convert to error') - del self.module.filters - self.assertRaises(UserWarning, self.module.warn, - 'convert to error') - - def test_onceregistry(self): - # Replacing or removing the onceregistry should be okay. - global __warningregistry__ - message = UserWarning('onceregistry test') - try: - original_registry = self.module.onceregistry - __warningregistry__ = {} - with original_warnings.catch_warnings(record=True, - module=self.module) as w: - self.module.resetwarnings() - self.module.filterwarnings("once", category=UserWarning) - self.module.warn_explicit(message, UserWarning, "file", 42) - self.assertEqual(w[-1].message, message) - del w[:] - self.module.warn_explicit(message, UserWarning, "file", 42) - self.assertEqual(len(w), 0) - # Test the resetting of onceregistry. - self.module.onceregistry = {} - __warningregistry__ = {} - self.module.warn('onceregistry test') - self.assertEqual(w[-1].message.args, message.args) - # Removal of onceregistry is okay. - del w[:] - del self.module.onceregistry - __warningregistry__ = {} - self.module.warn_explicit(message, UserWarning, "file", 42) - self.assertEqual(len(w), 0) - finally: - self.module.onceregistry = original_registry - - def test_default_action(self): - # Replacing or removing defaultaction should be okay. - message = UserWarning("defaultaction test") - original = self.module.defaultaction - try: - with original_warnings.catch_warnings(record=True, - module=self.module) as w: - self.module.resetwarnings() - registry = {} - self.module.warn_explicit(message, UserWarning, "", 42, - registry=registry) - self.assertEqual(w[-1].message, message) - self.assertEqual(len(w), 1) - # One actual registry key plus the "version" key - self.assertEqual(len(registry), 2) - self.assertIn("version", registry) - del w[:] - # Test removal. - del self.module.defaultaction - __warningregistry__ = {} - registry = {} - self.module.warn_explicit(message, UserWarning, "", 43, - registry=registry) - self.assertEqual(w[-1].message, message) - self.assertEqual(len(w), 1) - self.assertEqual(len(registry), 2) - del w[:] - # Test setting. - self.module.defaultaction = "ignore" - __warningregistry__ = {} - registry = {} - self.module.warn_explicit(message, UserWarning, "", 44, - registry=registry) - self.assertEqual(len(w), 0) - finally: - self.module.defaultaction = original - - def test_showwarning_missing(self): - # Test that showwarning() missing is okay. - text = 'del showwarning test' - with original_warnings.catch_warnings(module=self.module): - self.module.filterwarnings("always", category=UserWarning) - del self.module.showwarning - with support.captured_output('stderr') as stream: - self.module.warn(text) - result = stream.getvalue() - self.assertIn(text, result) - - def test_showwarning_not_callable(self): - with original_warnings.catch_warnings(module=self.module): - self.module.filterwarnings("always", category=UserWarning) - self.module.showwarning = print - with support.captured_output('stdout'): - self.module.warn('Warning!') - self.module.showwarning = 23 - self.assertRaises(TypeError, self.module.warn, "Warning!") - - def test_show_warning_output(self): - # With showarning() missing, make sure that output is okay. - text = 'test show_warning' - with original_warnings.catch_warnings(module=self.module): - self.module.filterwarnings("always", category=UserWarning) - del self.module.showwarning - with support.captured_output('stderr') as stream: - warning_tests.inner(text) - result = stream.getvalue() - self.assertEqual(result.count('\n'), 2, - "Too many newlines in %r" % result) - first_line, second_line = result.split('\n', 1) - expected_file = os.path.splitext(warning_tests.__file__)[0] + '.py' - first_line_parts = first_line.rsplit(':', 3) - path, line, warning_class, message = first_line_parts - line = int(line) - self.assertEqual(expected_file, path) - self.assertEqual(warning_class, ' ' + UserWarning.__name__) - self.assertEqual(message, ' ' + text) - expected_line = ' ' + linecache.getline(path, line).strip() + '\n' - assert expected_line - self.assertEqual(second_line, expected_line) - - def test_filename_none(self): - # issue #12467: race condition if a warning is emitted at shutdown - globals_dict = globals() - oldfile = globals_dict['__file__'] - try: - catch = original_warnings.catch_warnings(record=True, - module=self.module) - with catch as w: - self.module.filterwarnings("always", category=UserWarning) - globals_dict['__file__'] = None - original_warnings.warn('test', UserWarning) - self.assertTrue(len(w)) - finally: - globals_dict['__file__'] = oldfile - - def test_stderr_none(self): - rc, stdout, stderr = assert_python_ok("-c", - "import sys; sys.stderr = None; " - "import warnings; warnings.simplefilter('always'); " - "warnings.warn('Warning!')") - self.assertEqual(stdout, b'') - self.assertNotIn(b'Warning!', stderr) - self.assertNotIn(b'Error', stderr) - - -class WarningsDisplayTests(BaseTest): - - """Test the displaying of warnings and the ability to overload functions - related to displaying warnings.""" - - def test_formatwarning(self): - message = "msg" - category = Warning - file_name = os.path.splitext(warning_tests.__file__)[0] + '.py' - line_num = 3 - file_line = linecache.getline(file_name, line_num).strip() - format = "%s:%s: %s: %s\n %s\n" - expect = format % (file_name, line_num, category.__name__, message, - file_line) - self.assertEqual(expect, self.module.formatwarning(message, - category, file_name, line_num)) - # Test the 'line' argument. - file_line += " for the win!" - expect = format % (file_name, line_num, category.__name__, message, - file_line) - self.assertEqual(expect, self.module.formatwarning(message, - category, file_name, line_num, file_line)) - - def test_showwarning(self): - file_name = os.path.splitext(warning_tests.__file__)[0] + '.py' - line_num = 3 - expected_file_line = linecache.getline(file_name, line_num).strip() - message = 'msg' - category = Warning - file_object = StringIO() - expect = self.module.formatwarning(message, category, file_name, - line_num) - self.module.showwarning(message, category, file_name, line_num, - file_object) - self.assertEqual(file_object.getvalue(), expect) - # Test 'line' argument. - expected_file_line += "for the win!" - expect = self.module.formatwarning(message, category, file_name, - line_num, expected_file_line) - file_object = StringIO() - self.module.showwarning(message, category, file_name, line_num, - file_object, expected_file_line) - self.assertEqual(expect, file_object.getvalue()) - -class CWarningsDisplayTests(WarningsDisplayTests, unittest.TestCase): - module = c_warnings - -class PyWarningsDisplayTests(WarningsDisplayTests, unittest.TestCase): - module = py_warnings - - -class CatchWarningTests(BaseTest): - - """Test catch_warnings().""" - - def test_catch_warnings_restore(self): - wmod = self.module - orig_filters = wmod.filters - orig_showwarning = wmod.showwarning - # Ensure both showwarning and filters are restored when recording - with wmod.catch_warnings(module=wmod, record=True): - wmod.filters = wmod.showwarning = object() - self.assertTrue(wmod.filters is orig_filters) - self.assertTrue(wmod.showwarning is orig_showwarning) - # Same test, but with recording disabled - with wmod.catch_warnings(module=wmod, record=False): - wmod.filters = wmod.showwarning = object() - self.assertTrue(wmod.filters is orig_filters) - self.assertTrue(wmod.showwarning is orig_showwarning) - - def test_catch_warnings_recording(self): - wmod = self.module - # Ensure warnings are recorded when requested - with wmod.catch_warnings(module=wmod, record=True) as w: - self.assertEqual(w, []) - self.assertTrue(type(w) is list) - wmod.simplefilter("always") - wmod.warn("foo") - self.assertEqual(str(w[-1].message), "foo") - wmod.warn("bar") - self.assertEqual(str(w[-1].message), "bar") - self.assertEqual(str(w[0].message), "foo") - self.assertEqual(str(w[1].message), "bar") - del w[:] - self.assertEqual(w, []) - # Ensure warnings are not recorded when not requested - orig_showwarning = wmod.showwarning - with wmod.catch_warnings(module=wmod, record=False) as w: - self.assertTrue(w is None) - self.assertTrue(wmod.showwarning is orig_showwarning) - - def test_catch_warnings_reentry_guard(self): - wmod = self.module - # Ensure catch_warnings is protected against incorrect usage - x = wmod.catch_warnings(module=wmod, record=True) - self.assertRaises(RuntimeError, x.__exit__) - with x: - self.assertRaises(RuntimeError, x.__enter__) - # Same test, but with recording disabled - x = wmod.catch_warnings(module=wmod, record=False) - self.assertRaises(RuntimeError, x.__exit__) - with x: - self.assertRaises(RuntimeError, x.__enter__) - - def test_catch_warnings_defaults(self): - wmod = self.module - orig_filters = wmod.filters - orig_showwarning = wmod.showwarning - # Ensure default behaviour is not to record warnings - with wmod.catch_warnings(module=wmod) as w: - self.assertTrue(w is None) - self.assertTrue(wmod.showwarning is orig_showwarning) - self.assertTrue(wmod.filters is not orig_filters) - self.assertTrue(wmod.filters is orig_filters) - if wmod is sys.modules['warnings']: - # Ensure the default module is this one - with wmod.catch_warnings() as w: - self.assertTrue(w is None) - self.assertTrue(wmod.showwarning is orig_showwarning) - self.assertTrue(wmod.filters is not orig_filters) - self.assertTrue(wmod.filters is orig_filters) - - def test_check_warnings(self): - # Explicit tests for the test.support convenience wrapper - wmod = self.module - if wmod is not sys.modules['warnings']: - self.skipTest('module to test is not loaded warnings module') - with support.check_warnings(quiet=False) as w: - self.assertEqual(w.warnings, []) - wmod.simplefilter("always") - wmod.warn("foo") - self.assertEqual(str(w.message), "foo") - wmod.warn("bar") - self.assertEqual(str(w.message), "bar") - self.assertEqual(str(w.warnings[0].message), "foo") - self.assertEqual(str(w.warnings[1].message), "bar") - w.reset() - self.assertEqual(w.warnings, []) - - with support.check_warnings(): - # defaults to quiet=True without argument - pass - with support.check_warnings(('foo', UserWarning)): - wmod.warn("foo") - - with self.assertRaises(AssertionError): - with support.check_warnings(('', RuntimeWarning)): - # defaults to quiet=False with argument - pass - with self.assertRaises(AssertionError): - with support.check_warnings(('foo', RuntimeWarning)): - wmod.warn("foo") - -class CCatchWarningTests(CatchWarningTests, unittest.TestCase): - module = c_warnings - -class PyCatchWarningTests(CatchWarningTests, unittest.TestCase): - module = py_warnings - - -class EnvironmentVariableTests(BaseTest): - - def test_single_warning(self): - rc, stdout, stderr = assert_python_ok("-c", - "import sys; sys.stdout.write(str(sys.warnoptions))", - PYTHONWARNINGS="ignore::DeprecationWarning") - self.assertEqual(stdout, b"['ignore::DeprecationWarning']") - - def test_comma_separated_warnings(self): - rc, stdout, stderr = assert_python_ok("-c", - "import sys; sys.stdout.write(str(sys.warnoptions))", - PYTHONWARNINGS="ignore::DeprecationWarning,ignore::UnicodeWarning") - self.assertEqual(stdout, - b"['ignore::DeprecationWarning', 'ignore::UnicodeWarning']") - - def test_envvar_and_command_line(self): - rc, stdout, stderr = assert_python_ok("-Wignore::UnicodeWarning", "-c", - "import sys; sys.stdout.write(str(sys.warnoptions))", - PYTHONWARNINGS="ignore::DeprecationWarning") - self.assertEqual(stdout, - b"['ignore::DeprecationWarning', 'ignore::UnicodeWarning']") - - def test_conflicting_envvar_and_command_line(self): - rc, stdout, stderr = assert_python_failure("-Werror::DeprecationWarning", "-c", - "import sys, warnings; sys.stdout.write(str(sys.warnoptions)); " - "warnings.warn('Message', DeprecationWarning)", - PYTHONWARNINGS="default::DeprecationWarning") - self.assertEqual(stdout, - b"['default::DeprecationWarning', 'error::DeprecationWarning']") - self.assertEqual(stderr.splitlines(), - [b"Traceback (most recent call last):", - b" File \"\", line 1, in ", - b"DeprecationWarning: Message"]) - - @unittest.skipUnless(sys.getfilesystemencoding() != 'ascii', - 'requires non-ascii filesystemencoding') - def test_nonascii(self): - rc, stdout, stderr = assert_python_ok("-c", - "import sys; sys.stdout.write(str(sys.warnoptions))", - PYTHONIOENCODING="utf-8", - PYTHONWARNINGS="ignore:Deprecaci?nWarning") - self.assertEqual(stdout, - "['ignore:Deprecaci?nWarning']".encode('utf-8')) - -class CEnvironmentVariableTests(EnvironmentVariableTests, unittest.TestCase): - module = c_warnings - -class PyEnvironmentVariableTests(EnvironmentVariableTests, unittest.TestCase): - module = py_warnings - - -class BootstrapTest(unittest.TestCase): - def test_issue_8766(self): - # "import encodings" emits a warning whereas the warnings is not loaded - # or not completely loaded (warnings imports indirectly encodings by - # importing linecache) yet - with support.temp_cwd() as cwd, support.temp_cwd('encodings'): - # encodings loaded by initfsencoding() - assert_python_ok('-c', 'pass', PYTHONPATH=cwd) - - # Use -W to load warnings module at startup - assert_python_ok('-c', 'pass', '-W', 'always', PYTHONPATH=cwd) - -class FinalizationTest(unittest.TestCase): - def test_finalization(self): - # Issue #19421: warnings.warn() should not crash - # during Python finalization - code = """ -import warnings -warn = warnings.warn - -class A: - def __del__(self): - warn("test") - -a=A() - """ - rc, out, err = assert_python_ok("-c", code) - # note: "__main__" filename is not correct, it should be the name - # of the script - self.assertEqual(err, b'__main__:7: UserWarning: test') - - -def setUpModule(): - py_warnings.onceregistry.clear() - c_warnings.onceregistry.clear() - -tearDownModule = setUpModule - -if __name__ == "__main__": - unittest.main() diff --git a/Lib/test/test_warnings/__init__.py b/Lib/test/test_warnings/__init__.py new file mode 100644 --- /dev/null +++ b/Lib/test/test_warnings/__init__.py @@ -0,0 +1,961 @@ +from contextlib import contextmanager +import linecache +import os +from io import StringIO +import sys +import unittest +from test import support +from test.support.script_helper import assert_python_ok, assert_python_failure + +from test.test_warnings.data import stacklevel as warning_tests + +import warnings as original_warnings + +py_warnings = support.import_fresh_module('warnings', blocked=['_warnings']) +c_warnings = support.import_fresh_module('warnings', fresh=['_warnings']) + + at contextmanager +def warnings_state(module): + """Use a specific warnings implementation in warning_tests.""" + global __warningregistry__ + for to_clear in (sys, warning_tests): + try: + to_clear.__warningregistry__.clear() + except AttributeError: + pass + try: + __warningregistry__.clear() + except NameError: + pass + original_warnings = warning_tests.warnings + original_filters = module.filters + try: + module.filters = original_filters[:] + module.simplefilter("once") + warning_tests.warnings = module + yield + finally: + warning_tests.warnings = original_warnings + module.filters = original_filters + + +class BaseTest: + + """Basic bookkeeping required for testing.""" + + def setUp(self): + self.old_unittest_module = unittest.case.warnings + # The __warningregistry__ needs to be in a pristine state for tests + # to work properly. + if '__warningregistry__' in globals(): + del globals()['__warningregistry__'] + if hasattr(warning_tests, '__warningregistry__'): + del warning_tests.__warningregistry__ + if hasattr(sys, '__warningregistry__'): + del sys.__warningregistry__ + # The 'warnings' module must be explicitly set so that the proper + # interaction between _warnings and 'warnings' can be controlled. + sys.modules['warnings'] = self.module + # Ensure that unittest.TestCase.assertWarns() uses the same warnings + # module than warnings.catch_warnings(). Otherwise, + # warnings.catch_warnings() will be unable to remove the added filter. + unittest.case.warnings = self.module + super(BaseTest, self).setUp() + + def tearDown(self): + sys.modules['warnings'] = original_warnings + unittest.case.warnings = self.old_unittest_module + super(BaseTest, self).tearDown() + +class PublicAPITests(BaseTest): + + """Ensures that the correct values are exposed in the + public API. + """ + + def test_module_all_attribute(self): + self.assertTrue(hasattr(self.module, '__all__')) + target_api = ["warn", "warn_explicit", "showwarning", + "formatwarning", "filterwarnings", "simplefilter", + "resetwarnings", "catch_warnings"] + self.assertSetEqual(set(self.module.__all__), + set(target_api)) + +class CPublicAPITests(PublicAPITests, unittest.TestCase): + module = c_warnings + +class PyPublicAPITests(PublicAPITests, unittest.TestCase): + module = py_warnings + +class FilterTests(BaseTest): + + """Testing the filtering functionality.""" + + def test_error(self): + with original_warnings.catch_warnings(module=self.module) as w: + self.module.resetwarnings() + self.module.filterwarnings("error", category=UserWarning) + self.assertRaises(UserWarning, self.module.warn, + "FilterTests.test_error") + + def test_error_after_default(self): + with original_warnings.catch_warnings(module=self.module) as w: + self.module.resetwarnings() + message = "FilterTests.test_ignore_after_default" + def f(): + self.module.warn(message, UserWarning) + f() + self.module.filterwarnings("error", category=UserWarning) + self.assertRaises(UserWarning, f) + + def test_ignore(self): + with original_warnings.catch_warnings(record=True, + module=self.module) as w: + self.module.resetwarnings() + self.module.filterwarnings("ignore", category=UserWarning) + self.module.warn("FilterTests.test_ignore", UserWarning) + self.assertEqual(len(w), 0) + + def test_ignore_after_default(self): + with original_warnings.catch_warnings(record=True, + module=self.module) as w: + self.module.resetwarnings() + message = "FilterTests.test_ignore_after_default" + def f(): + self.module.warn(message, UserWarning) + f() + self.module.filterwarnings("ignore", category=UserWarning) + f() + f() + self.assertEqual(len(w), 1) + + def test_always(self): + with original_warnings.catch_warnings(record=True, + module=self.module) as w: + self.module.resetwarnings() + self.module.filterwarnings("always", category=UserWarning) + message = "FilterTests.test_always" + self.module.warn(message, UserWarning) + self.assertTrue(message, w[-1].message) + self.module.warn(message, UserWarning) + self.assertTrue(w[-1].message, message) + + def test_always_after_default(self): + with original_warnings.catch_warnings(record=True, + module=self.module) as w: + self.module.resetwarnings() + message = "FilterTests.test_always_after_ignore" + def f(): + self.module.warn(message, UserWarning) + f() + self.assertEqual(len(w), 1) + self.assertEqual(w[-1].message.args[0], message) + f() + self.assertEqual(len(w), 1) + self.module.filterwarnings("always", category=UserWarning) + f() + self.assertEqual(len(w), 2) + self.assertEqual(w[-1].message.args[0], message) + f() + self.assertEqual(len(w), 3) + self.assertEqual(w[-1].message.args[0], message) + + def test_default(self): + with original_warnings.catch_warnings(record=True, + module=self.module) as w: + self.module.resetwarnings() + self.module.filterwarnings("default", category=UserWarning) + message = UserWarning("FilterTests.test_default") + for x in range(2): + self.module.warn(message, UserWarning) + if x == 0: + self.assertEqual(w[-1].message, message) + del w[:] + elif x == 1: + self.assertEqual(len(w), 0) + else: + raise ValueError("loop variant unhandled") + + def test_module(self): + with original_warnings.catch_warnings(record=True, + module=self.module) as w: + self.module.resetwarnings() + self.module.filterwarnings("module", category=UserWarning) + message = UserWarning("FilterTests.test_module") + self.module.warn(message, UserWarning) + self.assertEqual(w[-1].message, message) + del w[:] + self.module.warn(message, UserWarning) + self.assertEqual(len(w), 0) + + def test_once(self): + with original_warnings.catch_warnings(record=True, + module=self.module) as w: + self.module.resetwarnings() + self.module.filterwarnings("once", category=UserWarning) + message = UserWarning("FilterTests.test_once") + self.module.warn_explicit(message, UserWarning, "__init__.py", + 42) + self.assertEqual(w[-1].message, message) + del w[:] + self.module.warn_explicit(message, UserWarning, "__init__.py", + 13) + self.assertEqual(len(w), 0) + self.module.warn_explicit(message, UserWarning, "test_warnings2.py", + 42) + self.assertEqual(len(w), 0) + + def test_inheritance(self): + with original_warnings.catch_warnings(module=self.module) as w: + self.module.resetwarnings() + self.module.filterwarnings("error", category=Warning) + self.assertRaises(UserWarning, self.module.warn, + "FilterTests.test_inheritance", UserWarning) + + def test_ordering(self): + with original_warnings.catch_warnings(record=True, + module=self.module) as w: + self.module.resetwarnings() + self.module.filterwarnings("ignore", category=UserWarning) + self.module.filterwarnings("error", category=UserWarning, + append=True) + del w[:] + try: + self.module.warn("FilterTests.test_ordering", UserWarning) + except UserWarning: + self.fail("order handling for actions failed") + self.assertEqual(len(w), 0) + + def test_filterwarnings(self): + # Test filterwarnings(). + # Implicitly also tests resetwarnings(). + with original_warnings.catch_warnings(record=True, + module=self.module) as w: + self.module.filterwarnings("error", "", Warning, "", 0) + self.assertRaises(UserWarning, self.module.warn, 'convert to error') + + self.module.resetwarnings() + text = 'handle normally' + self.module.warn(text) + self.assertEqual(str(w[-1].message), text) + self.assertTrue(w[-1].category is UserWarning) + + self.module.filterwarnings("ignore", "", Warning, "", 0) + text = 'filtered out' + self.module.warn(text) + self.assertNotEqual(str(w[-1].message), text) + + self.module.resetwarnings() + self.module.filterwarnings("error", "hex*", Warning, "", 0) + self.assertRaises(UserWarning, self.module.warn, 'hex/oct') + text = 'nonmatching text' + self.module.warn(text) + self.assertEqual(str(w[-1].message), text) + self.assertTrue(w[-1].category is UserWarning) + + def test_mutate_filter_list(self): + class X: + def match(self, a): + L[:] = [] + + L = [("default",X(),UserWarning,X(),0) for i in range(2)] + with original_warnings.catch_warnings(record=True, + module=self.module) as w: + self.module.filters = L + self.module.warn_explicit(UserWarning("b"), None, "f.py", 42) + self.assertEqual(str(w[-1].message), "b") + +class CFilterTests(FilterTests, unittest.TestCase): + module = c_warnings + +class PyFilterTests(FilterTests, unittest.TestCase): + module = py_warnings + + +class WarnTests(BaseTest): + + """Test warnings.warn() and warnings.warn_explicit().""" + + def test_message(self): + with original_warnings.catch_warnings(record=True, + module=self.module) as w: + self.module.simplefilter("once") + for i in range(4): + text = 'multi %d' %i # Different text on each call. + self.module.warn(text) + self.assertEqual(str(w[-1].message), text) + self.assertTrue(w[-1].category is UserWarning) + + # Issue 3639 + def test_warn_nonstandard_types(self): + # warn() should handle non-standard types without issue. + for ob in (Warning, None, 42): + with original_warnings.catch_warnings(record=True, + module=self.module) as w: + self.module.simplefilter("once") + self.module.warn(ob) + # Don't directly compare objects since + # ``Warning() != Warning()``. + self.assertEqual(str(w[-1].message), str(UserWarning(ob))) + + def test_filename(self): + with warnings_state(self.module): + with original_warnings.catch_warnings(record=True, + module=self.module) as w: + warning_tests.inner("spam1") + self.assertEqual(os.path.basename(w[-1].filename), + "stacklevel.py") + warning_tests.outer("spam2") + self.assertEqual(os.path.basename(w[-1].filename), + "stacklevel.py") + + def test_stacklevel(self): + # Test stacklevel argument + # make sure all messages are different, so the warning won't be skipped + with warnings_state(self.module): + with original_warnings.catch_warnings(record=True, + module=self.module) as w: + warning_tests.inner("spam3", stacklevel=1) + self.assertEqual(os.path.basename(w[-1].filename), + "stacklevel.py") + warning_tests.outer("spam4", stacklevel=1) + self.assertEqual(os.path.basename(w[-1].filename), + "stacklevel.py") + + warning_tests.inner("spam5", stacklevel=2) + self.assertEqual(os.path.basename(w[-1].filename), + "__init__.py") + warning_tests.outer("spam6", stacklevel=2) + self.assertEqual(os.path.basename(w[-1].filename), + "stacklevel.py") + warning_tests.outer("spam6.5", stacklevel=3) + self.assertEqual(os.path.basename(w[-1].filename), + "__init__.py") + + warning_tests.inner("spam7", stacklevel=9999) + self.assertEqual(os.path.basename(w[-1].filename), + "sys") + + def test_stacklevel_import(self): + # Issue #24305: With stacklevel=2, module-level warnings should work. + support.unload('test.test_warnings.data.import_warning') + with warnings_state(self.module): + with original_warnings.catch_warnings(record=True, + module=self.module) as w: + self.module.simplefilter('always') + import test.test_warnings.data.import_warning + self.assertEqual(len(w), 1) + self.assertEqual(w[0].filename, __file__) + + def test_missing_filename_not_main(self): + # If __file__ is not specified and __main__ is not the module name, + # then __file__ should be set to the module name. + filename = warning_tests.__file__ + try: + del warning_tests.__file__ + with warnings_state(self.module): + with original_warnings.catch_warnings(record=True, + module=self.module) as w: + warning_tests.inner("spam8", stacklevel=1) + self.assertEqual(w[-1].filename, warning_tests.__name__) + finally: + warning_tests.__file__ = filename + + @unittest.skipUnless(hasattr(sys, 'argv'), 'test needs sys.argv') + def test_missing_filename_main_with_argv(self): + # If __file__ is not specified and the caller is __main__ and sys.argv + # exists, then use sys.argv[0] as the file. + filename = warning_tests.__file__ + module_name = warning_tests.__name__ + try: + del warning_tests.__file__ + warning_tests.__name__ = '__main__' + with warnings_state(self.module): + with original_warnings.catch_warnings(record=True, + module=self.module) as w: + warning_tests.inner('spam9', stacklevel=1) + self.assertEqual(w[-1].filename, sys.argv[0]) + finally: + warning_tests.__file__ = filename + warning_tests.__name__ = module_name + + def test_missing_filename_main_without_argv(self): + # If __file__ is not specified, the caller is __main__, and sys.argv + # is not set, then '__main__' is the file name. + filename = warning_tests.__file__ + module_name = warning_tests.__name__ + argv = sys.argv + try: + del warning_tests.__file__ + warning_tests.__name__ = '__main__' + del sys.argv + with warnings_state(self.module): + with original_warnings.catch_warnings(record=True, + module=self.module) as w: + warning_tests.inner('spam10', stacklevel=1) + self.assertEqual(w[-1].filename, '__main__') + finally: + warning_tests.__file__ = filename + warning_tests.__name__ = module_name + sys.argv = argv + + def test_missing_filename_main_with_argv_empty_string(self): + # If __file__ is not specified, the caller is __main__, and sys.argv[0] + # is the empty string, then '__main__ is the file name. + # Tests issue 2743. + file_name = warning_tests.__file__ + module_name = warning_tests.__name__ + argv = sys.argv + try: + del warning_tests.__file__ + warning_tests.__name__ = '__main__' + sys.argv = [''] + with warnings_state(self.module): + with original_warnings.catch_warnings(record=True, + module=self.module) as w: + warning_tests.inner('spam11', stacklevel=1) + self.assertEqual(w[-1].filename, '__main__') + finally: + warning_tests.__file__ = file_name + warning_tests.__name__ = module_name + sys.argv = argv + + def test_warn_explicit_non_ascii_filename(self): + with original_warnings.catch_warnings(record=True, + module=self.module) as w: + self.module.resetwarnings() + self.module.filterwarnings("always", category=UserWarning) + for filename in ("nonascii\xe9\u20ac", "surrogate\udc80"): + try: + os.fsencode(filename) + except UnicodeEncodeError: + continue + self.module.warn_explicit("text", UserWarning, filename, 1) + self.assertEqual(w[-1].filename, filename) + + def test_warn_explicit_type_errors(self): + # warn_explicit() should error out gracefully if it is given objects + # of the wrong types. + # lineno is expected to be an integer. + self.assertRaises(TypeError, self.module.warn_explicit, + None, UserWarning, None, None) + # Either 'message' needs to be an instance of Warning or 'category' + # needs to be a subclass. + self.assertRaises(TypeError, self.module.warn_explicit, + None, None, None, 1) + # 'registry' must be a dict or None. + self.assertRaises((TypeError, AttributeError), + self.module.warn_explicit, + None, Warning, None, 1, registry=42) + + def test_bad_str(self): + # issue 6415 + # Warnings instance with a bad format string for __str__ should not + # trigger a bus error. + class BadStrWarning(Warning): + """Warning with a bad format string for __str__.""" + def __str__(self): + return ("A bad formatted string %(err)" % + {"err" : "there is no %(err)s"}) + + with self.assertRaises(ValueError): + self.module.warn(BadStrWarning()) + + def test_warning_classes(self): + class MyWarningClass(Warning): + pass + + class NonWarningSubclass: + pass + + # passing a non-subclass of Warning should raise a TypeError + with self.assertRaises(TypeError) as cm: + self.module.warn('bad warning category', '') + self.assertIn('category must be a Warning subclass, not ', + str(cm.exception)) + + with self.assertRaises(TypeError) as cm: + self.module.warn('bad warning category', NonWarningSubclass) + self.assertIn('category must be a Warning subclass, not ', + str(cm.exception)) + + # check that warning instances also raise a TypeError + with self.assertRaises(TypeError) as cm: + self.module.warn('bad warning category', MyWarningClass()) + self.assertIn('category must be a Warning subclass, not ', + str(cm.exception)) + + with original_warnings.catch_warnings(module=self.module): + self.module.resetwarnings() + self.module.filterwarnings('default') + with self.assertWarns(MyWarningClass) as cm: + self.module.warn('good warning category', MyWarningClass) + self.assertEqual('good warning category', str(cm.warning)) + + with self.assertWarns(UserWarning) as cm: + self.module.warn('good warning category', None) + self.assertEqual('good warning category', str(cm.warning)) + + with self.assertWarns(MyWarningClass) as cm: + self.module.warn('good warning category', MyWarningClass) + self.assertIsInstance(cm.warning, Warning) + +class CWarnTests(WarnTests, unittest.TestCase): + module = c_warnings + + # As an early adopter, we sanity check the + # test.support.import_fresh_module utility function + def test_accelerated(self): + self.assertFalse(original_warnings is self.module) + self.assertFalse(hasattr(self.module.warn, '__code__')) + +class PyWarnTests(WarnTests, unittest.TestCase): + module = py_warnings + + # As an early adopter, we sanity check the + # test.support.import_fresh_module utility function + def test_pure_python(self): + self.assertFalse(original_warnings is self.module) + self.assertTrue(hasattr(self.module.warn, '__code__')) + + +class WCmdLineTests(BaseTest): + + def test_improper_input(self): + # Uses the private _setoption() function to test the parsing + # of command-line warning arguments + with original_warnings.catch_warnings(module=self.module): + self.assertRaises(self.module._OptionError, + self.module._setoption, '1:2:3:4:5:6') + self.assertRaises(self.module._OptionError, + self.module._setoption, 'bogus::Warning') + self.assertRaises(self.module._OptionError, + self.module._setoption, 'ignore:2::4:-5') + self.module._setoption('error::Warning::0') + self.assertRaises(UserWarning, self.module.warn, 'convert to error') + + def test_improper_option(self): + # Same as above, but check that the message is printed out when + # the interpreter is executed. This also checks that options are + # actually parsed at all. + rc, out, err = assert_python_ok("-Wxxx", "-c", "pass") + self.assertIn(b"Invalid -W option ignored: invalid action: 'xxx'", err) + + def test_warnings_bootstrap(self): + # Check that the warnings module does get loaded when -W + # is used (see issue #10372 for an example of silent bootstrap failure). + rc, out, err = assert_python_ok("-Wi", "-c", + "import sys; sys.modules['warnings'].warn('foo', RuntimeWarning)") + # '-Wi' was observed + self.assertFalse(out.strip()) + self.assertNotIn(b'RuntimeWarning', err) + +class CWCmdLineTests(WCmdLineTests, unittest.TestCase): + module = c_warnings + +class PyWCmdLineTests(WCmdLineTests, unittest.TestCase): + module = py_warnings + + +class _WarningsTests(BaseTest, unittest.TestCase): + + """Tests specific to the _warnings module.""" + + module = c_warnings + + def test_filter(self): + # Everything should function even if 'filters' is not in warnings. + with original_warnings.catch_warnings(module=self.module) as w: + self.module.filterwarnings("error", "", Warning, "", 0) + self.assertRaises(UserWarning, self.module.warn, + 'convert to error') + del self.module.filters + self.assertRaises(UserWarning, self.module.warn, + 'convert to error') + + def test_onceregistry(self): + # Replacing or removing the onceregistry should be okay. + global __warningregistry__ + message = UserWarning('onceregistry test') + try: + original_registry = self.module.onceregistry + __warningregistry__ = {} + with original_warnings.catch_warnings(record=True, + module=self.module) as w: + self.module.resetwarnings() + self.module.filterwarnings("once", category=UserWarning) + self.module.warn_explicit(message, UserWarning, "file", 42) + self.assertEqual(w[-1].message, message) + del w[:] + self.module.warn_explicit(message, UserWarning, "file", 42) + self.assertEqual(len(w), 0) + # Test the resetting of onceregistry. + self.module.onceregistry = {} + __warningregistry__ = {} + self.module.warn('onceregistry test') + self.assertEqual(w[-1].message.args, message.args) + # Removal of onceregistry is okay. + del w[:] + del self.module.onceregistry + __warningregistry__ = {} + self.module.warn_explicit(message, UserWarning, "file", 42) + self.assertEqual(len(w), 0) + finally: + self.module.onceregistry = original_registry + + def test_default_action(self): + # Replacing or removing defaultaction should be okay. + message = UserWarning("defaultaction test") + original = self.module.defaultaction + try: + with original_warnings.catch_warnings(record=True, + module=self.module) as w: + self.module.resetwarnings() + registry = {} + self.module.warn_explicit(message, UserWarning, "", 42, + registry=registry) + self.assertEqual(w[-1].message, message) + self.assertEqual(len(w), 1) + # One actual registry key plus the "version" key + self.assertEqual(len(registry), 2) + self.assertIn("version", registry) + del w[:] + # Test removal. + del self.module.defaultaction + __warningregistry__ = {} + registry = {} + self.module.warn_explicit(message, UserWarning, "", 43, + registry=registry) + self.assertEqual(w[-1].message, message) + self.assertEqual(len(w), 1) + self.assertEqual(len(registry), 2) + del w[:] + # Test setting. + self.module.defaultaction = "ignore" + __warningregistry__ = {} + registry = {} + self.module.warn_explicit(message, UserWarning, "", 44, + registry=registry) + self.assertEqual(len(w), 0) + finally: + self.module.defaultaction = original + + def test_showwarning_missing(self): + # Test that showwarning() missing is okay. + text = 'del showwarning test' + with original_warnings.catch_warnings(module=self.module): + self.module.filterwarnings("always", category=UserWarning) + del self.module.showwarning + with support.captured_output('stderr') as stream: + self.module.warn(text) + result = stream.getvalue() + self.assertIn(text, result) + + def test_showwarning_not_callable(self): + with original_warnings.catch_warnings(module=self.module): + self.module.filterwarnings("always", category=UserWarning) + self.module.showwarning = print + with support.captured_output('stdout'): + self.module.warn('Warning!') + self.module.showwarning = 23 + self.assertRaises(TypeError, self.module.warn, "Warning!") + + def test_show_warning_output(self): + # With showarning() missing, make sure that output is okay. + text = 'test show_warning' + with original_warnings.catch_warnings(module=self.module): + self.module.filterwarnings("always", category=UserWarning) + del self.module.showwarning + with support.captured_output('stderr') as stream: + warning_tests.inner(text) + result = stream.getvalue() + self.assertEqual(result.count('\n'), 2, + "Too many newlines in %r" % result) + first_line, second_line = result.split('\n', 1) + expected_file = os.path.splitext(warning_tests.__file__)[0] + '.py' + first_line_parts = first_line.rsplit(':', 3) + path, line, warning_class, message = first_line_parts + line = int(line) + self.assertEqual(expected_file, path) + self.assertEqual(warning_class, ' ' + UserWarning.__name__) + self.assertEqual(message, ' ' + text) + expected_line = ' ' + linecache.getline(path, line).strip() + '\n' + assert expected_line + self.assertEqual(second_line, expected_line) + + def test_filename_none(self): + # issue #12467: race condition if a warning is emitted at shutdown + globals_dict = globals() + oldfile = globals_dict['__file__'] + try: + catch = original_warnings.catch_warnings(record=True, + module=self.module) + with catch as w: + self.module.filterwarnings("always", category=UserWarning) + globals_dict['__file__'] = None + original_warnings.warn('test', UserWarning) + self.assertTrue(len(w)) + finally: + globals_dict['__file__'] = oldfile + + def test_stderr_none(self): + rc, stdout, stderr = assert_python_ok("-c", + "import sys; sys.stderr = None; " + "import warnings; warnings.simplefilter('always'); " + "warnings.warn('Warning!')") + self.assertEqual(stdout, b'') + self.assertNotIn(b'Warning!', stderr) + self.assertNotIn(b'Error', stderr) + + +class WarningsDisplayTests(BaseTest): + + """Test the displaying of warnings and the ability to overload functions + related to displaying warnings.""" + + def test_formatwarning(self): + message = "msg" + category = Warning + file_name = os.path.splitext(warning_tests.__file__)[0] + '.py' + line_num = 3 + file_line = linecache.getline(file_name, line_num).strip() + format = "%s:%s: %s: %s\n %s\n" + expect = format % (file_name, line_num, category.__name__, message, + file_line) + self.assertEqual(expect, self.module.formatwarning(message, + category, file_name, line_num)) + # Test the 'line' argument. + file_line += " for the win!" + expect = format % (file_name, line_num, category.__name__, message, + file_line) + self.assertEqual(expect, self.module.formatwarning(message, + category, file_name, line_num, file_line)) + + def test_showwarning(self): + file_name = os.path.splitext(warning_tests.__file__)[0] + '.py' + line_num = 3 + expected_file_line = linecache.getline(file_name, line_num).strip() + message = 'msg' + category = Warning + file_object = StringIO() + expect = self.module.formatwarning(message, category, file_name, + line_num) + self.module.showwarning(message, category, file_name, line_num, + file_object) + self.assertEqual(file_object.getvalue(), expect) + # Test 'line' argument. + expected_file_line += "for the win!" + expect = self.module.formatwarning(message, category, file_name, + line_num, expected_file_line) + file_object = StringIO() + self.module.showwarning(message, category, file_name, line_num, + file_object, expected_file_line) + self.assertEqual(expect, file_object.getvalue()) + +class CWarningsDisplayTests(WarningsDisplayTests, unittest.TestCase): + module = c_warnings + +class PyWarningsDisplayTests(WarningsDisplayTests, unittest.TestCase): + module = py_warnings + + +class CatchWarningTests(BaseTest): + + """Test catch_warnings().""" + + def test_catch_warnings_restore(self): + wmod = self.module + orig_filters = wmod.filters + orig_showwarning = wmod.showwarning + # Ensure both showwarning and filters are restored when recording + with wmod.catch_warnings(module=wmod, record=True): + wmod.filters = wmod.showwarning = object() + self.assertTrue(wmod.filters is orig_filters) + self.assertTrue(wmod.showwarning is orig_showwarning) + # Same test, but with recording disabled + with wmod.catch_warnings(module=wmod, record=False): + wmod.filters = wmod.showwarning = object() + self.assertTrue(wmod.filters is orig_filters) + self.assertTrue(wmod.showwarning is orig_showwarning) + + def test_catch_warnings_recording(self): + wmod = self.module + # Ensure warnings are recorded when requested + with wmod.catch_warnings(module=wmod, record=True) as w: + self.assertEqual(w, []) + self.assertTrue(type(w) is list) + wmod.simplefilter("always") + wmod.warn("foo") + self.assertEqual(str(w[-1].message), "foo") + wmod.warn("bar") + self.assertEqual(str(w[-1].message), "bar") + self.assertEqual(str(w[0].message), "foo") + self.assertEqual(str(w[1].message), "bar") + del w[:] + self.assertEqual(w, []) + # Ensure warnings are not recorded when not requested + orig_showwarning = wmod.showwarning + with wmod.catch_warnings(module=wmod, record=False) as w: + self.assertTrue(w is None) + self.assertTrue(wmod.showwarning is orig_showwarning) + + def test_catch_warnings_reentry_guard(self): + wmod = self.module + # Ensure catch_warnings is protected against incorrect usage + x = wmod.catch_warnings(module=wmod, record=True) + self.assertRaises(RuntimeError, x.__exit__) + with x: + self.assertRaises(RuntimeError, x.__enter__) + # Same test, but with recording disabled + x = wmod.catch_warnings(module=wmod, record=False) + self.assertRaises(RuntimeError, x.__exit__) + with x: + self.assertRaises(RuntimeError, x.__enter__) + + def test_catch_warnings_defaults(self): + wmod = self.module + orig_filters = wmod.filters + orig_showwarning = wmod.showwarning + # Ensure default behaviour is not to record warnings + with wmod.catch_warnings(module=wmod) as w: + self.assertTrue(w is None) + self.assertTrue(wmod.showwarning is orig_showwarning) + self.assertTrue(wmod.filters is not orig_filters) + self.assertTrue(wmod.filters is orig_filters) + if wmod is sys.modules['warnings']: + # Ensure the default module is this one + with wmod.catch_warnings() as w: + self.assertTrue(w is None) + self.assertTrue(wmod.showwarning is orig_showwarning) + self.assertTrue(wmod.filters is not orig_filters) + self.assertTrue(wmod.filters is orig_filters) + + def test_check_warnings(self): + # Explicit tests for the test.support convenience wrapper + wmod = self.module + if wmod is not sys.modules['warnings']: + self.skipTest('module to test is not loaded warnings module') + with support.check_warnings(quiet=False) as w: + self.assertEqual(w.warnings, []) + wmod.simplefilter("always") + wmod.warn("foo") + self.assertEqual(str(w.message), "foo") + wmod.warn("bar") + self.assertEqual(str(w.message), "bar") + self.assertEqual(str(w.warnings[0].message), "foo") + self.assertEqual(str(w.warnings[1].message), "bar") + w.reset() + self.assertEqual(w.warnings, []) + + with support.check_warnings(): + # defaults to quiet=True without argument + pass + with support.check_warnings(('foo', UserWarning)): + wmod.warn("foo") + + with self.assertRaises(AssertionError): + with support.check_warnings(('', RuntimeWarning)): + # defaults to quiet=False with argument + pass + with self.assertRaises(AssertionError): + with support.check_warnings(('foo', RuntimeWarning)): + wmod.warn("foo") + +class CCatchWarningTests(CatchWarningTests, unittest.TestCase): + module = c_warnings + +class PyCatchWarningTests(CatchWarningTests, unittest.TestCase): + module = py_warnings + + +class EnvironmentVariableTests(BaseTest): + + def test_single_warning(self): + rc, stdout, stderr = assert_python_ok("-c", + "import sys; sys.stdout.write(str(sys.warnoptions))", + PYTHONWARNINGS="ignore::DeprecationWarning") + self.assertEqual(stdout, b"['ignore::DeprecationWarning']") + + def test_comma_separated_warnings(self): + rc, stdout, stderr = assert_python_ok("-c", + "import sys; sys.stdout.write(str(sys.warnoptions))", + PYTHONWARNINGS="ignore::DeprecationWarning,ignore::UnicodeWarning") + self.assertEqual(stdout, + b"['ignore::DeprecationWarning', 'ignore::UnicodeWarning']") + + def test_envvar_and_command_line(self): + rc, stdout, stderr = assert_python_ok("-Wignore::UnicodeWarning", "-c", + "import sys; sys.stdout.write(str(sys.warnoptions))", + PYTHONWARNINGS="ignore::DeprecationWarning") + self.assertEqual(stdout, + b"['ignore::DeprecationWarning', 'ignore::UnicodeWarning']") + + def test_conflicting_envvar_and_command_line(self): + rc, stdout, stderr = assert_python_failure("-Werror::DeprecationWarning", "-c", + "import sys, warnings; sys.stdout.write(str(sys.warnoptions)); " + "warnings.warn('Message', DeprecationWarning)", + PYTHONWARNINGS="default::DeprecationWarning") + self.assertEqual(stdout, + b"['default::DeprecationWarning', 'error::DeprecationWarning']") + self.assertEqual(stderr.splitlines(), + [b"Traceback (most recent call last):", + b" File \"\", line 1, in ", + b"DeprecationWarning: Message"]) + + @unittest.skipUnless(sys.getfilesystemencoding() != 'ascii', + 'requires non-ascii filesystemencoding') + def test_nonascii(self): + rc, stdout, stderr = assert_python_ok("-c", + "import sys; sys.stdout.write(str(sys.warnoptions))", + PYTHONIOENCODING="utf-8", + PYTHONWARNINGS="ignore:Deprecaci?nWarning") + self.assertEqual(stdout, + "['ignore:Deprecaci?nWarning']".encode('utf-8')) + +class CEnvironmentVariableTests(EnvironmentVariableTests, unittest.TestCase): + module = c_warnings + +class PyEnvironmentVariableTests(EnvironmentVariableTests, unittest.TestCase): + module = py_warnings + + +class BootstrapTest(unittest.TestCase): + def test_issue_8766(self): + # "import encodings" emits a warning whereas the warnings is not loaded + # or not completely loaded (warnings imports indirectly encodings by + # importing linecache) yet + with support.temp_cwd() as cwd, support.temp_cwd('encodings'): + # encodings loaded by initfsencoding() + assert_python_ok('-c', 'pass', PYTHONPATH=cwd) + + # Use -W to load warnings module at startup + assert_python_ok('-c', 'pass', '-W', 'always', PYTHONPATH=cwd) + +class FinalizationTest(unittest.TestCase): + def test_finalization(self): + # Issue #19421: warnings.warn() should not crash + # during Python finalization + code = """ +import warnings +warn = warnings.warn + +class A: + def __del__(self): + warn("test") + +a=A() + """ + rc, out, err = assert_python_ok("-c", code) + # note: "__main__" filename is not correct, it should be the name + # of the script + self.assertEqual(err, b'__main__:7: UserWarning: test') + + +def setUpModule(): + py_warnings.onceregistry.clear() + c_warnings.onceregistry.clear() + +tearDownModule = setUpModule + +if __name__ == "__main__": + unittest.main() diff --git a/Lib/test/test_warnings/__main__.py b/Lib/test/test_warnings/__main__.py new file mode 100644 --- /dev/null +++ b/Lib/test/test_warnings/__main__.py @@ -0,0 +1,3 @@ +import unittest + +unittest.main('test.test_warnings') diff --git a/Lib/test/test_warnings/data/import_warning.py b/Lib/test/test_warnings/data/import_warning.py new file mode 100644 --- /dev/null +++ b/Lib/test/test_warnings/data/import_warning.py @@ -0,0 +1,3 @@ +import warnings + +warnings.warn('module-level warning', DeprecationWarning, stacklevel=2) diff --git a/Lib/test/warning_tests.py b/Lib/test/test_warnings/data/stacklevel.py rename from Lib/test/warning_tests.py rename to Lib/test/test_warnings/data/stacklevel.py diff --git a/Lib/warnings.py b/Lib/warnings.py --- a/Lib/warnings.py +++ b/Lib/warnings.py @@ -160,6 +160,20 @@ return cat +def _is_internal_frame(frame): + """Signal whether the frame is an internal CPython implementation detail.""" + filename = frame.f_code.co_filename + return 'importlib' in filename and '_bootstrap' in filename + + +def _next_external_frame(frame): + """Find the next frame that doesn't involve CPython internals.""" + frame = frame.f_back + while frame is not None and _is_internal_frame(frame): + frame = frame.f_back + return frame + + # Code typically replaced by _warnings def warn(message, category=None, stacklevel=1): """Issue a warning, or maybe ignore it or raise an exception.""" @@ -174,13 +188,23 @@ "not '{:s}'".format(type(category).__name__)) # Get context information try: - caller = sys._getframe(stacklevel) + if stacklevel <= 1 or _is_internal_frame(sys._getframe(1)): + # If frame is too small to care or if the warning originated in + # internal code, then do not try to hide any frames. + frame = sys._getframe(stacklevel) + else: + frame = sys._getframe(1) + # Look for one frame less since the above line starts us off. + for x in range(stacklevel-1): + frame = _next_external_frame(frame) + if frame is None: + raise ValueError except ValueError: globals = sys.__dict__ lineno = 1 else: - globals = caller.f_globals - lineno = caller.f_lineno + globals = frame.f_globals + lineno = frame.f_lineno if '__name__' in globals: module = globals['__name__'] else: @@ -374,7 +398,6 @@ defaultaction = _defaultaction onceregistry = _onceregistry _warnings_defaults = True - except ImportError: filters = [] defaultaction = "default" diff --git a/Lib/webbrowser.py b/Lib/webbrowser.py --- a/Lib/webbrowser.py +++ b/Lib/webbrowser.py @@ -495,23 +495,10 @@ # if sys.platform[:3] == "win": - class WindowsDefault(BaseBrowser): - # Windows Default opening arguments. - - cmd = "start" - newwindow = "" - newtab = "" - def open(self, url, new=0, autoraise=True): - # Format the command for optional arguments and add the url. - if new == 1: - self.cmd += " " + self.newwindow - elif new == 2: - self.cmd += " " + self.newtab - self.cmd += " " + url try: - subprocess.call(self.cmd, shell=True) + os.startfile(url) except OSError: # [Error 22] No application is associated with the specified # file for this operation: '' @@ -519,108 +506,19 @@ else: return True - - # Windows Sub-Classes for commonly used browsers. - - class InternetExplorer(WindowsDefault): - """Launcher class for Internet Explorer browser""" - - cmd = "start iexplore.exe" - newwindow = "" - newtab = "" - - - class WinChrome(WindowsDefault): - """Launcher class for windows specific Google Chrome browser""" - - cmd = "start chrome.exe" - newwindow = "-new-window" - newtab = "-new-tab" - - - class WinFirefox(WindowsDefault): - """Launcher class for windows specific Firefox browser""" - - cmd = "start firefox.exe" - newwindow = "-new-window" - newtab = "-new-tab" - - - class WinOpera(WindowsDefault): - """Launcher class for windows specific Opera browser""" - - cmd = "start opera" - newwindow = "" - newtab = "" - - - class WinSeaMonkey(WindowsDefault): - """Launcher class for windows specific SeaMonkey browser""" - - cmd = "start seamonkey" - newwinow = "" - newtab = "" - - _tryorder = [] _browsers = {} - # First try to use the default Windows browser. + # First try to use the default Windows browser register("windows-default", WindowsDefault) - def find_windows_browsers(): - """ Access the windows registry to determine - what browsers are on the system. - """ - - import winreg - HKLM = winreg.HKEY_LOCAL_MACHINE - subkey = r'Software\Clients\StartMenuInternet' - read32 = winreg.KEY_READ | winreg.KEY_WOW64_32KEY - read64 = winreg.KEY_READ | winreg.KEY_WOW64_64KEY - key32 = winreg.OpenKey(HKLM, subkey, access=read32) - key64 = winreg.OpenKey(HKLM, subkey, access=read64) - - # Return a list of browsers found in the registry - # Check if there are any different browsers in the - # 32 bit location instead of the 64 bit location. - browsers = [] - i = 0 - while True: - try: - browsers.append(winreg.EnumKey(key32, i)) - except EnvironmentError: - break - i += 1 - - i = 0 - while True: - try: - browsers.append(winreg.EnumKey(key64, i)) - except EnvironmentError: - break - i += 1 - - winreg.CloseKey(key32) - winreg.CloseKey(key64) - - return browsers - - # Detect some common windows browsers - for browser in find_windows_browsers(): - browser = browser.lower() - if "iexplore" in browser: - register("iexplore", None, InternetExplorer("iexplore")) - elif "chrome" in browser: - register("chrome", None, WinChrome("chrome")) - elif "firefox" in browser: - register("firefox", None, WinFirefox("firefox")) - elif "opera" in browser: - register("opera", None, WinOpera("opera")) - elif "seamonkey" in browser: - register("seamonkey", None, WinSeaMonkey("seamonkey")) - else: - register(browser, None, WindowsDefault(browser)) + # Detect some common Windows browsers, fallback to IE + iexplore = os.path.join(os.environ.get("PROGRAMFILES", "C:\\Program Files"), + "Internet Explorer\\IEXPLORE.EXE") + for browser in ("firefox", "firebird", "seamonkey", "mozilla", + "netscape", "opera", iexplore): + if shutil.which(browser): + register(browser, None, BackgroundBrowser(browser)) # # Platform support for MacOS diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -1,4 +1,4 @@ -?+++++++++++ ++++++++++++ Python News +++++++++++ @@ -174,6 +174,9 @@ Core and Builtins ----------------- +- Issue #24305: Prevent import subsystem stack frames from being counted + by the warnings.warn(stacklevel=) parameter. + - Issue #24912: Prevent __class__ assignment to immutable built-in objects. - Issue #24975: Fix AST compilation for PEP 448 syntax. @@ -181,9 +184,15 @@ Library ------- +- Issue #24917: time_strftime() buffer over-read. - Issue #23144: Make sure that HTMLParser.feed() returns all the data, even when convert_charrefs is True. +- Issue #24748: To resolve a compatibility problem found with py2exe and + pywin32, imp.load_dynamic() once again ignores previously loaded modules + to support Python modules replacing themselves with extension modules. + Patch by Petr Viktorin. + - Issue #24635: Fixed a bug in typing.py where isinstance([], typing.Iterable) would return True once, then False on subsequent calls. @@ -468,9 +477,6 @@ - Issue #14373: C implementation of functools.lru_cache() now can be used with methods. -- Issue #8232: webbrowser support incomplete on Windows. Patch by Brandon - Milam - - Issue #24347: Set KeyError if PyDict_GetItemWithError returns NULL. - Issue #24348: Drop superfluous incref/decref. diff --git a/Modules/_testmultiphase.c b/Modules/_testmultiphase.c --- a/Modules/_testmultiphase.c +++ b/Modules/_testmultiphase.c @@ -582,3 +582,13 @@ { return PyModuleDef_Init(&def_exec_unreported_exception); } + +/*** Helper for imp test ***/ + +static PyModuleDef imp_dummy_def = TEST_MODULE_DEF("imp_dummy", main_slots, testexport_methods); + +PyMODINIT_FUNC +PyInit_imp_dummy(PyObject *spec) +{ + return PyModuleDef_Init(&imp_dummy_def); +} diff --git a/Modules/timemodule.c b/Modules/timemodule.c --- a/Modules/timemodule.c +++ b/Modules/timemodule.c @@ -610,14 +610,15 @@ #if defined(MS_WINDOWS) && !defined(HAVE_WCSFTIME) /* check that the format string contains only valid directives */ - for(outbuf = strchr(fmt, '%'); + for (outbuf = strchr(fmt, '%'); outbuf != NULL; outbuf = strchr(outbuf+2, '%')) { - if (outbuf[1]=='#') + if (outbuf[1] == '#') ++outbuf; /* not documented by python, */ - if ((outbuf[1] == 'y') && buf.tm_year < 0) - { + if (outbuf[1] == '\0') + break; + if ((outbuf[1] == 'y') && buf.tm_year < 0) { PyErr_SetString(PyExc_ValueError, "format %y requires year >= 1900 on Windows"); Py_DECREF(format); @@ -625,10 +626,12 @@ } } #elif (defined(_AIX) || defined(sun)) && defined(HAVE_WCSFTIME) - for(outbuf = wcschr(fmt, '%'); + for (outbuf = wcschr(fmt, '%'); outbuf != NULL; outbuf = wcschr(outbuf+2, '%')) { + if (outbuf[1] == L'\0') + break; /* Issue #19634: On AIX, wcsftime("y", (1899, 1, 1, 0, 0, 0, 0, 0, 0)) returns "0/" instead of "99" */ if (outbuf[1] == L'y' && buf.tm_year < 0) { @@ -659,7 +662,8 @@ #if defined _MSC_VER && _MSC_VER >= 1400 && defined(__STDC_SECURE_LIB__) err = errno; #endif - if (buflen > 0 || i >= 256 * fmtlen) { + if (buflen > 0 || fmtlen == 0 || + (fmtlen > 4 && i >= 256 * fmtlen)) { /* If the buffer is 256 times as long as the format, it's probably not failing for lack of room! More likely, the format yields an empty result, diff --git a/Python/_warnings.c b/Python/_warnings.c --- a/Python/_warnings.c +++ b/Python/_warnings.c @@ -513,6 +513,64 @@ return result; /* Py_None or NULL. */ } +static int +is_internal_frame(PyFrameObject *frame) +{ + static PyObject *importlib_string = NULL; + static PyObject *bootstrap_string = NULL; + PyObject *filename; + int contains; + + if (importlib_string == NULL) { + importlib_string = PyUnicode_FromString("importlib"); + if (importlib_string == NULL) { + return 0; + } + + bootstrap_string = PyUnicode_FromString("_bootstrap"); + if (bootstrap_string == NULL) { + Py_DECREF(importlib_string); + return 0; + } + Py_INCREF(importlib_string); + Py_INCREF(bootstrap_string); + } + + if (frame == NULL || frame->f_code == NULL || + frame->f_code->co_filename == NULL) { + return 0; + } + filename = frame->f_code->co_filename; + if (!PyUnicode_Check(filename)) { + return 0; + } + contains = PyUnicode_Contains(filename, importlib_string); + if (contains < 0) { + return 0; + } + else if (contains > 0) { + contains = PyUnicode_Contains(filename, bootstrap_string); + if (contains < 0) { + return 0; + } + else if (contains > 0) { + return 1; + } + } + + return 0; +} + +static PyFrameObject * +next_external_frame(PyFrameObject *frame) +{ + do { + frame = frame->f_back; + } while (frame != NULL && is_internal_frame(frame)); + + return frame; +} + /* filename, module, and registry are new refs, globals is borrowed */ /* Returns 0 on error (no new refs), 1 on success */ static int @@ -523,8 +581,18 @@ /* Setup globals and lineno. */ PyFrameObject *f = PyThreadState_GET()->frame; - while (--stack_level > 0 && f != NULL) - f = f->f_back; + // Stack level comparisons to Python code is off by one as there is no + // warnings-related stack level to avoid. + if (stack_level <= 0 || is_internal_frame(f)) { + while (--stack_level > 0 && f != NULL) { + f = f->f_back; + } + } + else { + while (--stack_level > 0 && f != NULL) { + f = next_external_frame(f); + } + } if (f == NULL) { globals = PyThreadState_Get()->interp->sysdict; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 7 07:59:40 2015 From: python-checkins at python.org (terry.reedy) Date: Mon, 07 Sep 2015 05:59:40 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzI0ODg5?= =?utf-8?q?=3A_When_starting_Idle=2C_force_focus_onto_Idle_window_if_not_a?= =?utf-8?q?lready?= Message-ID: <20150907055940.27703.75649@psf.io> https://hg.python.org/cpython/rev/741b033c5290 changeset: 97737:741b033c5290 branch: 2.7 parent: 97716:6c222848badd user: Terry Jan Reedy date: Mon Sep 07 01:58:05 2015 -0400 summary: Issue #24889: When starting Idle, force focus onto Idle window if not already there (as when opening Idle from interactive Python on Windows). files: Lib/idlelib/PyShell.py | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diff --git a/Lib/idlelib/PyShell.py b/Lib/idlelib/PyShell.py --- a/Lib/idlelib/PyShell.py +++ b/Lib/idlelib/PyShell.py @@ -1054,6 +1054,7 @@ nosub = "==== No Subprocess ====" self.write("Python %s on %s\n%s\n%s" % (sys.version, sys.platform, self.COPYRIGHT, nosub)) + self.text.focus_force() self.showprompt() import Tkinter Tkinter._default_root = None # 03Jan04 KBK What's this? -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 7 07:59:40 2015 From: python-checkins at python.org (terry.reedy) Date: Mon, 07 Sep 2015 05:59:40 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzI0ODg5?= =?utf-8?q?=3A_When_starting_Idle=2C_force_focus_onto_Idle_window_if_not_a?= =?utf-8?q?lready?= Message-ID: <20150907055940.114719.7468@psf.io> https://hg.python.org/cpython/rev/d7449bac2c6d changeset: 97738:d7449bac2c6d branch: 3.4 parent: 97732:f185917498ca user: Terry Jan Reedy date: Mon Sep 07 01:58:13 2015 -0400 summary: Issue #24889: When starting Idle, force focus onto Idle window if not already there (as when opening Idle from interactive Python on Windows). files: Lib/idlelib/PyShell.py | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diff --git a/Lib/idlelib/PyShell.py b/Lib/idlelib/PyShell.py --- a/Lib/idlelib/PyShell.py +++ b/Lib/idlelib/PyShell.py @@ -1043,6 +1043,7 @@ self.write("Python %s on %s\n%s\n%s" % (sys.version, sys.platform, self.COPYRIGHT, nosub)) + self.text.focus_force() self.showprompt() import tkinter tkinter._default_root = None # 03Jan04 KBK What's this? -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 7 07:59:40 2015 From: python-checkins at python.org (terry.reedy) Date: Mon, 07 Sep 2015 05:59:40 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Merge_with_3=2E4?= Message-ID: <20150907055940.27711.70146@psf.io> https://hg.python.org/cpython/rev/cd7fa421e0e9 changeset: 97739:cd7fa421e0e9 branch: 3.5 parent: 97735:a11d30b8fcdf parent: 97738:d7449bac2c6d user: Terry Jan Reedy date: Mon Sep 07 01:58:29 2015 -0400 summary: Merge with 3.4 files: Lib/idlelib/PyShell.py | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diff --git a/Lib/idlelib/PyShell.py b/Lib/idlelib/PyShell.py --- a/Lib/idlelib/PyShell.py +++ b/Lib/idlelib/PyShell.py @@ -1043,6 +1043,7 @@ self.write("Python %s on %s\n%s\n%s" % (sys.version, sys.platform, self.COPYRIGHT, nosub)) + self.text.focus_force() self.showprompt() import tkinter tkinter._default_root = None # 03Jan04 KBK What's this? -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 7 07:59:41 2015 From: python-checkins at python.org (terry.reedy) Date: Mon, 07 Sep 2015 05:59:41 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge_with_3=2E5?= Message-ID: <20150907055940.11250.18675@psf.io> https://hg.python.org/cpython/rev/1c76d7bc892f changeset: 97740:1c76d7bc892f parent: 97736:1e8fe240c575 parent: 97739:cd7fa421e0e9 user: Terry Jan Reedy date: Mon Sep 07 01:58:43 2015 -0400 summary: Merge with 3.5 files: Lib/idlelib/PyShell.py | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diff --git a/Lib/idlelib/PyShell.py b/Lib/idlelib/PyShell.py --- a/Lib/idlelib/PyShell.py +++ b/Lib/idlelib/PyShell.py @@ -1043,6 +1043,7 @@ self.write("Python %s on %s\n%s\n%s" % (sys.version, sys.platform, self.COPYRIGHT, nosub)) + self.text.focus_force() self.showprompt() import tkinter tkinter._default_root = None # 03Jan04 KBK What's this? -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 7 12:58:38 2015 From: python-checkins at python.org (serhiy.storchaka) Date: Mon, 07 Sep 2015 10:58:38 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_Explicitly_tes?= =?utf-8?q?t_archive_name_in_shutil=2Emake=5Farchive=28=29_tests_to_expose?= =?utf-8?q?_failure?= Message-ID: <20150907105838.68861.5201@psf.io> https://hg.python.org/cpython/rev/7bd6b8076c48 changeset: 97741:7bd6b8076c48 branch: 3.4 parent: 97738:d7449bac2c6d user: Serhiy Storchaka date: Mon Sep 07 13:55:25 2015 +0300 summary: Explicitly test archive name in shutil.make_archive() tests to expose failure details in issue25018. files: Lib/test/test_shutil.py | 22 +++++++++++----------- 1 files changed, 11 insertions(+), 11 deletions(-) diff --git a/Lib/test/test_shutil.py b/Lib/test/test_shutil.py --- a/Lib/test/test_shutil.py +++ b/Lib/test/test_shutil.py @@ -974,10 +974,10 @@ base_name = os.path.join(tmpdir2, 'archive') # working with relative paths to avoid tar warnings - make_archive(splitdrive(base_name)[1], 'gztar', root_dir, '.') + tarball = make_archive(splitdrive(base_name)[1], 'gztar', root_dir, '.') # check if the compressed tarball was created - tarball = base_name + '.tar.gz' + self.assertEqual(tarball, base_name + '.tar.gz') self.assertTrue(os.path.isfile(tarball)) self.assertTrue(tarfile.is_tarfile(tarball)) with tarfile.open(tarball, 'r:gz') as tf: @@ -986,9 +986,8 @@ './file1', './file2', './sub/file3']) # trying an uncompressed one - base_name = os.path.join(tmpdir2, 'archive') - make_archive(splitdrive(base_name)[1], 'tar', root_dir, '.') - tarball = base_name + '.tar' + tarball = make_archive(splitdrive(base_name)[1], 'tar', root_dir, '.') + self.assertEqual(tarball, base_name + '.tar') self.assertTrue(os.path.isfile(tarball)) self.assertTrue(tarfile.is_tarfile(tarball)) with tarfile.open(tarball, 'r') as tf: @@ -1022,10 +1021,10 @@ def test_tarfile_vs_tar(self): root_dir, base_dir = self._create_files() base_name = os.path.join(self.mkdtemp(), 'archive') - make_archive(base_name, 'gztar', root_dir, base_dir) + tarball = make_archive(base_name, 'gztar', root_dir, base_dir) # check if the compressed tarball was created - tarball = base_name + '.tar.gz' + self.assertEqual(tarball, base_name + '.tar.gz') self.assertTrue(os.path.isfile(tarball)) # now create another tarball using `tar` @@ -1039,13 +1038,14 @@ self.assertEqual(self._tarinfo(tarball), self._tarinfo(tarball2)) # trying an uncompressed one - make_archive(base_name, 'tar', root_dir, base_dir) - tarball = base_name + '.tar' + tarball = make_archive(base_name, 'tar', root_dir, base_dir) + self.assertEqual(tarball, base_name + '.tar') self.assertTrue(os.path.isfile(tarball)) # now for a dry_run - make_archive(base_name, 'tar', root_dir, base_dir, dry_run=True) - tarball = base_name + '.tar' + tarball = make_archive(base_name, 'tar', root_dir, base_dir, + dry_run=True) + self.assertEqual(tarball, base_name + '.tar') self.assertTrue(os.path.isfile(tarball)) @requires_zlib -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 7 12:58:39 2015 From: python-checkins at python.org (serhiy.storchaka) Date: Mon, 07 Sep 2015 10:58:39 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Explicitly_test_archive_name_in_shutil=2Emake=5Farchive?= =?utf-8?q?=28=29_tests_to_expose_failure?= Message-ID: <20150907105839.17961.73207@psf.io> https://hg.python.org/cpython/rev/738e227d0891 changeset: 97744:738e227d0891 parent: 97740:1c76d7bc892f parent: 97743:f842c2b710ed user: Serhiy Storchaka date: Mon Sep 07 13:57:21 2015 +0300 summary: Explicitly test archive name in shutil.make_archive() tests to expose failure details in issue25018. files: Lib/test/test_shutil.py | 22 +++++++++++----------- 1 files changed, 11 insertions(+), 11 deletions(-) diff --git a/Lib/test/test_shutil.py b/Lib/test/test_shutil.py --- a/Lib/test/test_shutil.py +++ b/Lib/test/test_shutil.py @@ -980,10 +980,10 @@ base_name = os.path.join(tmpdir2, 'archive') # working with relative paths to avoid tar warnings - make_archive(splitdrive(base_name)[1], 'gztar', root_dir, '.') + tarball = make_archive(splitdrive(base_name)[1], 'gztar', root_dir, '.') # check if the compressed tarball was created - tarball = base_name + '.tar.gz' + self.assertEqual(tarball, base_name + '.tar.gz') self.assertTrue(os.path.isfile(tarball)) self.assertTrue(tarfile.is_tarfile(tarball)) with tarfile.open(tarball, 'r:gz') as tf: @@ -992,9 +992,8 @@ './file1', './file2', './sub/file3']) # trying an uncompressed one - base_name = os.path.join(tmpdir2, 'archive') - make_archive(splitdrive(base_name)[1], 'tar', root_dir, '.') - tarball = base_name + '.tar' + tarball = make_archive(splitdrive(base_name)[1], 'tar', root_dir, '.') + self.assertEqual(tarball, base_name + '.tar') self.assertTrue(os.path.isfile(tarball)) self.assertTrue(tarfile.is_tarfile(tarball)) with tarfile.open(tarball, 'r') as tf: @@ -1028,10 +1027,10 @@ def test_tarfile_vs_tar(self): root_dir, base_dir = self._create_files() base_name = os.path.join(self.mkdtemp(), 'archive') - make_archive(base_name, 'gztar', root_dir, base_dir) + tarball = make_archive(base_name, 'gztar', root_dir, base_dir) # check if the compressed tarball was created - tarball = base_name + '.tar.gz' + self.assertEqual(tarball, base_name + '.tar.gz') self.assertTrue(os.path.isfile(tarball)) # now create another tarball using `tar` @@ -1045,13 +1044,14 @@ self.assertEqual(self._tarinfo(tarball), self._tarinfo(tarball2)) # trying an uncompressed one - make_archive(base_name, 'tar', root_dir, base_dir) - tarball = base_name + '.tar' + tarball = make_archive(base_name, 'tar', root_dir, base_dir) + self.assertEqual(tarball, base_name + '.tar') self.assertTrue(os.path.isfile(tarball)) # now for a dry_run - make_archive(base_name, 'tar', root_dir, base_dir, dry_run=True) - tarball = base_name + '.tar' + tarball = make_archive(base_name, 'tar', root_dir, base_dir, + dry_run=True) + self.assertEqual(tarball, base_name + '.tar') self.assertTrue(os.path.isfile(tarball)) @requires_zlib -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 7 12:58:39 2015 From: python-checkins at python.org (serhiy.storchaka) Date: Mon, 07 Sep 2015 10:58:39 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Explicitly_test_archive_name_in_shutil=2Emake=5Farchive=28=29_?= =?utf-8?q?tests_to_expose_failure?= Message-ID: <20150907105839.115088.50421@psf.io> https://hg.python.org/cpython/rev/f842c2b710ed changeset: 97743:f842c2b710ed branch: 3.5 parent: 97739:cd7fa421e0e9 parent: 97741:7bd6b8076c48 user: Serhiy Storchaka date: Mon Sep 07 13:56:49 2015 +0300 summary: Explicitly test archive name in shutil.make_archive() tests to expose failure details in issue25018. files: Lib/test/test_shutil.py | 22 +++++++++++----------- 1 files changed, 11 insertions(+), 11 deletions(-) diff --git a/Lib/test/test_shutil.py b/Lib/test/test_shutil.py --- a/Lib/test/test_shutil.py +++ b/Lib/test/test_shutil.py @@ -980,10 +980,10 @@ base_name = os.path.join(tmpdir2, 'archive') # working with relative paths to avoid tar warnings - make_archive(splitdrive(base_name)[1], 'gztar', root_dir, '.') + tarball = make_archive(splitdrive(base_name)[1], 'gztar', root_dir, '.') # check if the compressed tarball was created - tarball = base_name + '.tar.gz' + self.assertEqual(tarball, base_name + '.tar.gz') self.assertTrue(os.path.isfile(tarball)) self.assertTrue(tarfile.is_tarfile(tarball)) with tarfile.open(tarball, 'r:gz') as tf: @@ -992,9 +992,8 @@ './file1', './file2', './sub/file3']) # trying an uncompressed one - base_name = os.path.join(tmpdir2, 'archive') - make_archive(splitdrive(base_name)[1], 'tar', root_dir, '.') - tarball = base_name + '.tar' + tarball = make_archive(splitdrive(base_name)[1], 'tar', root_dir, '.') + self.assertEqual(tarball, base_name + '.tar') self.assertTrue(os.path.isfile(tarball)) self.assertTrue(tarfile.is_tarfile(tarball)) with tarfile.open(tarball, 'r') as tf: @@ -1028,10 +1027,10 @@ def test_tarfile_vs_tar(self): root_dir, base_dir = self._create_files() base_name = os.path.join(self.mkdtemp(), 'archive') - make_archive(base_name, 'gztar', root_dir, base_dir) + tarball = make_archive(base_name, 'gztar', root_dir, base_dir) # check if the compressed tarball was created - tarball = base_name + '.tar.gz' + self.assertEqual(tarball, base_name + '.tar.gz') self.assertTrue(os.path.isfile(tarball)) # now create another tarball using `tar` @@ -1045,13 +1044,14 @@ self.assertEqual(self._tarinfo(tarball), self._tarinfo(tarball2)) # trying an uncompressed one - make_archive(base_name, 'tar', root_dir, base_dir) - tarball = base_name + '.tar' + tarball = make_archive(base_name, 'tar', root_dir, base_dir) + self.assertEqual(tarball, base_name + '.tar') self.assertTrue(os.path.isfile(tarball)) # now for a dry_run - make_archive(base_name, 'tar', root_dir, base_dir, dry_run=True) - tarball = base_name + '.tar' + tarball = make_archive(base_name, 'tar', root_dir, base_dir, + dry_run=True) + self.assertEqual(tarball, base_name + '.tar') self.assertTrue(os.path.isfile(tarball)) @requires_zlib -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 7 12:58:39 2015 From: python-checkins at python.org (serhiy.storchaka) Date: Mon, 07 Sep 2015 10:58:39 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_Explicitly_tes?= =?utf-8?q?t_archive_name_in_shutil=2Emake=5Farchive=28=29_tests_to_expose?= =?utf-8?q?_failure?= Message-ID: <20150907105838.27695.22250@psf.io> https://hg.python.org/cpython/rev/beda04bf5991 changeset: 97742:beda04bf5991 branch: 2.7 parent: 97737:741b033c5290 user: Serhiy Storchaka date: Mon Sep 07 13:55:25 2015 +0300 summary: Explicitly test archive name in shutil.make_archive() tests to expose failure details in issue25018. files: Lib/test/test_shutil.py | 22 +++++++++++----------- 1 files changed, 11 insertions(+), 11 deletions(-) diff --git a/Lib/test/test_shutil.py b/Lib/test/test_shutil.py --- a/Lib/test/test_shutil.py +++ b/Lib/test/test_shutil.py @@ -385,10 +385,10 @@ base_name = os.path.join(tmpdir2, 'archive') # working with relative paths to avoid tar warnings - make_archive(splitdrive(base_name)[1], 'gztar', root_dir, '.') + tarball = make_archive(splitdrive(base_name)[1], 'gztar', root_dir, '.') # check if the compressed tarball was created - tarball = base_name + '.tar.gz' + self.assertEqual(tarball, base_name + '.tar.gz') self.assertTrue(os.path.isfile(tarball)) self.assertTrue(tarfile.is_tarfile(tarball)) with tarfile.open(tarball, 'r:gz') as tf: @@ -397,9 +397,8 @@ './sub', './sub/file3', './sub2']) # trying an uncompressed one - base_name = os.path.join(tmpdir2, 'archive') - make_archive(splitdrive(base_name)[1], 'tar', root_dir, '.') - tarball = base_name + '.tar' + tarball = make_archive(splitdrive(base_name)[1], 'tar', root_dir, '.') + self.assertEqual(tarball, base_name + '.tar') self.assertTrue(os.path.isfile(tarball)) self.assertTrue(tarfile.is_tarfile(tarball)) with tarfile.open(tarball, 'r') as tf: @@ -434,10 +433,10 @@ def test_tarfile_vs_tar(self): root_dir, base_dir = self._create_files() base_name = os.path.join(self.mkdtemp(), 'archive') - make_archive(base_name, 'gztar', root_dir, base_dir) + tarball = make_archive(base_name, 'gztar', root_dir, base_dir) # check if the compressed tarball was created - tarball = base_name + '.tar.gz' + self.assertEqual(tarball, base_name + '.tar.gz') self.assertTrue(os.path.isfile(tarball)) # now create another tarball using `tar` @@ -451,13 +450,14 @@ self.assertEqual(self._tarinfo(tarball), self._tarinfo(tarball2)) # trying an uncompressed one - make_archive(base_name, 'tar', root_dir, base_dir) - tarball = base_name + '.tar' + tarball = make_archive(base_name, 'tar', root_dir, base_dir) + self.assertEqual(tarball, base_name + '.tar') self.assertTrue(os.path.isfile(tarball)) # now for a dry_run - make_archive(base_name, 'tar', root_dir, base_dir, dry_run=True) - tarball = base_name + '.tar' + tarball = make_archive(base_name, 'tar', root_dir, base_dir, + dry_run=True) + self.assertEqual(tarball, base_name + '.tar') self.assertTrue(os.path.isfile(tarball)) @unittest.skipUnless(zlib, "Requires zlib") -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 7 14:18:30 2015 From: python-checkins at python.org (larry.hastings) Date: Mon, 07 Sep 2015 12:18:30 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_Added_tag_v3?= =?utf-8?q?=2E5=2E0rc3_for_changeset_66ed52375df8?= Message-ID: <20150907121824.101492.43696@psf.io> https://hg.python.org/cpython/rev/170af6e33c9c changeset: 97747:170af6e33c9c branch: 3.5 user: Larry Hastings date: Mon Sep 07 05:12:28 2015 -0700 summary: Added tag v3.5.0rc3 for changeset 66ed52375df8 files: .hgtags | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diff --git a/.hgtags b/.hgtags --- a/.hgtags +++ b/.hgtags @@ -154,3 +154,4 @@ c0d64105463581f85d0e368e8d6e59b7fd8f12b1 v3.5.0b4 1a58b1227501e046eee13d90f113417b60843301 v3.5.0rc1 cc15d736d860303b9da90d43cd32db39bab048df v3.5.0rc2 +66ed52375df802f9d0a34480daaa8ce79fc41313 v3.5.0rc3 -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 7 14:18:29 2015 From: python-checkins at python.org (larry.hastings) Date: Mon, 07 Sep 2015 12:18:29 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_Version_bump_f?= =?utf-8?q?or_Python_3=2E5=2E0rc3=2E?= Message-ID: <20150907121824.68859.45169@psf.io> https://hg.python.org/cpython/rev/66ed52375df8 changeset: 97746:66ed52375df8 branch: 3.5 tag: v3.5.0rc3 user: Larry Hastings date: Mon Sep 07 05:12:05 2015 -0700 summary: Version bump for Python 3.5.0rc3. files: Include/patchlevel.h | 4 ++-- Misc/NEWS | 2 +- README | 2 +- 3 files changed, 4 insertions(+), 4 deletions(-) diff --git a/Include/patchlevel.h b/Include/patchlevel.h --- a/Include/patchlevel.h +++ b/Include/patchlevel.h @@ -20,10 +20,10 @@ #define PY_MINOR_VERSION 5 #define PY_MICRO_VERSION 0 #define PY_RELEASE_LEVEL PY_RELEASE_LEVEL_GAMMA -#define PY_RELEASE_SERIAL 2 +#define PY_RELEASE_SERIAL 3 /* Version as a string */ -#define PY_VERSION "3.5.0rc2+" +#define PY_VERSION "3.5.0rc3" /*--end constants--*/ /* Version as a single 4-byte hex number, e.g. 0x010502B2 == 1.5.2b2. diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -5,7 +5,7 @@ What's New in Python 3.5.0 release candidate 3? =============================================== -Release date: 2015-09-06 +Release date: 2015-09-07 Core and Builtins ----------------- diff --git a/README b/README --- a/README +++ b/README @@ -1,4 +1,4 @@ -This is Python version 3.5.0 release candidate 2 +This is Python version 3.5.0 release candidate 3 ================================================ Copyright (c) 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010, 2011, -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 7 14:18:30 2015 From: python-checkins at python.org (larry.hastings) Date: Mon, 07 Sep 2015 12:18:30 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_Updated_topics?= =?utf-8?q?_=28again=29_for_Python_3=2E5=2E0rc3_=28second_try=29=2E?= Message-ID: <20150907121824.68871.7719@psf.io> https://hg.python.org/cpython/rev/34990fd30cb9 changeset: 97745:34990fd30cb9 branch: 3.5 parent: 97731:7d320c3bf9c6 user: Larry Hastings date: Mon Sep 07 05:10:55 2015 -0700 summary: Updated topics (again) for Python 3.5.0rc3 (second try). files: Lib/pydoc_data/topics.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Lib/pydoc_data/topics.py b/Lib/pydoc_data/topics.py --- a/Lib/pydoc_data/topics.py +++ b/Lib/pydoc_data/topics.py @@ -1,5 +1,5 @@ # -*- coding: utf-8 -*- -# Autogenerated by Sphinx on Mon Aug 24 20:29:23 2015 +# Autogenerated by Sphinx on Mon Sep 7 05:10:25 2015 topics = {'assert': u'\nThe "assert" statement\n**********************\n\nAssert statements are a convenient way to insert debugging assertions\ninto a program:\n\n assert_stmt ::= "assert" expression ["," expression]\n\nThe simple form, "assert expression", is equivalent to\n\n if __debug__:\n if not expression: raise AssertionError\n\nThe extended form, "assert expression1, expression2", is equivalent to\n\n if __debug__:\n if not expression1: raise AssertionError(expression2)\n\nThese equivalences assume that "__debug__" and "AssertionError" refer\nto the built-in variables with those names. In the current\nimplementation, the built-in variable "__debug__" is "True" under\nnormal circumstances, "False" when optimization is requested (command\nline option -O). The current code generator emits no code for an\nassert statement when optimization is requested at compile time. Note\nthat it is unnecessary to include the source code for the expression\nthat failed in the error message; it will be displayed as part of the\nstack trace.\n\nAssignments to "__debug__" are illegal. The value for the built-in\nvariable is determined when the interpreter starts.\n', 'assignment': u'\nAssignment statements\n*********************\n\nAssignment statements are used to (re)bind names to values and to\nmodify attributes or items of mutable objects:\n\n assignment_stmt ::= (target_list "=")+ (expression_list | yield_expression)\n target_list ::= target ("," target)* [","]\n target ::= identifier\n | "(" target_list ")"\n | "[" target_list "]"\n | attributeref\n | subscription\n | slicing\n | "*" target\n\n(See section *Primaries* for the syntax definitions for\n*attributeref*, *subscription*, and *slicing*.)\n\nAn assignment statement evaluates the expression list (remember that\nthis can be a single expression or a comma-separated list, the latter\nyielding a tuple) and assigns the single resulting object to each of\nthe target lists, from left to right.\n\nAssignment is defined recursively depending on the form of the target\n(list). When a target is part of a mutable object (an attribute\nreference, subscription or slicing), the mutable object must\nultimately perform the assignment and decide about its validity, and\nmay raise an exception if the assignment is unacceptable. The rules\nobserved by various types and the exceptions raised are given with the\ndefinition of the object types (see section *The standard type\nhierarchy*).\n\nAssignment of an object to a target list, optionally enclosed in\nparentheses or square brackets, is recursively defined as follows.\n\n* If the target list is a single target: The object is assigned to\n that target.\n\n* If the target list is a comma-separated list of targets: The\n object must be an iterable with the same number of items as there\n are targets in the target list, and the items are assigned, from\n left to right, to the corresponding targets.\n\n * If the target list contains one target prefixed with an\n asterisk, called a "starred" target: The object must be a sequence\n with at least as many items as there are targets in the target\n list, minus one. The first items of the sequence are assigned,\n from left to right, to the targets before the starred target. The\n final items of the sequence are assigned to the targets after the\n starred target. A list of the remaining items in the sequence is\n then assigned to the starred target (the list can be empty).\n\n * Else: The object must be a sequence with the same number of\n items as there are targets in the target list, and the items are\n assigned, from left to right, to the corresponding targets.\n\nAssignment of an object to a single target is recursively defined as\nfollows.\n\n* If the target is an identifier (name):\n\n * If the name does not occur in a "global" or "nonlocal" statement\n in the current code block: the name is bound to the object in the\n current local namespace.\n\n * Otherwise: the name is bound to the object in the global\n namespace or the outer namespace determined by "nonlocal",\n respectively.\n\n The name is rebound if it was already bound. This may cause the\n reference count for the object previously bound to the name to reach\n zero, causing the object to be deallocated and its destructor (if it\n has one) to be called.\n\n* If the target is a target list enclosed in parentheses or in\n square brackets: The object must be an iterable with the same number\n of items as there are targets in the target list, and its items are\n assigned, from left to right, to the corresponding targets.\n\n* If the target is an attribute reference: The primary expression in\n the reference is evaluated. It should yield an object with\n assignable attributes; if this is not the case, "TypeError" is\n raised. That object is then asked to assign the assigned object to\n the given attribute; if it cannot perform the assignment, it raises\n an exception (usually but not necessarily "AttributeError").\n\n Note: If the object is a class instance and the attribute reference\n occurs on both sides of the assignment operator, the RHS expression,\n "a.x" can access either an instance attribute or (if no instance\n attribute exists) a class attribute. The LHS target "a.x" is always\n set as an instance attribute, creating it if necessary. Thus, the\n two occurrences of "a.x" do not necessarily refer to the same\n attribute: if the RHS expression refers to a class attribute, the\n LHS creates a new instance attribute as the target of the\n assignment:\n\n class Cls:\n x = 3 # class variable\n inst = Cls()\n inst.x = inst.x + 1 # writes inst.x as 4 leaving Cls.x as 3\n\n This description does not necessarily apply to descriptor\n attributes, such as properties created with "property()".\n\n* If the target is a subscription: The primary expression in the\n reference is evaluated. It should yield either a mutable sequence\n object (such as a list) or a mapping object (such as a dictionary).\n Next, the subscript expression is evaluated.\n\n If the primary is a mutable sequence object (such as a list), the\n subscript must yield an integer. If it is negative, the sequence\'s\n length is added to it. The resulting value must be a nonnegative\n integer less than the sequence\'s length, and the sequence is asked\n to assign the assigned object to its item with that index. If the\n index is out of range, "IndexError" is raised (assignment to a\n subscripted sequence cannot add new items to a list).\n\n If the primary is a mapping object (such as a dictionary), the\n subscript must have a type compatible with the mapping\'s key type,\n and the mapping is then asked to create a key/datum pair which maps\n the subscript to the assigned object. This can either replace an\n existing key/value pair with the same key value, or insert a new\n key/value pair (if no key with the same value existed).\n\n For user-defined objects, the "__setitem__()" method is called with\n appropriate arguments.\n\n* If the target is a slicing: The primary expression in the\n reference is evaluated. It should yield a mutable sequence object\n (such as a list). The assigned object should be a sequence object\n of the same type. Next, the lower and upper bound expressions are\n evaluated, insofar they are present; defaults are zero and the\n sequence\'s length. The bounds should evaluate to integers. If\n either bound is negative, the sequence\'s length is added to it. The\n resulting bounds are clipped to lie between zero and the sequence\'s\n length, inclusive. Finally, the sequence object is asked to replace\n the slice with the items of the assigned sequence. The length of\n the slice may be different from the length of the assigned sequence,\n thus changing the length of the target sequence, if the target\n sequence allows it.\n\n**CPython implementation detail:** In the current implementation, the\nsyntax for targets is taken to be the same as for expressions, and\ninvalid syntax is rejected during the code generation phase, causing\nless detailed error messages.\n\nAlthough the definition of assignment implies that overlaps between\nthe left-hand side and the right-hand side are \'simultanenous\' (for\nexample "a, b = b, a" swaps two variables), overlaps *within* the\ncollection of assigned-to variables occur left-to-right, sometimes\nresulting in confusion. For instance, the following program prints\n"[0, 2]":\n\n x = [0, 1]\n i = 0\n i, x[i] = 1, 2 # i is updated, then x[i] is updated\n print(x)\n\nSee also: **PEP 3132** - Extended Iterable Unpacking\n\n The specification for the "*target" feature.\n\n\nAugmented assignment statements\n===============================\n\nAugmented assignment is the combination, in a single statement, of a\nbinary operation and an assignment statement:\n\n augmented_assignment_stmt ::= augtarget augop (expression_list | yield_expression)\n augtarget ::= identifier | attributeref | subscription | slicing\n augop ::= "+=" | "-=" | "*=" | "@=" | "/=" | "//=" | "%=" | "**="\n | ">>=" | "<<=" | "&=" | "^=" | "|="\n\n(See section *Primaries* for the syntax definitions of the last three\nsymbols.)\n\nAn augmented assignment evaluates the target (which, unlike normal\nassignment statements, cannot be an unpacking) and the expression\nlist, performs the binary operation specific to the type of assignment\non the two operands, and assigns the result to the original target.\nThe target is only evaluated once.\n\nAn augmented assignment expression like "x += 1" can be rewritten as\n"x = x + 1" to achieve a similar, but not exactly equal effect. In the\naugmented version, "x" is only evaluated once. Also, when possible,\nthe actual operation is performed *in-place*, meaning that rather than\ncreating a new object and assigning that to the target, the old object\nis modified instead.\n\nUnlike normal assignments, augmented assignments evaluate the left-\nhand side *before* evaluating the right-hand side. For example, "a[i]\n+= f(x)" first looks-up "a[i]", then it evaluates "f(x)" and performs\nthe addition, and lastly, it writes the result back to "a[i]".\n\nWith the exception of assigning to tuples and multiple targets in a\nsingle statement, the assignment done by augmented assignment\nstatements is handled the same way as normal assignments. Similarly,\nwith the exception of the possible *in-place* behavior, the binary\noperation performed by augmented assignment is the same as the normal\nbinary operations.\n\nFor targets which are attribute references, the same *caveat about\nclass and instance attributes* applies as for regular assignments.\n', 'atom-identifiers': u'\nIdentifiers (Names)\n*******************\n\nAn identifier occurring as an atom is a name. See section\n*Identifiers and keywords* for lexical definition and section *Naming\nand binding* for documentation of naming and binding.\n\nWhen the name is bound to an object, evaluation of the atom yields\nthat object. When a name is not bound, an attempt to evaluate it\nraises a "NameError" exception.\n\n**Private name mangling:** When an identifier that textually occurs in\na class definition begins with two or more underscore characters and\ndoes not end in two or more underscores, it is considered a *private\nname* of that class. Private names are transformed to a longer form\nbefore code is generated for them. The transformation inserts the\nclass name, with leading underscores removed and a single underscore\ninserted, in front of the name. For example, the identifier "__spam"\noccurring in a class named "Ham" will be transformed to "_Ham__spam".\nThis transformation is independent of the syntactical context in which\nthe identifier is used. If the transformed name is extremely long\n(longer than 255 characters), implementation defined truncation may\nhappen. If the class name consists only of underscores, no\ntransformation is done.\n', -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 7 14:18:29 2015 From: python-checkins at python.org (larry.hastings) Date: Mon, 07 Sep 2015 12:18:29 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy41IC0+IDMuNSk6?= =?utf-8?q?_Merge_heads=2E?= Message-ID: <20150907121825.101488.85359@psf.io> https://hg.python.org/cpython/rev/5939b50429a0 changeset: 97748:5939b50429a0 branch: 3.5 parent: 97747:170af6e33c9c parent: 97743:f842c2b710ed user: Larry Hastings date: Mon Sep 07 05:16:38 2015 -0700 summary: Merge heads. files: Doc/c-api/buffer.rst | 8 +- Doc/c-api/typeobj.rst | 10 +- Doc/extending/newtypes.rst | 6 +- Doc/faq/programming.rst | 2 + Doc/includes/typestruct.h | 16 +- Doc/library/_thread.rst | 3 +- Doc/library/asyncio-eventloop.rst | 2 +- Doc/library/asyncio-subprocess.rst | 8 +- Doc/library/asyncio.rst | 7 + Doc/library/collections.rst | 6 +- Doc/library/dis.rst | 4 +- Doc/library/html.parser.rst | 2 +- Doc/library/inspect.rst | 15 +- Doc/library/ipaddress.rst | 2 +- Doc/library/os.path.rst | 17 +- Doc/library/stdtypes.rst | 15 +- Doc/library/subprocess.rst | 2 +- Doc/library/tempfile.rst | 153 +++++---- Doc/library/threading.rst | 3 +- Doc/library/typing.rst | 97 +++--- Doc/library/unittest.rst | 4 +- Doc/library/xml.etree.elementtree.rst | 30 +- Doc/tools/susp-ignored.csv | 2 +- Doc/tutorial/datastructures.rst | 20 +- Doc/using/mac.rst | 5 +- Include/object.h | 3 +- Lib/_pyio.py | 3 +- Lib/cgi.py | 5 + Lib/collections/__init__.py | 11 +- Lib/configparser.py | 29 +- Lib/functools.py | 2 +- Lib/html/parser.py | 10 +- Lib/http/server.py | 3 +- Lib/idlelib/NEWS.txt | 13 + Lib/idlelib/PyShell.py | 1 + Lib/idlelib/ScriptBinding.py | 2 +- Lib/idlelib/StackViewer.py | 11 +- Lib/idlelib/configDialog.py | 81 +++-- Lib/pdb.py | 3 + Lib/test/test_cgi.py | 18 + Lib/test/test_collections.py | 12 +- Lib/test/test_configparser.py | 7 +- Lib/test/test_functools.py | 18 + Lib/test/test_gdb.py | 52 ++- Lib/test/test_glob.py | 6 +- Lib/test/test_htmlparser.py | 15 +- Lib/test/test_mmap.py | 5 +- Lib/test/test_pdb.py | 12 + Lib/test/test_pep277.py | 8 +- Lib/test/test_posixpath.py | 37 +- Lib/test/test_py_compile.py | 8 +- Lib/test/test_shutil.py | 212 +++++-------- Lib/test/test_subprocess.py | 7 +- Lib/test/test_sysconfig.py | 8 +- Lib/test/test_tarfile.py | 12 +- Lib/test/test_unicode_file.py | 8 +- Lib/test/test_warnings/__init__.py | 6 + Lib/test/test_wsgiref.py | 5 +- Lib/unittest/case.py | 25 +- Lib/unittest/test/test_assertions.py | 15 +- Lib/unittest/test/test_skipping.py | 33 ++ Misc/ACKS | 10 + Misc/NEWS | 81 +++++ Misc/Porting | 42 +-- Modules/_collectionsmodule.c | 16 +- Modules/_decimal/libmpdec/mpdecimal.c | 1 + Modules/posixmodule.c | 12 +- Objects/odictobject.c | 40 +- PC/pyconfig.h | 8 + PCbuild/build.bat | 77 +++- PCbuild/get_externals.bat | 21 +- PCbuild/pcbuild.proj | 14 +- PCbuild/rt.bat | 4 +- Python/codecs.c | 12 +- Python/pylifecycle.c | 79 ++-- Python/pytime.c | 16 +- Tools/buildbot/test.bat | 26 +- Tools/msi/build.bat | 8 +- Tools/msi/bundle/snapshot.wixproj | 7 +- Tools/msi/core/core_d.wxs | 2 +- Tools/msi/core/core_pdb.wxs | 2 +- Tools/msi/dev/dev_d.wxs | 2 +- Tools/msi/exe/exe_d.wxs | 2 +- Tools/msi/exe/exe_pdb.wxs | 2 +- Tools/msi/lib/lib_d.wxs | 2 +- Tools/msi/lib/lib_pdb.wxs | 2 +- Tools/msi/tcltk/tcltk_d.wxs | 2 +- Tools/msi/tcltk/tcltk_pdb.wxs | 2 +- Tools/msi/test/test_d.wxs | 2 +- Tools/msi/test/test_pdb.wxs | 2 +- 90 files changed, 962 insertions(+), 679 deletions(-) diff --git a/Doc/c-api/buffer.rst b/Doc/c-api/buffer.rst --- a/Doc/c-api/buffer.rst +++ b/Doc/c-api/buffer.rst @@ -133,15 +133,15 @@ called on non-NULL :c:member:`~Py_buffer.format` values. Important exception: If a consumer requests a buffer without the - :c:macro:`PyBUF_FORMAT` flag, :c:member:`~Py_Buffer.format` will + :c:macro:`PyBUF_FORMAT` flag, :c:member:`~Py_buffer.format` will be set to *NULL*, but :c:member:`~Py_buffer.itemsize` still has the value for the original format. - If :c:member:`~Py_Buffer.shape` is present, the equality + If :c:member:`~Py_buffer.shape` is present, the equality ``product(shape) * itemsize == len`` still holds and the consumer can use :c:member:`~Py_buffer.itemsize` to navigate the buffer. - If :c:member:`~Py_Buffer.shape` is *NULL* as a result of a :c:macro:`PyBUF_SIMPLE` + If :c:member:`~Py_buffer.shape` is *NULL* as a result of a :c:macro:`PyBUF_SIMPLE` or a :c:macro:`PyBUF_WRITABLE` request, the consumer must disregard :c:member:`~Py_buffer.itemsize` and assume ``itemsize == 1``. @@ -156,7 +156,7 @@ .. c:member:: int ndim The number of dimensions the memory represents as an n-dimensional array. - If it is 0, :c:member:`~Py_Buffer.buf` points to a single item representing + If it is 0, :c:member:`~Py_buffer.buf` points to a single item representing a scalar. In this case, :c:member:`~Py_buffer.shape`, :c:member:`~Py_buffer.strides` and :c:member:`~Py_buffer.suboffsets` MUST be *NULL*. diff --git a/Doc/c-api/typeobj.rst b/Doc/c-api/typeobj.rst --- a/Doc/c-api/typeobj.rst +++ b/Doc/c-api/typeobj.rst @@ -94,7 +94,7 @@ This field is not inherited by subtypes. -.. c:member:: char* PyTypeObject.tp_name +.. c:member:: const char* PyTypeObject.tp_name Pointer to a NUL-terminated string containing the name of the type. For types that are accessible as module globals, the string should be the full module @@ -372,7 +372,7 @@ inherited individually. -.. c:member:: long PyTypeObject.tp_flags +.. c:member:: unsigned long PyTypeObject.tp_flags This field is a bit mask of various flags. Some flags indicate variant semantics for certain situations; others are used to indicate that certain @@ -472,7 +472,7 @@ .. versionadded:: 3.4 -.. c:member:: char* PyTypeObject.tp_doc +.. c:member:: const char* PyTypeObject.tp_doc An optional pointer to a NUL-terminated C string giving the docstring for this type object. This is exposed as the :attr:`__doc__` attribute on the type and @@ -619,7 +619,7 @@ +----------------+------------+ -.. c:member:: long PyTypeObject.tp_weaklistoffset +.. c:member:: Py_ssize_t PyTypeObject.tp_weaklistoffset If the instances of this type are weakly referenceable, this field is greater than zero and contains the offset in the instance structure of the weak @@ -786,7 +786,7 @@ .. XXX explain. -.. c:member:: long PyTypeObject.tp_dictoffset +.. c:member:: Py_ssize_t PyTypeObject.tp_dictoffset If the instances of this type have a dictionary containing instance variables, this field is non-zero and contains the offset in the instances of the type of diff --git a/Doc/extending/newtypes.rst b/Doc/extending/newtypes.rst --- a/Doc/extending/newtypes.rst +++ b/Doc/extending/newtypes.rst @@ -893,20 +893,20 @@ all the fields you need (even if they're initialized to ``0``) and then change the values to suit your new type. :: - char *tp_name; /* For printing */ + const char *tp_name; /* For printing */ The name of the type - as mentioned in the last section, this will appear in various places, almost entirely for diagnostic purposes. Try to choose something that will be helpful in such a situation! :: - int tp_basicsize, tp_itemsize; /* For allocation */ + Py_ssize_t tp_basicsize, tp_itemsize; /* For allocation */ These fields tell the runtime how much memory to allocate when new objects of this type are created. Python has some built-in support for variable length structures (think: strings, lists) which is where the :c:member:`~PyTypeObject.tp_itemsize` field comes in. This will be dealt with later. :: - char *tp_doc; + const char *tp_doc; Here you can put a string (or its address) that you want returned when the Python script references ``obj.__doc__`` to retrieve the doc string. diff --git a/Doc/faq/programming.rst b/Doc/faq/programming.rst --- a/Doc/faq/programming.rst +++ b/Doc/faq/programming.rst @@ -1164,6 +1164,8 @@ usually a lot slower than using Python lists. +.. _faq-multidimensional-list: + How do I create a multidimensional list? ---------------------------------------- diff --git a/Doc/includes/typestruct.h b/Doc/includes/typestruct.h --- a/Doc/includes/typestruct.h +++ b/Doc/includes/typestruct.h @@ -1,7 +1,7 @@ typedef struct _typeobject { PyObject_VAR_HEAD - char *tp_name; /* For printing, in format "." */ - int tp_basicsize, tp_itemsize; /* For allocation */ + const char *tp_name; /* For printing, in format "." */ + Py_ssize_t tp_basicsize, tp_itemsize; /* For allocation */ /* Methods to implement standard operations */ @@ -9,7 +9,8 @@ printfunc tp_print; getattrfunc tp_getattr; setattrfunc tp_setattr; - PyAsyncMethods *tp_as_async; + PyAsyncMethods *tp_as_async; /* formerly known as tp_compare (Python 2) + or tp_reserved (Python 3) */ reprfunc tp_repr; /* Method suites for standard classes */ @@ -30,9 +31,9 @@ PyBufferProcs *tp_as_buffer; /* Flags to define presence of optional/expanded features */ - long tp_flags; + unsigned long tp_flags; - char *tp_doc; /* Documentation string */ + const char *tp_doc; /* Documentation string */ /* call function for all accessible objects */ traverseproc tp_traverse; @@ -44,7 +45,7 @@ richcmpfunc tp_richcompare; /* weak reference enabler */ - long tp_weaklistoffset; + Py_ssize_t tp_weaklistoffset; /* Iterators */ getiterfunc tp_iter; @@ -58,7 +59,7 @@ PyObject *tp_dict; descrgetfunc tp_descr_get; descrsetfunc tp_descr_set; - long tp_dictoffset; + Py_ssize_t tp_dictoffset; initproc tp_init; allocfunc tp_alloc; newfunc tp_new; @@ -69,7 +70,6 @@ PyObject *tp_cache; PyObject *tp_subclasses; PyObject *tp_weaklist; - destructor tp_del; /* Type attribute cache version tag. Added in version 2.6 */ diff --git a/Doc/library/_thread.rst b/Doc/library/_thread.rst --- a/Doc/library/_thread.rst +++ b/Doc/library/_thread.rst @@ -93,7 +93,8 @@ Return the thread stack size used when creating new threads. The optional *size* argument specifies the stack size to be used for subsequently created threads, and must be 0 (use platform or configured default) or a positive - integer value of at least 32,768 (32 KiB). If changing the thread stack size is + integer value of at least 32,768 (32 KiB). If *size* is not specified, + 0 is used. If changing the thread stack size is unsupported, a :exc:`RuntimeError` is raised. If the specified stack size is invalid, a :exc:`ValueError` is raised and the stack size is unmodified. 32 KiB is currently the minimum supported stack size value to guarantee sufficient diff --git a/Doc/library/asyncio-eventloop.rst b/Doc/library/asyncio-eventloop.rst --- a/Doc/library/asyncio-eventloop.rst +++ b/Doc/library/asyncio-eventloop.rst @@ -6,7 +6,7 @@ =============== The event loop is the central execution device provided by :mod:`asyncio`. -It provides multiple facilities, amongst which: +It provides multiple facilities, including: * Registering, executing and cancelling delayed calls (timeouts). diff --git a/Doc/library/asyncio-subprocess.rst b/Doc/library/asyncio-subprocess.rst --- a/Doc/library/asyncio-subprocess.rst +++ b/Doc/library/asyncio-subprocess.rst @@ -303,7 +303,7 @@ .. _asyncio-subprocess-threads: Subprocess and threads -====================== +---------------------- asyncio supports running subprocesses from different threads, but there are limits: @@ -322,10 +322,10 @@ Subprocess examples -=================== +------------------- Subprocess using transport and protocol ---------------------------------------- +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Example of a subprocess protocol using to get the output of a subprocess and to wait for the subprocess exit. The subprocess is created by the @@ -381,7 +381,7 @@ Subprocess using streams ------------------------- +^^^^^^^^^^^^^^^^^^^^^^^^ Example using the :class:`~asyncio.subprocess.Process` class to control the subprocess and the :class:`StreamReader` class to read from the standard diff --git a/Doc/library/asyncio.rst b/Doc/library/asyncio.rst --- a/Doc/library/asyncio.rst +++ b/Doc/library/asyncio.rst @@ -4,6 +4,13 @@ .. module:: asyncio :synopsis: Asynchronous I/O, event loop, coroutines and tasks. +.. note:: + + The asyncio package has been included in the standard library on a + :term:`provisional basis `. Backwards incompatible + changes (up to and including removal of the module) may occur if deemed + necessary by the core developers. + .. versionadded:: 3.4 **Source code:** :source:`Lib/asyncio/` diff --git a/Doc/library/collections.rst b/Doc/library/collections.rst --- a/Doc/library/collections.rst +++ b/Doc/library/collections.rst @@ -842,10 +842,10 @@ .. method:: somenamedtuple._asdict() Return a new :class:`OrderedDict` which maps field names to their corresponding - values. Note, this method is no longer needed now that the same effect can - be achieved by using the built-in :func:`vars` function:: + values:: - >>> vars(p) + >>> p = Point(x=11, y=22) + >>> p._asdict() OrderedDict([('x', 11), ('y', 22)]) .. versionchanged:: 3.1 diff --git a/Doc/library/dis.rst b/Doc/library/dis.rst --- a/Doc/library/dis.rst +++ b/Doc/library/dis.rst @@ -946,8 +946,8 @@ Creates a new function object, sets its *__closure__* slot, and pushes it on the stack. TOS is the :term:`qualified name` of the function, TOS1 is the code associated with the function, and TOS2 is the tuple containing cells for - the closure's free variables. The function also has *argc* default - parameters, which are found below the cells. + the closure's free variables. *argc* is interpreted as in ``MAKE_FUNCTION``; + the annotations and defaults are also in the same order below TOS2. .. opcode:: BUILD_SLICE (argc) diff --git a/Doc/library/html.parser.rst b/Doc/library/html.parser.rst --- a/Doc/library/html.parser.rst +++ b/Doc/library/html.parser.rst @@ -185,7 +185,7 @@ The content of Internet Explorer conditional comments (condcoms) will also be sent to this method, so, for ````, - this method will receive ``'[if IE 9]>IE-specific contentIE9-specific content>> os.path.commonprefix(['/usr/lib', '/usr/local/lib']) + '/usr/l' + + >>> os.path.commonpath(['/usr/lib', '/usr/local/lib']) + '/usr' .. function:: dirname(path) diff --git a/Doc/library/stdtypes.rst b/Doc/library/stdtypes.rst --- a/Doc/library/stdtypes.rst +++ b/Doc/library/stdtypes.rst @@ -854,8 +854,8 @@ | ``s + t`` | the concatenation of *s* and | (6)(7) | | | *t* | | +--------------------------+--------------------------------+----------+ -| ``s * n`` or | *n* shallow copies of *s* | (2)(7) | -| ``n * s`` | concatenated | | +| ``s * n`` or | equivalent to adding *s* to | (2)(7) | +| ``n * s`` | itself *n* times | | +--------------------------+--------------------------------+----------+ | ``s[i]`` | *i*\ th item of *s*, origin 0 | \(3) | +--------------------------+--------------------------------+----------+ @@ -897,9 +897,9 @@ (2) Values of *n* less than ``0`` are treated as ``0`` (which yields an empty - sequence of the same type as *s*). Note also that the copies are shallow; - nested structures are not copied. This often haunts new Python programmers; - consider:: + sequence of the same type as *s*). Note that items in the sequence *s* + are not copied; they are referenced multiple times. This often haunts + new Python programmers; consider:: >>> lists = [[]] * 3 >>> lists @@ -909,7 +909,7 @@ [[3], [3], [3]] What has happened is that ``[[]]`` is a one-element list containing an empty - list, so all three elements of ``[[]] * 3`` are (pointers to) this single empty + list, so all three elements of ``[[]] * 3`` are references to this single empty list. Modifying any of the elements of ``lists`` modifies this single list. You can create a list of different lists this way:: @@ -920,6 +920,9 @@ >>> lists [[3], [5], [7]] + Further explanation is available in the FAQ entry + :ref:`faq-multidimensional-list`. + (3) If *i* or *j* is negative, the index is relative to the end of the string: ``len(s) + i`` or ``len(s) + j`` is substituted. But note that ``-0`` is diff --git a/Doc/library/subprocess.rst b/Doc/library/subprocess.rst --- a/Doc/library/subprocess.rst +++ b/Doc/library/subprocess.rst @@ -1068,7 +1068,7 @@ if rc is not None and rc >> 8: print("There were some errors") ==> - process = Popen(cmd, 'w', stdin=PIPE) + process = Popen(cmd, stdin=PIPE) ... process.stdin.close() if process.wait() != 0: diff --git a/Doc/library/tempfile.rst b/Doc/library/tempfile.rst --- a/Doc/library/tempfile.rst +++ b/Doc/library/tempfile.rst @@ -16,16 +16,18 @@ -------------- -This module generates temporary files and directories. It works on all -supported platforms. It provides three new functions, -:func:`NamedTemporaryFile`, :func:`mkstemp`, and :func:`mkdtemp`, which should -eliminate all remaining need to use the insecure :func:`mktemp` function. -Temporary file names created by this module no longer contain the process ID; -instead a string of six random characters is used. +This module creates temporary files and directories. It works on all +supported platforms. :class:`TemporaryFile`, :class:`NamedTemporaryFile`, +:class:`TemporaryDirectory`, and :class:`SpooledTemporaryFile` are high-level +interfaces which provide automatic cleanup and can be used as +context managers. :func:`mkstemp` and +:func:`mkdtemp` are lower-level functions which require manual cleanup. -Also, all the user-callable functions now take additional arguments which -allow direct control over the location and name of temporary files. It is -no longer necessary to use the global *tempdir* variable. +All the user-callable functions and constructors take additional arguments which +allow direct control over the location and name of temporary files and +directories. Files names used by this module include a string of +random characters which allows those files to be securely created in +shared temporary directories. To maintain backward compatibility, the argument order is somewhat odd; it is recommended to use keyword arguments for clarity. @@ -34,28 +36,33 @@ .. function:: TemporaryFile(mode='w+b', buffering=None, encoding=None, newline=None, suffix='', prefix='tmp', dir=None) Return a :term:`file-like object` that can be used as a temporary storage area. - The file is created using :func:`mkstemp`. It will be destroyed as soon + The file is created securely, using the same rules as :func:`mkstemp`. It will be destroyed as soon as it is closed (including an implicit close when the object is garbage - collected). Under Unix, the directory entry for the file is removed + collected). Under Unix, the directory entry for the file is either not created at all or is removed immediately after the file is created. Other platforms do not support this; your code should not rely on a temporary file created using this function having or not having a visible name in the file system. + The resulting object can be used as a context manager (see + :ref:`tempfile-examples`). On completion of the context or + destruction of the file object the temporary file will be removed + from the filesystem. + The *mode* parameter defaults to ``'w+b'`` so that the file created can be read and written without being closed. Binary mode is used so that it behaves consistently on all platforms without regard for the data that is stored. *buffering*, *encoding* and *newline* are interpreted as for :func:`open`. - The *dir*, *prefix* and *suffix* parameters are passed to :func:`mkstemp`. + The *dir*, *prefix* and *suffix* parameters have the same meaning + as with :func:`mkstemp`. The returned object is a true file object on POSIX platforms. On other platforms, it is a file-like object whose :attr:`!file` attribute is the - underlying true file object. This file-like object can be used in a - :keyword:`with` statement, just like a normal file. + underlying true file object. The :py:data:`os.O_TMPFILE` flag is used if it is available and works - (Linux-specific, require Linux kernel 3.11 or later). + (Linux-specific, requires Linux kernel 3.11 or later). .. versionchanged:: 3.5 @@ -101,10 +108,9 @@ .. function:: TemporaryDirectory(suffix='', prefix='tmp', dir=None) - This function creates a temporary directory using :func:`mkdtemp` - (the supplied arguments are passed directly to the underlying function). + This function securely creates a temporary directory using the same rules as :func:`mkdtemp`. The resulting object can be used as a context manager (see - :ref:`context-managers`). On completion of the context or destruction + :ref:`tempfile-examples`). On completion of the context or destruction of the temporary directory object the newly created temporary directory and all its contents are removed from the filesystem. @@ -194,49 +200,14 @@ an appropriate default value to be used. -.. function:: mktemp(suffix='', prefix='tmp', dir=None) +.. function:: gettempdir() - .. deprecated:: 2.3 - Use :func:`mkstemp` instead. + Return the name of the directory used for temporary files. This + defines the default value for the *dir* argument to all functions + in this module. - Return an absolute pathname of a file that did not exist at the time the - call is made. The *prefix*, *suffix*, and *dir* arguments are the same - as for :func:`mkstemp`. - - .. warning:: - - Use of this function may introduce a security hole in your program. By - the time you get around to doing anything with the file name it returns, - someone else may have beaten you to the punch. :func:`mktemp` usage can - be replaced easily with :func:`NamedTemporaryFile`, passing it the - ``delete=False`` parameter:: - - >>> f = NamedTemporaryFile(delete=False) - >>> f.name - '/tmp/tmptjujjt' - >>> f.write(b"Hello World!\n") - 13 - >>> f.close() - >>> os.unlink(f.name) - >>> os.path.exists(f.name) - False - -The module uses a global variable that tell it how to construct a -temporary name. They are initialized at the first call to any of the -functions above. The caller may change them, but this is discouraged; use -the appropriate function arguments, instead. - - -.. data:: tempdir - - When set to a value other than ``None``, this variable defines the - default value for the *dir* argument to all the functions defined in this - module. - - If ``tempdir`` is unset or ``None`` at any call to any of the above - functions, Python searches a standard list of directories and sets - *tempdir* to the first one which the calling user can create files in. - The list is: + Python searches a standard list of directories to find one which + the calling user can create files in. The list is: #. The directory named by the :envvar:`TMPDIR` environment variable. @@ -254,12 +225,8 @@ #. As a last resort, the current working directory. - -.. function:: gettempdir() - - Return the directory currently selected to create temporary files in. If - :data:`tempdir` is not ``None``, this simply returns its contents; otherwise, - the search described above is performed, and the result returned. + The result of this search is cached, see the description of + :data:`tempdir` below. .. function:: gettempdirb() @@ -278,6 +245,23 @@ .. versionadded:: 3.5 +The module uses a global variable to store the name of the directory +used for temporary files returned by :func:`gettempdir`. It can be +set directly to override the selection process, but this is discouraged. +All functions in this module take a *dir* argument which can be used +to specify the directory and this is the recommend approach. + +.. data:: tempdir + + When set to a value other than ``None``, this variable defines the + default value for the *dir* argument to all the functions defined in this + module. + + If ``tempdir`` is unset or ``None`` at any call to any of the above + functions except :func:`gettempprefix` it is initalized following the + algorithm described in :func:`gettempdir`. + +.. _tempfile-examples: Examples -------- @@ -311,3 +295,42 @@ >>> # directory and contents have been removed + +Deprecated functions and variables +---------------------------------- + +A historical way to create temporary files was to first generate a +file name with the :func:`mktemp` function and then create a file +using this name. Unfortunately this is not secure, because a different +process may create a file with this name in the time between the call +to :func:`mktemp` and the subsequent attempt to create the file by the +first process. The solution is to combine the two steps and create the +file immediately. This approach is used by :func:`mkstemp` and the +other functions described above. + +.. function:: mktemp(suffix='', prefix='tmp', dir=None) + + .. deprecated:: 2.3 + Use :func:`mkstemp` instead. + + Return an absolute pathname of a file that did not exist at the time the + call is made. The *prefix*, *suffix*, and *dir* arguments are the same + as for :func:`mkstemp`. + + .. warning:: + + Use of this function may introduce a security hole in your program. By + the time you get around to doing anything with the file name it returns, + someone else may have beaten you to the punch. :func:`mktemp` usage can + be replaced easily with :func:`NamedTemporaryFile`, passing it the + ``delete=False`` parameter:: + + >>> f = NamedTemporaryFile(delete=False) + >>> f.name + '/tmp/tmptjujjt' + >>> f.write(b"Hello World!\n") + 13 + >>> f.close() + >>> os.unlink(f.name) + >>> os.path.exists(f.name) + False diff --git a/Doc/library/threading.rst b/Doc/library/threading.rst --- a/Doc/library/threading.rst +++ b/Doc/library/threading.rst @@ -89,7 +89,8 @@ Return the thread stack size used when creating new threads. The optional *size* argument specifies the stack size to be used for subsequently created threads, and must be 0 (use platform or configured default) or a positive - integer value of at least 32,768 (32 KiB). If changing the thread stack size is + integer value of at least 32,768 (32 KiB). If *size* is not specified, + 0 is used. If changing the thread stack size is unsupported, a :exc:`RuntimeError` is raised. If the specified stack size is invalid, a :exc:`ValueError` is raised and the stack size is unmodified. 32 KiB is currently the minimum supported stack size value to guarantee sufficient diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -20,8 +20,9 @@ def greeting(name: str) -> str: return 'Hello ' + name -In the function `greeting`, the argument `name` is expected to by of type `str` -and the return type `str`. Subtypes are accepted as arguments. +In the function ``greeting``, the argument ``name`` is expected to by of type +:class:`str` and the return type :class:`str`. Subtypes are accepted as +arguments. Type aliases ------------ @@ -49,8 +50,8 @@ It is possible to declare the return type of a callable without specifying the call signature by substituting a literal ellipsis -for the list of arguments in the type hint: `Callable[..., ReturnType]`. -`None` as a type hint is a special case and is replaced by `type(None)`. +for the list of arguments in the type hint: ``Callable[..., ReturnType]``. +``None`` as a type hint is a special case and is replaced by ``type(None)``. Generics -------- @@ -108,11 +109,12 @@ def log(self, message: str) -> None: self.logger.info('{}: {}'.format(self.name, message)) -`Generic[T]` as a base class defines that the class `LoggedVar` takes a single -type parameter `T` . This also makes `T` valid as a type within the class body. +``Generic[T]`` as a base class defines that the class ``LoggedVar`` takes a +single type parameter ``T`` . This also makes ``T`` valid as a type within the +class body. -The `Generic` base class uses a metaclass that defines `__getitem__` so that -`LoggedVar[t]` is valid as a type:: +The :class:`Generic` base class uses a metaclass that defines +:meth:`__getitem__` so that ``LoggedVar[t]`` is valid as a type:: from typing import Iterable @@ -132,7 +134,7 @@ class StrangePair(Generic[T, S]): ... -Each type variable argument to `Generic` must be distinct. +Each type variable argument to :class:`Generic` must be distinct. This is thus invalid:: from typing import TypeVar, Generic @@ -152,9 +154,9 @@ class LinkedList(Sized, Generic[T]): ... -Subclassing a generic class without specifying type parameters assumes `Any` -for each position. In the following example, `MyIterable` is not generic but -implicitly inherits from `Iterable[Any]`:: +Subclassing a generic class without specifying type parameters assumes +:class:`Any` for each position. In the following example, ``MyIterable`` is +not generic but implicitly inherits from ``Iterable[Any]``:: from typing import Iterable @@ -162,24 +164,24 @@ Generic metaclasses are not supported. -The `Any` type --------------- +The :class:`Any` type +--------------------- -A special kind of type is `Any`. Every type is a subtype of `Any`. -This is also true for the builtin type object. However, to the static type -checker these are completely different. +A special kind of type is :class:`Any`. Every type is a subtype of +:class:`Any`. This is also true for the builtin type object. However, to the +static type checker these are completely different. -When the type of a value is `object`, the type checker will reject almost all -operations on it, and assigning it to a variable (or using it as a return value) -of a more specialized type is a type error. On the other hand, when a value has -type `Any`, the type checker will allow all operations on it, and a value of -type `Any` can be assigned to a variable (or used as a return value) of a more -constrained type. +When the type of a value is :class:`object`, the type checker will reject +almost all operations on it, and assigning it to a variable (or using it as a +return value) of a more specialized type is a type error. On the other hand, +when a value has type :class:`Any`, the type checker will allow all operations +on it, and a value of type :class:`Any` can be assigned to a variable (or used +as a return value) of a more constrained type. Default argument values ----------------------- -Use a literal ellipsis `...` to declare an argument as having a default value:: +Use a literal ellipsis ``...`` to declare an argument as having a default value:: from typing import AnyStr @@ -195,9 +197,10 @@ Special type indicating an unconstrained type. - * Any object is an instance of `Any`. - * Any class is a subclass of `Any`. - * As a special case, `Any` and `object` are subclasses of each other. + * Any object is an instance of :class:`Any`. + * Any class is a subclass of :class:`Any`. + * As a special case, :class:`Any` and :class:`object` are subclasses of + each other. .. class:: TypeVar @@ -224,22 +227,22 @@ return x if len(x) >= len(y) else y The latter example's signature is essentially the overloading - of `(str, str) -> str` and `(bytes, bytes) -> bytes`. Also note - that if the arguments are instances of some subclass of `str`, - the return type is still plain `str`. + of ``(str, str) -> str`` and ``(bytes, bytes) -> bytes``. Also note + that if the arguments are instances of some subclass of :class:`str`, + the return type is still plain :class:`str`. - At runtime, `isinstance(x, T)` will raise `TypeError`. In general, - `isinstance` and `issublass` should not be used with types. + At runtime, ``isinstance(x, T)`` will raise :exc:`TypeError`. In general, + :func:`isinstance` and :func:`issublass` should not be used with types. Type variables may be marked covariant or contravariant by passing - `covariant=True` or `contravariant=True`. See :pep:`484` for more + ``covariant=True`` or ``contravariant=True``. See :pep:`484` for more details. By default type variables are invariant. .. class:: Union - Union type; `Union[X, Y]` means either X or Y. + Union type; ``Union[X, Y]`` means either X or Y. - To define a union, use e.g. `Union[int, str]`. Details: + To define a union, use e.g. ``Union[int, str]``. Details: * The arguments must be types and there must be at least one. @@ -259,37 +262,37 @@ Union[int, str] == Union[str, int] - * If `Any` is present it is the sole survivor, e.g.:: + * If :class:`Any` is present it is the sole survivor, e.g.:: Union[int, Any] == Any * You cannot subclass or instantiate a union. - * You cannot write `Union[X][Y]` + * You cannot write ``Union[X][Y]`` - * You can use `Optional[X]` as a shorthand for `Union[X, None]`. + * You can use ``Optional[X]`` as a shorthand for ``Union[X, None]``. .. class:: Optional Optional type. - `Optional[X]` is equivalent to `Union[X, type(None)]`. + ``Optional[X]`` is equivalent to ``Union[X, type(None)]``. .. class:: Tuple - Tuple type; `Tuple[X, Y]` is the is the type of a tuple of two items + Tuple type; ``Tuple[X, Y]`` is the is the type of a tuple of two items with the first item of type X and the second of type Y. - Example: `Tuple[T1, T2]` is a tuple of two elements corresponding - to type variables T1 and T2. `Tuple[int, float, str]` is a tuple + Example: ``Tuple[T1, T2]`` is a tuple of two elements corresponding + to type variables T1 and T2. ``Tuple[int, float, str]`` is a tuple of an int, a float and a string. To specify a variable-length tuple of homogeneous type, - use literal ellipsis, e.g. `Tuple[int, ...]`. + use literal ellipsis, e.g. ``Tuple[int, ...]``. .. class:: Callable - Callable type; `Callable[[int], str]` is a function of (int) -> str. + Callable type; ``Callable[[int], str]`` is a function of (int) -> str. The subscription syntax must always be used with exactly two values: the argument list and the return type. The argument list @@ -297,9 +300,9 @@ There is no syntax to indicate optional or keyword arguments, such function types are rarely used as callback types. - `Callable[..., ReturnType]` could be used to type hint a callable - taking any number of arguments and returning `ReturnType`. - A plain `Callable` is equivalent to `Callable[..., Any]`. + ``Callable[..., ReturnType]`` could be used to type hint a callable + taking any number of arguments and returning ``ReturnType``. + A plain :class:`Callable` is equivalent to ``Callable[..., Any]``. .. class:: Generic diff --git a/Doc/library/unittest.rst b/Doc/library/unittest.rst --- a/Doc/library/unittest.rst +++ b/Doc/library/unittest.rst @@ -278,8 +278,8 @@ as positional arguments in that order. The following two command lines are equivalent:: - python -m unittest discover -s project_directory -p '*_test.py' - python -m unittest discover project_directory '*_test.py' + python -m unittest discover -s project_directory -p "*_test.py" + python -m unittest discover project_directory "*_test.py" As well as being a path it is possible to pass a package name, for example ``myproject.subpackage.test``, as the start directory. The package name you diff --git a/Doc/library/xml.etree.elementtree.rst b/Doc/library/xml.etree.elementtree.rst --- a/Doc/library/xml.etree.elementtree.rst +++ b/Doc/library/xml.etree.elementtree.rst @@ -651,21 +651,29 @@ .. attribute:: text + tail - The *text* attribute can be used to hold additional data associated with - the element. As the name implies this attribute is usually a string but - may be any application-specific object. If the element is created from - an XML file the attribute will contain any text found between the element - tags. + These attributes can be used to hold additional data associated with + the element. Their values are usually strings but may be any + application-specific object. If the element is created from + an XML file, the *text* attribute holds either the text between + the element's start tag and its first child or end tag, or ``None``, and + the *tail* attribute holds either the text between the element's + end tag and the next tag, or ``None``. For the XML data + .. code-block:: xml - .. attribute:: tail + 1234 - The *tail* attribute can be used to hold additional data associated with - the element. This attribute is usually a string but may be any - application-specific object. If the element is created from an XML file - the attribute will contain any text found after the element's end tag and - before the next tag. + the *a* element has ``None`` for both *text* and *tail* attributes, + the *b* element has *text* ``"1"`` and *tail* ``"4"``, + the *c* element has *text* ``"2"`` and *tail* ``None``, + and the *d* element has *text* ``None`` and *tail* ``"3"``. + + To collect the inner text of an element, see :meth:`itertext`, for + example ``"".join(element.itertext())``. + + Applications may store arbitrary objects in these attributes. .. attribute:: attrib diff --git a/Doc/tools/susp-ignored.csv b/Doc/tools/susp-ignored.csv --- a/Doc/tools/susp-ignored.csv +++ b/Doc/tools/susp-ignored.csv @@ -288,6 +288,6 @@ library/zipapp,31,:main,"$ python -m zipapp myapp -m ""myapp:main""" library/zipapp,82,:fn,"argument should have the form ""pkg.mod:fn"", where ""pkg.mod"" is a" library/zipapp,155,:callable,"""pkg.module:callable"" and the archive will be run by importing" -library/stdtypes,3767,::,>>> m[::2].tolist() +library/stdtypes,,::,>>> m[::2].tolist() library/sys,1115,`,# ``wrapper`` creates a ``wrap(coro)`` coroutine: tutorial/venv,77,:c7b9645a6f35,"Python 3.4.3+ (3.4:c7b9645a6f35+, May 22 2015, 09:31:25)" diff --git a/Doc/tutorial/datastructures.rst b/Doc/tutorial/datastructures.rst --- a/Doc/tutorial/datastructures.rst +++ b/Doc/tutorial/datastructures.rst @@ -612,18 +612,18 @@ orange pear -To change a sequence you are iterating over while inside the loop (for -example to duplicate certain items), it is recommended that you first make -a copy. Looping over a sequence does not implicitly make a copy. The slice -notation makes this especially convenient:: +It is sometimes tempting to change a list while you are looping over it; +however, it is often simpler and safer to create a new list instead. :: - >>> words = ['cat', 'window', 'defenestrate'] - >>> for w in words[:]: # Loop over a slice copy of the entire list. - ... if len(w) > 6: - ... words.insert(0, w) + >>> import math + >>> raw_data = [56.2, float('NaN'), 51.7, 55.3, 52.5, float('NaN'), 47.8] + >>> filtered_data = [] + >>> for value in raw_data: + ... if not math.isnan(value): + ... filtered_data.append(value) ... - >>> words - ['defenestrate', 'cat', 'window', 'defenestrate'] + >>> filtered_data + [56.2, 51.7, 55.3, 52.5, 47.8] .. _tut-conditions: diff --git a/Doc/using/mac.rst b/Doc/using/mac.rst --- a/Doc/using/mac.rst +++ b/Doc/using/mac.rst @@ -102,8 +102,9 @@ Python on OS X honors all standard Unix environment variables such as :envvar:`PYTHONPATH`, but setting these variables for programs started from the Finder is non-standard as the Finder does not read your :file:`.profile` or -:file:`.cshrc` at startup. You need to create a file :file:`~ -/.MacOSX/environment.plist`. See Apple's Technical Document QA1067 for details. +:file:`.cshrc` at startup. You need to create a file +:file:`~/.MacOSX/environment.plist`. See Apple's Technical Document QA1067 for +details. For more information on installation Python packages in MacPython, see section :ref:`mac-package-manager`. diff --git a/Include/object.h b/Include/object.h --- a/Include/object.h +++ b/Include/object.h @@ -351,7 +351,8 @@ printfunc tp_print; getattrfunc tp_getattr; setattrfunc tp_setattr; - PyAsyncMethods *tp_as_async; /* formerly known as tp_compare or tp_reserved */ + PyAsyncMethods *tp_as_async; /* formerly known as tp_compare (Python 2) + or tp_reserved (Python 3) */ reprfunc tp_repr; /* Method suites for standard classes */ diff --git a/Lib/_pyio.py b/Lib/_pyio.py --- a/Lib/_pyio.py +++ b/Lib/_pyio.py @@ -8,12 +8,13 @@ import errno import array import stat +import sys # Import _thread instead of threading to reduce startup cost try: from _thread import allocate_lock as Lock except ImportError: from _dummy_thread import allocate_lock as Lock -if os.name == 'win32': +if sys.platform in {'win32', 'cygwin'}: from msvcrt import setmode as _setmode else: _setmode = None diff --git a/Lib/cgi.py b/Lib/cgi.py --- a/Lib/cgi.py +++ b/Lib/cgi.py @@ -720,6 +720,11 @@ self.bytes_read += len(hdr_text) parser.feed(hdr_text.decode(self.encoding, self.errors)) headers = parser.close() + + # Some clients add Content-Length for part headers, ignore them + if 'content-length' in headers: + del headers['content-length'] + part = klass(self.fp, headers, ib, environ, keep_blank_values, strict_parsing,self.limit-self.bytes_read, self.encoding, self.errors) diff --git a/Lib/collections/__init__.py b/Lib/collections/__init__.py --- a/Lib/collections/__init__.py +++ b/Lib/collections/__init__.py @@ -320,23 +320,14 @@ 'Return a nicely formatted representation string' return self.__class__.__name__ + '({repr_fmt})' % self - @property - def __dict__(self): - 'A new OrderedDict mapping field names to their values' - return OrderedDict(zip(self._fields, self)) - def _asdict(self): 'Return a new OrderedDict which maps field names to their values.' - return self.__dict__ + return OrderedDict(zip(self._fields, self)) def __getnewargs__(self): 'Return self as a plain tuple. Used by copy and pickle.' return tuple(self) - def __getstate__(self): - 'Exclude the OrderedDict from pickling' - return None - {field_defs} """ diff --git a/Lib/configparser.py b/Lib/configparser.py --- a/Lib/configparser.py +++ b/Lib/configparser.py @@ -263,12 +263,9 @@ """A string substitution required a setting which was not available.""" def __init__(self, option, section, rawval, reference): - msg = ("Bad value substitution:\n" - "\tsection: [%s]\n" - "\toption : %s\n" - "\tkey : %s\n" - "\trawval : %s\n" - % (section, option, reference, rawval)) + msg = ("Bad value substitution: option {!r} in section {!r} contains " + "an interpolation key {!r} which is not a valid option name. " + "Raw value: {!r}".format(option, section, reference, rawval)) InterpolationError.__init__(self, option, section, msg) self.reference = reference self.args = (option, section, rawval, reference) @@ -286,11 +283,11 @@ """Raised when substitutions are nested too deeply.""" def __init__(self, option, section, rawval): - msg = ("Value interpolation too deeply recursive:\n" - "\tsection: [%s]\n" - "\toption : %s\n" - "\trawval : %s\n" - % (section, option, rawval)) + msg = ("Recursion limit exceeded in value substitution: option {!r} " + "in section {!r} contains an interpolation key which " + "cannot be substituted in {} steps. Raw value: {!r}" + "".format(option, section, MAX_INTERPOLATION_DEPTH, + rawval)) InterpolationError.__init__(self, option, section, msg) self.args = (option, section, rawval) @@ -406,8 +403,9 @@ def _interpolate_some(self, parser, option, accum, rest, section, map, depth): + rawval = parser.get(section, option, raw=True, fallback=rest) if depth > MAX_INTERPOLATION_DEPTH: - raise InterpolationDepthError(option, section, rest) + raise InterpolationDepthError(option, section, rawval) while rest: p = rest.find("%") if p < 0: @@ -432,7 +430,7 @@ v = map[var] except KeyError: raise InterpolationMissingOptionError( - option, section, rest, var) from None + option, section, rawval, var) from None if "%" in v: self._interpolate_some(parser, option, accum, v, section, map, depth + 1) @@ -466,8 +464,9 @@ def _interpolate_some(self, parser, option, accum, rest, section, map, depth): + rawval = parser.get(section, option, raw=True, fallback=rest) if depth > MAX_INTERPOLATION_DEPTH: - raise InterpolationDepthError(option, section, rest) + raise InterpolationDepthError(option, section, rawval) while rest: p = rest.find("$") if p < 0: @@ -504,7 +503,7 @@ "More than one ':' found: %r" % (rest,)) except (KeyError, NoSectionError, NoOptionError): raise InterpolationMissingOptionError( - option, section, rest, ":".join(path)) from None + option, section, rawval, ":".join(path)) from None if "$" in v: self._interpolate_some(parser, opt, accum, v, sect, dict(parser.items(sect, raw=True)), diff --git a/Lib/functools.py b/Lib/functools.py --- a/Lib/functools.py +++ b/Lib/functools.py @@ -567,7 +567,7 @@ break # reject the current head, it appears later else: break - if not candidate: + if candidate is None: raise RuntimeError("Inconsistent hierarchy") result.append(candidate) # remove the chosen candidate diff --git a/Lib/html/parser.py b/Lib/html/parser.py --- a/Lib/html/parser.py +++ b/Lib/html/parser.py @@ -139,7 +139,15 @@ if self.convert_charrefs and not self.cdata_elem: j = rawdata.find('<', i) if j < 0: - if not end: + # if we can't find the next <, either we are at the end + # or there's more text incoming. If the latter is True, + # we can't pass the text to handle_data in case we have + # a charref cut in half at end. Try to determine if + # this is the case before proceding by looking for an + # & near the end and see if it's followed by a space or ;. + amppos = rawdata.rfind('&', max(i, n-34)) + if (amppos >= 0 and + not re.compile(r'[\s;]').search(rawdata, amppos)): break # wait till we get all the text j = n else: diff --git a/Lib/http/server.py b/Lib/http/server.py --- a/Lib/http/server.py +++ b/Lib/http/server.py @@ -1167,8 +1167,7 @@ ServerClass=HTTPServer, protocol="HTTP/1.0", port=8000, bind=""): """Test the HTTP request handler class. - This runs an HTTP server on port 8000 (or the first command line - argument). + This runs an HTTP server on port 8000 (or the port argument). """ server_address = (bind, port) diff --git a/Lib/idlelib/NEWS.txt b/Lib/idlelib/NEWS.txt --- a/Lib/idlelib/NEWS.txt +++ b/Lib/idlelib/NEWS.txt @@ -2,6 +2,19 @@ ========================= *Release date: 2015-09-13* ?? +- Issue #23672: Allow Idle to edit and run files with astral chars in name. + Patch by Mohd Sanad Zaki Rizvi. + +- Issue 24745: Idle editor default font. Switch from Courier to + platform-sensitive TkFixedFont. This should not affect current customized + font selections. If there is a problem, edit $HOME/.idlerc/config-main.cfg + and remove 'fontxxx' entries from [Editor Window]. Patch by Mark Roseman. + +- Issue #21192: Idle editor. When a file is run, put its name in the restart bar. + Do not print false prompts. Original patch by Adnan Umer. + +- Issue #13884: Idle menus. Remove tearoff lines. Patch by Roger Serwy. + - Issue #23184: remove unused names and imports in idlelib. Initial patch by Al Sweigart. diff --git a/Lib/idlelib/PyShell.py b/Lib/idlelib/PyShell.py --- a/Lib/idlelib/PyShell.py +++ b/Lib/idlelib/PyShell.py @@ -1043,6 +1043,7 @@ self.write("Python %s on %s\n%s\n%s" % (sys.version, sys.platform, self.COPYRIGHT, nosub)) + self.text.focus_force() self.showprompt() import tkinter tkinter._default_root = None # 03Jan04 KBK What's this? diff --git a/Lib/idlelib/ScriptBinding.py b/Lib/idlelib/ScriptBinding.py --- a/Lib/idlelib/ScriptBinding.py +++ b/Lib/idlelib/ScriptBinding.py @@ -69,7 +69,7 @@ try: tabnanny.process_tokens(tokenize.generate_tokens(f.readline)) except tokenize.TokenError as msg: - msgtxt, (lineno, start) = msg + msgtxt, (lineno, start) = msg.args self.editwin.gotoline(lineno) self.errorbox("Tabnanny Tokenizing Error", "Token Error: %s" % msgtxt) diff --git a/Lib/idlelib/StackViewer.py b/Lib/idlelib/StackViewer.py --- a/Lib/idlelib/StackViewer.py +++ b/Lib/idlelib/StackViewer.py @@ -10,8 +10,7 @@ def StackBrowser(root, flist=None, tb=None, top=None): if top is None: - from tkinter import Toplevel - top = Toplevel(root) + top = tk.Toplevel(root) sc = ScrolledCanvas(top, bg="white", highlightthickness=0) sc.frame.pack(expand=1, fill="both") item = StackTreeItem(flist, tb) @@ -108,12 +107,9 @@ def IsExpandable(self): return len(self.object) > 0 - def keys(self): - return list(self.object.keys()) - def GetSubList(self): sublist = [] - for key in self.keys(): + for key in self.object.keys(): try: value = self.object[key] except KeyError: @@ -124,6 +120,9 @@ sublist.append(item) return sublist + def keys(self): # unused, left for possible 3rd party use + return list(self.object.keys()) + def _stack_viewer(parent): root = tk.Tk() root.title("Test StackViewer") diff --git a/Lib/idlelib/configDialog.py b/Lib/idlelib/configDialog.py --- a/Lib/idlelib/configDialog.py +++ b/Lib/idlelib/configDialog.py @@ -1201,9 +1201,6 @@ # update the scrollbars to match the size of the inner frame size = (interior.winfo_reqwidth(), interior.winfo_reqheight()) canvas.config(scrollregion="0 0 %s %s" % size) - if interior.winfo_reqwidth() != canvas.winfo_width(): - # update the canvas's width to fit the inner frame - canvas.config(width=interior.winfo_reqwidth()) interior.bind('', _configure_interior) def _configure_canvas(event): @@ -1323,38 +1320,56 @@ def create_widgets(self): """Create the dialog's widgets.""" + self.extension_names = StringVar(self) self.rowconfigure(0, weight=1) - self.rowconfigure(1, weight=0) - self.columnconfigure(0, weight=1) + self.columnconfigure(2, weight=1) + self.extension_list = Listbox(self, listvariable=self.extension_names, + selectmode='browse') + self.extension_list.bind('<>', self.extension_selected) + scroll = Scrollbar(self, command=self.extension_list.yview) + self.extension_list.yscrollcommand=scroll.set + self.details_frame = LabelFrame(self, width=250, height=250) + self.extension_list.grid(column=0, row=0, sticky='nws') + scroll.grid(column=1, row=0, sticky='ns') + self.details_frame.grid(column=2, row=0, sticky='nsew', padx=[10, 0]) + self.configure(padx=10, pady=10) + self.config_frame = {} + self.current_extension = None - # create the tabbed pages - self.tabbed_page_set = TabbedPageSet( - self, page_names=self.extensions.keys(), - n_rows=None, max_tabs_per_row=5, - page_class=TabbedPageSet.PageRemove) - self.tabbed_page_set.grid(row=0, column=0, sticky=NSEW) - for ext_name in self.extensions: - self.create_tab_page(ext_name) + self.outerframe = self # TEMPORARY + self.tabbed_page_set = self.extension_list # TEMPORARY - self.create_action_buttons().grid(row=1) + # create the individual pages + ext_names = '' + for ext_name in sorted(self.extensions): + self.create_extension_frame(ext_name) + ext_names = ext_names + '{' + ext_name + '} ' + self.extension_names.set(ext_names) + self.extension_list.selection_set(0) + self.extension_selected(None) + self.create_action_buttons().grid(row=1, columnspan=3) + + def extension_selected(self, event): + newsel = self.extension_list.curselection() + if newsel: + newsel = self.extension_list.get(newsel) + if newsel is None or newsel != self.current_extension: + if self.current_extension: + self.details_frame.config(text='') + self.config_frame[self.current_extension].grid_forget() + self.current_extension = None + if newsel: + self.details_frame.config(text=newsel) + self.config_frame[newsel].grid(column=0, row=0, sticky='nsew') + self.current_extension = newsel create_action_buttons = ConfigDialog.create_action_buttons - def create_tab_page(self, ext_name): - """Create the page for an extension.""" - - page = LabelFrame(self.tabbed_page_set.pages[ext_name].frame, - border=2, padx=2, relief=GROOVE, - text=' %s ' % ext_name) - page.pack(fill=BOTH, expand=True, padx=12, pady=2) - - # create the scrollable frame which will contain the entries - scrolled_frame = VerticalScrolledFrame(page, pady=2, height=250) - scrolled_frame.pack(side=BOTTOM, fill=BOTH, expand=TRUE) - entry_area = scrolled_frame.interior - entry_area.columnconfigure(0, weight=0) - entry_area.columnconfigure(1, weight=1) - + def create_extension_frame(self, ext_name): + """Create a frame holding the widgets to configure one extension""" + f = VerticalScrolledFrame(self.details_frame, height=250, width=250) + self.config_frame[ext_name] = f + entry_area = f.interior # create an entry for each configuration option for row, opt in enumerate(self.extensions[ext_name]): # create a row with a label and entry/checkbutton @@ -1365,15 +1380,15 @@ Checkbutton(entry_area, textvariable=var, variable=var, onvalue='True', offvalue='False', indicatoron=FALSE, selectcolor='', width=8 - ).grid(row=row, column=1, sticky=W, padx=7) + ).grid(row=row, column=1, sticky=W, padx=7) elif opt['type'] == 'int': Entry(entry_area, textvariable=var, validate='key', - validatecommand=(self.is_int, '%P') - ).grid(row=row, column=1, sticky=NSEW, padx=7) + validatecommand=(self.is_int, '%P') + ).grid(row=row, column=1, sticky=NSEW, padx=7) else: Entry(entry_area, textvariable=var - ).grid(row=row, column=1, sticky=NSEW, padx=7) + ).grid(row=row, column=1, sticky=NSEW, padx=7) return diff --git a/Lib/pdb.py b/Lib/pdb.py --- a/Lib/pdb.py +++ b/Lib/pdb.py @@ -1669,6 +1669,9 @@ # In most cases SystemExit does not warrant a post-mortem session. print("The program exited via sys.exit(). Exit status:", end=' ') print(sys.exc_info()[1]) + except SyntaxError: + traceback.print_exc() + sys.exit(1) except: traceback.print_exc() print("Uncaught exception. Entering post mortem debugging") diff --git a/Lib/test/test_cgi.py b/Lib/test/test_cgi.py --- a/Lib/test/test_cgi.py +++ b/Lib/test/test_cgi.py @@ -326,6 +326,24 @@ got = getattr(files[x], k) self.assertEqual(got, exp) + def test_fieldstorage_part_content_length(self): + BOUNDARY = "JfISa01" + POSTDATA = """--JfISa01 +Content-Disposition: form-data; name="submit-name" +Content-Length: 5 + +Larry +--JfISa01""" + env = { + 'REQUEST_METHOD': 'POST', + 'CONTENT_TYPE': 'multipart/form-data; boundary={}'.format(BOUNDARY), + 'CONTENT_LENGTH': str(len(POSTDATA))} + fp = BytesIO(POSTDATA.encode('latin-1')) + fs = cgi.FieldStorage(fp, environ=env, encoding="latin-1") + self.assertEqual(len(fs.list), 1) + self.assertEqual(fs.list[0].name, 'submit-name') + self.assertEqual(fs.list[0].value, 'Larry') + def test_fieldstorage_as_context_manager(self): fp = BytesIO(b'x' * 10) env = {'REQUEST_METHOD': 'PUT'} diff --git a/Lib/test/test_collections.py b/Lib/test/test_collections.py --- a/Lib/test/test_collections.py +++ b/Lib/test/test_collections.py @@ -257,7 +257,6 @@ self.assertEqual(p._fields, ('x', 'y')) # test _fields attribute self.assertEqual(p._replace(x=1), (1, 22)) # test _replace method self.assertEqual(p._asdict(), dict(x=11, y=22)) # test _asdict method - self.assertEqual(vars(p), p._asdict()) # verify that vars() works try: p._replace(x=1, error=2) @@ -412,6 +411,17 @@ globals().pop('NTColor', None) # clean-up after this test + def test_namedtuple_subclass_issue_24931(self): + class Point(namedtuple('_Point', ['x', 'y'])): + pass + + a = Point(3, 4) + self.assertEqual(a._asdict(), OrderedDict([('x', 3), ('y', 4)])) + + a.w = 5 + self.assertEqual(a.__dict__, {'w': 5}) + + ################################################################################ ### Abstract Base Classes ################################################################################ diff --git a/Lib/test/test_configparser.py b/Lib/test/test_configparser.py --- a/Lib/test/test_configparser.py +++ b/Lib/test/test_configparser.py @@ -847,7 +847,8 @@ "something with lots of interpolation (10 steps)") e = self.get_error(cf, configparser.InterpolationDepthError, "Foo", "bar11") if self.interpolation == configparser._UNSET: - self.assertEqual(e.args, ("bar11", "Foo", "%(with1)s")) + self.assertEqual(e.args, ("bar11", "Foo", + "something %(with11)s lots of interpolation (11 steps)")) elif isinstance(self.interpolation, configparser.LegacyInterpolation): self.assertEqual(e.args, ("bar11", "Foo", "something %(with11)s lots of interpolation (11 steps)")) @@ -861,7 +862,7 @@ self.assertEqual(e.option, "name") if self.interpolation == configparser._UNSET: self.assertEqual(e.args, ('name', 'Interpolation Error', - '', 'reference')) + '%(reference)s', 'reference')) elif isinstance(self.interpolation, configparser.LegacyInterpolation): self.assertEqual(e.args, ('name', 'Interpolation Error', '%(reference)s', 'reference')) @@ -1177,7 +1178,7 @@ with self.assertRaises(exception_class) as cm: cf['interpolated']['$trying'] self.assertEqual(cm.exception.reference, 'dollars:${sick') - self.assertEqual(cm.exception.args[2], '}') #rawval + self.assertEqual(cm.exception.args[2], '${dollars:${sick}}') #rawval def test_case_sensitivity_basic(self): ini = textwrap.dedent(""" diff --git a/Lib/test/test_functools.py b/Lib/test/test_functools.py --- a/Lib/test/test_functools.py +++ b/Lib/test/test_functools.py @@ -1491,6 +1491,24 @@ many_abcs = [c.Mapping, c.Sized, c.Callable, c.Container, c.Iterable] self.assertEqual(mro(X, abcs=many_abcs), expected) + def test_false_meta(self): + # see issue23572 + class MetaA(type): + def __len__(self): + return 0 + class A(metaclass=MetaA): + pass + class AA(A): + pass + @functools.singledispatch + def fun(a): + return 'base A' + @fun.register(A) + def _(a): + return 'fun A' + aa = AA() + self.assertEqual(fun(aa), 'fun A') + def test_mro_conflicts(self): c = collections @functools.singledispatch diff --git a/Lib/test/test_gdb.py b/Lib/test/test_gdb.py --- a/Lib/test/test_gdb.py +++ b/Lib/test/test_gdb.py @@ -21,19 +21,34 @@ from test import support from test.support import run_unittest, findfile, python_is_optimized -try: - gdb_version, _ = subprocess.Popen(["gdb", "-nx", "--version"], - stdout=subprocess.PIPE).communicate() -except OSError: - # This is what "no gdb" looks like. There may, however, be other - # errors that manifest this way too. - raise unittest.SkipTest("Couldn't find gdb on the path") -gdb_version_number = re.search(b"^GNU gdb [^\d]*(\d+)\.(\d)", gdb_version) -gdb_major_version = int(gdb_version_number.group(1)) -gdb_minor_version = int(gdb_version_number.group(2)) +def get_gdb_version(): + try: + proc = subprocess.Popen(["gdb", "-nx", "--version"], + stdout=subprocess.PIPE, + universal_newlines=True) + with proc: + version = proc.communicate()[0] + except OSError: + # This is what "no gdb" looks like. There may, however, be other + # errors that manifest this way too. + raise unittest.SkipTest("Couldn't find gdb on the path") + + # Regex to parse: + # 'GNU gdb (GDB; SUSE Linux Enterprise 12) 7.7\n' -> 7.7 + # 'GNU gdb (GDB) Fedora 7.9.1-17.fc22\n' -> 7.9 + # 'GNU gdb 6.1.1 [FreeBSD]\n' -> 6.1 + # 'GNU gdb (GDB) Fedora (7.5.1-37.fc18)\n' -> 7.5 + match = re.search(r"^GNU gdb.*?\b(\d+)\.(\d)", version) + if match is None: + raise Exception("unable to parse GDB version: %r" % version) + return (version, int(match.group(1)), int(match.group(2))) + +gdb_version, gdb_major_version, gdb_minor_version = get_gdb_version() if gdb_major_version < 7: - raise unittest.SkipTest("gdb versions before 7.0 didn't support python embedding" - " Saw:\n" + gdb_version.decode('ascii', 'replace')) + raise unittest.SkipTest("gdb versions before 7.0 didn't support python " + "embedding. Saw %s.%s:\n%s" + % (gdb_major_version, gdb_minor_version, + gdb_version)) if not sysconfig.is_python_build(): raise unittest.SkipTest("test_gdb only works on source builds at the moment.") @@ -59,9 +74,12 @@ base_cmd = ('gdb', '--batch', '-nx') if (gdb_major_version, gdb_minor_version) >= (7, 4): base_cmd += ('-iex', 'add-auto-load-safe-path ' + checkout_hook_path) - out, err = subprocess.Popen(base_cmd + args, - stdout=subprocess.PIPE, stderr=subprocess.PIPE, env=env, - ).communicate() + proc = subprocess.Popen(base_cmd + args, + stdout=subprocess.PIPE, + stderr=subprocess.PIPE, + env=env) + with proc: + out, err = proc.communicate() return out.decode('utf-8', 'replace'), err.decode('utf-8', 'replace') # Verify that "gdb" was built with the embedded python support enabled: @@ -880,8 +898,8 @@ def test_main(): if support.verbose: - print("GDB version:") - for line in os.fsdecode(gdb_version).splitlines(): + print("GDB version %s.%s:" % (gdb_major_version, gdb_minor_version)) + for line in gdb_version.splitlines(): print(" " * 4 + line) run_unittest(PrettyPrintTests, PyListTests, diff --git a/Lib/test/test_glob.py b/Lib/test/test_glob.py --- a/Lib/test/test_glob.py +++ b/Lib/test/test_glob.py @@ -242,9 +242,7 @@ ('a', 'bcd', 'EF'), ('a', 'bcd', 'efg'))) eq(self.rglob('a', '**', 'bcd'), self.joins(('a', 'bcd'))) - predir = os.path.abspath(os.curdir) - try: - os.chdir(self.tempdir) + with change_cwd(self.tempdir): join = os.path.join eq(glob.glob('**', recursive=True), [join(*i) for i in full]) eq(glob.glob(join('**', ''), recursive=True), @@ -256,8 +254,6 @@ if can_symlink(): expect += [join('sym3', 'EF')] eq(glob.glob(join('**', 'EF'), recursive=True), expect) - finally: - os.chdir(predir) @skip_unless_symlink diff --git a/Lib/test/test_htmlparser.py b/Lib/test/test_htmlparser.py --- a/Lib/test/test_htmlparser.py +++ b/Lib/test/test_htmlparser.py @@ -72,9 +72,6 @@ class EventCollectorCharrefs(EventCollector): - def get_events(self): - return self.events - def handle_charref(self, data): self.fail('This should never be called with convert_charrefs=True') @@ -633,6 +630,18 @@ ] self._run_check(html, expected) + def test_convert_charrefs_dropped_text(self): + # #23144: make sure that all the events are triggered when + # convert_charrefs is True, even if we don't call .close() + parser = EventCollector(convert_charrefs=True) + # before the fix, bar & baz was missing + parser.feed("foo link bar & baz") + self.assertEqual( + parser.get_events(), + [('data', 'foo '), ('starttag', 'a', []), ('data', 'link'), + ('endtag', 'a'), ('data', ' bar & baz')] + ) + class AttributesTestCase(TestCaseBase): diff --git a/Lib/test/test_mmap.py b/Lib/test/test_mmap.py --- a/Lib/test/test_mmap.py +++ b/Lib/test/test_mmap.py @@ -731,7 +731,10 @@ f.write(tail) f.flush() except (OSError, OverflowError): - f.close() + try: + f.close() + except (OSError, OverflowError): + pass raise unittest.SkipTest("filesystem does not have largefile support") return f diff --git a/Lib/test/test_pdb.py b/Lib/test/test_pdb.py --- a/Lib/test/test_pdb.py +++ b/Lib/test/test_pdb.py @@ -1043,6 +1043,18 @@ self.assertNotIn('Error', stdout.decode(), "Got an error running test script under PDB") + def test_issue16180(self): + # A syntax error in the debuggee. + script = "def f: pass\n" + commands = '' + expected = "SyntaxError:" + stdout, stderr = self.run_pdb(script, commands) + self.assertIn(expected, stdout, + '\n\nExpected:\n{}\nGot:\n{}\n' + 'Fail to handle a syntax error in the debuggee.' + .format(expected, stdout)) + + def tearDown(self): support.unlink(support.TESTFN) diff --git a/Lib/test/test_pep277.py b/Lib/test/test_pep277.py --- a/Lib/test/test_pep277.py +++ b/Lib/test/test_pep277.py @@ -158,17 +158,11 @@ def test_directory(self): dirname = os.path.join(support.TESTFN, 'Gr\xfc\xdf-\u66e8\u66e9\u66eb') filename = '\xdf-\u66e8\u66e9\u66eb' - oldwd = os.getcwd() - os.mkdir(dirname) - os.chdir(dirname) - try: + with support.temp_cwd(dirname): with open(filename, 'wb') as f: f.write((filename + '\n').encode("utf-8")) os.access(filename,os.R_OK) os.remove(filename) - finally: - os.chdir(oldwd) - os.rmdir(dirname) class UnicodeNFCFileTests(UnicodeFileTests): diff --git a/Lib/test/test_posixpath.py b/Lib/test/test_posixpath.py --- a/Lib/test/test_posixpath.py +++ b/Lib/test/test_posixpath.py @@ -316,7 +316,6 @@ # Bug #930024, return the path unchanged if we get into an infinite # symlink loop. try: - old_path = abspath('.') os.symlink(ABSTFN, ABSTFN) self.assertEqual(realpath(ABSTFN), ABSTFN) @@ -342,10 +341,9 @@ self.assertEqual(realpath(ABSTFN+"c"), ABSTFN+"c") # Test using relative path as well. - os.chdir(dirname(ABSTFN)) - self.assertEqual(realpath(basename(ABSTFN)), ABSTFN) + with support.change_cwd(dirname(ABSTFN)): + self.assertEqual(realpath(basename(ABSTFN)), ABSTFN) finally: - os.chdir(old_path) support.unlink(ABSTFN) support.unlink(ABSTFN+"1") support.unlink(ABSTFN+"2") @@ -373,7 +371,6 @@ @skip_if_ABSTFN_contains_backslash def test_realpath_deep_recursion(self): depth = 10 - old_path = abspath('.') try: os.mkdir(ABSTFN) for i in range(depth): @@ -382,10 +379,9 @@ self.assertEqual(realpath(ABSTFN + '/%d' % depth), ABSTFN) # Test using relative path as well. - os.chdir(ABSTFN) - self.assertEqual(realpath('%d' % depth), ABSTFN) + with support.change_cwd(ABSTFN): + self.assertEqual(realpath('%d' % depth), ABSTFN) finally: - os.chdir(old_path) for i in range(depth + 1): support.unlink(ABSTFN + '/%d' % i) safe_rmdir(ABSTFN) @@ -399,15 +395,13 @@ # /usr/doc with 'doc' being a symlink to /usr/share/doc. We call # realpath("a"). This should return /usr/share/doc/a/. try: - old_path = abspath('.') os.mkdir(ABSTFN) os.mkdir(ABSTFN + "/y") os.symlink(ABSTFN + "/y", ABSTFN + "/k") - os.chdir(ABSTFN + "/k") - self.assertEqual(realpath("a"), ABSTFN + "/y/a") + with support.change_cwd(ABSTFN + "/k"): + self.assertEqual(realpath("a"), ABSTFN + "/y/a") finally: - os.chdir(old_path) support.unlink(ABSTFN + "/k") safe_rmdir(ABSTFN + "/y") safe_rmdir(ABSTFN) @@ -424,7 +418,6 @@ # and a symbolic link 'link-y' pointing to 'y' in directory 'a', # then realpath("link-y/..") should return 'k', not 'a'. try: - old_path = abspath('.') os.mkdir(ABSTFN) os.mkdir(ABSTFN + "/k") os.mkdir(ABSTFN + "/k/y") @@ -433,11 +426,10 @@ # Absolute path. self.assertEqual(realpath(ABSTFN + "/link-y/.."), ABSTFN + "/k") # Relative path. - os.chdir(dirname(ABSTFN)) - self.assertEqual(realpath(basename(ABSTFN) + "/link-y/.."), - ABSTFN + "/k") + with support.change_cwd(dirname(ABSTFN)): + self.assertEqual(realpath(basename(ABSTFN) + "/link-y/.."), + ABSTFN + "/k") finally: - os.chdir(old_path) support.unlink(ABSTFN + "/link-y") safe_rmdir(ABSTFN + "/k/y") safe_rmdir(ABSTFN + "/k") @@ -451,17 +443,14 @@ # must be resolved too. try: - old_path = abspath('.') os.mkdir(ABSTFN) os.mkdir(ABSTFN + "/k") os.symlink(ABSTFN, ABSTFN + "link") - os.chdir(dirname(ABSTFN)) - - base = basename(ABSTFN) - self.assertEqual(realpath(base + "link"), ABSTFN) - self.assertEqual(realpath(base + "link/k"), ABSTFN + "/k") + with support.change_cwd(dirname(ABSTFN)): + base = basename(ABSTFN) + self.assertEqual(realpath(base + "link"), ABSTFN) + self.assertEqual(realpath(base + "link/k"), ABSTFN + "/k") finally: - os.chdir(old_path) support.unlink(ABSTFN + "link") safe_rmdir(ABSTFN + "/k") safe_rmdir(ABSTFN) diff --git a/Lib/test/test_py_compile.py b/Lib/test/test_py_compile.py --- a/Lib/test/test_py_compile.py +++ b/Lib/test/test_py_compile.py @@ -63,11 +63,9 @@ self.assertTrue(os.path.exists(self.cache_path)) def test_cwd(self): - cwd = os.getcwd() - os.chdir(self.directory) - py_compile.compile(os.path.basename(self.source_path), - os.path.basename(self.pyc_path)) - os.chdir(cwd) + with support.change_cwd(self.directory): + py_compile.compile(os.path.basename(self.source_path), + os.path.basename(self.pyc_path)) self.assertTrue(os.path.exists(self.pyc_path)) self.assertFalse(os.path.exists(self.cache_path)) diff --git a/Lib/test/test_shutil.py b/Lib/test/test_shutil.py --- a/Lib/test/test_shutil.py +++ b/Lib/test/test_shutil.py @@ -12,11 +12,9 @@ import functools import subprocess from contextlib import ExitStack -from test import support -from test.support import TESTFN from os.path import splitdrive from distutils.spawn import find_executable, spawn -from shutil import (_make_tarball, _make_zipfile, make_archive, +from shutil import (make_archive, register_archive_format, unregister_archive_format, get_archive_formats, Error, unpack_archive, register_unpack_format, RegistryError, @@ -94,6 +92,18 @@ with open(path, 'rb' if binary else 'r') as fp: return fp.read() +def rlistdir(path): + res = [] + for name in sorted(os.listdir(path)): + p = os.path.join(path, name) + if os.path.isdir(p) and not os.path.islink(p): + res.append(name + '/') + for n in rlistdir(p): + res.append(name + '/' + n) + else: + res.append(name) + return res + class TestShutil(unittest.TestCase): @@ -959,138 +969,105 @@ @requires_zlib def test_make_tarball(self): # creating something to tar - tmpdir = self.mkdtemp() - write_file((tmpdir, 'file1'), 'xxx') - write_file((tmpdir, 'file2'), 'xxx') - os.mkdir(os.path.join(tmpdir, 'sub')) - write_file((tmpdir, 'sub', 'file3'), 'xxx') + root_dir, base_dir = self._create_files('') tmpdir2 = self.mkdtemp() # force shutil to create the directory os.rmdir(tmpdir2) - unittest.skipUnless(splitdrive(tmpdir)[0] == splitdrive(tmpdir2)[0], + unittest.skipUnless(splitdrive(root_dir)[0] == splitdrive(tmpdir2)[0], "source and target should be on same drive") base_name = os.path.join(tmpdir2, 'archive') # working with relative paths to avoid tar warnings - old_dir = os.getcwd() - os.chdir(tmpdir) - try: - _make_tarball(splitdrive(base_name)[1], '.') - finally: - os.chdir(old_dir) + tarball = make_archive(splitdrive(base_name)[1], 'gztar', root_dir, '.') # check if the compressed tarball was created - tarball = base_name + '.tar.gz' - self.assertTrue(os.path.exists(tarball)) + self.assertEqual(tarball, base_name + '.tar.gz') + self.assertTrue(os.path.isfile(tarball)) + self.assertTrue(tarfile.is_tarfile(tarball)) + with tarfile.open(tarball, 'r:gz') as tf: + self.assertCountEqual(tf.getnames(), + ['.', './sub', './sub2', + './file1', './file2', './sub/file3']) # trying an uncompressed one - base_name = os.path.join(tmpdir2, 'archive') - old_dir = os.getcwd() - os.chdir(tmpdir) - try: - _make_tarball(splitdrive(base_name)[1], '.', compress=None) - finally: - os.chdir(old_dir) - tarball = base_name + '.tar' - self.assertTrue(os.path.exists(tarball)) + tarball = make_archive(splitdrive(base_name)[1], 'tar', root_dir, '.') + self.assertEqual(tarball, base_name + '.tar') + self.assertTrue(os.path.isfile(tarball)) + self.assertTrue(tarfile.is_tarfile(tarball)) + with tarfile.open(tarball, 'r') as tf: + self.assertCountEqual(tf.getnames(), + ['.', './sub', './sub2', + './file1', './file2', './sub/file3']) def _tarinfo(self, path): - tar = tarfile.open(path) - try: + with tarfile.open(path) as tar: names = tar.getnames() names.sort() return tuple(names) - finally: - tar.close() - def _create_files(self): + def _create_files(self, base_dir='dist'): # creating something to tar - tmpdir = self.mkdtemp() - dist = os.path.join(tmpdir, 'dist') - os.mkdir(dist) + root_dir = self.mkdtemp() + dist = os.path.join(root_dir, base_dir) + os.makedirs(dist, exist_ok=True) write_file((dist, 'file1'), 'xxx') write_file((dist, 'file2'), 'xxx') os.mkdir(os.path.join(dist, 'sub')) write_file((dist, 'sub', 'file3'), 'xxx') os.mkdir(os.path.join(dist, 'sub2')) - tmpdir2 = self.mkdtemp() - base_name = os.path.join(tmpdir2, 'archive') - return tmpdir, tmpdir2, base_name + if base_dir: + write_file((root_dir, 'outer'), 'xxx') + return root_dir, base_dir @requires_zlib - @unittest.skipUnless(find_executable('tar') and find_executable('gzip'), + @unittest.skipUnless(find_executable('tar'), 'Need the tar command to run') def test_tarfile_vs_tar(self): - tmpdir, tmpdir2, base_name = self._create_files() - old_dir = os.getcwd() - os.chdir(tmpdir) - try: - _make_tarball(base_name, 'dist') - finally: - os.chdir(old_dir) + root_dir, base_dir = self._create_files() + base_name = os.path.join(self.mkdtemp(), 'archive') + tarball = make_archive(base_name, 'gztar', root_dir, base_dir) # check if the compressed tarball was created - tarball = base_name + '.tar.gz' - self.assertTrue(os.path.exists(tarball)) + self.assertEqual(tarball, base_name + '.tar.gz') + self.assertTrue(os.path.isfile(tarball)) # now create another tarball using `tar` - tarball2 = os.path.join(tmpdir, 'archive2.tar.gz') - tar_cmd = ['tar', '-cf', 'archive2.tar', 'dist'] - gzip_cmd = ['gzip', '-f9', 'archive2.tar'] - old_dir = os.getcwd() - os.chdir(tmpdir) - try: - with captured_stdout() as s: - spawn(tar_cmd) - spawn(gzip_cmd) - finally: - os.chdir(old_dir) + tarball2 = os.path.join(root_dir, 'archive2.tar') + tar_cmd = ['tar', '-cf', 'archive2.tar', base_dir] + with support.change_cwd(root_dir), captured_stdout(): + spawn(tar_cmd) - self.assertTrue(os.path.exists(tarball2)) + self.assertTrue(os.path.isfile(tarball2)) # let's compare both tarballs self.assertEqual(self._tarinfo(tarball), self._tarinfo(tarball2)) # trying an uncompressed one - base_name = os.path.join(tmpdir2, 'archive') - old_dir = os.getcwd() - os.chdir(tmpdir) - try: - _make_tarball(base_name, 'dist', compress=None) - finally: - os.chdir(old_dir) - tarball = base_name + '.tar' - self.assertTrue(os.path.exists(tarball)) + tarball = make_archive(base_name, 'tar', root_dir, base_dir) + self.assertEqual(tarball, base_name + '.tar') + self.assertTrue(os.path.isfile(tarball)) # now for a dry_run - base_name = os.path.join(tmpdir2, 'archive') - old_dir = os.getcwd() - os.chdir(tmpdir) - try: - _make_tarball(base_name, 'dist', compress=None, dry_run=True) - finally: - os.chdir(old_dir) - tarball = base_name + '.tar' - self.assertTrue(os.path.exists(tarball)) + tarball = make_archive(base_name, 'tar', root_dir, base_dir, + dry_run=True) + self.assertEqual(tarball, base_name + '.tar') + self.assertTrue(os.path.isfile(tarball)) @requires_zlib @unittest.skipUnless(ZIP_SUPPORT, 'Need zip support to run') def test_make_zipfile(self): - # creating something to tar - tmpdir = self.mkdtemp() - write_file((tmpdir, 'file1'), 'xxx') - write_file((tmpdir, 'file2'), 'xxx') + # creating something to zip + root_dir, base_dir = self._create_files() + base_name = os.path.join(self.mkdtemp(), 'archive') + res = make_archive(base_name, 'zip', root_dir, 'dist') - tmpdir2 = self.mkdtemp() - # force shutil to create the directory - os.rmdir(tmpdir2) - base_name = os.path.join(tmpdir2, 'archive') - _make_zipfile(base_name, tmpdir) - - # check if the compressed tarball was created - tarball = base_name + '.zip' - self.assertTrue(os.path.exists(tarball)) + self.assertEqual(res, base_name + '.zip') + self.assertTrue(os.path.isfile(res)) + self.assertTrue(zipfile.is_zipfile(res)) + with zipfile.ZipFile(res) as zf: + self.assertCountEqual(zf.namelist(), + ['dist/file1', 'dist/file2', 'dist/sub/file3']) def test_make_archive(self): @@ -1108,40 +1085,37 @@ else: group = owner = 'root' - base_dir, root_dir, base_name = self._create_files() - base_name = os.path.join(self.mkdtemp() , 'archive') + root_dir, base_dir = self._create_files() + base_name = os.path.join(self.mkdtemp(), 'archive') res = make_archive(base_name, 'zip', root_dir, base_dir, owner=owner, group=group) - self.assertTrue(os.path.exists(res)) + self.assertTrue(os.path.isfile(res)) res = make_archive(base_name, 'zip', root_dir, base_dir) - self.assertTrue(os.path.exists(res)) + self.assertTrue(os.path.isfile(res)) res = make_archive(base_name, 'tar', root_dir, base_dir, owner=owner, group=group) - self.assertTrue(os.path.exists(res)) + self.assertTrue(os.path.isfile(res)) res = make_archive(base_name, 'tar', root_dir, base_dir, owner='kjhkjhkjg', group='oihohoh') - self.assertTrue(os.path.exists(res)) + self.assertTrue(os.path.isfile(res)) @requires_zlib @unittest.skipUnless(UID_GID_SUPPORT, "Requires grp and pwd support") def test_tarfile_root_owner(self): - tmpdir, tmpdir2, base_name = self._create_files() - old_dir = os.getcwd() - os.chdir(tmpdir) + root_dir, base_dir = self._create_files() + base_name = os.path.join(self.mkdtemp(), 'archive') group = grp.getgrgid(0)[0] owner = pwd.getpwuid(0)[0] - try: - archive_name = _make_tarball(base_name, 'dist', compress=None, - owner=owner, group=group) - finally: - os.chdir(old_dir) + with support.change_cwd(root_dir): + archive_name = make_archive(base_name, 'gztar', root_dir, 'dist', + owner=owner, group=group) # check if the compressed tarball was created - self.assertTrue(os.path.exists(archive_name)) + self.assertTrue(os.path.isfile(archive_name)) # now checks the rights archive = tarfile.open(archive_name) @@ -1198,18 +1172,6 @@ formats = [name for name, params in get_archive_formats()] self.assertNotIn('xxx', formats) - def _compare_dirs(self, dir1, dir2): - # check that dir1 and dir2 are equivalent, - # return the diff - diff = [] - for root, dirs, files in os.walk(dir1): - for file_ in files: - path = os.path.join(root, file_) - target_path = os.path.join(dir2, os.path.split(path)[-1]) - if not os.path.exists(target_path): - diff.append(file_) - return diff - @requires_zlib def test_unpack_archive(self): formats = ['tar', 'gztar', 'zip'] @@ -1218,22 +1180,24 @@ if LZMA_SUPPORTED: formats.append('xztar') + root_dir, base_dir = self._create_files() for format in formats: - tmpdir = self.mkdtemp() - base_dir, root_dir, base_name = self._create_files() - tmpdir2 = self.mkdtemp() + expected = rlistdir(root_dir) + expected.remove('outer') + if format == 'zip': + expected.remove('dist/sub2/') + base_name = os.path.join(self.mkdtemp(), 'archive') filename = make_archive(base_name, format, root_dir, base_dir) # let's try to unpack it now + tmpdir2 = self.mkdtemp() unpack_archive(filename, tmpdir2) - diff = self._compare_dirs(tmpdir, tmpdir2) - self.assertEqual(diff, []) + self.assertEqual(rlistdir(tmpdir2), expected) # and again, this time with the format specified tmpdir3 = self.mkdtemp() unpack_archive(filename, tmpdir3, format=format) - diff = self._compare_dirs(tmpdir, tmpdir3) - self.assertEqual(diff, []) + self.assertEqual(rlistdir(tmpdir3), expected) self.assertRaises(shutil.ReadError, unpack_archive, TESTFN) self.assertRaises(ValueError, unpack_archive, TESTFN, format='xxx') diff --git a/Lib/test/test_subprocess.py b/Lib/test/test_subprocess.py --- a/Lib/test/test_subprocess.py +++ b/Lib/test/test_subprocess.py @@ -317,11 +317,8 @@ # Normalize an expected cwd (for Tru64 support). # We can't use os.path.realpath since it doesn't expand Tru64 {memb} # strings. See bug #1063571. - original_cwd = os.getcwd() - os.chdir(cwd) - cwd = os.getcwd() - os.chdir(original_cwd) - return cwd + with support.change_cwd(cwd): + return os.getcwd() # For use in the test_cwd* tests below. def _split_python_path(self): diff --git a/Lib/test/test_sysconfig.py b/Lib/test/test_sysconfig.py --- a/Lib/test/test_sysconfig.py +++ b/Lib/test/test_sysconfig.py @@ -6,7 +6,7 @@ from copy import copy from test.support import (run_unittest, TESTFN, unlink, check_warnings, - captured_stdout, skip_unless_symlink) + captured_stdout, skip_unless_symlink, change_cwd) import sysconfig from sysconfig import (get_paths, get_platform, get_config_vars, @@ -361,12 +361,8 @@ # srcdir should be independent of the current working directory # See Issues #15322, #15364. srcdir = sysconfig.get_config_var('srcdir') - cwd = os.getcwd() - try: - os.chdir('..') + with change_cwd(os.pardir): srcdir2 = sysconfig.get_config_var('srcdir') - finally: - os.chdir(cwd) self.assertEqual(srcdir, srcdir2) @unittest.skipIf(sysconfig.get_config_var('EXT_SUFFIX') is None, diff --git a/Lib/test/test_tarfile.py b/Lib/test/test_tarfile.py --- a/Lib/test/test_tarfile.py +++ b/Lib/test/test_tarfile.py @@ -1132,10 +1132,8 @@ self.assertEqual(tar.getnames(), [], "added the archive to itself") - cwd = os.getcwd() - os.chdir(TEMPDIR) - tar.add(dstname) - os.chdir(cwd) + with support.change_cwd(TEMPDIR): + tar.add(dstname) self.assertEqual(tar.getnames(), [], "added the archive to itself") finally: @@ -1292,9 +1290,7 @@ def test_cwd(self): # Test adding the current working directory. - cwd = os.getcwd() - os.chdir(TEMPDIR) - try: + with support.change_cwd(TEMPDIR): tar = tarfile.open(tmpname, self.mode) try: tar.add(".") @@ -1308,8 +1304,6 @@ self.assertTrue(t.name.startswith("./"), t.name) finally: tar.close() - finally: - os.chdir(cwd) def test_open_nonwritable_fileobj(self): for exctype in OSError, EOFError, RuntimeError: diff --git a/Lib/test/test_unicode_file.py b/Lib/test/test_unicode_file.py --- a/Lib/test/test_unicode_file.py +++ b/Lib/test/test_unicode_file.py @@ -5,7 +5,7 @@ import unicodedata import unittest -from test.support import (run_unittest, rmtree, +from test.support import (run_unittest, rmtree, change_cwd, TESTFN_ENCODING, TESTFN_UNICODE, TESTFN_UNENCODABLE, create_empty_file) if not os.path.supports_unicode_filenames: @@ -82,13 +82,11 @@ self.assertFalse(os.path.exists(filename2 + '.new')) def _do_directory(self, make_name, chdir_name): - cwd = os.getcwd() if os.path.isdir(make_name): rmtree(make_name) os.mkdir(make_name) try: - os.chdir(chdir_name) - try: + with change_cwd(chdir_name): cwd_result = os.getcwd() name_result = make_name @@ -96,8 +94,6 @@ name_result = unicodedata.normalize("NFD", name_result) self.assertEqual(os.path.basename(cwd_result),name_result) - finally: - os.chdir(cwd) finally: os.rmdir(make_name) diff --git a/Lib/test/test_warnings/__init__.py b/Lib/test/test_warnings/__init__.py --- a/Lib/test/test_warnings/__init__.py +++ b/Lib/test/test_warnings/__init__.py @@ -44,6 +44,7 @@ """Basic bookkeeping required for testing.""" def setUp(self): + self.old_unittest_module = unittest.case.warnings # The __warningregistry__ needs to be in a pristine state for tests # to work properly. if '__warningregistry__' in globals(): @@ -55,10 +56,15 @@ # The 'warnings' module must be explicitly set so that the proper # interaction between _warnings and 'warnings' can be controlled. sys.modules['warnings'] = self.module + # Ensure that unittest.TestCase.assertWarns() uses the same warnings + # module than warnings.catch_warnings(). Otherwise, + # warnings.catch_warnings() will be unable to remove the added filter. + unittest.case.warnings = self.module super(BaseTest, self).setUp() def tearDown(self): sys.modules['warnings'] = original_warnings + unittest.case.warnings = self.old_unittest_module super(BaseTest, self).tearDown() class PublicAPITests(BaseTest): diff --git a/Lib/test/test_wsgiref.py b/Lib/test/test_wsgiref.py --- a/Lib/test/test_wsgiref.py +++ b/Lib/test/test_wsgiref.py @@ -1,11 +1,10 @@ -from __future__ import nested_scopes # Backward compat for 2.1 from unittest import TestCase from wsgiref.util import setup_testing_defaults from wsgiref.headers import Headers from wsgiref.handlers import BaseHandler, BaseCGIHandler from wsgiref import util from wsgiref.validate import validator -from wsgiref.simple_server import WSGIServer, WSGIRequestHandler, demo_app +from wsgiref.simple_server import WSGIServer, WSGIRequestHandler from wsgiref.simple_server import make_server from io import StringIO, BytesIO, BufferedReader from socketserver import BaseServer @@ -14,8 +13,8 @@ import os import re import sys +import unittest -from test import support class MockServer(WSGIServer): """Non-socket HTTP server""" diff --git a/Lib/unittest/case.py b/Lib/unittest/case.py --- a/Lib/unittest/case.py +++ b/Lib/unittest/case.py @@ -583,8 +583,11 @@ finally: result.stopTest(self) return - expecting_failure = getattr(testMethod, - "__unittest_expecting_failure__", False) + expecting_failure_method = getattr(testMethod, + "__unittest_expecting_failure__", False) + expecting_failure_class = getattr(self, + "__unittest_expecting_failure__", False) + expecting_failure = expecting_failure_class or expecting_failure_method outcome = _Outcome(result) try: self._outcome = outcome @@ -1279,8 +1282,10 @@ assert expected_regex, "expected_regex must not be empty." expected_regex = re.compile(expected_regex) if not expected_regex.search(text): - msg = msg or "Regex didn't match" - msg = '%s: %r not found in %r' % (msg, expected_regex.pattern, text) + standardMsg = "Regex didn't match: %r not found in %r" % ( + expected_regex.pattern, text) + # _formatMessage ensures the longMessage option is respected + msg = self._formatMessage(msg, standardMsg) raise self.failureException(msg) def assertNotRegex(self, text, unexpected_regex, msg=None): @@ -1289,11 +1294,12 @@ unexpected_regex = re.compile(unexpected_regex) match = unexpected_regex.search(text) if match: - msg = msg or "Regex matched" - msg = '%s: %r matches %r in %r' % (msg, - text[match.start():match.end()], - unexpected_regex.pattern, - text) + standardMsg = 'Regex matched: %r matches %r in %r' % ( + text[match.start() : match.end()], + unexpected_regex.pattern, + text) + # _formatMessage ensures the longMessage option is respected + msg = self._formatMessage(msg, standardMsg) raise self.failureException(msg) @@ -1315,6 +1321,7 @@ failIf = _deprecate(assertFalse) assertRaisesRegexp = _deprecate(assertRaisesRegex) assertRegexpMatches = _deprecate(assertRegex) + assertNotRegexpMatches = _deprecate(assertNotRegex) diff --git a/Lib/unittest/test/test_assertions.py b/Lib/unittest/test/test_assertions.py --- a/Lib/unittest/test/test_assertions.py +++ b/Lib/unittest/test/test_assertions.py @@ -133,7 +133,6 @@ try: self.assertNotRegex('Ala ma kota', r'k.t', 'Message') except self.failureException as e: - self.assertIn("'kot'", e.args[0]) self.assertIn('Message', e.args[0]) else: self.fail('assertNotRegex should have failed.') @@ -329,6 +328,20 @@ "^unexpectedly identical: None$", "^unexpectedly identical: None : oops$"]) + def testAssertRegex(self): + self.assertMessages('assertRegex', ('foo', 'bar'), + ["^Regex didn't match:", + "^oops$", + "^Regex didn't match:", + "^Regex didn't match: (.*) : oops$"]) + + def testAssertNotRegex(self): + self.assertMessages('assertNotRegex', ('foo', 'foo'), + ["^Regex matched:", + "^oops$", + "^Regex matched:", + "^Regex matched: (.*) : oops$"]) + def assertMessagesCM(self, methodName, args, func, errors): """ diff --git a/Lib/unittest/test/test_skipping.py b/Lib/unittest/test/test_skipping.py --- a/Lib/unittest/test/test_skipping.py +++ b/Lib/unittest/test/test_skipping.py @@ -120,6 +120,39 @@ self.assertEqual(result.expectedFailures[0][0], test) self.assertTrue(result.wasSuccessful()) + def test_expected_failure_with_wrapped_class(self): + @unittest.expectedFailure + class Foo(unittest.TestCase): + def test_1(self): + self.assertTrue(False) + + events = [] + result = LoggingResult(events) + test = Foo("test_1") + test.run(result) + self.assertEqual(events, + ['startTest', 'addExpectedFailure', 'stopTest']) + self.assertEqual(result.expectedFailures[0][0], test) + self.assertTrue(result.wasSuccessful()) + + def test_expected_failure_with_wrapped_subclass(self): + class Foo(unittest.TestCase): + def test_1(self): + self.assertTrue(False) + + @unittest.expectedFailure + class Bar(Foo): + pass + + events = [] + result = LoggingResult(events) + test = Bar("test_1") + test.run(result) + self.assertEqual(events, + ['startTest', 'addExpectedFailure', 'stopTest']) + self.assertEqual(result.expectedFailures[0][0], test) + self.assertTrue(result.wasSuccessful()) + def test_expected_failure_subtests(self): # A failure in any subtest counts as the expected failure of the # whole test. diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -103,6 +103,7 @@ Samuel L. Bayer Donald Beaudry David Beazley +John Beck Ingolf Becker Neal Becker Robin Becker @@ -622,6 +623,7 @@ Brad Howes Mike Hoy Ben Hoyt +Chiu-Hsiang Hsu Chih-Hao Huang Christian Hudon Lawrence Hudson @@ -785,6 +787,7 @@ Dave Kuhlman Jon Kuhn Toshio Kuratomi +Ilia Kurenkov Vladimir Kushnir Erno Kuusela Ross Lagerwall @@ -794,6 +797,7 @@ Valerie Lambert Jean-Baptiste "Jiba" Lamy Ronan Lamy +Peter Landry Torsten Landschoff ?ukasz Langa Tino Lange @@ -911,12 +915,14 @@ Simon Mathieu Laura Matson Graham Matthews +mattip Martin Matusiak Dieter Maurer Daniel May Madison May Lucas Maystre Arnaud Mazin +Pam McA'Nulty Matt McClure Jack McCracken Rebecca McCreary @@ -1061,6 +1067,7 @@ Yongzhi Pan Martin Panter Mathias Panzenb?ck +Marco Paolini M. Papillon Peter Parente Alexandre Parenteau @@ -1121,6 +1128,7 @@ Iustin Pop Claudiu Popa John Popplewell +Matheus Vieira Portela Davin Potts Guillaume Pratte Florian Preinstorfer @@ -1182,6 +1190,7 @@ Wes Rishel Daniel Riti Juan M. Bello Rivas +Mohd Sanad Zaki Rizvi Davide Rizzo Anthony Roach Carl Robben @@ -1514,6 +1523,7 @@ Edward Welbourne Cliff Wells Rickard Westman +Joseph Weston Jeff Wheeler Christopher White David White diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -2,6 +2,84 @@ Python News +++++++++++ + +What's New in Python 3.5.1 +========================== + +Release date: TBA + +Core and Builtins +----------------- + +Library +------- + +- Issue #16180: Exit pdb if file has syntax error, instead of trapping user + in an infinite loop. Patch by Xavier de Gaye. + +- Issue #24891: Fix a race condition at Python startup if the file descriptor + of stdin (0), stdout (1) or stderr (2) is closed while Python is creating + sys.stdin, sys.stdout and sys.stderr objects. These attributes are now set + to None if the creation of the object failed, instead of raising an OSError + exception. Initial patch written by Marco Paolini. + +- Issue #24992: Fix error handling and a race condition (related to garbage + collection) in collections.OrderedDict constructor. + +- Issue #24881: Fixed setting binary mode in Python implementation of FileIO + on Windows and Cygwin. Patch from Akira Li. + +- Issue #21112: Fix regression in unittest.expectedFailure on subclasses. + Patch from Berker Peksag. + +- Issue #24764: cgi.FieldStorage.read_multi() now ignores the Content-Length + header in part headers. Patch written by Peter Landry and reviewed by Pierre + Quentel. + +- Issue #24913: Fix overrun error in deque.index(). + Found by John Leitch and Bryce Darling. + +- Issue #24774: Fix docstring in http.server.test. Patch from Chiu-Hsiang Hsu. + +- Issue #21159: Improve message in configparser.InterpolationMissingOptionError. + Patch from ?ukasz Langa. + +- Issue #20362: Honour TestCase.longMessage correctly in assertRegex. + Patch from Ilia Kurenkov. + +- Issue #23572: Fixed functools.singledispatch on classes with falsy + metaclasses. Patch by Ethan Furman. + +Documentation +------------- + +- Issue #24952: Clarify the default size argument of stack_size() in + the "threading" and "_thread" modules. Patch from Mattip. + +- Issue #23725: Overhaul tempfile docs. Note deprecated status of mktemp. + Patch from Zbigniew J?drzejewski-Szmek. + +- Issue #24808: Update the types of some PyTypeObject fields. Patch by + Joseph Weston. + +- Issue #22812: Fix unittest discovery examples. + Patch from Pam McA'Nulty. + +Tests +----- + +- PCbuild\rt.bat now accepts an unlimited number of arguments to pass along + to regrtest.py. Previously there was a limit of 9. + +Build +----- + +- Issue #24910: Windows MSIs now have unique display names. + +- Issue #24986: It is now possible to build Python on Windows without errors + when external libraries are not available. + + What's New in Python 3.5.0 release candidate 3? =============================================== @@ -21,6 +99,8 @@ ------- - Issue #24917: time_strftime() buffer over-read. +- Issue #23144: Make sure that HTMLParser.feed() returns all the data, even + when convert_charrefs is True. - Issue #24748: To resolve a compatibility problem found with py2exe and pywin32, imp.load_dynamic() once again ignores previously loaded modules @@ -30,6 +110,7 @@ - Issue #24635: Fixed a bug in typing.py where isinstance([], typing.Iterable) would return True once, then False on subsequent calls. + - Issue #24989: Fixed buffer overread in BytesIO.readline() if a position is set beyond size. Based on patch by John Leitch. diff --git a/Misc/Porting b/Misc/Porting --- a/Misc/Porting +++ b/Misc/Porting @@ -1,41 +1,1 @@ -Q. I want to port Python to a new platform. How do I begin? - -A. I guess the two things to start with is to familiarize yourself -with are the development system for your target platform and the -generic build process for Python. Make sure you can compile and run a -simple hello-world program on your target platform. Make sure you can -compile and run the Python interpreter on a platform to which it has -already been ported (preferably Unix, but Mac or Windows will do, -too). - -I also would never start something like this without at least -medium-level understanding of your target platform (i.e. how it is -generally used, how to write platform specific apps etc.) and Python -(or else you'll never know how to test the results). - -The build process for Python, in particular the Makefiles in the -source distribution, will give you a hint on which files to compile -for Python. Not all source files are relevant -- some are platform -specific, others are only used in emergencies (e.g. getopt.c). The -Makefiles tell the story. - -You'll also need a pyconfig.h file tailored for your platform. You can -start with pyconfig.h.in, read the comments and turn on definitions that -apply to your platform. - -And you'll need a config.c file, which lists the built-in modules you -support. Start with Modules/config.c.in. - -Finally, you'll run into some things that aren't supported on your -target platform. Forget about the posix module for now -- simply take -it out of the config.c file. - -Bang on it until you get a >>> prompt. (You may have to disable the -importing of "site.py" by passing the -S option.) - -Then bang on it until it executes very simple Python statements. - -Now bang on it some more. At some point you'll want to use the os -module; this is the time to start thinking about what to do with the -posix module. It's okay to simply #ifdef out those functions that -cause problems; the remaining ones will be quite useful. +This document is moved to https://docs.python.org/devguide/faq.html#how-do-i-port-python-to-a-new-platform diff --git a/Modules/_collectionsmodule.c b/Modules/_collectionsmodule.c --- a/Modules/_collectionsmodule.c +++ b/Modules/_collectionsmodule.c @@ -419,9 +419,11 @@ deque->rightblock->data[deque->rightindex] = item; deque_trim_left(deque); } + if (PyErr_Occurred()) { + Py_DECREF(it); + return NULL; + } Py_DECREF(it); - if (PyErr_Occurred()) - return NULL; Py_RETURN_NONE; } @@ -480,9 +482,11 @@ deque->leftblock->data[deque->leftindex] = item; deque_trim_right(deque); } + if (PyErr_Occurred()) { + Py_DECREF(it); + return NULL; + } Py_DECREF(it); - if (PyErr_Occurred()) - return NULL; Py_RETURN_NONE; } @@ -497,8 +501,8 @@ result = deque_extend(deque, other); if (result == NULL) return result; + Py_INCREF(deque); Py_DECREF(result); - Py_INCREF(deque); return (PyObject *)deque; } @@ -1260,8 +1264,8 @@ aslist, ((dequeobject *)deque)->maxlen); else result = PyUnicode_FromFormat("deque(%R)", aslist); + Py_ReprLeave(deque); Py_DECREF(aslist); - Py_ReprLeave(deque); return result; } diff --git a/Modules/_decimal/libmpdec/mpdecimal.c b/Modules/_decimal/libmpdec/mpdecimal.c --- a/Modules/_decimal/libmpdec/mpdecimal.c +++ b/Modules/_decimal/libmpdec/mpdecimal.c @@ -43,6 +43,7 @@ #ifdef PPRO #if defined(_MSC_VER) #include + #pragma float_control(precise, on) #pragma fenv_access(on) #elif !defined(__OpenBSD__) && !defined(__NetBSD__) /* C99 */ diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c --- a/Modules/posixmodule.c +++ b/Modules/posixmodule.c @@ -4560,9 +4560,7 @@ } \ -#define UTIME_HAVE_DIR_FD (defined(HAVE_FUTIMESAT) || defined(HAVE_UTIMENSAT)) - -#if UTIME_HAVE_DIR_FD +#if defined(HAVE_FUTIMESAT) || defined(HAVE_UTIMENSAT) static int utime_dir_fd(utime_t *ut, int dir_fd, char *path, int follow_symlinks) @@ -4588,9 +4586,7 @@ #define FUTIMENSAT_DIR_FD_CONVERTER dir_fd_unavailable #endif -#define UTIME_HAVE_FD (defined(HAVE_FUTIMES) || defined(HAVE_FUTIMENS)) - -#if UTIME_HAVE_FD +#if defined(HAVE_FUTIMES) || defined(HAVE_FUTIMENS) static int utime_fd(utime_t *ut, int fd) @@ -4835,13 +4831,13 @@ else #endif -#if UTIME_HAVE_DIR_FD +#if defined(HAVE_FUTIMESAT) || defined(HAVE_UTIMENSAT) if ((dir_fd != DEFAULT_DIR_FD) || (!follow_symlinks)) result = utime_dir_fd(&utime, dir_fd, path->narrow, follow_symlinks); else #endif -#if UTIME_HAVE_FD +#if defined(HAVE_FUTIMES) || defined(HAVE_FUTIMENS) if (path->fd != -1) result = utime_fd(&utime, path->fd); else diff --git a/Objects/odictobject.c b/Objects/odictobject.c --- a/Objects/odictobject.c +++ b/Objects/odictobject.c @@ -98,7 +98,6 @@ Others: -* _odict_initialize(od) * _odict_find_node(od, key) * _odict_keys_equal(od1, od2) @@ -602,15 +601,6 @@ return _odict_get_index_hash(od, key, hash); } -static int -_odict_initialize(PyODictObject *od) -{ - od->od_state = 0; - _odict_FIRST(od) = NULL; - _odict_LAST(od) = NULL; - return _odict_resize((PyODictObject *)od); -} - /* Returns NULL if there was some error or the key was not found. */ static _ODictNode * _odict_find_node(PyODictObject *od, PyObject *key) @@ -744,7 +734,7 @@ /* If someone calls PyDict_DelItem() directly on an OrderedDict, we'll get all sorts of problems here. In PyODict_DelItem we make sure to call _odict_clear_node first. - + This matters in the case of colliding keys. Suppose we add 3 keys: [A, B, C], where the hash of C collides with A and the next possible index in the hash table is occupied by B. If we remove B then for C @@ -1739,14 +1729,28 @@ static PyObject * odict_new(PyTypeObject *type, PyObject *args, PyObject *kwds) { - PyObject *od = PyDict_Type.tp_new(type, args, kwds); - if (od != NULL) { - if (_odict_initialize((PyODictObject *)od) < 0) - return NULL; - ((PyODictObject *)od)->od_inst_dict = PyDict_New(); - ((PyODictObject *)od)->od_weakreflist = NULL; + PyObject *dict; + PyODictObject *od; + + dict = PyDict_New(); + if (dict == NULL) + return NULL; + + od = (PyODictObject *)PyDict_Type.tp_new(type, args, kwds); + if (od == NULL) { + Py_DECREF(dict); + return NULL; } - return od; + + od->od_inst_dict = dict; + /* type constructor fills the memory with zeros (see + PyType_GenericAlloc()), there is no need to set them to zero again */ + if (_odict_resize(od) < 0) { + Py_DECREF(od); + return NULL; + } + + return (PyObject*)od; } /* PyODict_Type */ diff --git a/PC/pyconfig.h b/PC/pyconfig.h --- a/PC/pyconfig.h +++ b/PC/pyconfig.h @@ -147,7 +147,11 @@ #define MS_WINI64 #define PYD_PLATFORM_TAG "win_ia64" #elif defined(_M_X64) || defined(_M_AMD64) +#if defined(__INTEL_COMPILER) +#define COMPILER ("[ICC v." _Py_STRINGIZE(__INTEL_COMPILER) " 64 bit (amd64) with MSC v." _Py_STRINGIZE(_MSC_VER) " CRT]") +#else #define COMPILER _Py_PASTE_VERSION("64 bit (AMD64)") +#endif /* __INTEL_COMPILER */ #define MS_WINX64 #define PYD_PLATFORM_TAG "win_amd64" #else @@ -194,7 +198,11 @@ #if defined(MS_WIN32) && !defined(MS_WIN64) #if defined(_M_IX86) +#if defined(__INTEL_COMPILER) +#define COMPILER ("[ICC v." _Py_STRINGIZE(__INTEL_COMPILER) " 32 bit (Intel) with MSC v." _Py_STRINGIZE(_MSC_VER) " CRT]") +#else #define COMPILER _Py_PASTE_VERSION("32 bit (Intel)") +#endif /* __INTEL_COMPILER */ #define PYD_PLATFORM_TAG "win32" #elif defined(_M_ARM) #define COMPILER _Py_PASTE_VERSION("32 bit (ARM)") diff --git a/PCbuild/build.bat b/PCbuild/build.bat --- a/PCbuild/build.bat +++ b/PCbuild/build.bat @@ -1,19 +1,46 @@ @echo off -rem A batch program to build or rebuild a particular configuration, -rem just for convenience. +goto Run +:Usage +echo.%~nx0 [flags and arguments] [quoted MSBuild options] +echo. +echo.Build CPython from the command line. Requires the appropriate +echo.version(s) of Microsoft Visual Studio to be installed (see readme.txt). +echo.Also requires Subversion (svn.exe) to be on PATH if the '-e' flag is +echo.given. +echo. +echo.After the flags recognized by this script, up to 9 arguments to be passed +echo.directly to MSBuild may be passed. If the argument contains an '=', the +echo.entire argument must be quoted (e.g. `%~nx0 "/p:PlatformToolset=v100"`) +echo. +echo.Available flags: +echo. -h Display this help message +echo. -V Display version information for the current build +echo. -r Target Rebuild instead of Build +echo. -d Set the configuration to Debug +echo. -e Build external libraries fetched by get_externals.bat +echo. Extension modules that depend on external libraries will not attempt +echo. to build if this flag is not present +echo. -m Enable parallel build (enabled by default) +echo. -M Disable parallel build +echo. -v Increased output messages +echo. -k Attempt to kill any running Pythons before building (usually done +echo. automatically by the pythoncore project) +echo. +echo.Available flags to avoid building certain modules. +echo.These flags have no effect if '-e' is not given: +echo. --no-ssl Do not attempt to build _ssl +echo. --no-tkinter Do not attempt to build Tkinter +echo. +echo.Available arguments: +echo. -c Release ^| Debug ^| PGInstrument ^| PGUpdate +echo. Set the configuration (default: Release) +echo. -p x64 ^| Win32 +echo. Set the platform (default: Win32) +echo. -t Build ^| Rebuild ^| Clean ^| CleanAll +echo. Set the target manually +exit /b 127 -rem Arguments: -rem -c Set the configuration (default: Release) -rem -p Set the platform (x64 or Win32, default: Win32) -rem -r Target Rebuild instead of Build -rem -t Set the target manually (Build, Rebuild, Clean, or CleanAll) -rem -d Set the configuration to Debug -rem -e Pull in external libraries using get_externals.bat -rem -m Enable parallel build (enabled by default) -rem -M Disable parallel build -rem -v Increased output messages -rem -k Attempt to kill any running Pythons before building (usually unnecessary) - +:Run setlocal set platf=Win32 set vs_platf=x86 @@ -25,17 +52,29 @@ set kill= :CheckOpts +if "%~1"=="-h" goto Usage if "%~1"=="-c" (set conf=%2) & shift & shift & goto CheckOpts if "%~1"=="-p" (set platf=%2) & shift & shift & goto CheckOpts if "%~1"=="-r" (set target=Rebuild) & shift & goto CheckOpts if "%~1"=="-t" (set target=%2) & shift & shift & goto CheckOpts if "%~1"=="-d" (set conf=Debug) & shift & goto CheckOpts -if "%~1"=="-e" call "%dir%get_externals.bat" & shift & goto CheckOpts if "%~1"=="-m" (set parallel=/m) & shift & goto CheckOpts if "%~1"=="-M" (set parallel=) & shift & goto CheckOpts if "%~1"=="-v" (set verbose=/v:n) & shift & goto CheckOpts if "%~1"=="-k" (set kill=true) & shift & goto CheckOpts if "%~1"=="-V" shift & goto Version +rem These use the actual property names used by MSBuild. We could just let +rem them in through the environment, but we specify them on the command line +rem anyway for visibility so set defaults after this +if "%~1"=="-e" (set IncludeExternals=true) & shift & goto CheckOpts +if "%~1"=="--no-ssl" (set IncludeSSL=false) & shift & goto CheckOpts +if "%~1"=="--no-tkinter" (set IncludeTkinter=false) & shift & goto CheckOpts + +if "%IncludeExternals%"=="" set IncludeExternals=false +if "%IncludeSSL%"=="" set IncludeSSL=true +if "%IncludeTkinter%"=="" set IncludeTkinter=true + +if "%IncludeExternals%"=="true" call "%dir%get_externals.bat" if "%platf%"=="x64" (set vs_platf=x86_amd64) @@ -43,14 +82,18 @@ call "%dir%env.bat" %vs_platf% >nul if "%kill%"=="true" ( - msbuild /v:m /nologo /target:KillPython "%pcbuild%\pythoncore.vcxproj" /p:Configuration=%conf% /p:Platform=%platf% /p:KillPython=true + msbuild /v:m /nologo /target:KillPython "%dir%\pythoncore.vcxproj" /p:Configuration=%conf% /p:Platform=%platf% /p:KillPython=true ) rem Call on MSBuild to do the work, echo the command. rem Passing %1-9 is not the preferred option, but argument parsing in rem batch is, shall we say, "lackluster" echo on -msbuild "%dir%pcbuild.proj" /t:%target% %parallel% %verbose% /p:Configuration=%conf% /p:Platform=%platf% %1 %2 %3 %4 %5 %6 %7 %8 %9 +msbuild "%dir%pcbuild.proj" /t:%target% %parallel% %verbose%^ + /p:Configuration=%conf% /p:Platform=%platf%^ + /p:IncludeExternals=%IncludeExternals%^ + /p:IncludeSSL=%IncludeSSL% /p:IncludeTkinter=%IncludeTkinter%^ + %1 %2 %3 %4 %5 %6 %7 %8 %9 @goto :eof diff --git a/PCbuild/get_externals.bat b/PCbuild/get_externals.bat --- a/PCbuild/get_externals.bat +++ b/PCbuild/get_externals.bat @@ -51,16 +51,17 @@ echo.Fetching external libraries... -for %%e in ( - bzip2-1.0.6 - nasm-2.11.06 - openssl-1.0.2d - tcl-core-8.6.4.2 - tk-8.6.4.2 - tix-8.4.3.6 - sqlite-3.8.11.0 - xz-5.0.5 - ) do ( +set libraries= +set libraries=%libraries% bzip2-1.0.6 +if NOT "%IncludeSSL%"=="false" set libraries=%libraries% nasm-2.11.06 +if NOT "%IncludeSSL%"=="false" set libraries=%libraries% openssl-1.0.2d +set libraries=%libraries% sqlite-3.8.11.0 +if NOT "%IncludeTkinter%"=="false" set libraries=%libraries% tcl-core-8.6.4.2 +if NOT "%IncludeTkinter%"=="false" set libraries=%libraries% tk-8.6.4.2 +if NOT "%IncludeTkinter%"=="false" set libraries=%libraries% tix-8.4.3.6 +set libraries=%libraries% xz-5.0.5 + +for %%e in (%libraries%) do ( if exist %%e ( echo.%%e already exists, skipping. ) else ( diff --git a/PCbuild/pcbuild.proj b/PCbuild/pcbuild.proj --- a/PCbuild/pcbuild.proj +++ b/PCbuild/pcbuild.proj @@ -5,8 +5,10 @@ Win32 Release true + true true true + true @@ -25,7 +27,7 @@ @@ -40,10 +42,14 @@ - + + + - - + + + + diff --git a/PCbuild/rt.bat b/PCbuild/rt.bat --- a/PCbuild/rt.bat +++ b/PCbuild/rt.bat @@ -32,15 +32,17 @@ set suffix= set qmode= set dashO= +set regrtestargs= :CheckOpts if "%1"=="-O" (set dashO=-O) & shift & goto CheckOpts if "%1"=="-q" (set qmode=yes) & shift & goto CheckOpts if "%1"=="-d" (set suffix=_d) & shift & goto CheckOpts if "%1"=="-x64" (set prefix=%pcbuild%amd64\) & shift & goto CheckOpts +if NOT "%1"=="" (set regrtestargs=%regrtestargs% %1) & shift & goto CheckOpts set exe=%prefix%python%suffix%.exe -set cmd="%exe%" %dashO% -Wd -E -bb "%pcbuild%..\lib\test\regrtest.py" %1 %2 %3 %4 %5 %6 %7 %8 %9 +set cmd="%exe%" %dashO% -Wd -E -bb "%pcbuild%..\lib\test\regrtest.py" %regrtestargs% if defined qmode goto Qmode echo Deleting .pyc/.pyo files ... diff --git a/Python/codecs.c b/Python/codecs.c --- a/Python/codecs.c +++ b/Python/codecs.c @@ -966,7 +966,6 @@ } static _PyUnicode_Name_CAPI *ucnhash_CAPI = NULL; -static int ucnhash_initialized = 0; PyObject *PyCodec_NameReplaceErrors(PyObject *exc) { @@ -988,17 +987,17 @@ return NULL; if (!(object = PyUnicodeEncodeError_GetObject(exc))) return NULL; - if (!ucnhash_initialized) { + if (!ucnhash_CAPI) { /* load the unicode data module */ ucnhash_CAPI = (_PyUnicode_Name_CAPI *)PyCapsule_Import( PyUnicodeData_CAPSULE_NAME, 1); - ucnhash_initialized = 1; + if (!ucnhash_CAPI) + return NULL; } for (i = start, ressize = 0; i < end; ++i) { /* object is guaranteed to be "ready" */ c = PyUnicode_READ_CHAR(object, i); - if (ucnhash_CAPI && - ucnhash_CAPI->getname(NULL, c, buffer, sizeof(buffer), 1)) { + if (ucnhash_CAPI->getname(NULL, c, buffer, sizeof(buffer), 1)) { replsize = 1+1+1+(int)strlen(buffer)+1; } else if (c >= 0x10000) { @@ -1021,8 +1020,7 @@ i < end; ++i) { c = PyUnicode_READ_CHAR(object, i); *outp++ = '\\'; - if (ucnhash_CAPI && - ucnhash_CAPI->getname(NULL, c, buffer, sizeof(buffer), 1)) { + if (ucnhash_CAPI->getname(NULL, c, buffer, sizeof(buffer), 1)) { *outp++ = 'N'; *outp++ = '{'; strcpy((char *)outp, buffer); diff --git a/Python/pylifecycle.c b/Python/pylifecycle.c --- a/Python/pylifecycle.c +++ b/Python/pylifecycle.c @@ -1,4 +1,3 @@ - /* Python interpreter top-level routines, including init/exit */ #include "Python.h" @@ -963,6 +962,23 @@ } } +/* Check if a file descriptor is valid or not. + Return 0 if the file descriptor is invalid, return non-zero otherwise. */ +static int +is_valid_fd(int fd) +{ + int fd2; + if (fd < 0 || !_PyVerify_fd(fd)) + return 0; + _Py_BEGIN_SUPPRESS_IPH + fd2 = dup(fd); + if (fd2 >= 0) + close(fd2); + _Py_END_SUPPRESS_IPH + return fd2 >= 0; +} + +/* returns Py_None if the fd is not valid */ static PyObject* create_stdio(PyObject* io, int fd, int write_mode, char* name, @@ -978,6 +994,9 @@ _Py_IDENTIFIER(TextIOWrapper); _Py_IDENTIFIER(mode); + if (!is_valid_fd(fd)) + Py_RETURN_NONE; + /* stdin is always opened in buffered mode, first because it shouldn't make a difference in common use cases, second because TextIOWrapper depends on the presence of a read1() method which only exists on @@ -1059,23 +1078,17 @@ Py_XDECREF(stream); Py_XDECREF(text); Py_XDECREF(raw); + + if (PyErr_ExceptionMatches(PyExc_OSError) && !is_valid_fd(fd)) { + /* Issue #24891: the file descriptor was closed after the first + is_valid_fd() check was called. Ignore the OSError and set the + stream to None. */ + PyErr_Clear(); + Py_RETURN_NONE; + } return NULL; } -static int -is_valid_fd(int fd) -{ - int dummy_fd; - if (fd < 0 || !_PyVerify_fd(fd)) - return 0; - _Py_BEGIN_SUPPRESS_IPH - dummy_fd = dup(fd); - if (dummy_fd >= 0) - close(dummy_fd); - _Py_END_SUPPRESS_IPH - return dummy_fd >= 0; -} - /* Initialize sys.stdin, stdout, stderr and builtins.open */ static int initstdio(void) @@ -1158,30 +1171,18 @@ * and fileno() may point to an invalid file descriptor. For example * GUI apps don't have valid standard streams by default. */ - if (!is_valid_fd(fd)) { - std = Py_None; - Py_INCREF(std); - } - else { - std = create_stdio(iomod, fd, 0, "", encoding, errors); - if (std == NULL) - goto error; - } /* if (fd < 0) */ + std = create_stdio(iomod, fd, 0, "", encoding, errors); + if (std == NULL) + goto error; PySys_SetObject("__stdin__", std); _PySys_SetObjectId(&PyId_stdin, std); Py_DECREF(std); /* Set sys.stdout */ fd = fileno(stdout); - if (!is_valid_fd(fd)) { - std = Py_None; - Py_INCREF(std); - } - else { - std = create_stdio(iomod, fd, 1, "", encoding, errors); - if (std == NULL) - goto error; - } /* if (fd < 0) */ + std = create_stdio(iomod, fd, 1, "", encoding, errors); + if (std == NULL) + goto error; PySys_SetObject("__stdout__", std); _PySys_SetObjectId(&PyId_stdout, std); Py_DECREF(std); @@ -1189,15 +1190,9 @@ #if 1 /* Disable this if you have trouble debugging bootstrap stuff */ /* Set sys.stderr, replaces the preliminary stderr */ fd = fileno(stderr); - if (!is_valid_fd(fd)) { - std = Py_None; - Py_INCREF(std); - } - else { - std = create_stdio(iomod, fd, 1, "", encoding, "backslashreplace"); - if (std == NULL) - goto error; - } /* if (fd < 0) */ + std = create_stdio(iomod, fd, 1, "", encoding, "backslashreplace"); + if (std == NULL) + goto error; /* Same as hack above, pre-import stderr's codec to avoid recursion when import.c tries to write to stderr in verbose mode. */ diff --git a/Python/pytime.c b/Python/pytime.c --- a/Python/pytime.c +++ b/Python/pytime.c @@ -531,12 +531,8 @@ static int -pymonotonic_new(_PyTime_t *tp, _Py_clock_info_t *info, int raise) +pymonotonic(_PyTime_t *tp, _Py_clock_info_t *info, int raise) { -#ifdef Py_DEBUG - static int last_set = 0; - static _PyTime_t last = 0; -#endif #if defined(MS_WINDOWS) ULONGLONG result; @@ -628,12 +624,6 @@ if (_PyTime_FromTimespec(tp, &ts, raise) < 0) return -1; #endif -#ifdef Py_DEBUG - /* monotonic clock cannot go backward */ - assert(!last_set || last <= *tp); - last = *tp; - last_set = 1; -#endif return 0; } @@ -641,7 +631,7 @@ _PyTime_GetMonotonicClock(void) { _PyTime_t t; - if (pymonotonic_new(&t, NULL, 0) < 0) { + if (pymonotonic(&t, NULL, 0) < 0) { /* should not happen, _PyTime_Init() checked that monotonic clock at startup */ assert(0); @@ -655,7 +645,7 @@ int _PyTime_GetMonotonicClockWithInfo(_PyTime_t *tp, _Py_clock_info_t *info) { - return pymonotonic_new(tp, info, 1); + return pymonotonic(tp, info, 1); } int diff --git a/Tools/buildbot/test.bat b/Tools/buildbot/test.bat --- a/Tools/buildbot/test.bat +++ b/Tools/buildbot/test.bat @@ -1,15 +1,19 @@ - at rem Used by the buildbot "test" step. - at setlocal + at echo off +rem Used by the buildbot "test" step. +setlocal - at set here=%~dp0 - at set rt_opts=-q -d +set here=%~dp0 +set rt_opts=-q -d +set regrtest_args= :CheckOpts - at if '%1'=='-x64' (set rt_opts=%rt_opts% %1) & shift & goto CheckOpts - at if '%1'=='-d' (set rt_opts=%rt_opts% %1) & shift & goto CheckOpts - at if '%1'=='-O' (set rt_opts=%rt_opts% %1) & shift & goto CheckOpts - at if '%1'=='-q' (set rt_opts=%rt_opts% %1) & shift & goto CheckOpts - at if '%1'=='+d' (set rt_opts=%rt_opts:-d=%) & shift & goto CheckOpts - at if '%1'=='+q' (set rt_opts=%rt_opts:-q=%) & shift & goto CheckOpts +if "%1"=="-x64" (set rt_opts=%rt_opts% %1) & shift & goto CheckOpts +if "%1"=="-d" (set rt_opts=%rt_opts% %1) & shift & goto CheckOpts +if "%1"=="-O" (set rt_opts=%rt_opts% %1) & shift & goto CheckOpts +if "%1"=="-q" (set rt_opts=%rt_opts% %1) & shift & goto CheckOpts +if "%1"=="+d" (set rt_opts=%rt_opts:-d=%) & shift & goto CheckOpts +if "%1"=="+q" (set rt_opts=%rt_opts:-q=%) & shift & goto CheckOpts +if NOT "%1"=="" (set regrtest_args=%regrtest_args% %1) & shift & goto CheckOpts -call "%here%..\..\PCbuild\rt.bat" %rt_opts% -uall -rwW -n --timeout=3600 %1 %2 %3 %4 %5 %6 %7 %8 %9 +echo on +call "%here%..\..\PCbuild\rt.bat" %rt_opts% -uall -rwW -n --timeout=3600 %regrtest_args% diff --git a/Tools/msi/build.bat b/Tools/msi/build.bat --- a/Tools/msi/build.bat +++ b/Tools/msi/build.bat @@ -7,6 +7,7 @@ set BUILDX64= set BUILDDOC= set BUILDPX= +set BUILDPACK= :CheckOpts if "%~1" EQU "-h" goto Help @@ -14,6 +15,7 @@ if "%~1" EQU "-x64" (set BUILDX64=1) && shift && goto CheckOpts if "%~1" EQU "--doc" (set BUILDDOC=1) && shift && goto CheckOpts if "%~1" EQU "--test-marker" (set BUILDPX=1) && shift && goto CheckOpts +if "%~1" EQU "--pack" (set BUILDPACK=1) && shift && goto CheckOpts if not defined BUILDX86 if not defined BUILDX64 (set BUILDX86=1) && (set BUILDX64=1) @@ -41,6 +43,9 @@ if defined BUILDPX ( set BUILD_CMD=%BUILD_CMD% /p:UseTestMarker=true ) +if defined BUILDPACK ( + set BUILD_CMD=%BUILD_CMD% /p:Pack=true +) if defined BUILDX86 ( "%PCBUILD%win32\python.exe" "%D%get_wix.py" @@ -56,9 +61,10 @@ exit /B 0 :Help -echo build.bat [-x86] [-x64] [--doc] [-h] [--test-marker] +echo build.bat [-x86] [-x64] [--doc] [-h] [--test-marker] [--pack] echo. echo -x86 Build x86 installers echo -x64 Build x64 installers echo --doc Build CHM documentation echo --test-marker Build installers with 'x' markers +echo --pack Embed core MSIs into installer diff --git a/Tools/msi/bundle/snapshot.wixproj b/Tools/msi/bundle/snapshot.wixproj --- a/Tools/msi/bundle/snapshot.wixproj +++ b/Tools/msi/bundle/snapshot.wixproj @@ -9,9 +9,14 @@ + + $(DefineConstants);CompressMSI=no; + + + $(DefineConstants);CompressMSI=yes; + $(DefineConstants); - CompressMSI=no; CompressPDB=no; CompressMSI_D=no; diff --git a/Tools/msi/core/core_d.wxs b/Tools/msi/core/core_d.wxs --- a/Tools/msi/core/core_d.wxs +++ b/Tools/msi/core/core_d.wxs @@ -1,6 +1,6 @@ ? - + diff --git a/Tools/msi/core/core_pdb.wxs b/Tools/msi/core/core_pdb.wxs --- a/Tools/msi/core/core_pdb.wxs +++ b/Tools/msi/core/core_pdb.wxs @@ -1,6 +1,6 @@ ? - + diff --git a/Tools/msi/dev/dev_d.wxs b/Tools/msi/dev/dev_d.wxs --- a/Tools/msi/dev/dev_d.wxs +++ b/Tools/msi/dev/dev_d.wxs @@ -1,6 +1,6 @@ ? - + diff --git a/Tools/msi/exe/exe_d.wxs b/Tools/msi/exe/exe_d.wxs --- a/Tools/msi/exe/exe_d.wxs +++ b/Tools/msi/exe/exe_d.wxs @@ -1,6 +1,6 @@ ? - + diff --git a/Tools/msi/exe/exe_pdb.wxs b/Tools/msi/exe/exe_pdb.wxs --- a/Tools/msi/exe/exe_pdb.wxs +++ b/Tools/msi/exe/exe_pdb.wxs @@ -1,6 +1,6 @@ ? - + diff --git a/Tools/msi/lib/lib_d.wxs b/Tools/msi/lib/lib_d.wxs --- a/Tools/msi/lib/lib_d.wxs +++ b/Tools/msi/lib/lib_d.wxs @@ -1,6 +1,6 @@ ? - + diff --git a/Tools/msi/lib/lib_pdb.wxs b/Tools/msi/lib/lib_pdb.wxs --- a/Tools/msi/lib/lib_pdb.wxs +++ b/Tools/msi/lib/lib_pdb.wxs @@ -1,6 +1,6 @@ ? - + diff --git a/Tools/msi/tcltk/tcltk_d.wxs b/Tools/msi/tcltk/tcltk_d.wxs --- a/Tools/msi/tcltk/tcltk_d.wxs +++ b/Tools/msi/tcltk/tcltk_d.wxs @@ -1,6 +1,6 @@ ? - + diff --git a/Tools/msi/tcltk/tcltk_pdb.wxs b/Tools/msi/tcltk/tcltk_pdb.wxs --- a/Tools/msi/tcltk/tcltk_pdb.wxs +++ b/Tools/msi/tcltk/tcltk_pdb.wxs @@ -1,6 +1,6 @@ ? - + diff --git a/Tools/msi/test/test_d.wxs b/Tools/msi/test/test_d.wxs --- a/Tools/msi/test/test_d.wxs +++ b/Tools/msi/test/test_d.wxs @@ -1,6 +1,6 @@ ? - + diff --git a/Tools/msi/test/test_pdb.wxs b/Tools/msi/test/test_pdb.wxs --- a/Tools/msi/test/test_pdb.wxs +++ b/Tools/msi/test/test_pdb.wxs @@ -1,6 +1,6 @@ ? - + -- Repository URL: https://hg.python.org/cpython From lp_benchmark_robot at intel.com Mon Sep 7 15:54:19 2015 From: lp_benchmark_robot at intel.com (lp_benchmark_robot) Date: Mon, 7 Sep 2015 13:54:19 +0000 Subject: [Python-checkins] Benchmark Results for Python Default 2015-09-07 Message-ID: <078AA0FFE8C7034097F90205717F504611D79BE3@IRSMSX102.ger.corp.intel.com> Results for project python_default-nightly, build date 2015-09-07 13:11:42 commit: 1c76d7bc892f2d11bfb94ed86265c07125e83290 revision date: 2015-09-07 08:58:43 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v3.4.3, with hash b4cbecbc0781e89a309d03b60a1f75f8499250e6 from 2015-02-25 12:15:33+00:00 ------------------------------------------------------------------------------------------ benchmark relative change since change since current rev with std_dev* last run v3.4.3 regrtest PGO ------------------------------------------------------------------------------------------ :-) django_v2 0.19309% 3.30315% 9.80505% 16.14843% :-( pybench 0.15742% 0.20454% -2.15852% 9.12614% :-( regex_v8 6.68180% -1.75864% -5.82149% 6.85176% :-( nbody 0.17578% -7.94938% -7.66815% 13.80099% :-( json_dump_v2 0.30475% 0.09650% -2.23911% 13.69732% :-| normal_startup 0.90584% 0.36121% -0.19785% 5.45261% ------------------------------------------------------------------------------------------ Note: Benchmark results are measured in seconds. * Relative Standard Deviation (Standard Deviation/Average) Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. No license (express or implied, by estoppel or otherwise) to any intellectual property rights is granted by this document. Intel disclaims all express and implied warranties, including without limitation, the implied warranties of merchantability, fitness for a particular purpose, and non-infringement, as well as any warranty arising from course of performance, course of dealing, or usage in trade. This document may contain information on products, services and/or processes in development. Contact your Intel representative to obtain the latest forecast, schedule, specifications and roadmaps. The products and services described may contain defects or errors known as errata which may cause deviations from published specifications. Current characterized errata are available on request. (C) 2015 Intel Corporation. From python-checkins at python.org Mon Sep 7 19:00:45 2015 From: python-checkins at python.org (serhiy.storchaka) Date: Mon, 07 Sep 2015 17:00:45 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzI1MDE4?= =?utf-8?q?=3A_Fixed_testing_shutil=2Emake=5Farchive=28=29_with_relative_b?= =?utf-8?q?ase=5Fname_on?= Message-ID: <20150907170045.66850.79907@psf.io> https://hg.python.org/cpython/rev/58937cd19a85 changeset: 97750:58937cd19a85 branch: 3.4 parent: 97741:7bd6b8076c48 user: Serhiy Storchaka date: Mon Sep 07 19:58:23 2015 +0300 summary: Issue #25018: Fixed testing shutil.make_archive() with relative base_name on Windows. The test now makes sense on non-Windows. Added similar test for zip format. files: Lib/test/test_shutil.py | 28 +++++++++++++++++++--------- 1 files changed, 19 insertions(+), 9 deletions(-) diff --git a/Lib/test/test_shutil.py b/Lib/test/test_shutil.py --- a/Lib/test/test_shutil.py +++ b/Lib/test/test_shutil.py @@ -968,13 +968,13 @@ tmpdir2 = self.mkdtemp() # force shutil to create the directory os.rmdir(tmpdir2) - unittest.skipUnless(splitdrive(root_dir)[0] == splitdrive(tmpdir2)[0], - "source and target should be on same drive") + # working with relative paths + work_dir = os.path.dirname(tmpdir2) + rel_base_name = os.path.join(os.path.basename(tmpdir2), 'archive') + base_name = os.path.join(work_dir, rel_base_name) - base_name = os.path.join(tmpdir2, 'archive') - - # working with relative paths to avoid tar warnings - tarball = make_archive(splitdrive(base_name)[1], 'gztar', root_dir, '.') + with support.change_cwd(work_dir): + tarball = make_archive(rel_base_name, 'gztar', root_dir, '.') # check if the compressed tarball was created self.assertEqual(tarball, base_name + '.tar.gz') @@ -986,7 +986,8 @@ './file1', './file2', './sub/file3']) # trying an uncompressed one - tarball = make_archive(splitdrive(base_name)[1], 'tar', root_dir, '.') + with support.change_cwd(work_dir): + tarball = make_archive(rel_base_name, 'tar', root_dir, '.') self.assertEqual(tarball, base_name + '.tar') self.assertTrue(os.path.isfile(tarball)) self.assertTrue(tarfile.is_tarfile(tarball)) @@ -1053,8 +1054,17 @@ def test_make_zipfile(self): # creating something to zip root_dir, base_dir = self._create_files() - base_name = os.path.join(self.mkdtemp(), 'archive') - res = make_archive(base_name, 'zip', root_dir, 'dist') + + tmpdir2 = self.mkdtemp() + # force shutil to create the directory + os.rmdir(tmpdir2) + # working with relative paths + work_dir = os.path.dirname(tmpdir2) + rel_base_name = os.path.join(os.path.basename(tmpdir2), 'archive') + base_name = os.path.join(work_dir, rel_base_name) + + with support.change_cwd(work_dir): + res = make_archive(rel_base_name, 'zip', root_dir, 'dist') self.assertEqual(res, base_name + '.zip') self.assertTrue(os.path.isfile(res)) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 7 19:00:45 2015 From: python-checkins at python.org (serhiy.storchaka) Date: Mon, 07 Sep 2015 17:00:45 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Issue_=2325018=3A_Fixed_testing_shutil=2Emake=5Farchive=28=29_?= =?utf-8?q?with_relative_base=5Fname_on?= Message-ID: <20150907170045.17977.91798@psf.io> https://hg.python.org/cpython/rev/4312b6549590 changeset: 97752:4312b6549590 branch: 3.5 parent: 97748:5939b50429a0 parent: 97750:58937cd19a85 user: Serhiy Storchaka date: Mon Sep 07 19:59:24 2015 +0300 summary: Issue #25018: Fixed testing shutil.make_archive() with relative base_name on Windows. The test now makes sense on non-Windows. Added similar test for zip format. files: Lib/test/test_shutil.py | 28 +++++++++++++++++++--------- 1 files changed, 19 insertions(+), 9 deletions(-) diff --git a/Lib/test/test_shutil.py b/Lib/test/test_shutil.py --- a/Lib/test/test_shutil.py +++ b/Lib/test/test_shutil.py @@ -974,13 +974,13 @@ tmpdir2 = self.mkdtemp() # force shutil to create the directory os.rmdir(tmpdir2) - unittest.skipUnless(splitdrive(root_dir)[0] == splitdrive(tmpdir2)[0], - "source and target should be on same drive") + # working with relative paths + work_dir = os.path.dirname(tmpdir2) + rel_base_name = os.path.join(os.path.basename(tmpdir2), 'archive') + base_name = os.path.join(work_dir, rel_base_name) - base_name = os.path.join(tmpdir2, 'archive') - - # working with relative paths to avoid tar warnings - tarball = make_archive(splitdrive(base_name)[1], 'gztar', root_dir, '.') + with support.change_cwd(work_dir): + tarball = make_archive(rel_base_name, 'gztar', root_dir, '.') # check if the compressed tarball was created self.assertEqual(tarball, base_name + '.tar.gz') @@ -992,7 +992,8 @@ './file1', './file2', './sub/file3']) # trying an uncompressed one - tarball = make_archive(splitdrive(base_name)[1], 'tar', root_dir, '.') + with support.change_cwd(work_dir): + tarball = make_archive(rel_base_name, 'tar', root_dir, '.') self.assertEqual(tarball, base_name + '.tar') self.assertTrue(os.path.isfile(tarball)) self.assertTrue(tarfile.is_tarfile(tarball)) @@ -1059,8 +1060,17 @@ def test_make_zipfile(self): # creating something to zip root_dir, base_dir = self._create_files() - base_name = os.path.join(self.mkdtemp(), 'archive') - res = make_archive(base_name, 'zip', root_dir, 'dist') + + tmpdir2 = self.mkdtemp() + # force shutil to create the directory + os.rmdir(tmpdir2) + # working with relative paths + work_dir = os.path.dirname(tmpdir2) + rel_base_name = os.path.join(os.path.basename(tmpdir2), 'archive') + base_name = os.path.join(work_dir, rel_base_name) + + with support.change_cwd(work_dir): + res = make_archive(rel_base_name, 'zip', root_dir, 'dist') self.assertEqual(res, base_name + '.zip') self.assertTrue(os.path.isfile(res)) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 7 19:00:46 2015 From: python-checkins at python.org (serhiy.storchaka) Date: Mon, 07 Sep 2015 17:00:46 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2325018=3A_Fixed_testing_shutil=2Emake=5Farchive?= =?utf-8?q?=28=29_with_relative_base=5Fname_on?= Message-ID: <20150907170045.115118.6516@psf.io> https://hg.python.org/cpython/rev/b076b62731b7 changeset: 97753:b076b62731b7 parent: 97749:cfc8f1e57212 parent: 97752:4312b6549590 user: Serhiy Storchaka date: Mon Sep 07 19:59:38 2015 +0300 summary: Issue #25018: Fixed testing shutil.make_archive() with relative base_name on Windows. The test now makes sense on non-Windows. Added similar test for zip format. files: Lib/test/test_shutil.py | 28 +++++++++++++++++++--------- 1 files changed, 19 insertions(+), 9 deletions(-) diff --git a/Lib/test/test_shutil.py b/Lib/test/test_shutil.py --- a/Lib/test/test_shutil.py +++ b/Lib/test/test_shutil.py @@ -974,13 +974,13 @@ tmpdir2 = self.mkdtemp() # force shutil to create the directory os.rmdir(tmpdir2) - unittest.skipUnless(splitdrive(root_dir)[0] == splitdrive(tmpdir2)[0], - "source and target should be on same drive") + # working with relative paths + work_dir = os.path.dirname(tmpdir2) + rel_base_name = os.path.join(os.path.basename(tmpdir2), 'archive') + base_name = os.path.join(work_dir, rel_base_name) - base_name = os.path.join(tmpdir2, 'archive') - - # working with relative paths to avoid tar warnings - tarball = make_archive(splitdrive(base_name)[1], 'gztar', root_dir, '.') + with support.change_cwd(work_dir): + tarball = make_archive(rel_base_name, 'gztar', root_dir, '.') # check if the compressed tarball was created self.assertEqual(tarball, base_name + '.tar.gz') @@ -992,7 +992,8 @@ './file1', './file2', './sub/file3']) # trying an uncompressed one - tarball = make_archive(splitdrive(base_name)[1], 'tar', root_dir, '.') + with support.change_cwd(work_dir): + tarball = make_archive(rel_base_name, 'tar', root_dir, '.') self.assertEqual(tarball, base_name + '.tar') self.assertTrue(os.path.isfile(tarball)) self.assertTrue(tarfile.is_tarfile(tarball)) @@ -1059,8 +1060,17 @@ def test_make_zipfile(self): # creating something to zip root_dir, base_dir = self._create_files() - base_name = os.path.join(self.mkdtemp(), 'archive') - res = make_archive(base_name, 'zip', root_dir, 'dist') + + tmpdir2 = self.mkdtemp() + # force shutil to create the directory + os.rmdir(tmpdir2) + # working with relative paths + work_dir = os.path.dirname(tmpdir2) + rel_base_name = os.path.join(os.path.basename(tmpdir2), 'archive') + base_name = os.path.join(work_dir, rel_base_name) + + with support.change_cwd(work_dir): + res = make_archive(rel_base_name, 'zip', root_dir, 'dist') self.assertEqual(res, base_name + '.zip') self.assertTrue(os.path.isfile(res)) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 7 19:00:45 2015 From: python-checkins at python.org (serhiy.storchaka) Date: Mon, 07 Sep 2015 17:00:45 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzI1MDE4?= =?utf-8?q?=3A_Fixed_testing_shutil=2Emake=5Farchive=28=29_with_relative_b?= =?utf-8?q?ase=5Fname_on?= Message-ID: <20150907170045.11264.88252@psf.io> https://hg.python.org/cpython/rev/d9c4b35e3fdc changeset: 97751:d9c4b35e3fdc branch: 2.7 parent: 97742:beda04bf5991 user: Serhiy Storchaka date: Mon Sep 07 19:58:23 2015 +0300 summary: Issue #25018: Fixed testing shutil.make_archive() with relative base_name on Windows. The test now makes sense on non-Windows. Added similar test for zip format. files: Lib/test/test_shutil.py | 28 +++++++++++++++++++--------- 1 files changed, 19 insertions(+), 9 deletions(-) diff --git a/Lib/test/test_shutil.py b/Lib/test/test_shutil.py --- a/Lib/test/test_shutil.py +++ b/Lib/test/test_shutil.py @@ -379,13 +379,13 @@ tmpdir2 = self.mkdtemp() # force shutil to create the directory os.rmdir(tmpdir2) - unittest.skipUnless(splitdrive(root_dir)[0] == splitdrive(tmpdir2)[0], - "source and target should be on same drive") + # working with relative paths + work_dir = os.path.dirname(tmpdir2) + rel_base_name = os.path.join(os.path.basename(tmpdir2), 'archive') + base_name = os.path.join(work_dir, rel_base_name) - base_name = os.path.join(tmpdir2, 'archive') - - # working with relative paths to avoid tar warnings - tarball = make_archive(splitdrive(base_name)[1], 'gztar', root_dir, '.') + with support.change_cwd(work_dir): + tarball = make_archive(rel_base_name, 'gztar', root_dir, '.') # check if the compressed tarball was created self.assertEqual(tarball, base_name + '.tar.gz') @@ -397,7 +397,8 @@ './sub', './sub/file3', './sub2']) # trying an uncompressed one - tarball = make_archive(splitdrive(base_name)[1], 'tar', root_dir, '.') + with support.change_cwd(work_dir): + tarball = make_archive(rel_base_name, 'tar', root_dir, '.') self.assertEqual(tarball, base_name + '.tar') self.assertTrue(os.path.isfile(tarball)) self.assertTrue(tarfile.is_tarfile(tarball)) @@ -465,8 +466,17 @@ def test_make_zipfile(self): # creating something to zip root_dir, base_dir = self._create_files() - base_name = os.path.join(self.mkdtemp(), 'archive') - res = make_archive(base_name, 'zip', root_dir, 'dist') + + tmpdir2 = self.mkdtemp() + # force shutil to create the directory + os.rmdir(tmpdir2) + # working with relative paths + work_dir = os.path.dirname(tmpdir2) + rel_base_name = os.path.join(os.path.basename(tmpdir2), 'archive') + base_name = os.path.join(work_dir, rel_base_name) + + with support.change_cwd(work_dir): + res = make_archive(rel_base_name, 'zip', root_dir, 'dist') self.assertEqual(res, base_name + '.zip') self.assertTrue(os.path.isfile(res)) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 7 19:00:46 2015 From: python-checkins at python.org (serhiy.storchaka) Date: Mon, 07 Sep 2015 17:00:46 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogTWVyZ2UgMy41Lg==?= Message-ID: <20150907170045.15734.21855@psf.io> https://hg.python.org/cpython/rev/cfc8f1e57212 changeset: 97749:cfc8f1e57212 parent: 97744:738e227d0891 parent: 97748:5939b50429a0 user: Serhiy Storchaka date: Mon Sep 07 19:52:12 2015 +0300 summary: Merge 3.5. files: .hgtags | 1 + Misc/NEWS | 2 +- 2 files changed, 2 insertions(+), 1 deletions(-) diff --git a/.hgtags b/.hgtags --- a/.hgtags +++ b/.hgtags @@ -154,3 +154,4 @@ c0d64105463581f85d0e368e8d6e59b7fd8f12b1 v3.5.0b4 1a58b1227501e046eee13d90f113417b60843301 v3.5.0rc1 cc15d736d860303b9da90d43cd32db39bab048df v3.5.0rc2 +66ed52375df802f9d0a34480daaa8ce79fc41313 v3.5.0rc3 diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -169,7 +169,7 @@ What's New in Python 3.5.0 release candidate 3? =============================================== -Release date: 2015-09-06 +Release date: 2015-09-07 Core and Builtins ----------------- -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 7 21:56:25 2015 From: python-checkins at python.org (serhiy.storchaka) Date: Mon, 07 Sep 2015 19:56:25 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzI1MDE5?= =?utf-8?q?=3A_Fixed_a_crash_caused_by_setting_non-string_key_of_expat_par?= =?utf-8?q?ser=2E?= Message-ID: <20150907195624.17963.63401@psf.io> https://hg.python.org/cpython/rev/ff2c4f281720 changeset: 97754:ff2c4f281720 branch: 3.4 parent: 97750:58937cd19a85 user: Serhiy Storchaka date: Mon Sep 07 22:37:02 2015 +0300 summary: Issue #25019: Fixed a crash caused by setting non-string key of expat parser. Added additional tests for expat parser attributes. Based on patch by John Leitch. files: Lib/test/test_pyexpat.py | 56 ++++++++++++++++++++------- Misc/ACKS | 1 + Misc/NEWS | 3 + Modules/pyexpat.c | 7 +++- 4 files changed, 51 insertions(+), 16 deletions(-) diff --git a/Lib/test/test_pyexpat.py b/Lib/test/test_pyexpat.py --- a/Lib/test/test_pyexpat.py +++ b/Lib/test/test_pyexpat.py @@ -16,22 +16,47 @@ class SetAttributeTest(unittest.TestCase): def setUp(self): self.parser = expat.ParserCreate(namespace_separator='!') - self.set_get_pairs = [ - [0, 0], - [1, 1], - [2, 1], - [0, 0], - ] + + def test_buffer_text(self): + self.assertIs(self.parser.buffer_text, False) + for x in 0, 1, 2, 0: + self.parser.buffer_text = x + self.assertIs(self.parser.buffer_text, bool(x)) + + def test_namespace_prefixes(self): + self.assertIs(self.parser.namespace_prefixes, False) + for x in 0, 1, 2, 0: + self.parser.namespace_prefixes = x + self.assertIs(self.parser.namespace_prefixes, bool(x)) def test_ordered_attributes(self): - for x, y in self.set_get_pairs: + self.assertIs(self.parser.ordered_attributes, False) + for x in 0, 1, 2, 0: self.parser.ordered_attributes = x - self.assertEqual(self.parser.ordered_attributes, y) + self.assertIs(self.parser.ordered_attributes, bool(x)) def test_specified_attributes(self): - for x, y in self.set_get_pairs: + self.assertIs(self.parser.specified_attributes, False) + for x in 0, 1, 2, 0: self.parser.specified_attributes = x - self.assertEqual(self.parser.specified_attributes, y) + self.assertIs(self.parser.specified_attributes, bool(x)) + + def test_specified_attributes(self): + self.assertIs(self.parser.specified_attributes, False) + for x in 0, 1, 2, 0: + self.parser.specified_attributes = x + self.assertIs(self.parser.specified_attributes, bool(x)) + + def test_invalid_attributes(self): + with self.assertRaises(AttributeError): + self.parser.returns_unicode = 1 + with self.assertRaises(AttributeError): + self.parser.returns_unicode + + # Issue #25019 + self.assertRaises(TypeError, setattr, self.parser, range(0xF), 0) + self.assertRaises(TypeError, self.parser.__setattr__, range(0xF), 0) + self.assertRaises(TypeError, getattr, self.parser, range(0xF)) data = b'''\ @@ -514,11 +539,12 @@ def test_wrong_size(self): parser = expat.ParserCreate() parser.buffer_text = 1 - def f(size): - parser.buffer_size = size - - self.assertRaises(ValueError, f, -1) - self.assertRaises(ValueError, f, 0) + with self.assertRaises(ValueError): + parser.buffer_size = -1 + with self.assertRaises(ValueError): + parser.buffer_size = 0 + with self.assertRaises(TypeError): + parser.buffer_size = 512.0 def test_unchanged_size(self): xml1 = b"" + b'a' * 512 diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -807,6 +807,7 @@ Robert Lehmann Petri Lehtinen Luke Kenneth Casson Leighton +John Leitch Tshepang Lekhonkhobe Marc-Andr? Lemburg Mateusz Lenik diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -81,6 +81,9 @@ Library ------- +- Issue #25019: Fixed a crash caused by setting non-string key of expat parser. + Based on patch by John Leitch. + - Issue #24917: time_strftime() buffer over-read. - Issue #23144: Make sure that HTMLParser.feed() returns all the data, even diff --git a/Modules/pyexpat.c b/Modules/pyexpat.c --- a/Modules/pyexpat.c +++ b/Modules/pyexpat.c @@ -1341,11 +1341,16 @@ xmlparse_setattro(xmlparseobject *self, PyObject *name, PyObject *v) { /* Set attribute 'name' to value 'v'. v==NULL means delete */ + if (!PyUnicode_Check(name)) { + PyErr_Format(PyExc_TypeError, + "attribute name must be string, not '%.200s'", + name->ob_type->tp_name); + return -1; + } if (v == NULL) { PyErr_SetString(PyExc_RuntimeError, "Cannot delete attribute"); return -1; } - assert(PyUnicode_Check(name)); if (PyUnicode_CompareWithASCIIString(name, "buffer_text") == 0) { int b = PyObject_IsTrue(v); if (b < 0) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 7 21:56:29 2015 From: python-checkins at python.org (serhiy.storchaka) Date: Mon, 07 Sep 2015 19:56:29 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Issue_=2325019=3A_Fixed_a_crash_caused_by_setting_non-string_k?= =?utf-8?q?ey_of_expat_parser=2E?= Message-ID: <20150907195629.15706.58306@psf.io> https://hg.python.org/cpython/rev/6006231dcaae changeset: 97755:6006231dcaae branch: 3.5 parent: 97752:4312b6549590 parent: 97754:ff2c4f281720 user: Serhiy Storchaka date: Mon Sep 07 22:38:34 2015 +0300 summary: Issue #25019: Fixed a crash caused by setting non-string key of expat parser. Added additional tests for expat parser attributes. Based on patch by John Leitch. files: Lib/test/test_pyexpat.py | 56 ++++++++++++++++++++------- Misc/ACKS | 1 + Misc/NEWS | 3 + Modules/pyexpat.c | 7 +++- 4 files changed, 51 insertions(+), 16 deletions(-) diff --git a/Lib/test/test_pyexpat.py b/Lib/test/test_pyexpat.py --- a/Lib/test/test_pyexpat.py +++ b/Lib/test/test_pyexpat.py @@ -16,22 +16,47 @@ class SetAttributeTest(unittest.TestCase): def setUp(self): self.parser = expat.ParserCreate(namespace_separator='!') - self.set_get_pairs = [ - [0, 0], - [1, 1], - [2, 1], - [0, 0], - ] + + def test_buffer_text(self): + self.assertIs(self.parser.buffer_text, False) + for x in 0, 1, 2, 0: + self.parser.buffer_text = x + self.assertIs(self.parser.buffer_text, bool(x)) + + def test_namespace_prefixes(self): + self.assertIs(self.parser.namespace_prefixes, False) + for x in 0, 1, 2, 0: + self.parser.namespace_prefixes = x + self.assertIs(self.parser.namespace_prefixes, bool(x)) def test_ordered_attributes(self): - for x, y in self.set_get_pairs: + self.assertIs(self.parser.ordered_attributes, False) + for x in 0, 1, 2, 0: self.parser.ordered_attributes = x - self.assertEqual(self.parser.ordered_attributes, y) + self.assertIs(self.parser.ordered_attributes, bool(x)) def test_specified_attributes(self): - for x, y in self.set_get_pairs: + self.assertIs(self.parser.specified_attributes, False) + for x in 0, 1, 2, 0: self.parser.specified_attributes = x - self.assertEqual(self.parser.specified_attributes, y) + self.assertIs(self.parser.specified_attributes, bool(x)) + + def test_specified_attributes(self): + self.assertIs(self.parser.specified_attributes, False) + for x in 0, 1, 2, 0: + self.parser.specified_attributes = x + self.assertIs(self.parser.specified_attributes, bool(x)) + + def test_invalid_attributes(self): + with self.assertRaises(AttributeError): + self.parser.returns_unicode = 1 + with self.assertRaises(AttributeError): + self.parser.returns_unicode + + # Issue #25019 + self.assertRaises(TypeError, setattr, self.parser, range(0xF), 0) + self.assertRaises(TypeError, self.parser.__setattr__, range(0xF), 0) + self.assertRaises(TypeError, getattr, self.parser, range(0xF)) data = b'''\ @@ -514,11 +539,12 @@ def test_wrong_size(self): parser = expat.ParserCreate() parser.buffer_text = 1 - def f(size): - parser.buffer_size = size - - self.assertRaises(ValueError, f, -1) - self.assertRaises(ValueError, f, 0) + with self.assertRaises(ValueError): + parser.buffer_size = -1 + with self.assertRaises(ValueError): + parser.buffer_size = 0 + with self.assertRaises(TypeError): + parser.buffer_size = 512.0 def test_unchanged_size(self): xml1 = b"" + b'a' * 512 diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -831,6 +831,7 @@ Robert Lehmann Petri Lehtinen Luke Kenneth Casson Leighton +John Leitch Tshepang Lekhonkhobe Marc-Andr? Lemburg Mateusz Lenik diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -14,6 +14,9 @@ Library ------- +- Issue #25019: Fixed a crash caused by setting non-string key of expat parser. + Based on patch by John Leitch. + - Issue #16180: Exit pdb if file has syntax error, instead of trapping user in an infinite loop. Patch by Xavier de Gaye. diff --git a/Modules/pyexpat.c b/Modules/pyexpat.c --- a/Modules/pyexpat.c +++ b/Modules/pyexpat.c @@ -1378,11 +1378,16 @@ xmlparse_setattro(xmlparseobject *self, PyObject *name, PyObject *v) { /* Set attribute 'name' to value 'v'. v==NULL means delete */ + if (!PyUnicode_Check(name)) { + PyErr_Format(PyExc_TypeError, + "attribute name must be string, not '%.200s'", + name->ob_type->tp_name); + return -1; + } if (v == NULL) { PyErr_SetString(PyExc_RuntimeError, "Cannot delete attribute"); return -1; } - assert(PyUnicode_Check(name)); if (PyUnicode_CompareWithASCIIString(name, "buffer_text") == 0) { int b = PyObject_IsTrue(v); if (b < 0) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 7 21:56:30 2015 From: python-checkins at python.org (serhiy.storchaka) Date: Mon, 07 Sep 2015 19:56:30 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_Raise_more_cor?= =?utf-8?q?rect_exception_on_overflow_in_setting_buffer=5Fsize_attribute_o?= =?utf-8?q?f?= Message-ID: <20150907195629.17969.33343@psf.io> https://hg.python.org/cpython/rev/ef9346be45d0 changeset: 97758:ef9346be45d0 branch: 3.4 parent: 97754:ff2c4f281720 user: Serhiy Storchaka date: Mon Sep 07 22:51:56 2015 +0300 summary: Raise more correct exception on overflow in setting buffer_size attribute of expat parser. files: Lib/test/test_pyexpat.py | 3 +++ Modules/pyexpat.c | 13 +++++++------ 2 files changed, 10 insertions(+), 6 deletions(-) diff --git a/Lib/test/test_pyexpat.py b/Lib/test/test_pyexpat.py --- a/Lib/test/test_pyexpat.py +++ b/Lib/test/test_pyexpat.py @@ -3,6 +3,7 @@ from io import BytesIO import os +import sys import sysconfig import unittest import traceback @@ -543,6 +544,8 @@ parser.buffer_size = -1 with self.assertRaises(ValueError): parser.buffer_size = 0 + with self.assertRaises((ValueError, OverflowError)): + parser.buffer_size = sys.maxsize + 1 with self.assertRaises(TypeError): parser.buffer_size = 512.0 diff --git a/Modules/pyexpat.c b/Modules/pyexpat.c --- a/Modules/pyexpat.c +++ b/Modules/pyexpat.c @@ -1403,17 +1403,18 @@ return -1; } - new_buffer_size=PyLong_AS_LONG(v); + new_buffer_size = PyLong_AsLong(v); + if (new_buffer_size <= 0) { + if (!PyErr_Occurred()) + PyErr_SetString(PyExc_ValueError, "buffer_size must be greater than zero"); + return -1; + } + /* trivial case -- no change */ if (new_buffer_size == self->buffer_size) { return 0; } - if (new_buffer_size <= 0) { - PyErr_SetString(PyExc_ValueError, "buffer_size must be greater than zero"); - return -1; - } - /* check maximum */ if (new_buffer_size > INT_MAX) { char errmsg[100]; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 7 21:56:30 2015 From: python-checkins at python.org (serhiy.storchaka) Date: Mon, 07 Sep 2015 19:56:30 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_Backported_new?= =?utf-8?q?_tests_for_attribute_setting_of_expat_parser=2E?= Message-ID: <20150907195629.101478.39975@psf.io> https://hg.python.org/cpython/rev/b609e391c04d changeset: 97757:b609e391c04d branch: 2.7 parent: 97751:d9c4b35e3fdc user: Serhiy Storchaka date: Mon Sep 07 22:42:12 2015 +0300 summary: Backported new tests for attribute setting of expat parser. files: Lib/test/test_pyexpat.py | 53 ++++++++++++++++++--------- 1 files changed, 35 insertions(+), 18 deletions(-) diff --git a/Lib/test/test_pyexpat.py b/Lib/test/test_pyexpat.py --- a/Lib/test/test_pyexpat.py +++ b/Lib/test/test_pyexpat.py @@ -13,27 +13,42 @@ class SetAttributeTest(unittest.TestCase): def setUp(self): self.parser = expat.ParserCreate(namespace_separator='!') - self.set_get_pairs = [ - [0, 0], - [1, 1], - [2, 1], - [0, 0], - ] + + def test_buffer_text(self): + self.assertIs(self.parser.buffer_text, False) + for x in 0, 1, 2, 0: + self.parser.buffer_text = x + self.assertIs(self.parser.buffer_text, bool(x)) + + def test_namespace_prefixes(self): + self.assertIs(self.parser.namespace_prefixes, False) + for x in 0, 1, 2, 0: + self.parser.namespace_prefixes = x + self.assertIs(self.parser.namespace_prefixes, bool(x)) def test_returns_unicode(self): - for x, y in self.set_get_pairs: + self.assertIs(self.parser.returns_unicode, test_support.have_unicode) + for x in 0, 1, 2, 0: self.parser.returns_unicode = x - self.assertEqual(self.parser.returns_unicode, y) + self.assertIs(self.parser.returns_unicode, bool(x)) def test_ordered_attributes(self): - for x, y in self.set_get_pairs: + self.assertIs(self.parser.ordered_attributes, False) + for x in 0, 1, 2, 0: self.parser.ordered_attributes = x - self.assertEqual(self.parser.ordered_attributes, y) + self.assertIs(self.parser.ordered_attributes, bool(x)) def test_specified_attributes(self): - for x, y in self.set_get_pairs: + self.assertIs(self.parser.specified_attributes, False) + for x in 0, 1, 2, 0: self.parser.specified_attributes = x - self.assertEqual(self.parser.specified_attributes, y) + self.assertIs(self.parser.specified_attributes, bool(x)) + + def test_invalid_attributes(self): + with self.assertRaises(AttributeError): + self.parser.foo = 1 + with self.assertRaises(AttributeError): + self.parser.foo data = '''\ @@ -469,12 +484,14 @@ def test_wrong_size(self): parser = expat.ParserCreate() parser.buffer_text = 1 - def f(size): - parser.buffer_size = size - - self.assertRaises(TypeError, f, sys.maxint+1) - self.assertRaises(ValueError, f, -1) - self.assertRaises(ValueError, f, 0) + with self.assertRaises(ValueError): + parser.buffer_size = -1 + with self.assertRaises(ValueError): + parser.buffer_size = 0 + with self.assertRaises(TypeError): + parser.buffer_size = 512.0 + with self.assertRaises(TypeError): + parser.buffer_size = sys.maxint+1 def test_unchanged_size(self): xml1 = ("%s" % ('a' * 512)) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 7 21:56:30 2015 From: python-checkins at python.org (serhiy.storchaka) Date: Mon, 07 Sep 2015 19:56:30 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2325019=3A_Fixed_a_crash_caused_by_setting_non-st?= =?utf-8?q?ring_key_of_expat_parser=2E?= Message-ID: <20150907195629.68859.9978@psf.io> https://hg.python.org/cpython/rev/edf25acae637 changeset: 97756:edf25acae637 parent: 97753:b076b62731b7 parent: 97755:6006231dcaae user: Serhiy Storchaka date: Mon Sep 07 22:41:04 2015 +0300 summary: Issue #25019: Fixed a crash caused by setting non-string key of expat parser. Added additional tests for expat parser attributes. Based on patch by John Leitch. files: Lib/test/test_pyexpat.py | 56 ++++++++++++++++++++------- Misc/ACKS | 1 + Misc/NEWS | 3 + Modules/pyexpat.c | 7 +++- 4 files changed, 51 insertions(+), 16 deletions(-) diff --git a/Lib/test/test_pyexpat.py b/Lib/test/test_pyexpat.py --- a/Lib/test/test_pyexpat.py +++ b/Lib/test/test_pyexpat.py @@ -16,22 +16,47 @@ class SetAttributeTest(unittest.TestCase): def setUp(self): self.parser = expat.ParserCreate(namespace_separator='!') - self.set_get_pairs = [ - [0, 0], - [1, 1], - [2, 1], - [0, 0], - ] + + def test_buffer_text(self): + self.assertIs(self.parser.buffer_text, False) + for x in 0, 1, 2, 0: + self.parser.buffer_text = x + self.assertIs(self.parser.buffer_text, bool(x)) + + def test_namespace_prefixes(self): + self.assertIs(self.parser.namespace_prefixes, False) + for x in 0, 1, 2, 0: + self.parser.namespace_prefixes = x + self.assertIs(self.parser.namespace_prefixes, bool(x)) def test_ordered_attributes(self): - for x, y in self.set_get_pairs: + self.assertIs(self.parser.ordered_attributes, False) + for x in 0, 1, 2, 0: self.parser.ordered_attributes = x - self.assertEqual(self.parser.ordered_attributes, y) + self.assertIs(self.parser.ordered_attributes, bool(x)) def test_specified_attributes(self): - for x, y in self.set_get_pairs: + self.assertIs(self.parser.specified_attributes, False) + for x in 0, 1, 2, 0: self.parser.specified_attributes = x - self.assertEqual(self.parser.specified_attributes, y) + self.assertIs(self.parser.specified_attributes, bool(x)) + + def test_specified_attributes(self): + self.assertIs(self.parser.specified_attributes, False) + for x in 0, 1, 2, 0: + self.parser.specified_attributes = x + self.assertIs(self.parser.specified_attributes, bool(x)) + + def test_invalid_attributes(self): + with self.assertRaises(AttributeError): + self.parser.returns_unicode = 1 + with self.assertRaises(AttributeError): + self.parser.returns_unicode + + # Issue #25019 + self.assertRaises(TypeError, setattr, self.parser, range(0xF), 0) + self.assertRaises(TypeError, self.parser.__setattr__, range(0xF), 0) + self.assertRaises(TypeError, getattr, self.parser, range(0xF)) data = b'''\ @@ -514,11 +539,12 @@ def test_wrong_size(self): parser = expat.ParserCreate() parser.buffer_text = 1 - def f(size): - parser.buffer_size = size - - self.assertRaises(ValueError, f, -1) - self.assertRaises(ValueError, f, 0) + with self.assertRaises(ValueError): + parser.buffer_size = -1 + with self.assertRaises(ValueError): + parser.buffer_size = 0 + with self.assertRaises(TypeError): + parser.buffer_size = 512.0 def test_unchanged_size(self): xml1 = b"" + b'a' * 512 diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -831,6 +831,7 @@ Robert Lehmann Petri Lehtinen Luke Kenneth Casson Leighton +John Leitch Tshepang Lekhonkhobe Marc-Andr? Lemburg Mateusz Lenik diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -103,6 +103,9 @@ Library ------- +- Issue #25019: Fixed a crash caused by setting non-string key of expat parser. + Based on patch by John Leitch. + - Issue #16180: Exit pdb if file has syntax error, instead of trapping user in an infinite loop. Patch by Xavier de Gaye. diff --git a/Modules/pyexpat.c b/Modules/pyexpat.c --- a/Modules/pyexpat.c +++ b/Modules/pyexpat.c @@ -1378,11 +1378,16 @@ xmlparse_setattro(xmlparseobject *self, PyObject *name, PyObject *v) { /* Set attribute 'name' to value 'v'. v==NULL means delete */ + if (!PyUnicode_Check(name)) { + PyErr_Format(PyExc_TypeError, + "attribute name must be string, not '%.200s'", + name->ob_type->tp_name); + return -1; + } if (v == NULL) { PyErr_SetString(PyExc_RuntimeError, "Cannot delete attribute"); return -1; } - assert(PyUnicode_Check(name)); if (PyUnicode_CompareWithASCIIString(name, "buffer_text") == 0) { int b = PyObject_IsTrue(v); if (b < 0) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 7 21:56:30 2015 From: python-checkins at python.org (serhiy.storchaka) Date: Mon, 07 Sep 2015 19:56:30 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Raise_more_correct_exception_on_overflow_in_setting_buffer=5Fs?= =?utf-8?q?ize_attribute_of?= Message-ID: <20150907195630.115151.42222@psf.io> https://hg.python.org/cpython/rev/cacdd5f71cf4 changeset: 97759:cacdd5f71cf4 branch: 3.5 parent: 97755:6006231dcaae parent: 97758:ef9346be45d0 user: Serhiy Storchaka date: Mon Sep 07 22:54:08 2015 +0300 summary: Raise more correct exception on overflow in setting buffer_size attribute of expat parser. files: Lib/test/test_pyexpat.py | 3 +++ Modules/pyexpat.c | 13 +++++++------ 2 files changed, 10 insertions(+), 6 deletions(-) diff --git a/Lib/test/test_pyexpat.py b/Lib/test/test_pyexpat.py --- a/Lib/test/test_pyexpat.py +++ b/Lib/test/test_pyexpat.py @@ -3,6 +3,7 @@ from io import BytesIO import os +import sys import sysconfig import unittest import traceback @@ -543,6 +544,8 @@ parser.buffer_size = -1 with self.assertRaises(ValueError): parser.buffer_size = 0 + with self.assertRaises((ValueError, OverflowError)): + parser.buffer_size = sys.maxsize + 1 with self.assertRaises(TypeError): parser.buffer_size = 512.0 diff --git a/Modules/pyexpat.c b/Modules/pyexpat.c --- a/Modules/pyexpat.c +++ b/Modules/pyexpat.c @@ -1440,17 +1440,18 @@ return -1; } - new_buffer_size=PyLong_AS_LONG(v); + new_buffer_size = PyLong_AsLong(v); + if (new_buffer_size <= 0) { + if (!PyErr_Occurred()) + PyErr_SetString(PyExc_ValueError, "buffer_size must be greater than zero"); + return -1; + } + /* trivial case -- no change */ if (new_buffer_size == self->buffer_size) { return 0; } - if (new_buffer_size <= 0) { - PyErr_SetString(PyExc_ValueError, "buffer_size must be greater than zero"); - return -1; - } - /* check maximum */ if (new_buffer_size > INT_MAX) { char errmsg[100]; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 7 21:56:30 2015 From: python-checkins at python.org (serhiy.storchaka) Date: Mon, 07 Sep 2015 19:56:30 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Raise_more_correct_exception_on_overflow_in_setting_buff?= =?utf-8?q?er=5Fsize_attribute_of?= Message-ID: <20150907195630.66856.88173@psf.io> https://hg.python.org/cpython/rev/9d65e83c8b9e changeset: 97760:9d65e83c8b9e parent: 97756:edf25acae637 parent: 97759:cacdd5f71cf4 user: Serhiy Storchaka date: Mon Sep 07 22:54:33 2015 +0300 summary: Raise more correct exception on overflow in setting buffer_size attribute of expat parser. files: Lib/test/test_pyexpat.py | 3 +++ Modules/pyexpat.c | 13 +++++++------ 2 files changed, 10 insertions(+), 6 deletions(-) diff --git a/Lib/test/test_pyexpat.py b/Lib/test/test_pyexpat.py --- a/Lib/test/test_pyexpat.py +++ b/Lib/test/test_pyexpat.py @@ -3,6 +3,7 @@ from io import BytesIO import os +import sys import sysconfig import unittest import traceback @@ -543,6 +544,8 @@ parser.buffer_size = -1 with self.assertRaises(ValueError): parser.buffer_size = 0 + with self.assertRaises((ValueError, OverflowError)): + parser.buffer_size = sys.maxsize + 1 with self.assertRaises(TypeError): parser.buffer_size = 512.0 diff --git a/Modules/pyexpat.c b/Modules/pyexpat.c --- a/Modules/pyexpat.c +++ b/Modules/pyexpat.c @@ -1440,17 +1440,18 @@ return -1; } - new_buffer_size=PyLong_AS_LONG(v); + new_buffer_size = PyLong_AsLong(v); + if (new_buffer_size <= 0) { + if (!PyErr_Occurred()) + PyErr_SetString(PyExc_ValueError, "buffer_size must be greater than zero"); + return -1; + } + /* trivial case -- no change */ if (new_buffer_size == self->buffer_size) { return 0; } - if (new_buffer_size <= 0) { - PyErr_SetString(PyExc_ValueError, "buffer_size must be greater than zero"); - return -1; - } - /* check maximum */ if (new_buffer_size > INT_MAX) { char errmsg[100]; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 8 00:15:40 2015 From: python-checkins at python.org (victor.stinner) Date: Mon, 07 Sep 2015 22:15:40 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2322241=3A_Fix_a_co?= =?utf-8?q?mpiler_waring?= Message-ID: <20150907221540.15708.49819@psf.io> https://hg.python.org/cpython/rev/ea1bfb64e898 changeset: 97761:ea1bfb64e898 user: Victor Stinner date: Tue Sep 08 00:12:49 2015 +0200 summary: Issue #22241: Fix a compiler waring files: Modules/_datetimemodule.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Modules/_datetimemodule.c b/Modules/_datetimemodule.c --- a/Modules/_datetimemodule.c +++ b/Modules/_datetimemodule.c @@ -3267,7 +3267,7 @@ Py_INCREF(self->name); return self->name; } - if (self == PyDateTime_TimeZone_UTC || + if ((PyObject *)self == PyDateTime_TimeZone_UTC || (GET_TD_DAYS(self->offset) == 0 && GET_TD_SECONDS(self->offset) == 0 && GET_TD_MICROSECONDS(self->offset) == 0)) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 8 03:39:57 2015 From: python-checkins at python.org (larry.hastings) Date: Tue, 08 Sep 2015 01:39:57 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_Made_a_first_attempt_at_updat?= =?utf-8?q?ing_PEP_101_on_how_to_update_the_new_web_site=2E?= Message-ID: <20150908013957.27693.47109@psf.io> https://hg.python.org/peps/rev/2d3b15b78beb changeset: 6043:2d3b15b78beb user: Larry Hastings date: Mon Sep 07 18:39:54 2015 -0700 summary: Made a first attempt at updating PEP 101 on how to update the new web site. Also updated the 3.5 release schedule with final dates for rc2 and rc3. files: pep-0101.txt | 104 +++++++++++++++++++++++--------------- pep-0478.txt | 4 +- 2 files changed, 64 insertions(+), 44 deletions(-) diff --git a/pep-0101.txt b/pep-0101.txt --- a/pep-0101.txt +++ b/pep-0101.txt @@ -35,12 +35,15 @@ which hopefully will be on the "web of trust" with at least one of the other release managers. - * Access to ``dl-files.iad1.psf.io``, the server that hosts download files. + * Access to ``dl-files.iad1.psf.io``, the server that hosts download files, + and ``docs.iad1.psf.io``, the server that hosts the documentation. You'll be uploading files directly here. * Shell access to ``hg.python.org``, the Python Mercurial host. You'll have to adapt repository configuration there. + * An administratior account on www.python.org, including an "API key". + * Write access to the PEP repository. If you're reading this, you probably already have this--the first @@ -161,7 +164,7 @@ extra effort. ___ Make sure the current branch of your release clone is the branch you - want to release from. + want to release from. (``hg id``) ___ Check the docs for markup errors. @@ -299,7 +302,7 @@ ___ The WE builds the Windows helpfile, using (in Doc/) - > make.bat htmlhelp (on Windows) + % make.bat htmlhelp (on Windows) to create suitable input for HTML Help Workshop in build/htmlhelp. HTML Help Workshop is then fired up on the created python33.hhp file, finally @@ -460,7 +463,8 @@ To do these steps, you must have the permission to edit the website. If you don't have that, ask someone on pydotorg at python.org for the proper - permissions. It's insane for you not to have it. + permissions. (Or ask Ewa, who coordinated the effort for the new newbsite + with RevSys.) XXX This is completely out of date for Django based python.org. @@ -470,46 +474,72 @@ None of the web site updates are automated by release.py. - ___ Build the basic site. - - In the top directory, do an `svn update` to get the latest code. In the - build subdirectory, do `make` to build the site. Do `make serve` to - start service the pages on localhost:8005. Hit that url to see the site - as it is right now. At any time you can re-run `make` to update the - local site. You don't have to restart the server. - - Don't `svn commit` until you're all done! + ___ Log in to http://www.python.org/admin . ___ If this is the first release for this version (even a new patch - version), you'll need to create a subdirectory inside download/releases - to hold the new version files. It's probably a good idea to copy an - existing recent directory and twiddle the files in there for the new - version number. + version), you'll need to create a "page" for the version. + Find the "Pages" section and click on "Add", then fill in the + form. - ___ Update the version specific pages. + Note that the easiest thing is probably to copy fields from + an existing Python release "page", editing as you go. - ___ cd to `download/releases/X.Y.Z` - ___ Edit the version numbers in content.ht - ___ Update the md5 checksums + ___ If this isn't the first release for a version, open the existing + "page" for editing and update it to the new release. Don't save yet! - ___ Comment out the "This is a preview release" or the "This is a - production release" paragraph as appropriate + ___ Now create a new "release" for the release. Currently "Releases" are + sorted under "Downloads". - Note, you don't have to copy any release files into this directory; - they only live on dl-files.iad1.psf.io in the ftp directory. + Again, the easiest thing is probably to copy fields from + an existing Python release "page", editing as you go. - ___ Edit `download/releases/content.ht` to update the version numbers for - this release. There are a bunch of places you need to touch: + The mysterious "Release page" field on the form needs the + ID number of the "page" for this version. You can get that + by examining the URL for the "change page" form for this + version. For example, the URL for editing the "page" + for Python 3.5 is: - ___ The subdirectory name as the first element in the Nav rows. - ___ Possibly the Releases section, and possibly in the experimental - releases section if this is an alpha, beta or release candidate. + https://www.python.org/admin/pages/page/1232/ - ___ Update the download page, editing `download/content.ht`. Pre-releases are - added only to the "Testing versions" list. + The page's ID number is the last field; here it is 1232. + + ___ Note that by convention, the "Content" on the page and + the "Content" on the release are the same, *except* the + "page" has a section on where to download the software. + + ___ "Save" the release. + + ___ Populate the release with the downloadable files. + + Your friend and mine, Georg Brandl, made a lovely tool + called "add-to-pydotorg.py". You can find it in the + "release" tree (next to "release.py"). You run the + tool on dl-files.iad1.psf.io, like this: + + % AUTH_INFO=: python add-to-pydotorg.py + + This walks the correct download directory for , + looks for files marked with , and populates + the "Release Files" for the correct "release" on the web + site with these files. Note that clears the "Release Files" + for the relevant version each time it's run. You may run + it from any directory you like, and you can run it as + many times as you like if the files happen to change. + Keep a copy in your home directory on dl-files and + keep it fresh. + + If new types of files are added to the release + (e.g. the web-based installers or redistributable zip + files added to Python 3.5) someone will need to update + add-to-pydotorg.py so it recognizes these new files. + (It's best to update add-to-pydotorg.py when file types + are removed, too.) ___ If this is a final release... + ___ XXX I didn't update this section yet. I assume there are web site + changes to make for a final release...? XXX + ___ Update the 'Quick Links' section on the front page. Edit the top-level `content.ht` file. @@ -522,16 +552,6 @@ ___ Add the new version to `doc/versions/content.ht`. - ___ Add a news section item to the front page by editing newsindex.yml. The - format should be pretty self evident. - - ___ When everything looks good, `svn commit` in the data directory. This - will trigger the live site to update itself, and at that point the - release is live. - - ___ If this is a final release, create a new python.org/X.Y Apache alias - (or ask pydotorg to do so for you). - Now it's time to write the announcement for the mailing lists. This is the fuzzy bit because not much can be automated. You can use an earlier announcement as a template, but edit it for content! diff --git a/pep-0478.txt b/pep-0478.txt --- a/pep-0478.txt +++ b/pep-0478.txt @@ -46,11 +46,11 @@ - 3.5.0 beta 3: July 5, 2015 - 3.5.0 beta 4: July 26, 2015 - 3.5.0 release candidate 1: August 10, 2015 +- 3.5.0 release candidate 2: August 25, 2015 +- 3.5.0 release candidate 3: September 7, 2015 Planned future release dates: -- 3.5.0 release candidate 2: August 23, 2015 -- 3.5.0 release candidate 3: September 6, 2015 - 3.5.0 final: September 13, 2015 -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Tue Sep 8 04:54:26 2015 From: python-checkins at python.org (serhiy.storchaka) Date: Tue, 08 Sep 2015 02:54:26 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzI0OTgy?= =?utf-8?q?=3A_shutil=2Emake=5Farchive=28=29_with_the_=22zip=22_format_now?= =?utf-8?q?_adds_entries?= Message-ID: <20150908025426.12017.81134@psf.io> https://hg.python.org/cpython/rev/705ec4145f06 changeset: 97762:705ec4145f06 branch: 2.7 parent: 97757:b609e391c04d user: Serhiy Storchaka date: Tue Sep 08 05:47:01 2015 +0300 summary: Issue #24982: shutil.make_archive() with the "zip" format now adds entries for directories (including empty directories) in ZIP file. Added test for comparing shutil.make_archive() with the "zip" command. files: Lib/shutil.py | 9 ++++++++ Lib/test/test_shutil.py | 31 +++++++++++++++++++++++++++- Misc/NEWS | 3 ++ 3 files changed, 41 insertions(+), 2 deletions(-) diff --git a/Lib/shutil.py b/Lib/shutil.py --- a/Lib/shutil.py +++ b/Lib/shutil.py @@ -449,7 +449,16 @@ if not dry_run: with zipfile.ZipFile(zip_filename, "w", compression=zipfile.ZIP_DEFLATED) as zf: + path = os.path.normpath(base_dir) + zf.write(path, path) + if logger is not None: + logger.info("adding '%s'", path) for dirpath, dirnames, filenames in os.walk(base_dir): + for name in sorted(dirnames): + path = os.path.normpath(os.path.join(dirpath, name)) + zf.write(path, path) + if logger is not None: + logger.info("adding '%s'", path) for name in filenames: path = os.path.normpath(os.path.join(dirpath, name)) if os.path.isfile(path): diff --git a/Lib/test/test_shutil.py b/Lib/test/test_shutil.py --- a/Lib/test/test_shutil.py +++ b/Lib/test/test_shutil.py @@ -476,15 +476,42 @@ base_name = os.path.join(work_dir, rel_base_name) with support.change_cwd(work_dir): - res = make_archive(rel_base_name, 'zip', root_dir, 'dist') + res = make_archive(rel_base_name, 'zip', root_dir, base_dir) self.assertEqual(res, base_name + '.zip') self.assertTrue(os.path.isfile(res)) self.assertTrue(zipfile.is_zipfile(res)) with zipfile.ZipFile(res) as zf: self.assertEqual(sorted(zf.namelist()), - ['dist/file1', 'dist/file2', 'dist/sub/file3']) + ['dist/', 'dist/file1', 'dist/file2', + 'dist/sub/', 'dist/sub/file3', 'dist/sub2/']) + @unittest.skipUnless(zlib, "Requires zlib") + @unittest.skipUnless(ZIP_SUPPORT, 'Need zip support to run') + @unittest.skipUnless(find_executable('zip'), + 'Need the zip command to run') + def test_zipfile_vs_zip(self): + root_dir, base_dir = self._create_files() + base_name = os.path.join(self.mkdtemp(), 'archive') + archive = make_archive(base_name, 'zip', root_dir, base_dir) + + # check if ZIP file was created + self.assertEqual(archive, base_name + '.zip') + self.assertTrue(os.path.isfile(archive)) + + # now create another ZIP file using `zip` + archive2 = os.path.join(root_dir, 'archive2.zip') + zip_cmd = ['zip', '-q', '-r', 'archive2.zip', base_dir] + with support.change_cwd(root_dir): + spawn(zip_cmd) + + self.assertTrue(os.path.isfile(archive2)) + # let's compare both ZIP files + with zipfile.ZipFile(archive) as zf: + names = zf.namelist() + with zipfile.ZipFile(archive2) as zf: + names2 = zf.namelist() + self.assertEqual(sorted(names), sorted(names2)) def test_make_archive(self): tmpdir = self.mkdtemp() diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -37,6 +37,9 @@ Library ------- +- Issue #24982: shutil.make_archive() with the "zip" format now adds entries + for directories (including empty directories) in ZIP file. + - Issue #17849: Raise a sensible exception if an invalid response is received for a HTTP tunnel request, as seen with some servers that do not support tunnelling. Initial patch from Cory Benfield. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 8 04:54:26 2015 From: python-checkins at python.org (serhiy.storchaka) Date: Tue, 08 Sep 2015 02:54:26 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzI0OTgy?= =?utf-8?q?=3A_shutil=2Emake=5Farchive=28=29_with_the_=22zip=22_format_now?= =?utf-8?q?_adds_entries?= Message-ID: <20150908025426.11995.87367@psf.io> https://hg.python.org/cpython/rev/19216f5f6ee0 changeset: 97763:19216f5f6ee0 branch: 3.4 parent: 97758:ef9346be45d0 user: Serhiy Storchaka date: Tue Sep 08 05:47:23 2015 +0300 summary: Issue #24982: shutil.make_archive() with the "zip" format now adds entries for directories (including empty directories) in ZIP file. Added test for comparing shutil.make_archive() with the "zip" command. files: Lib/shutil.py | 9 +++++++ Lib/test/test_shutil.py | 37 ++++++++++++++++++++++++---- Misc/NEWS | 3 ++ 3 files changed, 43 insertions(+), 6 deletions(-) diff --git a/Lib/shutil.py b/Lib/shutil.py --- a/Lib/shutil.py +++ b/Lib/shutil.py @@ -687,7 +687,16 @@ if not dry_run: with zipfile.ZipFile(zip_filename, "w", compression=zipfile.ZIP_DEFLATED) as zf: + path = os.path.normpath(base_dir) + zf.write(path, path) + if logger is not None: + logger.info("adding '%s'", path) for dirpath, dirnames, filenames in os.walk(base_dir): + for name in sorted(dirnames): + path = os.path.normpath(os.path.join(dirpath, name)) + zf.write(path, path) + if logger is not None: + logger.info("adding '%s'", path) for name in filenames: path = os.path.normpath(os.path.join(dirpath, name)) if os.path.isfile(path): diff --git a/Lib/test/test_shutil.py b/Lib/test/test_shutil.py --- a/Lib/test/test_shutil.py +++ b/Lib/test/test_shutil.py @@ -1064,15 +1064,42 @@ base_name = os.path.join(work_dir, rel_base_name) with support.change_cwd(work_dir): - res = make_archive(rel_base_name, 'zip', root_dir, 'dist') + res = make_archive(rel_base_name, 'zip', root_dir, base_dir) self.assertEqual(res, base_name + '.zip') self.assertTrue(os.path.isfile(res)) self.assertTrue(zipfile.is_zipfile(res)) with zipfile.ZipFile(res) as zf: self.assertCountEqual(zf.namelist(), - ['dist/file1', 'dist/file2', 'dist/sub/file3']) + ['dist/', 'dist/sub/', 'dist/sub2/', + 'dist/file1', 'dist/file2', 'dist/sub/file3']) + @requires_zlib + @unittest.skipUnless(ZIP_SUPPORT, 'Need zip support to run') + @unittest.skipUnless(find_executable('zip'), + 'Need the zip command to run') + def test_zipfile_vs_zip(self): + root_dir, base_dir = self._create_files() + base_name = os.path.join(self.mkdtemp(), 'archive') + archive = make_archive(base_name, 'zip', root_dir, base_dir) + + # check if ZIP file was created + self.assertEqual(archive, base_name + '.zip') + self.assertTrue(os.path.isfile(archive)) + + # now create another ZIP file using `zip` + archive2 = os.path.join(root_dir, 'archive2.zip') + zip_cmd = ['zip', '-q', '-r', 'archive2.zip', base_dir] + with support.change_cwd(root_dir): + spawn(zip_cmd) + + self.assertTrue(os.path.isfile(archive2)) + # let's compare both ZIP files + with zipfile.ZipFile(archive) as zf: + names = zf.namelist() + with zipfile.ZipFile(archive2) as zf: + names2 = zf.namelist() + self.assertEqual(sorted(names), sorted(names2)) def test_make_archive(self): tmpdir = self.mkdtemp() @@ -1183,11 +1210,9 @@ formats.append('bztar') root_dir, base_dir = self._create_files() + expected = rlistdir(root_dir) + expected.remove('outer') for format in formats: - expected = rlistdir(root_dir) - expected.remove('outer') - if format == 'zip': - expected.remove('dist/sub2/') base_name = os.path.join(self.mkdtemp(), 'archive') filename = make_archive(base_name, format, root_dir, base_dir) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -81,6 +81,9 @@ Library ------- +- Issue #24982: shutil.make_archive() with the "zip" format now adds entries + for directories (including empty directories) in ZIP file. + - Issue #25019: Fixed a crash caused by setting non-string key of expat parser. Based on patch by John Leitch. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 8 04:54:26 2015 From: python-checkins at python.org (serhiy.storchaka) Date: Tue, 08 Sep 2015 02:54:26 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Issue_=2324982=3A_shutil=2Emake=5Farchive=28=29_with_the_=22zi?= =?utf-8?q?p=22_format_now_adds_entries?= Message-ID: <20150908025426.101492.25543@psf.io> https://hg.python.org/cpython/rev/142a5027ab3e changeset: 97764:142a5027ab3e branch: 3.5 parent: 97759:cacdd5f71cf4 parent: 97763:19216f5f6ee0 user: Serhiy Storchaka date: Tue Sep 08 05:51:00 2015 +0300 summary: Issue #24982: shutil.make_archive() with the "zip" format now adds entries for directories (including empty directories) in ZIP file. Added test for comparing shutil.make_archive() with the "zip" command. files: Lib/shutil.py | 9 +++++++ Lib/test/test_shutil.py | 37 ++++++++++++++++++++++++---- Misc/NEWS | 3 ++ 3 files changed, 43 insertions(+), 6 deletions(-) diff --git a/Lib/shutil.py b/Lib/shutil.py --- a/Lib/shutil.py +++ b/Lib/shutil.py @@ -679,7 +679,16 @@ if not dry_run: with zipfile.ZipFile(zip_filename, "w", compression=zipfile.ZIP_DEFLATED) as zf: + path = os.path.normpath(base_dir) + zf.write(path, path) + if logger is not None: + logger.info("adding '%s'", path) for dirpath, dirnames, filenames in os.walk(base_dir): + for name in sorted(dirnames): + path = os.path.normpath(os.path.join(dirpath, name)) + zf.write(path, path) + if logger is not None: + logger.info("adding '%s'", path) for name in filenames: path = os.path.normpath(os.path.join(dirpath, name)) if os.path.isfile(path): diff --git a/Lib/test/test_shutil.py b/Lib/test/test_shutil.py --- a/Lib/test/test_shutil.py +++ b/Lib/test/test_shutil.py @@ -1070,15 +1070,42 @@ base_name = os.path.join(work_dir, rel_base_name) with support.change_cwd(work_dir): - res = make_archive(rel_base_name, 'zip', root_dir, 'dist') + res = make_archive(rel_base_name, 'zip', root_dir, base_dir) self.assertEqual(res, base_name + '.zip') self.assertTrue(os.path.isfile(res)) self.assertTrue(zipfile.is_zipfile(res)) with zipfile.ZipFile(res) as zf: self.assertCountEqual(zf.namelist(), - ['dist/file1', 'dist/file2', 'dist/sub/file3']) + ['dist/', 'dist/sub/', 'dist/sub2/', + 'dist/file1', 'dist/file2', 'dist/sub/file3']) + @requires_zlib + @unittest.skipUnless(ZIP_SUPPORT, 'Need zip support to run') + @unittest.skipUnless(find_executable('zip'), + 'Need the zip command to run') + def test_zipfile_vs_zip(self): + root_dir, base_dir = self._create_files() + base_name = os.path.join(self.mkdtemp(), 'archive') + archive = make_archive(base_name, 'zip', root_dir, base_dir) + + # check if ZIP file was created + self.assertEqual(archive, base_name + '.zip') + self.assertTrue(os.path.isfile(archive)) + + # now create another ZIP file using `zip` + archive2 = os.path.join(root_dir, 'archive2.zip') + zip_cmd = ['zip', '-q', '-r', 'archive2.zip', base_dir] + with support.change_cwd(root_dir): + spawn(zip_cmd) + + self.assertTrue(os.path.isfile(archive2)) + # let's compare both ZIP files + with zipfile.ZipFile(archive) as zf: + names = zf.namelist() + with zipfile.ZipFile(archive2) as zf: + names2 = zf.namelist() + self.assertEqual(sorted(names), sorted(names2)) def test_make_archive(self): tmpdir = self.mkdtemp() @@ -1191,11 +1218,9 @@ formats.append('xztar') root_dir, base_dir = self._create_files() + expected = rlistdir(root_dir) + expected.remove('outer') for format in formats: - expected = rlistdir(root_dir) - expected.remove('outer') - if format == 'zip': - expected.remove('dist/sub2/') base_name = os.path.join(self.mkdtemp(), 'archive') filename = make_archive(base_name, format, root_dir, base_dir) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -14,6 +14,9 @@ Library ------- +- Issue #24982: shutil.make_archive() with the "zip" format now adds entries + for directories (including empty directories) in ZIP file. + - Issue #25019: Fixed a crash caused by setting non-string key of expat parser. Based on patch by John Leitch. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 8 04:54:27 2015 From: python-checkins at python.org (serhiy.storchaka) Date: Tue, 08 Sep 2015 02:54:27 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2324982=3A_shutil=2Emake=5Farchive=28=29_with_the?= =?utf-8?q?_=22zip=22_format_now_adds_entries?= Message-ID: <20150908025426.115020.17843@psf.io> https://hg.python.org/cpython/rev/6907716e7ccb changeset: 97765:6907716e7ccb parent: 97761:ea1bfb64e898 parent: 97764:142a5027ab3e user: Serhiy Storchaka date: Tue Sep 08 05:53:42 2015 +0300 summary: Issue #24982: shutil.make_archive() with the "zip" format now adds entries for directories (including empty directories) in ZIP file. Added test for comparing shutil.make_archive() with the "zip" command. files: Lib/shutil.py | 9 +++++++ Lib/test/test_shutil.py | 37 ++++++++++++++++++++++++---- Misc/NEWS | 3 ++ 3 files changed, 43 insertions(+), 6 deletions(-) diff --git a/Lib/shutil.py b/Lib/shutil.py --- a/Lib/shutil.py +++ b/Lib/shutil.py @@ -679,7 +679,16 @@ if not dry_run: with zipfile.ZipFile(zip_filename, "w", compression=zipfile.ZIP_DEFLATED) as zf: + path = os.path.normpath(base_dir) + zf.write(path, path) + if logger is not None: + logger.info("adding '%s'", path) for dirpath, dirnames, filenames in os.walk(base_dir): + for name in sorted(dirnames): + path = os.path.normpath(os.path.join(dirpath, name)) + zf.write(path, path) + if logger is not None: + logger.info("adding '%s'", path) for name in filenames: path = os.path.normpath(os.path.join(dirpath, name)) if os.path.isfile(path): diff --git a/Lib/test/test_shutil.py b/Lib/test/test_shutil.py --- a/Lib/test/test_shutil.py +++ b/Lib/test/test_shutil.py @@ -1070,15 +1070,42 @@ base_name = os.path.join(work_dir, rel_base_name) with support.change_cwd(work_dir): - res = make_archive(rel_base_name, 'zip', root_dir, 'dist') + res = make_archive(rel_base_name, 'zip', root_dir, base_dir) self.assertEqual(res, base_name + '.zip') self.assertTrue(os.path.isfile(res)) self.assertTrue(zipfile.is_zipfile(res)) with zipfile.ZipFile(res) as zf: self.assertCountEqual(zf.namelist(), - ['dist/file1', 'dist/file2', 'dist/sub/file3']) + ['dist/', 'dist/sub/', 'dist/sub2/', + 'dist/file1', 'dist/file2', 'dist/sub/file3']) + @requires_zlib + @unittest.skipUnless(ZIP_SUPPORT, 'Need zip support to run') + @unittest.skipUnless(find_executable('zip'), + 'Need the zip command to run') + def test_zipfile_vs_zip(self): + root_dir, base_dir = self._create_files() + base_name = os.path.join(self.mkdtemp(), 'archive') + archive = make_archive(base_name, 'zip', root_dir, base_dir) + + # check if ZIP file was created + self.assertEqual(archive, base_name + '.zip') + self.assertTrue(os.path.isfile(archive)) + + # now create another ZIP file using `zip` + archive2 = os.path.join(root_dir, 'archive2.zip') + zip_cmd = ['zip', '-q', '-r', 'archive2.zip', base_dir] + with support.change_cwd(root_dir): + spawn(zip_cmd) + + self.assertTrue(os.path.isfile(archive2)) + # let's compare both ZIP files + with zipfile.ZipFile(archive) as zf: + names = zf.namelist() + with zipfile.ZipFile(archive2) as zf: + names2 = zf.namelist() + self.assertEqual(sorted(names), sorted(names2)) def test_make_archive(self): tmpdir = self.mkdtemp() @@ -1191,11 +1218,9 @@ formats.append('xztar') root_dir, base_dir = self._create_files() + expected = rlistdir(root_dir) + expected.remove('outer') for format in formats: - expected = rlistdir(root_dir) - expected.remove('outer') - if format == 'zip': - expected.remove('dist/sub2/') base_name = os.path.join(self.mkdtemp(), 'archive') filename = make_archive(base_name, format, root_dir, base_dir) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -103,6 +103,9 @@ Library ------- +- Issue #24982: shutil.make_archive() with the "zip" format now adds entries + for directories (including empty directories) in ZIP file. + - Issue #25019: Fixed a crash caused by setting non-string key of expat parser. Based on patch by John Leitch. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 8 06:32:20 2015 From: python-checkins at python.org (zach.ware) Date: Tue, 08 Sep 2015 04:32:20 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Issue_=2325010=3A_Merge_with_3=2E4?= Message-ID: <20150908043220.15708.18662@psf.io> https://hg.python.org/cpython/rev/309c0331726b changeset: 97767:309c0331726b branch: 3.5 parent: 97764:142a5027ab3e parent: 97766:7f4102a9818b user: Zachary Ware date: Mon Sep 07 23:31:49 2015 -0500 summary: Issue #25010: Merge with 3.4 files: PCbuild/readme.txt | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/PCbuild/readme.txt b/PCbuild/readme.txt --- a/PCbuild/readme.txt +++ b/PCbuild/readme.txt @@ -338,5 +338,5 @@ ----------------------- If you want to create your own extension module DLL (.pyd), there's an -example with easy-to-follow instructions in ..\PC\example\; read the +example with easy-to-follow instructions in ..\PC\example_nt\; read the file readme.txt there first. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 8 06:32:20 2015 From: python-checkins at python.org (zach.ware) Date: Tue, 08 Sep 2015 04:32:20 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzI1MDEw?= =?utf-8?q?=3A_Fix_path_for_=2Epyd_example_project=2E?= Message-ID: <20150908043220.17983.79432@psf.io> https://hg.python.org/cpython/rev/7f4102a9818b changeset: 97766:7f4102a9818b branch: 3.4 parent: 97763:19216f5f6ee0 user: Zachary Ware date: Mon Sep 07 23:30:46 2015 -0500 summary: Issue #25010: Fix path for .pyd example project. Patch by Shaun Walbridge files: PCbuild/readme.txt | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/PCbuild/readme.txt b/PCbuild/readme.txt --- a/PCbuild/readme.txt +++ b/PCbuild/readme.txt @@ -318,5 +318,5 @@ ----------------------- If you want to create your own extension module DLL (.pyd), there's an -example with easy-to-follow instructions in ..\PC\example\; read the +example with easy-to-follow instructions in ..\PC\example_nt\; read the file readme.txt there first. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 8 06:32:20 2015 From: python-checkins at python.org (zach.ware) Date: Tue, 08 Sep 2015 04:32:20 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Closes_=2325010=3A_Merge_with_3=2E5?= Message-ID: <20150908043220.66864.37514@psf.io> https://hg.python.org/cpython/rev/3829f567d269 changeset: 97768:3829f567d269 parent: 97765:6907716e7ccb parent: 97767:309c0331726b user: Zachary Ware date: Mon Sep 07 23:32:10 2015 -0500 summary: Closes #25010: Merge with 3.5 files: PCbuild/readme.txt | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/PCbuild/readme.txt b/PCbuild/readme.txt --- a/PCbuild/readme.txt +++ b/PCbuild/readme.txt @@ -338,5 +338,5 @@ ----------------------- If you want to create your own extension module DLL (.pyd), there's an -example with easy-to-follow instructions in ..\PC\example\; read the +example with easy-to-follow instructions in ..\PC\example_nt\; read the file readme.txt there first. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 8 06:37:26 2015 From: python-checkins at python.org (raymond.hettinger) Date: Tue, 08 Sep 2015 04:37:26 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_Fix_whitespace?= =?utf-8?q?_in_comment=2E?= Message-ID: <20150908043725.27689.63291@psf.io> https://hg.python.org/cpython/rev/fc71979b80a5 changeset: 97769:fc71979b80a5 branch: 3.4 parent: 97766:7f4102a9818b user: Raymond Hettinger date: Tue Sep 08 00:36:29 2015 -0400 summary: Fix whitespace in comment. files: Lib/collections/__init__.py | 5 ++--- 1 files changed, 2 insertions(+), 3 deletions(-) diff --git a/Lib/collections/__init__.py b/Lib/collections/__init__.py --- a/Lib/collections/__init__.py +++ b/Lib/collections/__init__.py @@ -836,9 +836,8 @@ __copy__ = copy def new_child(self, m=None): # like Django's Context.push() - ''' - New ChainMap with a new map followed by all previous maps. If no - map is provided, an empty dict is used. + '''New ChainMap with a new map followed by all previous maps. + If no map is provided, an empty dict is used. ''' if m is None: m = {} -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 8 06:37:26 2015 From: python-checkins at python.org (raymond.hettinger) Date: Tue, 08 Sep 2015 04:37:26 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_merge?= Message-ID: <20150908043726.101480.37344@psf.io> https://hg.python.org/cpython/rev/c07664203a19 changeset: 97770:c07664203a19 branch: 3.5 parent: 97767:309c0331726b parent: 97769:fc71979b80a5 user: Raymond Hettinger date: Tue Sep 08 00:36:56 2015 -0400 summary: merge files: Lib/collections/__init__.py | 5 ++--- 1 files changed, 2 insertions(+), 3 deletions(-) diff --git a/Lib/collections/__init__.py b/Lib/collections/__init__.py --- a/Lib/collections/__init__.py +++ b/Lib/collections/__init__.py @@ -892,9 +892,8 @@ __copy__ = copy def new_child(self, m=None): # like Django's Context.push() - ''' - New ChainMap with a new map followed by all previous maps. If no - map is provided, an empty dict is used. + '''New ChainMap with a new map followed by all previous maps. + If no map is provided, an empty dict is used. ''' if m is None: m = {} -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 8 06:37:26 2015 From: python-checkins at python.org (raymond.hettinger) Date: Tue, 08 Sep 2015 04:37:26 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_merge?= Message-ID: <20150908043726.66870.39771@psf.io> https://hg.python.org/cpython/rev/5b31d21c6bb6 changeset: 97771:5b31d21c6bb6 parent: 97768:3829f567d269 parent: 97770:c07664203a19 user: Raymond Hettinger date: Tue Sep 08 00:37:17 2015 -0400 summary: merge files: Lib/collections/__init__.py | 5 ++--- 1 files changed, 2 insertions(+), 3 deletions(-) diff --git a/Lib/collections/__init__.py b/Lib/collections/__init__.py --- a/Lib/collections/__init__.py +++ b/Lib/collections/__init__.py @@ -892,9 +892,8 @@ __copy__ = copy def new_child(self, m=None): # like Django's Context.push() - ''' - New ChainMap with a new map followed by all previous maps. If no - map is provided, an empty dict is used. + '''New ChainMap with a new map followed by all previous maps. + If no map is provided, an empty dict is used. ''' if m is None: m = {} -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 8 08:13:09 2015 From: python-checkins at python.org (zach.ware) Date: Tue, 08 Sep 2015 06:13:09 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_Update_PCbuild?= =?utf-8?q?/readme=2Etxt?= Message-ID: <20150908061309.27689.11661@psf.io> https://hg.python.org/cpython/rev/e4a2089ad684 changeset: 97773:e4a2089ad684 branch: 3.5 parent: 97770:c07664203a19 user: Zachary Ware date: Tue Sep 08 01:12:00 2015 -0500 summary: Update PCbuild/readme.txt files: PCbuild/readme.txt | 44 +++------------------------------ 1 files changed, 5 insertions(+), 39 deletions(-) diff --git a/PCbuild/readme.txt b/PCbuild/readme.txt --- a/PCbuild/readme.txt +++ b/PCbuild/readme.txt @@ -41,8 +41,7 @@ used to build standard x86-compatible 32-bit binaries, output into the win32 sub-directory. The x64 platform is used for building 64-bit AMD64 (aka x86_64 or EM64T) binaries, output into the amd64 sub-directory. -The Itanium (IA-64) platform is no longer supported. See the "Building -for AMD64" section below for more information about 64-bit builds. +The Itanium (IA-64) platform is no longer supported. Four configuration options are supported by the solution: Debug @@ -76,29 +75,7 @@ By default, build.bat will build Python in Release configuration for the 32-bit Win32 platform. It accepts several arguments to change -this behavior: - - -c Set the configuration (see above) - -d Shortcut for "-c Debug" - -p Set the platform to build for ("Win32" or "x64") - -r Rebuild instead of just building - -t Set the target (Build, Rebuild, Clean or CleanAll) - -e Use get_externals.bat to fetch external sources - -M Don't build in parallel - -v Increased output messages - -Up to 9 MSBuild switches can also be passed, though they must be passed -after specifying any of the above switches. For example, use: - - build.bat -e -d /fl - -to do a debug build with externals fetched as needed and write detailed -build logs to a file. If the MSBuild switch requires an equal sign -("="), the entire switch must be quoted: - - build.bat -e -d "/p:ExternalsDir=P:\cpython-externals" - -There may also be other situations where quotes are necessary. +this behavior, try `build.bat -h` to learn more. C Runtime @@ -129,8 +106,6 @@ .dll and .lib python .exe -make_buildinfo, make_versioninfo - helpers to provide necessary information to the build process These sub-projects provide extra executables that are useful for running CPython in different ways: @@ -152,9 +127,6 @@ _freeze_importlib _freeze_importlib.exe, used to regenerate Python\importlib.h after changes have been made to Lib\importlib\_bootstrap.py -bdist_wininst - ..\Lib\distutils\command\wininst-14.0[-amd64].exe, the base - executable used by the distutils bdist_wininst command python3dll python3.dll, the PEP 384 Stable ABI dll xxlimited @@ -274,14 +246,8 @@ find them. This is an advanced topic and not necessarily fully supported. - -Building for AMD64 ------------------- - -The build process for AMD64 / x64 is very similar to standard builds, -you just have to set x64 as platform. In addition, the HOST_PYTHON -environment variable must point to a Python interpreter (at least 2.4), -to support cross-compilation from Win32. +The get_externals.bat script is called automatically by build.bat when +you pass the '-e' option to it. Profile Guided Optimization @@ -298,7 +264,7 @@ PGI python, and finally creates the optimized files. See - http://msdn.microsoft.com/en-us/library/e7k32f4k(VS.100).aspx + http://msdn.microsoft.com/en-us/library/e7k32f4k(VS.140).aspx for more on this topic. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 8 08:13:09 2015 From: python-checkins at python.org (zach.ware) Date: Tue, 08 Sep 2015 06:13:09 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Update_PCbuild/readme=2Etxt_=28merge_from_3=2E5=29?= Message-ID: <20150908061309.66864.57962@psf.io> https://hg.python.org/cpython/rev/73258658dc76 changeset: 97774:73258658dc76 parent: 97771:5b31d21c6bb6 parent: 97773:e4a2089ad684 user: Zachary Ware date: Tue Sep 08 01:12:56 2015 -0500 summary: Update PCbuild/readme.txt (merge from 3.5) files: PCbuild/readme.txt | 46 ++++----------------------------- 1 files changed, 6 insertions(+), 40 deletions(-) diff --git a/PCbuild/readme.txt b/PCbuild/readme.txt --- a/PCbuild/readme.txt +++ b/PCbuild/readme.txt @@ -41,15 +41,14 @@ used to build standard x86-compatible 32-bit binaries, output into the win32 sub-directory. The x64 platform is used for building 64-bit AMD64 (aka x86_64 or EM64T) binaries, output into the amd64 sub-directory. -The Itanium (IA-64) platform is no longer supported. See the "Building -for AMD64" section below for more information about 64-bit builds. +The Itanium (IA-64) platform is no longer supported. Four configuration options are supported by the solution: Debug Used to build Python with extra debugging capabilities, equivalent to using ./configure --with-pydebug on UNIX. All binaries built using this configuration have "_d" added to their name: - python35_d.dll, python_d.exe, parser_d.pyd, and so on. Both the + python36_d.dll, python_d.exe, parser_d.pyd, and so on. Both the build and rt (run test) batch files in this directory accept a -d option for debug builds. If you are building Python to help with development of CPython, you will most likely use this configuration. @@ -76,29 +75,7 @@ By default, build.bat will build Python in Release configuration for the 32-bit Win32 platform. It accepts several arguments to change -this behavior: - - -c Set the configuration (see above) - -d Shortcut for "-c Debug" - -p Set the platform to build for ("Win32" or "x64") - -r Rebuild instead of just building - -t Set the target (Build, Rebuild, Clean or CleanAll) - -e Use get_externals.bat to fetch external sources - -M Don't build in parallel - -v Increased output messages - -Up to 9 MSBuild switches can also be passed, though they must be passed -after specifying any of the above switches. For example, use: - - build.bat -e -d /fl - -to do a debug build with externals fetched as needed and write detailed -build logs to a file. If the MSBuild switch requires an equal sign -("="), the entire switch must be quoted: - - build.bat -e -d "/p:ExternalsDir=P:\cpython-externals" - -There may also be other situations where quotes are necessary. +this behavior, try `build.bat -h` to learn more. C Runtime @@ -129,8 +106,6 @@ .dll and .lib python .exe -make_buildinfo, make_versioninfo - helpers to provide necessary information to the build process These sub-projects provide extra executables that are useful for running CPython in different ways: @@ -152,9 +127,6 @@ _freeze_importlib _freeze_importlib.exe, used to regenerate Python\importlib.h after changes have been made to Lib\importlib\_bootstrap.py -bdist_wininst - ..\Lib\distutils\command\wininst-14.0[-amd64].exe, the base - executable used by the distutils bdist_wininst command python3dll python3.dll, the PEP 384 Stable ABI dll xxlimited @@ -274,14 +246,8 @@ find them. This is an advanced topic and not necessarily fully supported. - -Building for AMD64 ------------------- - -The build process for AMD64 / x64 is very similar to standard builds, -you just have to set x64 as platform. In addition, the HOST_PYTHON -environment variable must point to a Python interpreter (at least 2.4), -to support cross-compilation from Win32. +The get_externals.bat script is called automatically by build.bat when +you pass the '-e' option to it. Profile Guided Optimization @@ -298,7 +264,7 @@ PGI python, and finally creates the optimized files. See - http://msdn.microsoft.com/en-us/library/e7k32f4k(VS.100).aspx + http://msdn.microsoft.com/en-us/library/e7k32f4k(VS.140).aspx for more on this topic. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 8 08:13:09 2015 From: python-checkins at python.org (zach.ware) Date: Tue, 08 Sep 2015 06:13:09 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_Update_PCbuild?= =?utf-8?q?/readme=2Etxt?= Message-ID: <20150908061308.114820.37424@psf.io> https://hg.python.org/cpython/rev/1c5a171e7a13 changeset: 97772:1c5a171e7a13 branch: 2.7 parent: 97762:705ec4145f06 user: Zachary Ware date: Tue Sep 08 01:04:01 2015 -0500 summary: Update PCbuild/readme.txt It now better matches 3.5+ and the new reality of 2.7's PCbuild dir. files: PCbuild/readme.txt | 382 ++++++++++++++++++-------------- 1 files changed, 215 insertions(+), 167 deletions(-) diff --git a/PCbuild/readme.txt b/PCbuild/readme.txt --- a/PCbuild/readme.txt +++ b/PCbuild/readme.txt @@ -1,3 +1,14 @@ +Quick Start Guide +----------------- + +1. Install Microsoft Visual Studio 2008, any edition. +2. Install Microsoft Visual Studio 2010, any edition, or Windows SDK 7.1 + and any version of Microsoft Visual Studio newer than 2010. +3. Install Subversion, and make sure 'svn.exe' is on your PATH. +4. Run "build.bat -e" to build Python in 32-bit Release configuration. +5. (Optional, but recommended) Run the test suite with "rt.bat -q". + + Building Python using MSVC 9.0 via MSBuild ------------------------------------------ @@ -27,43 +38,63 @@ For other Windows platforms and compilers, see ../PC/readme.txt. -All you need to do is open the workspace "pcbuild.sln" in Visual Studio, -select the desired combination of configuration and platform and eventually -build the solution. Unless you are going to debug a problem in the core or -you are going to create an optimized build you want to select "Release" as -configuration. +All you need to do to build is open the solution "pcbuild.sln" in Visual +Studio, select the desired combination of configuration and platform, +then build with "Build Solution". You can also build from the command +line using the "build.bat" script in this directory; see below for +details. The solution is configured to build the projects in the correct +order. -The PCbuild directory is compatible with all versions of Visual Studio from -VS C++ Express Edition over the standard edition up to the professional -edition. However the express edition does not support features like solution -folders or profile guided optimization (PGO). The missing bits and pieces -won't stop you from building Python. +The solution currently supports two platforms. The Win32 platform is +used to build standard x86-compatible 32-bit binaries, output into this +directory. The x64 platform is used for building 64-bit AMD64 (aka +x86_64 or EM64T) binaries, output into the amd64 sub-directory. The +Itanium (IA-64) platform is no longer supported. -The solution is configured to build the projects in the correct order. "Build -Solution" or F7 takes care of dependencies except for x64 builds. To make -cross compiling x64 builds on a 32bit OS possible the x64 builds require a -32bit version of Python. +Four configuration options are supported by the solution: +Debug + Used to build Python with extra debugging capabilities, equivalent + to using ./configure --with-pydebug on UNIX. All binaries built + using this configuration have "_d" added to their name: + python27_d.dll, python_d.exe, parser_d.pyd, and so on. Both the + build and rt (run test) batch files in this directory accept a -d + option for debug builds. If you are building Python to help with + development of CPython, you will most likely use this configuration. +PGInstrument, PGUpdate + Used to build Python in Release configuration using PGO, which + requires Professional Edition of Visual Studio 2008. See the + "Profile Guided Optimization" section below for more information. + Build output from each of these configurations lands in its own + sub-directory of this directory. The official Python releases may + be built using these configurations. +Release + Used to build Python as it is meant to be used in production + settings, though without PGO. -NOTE: - You probably don't want to build most of the other subprojects, unless - you're building an entire Python distribution from scratch, or - specifically making changes to the subsystems they implement, or are - running a Python core buildbot test slave; see SUBPROJECTS below) -When using the Debug setting, the output files have a _d added to -their name: python27_d.dll, python_d.exe, parser_d.pyd, and so on. Both -the build and rt batch files accept a -d option for debug builds. +Building Python using the build.bat script +---------------------------------------------- -The 32bit builds end up in the solution folder PCbuild while the x64 builds -land in the amd64 subfolder. The PGI and PGO builds for profile guided -optimization end up in their own folders, too. +In this directory you can find build.bat, a script designed to make +building Python on Windows simpler. This script will use the env.bat +script to detect one of Visual Studio 2015, 2013, 2012, or 2010, any of +which contains a usable version of MSBuild. + +By default, build.bat will build Python in Release configuration for +the 32-bit Win32 platform. It accepts several arguments to change +this behavior, try `build.bat -h` to learn more. + Legacy support -------------- You can find build directories for older versions of Visual Studio and -Visual C++ in the PC directory. The legacy build directories are no longer -actively maintained and may not work out of the box. +Visual C++ in the PC directory. The project files in PC/VS9.0/ are +specific to Visual Studio 2008, and will be fully supported for the life +of Python 2.7. + +The following legacy build directories are no longer maintained and may +not work out of the box. PC/VC6/ Visual C++ 6.0 @@ -73,7 +104,7 @@ Visual Studio 2005 (8.0) -C RUNTIME +C Runtime --------- Visual Studio 2008 uses version 9 of the C runtime (MSVCRT9). The executables @@ -88,187 +119,204 @@ also set the PATH to this directory so that the dll can be found. For more info, see the Readme in the VC/Redist folder. -SUBPROJECTS ------------ -These subprojects should build out of the box. Subprojects other than the -main ones (pythoncore, python, pythonw) generally build a DLL (renamed to -.pyd) from a specific module so that users don't have to load the code -supporting that module unless they import the module. +Sub-Projects +------------ + +The CPython project is split up into several smaller sub-projects which +are managed by the pcbuild.sln solution file. Each sub-project is +represented by a .vcxproj and a .vcxproj.filters file starting with the +name of the sub-project. These sub-projects fall into a few general +categories: + +The following sub-projects represent the bare minimum required to build +a functioning CPython interpreter. If nothing else builds but these, +you'll have a very limited but usable python.exe: pythoncore .dll and .lib python .exe + +These sub-projects provide extra executables that are useful for running +CPython in different ways: pythonw - pythonw.exe, a variant of python.exe that doesn't pop up a DOS box + pythonw.exe, a variant of python.exe that doesn't open a Command + Prompt window +pylauncher + py.exe, the Python Launcher for Windows, see + http://docs.python.org/3/using/windows.html#launcher +pywlauncher + pyw.exe, a variant of py.exe that doesn't open a Command Prompt + window + +The following sub-projects are for individual modules of the standard +library which are implemented in C; each one builds a DLL (renamed to +.pyd) of the same name as the project: +_ctypes +_ctypes_test +_elementtree +_hashlib +_msi +_multiprocessing _socket - socketmodule.c _testcapi - tests of the Python C API, run via Lib/test/test_capi.py, and - implemented by module Modules/_testcapimodule.c pyexpat - Python wrapper for accelerated XML parsing, which incorporates stable - code from the Expat project: http://sourceforge.net/projects/expat/ select - selectmodule.c unicodedata - large tables of Unicode data winsound - play sounds (typically .wav files) under Windows -Python-controlled subprojects that wrap external projects: +There is also a w9xpopen project to build w9xpopen.exe, which is used +for platform.popen() on platforms whose COMSPEC points to 'command.com'. + +The following Python-controlled sub-projects wrap external projects. +Note that these external libraries are not necessary for a working +interpreter, but they do implement several major features. See the +"Getting External Sources" section below for additional information +about getting the source for building these libraries. The sub-projects +are: _bsddb - Wraps Berkeley DB 4.7.25, which is currently built by _bsddb.vcproj. - project. + Python wrapper for Berkeley DB version 4.7.25. + Homepage: + http://www.oracle.com/us/products/database/berkeley-db/ +_bz2 + Python wrapper for version 1.0.6 of the libbzip2 compression library + Homepage: + http://www.bzip.org/ +_ssl + Python wrapper for version 1.0.2d of the OpenSSL secure sockets + library, which is built by ssl.vcxproj + Homepage: + http://www.openssl.org/ + + Building OpenSSL requires nasm.exe (the Netwide Assembler), version + 2.10 or newer from + http://www.nasm.us/ + to be somewhere on your PATH. More recent versions of OpenSSL may + need a later version of NASM. If OpenSSL's self tests don't pass, + you should first try to update NASM and do a full rebuild of + OpenSSL. If you use the PCbuild\get_externals.bat method + for getting sources, it also downloads a version of NASM which the + libeay/ssleay sub-projects use. + + The libeay/ssleay sub-projects expect your OpenSSL sources to have + already been configured and be ready to build. If you get your sources + from svn.python.org as suggested in the "Getting External Sources" + section below, the OpenSSL source will already be ready to go. If + you want to build a different version, you will need to run + + PCbuild\prepare_ssl.py path\to\openssl-source-dir + + That script will prepare your OpenSSL sources in the same way that + those available on svn.python.org have been prepared. Note that + Perl must be installed and available on your PATH to configure + OpenSSL. ActivePerl is recommended and is available from + http://www.activestate.com/activeperl/ + + The libeay and ssleay sub-projects will build the modules of OpenSSL + required by _ssl and _hashlib and may need to be manually updated when + upgrading to a newer version of OpenSSL or when adding new + functionality to _ssl or _hashlib. They will not clean up their output + with the normal Clean target; CleanAll should be used instead. _sqlite3 - Wraps SQLite 3.6.21, which is currently built by sqlite3.vcproj. + Wraps SQLite 3.6.21, which is itself built by sqlite3.vcxproj + Homepage: + http://www.sqlite.org/ _tkinter - Wraps the Tk windowing system. Unlike _bsddb and _sqlite3, there's no - corresponding tcltk.vcproj-type project that builds Tcl/Tk from vcproj's - within our pcbuild.sln, which means this module expects to find a - pre-built Tcl/Tk in either ..\externals\tcltk for 32-bit or - ..\externals\tcltk64 for 64-bit (relative to this directory). See below - for instructions to build Tcl/Tk. -bz2 - Python wrapper for the libbz2 compression library. Homepage - http://sources.redhat.com/bzip2/ - Download the source from the python.org copy into the dist - directory: + Wraps version 8.5.15 of the Tk windowing system. + Homepage: + http://www.tcl.tk/ - svn export http://svn.python.org/projects/external/bzip2-1.0.6 + Tkinter's dependencies are built by the tcl.vcxproj and tk.vcxproj + projects. The tix.vcxproj project also builds the Tix extended + widget set for use with Tkinter. - ** NOTE: if you use the PCbuild\get_externals.bat approach for - obtaining external sources then you don't need to manually get the source - above via subversion. ** + Those three projects install their respective components in a + directory alongside the source directories called "tcltk" on + Win32 and "tcltk64" on x64. They also copy the Tcl and Tk DLLs + into the current output directory, which should ensure that Tkinter + is able to load Tcl/Tk without having to change your PATH. -_ssl - Python wrapper for the secure sockets library. + The tcl, tk, and tix sub-projects do not clean their builds with + the normal Clean target; if you need to rebuild, you should use the + CleanAll target or manually delete their builds. - Get the source code through - svn export http://svn.python.org/projects/external/openssl-1.0.2d +Getting External Sources +------------------------ - ** NOTE: if you use the PCbuild\get_externals.bat approach for - obtaining external sources then you don't need to manually get the source - above via subversion. ** +The last category of sub-projects listed above wrap external projects +Python doesn't control, and as such a little more work is required in +order to download the relevant source files for each project before they +can be built. However, a simple script is provided to make this as +painless as possible, called "get_externals.bat" and located in this +directory. This script extracts all the external sub-projects from + http://svn.python.org/projects/external +via Subversion (so you'll need svn.exe on your PATH) and places them +in ..\externals (relative to this directory). - The NASM assembler is required to build OpenSSL. If you use the - PCbuild\get_externals.bat script to get external library sources, it also - downloads a version of NASM, which the ssl build script will add to PATH. - Otherwise, you can download the NASM installer from - http://www.nasm.us/ - and add NASM to your PATH. +It is also possible to download sources from each project's homepage, +though you may have to change folder names or pass the names to MSBuild +as the values of certain properties in order for the build solution to +find them. This is an advanced topic and not necessarily fully +supported. - You can also install ActivePerl from - http://www.activestate.com/activeperl/ - if you like to use the official sources instead of the files from - python's subversion repository. The svn version contains pre-build - makefiles and assembly files. +The get_externals.bat script is called automatically by build.bat when +you pass the '-e' option to it. - The build process makes sure that no patented algorithms are included. - For now RC5, MDC2 and IDEA are excluded from the build. You may have - to manually remove $(OBJ_D)\i_*.obj from ms\nt.mak if the build process - complains about missing files or forbidden IDEA. Again the files provided - in the subversion repository are already fixed. - - The MSVC project simply invokes PCBuild/build_ssl.py to perform - the build. This Python script locates and builds your OpenSSL - installation, then invokes a simple makefile to build the final .pyd. - - build_ssl.py attempts to catch the most common errors (such as not - being able to find OpenSSL sources, or not being able to find a Perl - that works with OpenSSL) and give a reasonable error message. - If you have a problem that doesn't seem to be handled correctly - (eg, you know you have ActivePerl but we can't find it), please take - a peek at build_ssl.py and suggest patches. Note that build_ssl.py - should be able to be run directly from the command-line. - - build_ssl.py/MSVC isn't clever enough to clean OpenSSL - you must do - this by hand. - -The subprojects above wrap external projects Python doesn't control, and as -such, a little more work is required in order to download the relevant source -files for each project before they can be built. The easiest way to do this -is to use the `build.bat` script in this directory to build Python, and pass -the '-e' switch to tell it to use get_externals.bat to fetch external sources -and build Tcl/Tk and Tix. To use get_externals.bat, you'll need to have -Subversion installed and svn.exe on your PATH. The script will fetch external -library sources from http://svn.python.org/external and place them in -..\externals (relative to this directory). - -Building for Itanium --------------------- - -Official support for Itanium builds have been dropped from the build. Please -contact us and provide patches if you are interested in Itanium builds. - -Building for AMD64 ------------------- - -The build process for AMD64 / x64 is very similar to standard builds. You just -have to set x64 as platform. In addition, the HOST_PYTHON environment variable -must point to a Python interpreter (at least 2.4), to support cross-compilation. - -Building Python Using the free MS Toolkit Compiler --------------------------------------------------- - -Microsoft has withdrawn the free MS Toolkit Compiler, so this can no longer -be considered a supported option. Instead you can use the free VS C++ Express -Edition. Profile Guided Optimization --------------------------- The solution has two configurations for PGO. The PGInstrument -configuration must be build first. The PGInstrument binaries are -linked against a profiling library and contain extra debug -information. The PGUpdate configuration takes the profiling data and -generates optimized binaries. +configuration must be built first. The PGInstrument binaries are linked +against a profiling library and contain extra debug information. The +PGUpdate configuration takes the profiling data and generates optimized +binaries. -The build_pgo.bat script automates the creation of optimized binaries. It -creates the PGI files, runs the unit test suite or PyBench with the PGI -python and finally creates the optimized files. +The build_pgo.bat script automates the creation of optimized binaries. +It creates the PGI files, runs the unit test suite or PyBench with the +PGI python, and finally creates the optimized files. -http://msdn.microsoft.com/en-us/library/e7k32f4k(VS.90).aspx +See + http://msdn.microsoft.com/en-us/library/e7k32f4k(VS.90).aspx +for more on this topic. + Static library -------------- -The solution has no configuration for static libraries. However it is easy -it build a static library instead of a DLL. You simply have to set the -"Configuration Type" to "Static Library (.lib)" and alter the preprocessor -macro "Py_ENABLE_SHARED" to "Py_NO_ENABLE_SHARED". You may also have to -change the "Runtime Library" from "Multi-threaded DLL (/MD)" to -"Multi-threaded (/MT)". +The solution has no configuration for static libraries. However it is +easy to build a static library instead of a DLL. You simply have to set +the "Configuration Type" to "Static Library (.lib)" and alter the +preprocessor macro "Py_ENABLE_SHARED" to "Py_NO_ENABLE_SHARED". You may +also have to change the "Runtime Library" from "Multi-threaded DLL +(/MD)" to "Multi-threaded (/MT)". + Visual Studio properties ------------------------ -The PCbuild solution makes heavy use of Visual Studio property files -(*.vsprops). The properties can be viewed and altered in the Property -Manager (View -> Other Windows -> Property Manager). +The PCbuild solution makes use of Visual Studio property files (*.props) +to simplify each project. The properties can be viewed in the Property +Manager (View -> Other Windows -> Property Manager) but should be +carefully modified by hand. - * debug (debug macro: _DEBUG) - * pginstrument (PGO) - * pgupdate (PGO) - +-- pginstrument - * pyd (python extension, release build) - +-- release - +-- pyproject - * pyd_d (python extension, debug build) - +-- debug - +-- pyproject - * pyproject (base settings for all projects, user macros like PyDllName) - * release (release macro: NDEBUG) - * x64 (AMD64 / x64 platform specific settings) +The property files used are: + * python (versions, directories and build names) + * pyproject (base settings for all projects) + * openssl (used by libeay and ssleay projects) + * tcltk (used by _tkinter, tcl, tk and tix projects) -The pyproject propertyfile defines _WIN32 and x64 defines _WIN64 and _M_X64 -although the macros are set by the compiler, too. The GUI doesn't always know -about the macros and confuse the user with false information. +The pyproject property file defines all of the build settings for each +project, with some projects overriding certain specific values. The GUI +doesn't always reflect the correct settings and may confuse the user +with false information, especially for settings that automatically adapt +for diffirent configurations. -YOUR OWN EXTENSION DLLs + +Your Own Extension DLLs ----------------------- -If you want to create your own extension module DLL, there's an example -with easy-to-follow instructions in ../PC/example/; read the file -readme.txt there first. +If you want to create your own extension module DLL (.pyd), there's an +example with easy-to-follow instructions in ..\PC\example_nt\; read the +file readme.txt there first. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 8 09:01:20 2015 From: python-checkins at python.org (serhiy.storchaka) Date: Tue, 08 Sep 2015 07:01:20 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_Fixed_tests_fo?= =?utf-8?q?r_shutil=2Emake=5Farchive=28=29_with_relative_base=5Fname_in_th?= =?utf-8?q?e_case_when?= Message-ID: <20150908070120.11268.28654@psf.io> https://hg.python.org/cpython/rev/ab1dc73fe8b5 changeset: 97776:ab1dc73fe8b5 branch: 2.7 parent: 97772:1c5a171e7a13 user: Serhiy Storchaka date: Tue Sep 08 09:59:02 2015 +0300 summary: Fixed tests for shutil.make_archive() with relative base_name in the case when the path of the directory for temporary files contains symlinks. files: Lib/test/test_shutil.py | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Lib/test/test_shutil.py b/Lib/test/test_shutil.py --- a/Lib/test/test_shutil.py +++ b/Lib/test/test_shutil.py @@ -382,9 +382,9 @@ # working with relative paths work_dir = os.path.dirname(tmpdir2) rel_base_name = os.path.join(os.path.basename(tmpdir2), 'archive') - base_name = os.path.join(work_dir, rel_base_name) with support.change_cwd(work_dir): + base_name = os.path.abspath(rel_base_name) tarball = make_archive(rel_base_name, 'gztar', root_dir, '.') # check if the compressed tarball was created @@ -473,9 +473,9 @@ # working with relative paths work_dir = os.path.dirname(tmpdir2) rel_base_name = os.path.join(os.path.basename(tmpdir2), 'archive') - base_name = os.path.join(work_dir, rel_base_name) with support.change_cwd(work_dir): + base_name = os.path.abspath(rel_base_name) res = make_archive(rel_base_name, 'zip', root_dir, base_dir) self.assertEqual(res, base_name + '.zip') -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 8 09:01:20 2015 From: python-checkins at python.org (serhiy.storchaka) Date: Tue, 08 Sep 2015 07:01:20 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_Fixed_tests_fo?= =?utf-8?q?r_shutil=2Emake=5Farchive=28=29_with_relative_base=5Fname_in_th?= =?utf-8?q?e_case_when?= Message-ID: <20150908070120.14869.7572@psf.io> https://hg.python.org/cpython/rev/bbf72b164720 changeset: 97775:bbf72b164720 branch: 3.4 parent: 97769:fc71979b80a5 user: Serhiy Storchaka date: Tue Sep 08 09:59:02 2015 +0300 summary: Fixed tests for shutil.make_archive() with relative base_name in the case when the path of the directory for temporary files contains symlinks. files: Lib/test/test_shutil.py | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Lib/test/test_shutil.py b/Lib/test/test_shutil.py --- a/Lib/test/test_shutil.py +++ b/Lib/test/test_shutil.py @@ -971,9 +971,9 @@ # working with relative paths work_dir = os.path.dirname(tmpdir2) rel_base_name = os.path.join(os.path.basename(tmpdir2), 'archive') - base_name = os.path.join(work_dir, rel_base_name) with support.change_cwd(work_dir): + base_name = os.path.abspath(rel_base_name) tarball = make_archive(rel_base_name, 'gztar', root_dir, '.') # check if the compressed tarball was created @@ -1061,9 +1061,9 @@ # working with relative paths work_dir = os.path.dirname(tmpdir2) rel_base_name = os.path.join(os.path.basename(tmpdir2), 'archive') - base_name = os.path.join(work_dir, rel_base_name) with support.change_cwd(work_dir): + base_name = os.path.abspath(rel_base_name) res = make_archive(rel_base_name, 'zip', root_dir, base_dir) self.assertEqual(res, base_name + '.zip') -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 8 09:01:20 2015 From: python-checkins at python.org (serhiy.storchaka) Date: Tue, 08 Sep 2015 07:01:20 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Fixed_tests_for_shutil=2Emake=5Farchive=28=29_with_relat?= =?utf-8?q?ive_base=5Fname_in_the_case_when?= Message-ID: <20150908070120.11258.87764@psf.io> https://hg.python.org/cpython/rev/3c0c153d6b02 changeset: 97778:3c0c153d6b02 parent: 97774:73258658dc76 parent: 97777:7f380e3a8de9 user: Serhiy Storchaka date: Tue Sep 08 10:00:43 2015 +0300 summary: Fixed tests for shutil.make_archive() with relative base_name in the case when the path of the directory for temporary files contains symlinks. files: Lib/test/test_shutil.py | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Lib/test/test_shutil.py b/Lib/test/test_shutil.py --- a/Lib/test/test_shutil.py +++ b/Lib/test/test_shutil.py @@ -977,9 +977,9 @@ # working with relative paths work_dir = os.path.dirname(tmpdir2) rel_base_name = os.path.join(os.path.basename(tmpdir2), 'archive') - base_name = os.path.join(work_dir, rel_base_name) with support.change_cwd(work_dir): + base_name = os.path.abspath(rel_base_name) tarball = make_archive(rel_base_name, 'gztar', root_dir, '.') # check if the compressed tarball was created @@ -1067,9 +1067,9 @@ # working with relative paths work_dir = os.path.dirname(tmpdir2) rel_base_name = os.path.join(os.path.basename(tmpdir2), 'archive') - base_name = os.path.join(work_dir, rel_base_name) with support.change_cwd(work_dir): + base_name = os.path.abspath(rel_base_name) res = make_archive(rel_base_name, 'zip', root_dir, base_dir) self.assertEqual(res, base_name + '.zip') -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 8 09:01:21 2015 From: python-checkins at python.org (serhiy.storchaka) Date: Tue, 08 Sep 2015 07:01:21 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Fixed_tests_for_shutil=2Emake=5Farchive=28=29_with_relative_ba?= =?utf-8?q?se=5Fname_in_the_case_when?= Message-ID: <20150908070120.101478.10984@psf.io> https://hg.python.org/cpython/rev/7f380e3a8de9 changeset: 97777:7f380e3a8de9 branch: 3.5 parent: 97773:e4a2089ad684 parent: 97775:bbf72b164720 user: Serhiy Storchaka date: Tue Sep 08 10:00:22 2015 +0300 summary: Fixed tests for shutil.make_archive() with relative base_name in the case when the path of the directory for temporary files contains symlinks. files: Lib/test/test_shutil.py | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Lib/test/test_shutil.py b/Lib/test/test_shutil.py --- a/Lib/test/test_shutil.py +++ b/Lib/test/test_shutil.py @@ -977,9 +977,9 @@ # working with relative paths work_dir = os.path.dirname(tmpdir2) rel_base_name = os.path.join(os.path.basename(tmpdir2), 'archive') - base_name = os.path.join(work_dir, rel_base_name) with support.change_cwd(work_dir): + base_name = os.path.abspath(rel_base_name) tarball = make_archive(rel_base_name, 'gztar', root_dir, '.') # check if the compressed tarball was created @@ -1067,9 +1067,9 @@ # working with relative paths work_dir = os.path.dirname(tmpdir2) rel_base_name = os.path.join(os.path.basename(tmpdir2), 'archive') - base_name = os.path.join(work_dir, rel_base_name) with support.change_cwd(work_dir): + base_name = os.path.abspath(rel_base_name) res = make_archive(rel_base_name, 'zip', root_dir, base_dir) self.assertEqual(res, base_name + '.zip') -- Repository URL: https://hg.python.org/cpython From lp_benchmark_robot at intel.com Tue Sep 8 10:22:18 2015 From: lp_benchmark_robot at intel.com (lp_benchmark_robot) Date: Tue, 8 Sep 2015 08:22:18 +0000 Subject: [Python-checkins] Benchmark Results for Python Default 2015-09-08 Message-ID: <078AA0FFE8C7034097F90205717F504611D79CFA@IRSMSX102.ger.corp.intel.com> Results for project python_default-nightly, build date 2015-09-08 06:02:30 commit: 6907716e7ccb45525d75194374b2f390f12e0879 revision date: 2015-09-08 05:53:42 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v3.4.3, with hash b4cbecbc0781e89a309d03b60a1f75f8499250e6 from 2015-02-25 12:15:33+00:00 ------------------------------------------------------------------------------------------ benchmark relative change since change since current rev with std_dev* last run v3.4.3 regrtest PGO ------------------------------------------------------------------------------------------ :-) django_v2 0.28729% 0.05625% 9.85579% 15.88681% :-( pybench 0.11335% 0.08198% -2.07477% 9.30519% :-( regex_v8 8.24309% -2.37589% -8.33569% 9.63855% :-( nbody 0.17013% -0.72434% -8.44804% 13.78609% :-| json_dump_v2 0.23322% 0.42020% -1.80950% 13.19999% :-| normal_startup 0.86975% 0.08042% 0.12219% 5.29366% ------------------------------------------------------------------------------------------ Note: Benchmark results are measured in seconds. * Relative Standard Deviation (Standard Deviation/Average) Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. No license (express or implied, by estoppel or otherwise) to any intellectual property rights is granted by this document. Intel disclaims all express and implied warranties, including without limitation, the implied warranties of merchantability, fitness for a particular purpose, and non-infringement, as well as any warranty arising from course of performance, course of dealing, or usage in trade. This document may contain information on products, services and/or processes in development. Contact your Intel representative to obtain the latest forecast, schedule, specifications and roadmaps. The products and services described may contain defects or errors known as errata which may cause deviations from published specifications. Current characterized errata are available on request. (C) 2015 Intel Corporation. From python-checkins at python.org Tue Sep 8 21:33:56 2015 From: python-checkins at python.org (yury.selivanov) Date: Tue, 08 Sep 2015 19:33:56 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogTWVyZ2UgMy41?= Message-ID: <20150908193356.17961.1744@psf.io> https://hg.python.org/cpython/rev/8167951e1569 changeset: 97780:8167951e1569 parent: 97778:3c0c153d6b02 parent: 97779:60f5ca4c5a01 user: Yury Selivanov date: Tue Sep 08 15:33:33 2015 -0400 summary: Merge 3.5 files: Doc/whatsnew/3.5.rst | 268 +++++++++++++++++++++--------- 1 files changed, 188 insertions(+), 80 deletions(-) diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -5,6 +5,8 @@ :Release: |release| :Date: |today| +:Author: Elvis Pranskevichus (Editor) + .. Rules for maintenance: * Anyone can add text to this document. Do not spend very much time @@ -149,31 +151,183 @@ PEP written by Carl Meyer +New Features +============ + +.. _whatsnew-pep-492: + PEP 492 - Coroutines with async and await syntax ------------------------------------------------ -The PEP added dedicated syntax for declaring :term:`coroutines `, -:keyword:`await` expressions, new asynchronous :keyword:`async for` -and :keyword:`async with` statements. +:pep:`492` greatly improves support for asynchronous programming in Python +by adding :term:`awaitable objects `, +:term:`coroutine functions `, +:term:`asynchronous iteration `, +and :term:`asynchronous context managers `. -Example:: +Coroutine functions are declared using the new :keyword:`async def` syntax:: - async def read_data(db): - async with db.transaction(): - data = await db.fetch('SELECT ...') + >>> async def coro(): + ... return 'spam' -PEP written and implemented by Yury Selivanov. +Inside a coroutine function, a new :keyword:`await` expression can be used +to suspend coroutine execution until the result is available. Any object +can be *awaited*, as long as it implements the :term:`awaitable` protocol by +defining the :meth:`__await__` method. + +PEP 492 also adds :keyword:`async for` statement for convenient iteration +over asynchronous iterables. + +An example of a simple HTTP client written using the new syntax:: + + import asyncio + + async def http_get(domain): + reader, writer = await asyncio.open_connection(domain, 80) + + writer.write(b'\r\n'.join([ + b'GET / HTTP/1.1', + b'Host: %b' % domain.encode('latin-1'), + b'Connection: close', + b'', b'' + ])) + + async for line in reader: + print('>>>', line) + + writer.close() + + loop = asyncio.get_event_loop() + try: + loop.run_until_complete(http_get('example.com')) + finally: + loop.close() + + +Similarly to asynchronous iteration, there is a new syntax for asynchronous +context managers:: + + >>> import asyncio + >>> async def coro1(lock): + ... print('coro1: waiting for lock') + ... async with lock: + ... print('coro1: holding the lock') + ... await asyncio.sleep(1) + ... print('coro1: releasing the lock') + ... + >>> async def coro2(lock): + ... print('coro2: waiting for lock') + ... async with lock: + ... print('coro2: holding the lock') + ... await asyncio.sleep(1) + ... print('coro2: releasing the lock') + ... + >>> loop = asyncio.get_event_loop() + >>> lock = asyncio.Lock() + >>> coros = asyncio.gather(coro1(lock), coro2(lock), loop=loop) + >>> loop.run_until_complete(coros) + coro1: waiting for lock + coro1: holding the lock + coro2: waiting for lock + coro1: releasing the lock + coro2: holding the lock + coro2: releasing the lock + >>> loop.close() + +Note that both :keyword:`async for` and :keyword:`async with` can only +be used inside a coroutine function declared with :keyword:`async def`. + +Coroutine functions are intended to be ran inside a compatible event loop, +such as :class:`asyncio.Loop`. .. seealso:: :pep:`492` -- Coroutines with async and await syntax + PEP written and implemented by Yury Selivanov. -PEP 461 - Formatting support for bytes and bytearray ----------------------------------------------------- +PEP 465 - A dedicated infix operator for matrix multiplication +-------------------------------------------------------------- -This PEP proposes adding % formatting operations similar to Python 2's ``str`` -type to :class:`bytes` and :class:`bytearray`. +:pep:`465` adds the ``@`` infix operator for matrix multiplication. +Currently, no builtin Python types implement the new operator, however, it +can be implemented by defining :meth:`__matmul__`, :meth:`__rmatmul__`, +and :meth:`__imatmul__` for regular, reflected, and in-place matrix +multiplication. The semantics of these methods is similar to that of +methods defining other infix arithmetic operators. + +Matrix multiplication is a notably common operation in many fields of +mathematics, science, engineering, and the addition of ``@`` allows writing +cleaner code:: + + >>> S = (H @ beta - r).T @ inv(H @ V @ H.T) @ (H @ beta - r) + +An upcoming release of NumPy 1.10 will add support for the new operator:: + + >>> import numpy + + >>> x = numpy.ones(3) + >>> x + array([ 1., 1., 1.]) + + >>> m = numpy.eye(3) + >>> m + array([[ 1., 0., 0.], + [ 0., 1., 0.], + [ 0., 0., 1.]]) + + >>> x @ m + array([ 1., 1., 1.]) + + +.. seealso:: + + :pep:`465` -- A dedicated infix operator for matrix multiplication + PEP written by Nathaniel J. Smith; implemented by Benjamin Peterson. + + +PEP 448 - Additional Unpacking Generalizations +---------------------------------------------- + +:pep:`448` extends the allowed uses of the ``*`` iterable unpacking +operator and ``**`` dictionary unpacking operator. It is now possible +to use an arbitrary number of unpackings in function calls:: + + >>> print(*[1], *[2], 3, *[4, 5]) + 1 2 3 4 5 + + >>> def fn(a, b, c, d): + ... print(a, b, c, d) + ... + + >>> fn(**{'a': 1, 'c': 3}, **{'b': 2, 'd': 4}) + 1 2 3 4 + +Similarly, tuple, list, set, and dictionary displays allow multiple +unpackings:: + + >>> *range(4), 4 + (0, 1, 2, 3, 4) + >>> [*range(4), 4] + [0, 1, 2, 3, 4] + >>> {*range(4), 4, *(5, 6, 7)} + {0, 1, 2, 3, 4, 5, 6, 7} + >>> {'x': 1, **{'y': 2}} + {'x': 1, 'y': 2} + +.. seealso:: + + :pep:`448` -- Additional Unpacking Generalizations + PEP written by Joshua Landau; implemented by Neil Girdhar, + Thomas Wouters, and Joshua Landau. + + +PEP 461 - % formatting support for bytes and bytearray +------------------------------------------------------ + +PEP 461 adds % formatting to :class:`bytes` and :class:`bytearray`, aiding in +handling data that is a mixture of binary and ASCII compatible text. This +feature also eases porting such code from Python 2. Examples:: @@ -195,60 +349,8 @@ .. seealso:: :pep:`461` -- Adding % formatting to bytes and bytearray - - -PEP 465 - A dedicated infix operator for matrix multiplication --------------------------------------------------------------- - -This PEP proposes a new binary operator to be used for matrix multiplication, -called ``@``. (Mnemonic: ``@`` is ``*`` for mATrices.) - -.. seealso:: - - :pep:`465` -- A dedicated infix operator for matrix multiplication - - -PEP 448 - Additional Unpacking Generalizations ----------------------------------------------- - -This PEP proposes extended usages of the ``*`` iterable unpacking -operator and ``**`` dictionary unpacking operators -to allow unpacking in more positions, an arbitrary number of -times, and in additional circumstances. Specifically, -in function calls, in comprehensions and generator expressions, and -in displays. - -Function calls are proposed to support an arbitrary number of -unpackings rather than just one:: - - >>> print(*[1], *[2], 3) - 1 2 3 - >>> dict(**{'x': 1}, y=2, **{'z': 3}) - {'x': 1, 'y': 2, 'z': 3} - -Unpacking is proposed to be allowed inside tuple, list, set, -and dictionary displays:: - - >>> *range(4), 4 - (0, 1, 2, 3, 4) - >>> [*range(4), 4] - [0, 1, 2, 3, 4] - >>> {*range(4), 4} - {0, 1, 2, 3, 4} - >>> {'x': 1, **{'y': 2}} - {'x': 1, 'y': 2} - -In dictionaries, later values will always override earlier ones:: - - >>> {'x': 1, **{'x': 2}} - {'x': 2} - - >>> {**{'x': 2}, 'x': 1} - {'x': 1} - -.. seealso:: - - :pep:`448` -- Additional Unpacking Generalizations + PEP written by Ethan Furman; implemented by Neil Schemenauer and + Ethan Furman. PEP 484 - Type Hints @@ -261,8 +363,8 @@ For example, here is a simple function whose argument and return type are declared in the annotations:: - def greeting(name: str) -> str: - return 'Hello ' + name + def greeting(name: str) -> str: + return 'Hello ' + name The type system supports unions, generic types, and a special type named ``Any`` which is consistent with (i.e. assignable to and from) all @@ -270,8 +372,10 @@ .. seealso:: + * :mod:`typing` module documentation * :pep:`484` -- Type Hints - * :mod:`typing` module documentation + PEP written by Guido van Rossum, Jukka Lehtosalo, and ?ukasz Langa; + implemented by Guido van Rossum. PEP 471 - os.scandir() function -- a better and faster directory iterator @@ -282,12 +386,10 @@ implemented using :func:`os.scandir`, which speeds it up by 3-5 times on POSIX systems and by 7-20 times on Windows systems. -PEP and implementation written by Ben Hoyt with the help of Victor Stinner. - .. seealso:: - :pep:`471` -- os.scandir() function -- a better and faster directory - iterator + :pep:`471` -- os.scandir() function -- a better and faster directory iterator + PEP written and implemented by Ben Hoyt with the help of Victor Stinner. PEP 475: Retry system calls failing with EINTR @@ -357,12 +459,11 @@ * :func:`signal.sigtimedwait`, :func:`signal.sigwaitinfo` * :func:`time.sleep` -PEP and implementation written by Charles-Fran?ois Natali and Victor Stinner, -with the help of Antoine Pitrou (the french connection). - .. seealso:: :pep:`475` -- Retry system calls failing with EINTR + PEP and implementation written by Charles-Fran?ois Natali and + Victor Stinner, with the help of Antoine Pitrou (the french connection). PEP 479: Change StopIteration handling inside generators @@ -378,12 +479,11 @@ Without a ``__future__`` import, a :exc:`PendingDeprecationWarning` will be raised. -PEP written by Chris Angelico and Guido van Rossum. Implemented by -Chris Angelico, Yury Selivanov and Nick Coghlan. - .. seealso:: :pep:`479` -- Change StopIteration handling inside generators + PEP written by Chris Angelico and Guido van Rossum. Implemented by + Chris Angelico, Yury Selivanov and Nick Coghlan. PEP 486: Make the Python Launcher aware of virtual environments @@ -397,6 +497,7 @@ .. seealso:: :pep:`486` -- Make the Python Launcher aware of virtual environments + PEP written and implemented by Paul Moore. PEP 488: Elimination of PYO files @@ -414,6 +515,7 @@ .. seealso:: :pep:`488` -- Elimination of PYO files + PEP written and implemented by Brett Cannon. PEP 489: Multi-phase extension module initialization @@ -429,7 +531,10 @@ .. seealso:: - :pep:`488` -- Multi-phase extension module initialization + :pep:`489` -- Multi-phase extension module initialization + PEP written by Petr Viktorin, Stefan Behnel, and Nick Coghlan; + implementation by Petr Viktorin. + PEP 485: A function for testing approximate equality ---------------------------------------------------- @@ -442,6 +547,9 @@ .. seealso:: :pep:`485` -- A function for testing approximate equality + PEP written by Christopher Barker; implemented by Chris Barker and + Tal Einat. + Other Language Changes ====================== -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 8 21:34:03 2015 From: python-checkins at python.org (yury.selivanov) Date: Tue, 08 Sep 2015 19:34:03 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_docs/whatsnew/?= =?utf-8?q?3=2E5=3A_Update_peps_section?= Message-ID: <20150908193355.66862.76587@psf.io> https://hg.python.org/cpython/rev/60f5ca4c5a01 changeset: 97779:60f5ca4c5a01 branch: 3.5 parent: 97777:7f380e3a8de9 user: Yury Selivanov date: Tue Sep 08 15:33:15 2015 -0400 summary: docs/whatsnew/3.5: Update peps section Patch by Elvis Pranskevichus. files: Doc/whatsnew/3.5.rst | 268 +++++++++++++++++++++--------- 1 files changed, 188 insertions(+), 80 deletions(-) diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -5,6 +5,8 @@ :Release: |release| :Date: |today| +:Author: Elvis Pranskevichus (Editor) + .. Rules for maintenance: * Anyone can add text to this document. Do not spend very much time @@ -149,31 +151,183 @@ PEP written by Carl Meyer +New Features +============ + +.. _whatsnew-pep-492: + PEP 492 - Coroutines with async and await syntax ------------------------------------------------ -The PEP added dedicated syntax for declaring :term:`coroutines `, -:keyword:`await` expressions, new asynchronous :keyword:`async for` -and :keyword:`async with` statements. +:pep:`492` greatly improves support for asynchronous programming in Python +by adding :term:`awaitable objects `, +:term:`coroutine functions `, +:term:`asynchronous iteration `, +and :term:`asynchronous context managers `. -Example:: +Coroutine functions are declared using the new :keyword:`async def` syntax:: - async def read_data(db): - async with db.transaction(): - data = await db.fetch('SELECT ...') + >>> async def coro(): + ... return 'spam' -PEP written and implemented by Yury Selivanov. +Inside a coroutine function, a new :keyword:`await` expression can be used +to suspend coroutine execution until the result is available. Any object +can be *awaited*, as long as it implements the :term:`awaitable` protocol by +defining the :meth:`__await__` method. + +PEP 492 also adds :keyword:`async for` statement for convenient iteration +over asynchronous iterables. + +An example of a simple HTTP client written using the new syntax:: + + import asyncio + + async def http_get(domain): + reader, writer = await asyncio.open_connection(domain, 80) + + writer.write(b'\r\n'.join([ + b'GET / HTTP/1.1', + b'Host: %b' % domain.encode('latin-1'), + b'Connection: close', + b'', b'' + ])) + + async for line in reader: + print('>>>', line) + + writer.close() + + loop = asyncio.get_event_loop() + try: + loop.run_until_complete(http_get('example.com')) + finally: + loop.close() + + +Similarly to asynchronous iteration, there is a new syntax for asynchronous +context managers:: + + >>> import asyncio + >>> async def coro1(lock): + ... print('coro1: waiting for lock') + ... async with lock: + ... print('coro1: holding the lock') + ... await asyncio.sleep(1) + ... print('coro1: releasing the lock') + ... + >>> async def coro2(lock): + ... print('coro2: waiting for lock') + ... async with lock: + ... print('coro2: holding the lock') + ... await asyncio.sleep(1) + ... print('coro2: releasing the lock') + ... + >>> loop = asyncio.get_event_loop() + >>> lock = asyncio.Lock() + >>> coros = asyncio.gather(coro1(lock), coro2(lock), loop=loop) + >>> loop.run_until_complete(coros) + coro1: waiting for lock + coro1: holding the lock + coro2: waiting for lock + coro1: releasing the lock + coro2: holding the lock + coro2: releasing the lock + >>> loop.close() + +Note that both :keyword:`async for` and :keyword:`async with` can only +be used inside a coroutine function declared with :keyword:`async def`. + +Coroutine functions are intended to be ran inside a compatible event loop, +such as :class:`asyncio.Loop`. .. seealso:: :pep:`492` -- Coroutines with async and await syntax + PEP written and implemented by Yury Selivanov. -PEP 461 - Formatting support for bytes and bytearray ----------------------------------------------------- +PEP 465 - A dedicated infix operator for matrix multiplication +-------------------------------------------------------------- -This PEP proposes adding % formatting operations similar to Python 2's ``str`` -type to :class:`bytes` and :class:`bytearray`. +:pep:`465` adds the ``@`` infix operator for matrix multiplication. +Currently, no builtin Python types implement the new operator, however, it +can be implemented by defining :meth:`__matmul__`, :meth:`__rmatmul__`, +and :meth:`__imatmul__` for regular, reflected, and in-place matrix +multiplication. The semantics of these methods is similar to that of +methods defining other infix arithmetic operators. + +Matrix multiplication is a notably common operation in many fields of +mathematics, science, engineering, and the addition of ``@`` allows writing +cleaner code:: + + >>> S = (H @ beta - r).T @ inv(H @ V @ H.T) @ (H @ beta - r) + +An upcoming release of NumPy 1.10 will add support for the new operator:: + + >>> import numpy + + >>> x = numpy.ones(3) + >>> x + array([ 1., 1., 1.]) + + >>> m = numpy.eye(3) + >>> m + array([[ 1., 0., 0.], + [ 0., 1., 0.], + [ 0., 0., 1.]]) + + >>> x @ m + array([ 1., 1., 1.]) + + +.. seealso:: + + :pep:`465` -- A dedicated infix operator for matrix multiplication + PEP written by Nathaniel J. Smith; implemented by Benjamin Peterson. + + +PEP 448 - Additional Unpacking Generalizations +---------------------------------------------- + +:pep:`448` extends the allowed uses of the ``*`` iterable unpacking +operator and ``**`` dictionary unpacking operator. It is now possible +to use an arbitrary number of unpackings in function calls:: + + >>> print(*[1], *[2], 3, *[4, 5]) + 1 2 3 4 5 + + >>> def fn(a, b, c, d): + ... print(a, b, c, d) + ... + + >>> fn(**{'a': 1, 'c': 3}, **{'b': 2, 'd': 4}) + 1 2 3 4 + +Similarly, tuple, list, set, and dictionary displays allow multiple +unpackings:: + + >>> *range(4), 4 + (0, 1, 2, 3, 4) + >>> [*range(4), 4] + [0, 1, 2, 3, 4] + >>> {*range(4), 4, *(5, 6, 7)} + {0, 1, 2, 3, 4, 5, 6, 7} + >>> {'x': 1, **{'y': 2}} + {'x': 1, 'y': 2} + +.. seealso:: + + :pep:`448` -- Additional Unpacking Generalizations + PEP written by Joshua Landau; implemented by Neil Girdhar, + Thomas Wouters, and Joshua Landau. + + +PEP 461 - % formatting support for bytes and bytearray +------------------------------------------------------ + +PEP 461 adds % formatting to :class:`bytes` and :class:`bytearray`, aiding in +handling data that is a mixture of binary and ASCII compatible text. This +feature also eases porting such code from Python 2. Examples:: @@ -195,60 +349,8 @@ .. seealso:: :pep:`461` -- Adding % formatting to bytes and bytearray - - -PEP 465 - A dedicated infix operator for matrix multiplication --------------------------------------------------------------- - -This PEP proposes a new binary operator to be used for matrix multiplication, -called ``@``. (Mnemonic: ``@`` is ``*`` for mATrices.) - -.. seealso:: - - :pep:`465` -- A dedicated infix operator for matrix multiplication - - -PEP 448 - Additional Unpacking Generalizations ----------------------------------------------- - -This PEP proposes extended usages of the ``*`` iterable unpacking -operator and ``**`` dictionary unpacking operators -to allow unpacking in more positions, an arbitrary number of -times, and in additional circumstances. Specifically, -in function calls, in comprehensions and generator expressions, and -in displays. - -Function calls are proposed to support an arbitrary number of -unpackings rather than just one:: - - >>> print(*[1], *[2], 3) - 1 2 3 - >>> dict(**{'x': 1}, y=2, **{'z': 3}) - {'x': 1, 'y': 2, 'z': 3} - -Unpacking is proposed to be allowed inside tuple, list, set, -and dictionary displays:: - - >>> *range(4), 4 - (0, 1, 2, 3, 4) - >>> [*range(4), 4] - [0, 1, 2, 3, 4] - >>> {*range(4), 4} - {0, 1, 2, 3, 4} - >>> {'x': 1, **{'y': 2}} - {'x': 1, 'y': 2} - -In dictionaries, later values will always override earlier ones:: - - >>> {'x': 1, **{'x': 2}} - {'x': 2} - - >>> {**{'x': 2}, 'x': 1} - {'x': 1} - -.. seealso:: - - :pep:`448` -- Additional Unpacking Generalizations + PEP written by Ethan Furman; implemented by Neil Schemenauer and + Ethan Furman. PEP 484 - Type Hints @@ -261,8 +363,8 @@ For example, here is a simple function whose argument and return type are declared in the annotations:: - def greeting(name: str) -> str: - return 'Hello ' + name + def greeting(name: str) -> str: + return 'Hello ' + name The type system supports unions, generic types, and a special type named ``Any`` which is consistent with (i.e. assignable to and from) all @@ -270,8 +372,10 @@ .. seealso:: + * :mod:`typing` module documentation * :pep:`484` -- Type Hints - * :mod:`typing` module documentation + PEP written by Guido van Rossum, Jukka Lehtosalo, and ?ukasz Langa; + implemented by Guido van Rossum. PEP 471 - os.scandir() function -- a better and faster directory iterator @@ -282,12 +386,10 @@ implemented using :func:`os.scandir`, which speeds it up by 3-5 times on POSIX systems and by 7-20 times on Windows systems. -PEP and implementation written by Ben Hoyt with the help of Victor Stinner. - .. seealso:: - :pep:`471` -- os.scandir() function -- a better and faster directory - iterator + :pep:`471` -- os.scandir() function -- a better and faster directory iterator + PEP written and implemented by Ben Hoyt with the help of Victor Stinner. PEP 475: Retry system calls failing with EINTR @@ -357,12 +459,11 @@ * :func:`signal.sigtimedwait`, :func:`signal.sigwaitinfo` * :func:`time.sleep` -PEP and implementation written by Charles-Fran?ois Natali and Victor Stinner, -with the help of Antoine Pitrou (the french connection). - .. seealso:: :pep:`475` -- Retry system calls failing with EINTR + PEP and implementation written by Charles-Fran?ois Natali and + Victor Stinner, with the help of Antoine Pitrou (the french connection). PEP 479: Change StopIteration handling inside generators @@ -378,12 +479,11 @@ Without a ``__future__`` import, a :exc:`PendingDeprecationWarning` will be raised. -PEP written by Chris Angelico and Guido van Rossum. Implemented by -Chris Angelico, Yury Selivanov and Nick Coghlan. - .. seealso:: :pep:`479` -- Change StopIteration handling inside generators + PEP written by Chris Angelico and Guido van Rossum. Implemented by + Chris Angelico, Yury Selivanov and Nick Coghlan. PEP 486: Make the Python Launcher aware of virtual environments @@ -397,6 +497,7 @@ .. seealso:: :pep:`486` -- Make the Python Launcher aware of virtual environments + PEP written and implemented by Paul Moore. PEP 488: Elimination of PYO files @@ -414,6 +515,7 @@ .. seealso:: :pep:`488` -- Elimination of PYO files + PEP written and implemented by Brett Cannon. PEP 489: Multi-phase extension module initialization @@ -429,7 +531,10 @@ .. seealso:: - :pep:`488` -- Multi-phase extension module initialization + :pep:`489` -- Multi-phase extension module initialization + PEP written by Petr Viktorin, Stefan Behnel, and Nick Coghlan; + implementation by Petr Viktorin. + PEP 485: A function for testing approximate equality ---------------------------------------------------- @@ -442,6 +547,9 @@ .. seealso:: :pep:`485` -- A function for testing approximate equality + PEP written by Christopher Barker; implemented by Chris Barker and + Tal Einat. + Other Language Changes ====================== -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 8 23:59:20 2015 From: python-checkins at python.org (victor.stinner) Date: Tue, 08 Sep 2015 21:59:20 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Revert_change_0eb8c182131e?= =?utf-8?q?=3A?= Message-ID: <20150908215920.11260.45400@psf.io> https://hg.python.org/cpython/rev/d0bb896f9b14 changeset: 97781:d0bb896f9b14 user: Victor Stinner date: Tue Sep 08 23:58:54 2015 +0200 summary: Revert change 0eb8c182131e: """Issue #23517: datetime.timedelta constructor now rounds microseconds to nearest with ties going away from zero (ROUND_HALF_UP), as Python 2 and Python older than 3.3, instead of rounding to nearest with ties going to nearest even integer (ROUND_HALF_EVEN).""" datetime.timedelta uses rounding mode ROUND_HALF_EVEN again. files: Lib/datetime.py | 4 ++-- Lib/test/datetimetester.py | 18 ++++++++++++------ Misc/NEWS | 7 +------ Modules/_datetimemodule.c | 22 +++++++++++++++++++++- 4 files changed, 36 insertions(+), 15 deletions(-) diff --git a/Lib/datetime.py b/Lib/datetime.py --- a/Lib/datetime.py +++ b/Lib/datetime.py @@ -407,7 +407,7 @@ # secondsfrac isn't referenced again if isinstance(microseconds, float): - microseconds = _round_half_up(microseconds + usdouble) + microseconds = round(microseconds + usdouble) seconds, microseconds = divmod(microseconds, 1000000) days, seconds = divmod(seconds, 24*3600) d += days @@ -418,7 +418,7 @@ days, seconds = divmod(seconds, 24*3600) d += days s += seconds - microseconds = _round_half_up(microseconds + usdouble) + microseconds = round(microseconds + usdouble) assert isinstance(s, int) assert isinstance(microseconds, int) assert abs(s) <= 3 * 24 * 3600 diff --git a/Lib/test/datetimetester.py b/Lib/test/datetimetester.py --- a/Lib/test/datetimetester.py +++ b/Lib/test/datetimetester.py @@ -663,14 +663,16 @@ # Single-field rounding. eq(td(milliseconds=0.4/1000), td(0)) # rounds to 0 eq(td(milliseconds=-0.4/1000), td(0)) # rounds to 0 - eq(td(milliseconds=0.5/1000), td(microseconds=1)) - eq(td(milliseconds=-0.5/1000), td(microseconds=-1)) + eq(td(milliseconds=0.5/1000), td(microseconds=0)) + eq(td(milliseconds=-0.5/1000), td(microseconds=-0)) eq(td(milliseconds=0.6/1000), td(microseconds=1)) eq(td(milliseconds=-0.6/1000), td(microseconds=-1)) - eq(td(seconds=0.5/10**6), td(microseconds=1)) - eq(td(seconds=-0.5/10**6), td(microseconds=-1)) - eq(td(seconds=1/2**7), td(microseconds=7813)) - eq(td(seconds=-1/2**7), td(microseconds=-7813)) + eq(td(milliseconds=1.5/1000), td(microseconds=2)) + eq(td(milliseconds=-1.5/1000), td(microseconds=-2)) + eq(td(seconds=0.5/10**6), td(microseconds=0)) + eq(td(seconds=-0.5/10**6), td(microseconds=-0)) + eq(td(seconds=1/2**7), td(microseconds=7812)) + eq(td(seconds=-1/2**7), td(microseconds=-7812)) # Rounding due to contributions from more than one field. us_per_hour = 3600e6 @@ -683,6 +685,10 @@ eq(td(hours=-.2/us_per_hour), td(0)) eq(td(days=-.4/us_per_day, hours=-.2/us_per_hour), td(microseconds=-1)) + # Test for a patch in Issue 8860 + eq(td(microseconds=0.5), 0.5*td(microseconds=1.0)) + eq(td(microseconds=0.5)//td.resolution, 0.5*td.resolution//td.resolution) + def test_massive_normalization(self): td = timedelta(microseconds=-1) self.assertEqual((td.days, td.seconds, td.microseconds), diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -17,18 +17,13 @@ Library ------- -- Issue #22241: timezone.utc name is now plain 'UTC', not 'UTC-00:00'. +- Issue #22241: timezone.utc name is now plain 'UTC', not 'UTC-00:00'. - Issue #23517: fromtimestamp() and utcfromtimestamp() methods of datetime.datetime now round microseconds to nearest with ties going away from zero (ROUND_HALF_UP), as Python 2 and Python older than 3.3, instead of rounding towards -Infinity (ROUND_FLOOR). -- Issue #23517: datetime.timedelta constructor now rounds microseconds to - nearest with ties going away from zero (ROUND_HALF_UP), as Python 2 and - Python older than 3.3, instead of rounding to nearest with ties going to - nearest even integer (ROUND_HALF_EVEN). - - Issue #23552: Timeit now warns when there is substantial (4x) variance between best and worst times. Patch from Serhiy Storchaka. diff --git a/Modules/_datetimemodule.c b/Modules/_datetimemodule.c --- a/Modules/_datetimemodule.c +++ b/Modules/_datetimemodule.c @@ -2149,9 +2149,29 @@ if (leftover_us) { /* Round to nearest whole # of us, and add into x. */ double whole_us = round(leftover_us); + int x_is_odd; PyObject *temp; - whole_us = _PyTime_RoundHalfUp(leftover_us); + whole_us = round(leftover_us); + if (fabs(whole_us - leftover_us) == 0.5) { + /* We're exactly halfway between two integers. In order + * to do round-half-to-even, we must determine whether x + * is odd. Note that x is odd when it's last bit is 1. The + * code below uses bitwise and operation to check the last + * bit. */ + temp = PyNumber_And(x, one); /* temp <- x & 1 */ + if (temp == NULL) { + Py_DECREF(x); + goto Done; + } + x_is_odd = PyObject_IsTrue(temp); + Py_DECREF(temp); + if (x_is_odd == -1) { + Py_DECREF(x); + goto Done; + } + whole_us = 2.0 * round((leftover_us + x_is_odd) * 0.5) - x_is_odd; + } temp = PyLong_FromLong((long)whole_us); -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 9 01:04:29 2015 From: python-checkins at python.org (victor.stinner) Date: Tue, 08 Sep 2015 23:04:29 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2323517=3A_fromtime?= =?utf-8?q?stamp=28=29_and_utcfromtimestamp=28=29_methods_of?= Message-ID: <20150908230426.27689.37680@psf.io> https://hg.python.org/cpython/rev/171d5590ebc3 changeset: 97782:171d5590ebc3 user: Victor Stinner date: Wed Sep 09 01:02:23 2015 +0200 summary: Issue #23517: fromtimestamp() and utcfromtimestamp() methods of datetime.datetime now round microseconds to nearest with ties going to nearest even integer (ROUND_HALF_EVEN), as round(float), instead of rounding towards -Infinity (ROUND_FLOOR). pytime API: replace _PyTime_ROUND_HALF_UP with _PyTime_ROUND_HALF_EVEN. Fix also _PyTime_Divide() for negative numbers. _PyTime_AsTimeval_impl() now reuses _PyTime_Divide() instead of reimplementing rounding modes. files: Include/pytime.h | 9 +- Lib/datetime.py | 2 +- Lib/test/datetimetester.py | 4 +- Lib/test/test_time.py | 237 +++++++++++------------- Misc/NEWS | 6 +- Modules/_datetimemodule.c | 2 +- Modules/_testcapimodule.c | 2 +- Python/pytime.c | 71 +++---- 8 files changed, 148 insertions(+), 185 deletions(-) diff --git a/Include/pytime.h b/Include/pytime.h --- a/Include/pytime.h +++ b/Include/pytime.h @@ -31,9 +31,9 @@ /* Round towards infinity (+inf). For example, used for timeout to wait "at least" N seconds. */ _PyTime_ROUND_CEILING=1, - /* Round to nearest with ties going away from zero. + /* Round to nearest with ties going to nearest even integer. For example, used to round from a Python float. */ - _PyTime_ROUND_HALF_UP + _PyTime_ROUND_HALF_EVEN } _PyTime_round_t; /* Convert a time_t to a PyLong. */ @@ -44,8 +44,9 @@ PyAPI_FUNC(time_t) _PyLong_AsTime_t( PyObject *obj); -/* Round to nearest with ties going away from zero (_PyTime_ROUND_HALF_UP). */ -PyAPI_FUNC(double) _PyTime_RoundHalfUp( +/* Round to nearest with ties going to nearest even integer + (_PyTime_ROUND_HALF_EVEN) */ +PyAPI_FUNC(double) _PyTime_RoundHalfEven( double x); /* Convert a number of seconds, int or float, to time_t. */ diff --git a/Lib/datetime.py b/Lib/datetime.py --- a/Lib/datetime.py +++ b/Lib/datetime.py @@ -1380,7 +1380,7 @@ A timezone info object may be passed in as well. """ frac, t = _math.modf(t) - us = _round_half_up(frac * 1e6) + us = round(frac * 1e6) if us >= 1000000: t += 1 us -= 1000000 diff --git a/Lib/test/datetimetester.py b/Lib/test/datetimetester.py --- a/Lib/test/datetimetester.py +++ b/Lib/test/datetimetester.py @@ -1874,7 +1874,7 @@ self.assertEqual(t, zero) t = fts(-1/2**7) self.assertEqual(t.second, 59) - self.assertEqual(t.microsecond, 992187) + self.assertEqual(t.microsecond, 992188) t = fts(1e-7) self.assertEqual(t, zero) @@ -1888,7 +1888,7 @@ self.assertEqual(t.microsecond, 0) t = fts(1/2**7) self.assertEqual(t.second, 0) - self.assertEqual(t.microsecond, 7813) + self.assertEqual(t.microsecond, 7812) def test_insane_fromtimestamp(self): # It's possible that some platform maps time_t to double, diff --git a/Lib/test/test_time.py b/Lib/test/test_time.py --- a/Lib/test/test_time.py +++ b/Lib/test/test_time.py @@ -30,11 +30,11 @@ ROUND_FLOOR = 0 # Round towards infinity (+inf) ROUND_CEILING = 1 - # Round to nearest with ties going away from zero - ROUND_HALF_UP = 2 + # Round to nearest with ties going to nearest even integer + ROUND_HALF_EVEN = 2 ALL_ROUNDING_METHODS = (_PyTime.ROUND_FLOOR, _PyTime.ROUND_CEILING, - _PyTime.ROUND_HALF_UP) + _PyTime.ROUND_HALF_EVEN) class TimeTestCase(unittest.TestCase): @@ -639,27 +639,26 @@ # Conversion giving different results depending on the rounding method FLOOR = _PyTime.ROUND_FLOOR CEILING = _PyTime.ROUND_CEILING - HALF_UP = _PyTime.ROUND_HALF_UP - for obj, time_t, rnd in ( + HALF_EVEN = _PyTime.ROUND_HALF_EVEN + for obj, seconds, rnd in ( (-1.9, -2, FLOOR), (-1.9, -1, CEILING), - (-1.9, -2, HALF_UP), + (-1.9, -2, HALF_EVEN), (1.9, 1, FLOOR), (1.9, 2, CEILING), - (1.9, 2, HALF_UP), + (1.9, 2, HALF_EVEN), - # half up - (-0.999, -1, HALF_UP), - (-0.510, -1, HALF_UP), - (-0.500, -1, HALF_UP), - (-0.490, 0, HALF_UP), - ( 0.490, 0, HALF_UP), - ( 0.500, 1, HALF_UP), - ( 0.510, 1, HALF_UP), - ( 0.999, 1, HALF_UP), + # half even + (-1.5, -2, HALF_EVEN), + (-0.9, -1, HALF_EVEN), + (-0.5, 0, HALF_EVEN), + ( 0.5, 0, HALF_EVEN), + ( 0.9, 1, HALF_EVEN), + ( 1.5, 2, HALF_EVEN), ): - self.assertEqual(pytime_object_to_time_t(obj, rnd), time_t) + with self.subTest(obj=obj, round=rnd, seconds=seconds): + self.assertEqual(pytime_object_to_time_t(obj, rnd), seconds) # Test OverflowError rnd = _PyTime.ROUND_FLOOR @@ -691,15 +690,15 @@ # Conversion giving different results depending on the rounding method FLOOR = _PyTime.ROUND_FLOOR CEILING = _PyTime.ROUND_CEILING - HALF_UP = _PyTime.ROUND_HALF_UP + HALF_EVEN = _PyTime.ROUND_HALF_EVEN for obj, timespec, rnd in ( # Round towards minus infinity (-inf) (-1e-10, (0, 0), CEILING), (-1e-10, (-1, 999999999), FLOOR), - (-1e-10, (0, 0), HALF_UP), + (-1e-10, (0, 0), HALF_EVEN), (1e-10, (0, 0), FLOOR), (1e-10, (0, 1), CEILING), - (1e-10, (0, 0), HALF_UP), + (1e-10, (0, 0), HALF_EVEN), (0.9999999999, (0, 999999999), FLOOR), (0.9999999999, (1, 0), CEILING), @@ -714,15 +713,13 @@ (-1.1234567890, (-2, 876543211), CEILING), (-1.1234567891, (-2, 876543211), CEILING), - # half up - (-0.6e-9, (-1, 999999999), HALF_UP), - # skipped, 0.5e-6 is inexact in base 2 - #(-0.5e-9, (-1, 999999999), HALF_UP), - (-0.4e-9, (0, 0), HALF_UP), - - (0.4e-9, (0, 0), HALF_UP), - (0.5e-9, (0, 1), HALF_UP), - (0.6e-9, (0, 1), HALF_UP), + # half even + (-1.5e-9, (-1, 999999998), HALF_EVEN), + (-0.9e-9, (-1, 999999999), HALF_EVEN), + (-0.5e-9, (0, 0), HALF_EVEN), + (0.5e-9, (0, 0), HALF_EVEN), + (0.9e-9, (0, 1), HALF_EVEN), + (1.5e-9, (0, 2), HALF_EVEN), ): with self.subTest(obj=obj, round=rnd, timespec=timespec): self.assertEqual(pytime_object_to_timespec(obj, rnd), timespec) @@ -823,10 +820,10 @@ (-7.0, -7 * SEC_TO_NS), # nanosecond are kept for value <= 2^23 seconds, - # except 2**23-1e-9 with HALF_UP (2**22 - 1e-9, 4194303999999999), (2**22, 4194304000000000), (2**22 + 1e-9, 4194304000000001), + (2**23 - 1e-9, 8388607999999999), (2**23, 8388608000000000), # start loosing precision for value > 2^23 seconds @@ -859,38 +856,31 @@ # Conversion giving different results depending on the rounding method FLOOR = _PyTime.ROUND_FLOOR CEILING = _PyTime.ROUND_CEILING - HALF_UP = _PyTime.ROUND_HALF_UP + HALF_EVEN = _PyTime.ROUND_HALF_EVEN for obj, ts, rnd in ( # close to zero ( 1e-10, 0, FLOOR), ( 1e-10, 1, CEILING), - ( 1e-10, 0, HALF_UP), + ( 1e-10, 0, HALF_EVEN), (-1e-10, -1, FLOOR), (-1e-10, 0, CEILING), - (-1e-10, 0, HALF_UP), + (-1e-10, 0, HALF_EVEN), # test rounding of the last nanosecond ( 1.1234567899, 1123456789, FLOOR), ( 1.1234567899, 1123456790, CEILING), - ( 1.1234567899, 1123456790, HALF_UP), + ( 1.1234567899, 1123456790, HALF_EVEN), (-1.1234567899, -1123456790, FLOOR), (-1.1234567899, -1123456789, CEILING), - (-1.1234567899, -1123456790, HALF_UP), + (-1.1234567899, -1123456790, HALF_EVEN), # close to 1 second ( 0.9999999999, 999999999, FLOOR), ( 0.9999999999, 1000000000, CEILING), - ( 0.9999999999, 1000000000, HALF_UP), + ( 0.9999999999, 1000000000, HALF_EVEN), (-0.9999999999, -1000000000, FLOOR), (-0.9999999999, -999999999, CEILING), - (-0.9999999999, -1000000000, HALF_UP), - - # close to 2^23 seconds - (2**23 - 1e-9, 8388607999999999, FLOOR), - (2**23 - 1e-9, 8388607999999999, CEILING), - # Issue #23517: skip HALF_UP test because the result is different - # depending on the FPU and how the compiler optimize the code :-/ - #(2**23 - 1e-9, 8388608000000000, HALF_UP), + (-0.9999999999, -1000000000, HALF_EVEN), ): with self.subTest(obj=obj, round=rnd, timestamp=ts): self.assertEqual(PyTime_FromSecondsObject(obj, rnd), ts) @@ -958,33 +948,23 @@ FLOOR = _PyTime.ROUND_FLOOR CEILING = _PyTime.ROUND_CEILING - HALF_UP = _PyTime.ROUND_HALF_UP + HALF_EVEN = _PyTime.ROUND_HALF_EVEN for ns, tv, rnd in ( # nanoseconds (1, (0, 0), FLOOR), (1, (0, 1), CEILING), - (1, (0, 0), HALF_UP), + (1, (0, 0), HALF_EVEN), (-1, (-1, 999999), FLOOR), (-1, (0, 0), CEILING), - (-1, (0, 0), HALF_UP), + (-1, (0, 0), HALF_EVEN), - # seconds + nanoseconds - (1234567001, (1, 234567), FLOOR), - (1234567001, (1, 234568), CEILING), - (1234567001, (1, 234567), HALF_UP), - (-1234567001, (-2, 765432), FLOOR), - (-1234567001, (-2, 765433), CEILING), - (-1234567001, (-2, 765433), HALF_UP), - - # half up - (499, (0, 0), HALF_UP), - (500, (0, 1), HALF_UP), - (501, (0, 1), HALF_UP), - (999, (0, 1), HALF_UP), - (-499, (0, 0), HALF_UP), - (-500, (0, 0), HALF_UP), - (-501, (-1, 999999), HALF_UP), - (-999, (-1, 999999), HALF_UP), + # half even + (-1500, (-1, 999998), HALF_EVEN), + (-999, (-1, 999999), HALF_EVEN), + (-500, (0, 0), HALF_EVEN), + (500, (0, 0), HALF_EVEN), + (999, (0, 1), HALF_EVEN), + (1500, (0, 2), HALF_EVEN), ): with self.subTest(nanoseconds=ns, timeval=tv, round=rnd): self.assertEqual(PyTime_AsTimeval(ns, rnd), tv) @@ -1027,33 +1007,31 @@ FLOOR = _PyTime.ROUND_FLOOR CEILING = _PyTime.ROUND_CEILING - HALF_UP = _PyTime.ROUND_HALF_UP + HALF_EVEN = _PyTime.ROUND_HALF_EVEN for ns, ms, rnd in ( # nanoseconds (1, 0, FLOOR), (1, 1, CEILING), - (1, 0, HALF_UP), - (-1, 0, FLOOR), - (-1, -1, CEILING), - (-1, 0, HALF_UP), + (1, 0, HALF_EVEN), + (-1, -1, FLOOR), + (-1, 0, CEILING), + (-1, 0, HALF_EVEN), # seconds + nanoseconds (1234 * MS_TO_NS + 1, 1234, FLOOR), (1234 * MS_TO_NS + 1, 1235, CEILING), - (1234 * MS_TO_NS + 1, 1234, HALF_UP), - (-1234 * MS_TO_NS - 1, -1234, FLOOR), - (-1234 * MS_TO_NS - 1, -1235, CEILING), - (-1234 * MS_TO_NS - 1, -1234, HALF_UP), + (1234 * MS_TO_NS + 1, 1234, HALF_EVEN), + (-1234 * MS_TO_NS - 1, -1235, FLOOR), + (-1234 * MS_TO_NS - 1, -1234, CEILING), + (-1234 * MS_TO_NS - 1, -1234, HALF_EVEN), # half up - (499999, 0, HALF_UP), - (499999, 0, HALF_UP), - (500000, 1, HALF_UP), - (999999, 1, HALF_UP), - (-499999, 0, HALF_UP), - (-500000, -1, HALF_UP), - (-500001, -1, HALF_UP), - (-999999, -1, HALF_UP), + (-1500000, -2, HALF_EVEN), + (-999999, -1, HALF_EVEN), + (-500000, 0, HALF_EVEN), + (500000, 0, HALF_EVEN), + (999999, 1, HALF_EVEN), + (1500000, 2, HALF_EVEN), ): with self.subTest(nanoseconds=ns, milliseconds=ms, round=rnd): self.assertEqual(PyTime_AsMilliseconds(ns, rnd), ms) @@ -1079,31 +1057,31 @@ FLOOR = _PyTime.ROUND_FLOOR CEILING = _PyTime.ROUND_CEILING - HALF_UP = _PyTime.ROUND_HALF_UP + HALF_EVEN = _PyTime.ROUND_HALF_EVEN for ns, ms, rnd in ( # nanoseconds (1, 0, FLOOR), (1, 1, CEILING), - (1, 0, HALF_UP), - (-1, 0, FLOOR), - (-1, -1, CEILING), - (-1, 0, HALF_UP), + (1, 0, HALF_EVEN), + (-1, -1, FLOOR), + (-1, 0, CEILING), + (-1, 0, HALF_EVEN), # seconds + nanoseconds (1234 * US_TO_NS + 1, 1234, FLOOR), (1234 * US_TO_NS + 1, 1235, CEILING), - (1234 * US_TO_NS + 1, 1234, HALF_UP), - (-1234 * US_TO_NS - 1, -1234, FLOOR), - (-1234 * US_TO_NS - 1, -1235, CEILING), - (-1234 * US_TO_NS - 1, -1234, HALF_UP), + (1234 * US_TO_NS + 1, 1234, HALF_EVEN), + (-1234 * US_TO_NS - 1, -1235, FLOOR), + (-1234 * US_TO_NS - 1, -1234, CEILING), + (-1234 * US_TO_NS - 1, -1234, HALF_EVEN), # half up - (1499, 1, HALF_UP), - (1500, 2, HALF_UP), - (1501, 2, HALF_UP), - (-1499, -1, HALF_UP), - (-1500, -2, HALF_UP), - (-1501, -2, HALF_UP), + (-1500, -2, HALF_EVEN), + (-999, -1, HALF_EVEN), + (-500, 0, HALF_EVEN), + (500, 0, HALF_EVEN), + (999, 1, HALF_EVEN), + (1500, 2, HALF_EVEN), ): with self.subTest(nanoseconds=ns, milliseconds=ms, round=rnd): self.assertEqual(PyTime_AsMicroseconds(ns, rnd), ms) @@ -1142,23 +1120,23 @@ # Conversion giving different results depending on the rounding method FLOOR = _PyTime.ROUND_FLOOR CEILING = _PyTime.ROUND_CEILING - HALF_UP = _PyTime.ROUND_HALF_UP + HALF_EVEN = _PyTime.ROUND_HALF_EVEN for obj, time_t, rnd in ( (-1.9, -2, FLOOR), - (-1.9, -2, HALF_UP), + (-1.9, -2, HALF_EVEN), (-1.9, -1, CEILING), (1.9, 1, FLOOR), - (1.9, 2, HALF_UP), + (1.9, 2, HALF_EVEN), (1.9, 2, CEILING), - (-0.6, -1, HALF_UP), - (-0.5, -1, HALF_UP), - (-0.4, 0, HALF_UP), - - (0.4, 0, HALF_UP), - (0.5, 1, HALF_UP), - (0.6, 1, HALF_UP), + # half even + (-1.5, -2, HALF_EVEN), + (-0.9, -1, HALF_EVEN), + (-0.5, 0, HALF_EVEN), + ( 0.5, 0, HALF_EVEN), + ( 0.9, 1, HALF_EVEN), + ( 1.5, 2, HALF_EVEN), ): self.assertEqual(pytime_object_to_time_t(obj, rnd), time_t) @@ -1192,29 +1170,27 @@ # Conversion giving different results depending on the rounding method FLOOR = _PyTime.ROUND_FLOOR CEILING = _PyTime.ROUND_CEILING - HALF_UP = _PyTime.ROUND_HALF_UP + HALF_EVEN = _PyTime.ROUND_HALF_EVEN for obj, timeval, rnd in ( (-1e-7, (-1, 999999), FLOOR), (-1e-7, (0, 0), CEILING), - (-1e-7, (0, 0), HALF_UP), + (-1e-7, (0, 0), HALF_EVEN), (1e-7, (0, 0), FLOOR), (1e-7, (0, 1), CEILING), - (1e-7, (0, 0), HALF_UP), + (1e-7, (0, 0), HALF_EVEN), (0.9999999, (0, 999999), FLOOR), (0.9999999, (1, 0), CEILING), - (0.9999999, (1, 0), HALF_UP), + (0.9999999, (1, 0), HALF_EVEN), - (-0.6e-6, (-1, 999999), HALF_UP), - # skipped, -0.5e-6 is inexact in base 2 - #(-0.5e-6, (-1, 999999), HALF_UP), - (-0.4e-6, (0, 0), HALF_UP), - - (0.4e-6, (0, 0), HALF_UP), - # skipped, 0.5e-6 is inexact in base 2 - #(0.5e-6, (0, 1), HALF_UP), - (0.6e-6, (0, 1), HALF_UP), + # half even + (-1.5e-6, (-1, 999998), HALF_EVEN), + (-0.9e-6, (-1, 999999), HALF_EVEN), + (-0.5e-6, (0, 0), HALF_EVEN), + (0.5e-6, (0, 0), HALF_EVEN), + (0.9e-6, (0, 1), HALF_EVEN), + (1.5e-6, (0, 2), HALF_EVEN), ): with self.subTest(obj=obj, round=rnd, timeval=timeval): self.assertEqual(pytime_object_to_timeval(obj, rnd), timeval) @@ -1248,28 +1224,27 @@ # Conversion giving different results depending on the rounding method FLOOR = _PyTime.ROUND_FLOOR CEILING = _PyTime.ROUND_CEILING - HALF_UP = _PyTime.ROUND_HALF_UP + HALF_EVEN = _PyTime.ROUND_HALF_EVEN for obj, timespec, rnd in ( (-1e-10, (-1, 999999999), FLOOR), (-1e-10, (0, 0), CEILING), - (-1e-10, (0, 0), HALF_UP), + (-1e-10, (0, 0), HALF_EVEN), (1e-10, (0, 0), FLOOR), (1e-10, (0, 1), CEILING), - (1e-10, (0, 0), HALF_UP), + (1e-10, (0, 0), HALF_EVEN), (0.9999999999, (0, 999999999), FLOOR), (0.9999999999, (1, 0), CEILING), - (0.9999999999, (1, 0), HALF_UP), + (0.9999999999, (1, 0), HALF_EVEN), - (-0.6e-9, (-1, 999999999), HALF_UP), - # skipped, 0.5e-6 is inexact in base 2 - #(-0.5e-9, (-1, 999999999), HALF_UP), - (-0.4e-9, (0, 0), HALF_UP), - - (0.4e-9, (0, 0), HALF_UP), - (0.5e-9, (0, 1), HALF_UP), - (0.6e-9, (0, 1), HALF_UP), + # half even + (-1.5e-9, (-1, 999999998), HALF_EVEN), + (-0.9e-9, (-1, 999999999), HALF_EVEN), + (-0.5e-9, (0, 0), HALF_EVEN), + (0.5e-9, (0, 0), HALF_EVEN), + (0.9e-9, (0, 1), HALF_EVEN), + (1.5e-9, (0, 2), HALF_EVEN), ): with self.subTest(obj=obj, round=rnd, timespec=timespec): self.assertEqual(pytime_object_to_timespec(obj, rnd), timespec) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -20,9 +20,9 @@ - Issue #22241: timezone.utc name is now plain 'UTC', not 'UTC-00:00'. - Issue #23517: fromtimestamp() and utcfromtimestamp() methods of - datetime.datetime now round microseconds to nearest with ties going away from - zero (ROUND_HALF_UP), as Python 2 and Python older than 3.3, instead of - rounding towards -Infinity (ROUND_FLOOR). + datetime.datetime now round microseconds to nearest with ties going to + nearest even integer (ROUND_HALF_EVEN), as round(float), instead of rounding + towards -Infinity (ROUND_FLOOR). - Issue #23552: Timeit now warns when there is substantial (4x) variance between best and worst times. Patch from Serhiy Storchaka. diff --git a/Modules/_datetimemodule.c b/Modules/_datetimemodule.c --- a/Modules/_datetimemodule.c +++ b/Modules/_datetimemodule.c @@ -4103,7 +4103,7 @@ long us; if (_PyTime_ObjectToTimeval(timestamp, - &timet, &us, _PyTime_ROUND_HALF_UP) == -1) + &timet, &us, _PyTime_ROUND_HALF_EVEN) == -1) return NULL; return datetime_from_timet_and_us(cls, f, timet, (int)us, tzinfo); diff --git a/Modules/_testcapimodule.c b/Modules/_testcapimodule.c --- a/Modules/_testcapimodule.c +++ b/Modules/_testcapimodule.c @@ -2648,7 +2648,7 @@ { if (round != _PyTime_ROUND_FLOOR && round != _PyTime_ROUND_CEILING - && round != _PyTime_ROUND_HALF_UP) { + && round != _PyTime_ROUND_HALF_EVEN) { PyErr_SetString(PyExc_ValueError, "invalid rounding"); return -1; } diff --git a/Python/pytime.c b/Python/pytime.c --- a/Python/pytime.c +++ b/Python/pytime.c @@ -61,18 +61,15 @@ } double -_PyTime_RoundHalfUp(double x) +_PyTime_RoundHalfEven(double x) { - /* volatile avoids optimization changing how numbers are rounded */ - volatile double d = x; - if (d >= 0.0) - d = floor(d + 0.5); - else - d = ceil(d - 0.5); - return d; + double rounded = round(x); + if (fabs(x-rounded) == 0.5) + /* halfway case: round to even */ + rounded = 2.0*round(x/2.0); + return rounded; } - static int _PyTime_DoubleToDenominator(double d, time_t *sec, long *numerator, double denominator, _PyTime_round_t round) @@ -84,8 +81,8 @@ floatpart = modf(d, &intpart); floatpart *= denominator; - if (round == _PyTime_ROUND_HALF_UP) - floatpart = _PyTime_RoundHalfUp(floatpart); + if (round == _PyTime_ROUND_HALF_EVEN) + floatpart = _PyTime_RoundHalfEven(floatpart); else if (round == _PyTime_ROUND_CEILING) floatpart = ceil(floatpart); else @@ -140,8 +137,8 @@ volatile double d; d = PyFloat_AsDouble(obj); - if (round == _PyTime_ROUND_HALF_UP) - d = _PyTime_RoundHalfUp(d); + if (round == _PyTime_ROUND_HALF_EVEN) + d = _PyTime_RoundHalfEven(d); else if (round == _PyTime_ROUND_CEILING) d = ceil(d); else @@ -266,8 +263,8 @@ d = value; d *= to_nanoseconds; - if (round == _PyTime_ROUND_HALF_UP) - d = _PyTime_RoundHalfUp(d); + if (round == _PyTime_ROUND_HALF_EVEN) + d = _PyTime_RoundHalfEven(d); else if (round == _PyTime_ROUND_CEILING) d = ceil(d); else @@ -351,14 +348,16 @@ } static _PyTime_t -_PyTime_Divide(_PyTime_t t, _PyTime_t k, _PyTime_round_t round) +_PyTime_Divide(const _PyTime_t t, const _PyTime_t k, + const _PyTime_round_t round) { assert(k > 1); - if (round == _PyTime_ROUND_HALF_UP) { - _PyTime_t x, r; + if (round == _PyTime_ROUND_HALF_EVEN) { + _PyTime_t x, r, abs_r; x = t / k; r = t % k; - if (Py_ABS(r) >= k / 2) { + abs_r = Py_ABS(r); + if (abs_r > k / 2 || (abs_r == k / 2 && (Py_ABS(x) & 1))) { if (t >= 0) x++; else @@ -370,10 +369,14 @@ if (t >= 0) return (t + k - 1) / k; else + return t / k; + } + else { + if (t >= 0) + return t / k; + else return (t - (k - 1)) / k; } - else - return t / k; } _PyTime_t @@ -392,17 +395,12 @@ _PyTime_AsTimeval_impl(_PyTime_t t, struct timeval *tv, _PyTime_round_t round, int raise) { - const long k = US_TO_NS; _PyTime_t secs, ns; int res = 0; int usec; secs = t / SEC_TO_NS; ns = t % SEC_TO_NS; - if (ns < 0) { - ns += SEC_TO_NS; - secs -= 1; - } #ifdef MS_WINDOWS /* On Windows, timeval.tv_sec is a long (32 bit), @@ -427,23 +425,12 @@ res = -1; #endif - if (round == _PyTime_ROUND_HALF_UP) { - _PyTime_t r; - usec = (int)(ns / k); - r = ns % k; - if (Py_ABS(r) >= k / 2) { - if (ns >= 0) - usec++; - else - usec--; - } + usec = (int)_PyTime_Divide(ns, US_TO_NS, round); + if (usec < 0) { + usec += SEC_TO_US; + tv->tv_sec -= 1; } - else if (round == _PyTime_ROUND_CEILING) - usec = (int)((ns + k - 1) / k); - else - usec = (int)(ns / k); - - if (usec >= SEC_TO_US) { + else if (usec >= SEC_TO_US) { usec -= SEC_TO_US; tv->tv_sec += 1; } -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 9 01:09:41 2015 From: python-checkins at python.org (victor.stinner) Date: Tue, 08 Sep 2015 23:09:41 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_cleanup_datetime_code?= Message-ID: <20150908230938.66856.56459@psf.io> https://hg.python.org/cpython/rev/46d812637efd changeset: 97783:46d812637efd user: Victor Stinner date: Wed Sep 09 01:09:21 2015 +0200 summary: cleanup datetime code remove scories of round half up code and debug code. files: Lib/datetime.py | 7 ------- Lib/test/datetimetester.py | 2 +- 2 files changed, 1 insertions(+), 8 deletions(-) diff --git a/Lib/datetime.py b/Lib/datetime.py --- a/Lib/datetime.py +++ b/Lib/datetime.py @@ -316,13 +316,6 @@ return q -def _round_half_up(x): - """Round to nearest with ties going away from zero.""" - if x >= 0.0: - return _math.floor(x + 0.5) - else: - return _math.ceil(x - 0.5) - class timedelta: """Represent the difference between two datetime objects. diff --git a/Lib/test/datetimetester.py b/Lib/test/datetimetester.py --- a/Lib/test/datetimetester.py +++ b/Lib/test/datetimetester.py @@ -679,7 +679,7 @@ us_per_day = us_per_hour * 24 eq(td(days=.4/us_per_day), td(0)) eq(td(hours=.2/us_per_hour), td(0)) - eq(td(days=.4/us_per_day, hours=.2/us_per_hour), td(microseconds=1), td) + eq(td(days=.4/us_per_day, hours=.2/us_per_hour), td(microseconds=1)) eq(td(days=-.4/us_per_day), td(0)) eq(td(hours=-.2/us_per_hour), td(0)) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 9 02:10:43 2015 From: python-checkins at python.org (eric.smith) Date: Wed, 09 Sep 2015 00:10:43 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_Added_some_text_about_escapin?= =?utf-8?q?g_quote_chars=2C_and_about_raw_f-strings=2E?= Message-ID: <20150909001042.68875.9770@psf.io> https://hg.python.org/peps/rev/a593dc58bf31 changeset: 6044:a593dc58bf31 user: Eric V. Smith date: Tue Sep 08 20:10:44 2015 -0400 summary: Added some text about escaping quote chars, and about raw f-strings. files: pep-0498.txt | 28 +++++++++++++++++++++++++--- 1 files changed, 25 insertions(+), 3 deletions(-) diff --git a/pep-0498.txt b/pep-0498.txt --- a/pep-0498.txt +++ b/pep-0498.txt @@ -183,9 +183,9 @@ (parsing within an f-string is another matter, of course). Once tokenized, f-strings are decoded. This will convert backslash -escapes such as ``'\n'``, ``'\xhh'``, ``'\uxxxx'``, ``'\Uxxxxxxxx'``, -and named unicode characters ``'\N{name}'`` into their associated -Unicode characters [#]_. +escapes such as ``'\n'``, ``'\"'``, ``"\'"``, ``'\xhh'``, +``'\uxxxx'``, ``'\Uxxxxxxxx'``, and named unicode characters +``'\N{name}'`` into their associated Unicode characters [#]_. Up to this point, the processing of f-strings and normal strings is exactly the same. @@ -256,6 +256,28 @@ >>> f'{{{4*10}}}' '{40}' +Like all raw in Python, no escape processing is done for raw +f-strings:: + + >>> fr'x={4*10}\n' + 'x=40\\n' + +Due to Python's string tokenizing rules, the f-string +``f'abc {a['x']} def'`` is invalid. The tokenizer parses this as 3 +tokens: ``f'abc {a['``, ``x``, and ``']} def'``. Just like regular +strings, this cannot be fixed by using raw strings. There are a number +of correct ways to write this f-string: with escaped single quotes:: + + f'abc {a[\'x\']} def' + +With a different quote character:: + + f"abc {a['x']} def" + +Or with triple quotes:: + + f'''abc {a['x']} def''' + Code equivalence ---------------- -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Wed Sep 9 02:38:44 2015 From: python-checkins at python.org (eric.smith) Date: Wed, 09 Sep 2015 00:38:44 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_PEP_498_accepted=2E_Yay!_Than?= =?utf-8?q?ks=2C_all=2E?= Message-ID: <20150909003844.17969.14742@psf.io> https://hg.python.org/peps/rev/10a283dc6b1a changeset: 6045:10a283dc6b1a user: Eric V. Smith date: Tue Sep 08 20:38:45 2015 -0400 summary: PEP 498 accepted. Yay! Thanks, all. files: pep-0498.txt | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-) diff --git a/pep-0498.txt b/pep-0498.txt --- a/pep-0498.txt +++ b/pep-0498.txt @@ -3,12 +3,13 @@ Version: $Revision$ Last-Modified: $Date$ Author: Eric V. Smith -Status: Draft +Status: Accepted Type: Standards Track Content-Type: text/x-rst Created: 01-Aug-2015 Python-Version: 3.6 Post-History: 07-Aug-2015, 30-Aug-2015, 04-Sep-2015 +Resolution: https://mail.python.org/pipermail/python-dev/2015-September/141526.html Abstract ======== -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Wed Sep 9 04:41:05 2015 From: python-checkins at python.org (yury.selivanov) Date: Wed, 09 Sep 2015 02:41:05 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogd2hhdHNuZXcvMy41?= =?utf-8?q?=3A_Fix_library_news_till_Py3=2E5a1=2E_Update_other_docs=2E?= Message-ID: <20150909024105.68867.84516@psf.io> https://hg.python.org/cpython/rev/4a56a14ca4c1 changeset: 97784:4a56a14ca4c1 branch: 3.5 parent: 97779:60f5ca4c5a01 user: Yury Selivanov date: Tue Sep 08 22:40:30 2015 -0400 summary: whatsnew/3.5: Fix library news till Py3.5a1. Update other docs. files: Doc/library/compileall.rst | 12 +- Doc/library/imghdr.rst | 4 +- Doc/library/inspect.rst | 20 +++ Doc/whatsnew/3.5.rst | 163 ++++++++++++++++++++++++- 4 files changed, 185 insertions(+), 14 deletions(-) diff --git a/Doc/library/compileall.rst b/Doc/library/compileall.rst --- a/Doc/library/compileall.rst +++ b/Doc/library/compileall.rst @@ -88,14 +88,12 @@ Added the ``-i``, ``-b`` and ``-h`` options. .. versionchanged:: 3.5 - Added the ``-j`` and ``-r`` options. -.. versionchanged:: 3.5 - ``-q`` option was changed to a multilevel value. - -.. versionchanged:: 3.5 - ``-b`` will always produce a byte-code file ending in ``.pyc``, never - ``.pyo``. + * Added the ``-j`` and ``-r`` options. + * ``-q`` option was changed to a multilevel value. + * ``-qq`` option. + * ``-b`` will always produce a byte-code file ending in ``.pyc``, + never ``.pyo``. There is no command-line option to control the optimization level used by the diff --git a/Doc/library/imghdr.rst b/Doc/library/imghdr.rst --- a/Doc/library/imghdr.rst +++ b/Doc/library/imghdr.rst @@ -54,10 +54,8 @@ +------------+-----------------------------------+ .. versionadded:: 3.5 - The *exr* format was added. + The *exr* and *webp* formats were added. -.. versionchanged:: 3.5 - The *webp* type was added. You can extend the list of file types :mod:`imghdr` can recognize by appending to this variable: diff --git a/Doc/library/inspect.rst b/Doc/library/inspect.rst --- a/Doc/library/inspect.rst +++ b/Doc/library/inspect.rst @@ -1039,6 +1039,11 @@ returned list represents *frame*; the last entry represents the outermost call on *frame*'s stack. + .. versionchanged:: 3.5 + A list of :term:`named tuples ` + ``FrameInfo(frame, filename, lineno, function, code_context, index)`` + is returned. + .. function:: getinnerframes(traceback, context=1) @@ -1047,6 +1052,11 @@ list represents *traceback*; the last entry represents where the exception was raised. + .. versionchanged:: 3.5 + A list of :term:`named tuples ` + ``FrameInfo(frame, filename, lineno, function, code_context, index)`` + is returned. + .. function:: currentframe() @@ -1066,6 +1076,11 @@ returned list represents the caller; the last entry represents the outermost call on the stack. + .. versionchanged:: 3.5 + A list of :term:`named tuples ` + ``FrameInfo(frame, filename, lineno, function, code_context, index)`` + is returned. + .. function:: trace(context=1) @@ -1074,6 +1089,11 @@ entry in the list represents the caller; the last entry represents where the exception was raised. + .. versionchanged:: 3.5 + A list of :term:`named tuples ` + ``FrameInfo(frame, filename, lineno, function, code_context, index)`` + is returned. + Fetching attributes statically ------------------------------ diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -122,7 +122,14 @@ Security improvements: -* None yet. +* SSLv3 is now disabled throughout the standard library. + It can still be enabled by instantiating a :class:`ssl.SSLContext` + manually. (See :issue:`22638` for more details; this change was + backported to CPython 3.4 and 2.7.) + +* HTTP cookie parsing is now stricter, in order to protect + against potential injection attacks. (Contributed by Antoine Pitrou + in :issue:`22796`.) Windows improvements: @@ -606,6 +613,13 @@ :ref:`allow_abbrev` to ``False``. (Contributed by Jonathan Paugh, Steven Bethard, paul j3 and Daniel Eriksson.) +bz2 +--- + +* New option *max_length* for :meth:`~bz2.BZ2Decompressor.decompress` + to limit the maximum size of decompressed data. + (Contributed by Nikolaus Rath in :issue:`15955`.) + cgi --- @@ -645,6 +659,20 @@ can now do parallel bytecode compilation. (Contributed by Claudiu Popa in :issue:`16104`.) +* *quiet* parameter of :func:`compileall.compile_dir`, + :func:`compileall.compile_file`, and :func:`compileall.compile_path` + functions now has a multilevel value. New ``-qq`` command line option + is available for suppressing the output. + (Contributed by Thomas Kluyver in :issue:`21338`.) + +concurrent.futures +------------------ + +* :meth:`~concurrent.futures.Executor.map` now takes a *chunksize* + argument to allow batching of tasks in child processes and improve + performance of ProcessPoolExecutor. + (Contributed by Dan O'Reilly in :issue:`11271`.) + contextlib ---------- @@ -714,6 +742,12 @@ subdirectories using the "``**``" pattern. (Contributed by Serhiy Storchaka in :issue:`13968`.) +heapq +----- + +* :func:`~heapq.merge` has two new optional parameters ``reverse`` and + ``key``. (Contributed by Raymond Hettinger in :issue:`13742`.) + idlelib and IDLE ---------------- @@ -746,7 +780,9 @@ ------ * :func:`~imghdr.what` now recognizes the `OpenEXR `_ - format. (Contributed by Martin Vignali and Claudiu Popa in :issue:`20295`.) + format (contributed by Martin Vignali and Claudiu Popa in :issue:`20295`), + and the `WebP `_ format (contributed + by Fabrice Aneche and Claudiu Popa in :issue:`20197`.) importlib --------- @@ -763,6 +799,7 @@ * :func:`importlib.util.module_from_spec` is now the preferred way to create a new module. Compared to :class:`types.ModuleType`, this new function will set the various import-controlled attributes based on the passed-in spec object. + (Contributed by Brett Cannon in :issue:`20383`.) inspect ------- @@ -788,6 +825,10 @@ * New :func:`~inspect.getcoroutinelocals` and :func:`~inspect.getcoroutinestate` functions. (Contributed by Yury Selivanov in :issue:`24400`.) +* :func:`~inspect.stack`, :func:`~inspect.trace`, :func:`~inspect.getouterframes`, + and :func:`~inspect.getinnerframes` now return a list of named tuples. + (Contributed by Daniel Shahaf in :issue:`16808`.) + ipaddress --------- @@ -807,6 +848,35 @@ * JSON decoder now raises :exc:`json.JSONDecodeError` instead of :exc:`ValueError`. (Contributed by Serhiy Storchaka in :issue:`19361`.) +locale +------ + + * New :func:`~locale.delocalize` function to convert a string into a + normalized number string, following the ``LC_NUMERIC`` settings. + (Contributed by C?dric Krier in :issue:`13918`.) + +logging +------- + +* All logging methods :meth:`~logging.Logger.log`, :meth:`~logging.Logger.exception`, + :meth:`~logging.Logger.critical`, :meth:`~logging.Logger.debug`, etc, + now accept exception instances for ``exc_info`` parameter, in addition + to boolean values and exception tuples. + (Contributed by Yury Selivanov in :issue:`20537`.) + +* :class:`~logging.handlers.HTTPHandler` now accepts optional + :class:`ssl.SSLContext` instance to configure the SSL settings used + for HTTP connection. + (Contributed by Alex Gaynor in :issue:`22788`.) + +lzma +---- + +* New option *max_length* for :meth:`~lzma.LZMADecompressor.decompress` + to limit the maximum size of decompressed data. + (Contributed by Martin Panter in :issue:`15955`.) + + math ---- @@ -832,6 +902,10 @@ function is now used. These functions avoid the usage of an internal file descriptor. +* New :func:`os.get_blocking` and :func:`os.set_blocking` functions to + get and set the blocking mode of file descriptors. + (Contributed by Victor Stinner in :issue:`22054`.) + os.path ------- @@ -839,6 +913,25 @@ Unlike the :func:`~os.path.commonprefix` function, it always returns a valid path. (Contributed by Rafik Draoui and Serhiy Storchaka in :issue:`10395`.) +pathlib +------- + +* New :meth:`~pathlib.Path.samefile` method to check if other path object + points to the same file. (Contributed by Vajrasky Kok and Antoine Pitrou + in :issue:`19775`.) + +* :meth:`~pathlib.Path.mkdir` has a new optional parameter ``exist_ok`` + to mimic ``mkdir -p`` and :func:`os.makrdirs` functionality. + (Contributed by Berker Peksag in :issue:`21539`.) + +* New :meth:`~pathlib.Path.expanduser` to expand ``~`` and ``~user`` + constructs. + (Contributed by Serhiy Storchaka and Claudiu Popa in :issue:`19776`.) + +* New class method :meth:`~pathlib.Path.home` to get an instance of + :class:`~pathlib.Path` object representing the user?s home directory. + (Contributed by Victor Salgado and Mayank Tripathi in :issue:`19777`.) + pickle ------ @@ -862,6 +955,12 @@ * Now unmatched groups are replaced with empty strings in :func:`re.sub` and :func:`re.subn`. (Contributed by Serhiy Storchaka in :issue:`1519638`.) +readline +-------- + +* New :func:`~readline.append_history_file` function. + (Contributed by Bruno Cauet in :issue:`22940`.) + shutil ------ @@ -870,6 +969,9 @@ :func:`~shutil.copy2` if there is a need to ignore metadata. (Contributed by Claudiu Popa in :issue:`19840`.) +* :func:`~shutil.make_archive` now supports *xztar* format. + (Contributed by Serhiy Storchaka in :issue:`5411`.) + signal ------ @@ -948,6 +1050,14 @@ :meth:`SSLContext.wrap_bio ` method. (Contributed by Geert Jansen in :issue:`21965`.) +* New :meth:`~ssl.SSLSocket.version` to query the actual protocol version + in use. (Contributed by Antoine Pitrou in :issue:`20421`.) + +* New :meth:`~ssl.SSLObject.shared_ciphers` and + :meth:`~ssl.SSLSocket.shared_ciphers` methods to fetch the client's + list of ciphers sent at handshake. + (Contributed by Benjamin Peterson in :issue:`23186`.) + socket ------ @@ -961,6 +1071,9 @@ anymore each time bytes are received or sent. The socket timeout is now the maximum total duration to send all data. +* Functions with timeouts now use a monotonic clock, instead of a + system clock. (Contributed by Victor Stinner in :issue:`22043`.) + subprocess ---------- @@ -975,6 +1088,9 @@ * New :func:`~sys.set_coroutine_wrapper` and :func:`~sys.get_coroutine_wrapper` functions. (Contributed by Yury Selivanov in :issue:`24017`.) +* New :func:`~sys.is_finalizing` to check for :term:`interpreter shutdown`. + (Contributed by Antoine Pitrou in :issue:`22696`.) + sysconfig --------- @@ -991,9 +1107,16 @@ methods now take a keyword parameter *numeric_only*. If set to ``True``, the extracted files and directories will be owned by the numeric uid and gid from the tarfile. If set to ``False`` (the default, and the behavior in - versions prior to 3.5), they will be owned bythe named user and group in the + versions prior to 3.5), they will be owned by the named user and group in the tarfile. (Contributed by Michael Vogt and Eric Smith in :issue:`23193`.) +threading +--------- + +* :meth:`~threading.Lock.acquire` and :meth:`~threading.RLock.acquire` + now use a monotonic clock for managing timeouts. + (Contributed by Victor Stinner in :issue:`22043`.) + time ---- @@ -1008,6 +1131,17 @@ module which makes no permanent changes to environment variables. (Contributed by Zachary Ware in :issue:`20035`.) +traceback +--------- + +* New :func:`~traceback.walk_stack` and :func:`~traceback.walk_tb` + functions to conveniently traverse frame and traceback objects. + (Contributed by Robert Collins in :issue:`17911`.) + +* New lightweight classes: :class:`~traceback.TracebackException`, + :class:`~traceback.StackSummary`, and :class:`traceback.FrameSummary`. + (Contributed by Robert Collins in :issue:`17911`.) + types ----- @@ -1032,6 +1166,10 @@ control the encoding of query parts if needed. (Contributed by Samwyse and Arnon Yaari in :issue:`13866`.) +* :func:`~urllib.request.urlopen` accepts an :class:`ssl.SSLContext` + object as a *context* argument, which will be used for the HTTPS + connection. (Contributed by Alex Gaynor in :issue:`22366`.) + unicodedata ----------- @@ -1051,6 +1189,10 @@ * :class:`xmlrpc.client.ServerProxy` is now a :term:`context manager`. (Contributed by Claudiu Popa in :issue:`20627`.) +* :class:`~xmlrpc.client.ServerProxy` constructor now accepts an optional + :class:`ssl.SSLContext` instance. + (Contributed by Alex Gaynor in :issue:`22960`.) + xml.sax ------- @@ -1096,7 +1238,10 @@ :meth:`~ipaddress.IPv4Network.subnets`, :meth:`~ipaddress.IPv4Network.supernet`, :func:`~ipaddress.summarize_address_range`, :func:`~ipaddress.collapse_addresses`. The speed up can range from 3x to 15x. - (:issue:`21486`, :issue:`21487`, :issue:`20826`) + (See :issue:`21486`, :issue:`21487`, :issue:`20826`, :issue:`23266`.) + +* Pickling of :mod:`ipaddress` classes was optimized to produce significantly + smaller output. (Contributed by Serhiy Storchaka in :issue:`23133`.) * Many operations on :class:`io.BytesIO` are now 50% to 100% faster. (Contributed by Serhiy Storchaka in :issue:`15381` and David Wilson in @@ -1109,6 +1254,13 @@ * The UTF-32 encoder is now 3x to 7x faster. (Contributed by Serhiy Storchaka in :issue:`15027`.) +* Regular expressions are now parsed up to 10% faster. + (Contributed by Serhiy Storchaka in :issue:`19380`.) + +* :func:`json.dumps` was optimized to run with ``ensure_ascii=False`` + as fast as with ``ensure_ascii=True``. + (Contributed by Naoki Inada in :issue:`23206`.) + Build and C API Changes ======================= @@ -1185,6 +1337,9 @@ deprecated in favor of :func:`inspect.signature` API. (See :issue:`20438` for details.) +* Use of ``re.LOCALE`` flag with str patterns or ``re.ASCII`` is now + deprecated. (Contributed by Serhiy Storchaka in :issue:`22407`.) + Deprecated functions and types of the C API ------------------------------------------- -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 9 04:41:05 2015 From: python-checkins at python.org (yury.selivanov) Date: Wed, 09 Sep 2015 02:41:05 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogTWVyZ2UgMy41?= Message-ID: <20150909024105.114754.62976@psf.io> https://hg.python.org/cpython/rev/07c206831e28 changeset: 97785:07c206831e28 parent: 97783:46d812637efd parent: 97784:4a56a14ca4c1 user: Yury Selivanov date: Tue Sep 08 22:40:45 2015 -0400 summary: Merge 3.5 files: Doc/library/compileall.rst | 12 +- Doc/library/imghdr.rst | 4 +- Doc/library/inspect.rst | 20 +++ Doc/whatsnew/3.5.rst | 163 ++++++++++++++++++++++++- 4 files changed, 185 insertions(+), 14 deletions(-) diff --git a/Doc/library/compileall.rst b/Doc/library/compileall.rst --- a/Doc/library/compileall.rst +++ b/Doc/library/compileall.rst @@ -88,14 +88,12 @@ Added the ``-i``, ``-b`` and ``-h`` options. .. versionchanged:: 3.5 - Added the ``-j`` and ``-r`` options. -.. versionchanged:: 3.5 - ``-q`` option was changed to a multilevel value. - -.. versionchanged:: 3.5 - ``-b`` will always produce a byte-code file ending in ``.pyc``, never - ``.pyo``. + * Added the ``-j`` and ``-r`` options. + * ``-q`` option was changed to a multilevel value. + * ``-qq`` option. + * ``-b`` will always produce a byte-code file ending in ``.pyc``, + never ``.pyo``. There is no command-line option to control the optimization level used by the diff --git a/Doc/library/imghdr.rst b/Doc/library/imghdr.rst --- a/Doc/library/imghdr.rst +++ b/Doc/library/imghdr.rst @@ -54,10 +54,8 @@ +------------+-----------------------------------+ .. versionadded:: 3.5 - The *exr* format was added. + The *exr* and *webp* formats were added. -.. versionchanged:: 3.5 - The *webp* type was added. You can extend the list of file types :mod:`imghdr` can recognize by appending to this variable: diff --git a/Doc/library/inspect.rst b/Doc/library/inspect.rst --- a/Doc/library/inspect.rst +++ b/Doc/library/inspect.rst @@ -1000,6 +1000,11 @@ returned list represents *frame*; the last entry represents the outermost call on *frame*'s stack. + .. versionchanged:: 3.5 + A list of :term:`named tuples ` + ``FrameInfo(frame, filename, lineno, function, code_context, index)`` + is returned. + .. function:: getinnerframes(traceback, context=1) @@ -1008,6 +1013,11 @@ list represents *traceback*; the last entry represents where the exception was raised. + .. versionchanged:: 3.5 + A list of :term:`named tuples ` + ``FrameInfo(frame, filename, lineno, function, code_context, index)`` + is returned. + .. function:: currentframe() @@ -1027,6 +1037,11 @@ returned list represents the caller; the last entry represents the outermost call on the stack. + .. versionchanged:: 3.5 + A list of :term:`named tuples ` + ``FrameInfo(frame, filename, lineno, function, code_context, index)`` + is returned. + .. function:: trace(context=1) @@ -1035,6 +1050,11 @@ entry in the list represents the caller; the last entry represents where the exception was raised. + .. versionchanged:: 3.5 + A list of :term:`named tuples ` + ``FrameInfo(frame, filename, lineno, function, code_context, index)`` + is returned. + Fetching attributes statically ------------------------------ diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -122,7 +122,14 @@ Security improvements: -* None yet. +* SSLv3 is now disabled throughout the standard library. + It can still be enabled by instantiating a :class:`ssl.SSLContext` + manually. (See :issue:`22638` for more details; this change was + backported to CPython 3.4 and 2.7.) + +* HTTP cookie parsing is now stricter, in order to protect + against potential injection attacks. (Contributed by Antoine Pitrou + in :issue:`22796`.) Windows improvements: @@ -606,6 +613,13 @@ :ref:`allow_abbrev` to ``False``. (Contributed by Jonathan Paugh, Steven Bethard, paul j3 and Daniel Eriksson.) +bz2 +--- + +* New option *max_length* for :meth:`~bz2.BZ2Decompressor.decompress` + to limit the maximum size of decompressed data. + (Contributed by Nikolaus Rath in :issue:`15955`.) + cgi --- @@ -645,6 +659,20 @@ can now do parallel bytecode compilation. (Contributed by Claudiu Popa in :issue:`16104`.) +* *quiet* parameter of :func:`compileall.compile_dir`, + :func:`compileall.compile_file`, and :func:`compileall.compile_path` + functions now has a multilevel value. New ``-qq`` command line option + is available for suppressing the output. + (Contributed by Thomas Kluyver in :issue:`21338`.) + +concurrent.futures +------------------ + +* :meth:`~concurrent.futures.Executor.map` now takes a *chunksize* + argument to allow batching of tasks in child processes and improve + performance of ProcessPoolExecutor. + (Contributed by Dan O'Reilly in :issue:`11271`.) + contextlib ---------- @@ -714,6 +742,12 @@ subdirectories using the "``**``" pattern. (Contributed by Serhiy Storchaka in :issue:`13968`.) +heapq +----- + +* :func:`~heapq.merge` has two new optional parameters ``reverse`` and + ``key``. (Contributed by Raymond Hettinger in :issue:`13742`.) + idlelib and IDLE ---------------- @@ -746,7 +780,9 @@ ------ * :func:`~imghdr.what` now recognizes the `OpenEXR `_ - format. (Contributed by Martin Vignali and Claudiu Popa in :issue:`20295`.) + format (contributed by Martin Vignali and Claudiu Popa in :issue:`20295`), + and the `WebP `_ format (contributed + by Fabrice Aneche and Claudiu Popa in :issue:`20197`.) importlib --------- @@ -763,6 +799,7 @@ * :func:`importlib.util.module_from_spec` is now the preferred way to create a new module. Compared to :class:`types.ModuleType`, this new function will set the various import-controlled attributes based on the passed-in spec object. + (Contributed by Brett Cannon in :issue:`20383`.) inspect ------- @@ -788,6 +825,10 @@ * New :func:`~inspect.getcoroutinelocals` and :func:`~inspect.getcoroutinestate` functions. (Contributed by Yury Selivanov in :issue:`24400`.) +* :func:`~inspect.stack`, :func:`~inspect.trace`, :func:`~inspect.getouterframes`, + and :func:`~inspect.getinnerframes` now return a list of named tuples. + (Contributed by Daniel Shahaf in :issue:`16808`.) + ipaddress --------- @@ -807,6 +848,35 @@ * JSON decoder now raises :exc:`json.JSONDecodeError` instead of :exc:`ValueError`. (Contributed by Serhiy Storchaka in :issue:`19361`.) +locale +------ + + * New :func:`~locale.delocalize` function to convert a string into a + normalized number string, following the ``LC_NUMERIC`` settings. + (Contributed by C?dric Krier in :issue:`13918`.) + +logging +------- + +* All logging methods :meth:`~logging.Logger.log`, :meth:`~logging.Logger.exception`, + :meth:`~logging.Logger.critical`, :meth:`~logging.Logger.debug`, etc, + now accept exception instances for ``exc_info`` parameter, in addition + to boolean values and exception tuples. + (Contributed by Yury Selivanov in :issue:`20537`.) + +* :class:`~logging.handlers.HTTPHandler` now accepts optional + :class:`ssl.SSLContext` instance to configure the SSL settings used + for HTTP connection. + (Contributed by Alex Gaynor in :issue:`22788`.) + +lzma +---- + +* New option *max_length* for :meth:`~lzma.LZMADecompressor.decompress` + to limit the maximum size of decompressed data. + (Contributed by Martin Panter in :issue:`15955`.) + + math ---- @@ -832,6 +902,10 @@ function is now used. These functions avoid the usage of an internal file descriptor. +* New :func:`os.get_blocking` and :func:`os.set_blocking` functions to + get and set the blocking mode of file descriptors. + (Contributed by Victor Stinner in :issue:`22054`.) + os.path ------- @@ -839,6 +913,25 @@ Unlike the :func:`~os.path.commonprefix` function, it always returns a valid path. (Contributed by Rafik Draoui and Serhiy Storchaka in :issue:`10395`.) +pathlib +------- + +* New :meth:`~pathlib.Path.samefile` method to check if other path object + points to the same file. (Contributed by Vajrasky Kok and Antoine Pitrou + in :issue:`19775`.) + +* :meth:`~pathlib.Path.mkdir` has a new optional parameter ``exist_ok`` + to mimic ``mkdir -p`` and :func:`os.makrdirs` functionality. + (Contributed by Berker Peksag in :issue:`21539`.) + +* New :meth:`~pathlib.Path.expanduser` to expand ``~`` and ``~user`` + constructs. + (Contributed by Serhiy Storchaka and Claudiu Popa in :issue:`19776`.) + +* New class method :meth:`~pathlib.Path.home` to get an instance of + :class:`~pathlib.Path` object representing the user?s home directory. + (Contributed by Victor Salgado and Mayank Tripathi in :issue:`19777`.) + pickle ------ @@ -862,6 +955,12 @@ * Now unmatched groups are replaced with empty strings in :func:`re.sub` and :func:`re.subn`. (Contributed by Serhiy Storchaka in :issue:`1519638`.) +readline +-------- + +* New :func:`~readline.append_history_file` function. + (Contributed by Bruno Cauet in :issue:`22940`.) + shutil ------ @@ -870,6 +969,9 @@ :func:`~shutil.copy2` if there is a need to ignore metadata. (Contributed by Claudiu Popa in :issue:`19840`.) +* :func:`~shutil.make_archive` now supports *xztar* format. + (Contributed by Serhiy Storchaka in :issue:`5411`.) + signal ------ @@ -948,6 +1050,14 @@ :meth:`SSLContext.wrap_bio ` method. (Contributed by Geert Jansen in :issue:`21965`.) +* New :meth:`~ssl.SSLSocket.version` to query the actual protocol version + in use. (Contributed by Antoine Pitrou in :issue:`20421`.) + +* New :meth:`~ssl.SSLObject.shared_ciphers` and + :meth:`~ssl.SSLSocket.shared_ciphers` methods to fetch the client's + list of ciphers sent at handshake. + (Contributed by Benjamin Peterson in :issue:`23186`.) + socket ------ @@ -961,6 +1071,9 @@ anymore each time bytes are received or sent. The socket timeout is now the maximum total duration to send all data. +* Functions with timeouts now use a monotonic clock, instead of a + system clock. (Contributed by Victor Stinner in :issue:`22043`.) + subprocess ---------- @@ -975,6 +1088,9 @@ * New :func:`~sys.set_coroutine_wrapper` and :func:`~sys.get_coroutine_wrapper` functions. (Contributed by Yury Selivanov in :issue:`24017`.) +* New :func:`~sys.is_finalizing` to check for :term:`interpreter shutdown`. + (Contributed by Antoine Pitrou in :issue:`22696`.) + sysconfig --------- @@ -991,9 +1107,16 @@ methods now take a keyword parameter *numeric_only*. If set to ``True``, the extracted files and directories will be owned by the numeric uid and gid from the tarfile. If set to ``False`` (the default, and the behavior in - versions prior to 3.5), they will be owned bythe named user and group in the + versions prior to 3.5), they will be owned by the named user and group in the tarfile. (Contributed by Michael Vogt and Eric Smith in :issue:`23193`.) +threading +--------- + +* :meth:`~threading.Lock.acquire` and :meth:`~threading.RLock.acquire` + now use a monotonic clock for managing timeouts. + (Contributed by Victor Stinner in :issue:`22043`.) + time ---- @@ -1008,6 +1131,17 @@ module which makes no permanent changes to environment variables. (Contributed by Zachary Ware in :issue:`20035`.) +traceback +--------- + +* New :func:`~traceback.walk_stack` and :func:`~traceback.walk_tb` + functions to conveniently traverse frame and traceback objects. + (Contributed by Robert Collins in :issue:`17911`.) + +* New lightweight classes: :class:`~traceback.TracebackException`, + :class:`~traceback.StackSummary`, and :class:`traceback.FrameSummary`. + (Contributed by Robert Collins in :issue:`17911`.) + types ----- @@ -1032,6 +1166,10 @@ control the encoding of query parts if needed. (Contributed by Samwyse and Arnon Yaari in :issue:`13866`.) +* :func:`~urllib.request.urlopen` accepts an :class:`ssl.SSLContext` + object as a *context* argument, which will be used for the HTTPS + connection. (Contributed by Alex Gaynor in :issue:`22366`.) + unicodedata ----------- @@ -1051,6 +1189,10 @@ * :class:`xmlrpc.client.ServerProxy` is now a :term:`context manager`. (Contributed by Claudiu Popa in :issue:`20627`.) +* :class:`~xmlrpc.client.ServerProxy` constructor now accepts an optional + :class:`ssl.SSLContext` instance. + (Contributed by Alex Gaynor in :issue:`22960`.) + xml.sax ------- @@ -1096,7 +1238,10 @@ :meth:`~ipaddress.IPv4Network.subnets`, :meth:`~ipaddress.IPv4Network.supernet`, :func:`~ipaddress.summarize_address_range`, :func:`~ipaddress.collapse_addresses`. The speed up can range from 3x to 15x. - (:issue:`21486`, :issue:`21487`, :issue:`20826`) + (See :issue:`21486`, :issue:`21487`, :issue:`20826`, :issue:`23266`.) + +* Pickling of :mod:`ipaddress` classes was optimized to produce significantly + smaller output. (Contributed by Serhiy Storchaka in :issue:`23133`.) * Many operations on :class:`io.BytesIO` are now 50% to 100% faster. (Contributed by Serhiy Storchaka in :issue:`15381` and David Wilson in @@ -1109,6 +1254,13 @@ * The UTF-32 encoder is now 3x to 7x faster. (Contributed by Serhiy Storchaka in :issue:`15027`.) +* Regular expressions are now parsed up to 10% faster. + (Contributed by Serhiy Storchaka in :issue:`19380`.) + +* :func:`json.dumps` was optimized to run with ``ensure_ascii=False`` + as fast as with ``ensure_ascii=True``. + (Contributed by Naoki Inada in :issue:`23206`.) + Build and C API Changes ======================= @@ -1185,6 +1337,9 @@ deprecated in favor of :func:`inspect.signature` API. (See :issue:`20438` for details.) +* Use of ``re.LOCALE`` flag with str patterns or ``re.ASCII`` is now + deprecated. (Contributed by Serhiy Storchaka in :issue:`22407`.) + Deprecated functions and types of the C API ------------------------------------------- -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 9 05:28:37 2015 From: python-checkins at python.org (yury.selivanov) Date: Wed, 09 Sep 2015 03:28:37 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogd2hhdHNuZXcvMy41?= =?utf-8?q?=3A_Better_formatting=3B_add_traceback_to_significantly_improve?= =?utf-8?q?d_stdlib?= Message-ID: <20150909032836.27687.11419@psf.io> https://hg.python.org/cpython/rev/b99fcf1e64b7 changeset: 97786:b99fcf1e64b7 branch: 3.5 parent: 97784:4a56a14ca4c1 user: Yury Selivanov date: Tue Sep 08 23:28:06 2015 -0400 summary: whatsnew/3.5: Better formatting; add traceback to significantly improved stdlib files: Doc/whatsnew/3.5.rst | 12 +++++++++++- 1 files changed, 11 insertions(+), 1 deletions(-) diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -83,13 +83,16 @@ New built-in features: * ``bytes % args``, ``bytearray % args``: :pep:`461` - Adding ``%`` formatting - to bytes and bytearray + to bytes and bytearray. + * ``b'\xf0\x9f\x90\x8d'.hex()``, ``bytearray(b'\xf0\x9f\x90\x8d').hex()``, ``memoryview(b'\xf0\x9f\x90\x8d').hex()``: :issue:`9951` - A ``hex`` method has been added to bytes, bytearray, and memoryview. + * Generators have new ``gi_yieldfrom`` attribute, which returns the object being iterated by ``yield from`` expressions. (Contributed by Benno Leslie and Yury Selivanov in :issue:`24450`.) + * New :exc:`RecursionError` exception. (Contributed by Georg Brandl in :issue:`19235`.) @@ -101,6 +104,7 @@ (:issue:`19977`). * :pep:`488`, the elimination of ``.pyo`` files. + * :pep:`489`, multi-phase initialization of extension modules. Significantly Improved Library Modules: @@ -120,6 +124,11 @@ protocol handling from network IO. (Contributed by Geert Jansen in :issue:`21965`.) +* :mod:`traceback` has new lightweight and convenient to work with + classes :class:`~traceback.TracebackException`, + :class:`~traceback.StackSummary`, and :class:`traceback.FrameSummary`. + (Contributed by Robert Collins in :issue:`17911`.) + Security improvements: * SSLv3 is now disabled throughout the standard library. @@ -135,6 +144,7 @@ * A new installer for Windows has replaced the old MSI. See :ref:`using-on-windows` for more information. + * Windows builds now use Microsoft Visual C++ 14.0, and extension modules should use the same. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 9 05:28:37 2015 From: python-checkins at python.org (yury.selivanov) Date: Wed, 09 Sep 2015 03:28:37 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogTWVyZ2UgMy41?= Message-ID: <20150909032837.115052.48365@psf.io> https://hg.python.org/cpython/rev/71815c9bb860 changeset: 97787:71815c9bb860 parent: 97785:07c206831e28 parent: 97786:b99fcf1e64b7 user: Yury Selivanov date: Tue Sep 08 23:28:17 2015 -0400 summary: Merge 3.5 files: Doc/whatsnew/3.5.rst | 12 +++++++++++- 1 files changed, 11 insertions(+), 1 deletions(-) diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -83,13 +83,16 @@ New built-in features: * ``bytes % args``, ``bytearray % args``: :pep:`461` - Adding ``%`` formatting - to bytes and bytearray + to bytes and bytearray. + * ``b'\xf0\x9f\x90\x8d'.hex()``, ``bytearray(b'\xf0\x9f\x90\x8d').hex()``, ``memoryview(b'\xf0\x9f\x90\x8d').hex()``: :issue:`9951` - A ``hex`` method has been added to bytes, bytearray, and memoryview. + * Generators have new ``gi_yieldfrom`` attribute, which returns the object being iterated by ``yield from`` expressions. (Contributed by Benno Leslie and Yury Selivanov in :issue:`24450`.) + * New :exc:`RecursionError` exception. (Contributed by Georg Brandl in :issue:`19235`.) @@ -101,6 +104,7 @@ (:issue:`19977`). * :pep:`488`, the elimination of ``.pyo`` files. + * :pep:`489`, multi-phase initialization of extension modules. Significantly Improved Library Modules: @@ -120,6 +124,11 @@ protocol handling from network IO. (Contributed by Geert Jansen in :issue:`21965`.) +* :mod:`traceback` has new lightweight and convenient to work with + classes :class:`~traceback.TracebackException`, + :class:`~traceback.StackSummary`, and :class:`traceback.FrameSummary`. + (Contributed by Robert Collins in :issue:`17911`.) + Security improvements: * SSLv3 is now disabled throughout the standard library. @@ -135,6 +144,7 @@ * A new installer for Windows has replaced the old MSI. See :ref:`using-on-windows` for more information. + * Windows builds now use Microsoft Visual C++ 14.0, and extension modules should use the same. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 9 05:41:35 2015 From: python-checkins at python.org (yury.selivanov) Date: Wed, 09 Sep 2015 03:41:35 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogTWVyZ2UgMy41?= Message-ID: <20150909034135.27699.25408@psf.io> https://hg.python.org/cpython/rev/6c3c3d4b505b changeset: 97789:6c3c3d4b505b parent: 97787:71815c9bb860 parent: 97788:0ea528234bc5 user: Yury Selivanov date: Tue Sep 08 23:41:22 2015 -0400 summary: Merge 3.5 files: Doc/whatsnew/3.5.rst | 103 ++++++++++-------------------- 1 files changed, 35 insertions(+), 68 deletions(-) diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -48,7 +48,6 @@ when researching a change. This article explains the new features in Python 3.5, compared to 3.4. - For full details, see the :source:`Misc/NEWS` file. .. note:: @@ -148,24 +147,9 @@ * Windows builds now use Microsoft Visual C++ 14.0, and extension modules should use the same. - -Please read on for a comprehensive list of user-facing changes. - - -.. PEP-sized items next. - -.. _pep-4XX: - -.. PEP 4XX: Virtual Environments -.. ============================= - - -.. (Implemented by Foo Bar.) - -.. .. seealso:: - - :pep:`4XX` - Python Virtual Environments - PEP written by Carl Meyer +Please read on for a comprehensive list of user-facing changes, including many +other smaller improvements, CPython optimizations, deprecations, and potential +porting issues. New Features @@ -277,7 +261,12 @@ mathematics, science, engineering, and the addition of ``@`` allows writing cleaner code:: - >>> S = (H @ beta - r).T @ inv(H @ V @ H.T) @ (H @ beta - r) + S = (H @ beta - r).T @ inv(H @ V @ H.T) @ (H @ beta - r) + +instead of:: + + S = dot((dot(H, beta) - r).T, + dot(inv(dot(dot(H, V), H.T)), dot(H, beta) - r)) An upcoming release of NumPy 1.10 will add support for the new operator:: @@ -421,60 +410,38 @@ instead of raising :exc:`InterruptedError` if the Python signal handler does not raise an exception: -* :func:`open`, :func:`os.open`, :func:`io.open` -* functions of the :mod:`faulthandler` module -* :mod:`os` functions: +* :func:`open`, :func:`os.open`, :func:`io.open`; - - :func:`os.fchdir` - - :func:`os.fchmod` - - :func:`os.fchown` - - :func:`os.fdatasync` - - :func:`os.fstat` - - :func:`os.fstatvfs` - - :func:`os.fsync` - - :func:`os.ftruncate` - - :func:`os.mkfifo` - - :func:`os.mknod` - - :func:`os.posix_fadvise` - - :func:`os.posix_fallocate` - - :func:`os.pread` - - :func:`os.pwrite` - - :func:`os.read` - - :func:`os.readv` - - :func:`os.sendfile` - - :func:`os.wait3` - - :func:`os.wait4` - - :func:`os.wait` - - :func:`os.waitid` - - :func:`os.waitpid` - - :func:`os.write` - - :func:`os.writev` - - special cases: :func:`os.close` and :func:`os.dup2` now ignore - :py:data:`~errno.EINTR` error, the syscall is not retried (see the PEP - for the rationale) +* functions of the :mod:`faulthandler` module; -* :mod:`select` functions: +* :mod:`os` functions: :func:`~os.fchdir`, :func:`~os.fchmod`, + :func:`~os.fchown`, :func:`~os.fdatasync`, :func:`~os.fstat`, + :func:`~os.fstatvfs`, :func:`~os.fsync`, :func:`~os.ftruncate`, + :func:`~os.mkfifo`, :func:`~os.mknod`, :func:`~os.posix_fadvise`, + :func:`~os.posix_fallocate`, :func:`~os.pread`, :func:`~os.pwrite`, + :func:`~os.read`, :func:`~os.readv`, :func:`~os.sendfile`, + :func:`~os.wait3`, :func:`~os.wait4`, :func:`~os.wait`, + :func:`~os.waitid`, :func:`~os.waitpid`, :func:`~os.write`, + :func:`~os.writev`; - - :func:`select.devpoll.poll` - - :func:`select.epoll.poll` - - :func:`select.kqueue.control` - - :func:`select.poll.poll` - - :func:`select.select` +* special cases: :func:`os.close` and :func:`os.dup2` now ignore + :py:data:`~errno.EINTR` error, the syscall is not retried (see the PEP + for the rationale); -* :func:`socket.socket` methods: +* :mod:`select` functions: :func:`~select.devpoll.poll`, + :func:`~select.epoll.poll`, :func:`~select.kqueue.control`, + :func:`~select.poll.poll`, :func:`~select.select`; - - :meth:`~socket.socket.accept` - - :meth:`~socket.socket.connect` (except for non-blocking sockets) - - :meth:`~socket.socket.recv` - - :meth:`~socket.socket.recvfrom` - - :meth:`~socket.socket.recvmsg` - - :meth:`~socket.socket.send` - - :meth:`~socket.socket.sendall` - - :meth:`~socket.socket.sendmsg` - - :meth:`~socket.socket.sendto` +* :func:`socket.socket` methods: :meth:`~socket.socket.accept`, + :meth:`~socket.socket.connect` (except for non-blocking sockets), + :meth:`~socket.socket.recv`, :meth:`~socket.socket.recvfrom`, + :meth:`~socket.socket.recvmsg`, :meth:`~socket.socket.send`, + :meth:`~socket.socket.sendall`, :meth:`~socket.socket.sendmsg`, + :meth:`~socket.socket.sendto`; -* :func:`signal.sigtimedwait`, :func:`signal.sigwaitinfo` -* :func:`time.sleep` +* :func:`signal.sigtimedwait`, :func:`signal.sigwaitinfo`; + +* :func:`time.sleep`. .. seealso:: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 9 05:41:35 2015 From: python-checkins at python.org (yury.selivanov) Date: Wed, 09 Sep 2015 03:41:35 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogd2hhdHNuZXcvMy41?= =?utf-8?q?=3A_Reformat_PEP_475_to_render_in_less_space=3B_add_=22ugly=22_?= =?utf-8?q?ex_in_465?= Message-ID: <20150909034134.15730.32960@psf.io> https://hg.python.org/cpython/rev/0ea528234bc5 changeset: 97788:0ea528234bc5 branch: 3.5 parent: 97786:b99fcf1e64b7 user: Yury Selivanov date: Tue Sep 08 23:40:46 2015 -0400 summary: whatsnew/3.5: Reformat PEP 475 to render in less space; add "ugly" ex in 465 files: Doc/whatsnew/3.5.rst | 103 ++++++++++-------------------- 1 files changed, 35 insertions(+), 68 deletions(-) diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -48,7 +48,6 @@ when researching a change. This article explains the new features in Python 3.5, compared to 3.4. - For full details, see the :source:`Misc/NEWS` file. .. note:: @@ -148,24 +147,9 @@ * Windows builds now use Microsoft Visual C++ 14.0, and extension modules should use the same. - -Please read on for a comprehensive list of user-facing changes. - - -.. PEP-sized items next. - -.. _pep-4XX: - -.. PEP 4XX: Virtual Environments -.. ============================= - - -.. (Implemented by Foo Bar.) - -.. .. seealso:: - - :pep:`4XX` - Python Virtual Environments - PEP written by Carl Meyer +Please read on for a comprehensive list of user-facing changes, including many +other smaller improvements, CPython optimizations, deprecations, and potential +porting issues. New Features @@ -277,7 +261,12 @@ mathematics, science, engineering, and the addition of ``@`` allows writing cleaner code:: - >>> S = (H @ beta - r).T @ inv(H @ V @ H.T) @ (H @ beta - r) + S = (H @ beta - r).T @ inv(H @ V @ H.T) @ (H @ beta - r) + +instead of:: + + S = dot((dot(H, beta) - r).T, + dot(inv(dot(dot(H, V), H.T)), dot(H, beta) - r)) An upcoming release of NumPy 1.10 will add support for the new operator:: @@ -421,60 +410,38 @@ instead of raising :exc:`InterruptedError` if the Python signal handler does not raise an exception: -* :func:`open`, :func:`os.open`, :func:`io.open` -* functions of the :mod:`faulthandler` module -* :mod:`os` functions: +* :func:`open`, :func:`os.open`, :func:`io.open`; - - :func:`os.fchdir` - - :func:`os.fchmod` - - :func:`os.fchown` - - :func:`os.fdatasync` - - :func:`os.fstat` - - :func:`os.fstatvfs` - - :func:`os.fsync` - - :func:`os.ftruncate` - - :func:`os.mkfifo` - - :func:`os.mknod` - - :func:`os.posix_fadvise` - - :func:`os.posix_fallocate` - - :func:`os.pread` - - :func:`os.pwrite` - - :func:`os.read` - - :func:`os.readv` - - :func:`os.sendfile` - - :func:`os.wait3` - - :func:`os.wait4` - - :func:`os.wait` - - :func:`os.waitid` - - :func:`os.waitpid` - - :func:`os.write` - - :func:`os.writev` - - special cases: :func:`os.close` and :func:`os.dup2` now ignore - :py:data:`~errno.EINTR` error, the syscall is not retried (see the PEP - for the rationale) +* functions of the :mod:`faulthandler` module; -* :mod:`select` functions: +* :mod:`os` functions: :func:`~os.fchdir`, :func:`~os.fchmod`, + :func:`~os.fchown`, :func:`~os.fdatasync`, :func:`~os.fstat`, + :func:`~os.fstatvfs`, :func:`~os.fsync`, :func:`~os.ftruncate`, + :func:`~os.mkfifo`, :func:`~os.mknod`, :func:`~os.posix_fadvise`, + :func:`~os.posix_fallocate`, :func:`~os.pread`, :func:`~os.pwrite`, + :func:`~os.read`, :func:`~os.readv`, :func:`~os.sendfile`, + :func:`~os.wait3`, :func:`~os.wait4`, :func:`~os.wait`, + :func:`~os.waitid`, :func:`~os.waitpid`, :func:`~os.write`, + :func:`~os.writev`; - - :func:`select.devpoll.poll` - - :func:`select.epoll.poll` - - :func:`select.kqueue.control` - - :func:`select.poll.poll` - - :func:`select.select` +* special cases: :func:`os.close` and :func:`os.dup2` now ignore + :py:data:`~errno.EINTR` error, the syscall is not retried (see the PEP + for the rationale); -* :func:`socket.socket` methods: +* :mod:`select` functions: :func:`~select.devpoll.poll`, + :func:`~select.epoll.poll`, :func:`~select.kqueue.control`, + :func:`~select.poll.poll`, :func:`~select.select`; - - :meth:`~socket.socket.accept` - - :meth:`~socket.socket.connect` (except for non-blocking sockets) - - :meth:`~socket.socket.recv` - - :meth:`~socket.socket.recvfrom` - - :meth:`~socket.socket.recvmsg` - - :meth:`~socket.socket.send` - - :meth:`~socket.socket.sendall` - - :meth:`~socket.socket.sendmsg` - - :meth:`~socket.socket.sendto` +* :func:`socket.socket` methods: :meth:`~socket.socket.accept`, + :meth:`~socket.socket.connect` (except for non-blocking sockets), + :meth:`~socket.socket.recv`, :meth:`~socket.socket.recvfrom`, + :meth:`~socket.socket.recvmsg`, :meth:`~socket.socket.send`, + :meth:`~socket.socket.sendall`, :meth:`~socket.socket.sendmsg`, + :meth:`~socket.socket.sendto`; -* :func:`signal.sigtimedwait`, :func:`signal.sigwaitinfo` -* :func:`time.sleep` +* :func:`signal.sigtimedwait`, :func:`signal.sigwaitinfo`; + +* :func:`time.sleep`. .. seealso:: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 9 05:55:58 2015 From: python-checkins at python.org (martin.panter) Date: Wed, 09 Sep 2015 03:55:58 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy41IC0+IDMuNSk6?= =?utf-8?q?_Merge_3=2E5_branches?= Message-ID: <20150909035557.14885.20341@psf.io> https://hg.python.org/cpython/rev/981102699c00 changeset: 97792:981102699c00 branch: 3.5 parent: 97791:40bf1ca3f715 parent: 97784:4a56a14ca4c1 user: Martin Panter date: Wed Sep 09 02:51:53 2015 +0000 summary: Merge 3.5 branches files: Doc/library/compileall.rst | 12 +- Doc/library/imghdr.rst | 4 +- Doc/library/inspect.rst | 20 +++ Doc/whatsnew/3.5.rst | 163 ++++++++++++++++++++++++- 4 files changed, 185 insertions(+), 14 deletions(-) diff --git a/Doc/library/compileall.rst b/Doc/library/compileall.rst --- a/Doc/library/compileall.rst +++ b/Doc/library/compileall.rst @@ -88,14 +88,12 @@ Added the ``-i``, ``-b`` and ``-h`` options. .. versionchanged:: 3.5 - Added the ``-j`` and ``-r`` options. -.. versionchanged:: 3.5 - ``-q`` option was changed to a multilevel value. - -.. versionchanged:: 3.5 - ``-b`` will always produce a byte-code file ending in ``.pyc``, never - ``.pyo``. + * Added the ``-j`` and ``-r`` options. + * ``-q`` option was changed to a multilevel value. + * ``-qq`` option. + * ``-b`` will always produce a byte-code file ending in ``.pyc``, + never ``.pyo``. There is no command-line option to control the optimization level used by the diff --git a/Doc/library/imghdr.rst b/Doc/library/imghdr.rst --- a/Doc/library/imghdr.rst +++ b/Doc/library/imghdr.rst @@ -54,10 +54,8 @@ +------------+-----------------------------------+ .. versionadded:: 3.5 - The *exr* format was added. + The *exr* and *webp* formats were added. -.. versionchanged:: 3.5 - The *webp* type was added. You can extend the list of file types :mod:`imghdr` can recognize by appending to this variable: diff --git a/Doc/library/inspect.rst b/Doc/library/inspect.rst --- a/Doc/library/inspect.rst +++ b/Doc/library/inspect.rst @@ -1039,6 +1039,11 @@ returned list represents *frame*; the last entry represents the outermost call on *frame*'s stack. + .. versionchanged:: 3.5 + A list of :term:`named tuples ` + ``FrameInfo(frame, filename, lineno, function, code_context, index)`` + is returned. + .. function:: getinnerframes(traceback, context=1) @@ -1047,6 +1052,11 @@ list represents *traceback*; the last entry represents where the exception was raised. + .. versionchanged:: 3.5 + A list of :term:`named tuples ` + ``FrameInfo(frame, filename, lineno, function, code_context, index)`` + is returned. + .. function:: currentframe() @@ -1066,6 +1076,11 @@ returned list represents the caller; the last entry represents the outermost call on the stack. + .. versionchanged:: 3.5 + A list of :term:`named tuples ` + ``FrameInfo(frame, filename, lineno, function, code_context, index)`` + is returned. + .. function:: trace(context=1) @@ -1074,6 +1089,11 @@ entry in the list represents the caller; the last entry represents where the exception was raised. + .. versionchanged:: 3.5 + A list of :term:`named tuples ` + ``FrameInfo(frame, filename, lineno, function, code_context, index)`` + is returned. + Fetching attributes statically ------------------------------ diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -122,7 +122,14 @@ Security improvements: -* None yet. +* SSLv3 is now disabled throughout the standard library. + It can still be enabled by instantiating a :class:`ssl.SSLContext` + manually. (See :issue:`22638` for more details; this change was + backported to CPython 3.4 and 2.7.) + +* HTTP cookie parsing is now stricter, in order to protect + against potential injection attacks. (Contributed by Antoine Pitrou + in :issue:`22796`.) Windows improvements: @@ -606,6 +613,13 @@ :ref:`allow_abbrev` to ``False``. (Contributed by Jonathan Paugh, Steven Bethard, paul j3 and Daniel Eriksson.) +bz2 +--- + +* New option *max_length* for :meth:`~bz2.BZ2Decompressor.decompress` + to limit the maximum size of decompressed data. + (Contributed by Nikolaus Rath in :issue:`15955`.) + cgi --- @@ -645,6 +659,20 @@ can now do parallel bytecode compilation. (Contributed by Claudiu Popa in :issue:`16104`.) +* *quiet* parameter of :func:`compileall.compile_dir`, + :func:`compileall.compile_file`, and :func:`compileall.compile_path` + functions now has a multilevel value. New ``-qq`` command line option + is available for suppressing the output. + (Contributed by Thomas Kluyver in :issue:`21338`.) + +concurrent.futures +------------------ + +* :meth:`~concurrent.futures.Executor.map` now takes a *chunksize* + argument to allow batching of tasks in child processes and improve + performance of ProcessPoolExecutor. + (Contributed by Dan O'Reilly in :issue:`11271`.) + contextlib ---------- @@ -714,6 +742,12 @@ subdirectories using the "``**``" pattern. (Contributed by Serhiy Storchaka in :issue:`13968`.) +heapq +----- + +* :func:`~heapq.merge` has two new optional parameters ``reverse`` and + ``key``. (Contributed by Raymond Hettinger in :issue:`13742`.) + idlelib and IDLE ---------------- @@ -746,7 +780,9 @@ ------ * :func:`~imghdr.what` now recognizes the `OpenEXR `_ - format. (Contributed by Martin Vignali and Claudiu Popa in :issue:`20295`.) + format (contributed by Martin Vignali and Claudiu Popa in :issue:`20295`), + and the `WebP `_ format (contributed + by Fabrice Aneche and Claudiu Popa in :issue:`20197`.) importlib --------- @@ -763,6 +799,7 @@ * :func:`importlib.util.module_from_spec` is now the preferred way to create a new module. Compared to :class:`types.ModuleType`, this new function will set the various import-controlled attributes based on the passed-in spec object. + (Contributed by Brett Cannon in :issue:`20383`.) inspect ------- @@ -788,6 +825,10 @@ * New :func:`~inspect.getcoroutinelocals` and :func:`~inspect.getcoroutinestate` functions. (Contributed by Yury Selivanov in :issue:`24400`.) +* :func:`~inspect.stack`, :func:`~inspect.trace`, :func:`~inspect.getouterframes`, + and :func:`~inspect.getinnerframes` now return a list of named tuples. + (Contributed by Daniel Shahaf in :issue:`16808`.) + ipaddress --------- @@ -807,6 +848,35 @@ * JSON decoder now raises :exc:`json.JSONDecodeError` instead of :exc:`ValueError`. (Contributed by Serhiy Storchaka in :issue:`19361`.) +locale +------ + + * New :func:`~locale.delocalize` function to convert a string into a + normalized number string, following the ``LC_NUMERIC`` settings. + (Contributed by C?dric Krier in :issue:`13918`.) + +logging +------- + +* All logging methods :meth:`~logging.Logger.log`, :meth:`~logging.Logger.exception`, + :meth:`~logging.Logger.critical`, :meth:`~logging.Logger.debug`, etc, + now accept exception instances for ``exc_info`` parameter, in addition + to boolean values and exception tuples. + (Contributed by Yury Selivanov in :issue:`20537`.) + +* :class:`~logging.handlers.HTTPHandler` now accepts optional + :class:`ssl.SSLContext` instance to configure the SSL settings used + for HTTP connection. + (Contributed by Alex Gaynor in :issue:`22788`.) + +lzma +---- + +* New option *max_length* for :meth:`~lzma.LZMADecompressor.decompress` + to limit the maximum size of decompressed data. + (Contributed by Martin Panter in :issue:`15955`.) + + math ---- @@ -832,6 +902,10 @@ function is now used. These functions avoid the usage of an internal file descriptor. +* New :func:`os.get_blocking` and :func:`os.set_blocking` functions to + get and set the blocking mode of file descriptors. + (Contributed by Victor Stinner in :issue:`22054`.) + os.path ------- @@ -839,6 +913,25 @@ Unlike the :func:`~os.path.commonprefix` function, it always returns a valid path. (Contributed by Rafik Draoui and Serhiy Storchaka in :issue:`10395`.) +pathlib +------- + +* New :meth:`~pathlib.Path.samefile` method to check if other path object + points to the same file. (Contributed by Vajrasky Kok and Antoine Pitrou + in :issue:`19775`.) + +* :meth:`~pathlib.Path.mkdir` has a new optional parameter ``exist_ok`` + to mimic ``mkdir -p`` and :func:`os.makrdirs` functionality. + (Contributed by Berker Peksag in :issue:`21539`.) + +* New :meth:`~pathlib.Path.expanduser` to expand ``~`` and ``~user`` + constructs. + (Contributed by Serhiy Storchaka and Claudiu Popa in :issue:`19776`.) + +* New class method :meth:`~pathlib.Path.home` to get an instance of + :class:`~pathlib.Path` object representing the user?s home directory. + (Contributed by Victor Salgado and Mayank Tripathi in :issue:`19777`.) + pickle ------ @@ -862,6 +955,12 @@ * Now unmatched groups are replaced with empty strings in :func:`re.sub` and :func:`re.subn`. (Contributed by Serhiy Storchaka in :issue:`1519638`.) +readline +-------- + +* New :func:`~readline.append_history_file` function. + (Contributed by Bruno Cauet in :issue:`22940`.) + shutil ------ @@ -870,6 +969,9 @@ :func:`~shutil.copy2` if there is a need to ignore metadata. (Contributed by Claudiu Popa in :issue:`19840`.) +* :func:`~shutil.make_archive` now supports *xztar* format. + (Contributed by Serhiy Storchaka in :issue:`5411`.) + signal ------ @@ -948,6 +1050,14 @@ :meth:`SSLContext.wrap_bio ` method. (Contributed by Geert Jansen in :issue:`21965`.) +* New :meth:`~ssl.SSLSocket.version` to query the actual protocol version + in use. (Contributed by Antoine Pitrou in :issue:`20421`.) + +* New :meth:`~ssl.SSLObject.shared_ciphers` and + :meth:`~ssl.SSLSocket.shared_ciphers` methods to fetch the client's + list of ciphers sent at handshake. + (Contributed by Benjamin Peterson in :issue:`23186`.) + socket ------ @@ -961,6 +1071,9 @@ anymore each time bytes are received or sent. The socket timeout is now the maximum total duration to send all data. +* Functions with timeouts now use a monotonic clock, instead of a + system clock. (Contributed by Victor Stinner in :issue:`22043`.) + subprocess ---------- @@ -975,6 +1088,9 @@ * New :func:`~sys.set_coroutine_wrapper` and :func:`~sys.get_coroutine_wrapper` functions. (Contributed by Yury Selivanov in :issue:`24017`.) +* New :func:`~sys.is_finalizing` to check for :term:`interpreter shutdown`. + (Contributed by Antoine Pitrou in :issue:`22696`.) + sysconfig --------- @@ -991,9 +1107,16 @@ methods now take a keyword parameter *numeric_only*. If set to ``True``, the extracted files and directories will be owned by the numeric uid and gid from the tarfile. If set to ``False`` (the default, and the behavior in - versions prior to 3.5), they will be owned bythe named user and group in the + versions prior to 3.5), they will be owned by the named user and group in the tarfile. (Contributed by Michael Vogt and Eric Smith in :issue:`23193`.) +threading +--------- + +* :meth:`~threading.Lock.acquire` and :meth:`~threading.RLock.acquire` + now use a monotonic clock for managing timeouts. + (Contributed by Victor Stinner in :issue:`22043`.) + time ---- @@ -1008,6 +1131,17 @@ module which makes no permanent changes to environment variables. (Contributed by Zachary Ware in :issue:`20035`.) +traceback +--------- + +* New :func:`~traceback.walk_stack` and :func:`~traceback.walk_tb` + functions to conveniently traverse frame and traceback objects. + (Contributed by Robert Collins in :issue:`17911`.) + +* New lightweight classes: :class:`~traceback.TracebackException`, + :class:`~traceback.StackSummary`, and :class:`traceback.FrameSummary`. + (Contributed by Robert Collins in :issue:`17911`.) + types ----- @@ -1032,6 +1166,10 @@ control the encoding of query parts if needed. (Contributed by Samwyse and Arnon Yaari in :issue:`13866`.) +* :func:`~urllib.request.urlopen` accepts an :class:`ssl.SSLContext` + object as a *context* argument, which will be used for the HTTPS + connection. (Contributed by Alex Gaynor in :issue:`22366`.) + unicodedata ----------- @@ -1051,6 +1189,10 @@ * :class:`xmlrpc.client.ServerProxy` is now a :term:`context manager`. (Contributed by Claudiu Popa in :issue:`20627`.) +* :class:`~xmlrpc.client.ServerProxy` constructor now accepts an optional + :class:`ssl.SSLContext` instance. + (Contributed by Alex Gaynor in :issue:`22960`.) + xml.sax ------- @@ -1096,7 +1238,10 @@ :meth:`~ipaddress.IPv4Network.subnets`, :meth:`~ipaddress.IPv4Network.supernet`, :func:`~ipaddress.summarize_address_range`, :func:`~ipaddress.collapse_addresses`. The speed up can range from 3x to 15x. - (:issue:`21486`, :issue:`21487`, :issue:`20826`) + (See :issue:`21486`, :issue:`21487`, :issue:`20826`, :issue:`23266`.) + +* Pickling of :mod:`ipaddress` classes was optimized to produce significantly + smaller output. (Contributed by Serhiy Storchaka in :issue:`23133`.) * Many operations on :class:`io.BytesIO` are now 50% to 100% faster. (Contributed by Serhiy Storchaka in :issue:`15381` and David Wilson in @@ -1109,6 +1254,13 @@ * The UTF-32 encoder is now 3x to 7x faster. (Contributed by Serhiy Storchaka in :issue:`15027`.) +* Regular expressions are now parsed up to 10% faster. + (Contributed by Serhiy Storchaka in :issue:`19380`.) + +* :func:`json.dumps` was optimized to run with ``ensure_ascii=False`` + as fast as with ``ensure_ascii=True``. + (Contributed by Naoki Inada in :issue:`23206`.) + Build and C API Changes ======================= @@ -1185,6 +1337,9 @@ deprecated in favor of :func:`inspect.signature` API. (See :issue:`20438` for details.) +* Use of ``re.LOCALE`` flag with str patterns or ``re.ASCII`` is now + deprecated. (Contributed by Serhiy Storchaka in :issue:`22407`.) + Deprecated functions and types of the C API ------------------------------------------- -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 9 05:55:58 2015 From: python-checkins at python.org (martin.panter) Date: Wed, 09 Sep 2015 03:55:58 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2323738=3A_Merge_3=2E5_into_3=2E6?= Message-ID: <20150909035558.27687.90255@psf.io> https://hg.python.org/cpython/rev/6177482ce6a1 changeset: 97793:6177482ce6a1 parent: 97785:07c206831e28 parent: 97792:981102699c00 user: Martin Panter date: Wed Sep 09 03:01:17 2015 +0000 summary: Issue #23738: Merge 3.5 into 3.6 files: Doc/library/binascii.rst | 2 +- Doc/library/os.rst | 39 +++++++++++---------- Doc/library/zlib.rst | 8 ++-- Lib/test/test_binascii.py | 3 + Lib/test/test_os.py | 23 ++++++++++++- Lib/test/test_popen.py | 4 ++ Lib/test/test_posix.py | 8 ++++ Lib/test/test_zlib.py | 8 +++- Modules/clinic/posixmodule.c.h | 6 +- Modules/posixmodule.c | 15 ++++--- 10 files changed, 79 insertions(+), 37 deletions(-) diff --git a/Doc/library/binascii.rst b/Doc/library/binascii.rst --- a/Doc/library/binascii.rst +++ b/Doc/library/binascii.rst @@ -59,7 +59,7 @@ should be at most 57 to adhere to the base64 standard. -.. function:: a2b_qp(string, header=False) +.. function:: a2b_qp(data, header=False) Convert a block of quoted-printable data back to binary and return the binary data. More than one line may be passed at a time. If the optional argument diff --git a/Doc/library/os.rst b/Doc/library/os.rst --- a/Doc/library/os.rst +++ b/Doc/library/os.rst @@ -857,9 +857,9 @@ :data:`os.SEEK_HOLE` or :data:`os.SEEK_DATA`. -.. function:: open(file, flags, mode=0o777, *, dir_fd=None) - - Open the file *file* and set various flags according to *flags* and possibly +.. function:: open(path, flags, mode=0o777, *, dir_fd=None) + + Open the file *path* and set various flags according to *flags* and possibly its mode according to *mode*. When computing *mode*, the current umask value is first masked out. Return the file descriptor for the newly opened file. The new file descriptor is :ref:`non-inheritable `. @@ -1071,10 +1071,10 @@ :exc:`InterruptedError` exception (see :pep:`475` for the rationale). -.. function:: sendfile(out, in, offset, nbytes) - sendfile(out, in, offset, nbytes, headers=None, trailers=None, flags=0) - - Copy *nbytes* bytes from file descriptor *in* to file descriptor *out* +.. function:: sendfile(out, in, offset, count) + sendfile(out, in, offset, count, headers=None, trailers=None, flags=0) + + Copy *count* bytes from file descriptor *in* to file descriptor *out* starting at *offset*. Return the number of bytes sent. When EOF is reached return 0. @@ -1088,7 +1088,7 @@ *trailers* are arbitrary sequences of buffers that are written before and after the data from *in* is written. It returns the same as the first case. - On Mac OS X and FreeBSD, a value of 0 for *nbytes* specifies to send until + On Mac OS X and FreeBSD, a value of 0 for *count* specifies to send until the end of *in* is reached. All platforms support sockets as *out* file descriptor, and some platforms @@ -1690,10 +1690,10 @@ The *dir_fd* argument. -.. function:: mknod(filename, mode=0o600, device=0, *, dir_fd=None) +.. function:: mknod(path, mode=0o600, device=0, *, dir_fd=None) Create a filesystem node (file, device special file or named pipe) named - *filename*. *mode* specifies both the permissions to use and the type of node + *path*. *mode* specifies both the permissions to use and the type of node to be created, being combined (bitwise OR) with one of ``stat.S_IFREG``, ``stat.S_IFCHR``, ``stat.S_IFBLK``, and ``stat.S_IFIFO`` (those constants are available in :mod:`stat`). For ``stat.S_IFCHR`` and ``stat.S_IFBLK``, @@ -2389,9 +2389,9 @@ .. versionadded:: 3.3 -.. function:: symlink(source, link_name, target_is_directory=False, *, dir_fd=None) - - Create a symbolic link pointing to *source* named *link_name*. +.. function:: symlink(src, dst, target_is_directory=False, *, dir_fd=None) + + Create a symbolic link pointing to *src* named *dst*. On Windows, a symlink represents either a file or a directory, and does not morph to the target dynamically. If the target is present, the type of the @@ -2461,20 +2461,20 @@ The *dir_fd* parameter. -.. function:: utime(path, times=None, *, ns=None, dir_fd=None, follow_symlinks=True) +.. function:: utime(path, times=None, *[, ns], dir_fd=None, follow_symlinks=True) Set the access and modified times of the file specified by *path*. :func:`utime` takes two optional parameters, *times* and *ns*. These specify the times set on *path* and are used as follows: - - If *ns* is not ``None``, + - If *ns* is specified, it must be a 2-tuple of the form ``(atime_ns, mtime_ns)`` where each member is an int expressing nanoseconds. - If *times* is not ``None``, it must be a 2-tuple of the form ``(atime, mtime)`` where each member is an int or float expressing seconds. - - If *times* and *ns* are both ``None``, + - If *times* is ``None`` and *ns* is unspecified, this is equivalent to specifying ``ns=(atime_ns, mtime_ns)`` where both times are the current time. @@ -3023,9 +3023,10 @@ Availability: Unix. -.. function:: popen(command, mode='r', buffering=-1) - - Open a pipe to or from *command*. The return value is an open file object +.. function:: popen(cmd, mode='r', buffering=-1) + + Open a pipe to or from command *cmd*. + The return value is an open file object connected to the pipe, which can be read or written depending on whether *mode* is ``'r'`` (default) or ``'w'``. The *buffering* argument has the same meaning as the corresponding argument to the built-in :func:`open` function. The diff --git a/Doc/library/zlib.rst b/Doc/library/zlib.rst --- a/Doc/library/zlib.rst +++ b/Doc/library/zlib.rst @@ -58,7 +58,7 @@ Raises the :exc:`error` exception if any error occurs. -.. function:: compressobj(level=-1, method=DEFLATED, wbits=15, memlevel=8, strategy=Z_DEFAULT_STRATEGY[, zdict]) +.. function:: compressobj(level=-1, method=DEFLATED, wbits=15, memLevel=8, strategy=Z_DEFAULT_STRATEGY[, zdict]) Returns a compression object, to be used for compressing data streams that won't fit into memory at once. @@ -75,9 +75,9 @@ should be an integer from ``8`` to ``15``. Higher values give better compression, but use more memory. - *memlevel* controls the amount of memory used for internal compression state. - Valid values range from ``1`` to ``9``. Higher values using more memory, - but are faster and produce smaller output. + The *memLevel* argument controls the amount of memory used for the + internal compression state. Valid values range from ``1`` to ``9``. + Higher values use more memory, but are faster and produce smaller output. *strategy* is used to tune the compression algorithm. Possible values are ``Z_DEFAULT_STRATEGY``, ``Z_FILTERED``, and ``Z_HUFFMAN_ONLY``. diff --git a/Lib/test/test_binascii.py b/Lib/test/test_binascii.py --- a/Lib/test/test_binascii.py +++ b/Lib/test/test_binascii.py @@ -178,6 +178,8 @@ self.assertEqual(binascii.unhexlify(self.type2test(t)), u) def test_qp(self): + binascii.a2b_qp(data=b"", header=False) # Keyword arguments allowed + # A test for SF bug 534347 (segfaults without the proper fix) try: binascii.a2b_qp(b"", **{1:1}) @@ -185,6 +187,7 @@ pass else: self.fail("binascii.a2b_qp(**{1:1}) didn't raise TypeError") + self.assertEqual(binascii.a2b_qp(b"= "), b"= ") self.assertEqual(binascii.a2b_qp(b"=="), b"=") self.assertEqual(binascii.a2b_qp(b"=AX"), b"=AX") diff --git a/Lib/test/test_os.py b/Lib/test/test_os.py --- a/Lib/test/test_os.py +++ b/Lib/test/test_os.py @@ -85,7 +85,7 @@ # Tests creating TESTFN class FileTests(unittest.TestCase): def setUp(self): - if os.path.exists(support.TESTFN): + if os.path.lexists(support.TESTFN): os.unlink(support.TESTFN) tearDown = setUp @@ -209,6 +209,19 @@ with open(TESTFN2, 'r') as f: self.assertEqual(f.read(), "1") + def test_open_keywords(self): + f = os.open(path=__file__, flags=os.O_RDONLY, mode=0o777, + dir_fd=None) + os.close(f) + + def test_symlink_keywords(self): + symlink = support.get_attribute(os, "symlink") + try: + symlink(src='target', dst=support.TESTFN, + target_is_directory=False, dir_fd=None) + except (NotImplementedError, OSError): + pass # No OS support or unprivileged user + # Test attributes on return values from os.*stat* family. class StatAttributeTests(unittest.TestCase): @@ -2312,6 +2325,14 @@ os.sendfile(self.sockno, self.fileno, -1, 4096) self.assertEqual(cm.exception.errno, errno.EINVAL) + def test_keywords(self): + # Keyword arguments should be supported + os.sendfile(out=self.sockno, offset=0, count=4096, + **{'in': self.fileno}) + if self.SUPPORT_HEADERS_TRAILERS: + os.sendfile(self.sockno, self.fileno, offset=0, count=4096, + headers=None, trailers=None, flags=0) + # --- headers / trailers tests @requires_headers_trailers diff --git a/Lib/test/test_popen.py b/Lib/test/test_popen.py --- a/Lib/test/test_popen.py +++ b/Lib/test/test_popen.py @@ -57,5 +57,9 @@ with os.popen("echo hello") as f: self.assertEqual(list(f), ["hello\n"]) + def test_keywords(self): + with os.popen(cmd="exit 0", mode="w", buffering=-1): + pass + if __name__ == "__main__": unittest.main() diff --git a/Lib/test/test_posix.py b/Lib/test/test_posix.py --- a/Lib/test/test_posix.py +++ b/Lib/test/test_posix.py @@ -442,6 +442,14 @@ else: self.assertTrue(stat.S_ISFIFO(posix.stat(support.TESTFN).st_mode)) + # Keyword arguments are also supported + support.unlink(support.TESTFN) + try: + posix.mknod(path=support.TESTFN, mode=mode, device=0, + dir_fd=None) + except OSError as e: + self.assertIn(e.errno, (errno.EPERM, errno.EINVAL)) + @unittest.skipUnless(hasattr(posix, 'stat'), 'test needs posix.stat()') @unittest.skipUnless(hasattr(posix, 'makedev'), 'test needs posix.makedev()') def test_makedev(self): diff --git a/Lib/test/test_zlib.py b/Lib/test/test_zlib.py --- a/Lib/test/test_zlib.py +++ b/Lib/test/test_zlib.py @@ -222,9 +222,9 @@ level = 2 method = zlib.DEFLATED wbits = -12 - memlevel = 9 + memLevel = 9 strategy = zlib.Z_FILTERED - co = zlib.compressobj(level, method, wbits, memlevel, strategy) + co = zlib.compressobj(level, method, wbits, memLevel, strategy) x1 = co.compress(HAMLET_SCENE) x2 = co.flush() dco = zlib.decompressobj(wbits) @@ -232,6 +232,10 @@ y2 = dco.flush() self.assertEqual(HAMLET_SCENE, y1 + y2) + # keyword arguments should also be supported + zlib.compressobj(level=level, method=method, wbits=wbits, + memLevel=memLevel, strategy=strategy, zdict=b"") + def test_compressincremental(self): # compress object in steps, decompress object as one-shot data = HAMLET_SCENE * 128 diff --git a/Modules/clinic/posixmodule.c.h b/Modules/clinic/posixmodule.c.h --- a/Modules/clinic/posixmodule.c.h +++ b/Modules/clinic/posixmodule.c.h @@ -1487,10 +1487,10 @@ "\n" "If times is not None, it must be a tuple (atime, mtime);\n" " atime and mtime should be expressed as float seconds since the epoch.\n" -"If ns is not None, it must be a tuple (atime_ns, mtime_ns);\n" +"If ns is specified, it must be a tuple (atime_ns, mtime_ns);\n" " atime_ns and mtime_ns should be expressed as integer nanoseconds\n" " since the epoch.\n" -"If both times and ns are None, utime uses the current time.\n" +"If times is None and ns is unspecified, utime uses the current time.\n" "Specifying tuples for both times and ns is an error.\n" "\n" "If dir_fd is not None, it should be a file descriptor open to a directory,\n" @@ -5792,4 +5792,4 @@ #ifndef OS_SET_HANDLE_INHERITABLE_METHODDEF #define OS_SET_HANDLE_INHERITABLE_METHODDEF #endif /* !defined(OS_SET_HANDLE_INHERITABLE_METHODDEF) */ -/*[clinic end generated code: output=35b50461dbecd65d input=a9049054013a1b77]*/ +/*[clinic end generated code: output=a5c9bef9ad11a20b input=a9049054013a1b77]*/ diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c --- a/Modules/posixmodule.c +++ b/Modules/posixmodule.c @@ -4679,7 +4679,7 @@ dir_fd: dir_fd(requires='futimensat') = None follow_symlinks: bool=True -# "utime(path, times=None, *, ns=None, dir_fd=None, follow_symlinks=True)\n\ +# "utime(path, times=None, *[, ns], dir_fd=None, follow_symlinks=True)\n\ Set the access and modified time of path. @@ -4689,10 +4689,10 @@ If times is not None, it must be a tuple (atime, mtime); atime and mtime should be expressed as float seconds since the epoch. -If ns is not None, it must be a tuple (atime_ns, mtime_ns); +If ns is specified, it must be a tuple (atime_ns, mtime_ns); atime_ns and mtime_ns should be expressed as integer nanoseconds since the epoch. -If both times and ns are None, utime uses the current time. +If times is None and ns is unspecified, utime uses the current time. Specifying tuples for both times and ns is an error. If dir_fd is not None, it should be a file descriptor open to a directory, @@ -4710,7 +4710,7 @@ static PyObject * os_utime_impl(PyModuleDef *module, path_t *path, PyObject *times, PyObject *ns, int dir_fd, int follow_symlinks) -/*[clinic end generated code: output=31f3434e560ba2f0 input=1f18c17d5941aa82]*/ +/*[clinic end generated code: output=31f3434e560ba2f0 input=081cdc54ca685385]*/ { #ifdef MS_WINDOWS HANDLE hFile; @@ -8235,10 +8235,10 @@ #ifdef HAVE_SENDFILE PyDoc_STRVAR(posix_sendfile__doc__, -"sendfile(out, in, offset, nbytes) -> byteswritten\n\ -sendfile(out, in, offset, nbytes, headers=None, trailers=None, flags=0)\n\ +"sendfile(out, in, offset, count) -> byteswritten\n\ +sendfile(out, in, offset, count, headers=None, trailers=None, flags=0)\n\ -> byteswritten\n\ -Copy nbytes bytes from file descriptor in to file descriptor out."); +Copy count bytes from file descriptor in to file descriptor out."); /* AC 3.5: don't bother converting, has optional group*/ static PyObject * @@ -8258,6 +8258,7 @@ off_t sbytes; struct sf_hdtr sf; int flags = 0; + /* Beware that "in" clashes with Python's own "in" operator keyword */ static char *keywords[] = {"out", "in", "offset", "count", "headers", "trailers", "flags", NULL}; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 9 05:55:57 2015 From: python-checkins at python.org (martin.panter) Date: Wed, 09 Sep 2015 03:55:57 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Issue_=2323738=3A_Merge_3=2E4_into_3=2E5?= Message-ID: <20150909035557.115020.63808@psf.io> https://hg.python.org/cpython/rev/40bf1ca3f715 changeset: 97791:40bf1ca3f715 branch: 3.5 parent: 97779:60f5ca4c5a01 parent: 97790:fdb5d84f9948 user: Martin Panter date: Wed Sep 09 01:56:53 2015 +0000 summary: Issue #23738: Merge 3.4 into 3.5 files: Doc/library/binascii.rst | 2 +- Doc/library/os.rst | 39 +++++++++++---------- Doc/library/zlib.rst | 8 ++-- Lib/test/test_binascii.py | 3 + Lib/test/test_os.py | 23 ++++++++++++- Lib/test/test_popen.py | 4 ++ Lib/test/test_posix.py | 8 ++++ Lib/test/test_zlib.py | 8 +++- Modules/clinic/posixmodule.c.h | 6 +- Modules/posixmodule.c | 15 ++++--- 10 files changed, 79 insertions(+), 37 deletions(-) diff --git a/Doc/library/binascii.rst b/Doc/library/binascii.rst --- a/Doc/library/binascii.rst +++ b/Doc/library/binascii.rst @@ -59,7 +59,7 @@ should be at most 57 to adhere to the base64 standard. -.. function:: a2b_qp(string, header=False) +.. function:: a2b_qp(data, header=False) Convert a block of quoted-printable data back to binary and return the binary data. More than one line may be passed at a time. If the optional argument diff --git a/Doc/library/os.rst b/Doc/library/os.rst --- a/Doc/library/os.rst +++ b/Doc/library/os.rst @@ -857,9 +857,9 @@ :data:`os.SEEK_HOLE` or :data:`os.SEEK_DATA`. -.. function:: open(file, flags, mode=0o777, *, dir_fd=None) - - Open the file *file* and set various flags according to *flags* and possibly +.. function:: open(path, flags, mode=0o777, *, dir_fd=None) + + Open the file *path* and set various flags according to *flags* and possibly its mode according to *mode*. When computing *mode*, the current umask value is first masked out. Return the file descriptor for the newly opened file. The new file descriptor is :ref:`non-inheritable `. @@ -1071,10 +1071,10 @@ :exc:`InterruptedError` exception (see :pep:`475` for the rationale). -.. function:: sendfile(out, in, offset, nbytes) - sendfile(out, in, offset, nbytes, headers=None, trailers=None, flags=0) - - Copy *nbytes* bytes from file descriptor *in* to file descriptor *out* +.. function:: sendfile(out, in, offset, count) + sendfile(out, in, offset, count, headers=None, trailers=None, flags=0) + + Copy *count* bytes from file descriptor *in* to file descriptor *out* starting at *offset*. Return the number of bytes sent. When EOF is reached return 0. @@ -1088,7 +1088,7 @@ *trailers* are arbitrary sequences of buffers that are written before and after the data from *in* is written. It returns the same as the first case. - On Mac OS X and FreeBSD, a value of 0 for *nbytes* specifies to send until + On Mac OS X and FreeBSD, a value of 0 for *count* specifies to send until the end of *in* is reached. All platforms support sockets as *out* file descriptor, and some platforms @@ -1690,10 +1690,10 @@ The *dir_fd* argument. -.. function:: mknod(filename, mode=0o600, device=0, *, dir_fd=None) +.. function:: mknod(path, mode=0o600, device=0, *, dir_fd=None) Create a filesystem node (file, device special file or named pipe) named - *filename*. *mode* specifies both the permissions to use and the type of node + *path*. *mode* specifies both the permissions to use and the type of node to be created, being combined (bitwise OR) with one of ``stat.S_IFREG``, ``stat.S_IFCHR``, ``stat.S_IFBLK``, and ``stat.S_IFIFO`` (those constants are available in :mod:`stat`). For ``stat.S_IFCHR`` and ``stat.S_IFBLK``, @@ -2389,9 +2389,9 @@ .. versionadded:: 3.3 -.. function:: symlink(source, link_name, target_is_directory=False, *, dir_fd=None) - - Create a symbolic link pointing to *source* named *link_name*. +.. function:: symlink(src, dst, target_is_directory=False, *, dir_fd=None) + + Create a symbolic link pointing to *src* named *dst*. On Windows, a symlink represents either a file or a directory, and does not morph to the target dynamically. If the target is present, the type of the @@ -2461,20 +2461,20 @@ The *dir_fd* parameter. -.. function:: utime(path, times=None, *, ns=None, dir_fd=None, follow_symlinks=True) +.. function:: utime(path, times=None, *[, ns], dir_fd=None, follow_symlinks=True) Set the access and modified times of the file specified by *path*. :func:`utime` takes two optional parameters, *times* and *ns*. These specify the times set on *path* and are used as follows: - - If *ns* is not ``None``, + - If *ns* is specified, it must be a 2-tuple of the form ``(atime_ns, mtime_ns)`` where each member is an int expressing nanoseconds. - If *times* is not ``None``, it must be a 2-tuple of the form ``(atime, mtime)`` where each member is an int or float expressing seconds. - - If *times* and *ns* are both ``None``, + - If *times* is ``None`` and *ns* is unspecified, this is equivalent to specifying ``ns=(atime_ns, mtime_ns)`` where both times are the current time. @@ -3023,9 +3023,10 @@ Availability: Unix. -.. function:: popen(command, mode='r', buffering=-1) - - Open a pipe to or from *command*. The return value is an open file object +.. function:: popen(cmd, mode='r', buffering=-1) + + Open a pipe to or from command *cmd*. + The return value is an open file object connected to the pipe, which can be read or written depending on whether *mode* is ``'r'`` (default) or ``'w'``. The *buffering* argument has the same meaning as the corresponding argument to the built-in :func:`open` function. The diff --git a/Doc/library/zlib.rst b/Doc/library/zlib.rst --- a/Doc/library/zlib.rst +++ b/Doc/library/zlib.rst @@ -58,7 +58,7 @@ Raises the :exc:`error` exception if any error occurs. -.. function:: compressobj(level=-1, method=DEFLATED, wbits=15, memlevel=8, strategy=Z_DEFAULT_STRATEGY[, zdict]) +.. function:: compressobj(level=-1, method=DEFLATED, wbits=15, memLevel=8, strategy=Z_DEFAULT_STRATEGY[, zdict]) Returns a compression object, to be used for compressing data streams that won't fit into memory at once. @@ -75,9 +75,9 @@ should be an integer from ``8`` to ``15``. Higher values give better compression, but use more memory. - *memlevel* controls the amount of memory used for internal compression state. - Valid values range from ``1`` to ``9``. Higher values using more memory, - but are faster and produce smaller output. + The *memLevel* argument controls the amount of memory used for the + internal compression state. Valid values range from ``1`` to ``9``. + Higher values use more memory, but are faster and produce smaller output. *strategy* is used to tune the compression algorithm. Possible values are ``Z_DEFAULT_STRATEGY``, ``Z_FILTERED``, and ``Z_HUFFMAN_ONLY``. diff --git a/Lib/test/test_binascii.py b/Lib/test/test_binascii.py --- a/Lib/test/test_binascii.py +++ b/Lib/test/test_binascii.py @@ -178,6 +178,8 @@ self.assertEqual(binascii.unhexlify(self.type2test(t)), u) def test_qp(self): + binascii.a2b_qp(data=b"", header=False) # Keyword arguments allowed + # A test for SF bug 534347 (segfaults without the proper fix) try: binascii.a2b_qp(b"", **{1:1}) @@ -185,6 +187,7 @@ pass else: self.fail("binascii.a2b_qp(**{1:1}) didn't raise TypeError") + self.assertEqual(binascii.a2b_qp(b"= "), b"= ") self.assertEqual(binascii.a2b_qp(b"=="), b"=") self.assertEqual(binascii.a2b_qp(b"=AX"), b"=AX") diff --git a/Lib/test/test_os.py b/Lib/test/test_os.py --- a/Lib/test/test_os.py +++ b/Lib/test/test_os.py @@ -85,7 +85,7 @@ # Tests creating TESTFN class FileTests(unittest.TestCase): def setUp(self): - if os.path.exists(support.TESTFN): + if os.path.lexists(support.TESTFN): os.unlink(support.TESTFN) tearDown = setUp @@ -209,6 +209,19 @@ with open(TESTFN2, 'r') as f: self.assertEqual(f.read(), "1") + def test_open_keywords(self): + f = os.open(path=__file__, flags=os.O_RDONLY, mode=0o777, + dir_fd=None) + os.close(f) + + def test_symlink_keywords(self): + symlink = support.get_attribute(os, "symlink") + try: + symlink(src='target', dst=support.TESTFN, + target_is_directory=False, dir_fd=None) + except (NotImplementedError, OSError): + pass # No OS support or unprivileged user + # Test attributes on return values from os.*stat* family. class StatAttributeTests(unittest.TestCase): @@ -2313,6 +2326,14 @@ os.sendfile(self.sockno, self.fileno, -1, 4096) self.assertEqual(cm.exception.errno, errno.EINVAL) + def test_keywords(self): + # Keyword arguments should be supported + os.sendfile(out=self.sockno, offset=0, count=4096, + **{'in': self.fileno}) + if self.SUPPORT_HEADERS_TRAILERS: + os.sendfile(self.sockno, self.fileno, offset=0, count=4096, + headers=None, trailers=None, flags=0) + # --- headers / trailers tests @requires_headers_trailers diff --git a/Lib/test/test_popen.py b/Lib/test/test_popen.py --- a/Lib/test/test_popen.py +++ b/Lib/test/test_popen.py @@ -57,5 +57,9 @@ with os.popen("echo hello") as f: self.assertEqual(list(f), ["hello\n"]) + def test_keywords(self): + with os.popen(cmd="exit 0", mode="w", buffering=-1): + pass + if __name__ == "__main__": unittest.main() diff --git a/Lib/test/test_posix.py b/Lib/test/test_posix.py --- a/Lib/test/test_posix.py +++ b/Lib/test/test_posix.py @@ -442,6 +442,14 @@ else: self.assertTrue(stat.S_ISFIFO(posix.stat(support.TESTFN).st_mode)) + # Keyword arguments are also supported + support.unlink(support.TESTFN) + try: + posix.mknod(path=support.TESTFN, mode=mode, device=0, + dir_fd=None) + except OSError as e: + self.assertIn(e.errno, (errno.EPERM, errno.EINVAL)) + @unittest.skipUnless(hasattr(posix, 'stat'), 'test needs posix.stat()') @unittest.skipUnless(hasattr(posix, 'makedev'), 'test needs posix.makedev()') def test_makedev(self): diff --git a/Lib/test/test_zlib.py b/Lib/test/test_zlib.py --- a/Lib/test/test_zlib.py +++ b/Lib/test/test_zlib.py @@ -222,9 +222,9 @@ level = 2 method = zlib.DEFLATED wbits = -12 - memlevel = 9 + memLevel = 9 strategy = zlib.Z_FILTERED - co = zlib.compressobj(level, method, wbits, memlevel, strategy) + co = zlib.compressobj(level, method, wbits, memLevel, strategy) x1 = co.compress(HAMLET_SCENE) x2 = co.flush() dco = zlib.decompressobj(wbits) @@ -232,6 +232,10 @@ y2 = dco.flush() self.assertEqual(HAMLET_SCENE, y1 + y2) + # keyword arguments should also be supported + zlib.compressobj(level=level, method=method, wbits=wbits, + memLevel=memLevel, strategy=strategy, zdict=b"") + def test_compressincremental(self): # compress object in steps, decompress object as one-shot data = HAMLET_SCENE * 128 diff --git a/Modules/clinic/posixmodule.c.h b/Modules/clinic/posixmodule.c.h --- a/Modules/clinic/posixmodule.c.h +++ b/Modules/clinic/posixmodule.c.h @@ -1487,10 +1487,10 @@ "\n" "If times is not None, it must be a tuple (atime, mtime);\n" " atime and mtime should be expressed as float seconds since the epoch.\n" -"If ns is not None, it must be a tuple (atime_ns, mtime_ns);\n" +"If ns is specified, it must be a tuple (atime_ns, mtime_ns);\n" " atime_ns and mtime_ns should be expressed as integer nanoseconds\n" " since the epoch.\n" -"If both times and ns are None, utime uses the current time.\n" +"If times is None and ns is unspecified, utime uses the current time.\n" "Specifying tuples for both times and ns is an error.\n" "\n" "If dir_fd is not None, it should be a file descriptor open to a directory,\n" @@ -5788,4 +5788,4 @@ #ifndef OS_SET_HANDLE_INHERITABLE_METHODDEF #define OS_SET_HANDLE_INHERITABLE_METHODDEF #endif /* !defined(OS_SET_HANDLE_INHERITABLE_METHODDEF) */ -/*[clinic end generated code: output=f3f92b2d2e2c3fe3 input=a9049054013a1b77]*/ +/*[clinic end generated code: output=95824c52fd034654 input=a9049054013a1b77]*/ diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c --- a/Modules/posixmodule.c +++ b/Modules/posixmodule.c @@ -4679,7 +4679,7 @@ dir_fd: dir_fd(requires='futimensat') = None follow_symlinks: bool=True -# "utime(path, times=None, *, ns=None, dir_fd=None, follow_symlinks=True)\n\ +# "utime(path, times=None, *[, ns], dir_fd=None, follow_symlinks=True)\n\ Set the access and modified time of path. @@ -4689,10 +4689,10 @@ If times is not None, it must be a tuple (atime, mtime); atime and mtime should be expressed as float seconds since the epoch. -If ns is not None, it must be a tuple (atime_ns, mtime_ns); +If ns is specified, it must be a tuple (atime_ns, mtime_ns); atime_ns and mtime_ns should be expressed as integer nanoseconds since the epoch. -If both times and ns are None, utime uses the current time. +If times is None and ns is unspecified, utime uses the current time. Specifying tuples for both times and ns is an error. If dir_fd is not None, it should be a file descriptor open to a directory, @@ -4710,7 +4710,7 @@ static PyObject * os_utime_impl(PyModuleDef *module, path_t *path, PyObject *times, PyObject *ns, int dir_fd, int follow_symlinks) -/*[clinic end generated code: output=31f3434e560ba2f0 input=1f18c17d5941aa82]*/ +/*[clinic end generated code: output=31f3434e560ba2f0 input=081cdc54ca685385]*/ { #ifdef MS_WINDOWS HANDLE hFile; @@ -8235,10 +8235,10 @@ #ifdef HAVE_SENDFILE PyDoc_STRVAR(posix_sendfile__doc__, -"sendfile(out, in, offset, nbytes) -> byteswritten\n\ -sendfile(out, in, offset, nbytes, headers=None, trailers=None, flags=0)\n\ +"sendfile(out, in, offset, count) -> byteswritten\n\ +sendfile(out, in, offset, count, headers=None, trailers=None, flags=0)\n\ -> byteswritten\n\ -Copy nbytes bytes from file descriptor in to file descriptor out."); +Copy count bytes from file descriptor in to file descriptor out."); /* AC 3.5: don't bother converting, has optional group*/ static PyObject * @@ -8258,6 +8258,7 @@ off_t sbytes; struct sf_hdtr sf; int flags = 0; + /* Beware that "in" clashes with Python's own "in" operator keyword */ static char *keywords[] = {"out", "in", "offset", "count", "headers", "trailers", "flags", NULL}; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 9 05:55:57 2015 From: python-checkins at python.org (martin.panter) Date: Wed, 09 Sep 2015 03:55:57 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzIzNzM4?= =?utf-8?q?=3A_Document_and_test_actual_keyword_parameter_names?= Message-ID: <20150909035557.115118.66175@psf.io> https://hg.python.org/cpython/rev/fdb5d84f9948 changeset: 97790:fdb5d84f9948 branch: 3.4 parent: 97775:bbf72b164720 user: Martin Panter date: Wed Sep 09 01:01:13 2015 +0000 summary: Issue #23738: Document and test actual keyword parameter names Also fix signature because os.utime(..., ns=None) is not allowed. files: Doc/library/binascii.rst | 2 +- Doc/library/os.rst | 39 +++++++++++++------------- Doc/library/zlib.rst | 8 ++-- Lib/test/test_binascii.py | 3 ++ Lib/test/test_os.py | 23 +++++++++++++++- Lib/test/test_popen.py | 4 ++ Lib/test/test_posix.py | 8 +++++ Lib/test/test_zlib.py | 8 ++++- Modules/posixmodule.c | 17 ++++++----- 9 files changed, 77 insertions(+), 35 deletions(-) diff --git a/Doc/library/binascii.rst b/Doc/library/binascii.rst --- a/Doc/library/binascii.rst +++ b/Doc/library/binascii.rst @@ -59,7 +59,7 @@ should be at most 57 to adhere to the base64 standard. -.. function:: a2b_qp(string, header=False) +.. function:: a2b_qp(data, header=False) Convert a block of quoted-printable data back to binary and return the binary data. More than one line may be passed at a time. If the optional argument diff --git a/Doc/library/os.rst b/Doc/library/os.rst --- a/Doc/library/os.rst +++ b/Doc/library/os.rst @@ -863,9 +863,9 @@ :data:`os.SEEK_HOLE` or :data:`os.SEEK_DATA`. -.. function:: open(file, flags, mode=0o777, *, dir_fd=None) - - Open the file *file* and set various flags according to *flags* and possibly +.. function:: open(path, flags, mode=0o777, *, dir_fd=None) + + Open the file *path* and set various flags according to *flags* and possibly its mode according to *mode*. When computing *mode*, the current umask value is first masked out. Return the file descriptor for the newly opened file. The new file descriptor is :ref:`non-inheritable `. @@ -1071,10 +1071,10 @@ :meth:`~file.read` or :meth:`~file.readline` methods. -.. function:: sendfile(out, in, offset, nbytes) - sendfile(out, in, offset, nbytes, headers=None, trailers=None, flags=0) - - Copy *nbytes* bytes from file descriptor *in* to file descriptor *out* +.. function:: sendfile(out, in, offset, count) + sendfile(out, in, offset, count, headers=None, trailers=None, flags=0) + + Copy *count* bytes from file descriptor *in* to file descriptor *out* starting at *offset*. Return the number of bytes sent. When EOF is reached return 0. @@ -1088,7 +1088,7 @@ *trailers* are arbitrary sequences of buffers that are written before and after the data from *in* is written. It returns the same as the first case. - On Mac OS X and FreeBSD, a value of 0 for *nbytes* specifies to send until + On Mac OS X and FreeBSD, a value of 0 for *count* specifies to send until the end of *in* is reached. All platforms support sockets as *out* file descriptor, and some platforms @@ -1683,10 +1683,10 @@ The *dir_fd* argument. -.. function:: mknod(filename, mode=0o600, device=0, *, dir_fd=None) +.. function:: mknod(path, mode=0o600, device=0, *, dir_fd=None) Create a filesystem node (file, device special file or named pipe) named - *filename*. *mode* specifies both the permissions to use and the type of node + *path*. *mode* specifies both the permissions to use and the type of node to be created, being combined (bitwise OR) with one of ``stat.S_IFREG``, ``stat.S_IFCHR``, ``stat.S_IFBLK``, and ``stat.S_IFIFO`` (those constants are available in :mod:`stat`). For ``stat.S_IFCHR`` and ``stat.S_IFBLK``, @@ -2210,9 +2210,9 @@ .. versionadded:: 3.3 -.. function:: symlink(source, link_name, target_is_directory=False, *, dir_fd=None) - - Create a symbolic link pointing to *source* named *link_name*. +.. function:: symlink(src, dst, target_is_directory=False, *, dir_fd=None) + + Create a symbolic link pointing to *src* named *dst*. On Windows, a symlink represents either a file or a directory, and does not morph to the target dynamically. If the target is present, the type of the @@ -2282,20 +2282,20 @@ The *dir_fd* parameter. -.. function:: utime(path, times=None, *, ns=None, dir_fd=None, follow_symlinks=True) +.. function:: utime(path, times=None, *[, ns], dir_fd=None, follow_symlinks=True) Set the access and modified times of the file specified by *path*. :func:`utime` takes two optional parameters, *times* and *ns*. These specify the times set on *path* and are used as follows: - - If *ns* is not ``None``, + - If *ns* is specified, it must be a 2-tuple of the form ``(atime_ns, mtime_ns)`` where each member is an int expressing nanoseconds. - If *times* is not ``None``, it must be a 2-tuple of the form ``(atime, mtime)`` where each member is an int or float expressing seconds. - - If *times* and *ns* are both ``None``, + - If *times* is ``None`` and *ns* is unspecified, this is equivalent to specifying ``ns=(atime_ns, mtime_ns)`` where both times are the current time. @@ -2846,9 +2846,10 @@ Availability: Unix. -.. function:: popen(command, mode='r', buffering=-1) - - Open a pipe to or from *command*. The return value is an open file object +.. function:: popen(cmd, mode='r', buffering=-1) + + Open a pipe to or from command *cmd*. + The return value is an open file object connected to the pipe, which can be read or written depending on whether *mode* is ``'r'`` (default) or ``'w'``. The *buffering* argument has the same meaning as the corresponding argument to the built-in :func:`open` function. The diff --git a/Doc/library/zlib.rst b/Doc/library/zlib.rst --- a/Doc/library/zlib.rst +++ b/Doc/library/zlib.rst @@ -58,7 +58,7 @@ Raises the :exc:`error` exception if any error occurs. -.. function:: compressobj(level=-1, method=DEFLATED, wbits=15, memlevel=8, strategy=Z_DEFAULT_STRATEGY[, zdict]) +.. function:: compressobj(level=-1, method=DEFLATED, wbits=15, memLevel=8, strategy=Z_DEFAULT_STRATEGY[, zdict]) Returns a compression object, to be used for compressing data streams that won't fit into memory at once. @@ -75,9 +75,9 @@ should be an integer from ``8`` to ``15``. Higher values give better compression, but use more memory. - *memlevel* controls the amount of memory used for internal compression state. - Valid values range from ``1`` to ``9``. Higher values using more memory, - but are faster and produce smaller output. + The *memLevel* argument controls the amount of memory used for the + internal compression state. Valid values range from ``1`` to ``9``. + Higher values use more memory, but are faster and produce smaller output. *strategy* is used to tune the compression algorithm. Possible values are ``Z_DEFAULT_STRATEGY``, ``Z_FILTERED``, and ``Z_HUFFMAN_ONLY``. diff --git a/Lib/test/test_binascii.py b/Lib/test/test_binascii.py --- a/Lib/test/test_binascii.py +++ b/Lib/test/test_binascii.py @@ -179,6 +179,8 @@ self.assertEqual(binascii.unhexlify(self.type2test(t)), u) def test_qp(self): + binascii.a2b_qp(data=b"", header=False) # Keyword arguments allowed + # A test for SF bug 534347 (segfaults without the proper fix) try: binascii.a2b_qp(b"", **{1:1}) @@ -186,6 +188,7 @@ pass else: self.fail("binascii.a2b_qp(**{1:1}) didn't raise TypeError") + self.assertEqual(binascii.a2b_qp(b"= "), b"= ") self.assertEqual(binascii.a2b_qp(b"=="), b"=") self.assertEqual(binascii.a2b_qp(b"=AX"), b"=AX") diff --git a/Lib/test/test_os.py b/Lib/test/test_os.py --- a/Lib/test/test_os.py +++ b/Lib/test/test_os.py @@ -58,7 +58,7 @@ # Tests creating TESTFN class FileTests(unittest.TestCase): def setUp(self): - if os.path.exists(support.TESTFN): + if os.path.lexists(support.TESTFN): os.unlink(support.TESTFN) tearDown = setUp @@ -162,6 +162,19 @@ with open(TESTFN2, 'r') as f: self.assertEqual(f.read(), "1") + def test_open_keywords(self): + f = os.open(path=__file__, flags=os.O_RDONLY, mode=0o777, + dir_fd=None) + os.close(f) + + def test_symlink_keywords(self): + symlink = support.get_attribute(os, "symlink") + try: + symlink(src='target', dst=support.TESTFN, + target_is_directory=False, dir_fd=None) + except (NotImplementedError, OSError): + pass # No OS support or unprivileged user + # Test attributes on return values from os.*stat* family. class StatAttributeTests(unittest.TestCase): @@ -2151,6 +2164,14 @@ os.sendfile(self.sockno, self.fileno, -1, 4096) self.assertEqual(cm.exception.errno, errno.EINVAL) + def test_keywords(self): + # Keyword arguments should be supported + os.sendfile(out=self.sockno, offset=0, count=4096, + **{'in': self.fileno}) + if self.SUPPORT_HEADERS_TRAILERS: + os.sendfile(self.sockno, self.fileno, offset=0, count=4096, + headers=None, trailers=None, flags=0) + # --- headers / trailers tests @requires_headers_trailers diff --git a/Lib/test/test_popen.py b/Lib/test/test_popen.py --- a/Lib/test/test_popen.py +++ b/Lib/test/test_popen.py @@ -57,6 +57,10 @@ with os.popen("echo hello") as f: self.assertEqual(list(f), ["hello\n"]) + def test_keywords(self): + with os.popen(cmd="exit 0", mode="w", buffering=-1): + pass + def test_main(): support.run_unittest(PopenTest) diff --git a/Lib/test/test_posix.py b/Lib/test/test_posix.py --- a/Lib/test/test_posix.py +++ b/Lib/test/test_posix.py @@ -443,6 +443,14 @@ else: self.assertTrue(stat.S_ISFIFO(posix.stat(support.TESTFN).st_mode)) + # Keyword arguments are also supported + support.unlink(support.TESTFN) + try: + posix.mknod(path=support.TESTFN, mode=mode, device=0, + dir_fd=None) + except OSError as e: + self.assertIn(e.errno, (errno.EPERM, errno.EINVAL)) + @unittest.skipUnless(hasattr(posix, 'stat'), 'test needs posix.stat()') @unittest.skipUnless(hasattr(posix, 'makedev'), 'test needs posix.makedev()') def test_makedev(self): diff --git a/Lib/test/test_zlib.py b/Lib/test/test_zlib.py --- a/Lib/test/test_zlib.py +++ b/Lib/test/test_zlib.py @@ -222,9 +222,9 @@ level = 2 method = zlib.DEFLATED wbits = -12 - memlevel = 9 + memLevel = 9 strategy = zlib.Z_FILTERED - co = zlib.compressobj(level, method, wbits, memlevel, strategy) + co = zlib.compressobj(level, method, wbits, memLevel, strategy) x1 = co.compress(HAMLET_SCENE) x2 = co.flush() dco = zlib.decompressobj(wbits) @@ -232,6 +232,10 @@ y2 = dco.flush() self.assertEqual(HAMLET_SCENE, y1 + y2) + # keyword arguments should also be supported + zlib.compressobj(level=level, method=method, wbits=wbits, + memLevel=memLevel, strategy=strategy, zdict=b"") + def test_compressincremental(self): # compress object in steps, decompress object as one-shot data = HAMLET_SCENE * 128 diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c --- a/Modules/posixmodule.c +++ b/Modules/posixmodule.c @@ -4695,7 +4695,7 @@ PyDoc_STRVAR(posix_utime__doc__, -"utime(path, times=None, *, ns=None, dir_fd=None, follow_symlinks=True)\n\ +"utime(path, times=None, *[, ns], dir_fd=None, follow_symlinks=True)\n\ Set the access and modified time of path.\n\ \n\ path may always be specified as a string.\n\ @@ -4704,10 +4704,10 @@ \n\ If times is not None, it must be a tuple (atime, mtime);\n\ atime and mtime should be expressed as float seconds since the epoch.\n\ -If ns is not None, it must be a tuple (atime_ns, mtime_ns);\n\ +If ns is specified, it must be a tuple (atime_ns, mtime_ns);\n\ atime_ns and mtime_ns should be expressed as integer nanoseconds\n\ since the epoch.\n\ -If both times and ns are None, utime uses the current time.\n\ +If times is None and ns is unspecified, utime uses the current time.\n\ Specifying tuples for both times and ns is an error.\n\ \n\ If dir_fd is not None, it should be a file descriptor open to a directory,\n\ @@ -8245,10 +8245,10 @@ #ifdef HAVE_SENDFILE PyDoc_STRVAR(posix_sendfile__doc__, -"sendfile(out, in, offset, nbytes) -> byteswritten\n\ -sendfile(out, in, offset, nbytes, headers=None, trailers=None, flags=0)\n\ +"sendfile(out, in, offset, count) -> byteswritten\n\ +sendfile(out, in, offset, count, headers=None, trailers=None, flags=0)\n\ -> byteswritten\n\ -Copy nbytes bytes from file descriptor in to file descriptor out."); +Copy count bytes from file descriptor in to file descriptor out."); static PyObject * posix_sendfile(PyObject *self, PyObject *args, PyObject *kwdict) @@ -8266,6 +8266,7 @@ off_t sbytes; struct sf_hdtr sf; int flags = 0; + /* Beware that "in" clashes with Python's own "in" operator keyword */ static char *keywords[] = {"out", "in", "offset", "count", "headers", "trailers", "flags", NULL}; @@ -8655,9 +8656,9 @@ #if defined(HAVE_MKNOD) && defined(HAVE_MAKEDEV) PyDoc_STRVAR(posix_mknod__doc__, -"mknod(filename, mode=0o600, device=0, *, dir_fd=None)\n\n\ +"mknod(path, mode=0o600, device=0, *, dir_fd=None)\n\n\ Create a filesystem node (file, device special file or named pipe)\n\ -named filename. mode specifies both the permissions to use and the\n\ +named path. mode specifies both the permissions to use and the\n\ type of node to be created, being combined (bitwise OR) with one of\n\ S_IFREG, S_IFCHR, S_IFBLK, and S_IFIFO. For S_IFCHR and S_IFBLK,\n\ device defines the newly created device special file (probably using\n\ -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 9 05:56:03 2015 From: python-checkins at python.org (martin.panter) Date: Wed, 09 Sep 2015 03:56:03 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_default_-=3E_default?= =?utf-8?q?=29=3A_Merge_3=2E6_branches?= Message-ID: <20150909035603.14857.92165@psf.io> https://hg.python.org/cpython/rev/7a65ba35275f changeset: 97795:7a65ba35275f parent: 97793:6177482ce6a1 parent: 97789:6c3c3d4b505b user: Martin Panter date: Wed Sep 09 03:46:24 2015 +0000 summary: Merge 3.6 branches files: Doc/whatsnew/3.5.rst | 115 ++++++++++++------------------ 1 files changed, 46 insertions(+), 69 deletions(-) diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -48,7 +48,6 @@ when researching a change. This article explains the new features in Python 3.5, compared to 3.4. - For full details, see the :source:`Misc/NEWS` file. .. note:: @@ -83,13 +82,16 @@ New built-in features: * ``bytes % args``, ``bytearray % args``: :pep:`461` - Adding ``%`` formatting - to bytes and bytearray + to bytes and bytearray. + * ``b'\xf0\x9f\x90\x8d'.hex()``, ``bytearray(b'\xf0\x9f\x90\x8d').hex()``, ``memoryview(b'\xf0\x9f\x90\x8d').hex()``: :issue:`9951` - A ``hex`` method has been added to bytes, bytearray, and memoryview. + * Generators have new ``gi_yieldfrom`` attribute, which returns the object being iterated by ``yield from`` expressions. (Contributed by Benno Leslie and Yury Selivanov in :issue:`24450`.) + * New :exc:`RecursionError` exception. (Contributed by Georg Brandl in :issue:`19235`.) @@ -101,6 +103,7 @@ (:issue:`19977`). * :pep:`488`, the elimination of ``.pyo`` files. + * :pep:`489`, multi-phase initialization of extension modules. Significantly Improved Library Modules: @@ -120,6 +123,11 @@ protocol handling from network IO. (Contributed by Geert Jansen in :issue:`21965`.) +* :mod:`traceback` has new lightweight and convenient to work with + classes :class:`~traceback.TracebackException`, + :class:`~traceback.StackSummary`, and :class:`traceback.FrameSummary`. + (Contributed by Robert Collins in :issue:`17911`.) + Security improvements: * SSLv3 is now disabled throughout the standard library. @@ -135,27 +143,13 @@ * A new installer for Windows has replaced the old MSI. See :ref:`using-on-windows` for more information. + * Windows builds now use Microsoft Visual C++ 14.0, and extension modules should use the same. - -Please read on for a comprehensive list of user-facing changes. - - -.. PEP-sized items next. - -.. _pep-4XX: - -.. PEP 4XX: Virtual Environments -.. ============================= - - -.. (Implemented by Foo Bar.) - -.. .. seealso:: - - :pep:`4XX` - Python Virtual Environments - PEP written by Carl Meyer +Please read on for a comprehensive list of user-facing changes, including many +other smaller improvements, CPython optimizations, deprecations, and potential +porting issues. New Features @@ -267,7 +261,12 @@ mathematics, science, engineering, and the addition of ``@`` allows writing cleaner code:: - >>> S = (H @ beta - r).T @ inv(H @ V @ H.T) @ (H @ beta - r) + S = (H @ beta - r).T @ inv(H @ V @ H.T) @ (H @ beta - r) + +instead of:: + + S = dot((dot(H, beta) - r).T, + dot(inv(dot(dot(H, V), H.T)), dot(H, beta) - r)) An upcoming release of NumPy 1.10 will add support for the new operator:: @@ -411,60 +410,38 @@ instead of raising :exc:`InterruptedError` if the Python signal handler does not raise an exception: -* :func:`open`, :func:`os.open`, :func:`io.open` -* functions of the :mod:`faulthandler` module -* :mod:`os` functions: +* :func:`open`, :func:`os.open`, :func:`io.open`; - - :func:`os.fchdir` - - :func:`os.fchmod` - - :func:`os.fchown` - - :func:`os.fdatasync` - - :func:`os.fstat` - - :func:`os.fstatvfs` - - :func:`os.fsync` - - :func:`os.ftruncate` - - :func:`os.mkfifo` - - :func:`os.mknod` - - :func:`os.posix_fadvise` - - :func:`os.posix_fallocate` - - :func:`os.pread` - - :func:`os.pwrite` - - :func:`os.read` - - :func:`os.readv` - - :func:`os.sendfile` - - :func:`os.wait3` - - :func:`os.wait4` - - :func:`os.wait` - - :func:`os.waitid` - - :func:`os.waitpid` - - :func:`os.write` - - :func:`os.writev` - - special cases: :func:`os.close` and :func:`os.dup2` now ignore - :py:data:`~errno.EINTR` error, the syscall is not retried (see the PEP - for the rationale) +* functions of the :mod:`faulthandler` module; -* :mod:`select` functions: +* :mod:`os` functions: :func:`~os.fchdir`, :func:`~os.fchmod`, + :func:`~os.fchown`, :func:`~os.fdatasync`, :func:`~os.fstat`, + :func:`~os.fstatvfs`, :func:`~os.fsync`, :func:`~os.ftruncate`, + :func:`~os.mkfifo`, :func:`~os.mknod`, :func:`~os.posix_fadvise`, + :func:`~os.posix_fallocate`, :func:`~os.pread`, :func:`~os.pwrite`, + :func:`~os.read`, :func:`~os.readv`, :func:`~os.sendfile`, + :func:`~os.wait3`, :func:`~os.wait4`, :func:`~os.wait`, + :func:`~os.waitid`, :func:`~os.waitpid`, :func:`~os.write`, + :func:`~os.writev`; - - :func:`select.devpoll.poll` - - :func:`select.epoll.poll` - - :func:`select.kqueue.control` - - :func:`select.poll.poll` - - :func:`select.select` +* special cases: :func:`os.close` and :func:`os.dup2` now ignore + :py:data:`~errno.EINTR` error, the syscall is not retried (see the PEP + for the rationale); -* :func:`socket.socket` methods: +* :mod:`select` functions: :func:`~select.devpoll.poll`, + :func:`~select.epoll.poll`, :func:`~select.kqueue.control`, + :func:`~select.poll.poll`, :func:`~select.select`; - - :meth:`~socket.socket.accept` - - :meth:`~socket.socket.connect` (except for non-blocking sockets) - - :meth:`~socket.socket.recv` - - :meth:`~socket.socket.recvfrom` - - :meth:`~socket.socket.recvmsg` - - :meth:`~socket.socket.send` - - :meth:`~socket.socket.sendall` - - :meth:`~socket.socket.sendmsg` - - :meth:`~socket.socket.sendto` +* :func:`socket.socket` methods: :meth:`~socket.socket.accept`, + :meth:`~socket.socket.connect` (except for non-blocking sockets), + :meth:`~socket.socket.recv`, :meth:`~socket.socket.recvfrom`, + :meth:`~socket.socket.recvmsg`, :meth:`~socket.socket.send`, + :meth:`~socket.socket.sendall`, :meth:`~socket.socket.sendmsg`, + :meth:`~socket.socket.sendto`; -* :func:`signal.sigtimedwait`, :func:`signal.sigwaitinfo` -* :func:`time.sleep` +* :func:`signal.sigtimedwait`, :func:`signal.sigwaitinfo`; + +* :func:`time.sleep`. .. seealso:: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 9 05:56:03 2015 From: python-checkins at python.org (martin.panter) Date: Wed, 09 Sep 2015 03:56:03 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy41IC0+IDMuNSk6?= =?utf-8?q?_Merge_3=2E5_branches?= Message-ID: <20150909035603.101502.33464@psf.io> https://hg.python.org/cpython/rev/daa0ef5c8665 changeset: 97794:daa0ef5c8665 branch: 3.5 parent: 97792:981102699c00 parent: 97788:0ea528234bc5 user: Martin Panter date: Wed Sep 09 03:45:58 2015 +0000 summary: Merge 3.5 branches files: Doc/whatsnew/3.5.rst | 115 ++++++++++++------------------ 1 files changed, 46 insertions(+), 69 deletions(-) diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -48,7 +48,6 @@ when researching a change. This article explains the new features in Python 3.5, compared to 3.4. - For full details, see the :source:`Misc/NEWS` file. .. note:: @@ -83,13 +82,16 @@ New built-in features: * ``bytes % args``, ``bytearray % args``: :pep:`461` - Adding ``%`` formatting - to bytes and bytearray + to bytes and bytearray. + * ``b'\xf0\x9f\x90\x8d'.hex()``, ``bytearray(b'\xf0\x9f\x90\x8d').hex()``, ``memoryview(b'\xf0\x9f\x90\x8d').hex()``: :issue:`9951` - A ``hex`` method has been added to bytes, bytearray, and memoryview. + * Generators have new ``gi_yieldfrom`` attribute, which returns the object being iterated by ``yield from`` expressions. (Contributed by Benno Leslie and Yury Selivanov in :issue:`24450`.) + * New :exc:`RecursionError` exception. (Contributed by Georg Brandl in :issue:`19235`.) @@ -101,6 +103,7 @@ (:issue:`19977`). * :pep:`488`, the elimination of ``.pyo`` files. + * :pep:`489`, multi-phase initialization of extension modules. Significantly Improved Library Modules: @@ -120,6 +123,11 @@ protocol handling from network IO. (Contributed by Geert Jansen in :issue:`21965`.) +* :mod:`traceback` has new lightweight and convenient to work with + classes :class:`~traceback.TracebackException`, + :class:`~traceback.StackSummary`, and :class:`traceback.FrameSummary`. + (Contributed by Robert Collins in :issue:`17911`.) + Security improvements: * SSLv3 is now disabled throughout the standard library. @@ -135,27 +143,13 @@ * A new installer for Windows has replaced the old MSI. See :ref:`using-on-windows` for more information. + * Windows builds now use Microsoft Visual C++ 14.0, and extension modules should use the same. - -Please read on for a comprehensive list of user-facing changes. - - -.. PEP-sized items next. - -.. _pep-4XX: - -.. PEP 4XX: Virtual Environments -.. ============================= - - -.. (Implemented by Foo Bar.) - -.. .. seealso:: - - :pep:`4XX` - Python Virtual Environments - PEP written by Carl Meyer +Please read on for a comprehensive list of user-facing changes, including many +other smaller improvements, CPython optimizations, deprecations, and potential +porting issues. New Features @@ -267,7 +261,12 @@ mathematics, science, engineering, and the addition of ``@`` allows writing cleaner code:: - >>> S = (H @ beta - r).T @ inv(H @ V @ H.T) @ (H @ beta - r) + S = (H @ beta - r).T @ inv(H @ V @ H.T) @ (H @ beta - r) + +instead of:: + + S = dot((dot(H, beta) - r).T, + dot(inv(dot(dot(H, V), H.T)), dot(H, beta) - r)) An upcoming release of NumPy 1.10 will add support for the new operator:: @@ -411,60 +410,38 @@ instead of raising :exc:`InterruptedError` if the Python signal handler does not raise an exception: -* :func:`open`, :func:`os.open`, :func:`io.open` -* functions of the :mod:`faulthandler` module -* :mod:`os` functions: +* :func:`open`, :func:`os.open`, :func:`io.open`; - - :func:`os.fchdir` - - :func:`os.fchmod` - - :func:`os.fchown` - - :func:`os.fdatasync` - - :func:`os.fstat` - - :func:`os.fstatvfs` - - :func:`os.fsync` - - :func:`os.ftruncate` - - :func:`os.mkfifo` - - :func:`os.mknod` - - :func:`os.posix_fadvise` - - :func:`os.posix_fallocate` - - :func:`os.pread` - - :func:`os.pwrite` - - :func:`os.read` - - :func:`os.readv` - - :func:`os.sendfile` - - :func:`os.wait3` - - :func:`os.wait4` - - :func:`os.wait` - - :func:`os.waitid` - - :func:`os.waitpid` - - :func:`os.write` - - :func:`os.writev` - - special cases: :func:`os.close` and :func:`os.dup2` now ignore - :py:data:`~errno.EINTR` error, the syscall is not retried (see the PEP - for the rationale) +* functions of the :mod:`faulthandler` module; -* :mod:`select` functions: +* :mod:`os` functions: :func:`~os.fchdir`, :func:`~os.fchmod`, + :func:`~os.fchown`, :func:`~os.fdatasync`, :func:`~os.fstat`, + :func:`~os.fstatvfs`, :func:`~os.fsync`, :func:`~os.ftruncate`, + :func:`~os.mkfifo`, :func:`~os.mknod`, :func:`~os.posix_fadvise`, + :func:`~os.posix_fallocate`, :func:`~os.pread`, :func:`~os.pwrite`, + :func:`~os.read`, :func:`~os.readv`, :func:`~os.sendfile`, + :func:`~os.wait3`, :func:`~os.wait4`, :func:`~os.wait`, + :func:`~os.waitid`, :func:`~os.waitpid`, :func:`~os.write`, + :func:`~os.writev`; - - :func:`select.devpoll.poll` - - :func:`select.epoll.poll` - - :func:`select.kqueue.control` - - :func:`select.poll.poll` - - :func:`select.select` +* special cases: :func:`os.close` and :func:`os.dup2` now ignore + :py:data:`~errno.EINTR` error, the syscall is not retried (see the PEP + for the rationale); -* :func:`socket.socket` methods: +* :mod:`select` functions: :func:`~select.devpoll.poll`, + :func:`~select.epoll.poll`, :func:`~select.kqueue.control`, + :func:`~select.poll.poll`, :func:`~select.select`; - - :meth:`~socket.socket.accept` - - :meth:`~socket.socket.connect` (except for non-blocking sockets) - - :meth:`~socket.socket.recv` - - :meth:`~socket.socket.recvfrom` - - :meth:`~socket.socket.recvmsg` - - :meth:`~socket.socket.send` - - :meth:`~socket.socket.sendall` - - :meth:`~socket.socket.sendmsg` - - :meth:`~socket.socket.sendto` +* :func:`socket.socket` methods: :meth:`~socket.socket.accept`, + :meth:`~socket.socket.connect` (except for non-blocking sockets), + :meth:`~socket.socket.recv`, :meth:`~socket.socket.recvfrom`, + :meth:`~socket.socket.recvmsg`, :meth:`~socket.socket.send`, + :meth:`~socket.socket.sendall`, :meth:`~socket.socket.sendmsg`, + :meth:`~socket.socket.sendto`; -* :func:`signal.sigtimedwait`, :func:`signal.sigwaitinfo` -* :func:`time.sleep` +* :func:`signal.sigtimedwait`, :func:`signal.sigwaitinfo`; + +* :func:`time.sleep`. .. seealso:: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 9 08:03:06 2015 From: python-checkins at python.org (terry.reedy) Date: Wed, 09 Sep 2015 06:03:06 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_merge_dangling_3=2E5_into_default?= Message-ID: <20150909060305.11262.77262@psf.io> https://hg.python.org/cpython/rev/08783af6e9a7 changeset: 97796:08783af6e9a7 parent: 97795:7a65ba35275f parent: 97794:daa0ef5c8665 user: Terry Jan Reedy date: Wed Sep 09 02:02:25 2015 -0400 summary: merge dangling 3.5 into default files: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 9 08:11:46 2015 From: python-checkins at python.org (terry.reedy) Date: Wed, 09 Sep 2015 06:11:46 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgMjQxOTk6?= =?utf-8?q?_Deprecate_idlelib=2Eidlever_with_a_warning_on_import=2E?= Message-ID: <20150909061146.27705.23588@psf.io> https://hg.python.org/cpython/rev/55b62e2c59f8 changeset: 97797:55b62e2c59f8 branch: 2.7 parent: 97776:ab1dc73fe8b5 user: Terry Jan Reedy date: Wed Sep 09 02:10:10 2015 -0400 summary: Issue 24199: Deprecate idlelib.idlever with a warning on import. files: Lib/idlelib/idle_test/test_warning.py | 9 +++++++++ Lib/idlelib/idlever.py | 12 ++++++++++-- 2 files changed, 19 insertions(+), 2 deletions(-) diff --git a/Lib/idlelib/idle_test/test_warning.py b/Lib/idlelib/idle_test/test_warning.py --- a/Lib/idlelib/idle_test/test_warning.py +++ b/Lib/idlelib/idle_test/test_warning.py @@ -68,6 +68,15 @@ 'Test', UserWarning, 'test_warning.py', 99, f, 'Line of code') self.assertEqual(shellmsg.splitlines(), f.getvalue().splitlines()) +class ImportWarnTest(unittest.TestCase): + def test_idlever(self): + with warnings.catch_warnings(record=True) as w: + warnings.simplefilter("always") + import idlelib.idlever + self.assertEqual(len(w), 1) + self.assertTrue(issubclass(w[-1].category, DeprecationWarning)) + self.assertIn("version", str(w[-1].message)) + if __name__ == '__main__': unittest.main(verbosity=2, exit=False) diff --git a/Lib/idlelib/idlever.py b/Lib/idlelib/idlever.py --- a/Lib/idlelib/idlever.py +++ b/Lib/idlelib/idlever.py @@ -1,4 +1,12 @@ -"""Unused by Idle: there is no separate Idle version anymore. -Kept only for possible existing extension use.""" +""" +The separate Idle version was eliminated years ago; +idlelib.idlever is no longer used by Idle +and will be removed in 3.6 or later. Use + from sys import version + IDLE_VERSION = version[:version.index(' ')] +""" +# Kept for now only for possible existing extension use +import warnings as w +w.warn(__doc__, DeprecationWarning) from sys import version IDLE_VERSION = version[:version.index(' ')] -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 9 08:11:47 2015 From: python-checkins at python.org (terry.reedy) Date: Wed, 09 Sep 2015 06:11:47 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Merge_with_3=2E4?= Message-ID: <20150909061146.27713.44384@psf.io> https://hg.python.org/cpython/rev/c27490d2372c changeset: 97799:c27490d2372c branch: 3.5 parent: 97794:daa0ef5c8665 parent: 97798:c51514826126 user: Terry Jan Reedy date: Wed Sep 09 02:10:35 2015 -0400 summary: Merge with 3.4 files: Lib/idlelib/idle_test/test_warning.py | 9 +++++++++ Lib/idlelib/idlever.py | 12 ++++++++++-- 2 files changed, 19 insertions(+), 2 deletions(-) diff --git a/Lib/idlelib/idle_test/test_warning.py b/Lib/idlelib/idle_test/test_warning.py --- a/Lib/idlelib/idle_test/test_warning.py +++ b/Lib/idlelib/idle_test/test_warning.py @@ -68,6 +68,15 @@ 'Test', UserWarning, 'test_warning.py', 99, f, 'Line of code') self.assertEqual(shellmsg.splitlines(), f.getvalue().splitlines()) +class ImportWarnTest(unittest.TestCase): + def test_idlever(self): + with warnings.catch_warnings(record=True) as w: + warnings.simplefilter("always") + import idlelib.idlever + self.assertEqual(len(w), 1) + self.assertTrue(issubclass(w[-1].category, DeprecationWarning)) + self.assertIn("version", str(w[-1].message)) + if __name__ == '__main__': unittest.main(verbosity=2, exit=False) diff --git a/Lib/idlelib/idlever.py b/Lib/idlelib/idlever.py --- a/Lib/idlelib/idlever.py +++ b/Lib/idlelib/idlever.py @@ -1,4 +1,12 @@ -"""Unused by Idle: there is no separate Idle version anymore. -Kept only for possible existing extension use.""" +""" +The separate Idle version was eliminated years ago; +idlelib.idlever is no longer used by Idle +and will be removed in 3.6 or later. Use + from sys import version + IDLE_VERSION = version[:version.index(' ')] +""" +# Kept for now only for possible existing extension use +import warnings as w +w.warn(__doc__, DeprecationWarning) from sys import version IDLE_VERSION = version[:version.index(' ')] -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 9 08:11:47 2015 From: python-checkins at python.org (terry.reedy) Date: Wed, 09 Sep 2015 06:11:47 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgMjQxOTk6?= =?utf-8?q?_Deprecate_idlelib=2Eidlever_with_a_warning_on_import=2E?= Message-ID: <20150909061146.68887.38677@psf.io> https://hg.python.org/cpython/rev/c51514826126 changeset: 97798:c51514826126 branch: 3.4 parent: 97790:fdb5d84f9948 user: Terry Jan Reedy date: Wed Sep 09 02:10:17 2015 -0400 summary: Issue 24199: Deprecate idlelib.idlever with a warning on import. files: Lib/idlelib/idle_test/test_warning.py | 9 +++++++++ Lib/idlelib/idlever.py | 13 ++++++++++--- 2 files changed, 19 insertions(+), 3 deletions(-) diff --git a/Lib/idlelib/idle_test/test_warning.py b/Lib/idlelib/idle_test/test_warning.py --- a/Lib/idlelib/idle_test/test_warning.py +++ b/Lib/idlelib/idle_test/test_warning.py @@ -68,6 +68,15 @@ 'Test', UserWarning, 'test_warning.py', 99, f, 'Line of code') self.assertEqual(shellmsg.splitlines(), f.getvalue().splitlines()) +class ImportWarnTest(unittest.TestCase): + def test_idlever(self): + with warnings.catch_warnings(record=True) as w: + warnings.simplefilter("always") + import idlelib.idlever + self.assertEqual(len(w), 1) + self.assertTrue(issubclass(w[-1].category, DeprecationWarning)) + self.assertIn("version", str(w[-1].message)) + if __name__ == '__main__': unittest.main(verbosity=2, exit=False) diff --git a/Lib/idlelib/idlever.py b/Lib/idlelib/idlever.py --- a/Lib/idlelib/idlever.py +++ b/Lib/idlelib/idlever.py @@ -1,5 +1,12 @@ -"""Unused by Idle: there is no separate Idle version anymore. -Kept only for possible existing extension use.""" +""" +The separate Idle version was eliminated years ago; +idlelib.idlever is no longer used by Idle +and will be removed in 3.6 or later. Use + from sys import version + IDLE_VERSION = version[:version.index(' ')] +""" +# Kept for now only for possible existing extension use +import warnings as w +w.warn(__doc__, DeprecationWarning) from sys import version IDLE_VERSION = version[:version.index(' ')] - -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 9 08:11:47 2015 From: python-checkins at python.org (terry.reedy) Date: Wed, 09 Sep 2015 06:11:47 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge_with_3=2E5?= Message-ID: <20150909061147.114820.89991@psf.io> https://hg.python.org/cpython/rev/b4a7600d4314 changeset: 97800:b4a7600d4314 parent: 97796:08783af6e9a7 parent: 97799:c27490d2372c user: Terry Jan Reedy date: Wed Sep 09 02:10:47 2015 -0400 summary: Merge with 3.5 files: Lib/idlelib/idle_test/test_warning.py | 9 +++++++++ Lib/idlelib/idlever.py | 12 ++++++++++-- 2 files changed, 19 insertions(+), 2 deletions(-) diff --git a/Lib/idlelib/idle_test/test_warning.py b/Lib/idlelib/idle_test/test_warning.py --- a/Lib/idlelib/idle_test/test_warning.py +++ b/Lib/idlelib/idle_test/test_warning.py @@ -68,6 +68,15 @@ 'Test', UserWarning, 'test_warning.py', 99, f, 'Line of code') self.assertEqual(shellmsg.splitlines(), f.getvalue().splitlines()) +class ImportWarnTest(unittest.TestCase): + def test_idlever(self): + with warnings.catch_warnings(record=True) as w: + warnings.simplefilter("always") + import idlelib.idlever + self.assertEqual(len(w), 1) + self.assertTrue(issubclass(w[-1].category, DeprecationWarning)) + self.assertIn("version", str(w[-1].message)) + if __name__ == '__main__': unittest.main(verbosity=2, exit=False) diff --git a/Lib/idlelib/idlever.py b/Lib/idlelib/idlever.py --- a/Lib/idlelib/idlever.py +++ b/Lib/idlelib/idlever.py @@ -1,4 +1,12 @@ -"""Unused by Idle: there is no separate Idle version anymore. -Kept only for possible existing extension use.""" +""" +The separate Idle version was eliminated years ago; +idlelib.idlever is no longer used by Idle +and will be removed in 3.6 or later. Use + from sys import version + IDLE_VERSION = version[:version.index(' ')] +""" +# Kept for now only for possible existing extension use +import warnings as w +w.warn(__doc__, DeprecationWarning) from sys import version IDLE_VERSION = version[:version.index(' ')] -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 9 08:31:04 2015 From: python-checkins at python.org (martin.panter) Date: Wed, 09 Sep 2015 06:31:04 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Merge_3=2E4_into_3=2E5?= Message-ID: <20150909063104.15728.52169@psf.io> https://hg.python.org/cpython/rev/31745f7dc802 changeset: 97802:31745f7dc802 branch: 3.5 parent: 97799:c27490d2372c parent: 97801:e42e2bd47168 user: Martin Panter date: Wed Sep 09 06:27:43 2015 +0000 summary: Merge 3.4 into 3.5 files: Doc/library/os.rst | 2 +- Lib/test/test_os.py | 2 +- Modules/posixmodule.c | 6 +++--- 3 files changed, 5 insertions(+), 5 deletions(-) diff --git a/Doc/library/os.rst b/Doc/library/os.rst --- a/Doc/library/os.rst +++ b/Doc/library/os.rst @@ -1072,7 +1072,7 @@ .. function:: sendfile(out, in, offset, count) - sendfile(out, in, offset, count, headers=None, trailers=None, flags=0) + sendfile(out, in, offset, count, [headers], [trailers], flags=0) Copy *count* bytes from file descriptor *in* to file descriptor *out* starting at *offset*. diff --git a/Lib/test/test_os.py b/Lib/test/test_os.py --- a/Lib/test/test_os.py +++ b/Lib/test/test_os.py @@ -2332,7 +2332,7 @@ **{'in': self.fileno}) if self.SUPPORT_HEADERS_TRAILERS: os.sendfile(self.sockno, self.fileno, offset=0, count=4096, - headers=None, trailers=None, flags=0) + headers=(), trailers=(), flags=0) # --- headers / trailers tests diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c --- a/Modules/posixmodule.c +++ b/Modules/posixmodule.c @@ -8236,7 +8236,7 @@ #ifdef HAVE_SENDFILE PyDoc_STRVAR(posix_sendfile__doc__, "sendfile(out, in, offset, count) -> byteswritten\n\ -sendfile(out, in, offset, count, headers=None, trailers=None, flags=0)\n\ +sendfile(out, in, offset, count[, headers][, trailers], flags=0)\n\ -> byteswritten\n\ Copy count bytes from file descriptor in to file descriptor out."); @@ -8278,7 +8278,7 @@ if (headers != NULL) { if (!PySequence_Check(headers)) { PyErr_SetString(PyExc_TypeError, - "sendfile() headers must be a sequence or None"); + "sendfile() headers must be a sequence"); return NULL; } else { Py_ssize_t i = 0; /* Avoid uninitialized warning */ @@ -8295,7 +8295,7 @@ if (trailers != NULL) { if (!PySequence_Check(trailers)) { PyErr_SetString(PyExc_TypeError, - "sendfile() trailers must be a sequence or None"); + "sendfile() trailers must be a sequence"); return NULL; } else { Py_ssize_t i = 0; /* Avoid uninitialized warning */ -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 9 08:31:05 2015 From: python-checkins at python.org (martin.panter) Date: Wed, 09 Sep 2015 06:31:05 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogb3Muc2VuZGZpbGUo?= =?utf-8?q?headers=3DNone=2C_trailers=3DNone=29_arguments_are_not_actually?= =?utf-8?q?_accepted?= Message-ID: <20150909063104.12021.93595@psf.io> https://hg.python.org/cpython/rev/e42e2bd47168 changeset: 97801:e42e2bd47168 branch: 3.4 parent: 97798:c51514826126 user: Martin Panter date: Wed Sep 09 05:29:24 2015 +0000 summary: os.sendfile(headers=None, trailers=None) arguments are not actually accepted Needs to be tested on a BSD. files: Doc/library/os.rst | 2 +- Lib/test/test_os.py | 2 +- Modules/posixmodule.c | 6 +++--- 3 files changed, 5 insertions(+), 5 deletions(-) diff --git a/Doc/library/os.rst b/Doc/library/os.rst --- a/Doc/library/os.rst +++ b/Doc/library/os.rst @@ -1072,7 +1072,7 @@ .. function:: sendfile(out, in, offset, count) - sendfile(out, in, offset, count, headers=None, trailers=None, flags=0) + sendfile(out, in, offset, count, [headers], [trailers], flags=0) Copy *count* bytes from file descriptor *in* to file descriptor *out* starting at *offset*. diff --git a/Lib/test/test_os.py b/Lib/test/test_os.py --- a/Lib/test/test_os.py +++ b/Lib/test/test_os.py @@ -2170,7 +2170,7 @@ **{'in': self.fileno}) if self.SUPPORT_HEADERS_TRAILERS: os.sendfile(self.sockno, self.fileno, offset=0, count=4096, - headers=None, trailers=None, flags=0) + headers=(), trailers=(), flags=0) # --- headers / trailers tests diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c --- a/Modules/posixmodule.c +++ b/Modules/posixmodule.c @@ -8246,7 +8246,7 @@ #ifdef HAVE_SENDFILE PyDoc_STRVAR(posix_sendfile__doc__, "sendfile(out, in, offset, count) -> byteswritten\n\ -sendfile(out, in, offset, count, headers=None, trailers=None, flags=0)\n\ +sendfile(out, in, offset, count[, headers][, trailers], flags=0)\n\ -> byteswritten\n\ Copy count bytes from file descriptor in to file descriptor out."); @@ -8286,7 +8286,7 @@ if (headers != NULL) { if (!PySequence_Check(headers)) { PyErr_SetString(PyExc_TypeError, - "sendfile() headers must be a sequence or None"); + "sendfile() headers must be a sequence"); return NULL; } else { Py_ssize_t i = 0; /* Avoid uninitialized warning */ @@ -8303,7 +8303,7 @@ if (trailers != NULL) { if (!PySequence_Check(trailers)) { PyErr_SetString(PyExc_TypeError, - "sendfile() trailers must be a sequence or None"); + "sendfile() trailers must be a sequence"); return NULL; } else { Py_ssize_t i = 0; /* Avoid uninitialized warning */ -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 9 08:31:06 2015 From: python-checkins at python.org (martin.panter) Date: Wed, 09 Sep 2015 06:31:06 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogTWVyZ2UgMy41IGludG8gMy42?= Message-ID: <20150909063104.101488.77594@psf.io> https://hg.python.org/cpython/rev/ae6eb59f7535 changeset: 97803:ae6eb59f7535 parent: 97800:b4a7600d4314 parent: 97802:31745f7dc802 user: Martin Panter date: Wed Sep 09 06:28:08 2015 +0000 summary: Merge 3.5 into 3.6 files: Doc/library/os.rst | 2 +- Lib/test/test_os.py | 2 +- Modules/posixmodule.c | 6 +++--- 3 files changed, 5 insertions(+), 5 deletions(-) diff --git a/Doc/library/os.rst b/Doc/library/os.rst --- a/Doc/library/os.rst +++ b/Doc/library/os.rst @@ -1072,7 +1072,7 @@ .. function:: sendfile(out, in, offset, count) - sendfile(out, in, offset, count, headers=None, trailers=None, flags=0) + sendfile(out, in, offset, count, [headers], [trailers], flags=0) Copy *count* bytes from file descriptor *in* to file descriptor *out* starting at *offset*. diff --git a/Lib/test/test_os.py b/Lib/test/test_os.py --- a/Lib/test/test_os.py +++ b/Lib/test/test_os.py @@ -2331,7 +2331,7 @@ **{'in': self.fileno}) if self.SUPPORT_HEADERS_TRAILERS: os.sendfile(self.sockno, self.fileno, offset=0, count=4096, - headers=None, trailers=None, flags=0) + headers=(), trailers=(), flags=0) # --- headers / trailers tests diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c --- a/Modules/posixmodule.c +++ b/Modules/posixmodule.c @@ -8236,7 +8236,7 @@ #ifdef HAVE_SENDFILE PyDoc_STRVAR(posix_sendfile__doc__, "sendfile(out, in, offset, count) -> byteswritten\n\ -sendfile(out, in, offset, count, headers=None, trailers=None, flags=0)\n\ +sendfile(out, in, offset, count[, headers][, trailers], flags=0)\n\ -> byteswritten\n\ Copy count bytes from file descriptor in to file descriptor out."); @@ -8278,7 +8278,7 @@ if (headers != NULL) { if (!PySequence_Check(headers)) { PyErr_SetString(PyExc_TypeError, - "sendfile() headers must be a sequence or None"); + "sendfile() headers must be a sequence"); return NULL; } else { Py_ssize_t i = 0; /* Avoid uninitialized warning */ @@ -8295,7 +8295,7 @@ if (trailers != NULL) { if (!PySequence_Check(trailers)) { PyErr_SetString(PyExc_TypeError, - "sendfile() trailers must be a sequence or None"); + "sendfile() trailers must be a sequence"); return NULL; } else { Py_ssize_t i = 0; /* Avoid uninitialized warning */ -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 9 08:59:24 2015 From: python-checkins at python.org (martin.panter) Date: Wed, 09 Sep 2015 06:59:24 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzI0OTg0?= =?utf-8?q?=3A_Document_AF=5FBLUETOOTH_socket_address_formats?= Message-ID: <20150909065924.11997.18340@psf.io> https://hg.python.org/cpython/rev/da9b26670e44 changeset: 97804:da9b26670e44 branch: 3.4 parent: 97801:e42e2bd47168 user: Martin Panter date: Wed Sep 09 06:47:58 2015 +0000 summary: Issue #24984: Document AF_BLUETOOTH socket address formats Patch from Tim Tisdall. files: Doc/library/socket.rst | 26 ++++++++++++++++++++++++-- Misc/ACKS | 1 + 2 files changed, 25 insertions(+), 2 deletions(-) diff --git a/Doc/library/socket.rst b/Doc/library/socket.rst --- a/Doc/library/socket.rst +++ b/Doc/library/socket.rst @@ -103,8 +103,30 @@ .. versionadded:: 3.3 -- Certain other address families (:const:`AF_BLUETOOTH`, :const:`AF_PACKET`, - :const:`AF_CAN`) support specific representations. +- :const:`AF_BLUETOOTH` supports the following protocols and address + formats: + + - :const:`BTPROTO_L2CAP` accepts ``(bdaddr, psm)`` where ``bdaddr`` is + the Bluetooth address as a string and ``psm`` is an integer. + + - :const:`BTPROTO_RFCOMM` accepts ``(bdaddr, channel)`` where ``bdaddr`` + is the Bluetooth address as a string and ``channel`` is an integer. + + - :const:`BTPROTO_HCI` accepts ``(device_id,)`` where ``device_id`` is + either an integer or a string with the Bluetooth address of the + interface. (This depends on your OS; NetBSD and DragonFlyBSD expect + a Bluetooth address while everything else expects an integer.) + + .. versionchanged:: 3.2 + NetBSD and DragonFlyBSD support added. + + - :const:`BTPROTO_SCO` accepts ``bdaddr`` where ``bdaddr`` is a + :term:`bytes-like object` containing the Bluetooth address in a + string format. (ex. ``b'12:23:34:45:56:67'``) This protocol is not + supported under FreeBSD. + +- Certain other address families (:const:`AF_PACKET`, :const:`AF_CAN`) + support specific representations. .. XXX document them! diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -1385,6 +1385,7 @@ July Tikhonov Tracy Tims Oren Tirosh +Tim Tisdall Jason Tishler Christian Tismer Jim Tittsler -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 9 08:59:25 2015 From: python-checkins at python.org (martin.panter) Date: Wed, 09 Sep 2015 06:59:25 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2324984=3A_Merge_3=2E5_into_3=2E6?= Message-ID: <20150909065925.27685.3924@psf.io> https://hg.python.org/cpython/rev/ec4ba0cb1ce0 changeset: 97806:ec4ba0cb1ce0 parent: 97803:ae6eb59f7535 parent: 97805:4ce8450da22d user: Martin Panter date: Wed Sep 09 06:50:07 2015 +0000 summary: Issue #24984: Merge 3.5 into 3.6 files: Doc/library/socket.rst | 26 ++++++++++++++++++++++++-- Misc/ACKS | 1 + 2 files changed, 25 insertions(+), 2 deletions(-) diff --git a/Doc/library/socket.rst b/Doc/library/socket.rst --- a/Doc/library/socket.rst +++ b/Doc/library/socket.rst @@ -106,8 +106,30 @@ .. versionadded:: 3.3 -- Certain other address families (:const:`AF_BLUETOOTH`, :const:`AF_PACKET`, - :const:`AF_CAN`) support specific representations. +- :const:`AF_BLUETOOTH` supports the following protocols and address + formats: + + - :const:`BTPROTO_L2CAP` accepts ``(bdaddr, psm)`` where ``bdaddr`` is + the Bluetooth address as a string and ``psm`` is an integer. + + - :const:`BTPROTO_RFCOMM` accepts ``(bdaddr, channel)`` where ``bdaddr`` + is the Bluetooth address as a string and ``channel`` is an integer. + + - :const:`BTPROTO_HCI` accepts ``(device_id,)`` where ``device_id`` is + either an integer or a string with the Bluetooth address of the + interface. (This depends on your OS; NetBSD and DragonFlyBSD expect + a Bluetooth address while everything else expects an integer.) + + .. versionchanged:: 3.2 + NetBSD and DragonFlyBSD support added. + + - :const:`BTPROTO_SCO` accepts ``bdaddr`` where ``bdaddr`` is a + :term:`bytes-like object` containing the Bluetooth address in a + string format. (ex. ``b'12:23:34:45:56:67'``) This protocol is not + supported under FreeBSD. + +- Certain other address families (:const:`AF_PACKET`, :const:`AF_CAN`) + support specific representations. .. XXX document them! diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -1425,6 +1425,7 @@ July Tikhonov Tracy Tims Oren Tirosh +Tim Tisdall Jason Tishler Christian Tismer Jim Tittsler -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 9 08:59:24 2015 From: python-checkins at python.org (martin.panter) Date: Wed, 09 Sep 2015 06:59:24 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Issue_=2324984=3A_Merge_3=2E4_into_3=2E5?= Message-ID: <20150909065924.14855.134@psf.io> https://hg.python.org/cpython/rev/4ce8450da22d changeset: 97805:4ce8450da22d branch: 3.5 parent: 97802:31745f7dc802 parent: 97804:da9b26670e44 user: Martin Panter date: Wed Sep 09 06:48:55 2015 +0000 summary: Issue #24984: Merge 3.4 into 3.5 files: Doc/library/socket.rst | 26 ++++++++++++++++++++++++-- Misc/ACKS | 1 + 2 files changed, 25 insertions(+), 2 deletions(-) diff --git a/Doc/library/socket.rst b/Doc/library/socket.rst --- a/Doc/library/socket.rst +++ b/Doc/library/socket.rst @@ -106,8 +106,30 @@ .. versionadded:: 3.3 -- Certain other address families (:const:`AF_BLUETOOTH`, :const:`AF_PACKET`, - :const:`AF_CAN`) support specific representations. +- :const:`AF_BLUETOOTH` supports the following protocols and address + formats: + + - :const:`BTPROTO_L2CAP` accepts ``(bdaddr, psm)`` where ``bdaddr`` is + the Bluetooth address as a string and ``psm`` is an integer. + + - :const:`BTPROTO_RFCOMM` accepts ``(bdaddr, channel)`` where ``bdaddr`` + is the Bluetooth address as a string and ``channel`` is an integer. + + - :const:`BTPROTO_HCI` accepts ``(device_id,)`` where ``device_id`` is + either an integer or a string with the Bluetooth address of the + interface. (This depends on your OS; NetBSD and DragonFlyBSD expect + a Bluetooth address while everything else expects an integer.) + + .. versionchanged:: 3.2 + NetBSD and DragonFlyBSD support added. + + - :const:`BTPROTO_SCO` accepts ``bdaddr`` where ``bdaddr`` is a + :term:`bytes-like object` containing the Bluetooth address in a + string format. (ex. ``b'12:23:34:45:56:67'``) This protocol is not + supported under FreeBSD. + +- Certain other address families (:const:`AF_PACKET`, :const:`AF_CAN`) + support specific representations. .. XXX document them! diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -1424,6 +1424,7 @@ July Tikhonov Tracy Tims Oren Tirosh +Tim Tisdall Jason Tishler Christian Tismer Jim Tittsler -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 9 11:20:41 2015 From: python-checkins at python.org (serhiy.storchaka) Date: Wed, 09 Sep 2015 09:20:41 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogRG9uJ3QgZW5jb2Rl?= =?utf-8?q?_unicode_dirname_in_test=5Fsupport=2Etemp=5Fcwd=28=29_if_unicod?= =?utf-8?q?e_file_names?= Message-ID: <20150909092041.68887.19359@psf.io> https://hg.python.org/cpython/rev/32893d8a52a9 changeset: 97807:32893d8a52a9 branch: 2.7 parent: 97797:55b62e2c59f8 user: Serhiy Storchaka date: Wed Sep 09 12:18:36 2015 +0300 summary: Don't encode unicode dirname in test_support.temp_cwd() if unicode file names are supported by the filesystem. On Windows the encoding can convert some characters to '?' that is not legal in file name. files: Lib/test/test_support.py | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-) diff --git a/Lib/test/test_support.py b/Lib/test/test_support.py --- a/Lib/test/test_support.py +++ b/Lib/test/test_support.py @@ -705,7 +705,8 @@ the CWD, an error is raised. If it's True, only a warning is raised and the original CWD is used. """ - if have_unicode and isinstance(name, unicode): + if (have_unicode and isinstance(name, unicode) and + not os.path.supports_unicode_filenames): try: name = name.encode(sys.getfilesystemencoding() or 'ascii') except UnicodeEncodeError: -- Repository URL: https://hg.python.org/cpython From lp_benchmark_robot at intel.com Wed Sep 9 12:41:52 2015 From: lp_benchmark_robot at intel.com (lp_benchmark_robot) Date: Wed, 9 Sep 2015 10:41:52 +0000 Subject: [Python-checkins] Benchmark Results for Python Default 2015-09-09 Message-ID: <078AA0FFE8C7034097F90205717F504611D79F78@IRSMSX102.ger.corp.intel.com> Results for project python_default-nightly, build date 2015-09-09 06:02:54 commit: 07c206831e2862f141e790d6fc2ea7de960f6ac3 revision date: 2015-09-09 05:40:45 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v3.4.3, with hash b4cbecbc0781e89a309d03b60a1f75f8499250e6 from 2015-02-25 12:15:33+00:00 ------------------------------------------------------------------------------------------ benchmark relative change since change since current rev with std_dev* last run v3.4.3 regrtest PGO ------------------------------------------------------------------------------------------ :-) django_v2 0.45309% -1.00676% 8.94825% 16.49552% :-( pybench 0.10717% -0.08578% -2.16233% 8.86123% :-) regex_v8 2.95604% 4.17396% -3.81380% 5.93531% :-( nbody 0.31103% 1.84706% -6.44494% 15.79196% :-( json_dump_v2 0.30113% -1.91992% -3.76416% 15.85307% :-| normal_startup 0.72699% -0.59895% -0.72628% 5.46638% ------------------------------------------------------------------------------------------ Note: Benchmark results are measured in seconds. * Relative Standard Deviation (Standard Deviation/Average) Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. No license (express or implied, by estoppel or otherwise) to any intellectual property rights is granted by this document. Intel disclaims all express and implied warranties, including without limitation, the implied warranties of merchantability, fitness for a particular purpose, and non-infringement, as well as any warranty arising from course of performance, course of dealing, or usage in trade. This document may contain information on products, services and/or processes in development. Contact your Intel representative to obtain the latest forecast, schedule, specifications and roadmaps. The products and services described may contain defects or errors known as errata which may cause deviations from published specifications. Current characterized errata are available on request. (C) 2015 Intel Corporation. From python-checkins at python.org Wed Sep 9 15:28:01 2015 From: python-checkins at python.org (yury.selivanov) Date: Wed, 09 Sep 2015 13:28:01 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogTWVyZ2UgMy41?= Message-ID: <20150909132801.15712.3395@psf.io> https://hg.python.org/cpython/rev/a96cc32f8160 changeset: 97809:a96cc32f8160 parent: 97806:ec4ba0cb1ce0 parent: 97808:8b9640aa3f1f user: Yury Selivanov date: Wed Sep 09 09:27:46 2015 -0400 summary: Merge 3.5 files: Doc/whatsnew/3.5.rst | 5 ++--- 1 files changed, 2 insertions(+), 3 deletions(-) diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -4,8 +4,7 @@ :Release: |release| :Date: |today| - -:Author: Elvis Pranskevichus (Editor) +:Editor: Elvis Pranskevichus .. Rules for maintenance: @@ -517,7 +516,7 @@ :pep:`489` -- Multi-phase extension module initialization PEP written by Petr Viktorin, Stefan Behnel, and Nick Coghlan; - implementation by Petr Viktorin. + implemented by Petr Viktorin. PEP 485: A function for testing approximate equality -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 9 15:28:02 2015 From: python-checkins at python.org (yury.selivanov) Date: Wed, 09 Sep 2015 13:28:02 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogd2hhdHNuZXcvMy41?= =?utf-8?q?=3A_Fix_nits_per_Berker_Peksag_suggestion?= Message-ID: <20150909132801.17973.16766@psf.io> https://hg.python.org/cpython/rev/8b9640aa3f1f changeset: 97808:8b9640aa3f1f branch: 3.5 parent: 97805:4ce8450da22d user: Yury Selivanov date: Wed Sep 09 09:27:29 2015 -0400 summary: whatsnew/3.5: Fix nits per Berker Peksag suggestion files: Doc/whatsnew/3.5.rst | 5 ++--- 1 files changed, 2 insertions(+), 3 deletions(-) diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -4,8 +4,7 @@ :Release: |release| :Date: |today| - -:Author: Elvis Pranskevichus (Editor) +:Editor: Elvis Pranskevichus .. Rules for maintenance: @@ -517,7 +516,7 @@ :pep:`489` -- Multi-phase extension module initialization PEP written by Petr Viktorin, Stefan Behnel, and Nick Coghlan; - implementation by Petr Viktorin. + implemented by Petr Viktorin. PEP 485: A function for testing approximate equality -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 9 15:32:35 2015 From: python-checkins at python.org (yury.selivanov) Date: Wed, 09 Sep 2015 13:32:35 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogTWVyZ2UgMy41?= Message-ID: <20150909133235.12006.83476@psf.io> https://hg.python.org/cpython/rev/90e79bf21eac changeset: 97811:90e79bf21eac parent: 97809:a96cc32f8160 parent: 97810:a172a8035a93 user: Yury Selivanov date: Wed Sep 09 09:32:17 2015 -0400 summary: Merge 3.5 files: Doc/library/compileall.rst | 9 +++------ 1 files changed, 3 insertions(+), 6 deletions(-) diff --git a/Doc/library/compileall.rst b/Doc/library/compileall.rst --- a/Doc/library/compileall.rst +++ b/Doc/library/compileall.rst @@ -88,12 +88,9 @@ Added the ``-i``, ``-b`` and ``-h`` options. .. versionchanged:: 3.5 - - * Added the ``-j`` and ``-r`` options. - * ``-q`` option was changed to a multilevel value. - * ``-qq`` option. - * ``-b`` will always produce a byte-code file ending in ``.pyc``, - never ``.pyo``. + Added the ``-j``, ``-r``, and ``-qq`` options. ``-q`` option + was changed to a multilevel value. ``b`` will always produce a + byte-code file ending in ``.pyc``, never ``.pyo``. There is no command-line option to control the optimization level used by the -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 9 15:32:36 2015 From: python-checkins at python.org (yury.selivanov) Date: Wed, 09 Sep 2015 13:32:36 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogZG9jcy5jb21waWxl?= =?utf-8?q?all=3A_Fix_markup_=28rendering_was_off=2C_noticed_by_Berker_Pek?= =?utf-8?q?sag=29?= Message-ID: <20150909133235.17961.43073@psf.io> https://hg.python.org/cpython/rev/a172a8035a93 changeset: 97810:a172a8035a93 branch: 3.5 parent: 97808:8b9640aa3f1f user: Yury Selivanov date: Wed Sep 09 09:32:07 2015 -0400 summary: docs.compileall: Fix markup (rendering was off, noticed by Berker Peksag) files: Doc/library/compileall.rst | 9 +++------ 1 files changed, 3 insertions(+), 6 deletions(-) diff --git a/Doc/library/compileall.rst b/Doc/library/compileall.rst --- a/Doc/library/compileall.rst +++ b/Doc/library/compileall.rst @@ -88,12 +88,9 @@ Added the ``-i``, ``-b`` and ``-h`` options. .. versionchanged:: 3.5 - - * Added the ``-j`` and ``-r`` options. - * ``-q`` option was changed to a multilevel value. - * ``-qq`` option. - * ``-b`` will always produce a byte-code file ending in ``.pyc``, - never ``.pyo``. + Added the ``-j``, ``-r``, and ``-qq`` options. ``-q`` option + was changed to a multilevel value. ``b`` will always produce a + byte-code file ending in ``.pyc``, never ``.pyo``. There is no command-line option to control the optimization level used by the -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 9 15:56:34 2015 From: python-checkins at python.org (larry.hastings) Date: Wed, 09 Sep 2015 13:56:34 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_Added_Misc/NEW?= =?utf-8?q?S_section_for_Python_3=2E5=2E0_final=2E?= Message-ID: <20150909135634.15732.68765@psf.io> https://hg.python.org/cpython/rev/ebb8fabd132c changeset: 97812:ebb8fabd132c branch: 3.5 parent: 97731:7d320c3bf9c6 user: Larry Hastings date: Tue Sep 08 21:19:48 2015 -0700 summary: Added Misc/NEWS section for Python 3.5.0 final. files: Misc/NEWS | 6 ++++++ 1 files changed, 6 insertions(+), 0 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -2,6 +2,12 @@ Python News +++++++++++ +What's New in Python 3.5.0 final? +=============================================== + +Release date: 2015-09-13 + + What's New in Python 3.5.0 release candidate 3? =============================================== -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 9 15:56:35 2015 From: python-checkins at python.org (larry.hastings) Date: Wed, 09 Sep 2015 13:56:35 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI1MDI5?= =?utf-8?q?=3A_MemoryError_in_test=5Fstrptime?= Message-ID: <20150909135634.68887.35111@psf.io> https://hg.python.org/cpython/rev/bd7aed300a1b changeset: 97813:bd7aed300a1b branch: 3.5 parent: 97731:7d320c3bf9c6 user: Steve Dower date: Tue Sep 08 19:12:51 2015 -0700 summary: Issue #25029: MemoryError in test_strptime files: Modules/timemodule.c | 20 +++++++------------- 1 files changed, 7 insertions(+), 13 deletions(-) diff --git a/Modules/timemodule.c b/Modules/timemodule.c --- a/Modules/timemodule.c +++ b/Modules/timemodule.c @@ -648,9 +648,6 @@ * will be ahead of time... */ for (i = 1024; ; i += i) { -#if defined _MSC_VER && _MSC_VER >= 1400 && defined(__STDC_SECURE_LIB__) - int err; -#endif outbuf = (time_char *)PyMem_Malloc(i*sizeof(time_char)); if (outbuf == NULL) { PyErr_NoMemory(); @@ -660,10 +657,14 @@ buflen = format_time(outbuf, i, fmt, &buf); _Py_END_SUPPRESS_IPH #if defined _MSC_VER && _MSC_VER >= 1400 && defined(__STDC_SECURE_LIB__) - err = errno; + /* VisualStudio .NET 2005 does this properly */ + if (buflen == 0 && errno == EINVAL) { + PyErr_SetString(PyExc_ValueError, "Invalid format string"); + PyMem_Free(outbuf); + break; + } #endif - if (buflen > 0 || fmtlen == 0 || - (fmtlen > 4 && i >= 256 * fmtlen)) { + if (buflen > 0 || i >= 256 * fmtlen) { /* If the buffer is 256 times as long as the format, it's probably not failing for lack of room! More likely, the format yields an empty result, @@ -679,13 +680,6 @@ break; } PyMem_Free(outbuf); -#if defined _MSC_VER && _MSC_VER >= 1400 && defined(__STDC_SECURE_LIB__) - /* VisualStudio .NET 2005 does this properly */ - if (buflen == 0 && err == EINVAL) { - PyErr_SetString(PyExc_ValueError, "Invalid format string"); - break; - } -#endif } #ifdef HAVE_WCSFTIME PyMem_Free(format); -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 9 15:56:35 2015 From: python-checkins at python.org (larry.hastings) Date: Wed, 09 Sep 2015 13:56:35 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_Adds_Mics/NEWS?= =?utf-8?q?_entry_for_issue_=2325029=2E?= Message-ID: <20150909135635.27701.81248@psf.io> https://hg.python.org/cpython/rev/cceaccb6ec5a changeset: 97815:cceaccb6ec5a branch: 3.5 user: Steve Dower date: Tue Sep 08 21:27:47 2015 -0700 summary: Adds Mics/NEWS entry for issue #25029. files: Misc/NEWS | 4 ++++ 1 files changed, 4 insertions(+), 0 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -7,6 +7,10 @@ Release date: 2015-09-13 +Library +------- + +- Issue #25029: Fixes MemoryError in test_strptime What's New in Python 3.5.0 release candidate 3? =============================================== -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 9 15:56:35 2015 From: python-checkins at python.org (larry.hastings) Date: Wed, 09 Sep 2015 13:56:35 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy41IC0+IDMuNSk6?= =?utf-8?q?_Merge_fix_for_=2325029?= Message-ID: <20150909135634.115020.87314@psf.io> https://hg.python.org/cpython/rev/706b9eaba734 changeset: 97814:706b9eaba734 branch: 3.5 parent: 97812:ebb8fabd132c parent: 97813:bd7aed300a1b user: Steve Dower date: Tue Sep 08 21:26:59 2015 -0700 summary: Merge fix for #25029 files: Modules/timemodule.c | 20 +++++++------------- 1 files changed, 7 insertions(+), 13 deletions(-) diff --git a/Modules/timemodule.c b/Modules/timemodule.c --- a/Modules/timemodule.c +++ b/Modules/timemodule.c @@ -648,9 +648,6 @@ * will be ahead of time... */ for (i = 1024; ; i += i) { -#if defined _MSC_VER && _MSC_VER >= 1400 && defined(__STDC_SECURE_LIB__) - int err; -#endif outbuf = (time_char *)PyMem_Malloc(i*sizeof(time_char)); if (outbuf == NULL) { PyErr_NoMemory(); @@ -660,10 +657,14 @@ buflen = format_time(outbuf, i, fmt, &buf); _Py_END_SUPPRESS_IPH #if defined _MSC_VER && _MSC_VER >= 1400 && defined(__STDC_SECURE_LIB__) - err = errno; + /* VisualStudio .NET 2005 does this properly */ + if (buflen == 0 && errno == EINVAL) { + PyErr_SetString(PyExc_ValueError, "Invalid format string"); + PyMem_Free(outbuf); + break; + } #endif - if (buflen > 0 || fmtlen == 0 || - (fmtlen > 4 && i >= 256 * fmtlen)) { + if (buflen > 0 || i >= 256 * fmtlen) { /* If the buffer is 256 times as long as the format, it's probably not failing for lack of room! More likely, the format yields an empty result, @@ -679,13 +680,6 @@ break; } PyMem_Free(outbuf); -#if defined _MSC_VER && _MSC_VER >= 1400 && defined(__STDC_SECURE_LIB__) - /* VisualStudio .NET 2005 does this properly */ - if (buflen == 0 && err == EINVAL) { - PyErr_SetString(PyExc_ValueError, "Invalid format string"); - break; - } -#endif } #ifdef HAVE_WCSFTIME PyMem_Free(format); -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 9 15:56:35 2015 From: python-checkins at python.org (larry.hastings) Date: Wed, 09 Sep 2015 13:56:35 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI1MDI3?= =?utf-8?q?=3A_Reverts_partial-static_build_options_and_adds_vcruntime140?= =?utf-8?q?=2Edll_to?= Message-ID: <20150909135635.68885.80124@psf.io> https://hg.python.org/cpython/rev/8374472c6a6e changeset: 97816:8374472c6a6e branch: 3.5 user: Steve Dower date: Tue Sep 08 21:39:01 2015 -0700 summary: Issue #25027: Reverts partial-static build options and adds vcruntime140.dll to Windows installation. files: Lib/distutils/_msvccompiler.py | 76 +++++++++-- Lib/distutils/tests/test_msvccompiler.py | 58 ++++++++- Misc/NEWS | 7 + PCbuild/pyproject.props | 8 +- PCbuild/tcl.vcxproj | 4 +- PCbuild/tix.vcxproj | 4 +- PCbuild/tk.vcxproj | 4 +- Tools/msi/build.bat | 8 +- Tools/msi/buildrelease.bat | 2 +- Tools/msi/exe/exe_files.wxs | 3 + Tools/msi/make_zip.proj | 3 +- Tools/msi/make_zip.py | 9 +- Tools/msi/msi.props | 3 + 13 files changed, 151 insertions(+), 38 deletions(-) diff --git a/Lib/distutils/_msvccompiler.py b/Lib/distutils/_msvccompiler.py --- a/Lib/distutils/_msvccompiler.py +++ b/Lib/distutils/_msvccompiler.py @@ -14,6 +14,8 @@ # ported to VS 2015 by Steve Dower import os +import shutil +import stat import subprocess from distutils.errors import DistutilsExecError, DistutilsPlatformError, \ @@ -25,7 +27,7 @@ import winreg from itertools import count -def _find_vcvarsall(): +def _find_vcvarsall(plat_spec): with winreg.OpenKeyEx( winreg.HKEY_LOCAL_MACHINE, r"Software\Microsoft\VisualStudio\SxS\VC7", @@ -33,7 +35,7 @@ ) as key: if not key: log.debug("Visual C++ is not registered") - return None + return None, None best_version = 0 best_dir = None @@ -51,14 +53,23 @@ best_version, best_dir = version, vc_dir if not best_version: log.debug("No suitable Visual C++ version found") - return None + return None, None vcvarsall = os.path.join(best_dir, "vcvarsall.bat") if not os.path.isfile(vcvarsall): log.debug("%s cannot be found", vcvarsall) - return None + return None, None - return vcvarsall + vcruntime = None + vcruntime_spec = _VCVARS_PLAT_TO_VCRUNTIME_REDIST.get(plat_spec) + if vcruntime_spec: + vcruntime = os.path.join(best_dir, + vcruntime_spec.format(best_version)) + if not os.path.isfile(vcruntime): + log.debug("%s cannot be found", vcruntime) + vcruntime = None + + return vcvarsall, vcruntime def _get_vc_env(plat_spec): if os.getenv("DISTUTILS_USE_SDK"): @@ -67,7 +78,7 @@ for key, value in os.environ.items() } - vcvarsall = _find_vcvarsall() + vcvarsall, vcruntime = _find_vcvarsall(plat_spec) if not vcvarsall: raise DistutilsPlatformError("Unable to find vcvarsall.bat") @@ -83,12 +94,16 @@ raise DistutilsPlatformError("Error executing {}" .format(exc.cmd)) - return { + env = { key.lower(): value for key, _, value in (line.partition('=') for line in out.splitlines()) if key and value } + + if vcruntime: + env['py_vcruntime_redist'] = vcruntime + return env def _find_exe(exe, paths=None): """Return path to an MSVC executable program. @@ -115,6 +130,20 @@ 'win-amd64' : 'amd64', } +# A map keyed by get_platform() return values to the file under +# the VC install directory containing the vcruntime redistributable. +_VCVARS_PLAT_TO_VCRUNTIME_REDIST = { + 'x86' : 'redist\\x86\\Microsoft.VC{0}0.CRT\\vcruntime{0}0.dll', + 'amd64' : 'redist\\x64\\Microsoft.VC{0}0.CRT\\vcruntime{0}0.dll', + 'x86_amd64' : 'redist\\x64\\Microsoft.VC{0}0.CRT\\vcruntime{0}0.dll', +} + +# A set containing the DLLs that are guaranteed to be available for +# all micro versions of this Python version. Known extension +# dependencies that are not in this set will be copied to the output +# path. +_BUNDLED_DLLS = frozenset(['vcruntime140.dll']) + class MSVCCompiler(CCompiler) : """Concrete class that implements an interface to Microsoft Visual C++, as defined by the CCompiler abstract class.""" @@ -189,6 +218,7 @@ self.rc = _find_exe("rc.exe", paths) # resource compiler self.mc = _find_exe("mc.exe", paths) # message compiler self.mt = _find_exe("mt.exe", paths) # message compiler + self._vcruntime_redist = vc_env.get('py_vcruntime_redist', '') for dir in vc_env.get('include', '').split(os.pathsep): if dir: @@ -199,20 +229,26 @@ self.add_library_dir(dir) self.preprocess_options = None - # Use /MT[d] to build statically, then switch from libucrt[d].lib to ucrt[d].lib + # If vcruntime_redist is available, link against it dynamically. Otherwise, + # use /MT[d] to build statically, then switch from libucrt[d].lib to ucrt[d].lib # later to dynamically link to ucrtbase but not vcruntime. self.compile_options = [ - '/nologo', '/Ox', '/MT', '/W3', '/GL', '/DNDEBUG' + '/nologo', '/Ox', '/W3', '/GL', '/DNDEBUG' ] + self.compile_options.append('/MD' if self._vcruntime_redist else '/MT') + self.compile_options_debug = [ - '/nologo', '/Od', '/MTd', '/Zi', '/W3', '/D_DEBUG' + '/nologo', '/Od', '/MDd', '/Zi', '/W3', '/D_DEBUG' ] ldflags = [ - '/nologo', '/INCREMENTAL:NO', '/LTCG', '/nodefaultlib:libucrt.lib', 'ucrt.lib', + '/nologo', '/INCREMENTAL:NO', '/LTCG' ] + if not self._vcruntime_redist: + ldflags.extend(('/nodefaultlib:libucrt.lib', 'ucrt.lib')) + ldflags_debug = [ - '/nologo', '/INCREMENTAL:NO', '/LTCG', '/DEBUG:FULL', '/nodefaultlib:libucrtd.lib', 'ucrtd.lib', + '/nologo', '/INCREMENTAL:NO', '/LTCG', '/DEBUG:FULL' ] self.ldflags_exe = [*ldflags, '/MANIFEST:EMBED,ID=1'] @@ -446,15 +482,29 @@ if extra_postargs: ld_args.extend(extra_postargs) - self.mkpath(os.path.dirname(output_filename)) + output_dir = os.path.dirname(os.path.abspath(output_filename)) + self.mkpath(output_dir) try: log.debug('Executing "%s" %s', self.linker, ' '.join(ld_args)) self.spawn([self.linker] + ld_args) + self._copy_vcruntime(output_dir) except DistutilsExecError as msg: raise LinkError(msg) else: log.debug("skipping %s (up-to-date)", output_filename) + def _copy_vcruntime(self, output_dir): + vcruntime = self._vcruntime_redist + if not vcruntime or not os.path.isfile(vcruntime): + return + + if os.path.basename(vcruntime).lower() in _BUNDLED_DLLS: + return + + log.debug('Copying "%s"', vcruntime) + vcruntime = shutil.copy(vcruntime, output_dir) + os.chmod(vcruntime, stat.S_IWRITE) + def spawn(self, cmd): old_path = os.getenv('path') try: diff --git a/Lib/distutils/tests/test_msvccompiler.py b/Lib/distutils/tests/test_msvccompiler.py --- a/Lib/distutils/tests/test_msvccompiler.py +++ b/Lib/distutils/tests/test_msvccompiler.py @@ -3,6 +3,8 @@ import unittest import os +import distutils._msvccompiler as _msvccompiler + from distutils.errors import DistutilsPlatformError from distutils.tests import support from test.support import run_unittest @@ -19,19 +21,65 @@ # makes sure query_vcvarsall raises # a DistutilsPlatformError if the compiler # is not found - from distutils._msvccompiler import _get_vc_env - def _find_vcvarsall(): - return None + def _find_vcvarsall(plat_spec): + return None, None - import distutils._msvccompiler as _msvccompiler old_find_vcvarsall = _msvccompiler._find_vcvarsall _msvccompiler._find_vcvarsall = _find_vcvarsall try: - self.assertRaises(DistutilsPlatformError, _get_vc_env, + self.assertRaises(DistutilsPlatformError, + _msvccompiler._get_vc_env, 'wont find this version') finally: _msvccompiler._find_vcvarsall = old_find_vcvarsall + def test_compiler_options(self): + # suppress path to vcruntime from _find_vcvarsall to + # check that /MT is added to compile options + old_find_vcvarsall = _msvccompiler._find_vcvarsall + def _find_vcvarsall(plat_spec): + return old_find_vcvarsall(plat_spec)[0], None + _msvccompiler._find_vcvarsall = _find_vcvarsall + try: + compiler = _msvccompiler.MSVCCompiler() + compiler.initialize() + + self.assertIn('/MT', compiler.compile_options) + self.assertNotIn('/MD', compiler.compile_options) + finally: + _msvccompiler._find_vcvarsall = old_find_vcvarsall + + def test_vcruntime_copy(self): + # force path to a known file - it doesn't matter + # what we copy as long as its name is not in + # _msvccompiler._BUNDLED_DLLS + old_find_vcvarsall = _msvccompiler._find_vcvarsall + def _find_vcvarsall(plat_spec): + return old_find_vcvarsall(plat_spec)[0], __file__ + _msvccompiler._find_vcvarsall = _find_vcvarsall + try: + tempdir = self.mkdtemp() + compiler = _msvccompiler.MSVCCompiler() + compiler.initialize() + compiler._copy_vcruntime(tempdir) + + self.assertTrue(os.path.isfile(os.path.join( + tempdir, os.path.basename(__file__)))) + finally: + _msvccompiler._find_vcvarsall = old_find_vcvarsall + + def test_vcruntime_skip_copy(self): + tempdir = self.mkdtemp() + compiler = _msvccompiler.MSVCCompiler() + compiler.initialize() + dll = compiler._vcruntime_redist + self.assertTrue(os.path.isfile(dll)) + + compiler._copy_vcruntime(tempdir) + + self.assertFalse(os.path.isfile(os.path.join( + tempdir, os.path.basename(dll)))) + def test_suite(): return unittest.makeSuite(msvccompilerTestCase) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -12,6 +12,13 @@ - Issue #25029: Fixes MemoryError in test_strptime +Build +----- + +- Issue #25027: Reverts partial-static build options and adds + vcruntime140.dll to Windows installation. + + What's New in Python 3.5.0 release candidate 3? =============================================== diff --git a/PCbuild/pyproject.props b/PCbuild/pyproject.props --- a/PCbuild/pyproject.props +++ b/PCbuild/pyproject.props @@ -36,7 +36,7 @@ true true - MultiThreaded + MultiThreadedDLL true Level3 ProgramDatabase @@ -47,7 +47,7 @@ Disabled false - MultiThreadedDebug + MultiThreadedDebugDLL $(OutDir);%(AdditionalLibraryDirectories) @@ -57,9 +57,7 @@ true true true - ucrtd.lib;%(AdditionalDependencies) - ucrt.lib;%(AdditionalDependencies) - LIBC;libucrt.lib;libucrtd.lib;%(IgnoreSpecificDefaultLibraries) + LIBC;%(IgnoreSpecificDefaultLibraries) MachineX86 MachineX64 $(OutDir)$(TargetName).pgd diff --git a/PCbuild/tcl.vcxproj b/PCbuild/tcl.vcxproj --- a/PCbuild/tcl.vcxproj +++ b/PCbuild/tcl.vcxproj @@ -61,8 +61,8 @@ - ucrt - symbols,ucrt + msvcrt + symbols,msvcrt INSTALLDIR="$(OutDir.TrimEnd(`\`))" INSTALL_DIR="$(OutDir.TrimEnd(`\`))" DEBUGFLAGS="-wd4456 -wd4457 -wd4458 -wd4459 -wd4996" setlocal diff --git a/PCbuild/tix.vcxproj b/PCbuild/tix.vcxproj --- a/PCbuild/tix.vcxproj +++ b/PCbuild/tix.vcxproj @@ -57,8 +57,8 @@ BUILDDIRTOP="$(BuildDirTop)" TCL_DIR="$(tclDir.TrimEnd(`\`))" TK_DIR="$(tkDir.TrimEnd(`\`))" INSTALL_DIR="$(OutDir.TrimEnd(`\`))" - DEBUG=1 NODEBUG=0 UCRT=1 TCL_DBGX=g TK_DBGX=g - DEBUG=0 NODEBUG=1 UCRT=1 + DEBUG=1 NODEBUG=0 TCL_DBGX=g TK_DBGX=g + DEBUG=0 NODEBUG=1 setlocal @(ExpectedOutputs->'if not exist "%(FullPath)" goto build',' ') diff --git a/PCbuild/tk.vcxproj b/PCbuild/tk.vcxproj --- a/PCbuild/tk.vcxproj +++ b/PCbuild/tk.vcxproj @@ -60,8 +60,8 @@ - ucrt - symbols,ucrt + msvcrt + symbols,msvcrt TCLDIR="$(tclDir.TrimEnd(`\`))" INSTALLDIR="$(OutDir.TrimEnd(`\`))" DEBUGFLAGS="-wd4456 -wd4457 -wd4458 -wd4459 -wd4996" setlocal diff --git a/Tools/msi/build.bat b/Tools/msi/build.bat --- a/Tools/msi/build.bat +++ b/Tools/msi/build.bat @@ -20,15 +20,15 @@ call "%PCBUILD%env.bat" x86 if defined BUILDX86 ( - call "%PCBUILD%build.bat" -d + call "%PCBUILD%build.bat" -d -e if errorlevel 1 goto :eof - call "%PCBUILD%build.bat" + call "%PCBUILD%build.bat" -e if errorlevel 1 goto :eof ) if defined BUILDX64 ( - call "%PCBUILD%build.bat" -p x64 -d + call "%PCBUILD%build.bat" -p x64 -d -e if errorlevel 1 goto :eof - call "%PCBUILD%build.bat" -p x64 + call "%PCBUILD%build.bat" -p x64 -e if errorlevel 1 goto :eof ) diff --git a/Tools/msi/buildrelease.bat b/Tools/msi/buildrelease.bat --- a/Tools/msi/buildrelease.bat +++ b/Tools/msi/buildrelease.bat @@ -121,7 +121,7 @@ if not "%SKIPBUILD%" EQU "1" ( call "%PCBUILD%build.bat" -e -p %BUILD_PLAT% -d -t %TARGET% %CERTOPTS% if errorlevel 1 exit /B - call "%PCBUILD%build.bat" -p %BUILD_PLAT% -t %TARGET% %CERTOPTS% + call "%PCBUILD%build.bat" -e -p %BUILD_PLAT% -t %TARGET% %CERTOPTS% if errorlevel 1 exit /B @rem build.bat turns echo back on, so we disable it again @echo off diff --git a/Tools/msi/exe/exe_files.wxs b/Tools/msi/exe/exe_files.wxs --- a/Tools/msi/exe/exe_files.wxs +++ b/Tools/msi/exe/exe_files.wxs @@ -29,6 +29,9 @@ + + + diff --git a/Tools/msi/make_zip.proj b/Tools/msi/make_zip.proj --- a/Tools/msi/make_zip.proj +++ b/Tools/msi/make_zip.proj @@ -16,7 +16,8 @@ $(OutputPath)\en-us\$(TargetName)$(TargetExt) "$(PythonExe)" "$(MSBuildThisFileDirectory)\make_zip.py" $(Arguments) -e -o "$(TargetPath)" -t "$(IntermediateOutputPath)\zip_$(ArchName)" -a $(ArchName) - set DOC_FILENAME=python$(PythonVersion).chm + set DOC_FILENAME=python$(PythonVersion).chm +set VCREDIST_PATH=$(VS140COMNTOOLS)\..\..\VC\redist\$(Platform)\Microsoft.VC140.CRT diff --git a/Tools/msi/make_zip.py b/Tools/msi/make_zip.py --- a/Tools/msi/make_zip.py +++ b/Tools/msi/make_zip.py @@ -64,9 +64,6 @@ ('Tools/', 'Tools', '**/*', include_in_tools), ] -if os.getenv('DOC_FILENAME'): - FULL_LAYOUT.append(('Doc/', 'Doc/build/htmlhelp', os.getenv('DOC_FILENAME'), None)) - EMBED_LAYOUT = [ ('/', 'PCBuild/$arch', 'python*.exe', is_not_debug), ('/', 'PCBuild/$arch', '*.pyd', is_not_debug), @@ -74,6 +71,12 @@ ('python35.zip', 'Lib', '**/*', include_in_lib), ] +if os.getenv('DOC_FILENAME'): + FULL_LAYOUT.append(('Doc/', 'Doc/build/htmlhelp', os.getenv('DOC_FILENAME'), None)) +if os.getenv('VCREDIST_PATH'): + FULL_LAYOUT.append(('/', os.getenv('VCREDIST_PATH'), 'vcruntime*.dll', None)) + EMBED_LAYOUT.append(('/', os.getenv('VCREDIST_PATH'), 'vcruntime*.dll', None)) + def copy_to_layout(target, rel_sources): count = 0 diff --git a/Tools/msi/msi.props b/Tools/msi/msi.props --- a/Tools/msi/msi.props +++ b/Tools/msi/msi.props @@ -118,6 +118,9 @@ redist + + redist + -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 9 15:56:35 2015 From: python-checkins at python.org (larry.hastings) Date: Wed, 09 Sep 2015 13:56:35 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy41IC0+IDMuNSk6?= =?utf-8?q?_Merge_3=2E5=2E0rc3_revisions_back_into_current_3=2E5=2E0_head?= =?utf-8?q?=2E?= Message-ID: <20150909135635.14855.93283@psf.io> https://hg.python.org/cpython/rev/7a33f97feaa1 changeset: 97817:7a33f97feaa1 branch: 3.5 parent: 97816:8374472c6a6e parent: 97747:170af6e33c9c user: Larry Hastings date: Tue Sep 08 22:45:37 2015 -0700 summary: Merge 3.5.0rc3 revisions back into current 3.5.0 head. files: .hgtags | 1 + Include/patchlevel.h | 4 ++-- Lib/pydoc_data/topics.py | 2 +- Misc/NEWS | 2 +- README | 2 +- 5 files changed, 6 insertions(+), 5 deletions(-) diff --git a/.hgtags b/.hgtags --- a/.hgtags +++ b/.hgtags @@ -154,3 +154,4 @@ c0d64105463581f85d0e368e8d6e59b7fd8f12b1 v3.5.0b4 1a58b1227501e046eee13d90f113417b60843301 v3.5.0rc1 cc15d736d860303b9da90d43cd32db39bab048df v3.5.0rc2 +66ed52375df802f9d0a34480daaa8ce79fc41313 v3.5.0rc3 diff --git a/Include/patchlevel.h b/Include/patchlevel.h --- a/Include/patchlevel.h +++ b/Include/patchlevel.h @@ -20,10 +20,10 @@ #define PY_MINOR_VERSION 5 #define PY_MICRO_VERSION 0 #define PY_RELEASE_LEVEL PY_RELEASE_LEVEL_GAMMA -#define PY_RELEASE_SERIAL 2 +#define PY_RELEASE_SERIAL 3 /* Version as a string */ -#define PY_VERSION "3.5.0rc2+" +#define PY_VERSION "3.5.0rc3" /*--end constants--*/ /* Version as a single 4-byte hex number, e.g. 0x010502B2 == 1.5.2b2. diff --git a/Lib/pydoc_data/topics.py b/Lib/pydoc_data/topics.py --- a/Lib/pydoc_data/topics.py +++ b/Lib/pydoc_data/topics.py @@ -1,5 +1,5 @@ # -*- coding: utf-8 -*- -# Autogenerated by Sphinx on Mon Aug 24 20:29:23 2015 +# Autogenerated by Sphinx on Mon Sep 7 05:10:25 2015 topics = {'assert': u'\nThe "assert" statement\n**********************\n\nAssert statements are a convenient way to insert debugging assertions\ninto a program:\n\n assert_stmt ::= "assert" expression ["," expression]\n\nThe simple form, "assert expression", is equivalent to\n\n if __debug__:\n if not expression: raise AssertionError\n\nThe extended form, "assert expression1, expression2", is equivalent to\n\n if __debug__:\n if not expression1: raise AssertionError(expression2)\n\nThese equivalences assume that "__debug__" and "AssertionError" refer\nto the built-in variables with those names. In the current\nimplementation, the built-in variable "__debug__" is "True" under\nnormal circumstances, "False" when optimization is requested (command\nline option -O). The current code generator emits no code for an\nassert statement when optimization is requested at compile time. Note\nthat it is unnecessary to include the source code for the expression\nthat failed in the error message; it will be displayed as part of the\nstack trace.\n\nAssignments to "__debug__" are illegal. The value for the built-in\nvariable is determined when the interpreter starts.\n', 'assignment': u'\nAssignment statements\n*********************\n\nAssignment statements are used to (re)bind names to values and to\nmodify attributes or items of mutable objects:\n\n assignment_stmt ::= (target_list "=")+ (expression_list | yield_expression)\n target_list ::= target ("," target)* [","]\n target ::= identifier\n | "(" target_list ")"\n | "[" target_list "]"\n | attributeref\n | subscription\n | slicing\n | "*" target\n\n(See section *Primaries* for the syntax definitions for\n*attributeref*, *subscription*, and *slicing*.)\n\nAn assignment statement evaluates the expression list (remember that\nthis can be a single expression or a comma-separated list, the latter\nyielding a tuple) and assigns the single resulting object to each of\nthe target lists, from left to right.\n\nAssignment is defined recursively depending on the form of the target\n(list). When a target is part of a mutable object (an attribute\nreference, subscription or slicing), the mutable object must\nultimately perform the assignment and decide about its validity, and\nmay raise an exception if the assignment is unacceptable. The rules\nobserved by various types and the exceptions raised are given with the\ndefinition of the object types (see section *The standard type\nhierarchy*).\n\nAssignment of an object to a target list, optionally enclosed in\nparentheses or square brackets, is recursively defined as follows.\n\n* If the target list is a single target: The object is assigned to\n that target.\n\n* If the target list is a comma-separated list of targets: The\n object must be an iterable with the same number of items as there\n are targets in the target list, and the items are assigned, from\n left to right, to the corresponding targets.\n\n * If the target list contains one target prefixed with an\n asterisk, called a "starred" target: The object must be a sequence\n with at least as many items as there are targets in the target\n list, minus one. The first items of the sequence are assigned,\n from left to right, to the targets before the starred target. The\n final items of the sequence are assigned to the targets after the\n starred target. A list of the remaining items in the sequence is\n then assigned to the starred target (the list can be empty).\n\n * Else: The object must be a sequence with the same number of\n items as there are targets in the target list, and the items are\n assigned, from left to right, to the corresponding targets.\n\nAssignment of an object to a single target is recursively defined as\nfollows.\n\n* If the target is an identifier (name):\n\n * If the name does not occur in a "global" or "nonlocal" statement\n in the current code block: the name is bound to the object in the\n current local namespace.\n\n * Otherwise: the name is bound to the object in the global\n namespace or the outer namespace determined by "nonlocal",\n respectively.\n\n The name is rebound if it was already bound. This may cause the\n reference count for the object previously bound to the name to reach\n zero, causing the object to be deallocated and its destructor (if it\n has one) to be called.\n\n* If the target is a target list enclosed in parentheses or in\n square brackets: The object must be an iterable with the same number\n of items as there are targets in the target list, and its items are\n assigned, from left to right, to the corresponding targets.\n\n* If the target is an attribute reference: The primary expression in\n the reference is evaluated. It should yield an object with\n assignable attributes; if this is not the case, "TypeError" is\n raised. That object is then asked to assign the assigned object to\n the given attribute; if it cannot perform the assignment, it raises\n an exception (usually but not necessarily "AttributeError").\n\n Note: If the object is a class instance and the attribute reference\n occurs on both sides of the assignment operator, the RHS expression,\n "a.x" can access either an instance attribute or (if no instance\n attribute exists) a class attribute. The LHS target "a.x" is always\n set as an instance attribute, creating it if necessary. Thus, the\n two occurrences of "a.x" do not necessarily refer to the same\n attribute: if the RHS expression refers to a class attribute, the\n LHS creates a new instance attribute as the target of the\n assignment:\n\n class Cls:\n x = 3 # class variable\n inst = Cls()\n inst.x = inst.x + 1 # writes inst.x as 4 leaving Cls.x as 3\n\n This description does not necessarily apply to descriptor\n attributes, such as properties created with "property()".\n\n* If the target is a subscription: The primary expression in the\n reference is evaluated. It should yield either a mutable sequence\n object (such as a list) or a mapping object (such as a dictionary).\n Next, the subscript expression is evaluated.\n\n If the primary is a mutable sequence object (such as a list), the\n subscript must yield an integer. If it is negative, the sequence\'s\n length is added to it. The resulting value must be a nonnegative\n integer less than the sequence\'s length, and the sequence is asked\n to assign the assigned object to its item with that index. If the\n index is out of range, "IndexError" is raised (assignment to a\n subscripted sequence cannot add new items to a list).\n\n If the primary is a mapping object (such as a dictionary), the\n subscript must have a type compatible with the mapping\'s key type,\n and the mapping is then asked to create a key/datum pair which maps\n the subscript to the assigned object. This can either replace an\n existing key/value pair with the same key value, or insert a new\n key/value pair (if no key with the same value existed).\n\n For user-defined objects, the "__setitem__()" method is called with\n appropriate arguments.\n\n* If the target is a slicing: The primary expression in the\n reference is evaluated. It should yield a mutable sequence object\n (such as a list). The assigned object should be a sequence object\n of the same type. Next, the lower and upper bound expressions are\n evaluated, insofar they are present; defaults are zero and the\n sequence\'s length. The bounds should evaluate to integers. If\n either bound is negative, the sequence\'s length is added to it. The\n resulting bounds are clipped to lie between zero and the sequence\'s\n length, inclusive. Finally, the sequence object is asked to replace\n the slice with the items of the assigned sequence. The length of\n the slice may be different from the length of the assigned sequence,\n thus changing the length of the target sequence, if the target\n sequence allows it.\n\n**CPython implementation detail:** In the current implementation, the\nsyntax for targets is taken to be the same as for expressions, and\ninvalid syntax is rejected during the code generation phase, causing\nless detailed error messages.\n\nAlthough the definition of assignment implies that overlaps between\nthe left-hand side and the right-hand side are \'simultanenous\' (for\nexample "a, b = b, a" swaps two variables), overlaps *within* the\ncollection of assigned-to variables occur left-to-right, sometimes\nresulting in confusion. For instance, the following program prints\n"[0, 2]":\n\n x = [0, 1]\n i = 0\n i, x[i] = 1, 2 # i is updated, then x[i] is updated\n print(x)\n\nSee also: **PEP 3132** - Extended Iterable Unpacking\n\n The specification for the "*target" feature.\n\n\nAugmented assignment statements\n===============================\n\nAugmented assignment is the combination, in a single statement, of a\nbinary operation and an assignment statement:\n\n augmented_assignment_stmt ::= augtarget augop (expression_list | yield_expression)\n augtarget ::= identifier | attributeref | subscription | slicing\n augop ::= "+=" | "-=" | "*=" | "@=" | "/=" | "//=" | "%=" | "**="\n | ">>=" | "<<=" | "&=" | "^=" | "|="\n\n(See section *Primaries* for the syntax definitions of the last three\nsymbols.)\n\nAn augmented assignment evaluates the target (which, unlike normal\nassignment statements, cannot be an unpacking) and the expression\nlist, performs the binary operation specific to the type of assignment\non the two operands, and assigns the result to the original target.\nThe target is only evaluated once.\n\nAn augmented assignment expression like "x += 1" can be rewritten as\n"x = x + 1" to achieve a similar, but not exactly equal effect. In the\naugmented version, "x" is only evaluated once. Also, when possible,\nthe actual operation is performed *in-place*, meaning that rather than\ncreating a new object and assigning that to the target, the old object\nis modified instead.\n\nUnlike normal assignments, augmented assignments evaluate the left-\nhand side *before* evaluating the right-hand side. For example, "a[i]\n+= f(x)" first looks-up "a[i]", then it evaluates "f(x)" and performs\nthe addition, and lastly, it writes the result back to "a[i]".\n\nWith the exception of assigning to tuples and multiple targets in a\nsingle statement, the assignment done by augmented assignment\nstatements is handled the same way as normal assignments. Similarly,\nwith the exception of the possible *in-place* behavior, the binary\noperation performed by augmented assignment is the same as the normal\nbinary operations.\n\nFor targets which are attribute references, the same *caveat about\nclass and instance attributes* applies as for regular assignments.\n', 'atom-identifiers': u'\nIdentifiers (Names)\n*******************\n\nAn identifier occurring as an atom is a name. See section\n*Identifiers and keywords* for lexical definition and section *Naming\nand binding* for documentation of naming and binding.\n\nWhen the name is bound to an object, evaluation of the atom yields\nthat object. When a name is not bound, an attempt to evaluate it\nraises a "NameError" exception.\n\n**Private name mangling:** When an identifier that textually occurs in\na class definition begins with two or more underscore characters and\ndoes not end in two or more underscores, it is considered a *private\nname* of that class. Private names are transformed to a longer form\nbefore code is generated for them. The transformation inserts the\nclass name, with leading underscores removed and a single underscore\ninserted, in front of the name. For example, the identifier "__spam"\noccurring in a class named "Ham" will be transformed to "_Ham__spam".\nThis transformation is independent of the syntactical context in which\nthe identifier is used. If the transformed name is extremely long\n(longer than 255 characters), implementation defined truncation may\nhappen. If the class name consists only of underscores, no\ntransformation is done.\n', diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -22,7 +22,7 @@ What's New in Python 3.5.0 release candidate 3? =============================================== -Release date: 2015-09-06 +Release date: 2015-09-07 Core and Builtins ----------------- diff --git a/README b/README --- a/README +++ b/README @@ -1,4 +1,4 @@ -This is Python version 3.5.0 release candidate 2 +This is Python version 3.5.0 release candidate 3 ================================================ Copyright (c) 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010, 2011, -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 9 15:56:40 2015 From: python-checkins at python.org (larry.hastings) Date: Wed, 09 Sep 2015 13:56:40 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy41IC0+IDMuNSk6?= =?utf-8?q?_Merged_in_stevedower/cpython350_=28pull_request_=2323=29?= Message-ID: <20150909135635.66876.76248@psf.io> https://hg.python.org/cpython/rev/00ca2c8e2506 changeset: 97819:00ca2c8e2506 branch: 3.5 parent: 97817:7a33f97feaa1 parent: 97818:1fb824c2e1d7 user: Larry Hastings date: Tue Sep 08 23:45:23 2015 -0700 summary: Merged in stevedower/cpython350 (pull request #23) Moves distutils test import within skippable class. files: Lib/distutils/tests/test_msvccompiler.py | 7 +++++-- 1 files changed, 5 insertions(+), 2 deletions(-) diff --git a/Lib/distutils/tests/test_msvccompiler.py b/Lib/distutils/tests/test_msvccompiler.py --- a/Lib/distutils/tests/test_msvccompiler.py +++ b/Lib/distutils/tests/test_msvccompiler.py @@ -3,8 +3,6 @@ import unittest import os -import distutils._msvccompiler as _msvccompiler - from distutils.errors import DistutilsPlatformError from distutils.tests import support from test.support import run_unittest @@ -18,6 +16,7 @@ unittest.TestCase): def test_no_compiler(self): + import distutils._msvccompiler as _msvccompiler # makes sure query_vcvarsall raises # a DistutilsPlatformError if the compiler # is not found @@ -34,6 +33,7 @@ _msvccompiler._find_vcvarsall = old_find_vcvarsall def test_compiler_options(self): + import distutils._msvccompiler as _msvccompiler # suppress path to vcruntime from _find_vcvarsall to # check that /MT is added to compile options old_find_vcvarsall = _msvccompiler._find_vcvarsall @@ -50,6 +50,7 @@ _msvccompiler._find_vcvarsall = old_find_vcvarsall def test_vcruntime_copy(self): + import distutils._msvccompiler as _msvccompiler # force path to a known file - it doesn't matter # what we copy as long as its name is not in # _msvccompiler._BUNDLED_DLLS @@ -69,6 +70,8 @@ _msvccompiler._find_vcvarsall = old_find_vcvarsall def test_vcruntime_skip_copy(self): + import distutils._msvccompiler as _msvccompiler + tempdir = self.mkdtemp() compiler = _msvccompiler.MSVCCompiler() compiler.initialize() -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 9 15:56:42 2015 From: python-checkins at python.org (larry.hastings) Date: Wed, 09 Sep 2015 13:56:42 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_Whitespace_fix?= =?utf-8?q?es_to_make_the_commit_hook_on_hg=2Epython=2Eorg_happy=2E?= Message-ID: <20150909135642.17979.43174@psf.io> https://hg.python.org/cpython/rev/7eef1e886138 changeset: 97824:7eef1e886138 branch: 3.5 user: Larry Hastings date: Wed Sep 09 06:54:57 2015 -0700 summary: Whitespace fixes to make the commit hook on hg.python.org happy. files: Lib/distutils/_msvccompiler.py | 4 ++-- Lib/distutils/tests/test_msvccompiler.py | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/Lib/distutils/_msvccompiler.py b/Lib/distutils/_msvccompiler.py --- a/Lib/distutils/_msvccompiler.py +++ b/Lib/distutils/_msvccompiler.py @@ -100,7 +100,7 @@ (line.partition('=') for line in out.splitlines()) if key and value } - + if vcruntime: env['py_vcruntime_redist'] = vcruntime return env @@ -236,7 +236,7 @@ '/nologo', '/Ox', '/W3', '/GL', '/DNDEBUG' ] self.compile_options.append('/MD' if self._vcruntime_redist else '/MT') - + self.compile_options_debug = [ '/nologo', '/Od', '/MDd', '/Zi', '/W3', '/D_DEBUG' ] diff --git a/Lib/distutils/tests/test_msvccompiler.py b/Lib/distutils/tests/test_msvccompiler.py --- a/Lib/distutils/tests/test_msvccompiler.py +++ b/Lib/distutils/tests/test_msvccompiler.py @@ -77,7 +77,7 @@ compiler.initialize() dll = compiler._vcruntime_redist self.assertTrue(os.path.isfile(dll)) - + compiler._copy_vcruntime(tempdir) self.assertFalse(os.path.isfile(os.path.join( -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 9 15:56:50 2015 From: python-checkins at python.org (larry.hastings) Date: Wed, 09 Sep 2015 13:56:50 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_Post-release_u?= =?utf-8?q?pdate_for_Python_3=2E5=2E0rc4=2E?= Message-ID: <20150909135641.114864.85968@psf.io> https://hg.python.org/cpython/rev/2d2c84821f2a changeset: 97822:2d2c84821f2a branch: 3.5 user: Larry Hastings date: Wed Sep 09 06:45:19 2015 -0700 summary: Post-release update for Python 3.5.0rc4. files: Include/patchlevel.h | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Include/patchlevel.h b/Include/patchlevel.h --- a/Include/patchlevel.h +++ b/Include/patchlevel.h @@ -23,7 +23,7 @@ #define PY_RELEASE_SERIAL 4 /* Version as a string */ -#define PY_VERSION "3.5.0rc4" +#define PY_VERSION "3.5.0rc4+" /*--end constants--*/ /* Version as a single 4-byte hex number, e.g. 0x010502B2 == 1.5.2b2. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 9 15:56:50 2015 From: python-checkins at python.org (larry.hastings) Date: Wed, 09 Sep 2015 13:56:50 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_Added_tag_v3?= =?utf-8?q?=2E5=2E0rc4_for_changeset_2d033fedfa7f?= Message-ID: <20150909135641.66850.97618@psf.io> https://hg.python.org/cpython/rev/9e1b2a4c47d2 changeset: 97821:9e1b2a4c47d2 branch: 3.5 user: Larry Hastings date: Tue Sep 08 23:58:21 2015 -0700 summary: Added tag v3.5.0rc4 for changeset 2d033fedfa7f files: .hgtags | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diff --git a/.hgtags b/.hgtags --- a/.hgtags +++ b/.hgtags @@ -155,3 +155,4 @@ 1a58b1227501e046eee13d90f113417b60843301 v3.5.0rc1 cc15d736d860303b9da90d43cd32db39bab048df v3.5.0rc2 66ed52375df802f9d0a34480daaa8ce79fc41313 v3.5.0rc3 +2d033fedfa7f1e325fd14ccdaa9cb42155da206f v3.5.0rc4 -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 9 15:56:50 2015 From: python-checkins at python.org (larry.hastings) Date: Wed, 09 Sep 2015 13:56:50 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy41IC0+IDMuNSk6?= =?utf-8?q?_Merge_Python_3=2E5=2E0rc4_back_to_hg=2Epython=2Eorg=2E?= Message-ID: <20150909135642.68857.72039@psf.io> https://hg.python.org/cpython/rev/28b52c252205 changeset: 97823:28b52c252205 branch: 3.5 parent: 97810:a172a8035a93 parent: 97822:2d2c84821f2a user: Larry Hastings date: Wed Sep 09 06:52:38 2015 -0700 summary: Merge Python 3.5.0rc4 back to hg.python.org. files: .hgtags | 1 + Include/patchlevel.h | 4 +- Lib/distutils/_msvccompiler.py | 76 +++++++++-- Lib/distutils/tests/test_msvccompiler.py | 61 ++++++++- Misc/NEWS | 23 +++- Modules/timemodule.c | 20 +-- PCbuild/pyproject.props | 8 +- PCbuild/tcl.vcxproj | 4 +- PCbuild/tix.vcxproj | 4 +- PCbuild/tk.vcxproj | 4 +- README | 2 +- Tools/msi/build.bat | 8 +- Tools/msi/buildrelease.bat | 2 +- Tools/msi/exe/exe_files.wxs | 3 + Tools/msi/make_zip.proj | 3 +- Tools/msi/make_zip.py | 9 +- Tools/msi/msi.props | 3 + 17 files changed, 178 insertions(+), 57 deletions(-) diff --git a/.hgtags b/.hgtags --- a/.hgtags +++ b/.hgtags @@ -155,3 +155,4 @@ 1a58b1227501e046eee13d90f113417b60843301 v3.5.0rc1 cc15d736d860303b9da90d43cd32db39bab048df v3.5.0rc2 66ed52375df802f9d0a34480daaa8ce79fc41313 v3.5.0rc3 +2d033fedfa7f1e325fd14ccdaa9cb42155da206f v3.5.0rc4 diff --git a/Include/patchlevel.h b/Include/patchlevel.h --- a/Include/patchlevel.h +++ b/Include/patchlevel.h @@ -20,10 +20,10 @@ #define PY_MINOR_VERSION 5 #define PY_MICRO_VERSION 0 #define PY_RELEASE_LEVEL PY_RELEASE_LEVEL_GAMMA -#define PY_RELEASE_SERIAL 3 +#define PY_RELEASE_SERIAL 4 /* Version as a string */ -#define PY_VERSION "3.5.0rc3" +#define PY_VERSION "3.5.0rc4+" /*--end constants--*/ /* Version as a single 4-byte hex number, e.g. 0x010502B2 == 1.5.2b2. diff --git a/Lib/distutils/_msvccompiler.py b/Lib/distutils/_msvccompiler.py --- a/Lib/distutils/_msvccompiler.py +++ b/Lib/distutils/_msvccompiler.py @@ -14,6 +14,8 @@ # ported to VS 2015 by Steve Dower import os +import shutil +import stat import subprocess from distutils.errors import DistutilsExecError, DistutilsPlatformError, \ @@ -25,7 +27,7 @@ import winreg from itertools import count -def _find_vcvarsall(): +def _find_vcvarsall(plat_spec): with winreg.OpenKeyEx( winreg.HKEY_LOCAL_MACHINE, r"Software\Microsoft\VisualStudio\SxS\VC7", @@ -33,7 +35,7 @@ ) as key: if not key: log.debug("Visual C++ is not registered") - return None + return None, None best_version = 0 best_dir = None @@ -51,14 +53,23 @@ best_version, best_dir = version, vc_dir if not best_version: log.debug("No suitable Visual C++ version found") - return None + return None, None vcvarsall = os.path.join(best_dir, "vcvarsall.bat") if not os.path.isfile(vcvarsall): log.debug("%s cannot be found", vcvarsall) - return None + return None, None - return vcvarsall + vcruntime = None + vcruntime_spec = _VCVARS_PLAT_TO_VCRUNTIME_REDIST.get(plat_spec) + if vcruntime_spec: + vcruntime = os.path.join(best_dir, + vcruntime_spec.format(best_version)) + if not os.path.isfile(vcruntime): + log.debug("%s cannot be found", vcruntime) + vcruntime = None + + return vcvarsall, vcruntime def _get_vc_env(plat_spec): if os.getenv("DISTUTILS_USE_SDK"): @@ -67,7 +78,7 @@ for key, value in os.environ.items() } - vcvarsall = _find_vcvarsall() + vcvarsall, vcruntime = _find_vcvarsall(plat_spec) if not vcvarsall: raise DistutilsPlatformError("Unable to find vcvarsall.bat") @@ -83,12 +94,16 @@ raise DistutilsPlatformError("Error executing {}" .format(exc.cmd)) - return { + env = { key.lower(): value for key, _, value in (line.partition('=') for line in out.splitlines()) if key and value } + + if vcruntime: + env['py_vcruntime_redist'] = vcruntime + return env def _find_exe(exe, paths=None): """Return path to an MSVC executable program. @@ -115,6 +130,20 @@ 'win-amd64' : 'amd64', } +# A map keyed by get_platform() return values to the file under +# the VC install directory containing the vcruntime redistributable. +_VCVARS_PLAT_TO_VCRUNTIME_REDIST = { + 'x86' : 'redist\\x86\\Microsoft.VC{0}0.CRT\\vcruntime{0}0.dll', + 'amd64' : 'redist\\x64\\Microsoft.VC{0}0.CRT\\vcruntime{0}0.dll', + 'x86_amd64' : 'redist\\x64\\Microsoft.VC{0}0.CRT\\vcruntime{0}0.dll', +} + +# A set containing the DLLs that are guaranteed to be available for +# all micro versions of this Python version. Known extension +# dependencies that are not in this set will be copied to the output +# path. +_BUNDLED_DLLS = frozenset(['vcruntime140.dll']) + class MSVCCompiler(CCompiler) : """Concrete class that implements an interface to Microsoft Visual C++, as defined by the CCompiler abstract class.""" @@ -189,6 +218,7 @@ self.rc = _find_exe("rc.exe", paths) # resource compiler self.mc = _find_exe("mc.exe", paths) # message compiler self.mt = _find_exe("mt.exe", paths) # message compiler + self._vcruntime_redist = vc_env.get('py_vcruntime_redist', '') for dir in vc_env.get('include', '').split(os.pathsep): if dir: @@ -199,20 +229,26 @@ self.add_library_dir(dir) self.preprocess_options = None - # Use /MT[d] to build statically, then switch from libucrt[d].lib to ucrt[d].lib + # If vcruntime_redist is available, link against it dynamically. Otherwise, + # use /MT[d] to build statically, then switch from libucrt[d].lib to ucrt[d].lib # later to dynamically link to ucrtbase but not vcruntime. self.compile_options = [ - '/nologo', '/Ox', '/MT', '/W3', '/GL', '/DNDEBUG' + '/nologo', '/Ox', '/W3', '/GL', '/DNDEBUG' ] + self.compile_options.append('/MD' if self._vcruntime_redist else '/MT') + self.compile_options_debug = [ - '/nologo', '/Od', '/MTd', '/Zi', '/W3', '/D_DEBUG' + '/nologo', '/Od', '/MDd', '/Zi', '/W3', '/D_DEBUG' ] ldflags = [ - '/nologo', '/INCREMENTAL:NO', '/LTCG', '/nodefaultlib:libucrt.lib', 'ucrt.lib', + '/nologo', '/INCREMENTAL:NO', '/LTCG' ] + if not self._vcruntime_redist: + ldflags.extend(('/nodefaultlib:libucrt.lib', 'ucrt.lib')) + ldflags_debug = [ - '/nologo', '/INCREMENTAL:NO', '/LTCG', '/DEBUG:FULL', '/nodefaultlib:libucrtd.lib', 'ucrtd.lib', + '/nologo', '/INCREMENTAL:NO', '/LTCG', '/DEBUG:FULL' ] self.ldflags_exe = [*ldflags, '/MANIFEST:EMBED,ID=1'] @@ -446,15 +482,29 @@ if extra_postargs: ld_args.extend(extra_postargs) - self.mkpath(os.path.dirname(output_filename)) + output_dir = os.path.dirname(os.path.abspath(output_filename)) + self.mkpath(output_dir) try: log.debug('Executing "%s" %s', self.linker, ' '.join(ld_args)) self.spawn([self.linker] + ld_args) + self._copy_vcruntime(output_dir) except DistutilsExecError as msg: raise LinkError(msg) else: log.debug("skipping %s (up-to-date)", output_filename) + def _copy_vcruntime(self, output_dir): + vcruntime = self._vcruntime_redist + if not vcruntime or not os.path.isfile(vcruntime): + return + + if os.path.basename(vcruntime).lower() in _BUNDLED_DLLS: + return + + log.debug('Copying "%s"', vcruntime) + vcruntime = shutil.copy(vcruntime, output_dir) + os.chmod(vcruntime, stat.S_IWRITE) + def spawn(self, cmd): old_path = os.getenv('path') try: diff --git a/Lib/distutils/tests/test_msvccompiler.py b/Lib/distutils/tests/test_msvccompiler.py --- a/Lib/distutils/tests/test_msvccompiler.py +++ b/Lib/distutils/tests/test_msvccompiler.py @@ -16,22 +16,73 @@ unittest.TestCase): def test_no_compiler(self): + import distutils._msvccompiler as _msvccompiler # makes sure query_vcvarsall raises # a DistutilsPlatformError if the compiler # is not found - from distutils._msvccompiler import _get_vc_env - def _find_vcvarsall(): - return None + def _find_vcvarsall(plat_spec): + return None, None - import distutils._msvccompiler as _msvccompiler old_find_vcvarsall = _msvccompiler._find_vcvarsall _msvccompiler._find_vcvarsall = _find_vcvarsall try: - self.assertRaises(DistutilsPlatformError, _get_vc_env, + self.assertRaises(DistutilsPlatformError, + _msvccompiler._get_vc_env, 'wont find this version') finally: _msvccompiler._find_vcvarsall = old_find_vcvarsall + def test_compiler_options(self): + import distutils._msvccompiler as _msvccompiler + # suppress path to vcruntime from _find_vcvarsall to + # check that /MT is added to compile options + old_find_vcvarsall = _msvccompiler._find_vcvarsall + def _find_vcvarsall(plat_spec): + return old_find_vcvarsall(plat_spec)[0], None + _msvccompiler._find_vcvarsall = _find_vcvarsall + try: + compiler = _msvccompiler.MSVCCompiler() + compiler.initialize() + + self.assertIn('/MT', compiler.compile_options) + self.assertNotIn('/MD', compiler.compile_options) + finally: + _msvccompiler._find_vcvarsall = old_find_vcvarsall + + def test_vcruntime_copy(self): + import distutils._msvccompiler as _msvccompiler + # force path to a known file - it doesn't matter + # what we copy as long as its name is not in + # _msvccompiler._BUNDLED_DLLS + old_find_vcvarsall = _msvccompiler._find_vcvarsall + def _find_vcvarsall(plat_spec): + return old_find_vcvarsall(plat_spec)[0], __file__ + _msvccompiler._find_vcvarsall = _find_vcvarsall + try: + tempdir = self.mkdtemp() + compiler = _msvccompiler.MSVCCompiler() + compiler.initialize() + compiler._copy_vcruntime(tempdir) + + self.assertTrue(os.path.isfile(os.path.join( + tempdir, os.path.basename(__file__)))) + finally: + _msvccompiler._find_vcvarsall = old_find_vcvarsall + + def test_vcruntime_skip_copy(self): + import distutils._msvccompiler as _msvccompiler + + tempdir = self.mkdtemp() + compiler = _msvccompiler.MSVCCompiler() + compiler.initialize() + dll = compiler._vcruntime_redist + self.assertTrue(os.path.isfile(dll)) + + compiler._copy_vcruntime(tempdir) + + self.assertFalse(os.path.isfile(os.path.join( + tempdir, os.path.basename(dll)))) + def test_suite(): return unittest.makeSuite(msvccompilerTestCase) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -14,6 +14,9 @@ Library ------- +- Issue #23144: Make sure that HTMLParser.feed() returns all the data, even + when convert_charrefs is True. + - Issue #24982: shutil.make_archive() with the "zip" format now adds entries for directories (including empty directories) in ZIP file. @@ -86,6 +89,23 @@ when external libraries are not available. +What's New in Python 3.5.0 release candidate 4? +=============================================== + +Release date: 2015-09-09 + +Library +------- + +- Issue #25029: Fixes MemoryError in test_strptime. + +Build +----- + +- Issue #25027: Reverts partial-static build options and adds + vcruntime140.dll to Windows installation. + + What's New in Python 3.5.0 release candidate 3? =============================================== @@ -105,8 +125,6 @@ ------- - Issue #24917: time_strftime() buffer over-read. -- Issue #23144: Make sure that HTMLParser.feed() returns all the data, even - when convert_charrefs is True. - Issue #24748: To resolve a compatibility problem found with py2exe and pywin32, imp.load_dynamic() once again ignores previously loaded modules @@ -116,7 +134,6 @@ - Issue #24635: Fixed a bug in typing.py where isinstance([], typing.Iterable) would return True once, then False on subsequent calls. - - Issue #24989: Fixed buffer overread in BytesIO.readline() if a position is set beyond size. Based on patch by John Leitch. diff --git a/Modules/timemodule.c b/Modules/timemodule.c --- a/Modules/timemodule.c +++ b/Modules/timemodule.c @@ -648,9 +648,6 @@ * will be ahead of time... */ for (i = 1024; ; i += i) { -#if defined _MSC_VER && _MSC_VER >= 1400 && defined(__STDC_SECURE_LIB__) - int err; -#endif outbuf = (time_char *)PyMem_Malloc(i*sizeof(time_char)); if (outbuf == NULL) { PyErr_NoMemory(); @@ -660,10 +657,14 @@ buflen = format_time(outbuf, i, fmt, &buf); _Py_END_SUPPRESS_IPH #if defined _MSC_VER && _MSC_VER >= 1400 && defined(__STDC_SECURE_LIB__) - err = errno; + /* VisualStudio .NET 2005 does this properly */ + if (buflen == 0 && errno == EINVAL) { + PyErr_SetString(PyExc_ValueError, "Invalid format string"); + PyMem_Free(outbuf); + break; + } #endif - if (buflen > 0 || fmtlen == 0 || - (fmtlen > 4 && i >= 256 * fmtlen)) { + if (buflen > 0 || i >= 256 * fmtlen) { /* If the buffer is 256 times as long as the format, it's probably not failing for lack of room! More likely, the format yields an empty result, @@ -679,13 +680,6 @@ break; } PyMem_Free(outbuf); -#if defined _MSC_VER && _MSC_VER >= 1400 && defined(__STDC_SECURE_LIB__) - /* VisualStudio .NET 2005 does this properly */ - if (buflen == 0 && err == EINVAL) { - PyErr_SetString(PyExc_ValueError, "Invalid format string"); - break; - } -#endif } #ifdef HAVE_WCSFTIME PyMem_Free(format); diff --git a/PCbuild/pyproject.props b/PCbuild/pyproject.props --- a/PCbuild/pyproject.props +++ b/PCbuild/pyproject.props @@ -36,7 +36,7 @@ true true - MultiThreaded + MultiThreadedDLL true Level3 ProgramDatabase @@ -47,7 +47,7 @@ Disabled false - MultiThreadedDebug + MultiThreadedDebugDLL $(OutDir);%(AdditionalLibraryDirectories) @@ -57,9 +57,7 @@ true true true - ucrtd.lib;%(AdditionalDependencies) - ucrt.lib;%(AdditionalDependencies) - LIBC;libucrt.lib;libucrtd.lib;%(IgnoreSpecificDefaultLibraries) + LIBC;%(IgnoreSpecificDefaultLibraries) MachineX86 MachineX64 $(OutDir)$(TargetName).pgd diff --git a/PCbuild/tcl.vcxproj b/PCbuild/tcl.vcxproj --- a/PCbuild/tcl.vcxproj +++ b/PCbuild/tcl.vcxproj @@ -61,8 +61,8 @@ - ucrt - symbols,ucrt + msvcrt + symbols,msvcrt INSTALLDIR="$(OutDir.TrimEnd(`\`))" INSTALL_DIR="$(OutDir.TrimEnd(`\`))" DEBUGFLAGS="-wd4456 -wd4457 -wd4458 -wd4459 -wd4996" setlocal diff --git a/PCbuild/tix.vcxproj b/PCbuild/tix.vcxproj --- a/PCbuild/tix.vcxproj +++ b/PCbuild/tix.vcxproj @@ -57,8 +57,8 @@ BUILDDIRTOP="$(BuildDirTop)" TCL_DIR="$(tclDir.TrimEnd(`\`))" TK_DIR="$(tkDir.TrimEnd(`\`))" INSTALL_DIR="$(OutDir.TrimEnd(`\`))" - DEBUG=1 NODEBUG=0 UCRT=1 TCL_DBGX=g TK_DBGX=g - DEBUG=0 NODEBUG=1 UCRT=1 + DEBUG=1 NODEBUG=0 TCL_DBGX=g TK_DBGX=g + DEBUG=0 NODEBUG=1 setlocal @(ExpectedOutputs->'if not exist "%(FullPath)" goto build',' ') diff --git a/PCbuild/tk.vcxproj b/PCbuild/tk.vcxproj --- a/PCbuild/tk.vcxproj +++ b/PCbuild/tk.vcxproj @@ -60,8 +60,8 @@ - ucrt - symbols,ucrt + msvcrt + symbols,msvcrt TCLDIR="$(tclDir.TrimEnd(`\`))" INSTALLDIR="$(OutDir.TrimEnd(`\`))" DEBUGFLAGS="-wd4456 -wd4457 -wd4458 -wd4459 -wd4996" setlocal diff --git a/README b/README --- a/README +++ b/README @@ -1,4 +1,4 @@ -This is Python version 3.5.0 release candidate 3 +This is Python version 3.5.0 release candidate 4 ================================================ Copyright (c) 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010, 2011, diff --git a/Tools/msi/build.bat b/Tools/msi/build.bat --- a/Tools/msi/build.bat +++ b/Tools/msi/build.bat @@ -22,15 +22,15 @@ call "%PCBUILD%env.bat" x86 if defined BUILDX86 ( - call "%PCBUILD%build.bat" -d + call "%PCBUILD%build.bat" -d -e if errorlevel 1 goto :eof - call "%PCBUILD%build.bat" + call "%PCBUILD%build.bat" -e if errorlevel 1 goto :eof ) if defined BUILDX64 ( - call "%PCBUILD%build.bat" -p x64 -d + call "%PCBUILD%build.bat" -p x64 -d -e if errorlevel 1 goto :eof - call "%PCBUILD%build.bat" -p x64 + call "%PCBUILD%build.bat" -p x64 -e if errorlevel 1 goto :eof ) diff --git a/Tools/msi/buildrelease.bat b/Tools/msi/buildrelease.bat --- a/Tools/msi/buildrelease.bat +++ b/Tools/msi/buildrelease.bat @@ -121,7 +121,7 @@ if not "%SKIPBUILD%" EQU "1" ( call "%PCBUILD%build.bat" -e -p %BUILD_PLAT% -d -t %TARGET% %CERTOPTS% if errorlevel 1 exit /B - call "%PCBUILD%build.bat" -p %BUILD_PLAT% -t %TARGET% %CERTOPTS% + call "%PCBUILD%build.bat" -e -p %BUILD_PLAT% -t %TARGET% %CERTOPTS% if errorlevel 1 exit /B @rem build.bat turns echo back on, so we disable it again @echo off diff --git a/Tools/msi/exe/exe_files.wxs b/Tools/msi/exe/exe_files.wxs --- a/Tools/msi/exe/exe_files.wxs +++ b/Tools/msi/exe/exe_files.wxs @@ -29,6 +29,9 @@ + + + diff --git a/Tools/msi/make_zip.proj b/Tools/msi/make_zip.proj --- a/Tools/msi/make_zip.proj +++ b/Tools/msi/make_zip.proj @@ -16,7 +16,8 @@ $(OutputPath)\en-us\$(TargetName)$(TargetExt) "$(PythonExe)" "$(MSBuildThisFileDirectory)\make_zip.py" $(Arguments) -e -o "$(TargetPath)" -t "$(IntermediateOutputPath)\zip_$(ArchName)" -a $(ArchName) - set DOC_FILENAME=python$(PythonVersion).chm + set DOC_FILENAME=python$(PythonVersion).chm +set VCREDIST_PATH=$(VS140COMNTOOLS)\..\..\VC\redist\$(Platform)\Microsoft.VC140.CRT diff --git a/Tools/msi/make_zip.py b/Tools/msi/make_zip.py --- a/Tools/msi/make_zip.py +++ b/Tools/msi/make_zip.py @@ -64,9 +64,6 @@ ('Tools/', 'Tools', '**/*', include_in_tools), ] -if os.getenv('DOC_FILENAME'): - FULL_LAYOUT.append(('Doc/', 'Doc/build/htmlhelp', os.getenv('DOC_FILENAME'), None)) - EMBED_LAYOUT = [ ('/', 'PCBuild/$arch', 'python*.exe', is_not_debug), ('/', 'PCBuild/$arch', '*.pyd', is_not_debug), @@ -74,6 +71,12 @@ ('python35.zip', 'Lib', '**/*', include_in_lib), ] +if os.getenv('DOC_FILENAME'): + FULL_LAYOUT.append(('Doc/', 'Doc/build/htmlhelp', os.getenv('DOC_FILENAME'), None)) +if os.getenv('VCREDIST_PATH'): + FULL_LAYOUT.append(('/', os.getenv('VCREDIST_PATH'), 'vcruntime*.dll', None)) + EMBED_LAYOUT.append(('/', os.getenv('VCREDIST_PATH'), 'vcruntime*.dll', None)) + def copy_to_layout(target, rel_sources): count = 0 diff --git a/Tools/msi/msi.props b/Tools/msi/msi.props --- a/Tools/msi/msi.props +++ b/Tools/msi/msi.props @@ -118,6 +118,9 @@ redist + + redist + -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 9 15:56:52 2015 From: python-checkins at python.org (larry.hastings) Date: Wed, 09 Sep 2015 13:56:52 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_Version_bump_f?= =?utf-8?q?or_Python_3=2E5=2E0rc4=2E?= Message-ID: <20150909135636.27685.22041@psf.io> https://hg.python.org/cpython/rev/2d033fedfa7f changeset: 97820:2d033fedfa7f branch: 3.5 tag: v3.5.0rc4 user: Larry Hastings date: Tue Sep 08 23:58:10 2015 -0700 summary: Version bump for Python 3.5.0rc4. files: Include/patchlevel.h | 4 ++-- Misc/NEWS | 10 ++++++++-- README | 2 +- 3 files changed, 11 insertions(+), 5 deletions(-) diff --git a/Include/patchlevel.h b/Include/patchlevel.h --- a/Include/patchlevel.h +++ b/Include/patchlevel.h @@ -20,10 +20,10 @@ #define PY_MINOR_VERSION 5 #define PY_MICRO_VERSION 0 #define PY_RELEASE_LEVEL PY_RELEASE_LEVEL_GAMMA -#define PY_RELEASE_SERIAL 3 +#define PY_RELEASE_SERIAL 4 /* Version as a string */ -#define PY_VERSION "3.5.0rc3" +#define PY_VERSION "3.5.0rc4" /*--end constants--*/ /* Version as a single 4-byte hex number, e.g. 0x010502B2 == 1.5.2b2. diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -3,14 +3,20 @@ +++++++++++ What's New in Python 3.5.0 final? +================================= + +Release date: 2015-09-13 + + +What's New in Python 3.5.0 release candidate 4? =============================================== -Release date: 2015-09-13 +Release date: 2015-09-09 Library ------- -- Issue #25029: Fixes MemoryError in test_strptime +- Issue #25029: Fixes MemoryError in test_strptime. Build ----- diff --git a/README b/README --- a/README +++ b/README @@ -1,4 +1,4 @@ -This is Python version 3.5.0 release candidate 3 +This is Python version 3.5.0 release candidate 4 ================================================ Copyright (c) 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010, 2011, -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 9 15:56:52 2015 From: python-checkins at python.org (larry.hastings) Date: Wed, 09 Sep 2015 13:56:52 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_Moves_distutil?= =?utf-8?q?s_test_import_within_skippable_class=2E?= Message-ID: <20150909135635.101494.33161@psf.io> https://hg.python.org/cpython/rev/1fb824c2e1d7 changeset: 97818:1fb824c2e1d7 branch: 3.5 parent: 97816:8374472c6a6e user: Steve Dower date: Tue Sep 08 23:42:51 2015 -0700 summary: Moves distutils test import within skippable class. files: Lib/distutils/tests/test_msvccompiler.py | 7 +++++-- 1 files changed, 5 insertions(+), 2 deletions(-) diff --git a/Lib/distutils/tests/test_msvccompiler.py b/Lib/distutils/tests/test_msvccompiler.py --- a/Lib/distutils/tests/test_msvccompiler.py +++ b/Lib/distutils/tests/test_msvccompiler.py @@ -3,8 +3,6 @@ import unittest import os -import distutils._msvccompiler as _msvccompiler - from distutils.errors import DistutilsPlatformError from distutils.tests import support from test.support import run_unittest @@ -18,6 +16,7 @@ unittest.TestCase): def test_no_compiler(self): + import distutils._msvccompiler as _msvccompiler # makes sure query_vcvarsall raises # a DistutilsPlatformError if the compiler # is not found @@ -34,6 +33,7 @@ _msvccompiler._find_vcvarsall = old_find_vcvarsall def test_compiler_options(self): + import distutils._msvccompiler as _msvccompiler # suppress path to vcruntime from _find_vcvarsall to # check that /MT is added to compile options old_find_vcvarsall = _msvccompiler._find_vcvarsall @@ -50,6 +50,7 @@ _msvccompiler._find_vcvarsall = old_find_vcvarsall def test_vcruntime_copy(self): + import distutils._msvccompiler as _msvccompiler # force path to a known file - it doesn't matter # what we copy as long as its name is not in # _msvccompiler._BUNDLED_DLLS @@ -69,6 +70,8 @@ _msvccompiler._find_vcvarsall = old_find_vcvarsall def test_vcruntime_skip_copy(self): + import distutils._msvccompiler as _msvccompiler + tempdir = self.mkdtemp() compiler = _msvccompiler.MSVCCompiler() compiler.initialize() -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 9 16:01:54 2015 From: python-checkins at python.org (larry.hastings) Date: Wed, 09 Sep 2015 14:01:54 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogTWVyZ2UgZnJvbSAzLjUu?= Message-ID: <20150909140131.101494.17232@psf.io> https://hg.python.org/cpython/rev/35f3442661a8 changeset: 97825:35f3442661a8 parent: 97811:90e79bf21eac parent: 97824:7eef1e886138 user: Larry Hastings date: Wed Sep 09 07:00:54 2015 -0700 summary: Merge from 3.5. files: .hgtags | 1 + Lib/distutils/_msvccompiler.py | 76 +++++++++-- Lib/distutils/tests/test_msvccompiler.py | 61 ++++++++- Misc/NEWS | 26 +++- Modules/timemodule.c | 20 +-- PCbuild/pyproject.props | 8 +- PCbuild/tcl.vcxproj | 4 +- PCbuild/tix.vcxproj | 4 +- PCbuild/tk.vcxproj | 4 +- Tools/msi/build.bat | 8 +- Tools/msi/buildrelease.bat | 2 +- Tools/msi/exe/exe_files.wxs | 3 + Tools/msi/make_zip.proj | 3 +- Tools/msi/make_zip.py | 9 +- Tools/msi/msi.props | 3 + 15 files changed, 178 insertions(+), 54 deletions(-) diff --git a/.hgtags b/.hgtags --- a/.hgtags +++ b/.hgtags @@ -155,3 +155,4 @@ 1a58b1227501e046eee13d90f113417b60843301 v3.5.0rc1 cc15d736d860303b9da90d43cd32db39bab048df v3.5.0rc2 66ed52375df802f9d0a34480daaa8ce79fc41313 v3.5.0rc3 +2d033fedfa7f1e325fd14ccdaa9cb42155da206f v3.5.0rc4 diff --git a/Lib/distutils/_msvccompiler.py b/Lib/distutils/_msvccompiler.py --- a/Lib/distutils/_msvccompiler.py +++ b/Lib/distutils/_msvccompiler.py @@ -14,6 +14,8 @@ # ported to VS 2015 by Steve Dower import os +import shutil +import stat import subprocess from distutils.errors import DistutilsExecError, DistutilsPlatformError, \ @@ -25,7 +27,7 @@ import winreg from itertools import count -def _find_vcvarsall(): +def _find_vcvarsall(plat_spec): with winreg.OpenKeyEx( winreg.HKEY_LOCAL_MACHINE, r"Software\Microsoft\VisualStudio\SxS\VC7", @@ -33,7 +35,7 @@ ) as key: if not key: log.debug("Visual C++ is not registered") - return None + return None, None best_version = 0 best_dir = None @@ -51,14 +53,23 @@ best_version, best_dir = version, vc_dir if not best_version: log.debug("No suitable Visual C++ version found") - return None + return None, None vcvarsall = os.path.join(best_dir, "vcvarsall.bat") if not os.path.isfile(vcvarsall): log.debug("%s cannot be found", vcvarsall) - return None + return None, None - return vcvarsall + vcruntime = None + vcruntime_spec = _VCVARS_PLAT_TO_VCRUNTIME_REDIST.get(plat_spec) + if vcruntime_spec: + vcruntime = os.path.join(best_dir, + vcruntime_spec.format(best_version)) + if not os.path.isfile(vcruntime): + log.debug("%s cannot be found", vcruntime) + vcruntime = None + + return vcvarsall, vcruntime def _get_vc_env(plat_spec): if os.getenv("DISTUTILS_USE_SDK"): @@ -67,7 +78,7 @@ for key, value in os.environ.items() } - vcvarsall = _find_vcvarsall() + vcvarsall, vcruntime = _find_vcvarsall(plat_spec) if not vcvarsall: raise DistutilsPlatformError("Unable to find vcvarsall.bat") @@ -83,13 +94,17 @@ raise DistutilsPlatformError("Error executing {}" .format(exc.cmd)) - return { + env = { key.lower(): value for key, _, value in (line.partition('=') for line in out.splitlines()) if key and value } + if vcruntime: + env['py_vcruntime_redist'] = vcruntime + return env + def _find_exe(exe, paths=None): """Return path to an MSVC executable program. @@ -115,6 +130,20 @@ 'win-amd64' : 'amd64', } +# A map keyed by get_platform() return values to the file under +# the VC install directory containing the vcruntime redistributable. +_VCVARS_PLAT_TO_VCRUNTIME_REDIST = { + 'x86' : 'redist\\x86\\Microsoft.VC{0}0.CRT\\vcruntime{0}0.dll', + 'amd64' : 'redist\\x64\\Microsoft.VC{0}0.CRT\\vcruntime{0}0.dll', + 'x86_amd64' : 'redist\\x64\\Microsoft.VC{0}0.CRT\\vcruntime{0}0.dll', +} + +# A set containing the DLLs that are guaranteed to be available for +# all micro versions of this Python version. Known extension +# dependencies that are not in this set will be copied to the output +# path. +_BUNDLED_DLLS = frozenset(['vcruntime140.dll']) + class MSVCCompiler(CCompiler) : """Concrete class that implements an interface to Microsoft Visual C++, as defined by the CCompiler abstract class.""" @@ -189,6 +218,7 @@ self.rc = _find_exe("rc.exe", paths) # resource compiler self.mc = _find_exe("mc.exe", paths) # message compiler self.mt = _find_exe("mt.exe", paths) # message compiler + self._vcruntime_redist = vc_env.get('py_vcruntime_redist', '') for dir in vc_env.get('include', '').split(os.pathsep): if dir: @@ -199,20 +229,26 @@ self.add_library_dir(dir) self.preprocess_options = None - # Use /MT[d] to build statically, then switch from libucrt[d].lib to ucrt[d].lib + # If vcruntime_redist is available, link against it dynamically. Otherwise, + # use /MT[d] to build statically, then switch from libucrt[d].lib to ucrt[d].lib # later to dynamically link to ucrtbase but not vcruntime. self.compile_options = [ - '/nologo', '/Ox', '/MT', '/W3', '/GL', '/DNDEBUG' + '/nologo', '/Ox', '/W3', '/GL', '/DNDEBUG' ] + self.compile_options.append('/MD' if self._vcruntime_redist else '/MT') + self.compile_options_debug = [ - '/nologo', '/Od', '/MTd', '/Zi', '/W3', '/D_DEBUG' + '/nologo', '/Od', '/MDd', '/Zi', '/W3', '/D_DEBUG' ] ldflags = [ - '/nologo', '/INCREMENTAL:NO', '/LTCG', '/nodefaultlib:libucrt.lib', 'ucrt.lib', + '/nologo', '/INCREMENTAL:NO', '/LTCG' ] + if not self._vcruntime_redist: + ldflags.extend(('/nodefaultlib:libucrt.lib', 'ucrt.lib')) + ldflags_debug = [ - '/nologo', '/INCREMENTAL:NO', '/LTCG', '/DEBUG:FULL', '/nodefaultlib:libucrtd.lib', 'ucrtd.lib', + '/nologo', '/INCREMENTAL:NO', '/LTCG', '/DEBUG:FULL' ] self.ldflags_exe = [*ldflags, '/MANIFEST:EMBED,ID=1'] @@ -446,15 +482,29 @@ if extra_postargs: ld_args.extend(extra_postargs) - self.mkpath(os.path.dirname(output_filename)) + output_dir = os.path.dirname(os.path.abspath(output_filename)) + self.mkpath(output_dir) try: log.debug('Executing "%s" %s', self.linker, ' '.join(ld_args)) self.spawn([self.linker] + ld_args) + self._copy_vcruntime(output_dir) except DistutilsExecError as msg: raise LinkError(msg) else: log.debug("skipping %s (up-to-date)", output_filename) + def _copy_vcruntime(self, output_dir): + vcruntime = self._vcruntime_redist + if not vcruntime or not os.path.isfile(vcruntime): + return + + if os.path.basename(vcruntime).lower() in _BUNDLED_DLLS: + return + + log.debug('Copying "%s"', vcruntime) + vcruntime = shutil.copy(vcruntime, output_dir) + os.chmod(vcruntime, stat.S_IWRITE) + def spawn(self, cmd): old_path = os.getenv('path') try: diff --git a/Lib/distutils/tests/test_msvccompiler.py b/Lib/distutils/tests/test_msvccompiler.py --- a/Lib/distutils/tests/test_msvccompiler.py +++ b/Lib/distutils/tests/test_msvccompiler.py @@ -16,22 +16,73 @@ unittest.TestCase): def test_no_compiler(self): + import distutils._msvccompiler as _msvccompiler # makes sure query_vcvarsall raises # a DistutilsPlatformError if the compiler # is not found - from distutils._msvccompiler import _get_vc_env - def _find_vcvarsall(): - return None + def _find_vcvarsall(plat_spec): + return None, None - import distutils._msvccompiler as _msvccompiler old_find_vcvarsall = _msvccompiler._find_vcvarsall _msvccompiler._find_vcvarsall = _find_vcvarsall try: - self.assertRaises(DistutilsPlatformError, _get_vc_env, + self.assertRaises(DistutilsPlatformError, + _msvccompiler._get_vc_env, 'wont find this version') finally: _msvccompiler._find_vcvarsall = old_find_vcvarsall + def test_compiler_options(self): + import distutils._msvccompiler as _msvccompiler + # suppress path to vcruntime from _find_vcvarsall to + # check that /MT is added to compile options + old_find_vcvarsall = _msvccompiler._find_vcvarsall + def _find_vcvarsall(plat_spec): + return old_find_vcvarsall(plat_spec)[0], None + _msvccompiler._find_vcvarsall = _find_vcvarsall + try: + compiler = _msvccompiler.MSVCCompiler() + compiler.initialize() + + self.assertIn('/MT', compiler.compile_options) + self.assertNotIn('/MD', compiler.compile_options) + finally: + _msvccompiler._find_vcvarsall = old_find_vcvarsall + + def test_vcruntime_copy(self): + import distutils._msvccompiler as _msvccompiler + # force path to a known file - it doesn't matter + # what we copy as long as its name is not in + # _msvccompiler._BUNDLED_DLLS + old_find_vcvarsall = _msvccompiler._find_vcvarsall + def _find_vcvarsall(plat_spec): + return old_find_vcvarsall(plat_spec)[0], __file__ + _msvccompiler._find_vcvarsall = _find_vcvarsall + try: + tempdir = self.mkdtemp() + compiler = _msvccompiler.MSVCCompiler() + compiler.initialize() + compiler._copy_vcruntime(tempdir) + + self.assertTrue(os.path.isfile(os.path.join( + tempdir, os.path.basename(__file__)))) + finally: + _msvccompiler._find_vcvarsall = old_find_vcvarsall + + def test_vcruntime_skip_copy(self): + import distutils._msvccompiler as _msvccompiler + + tempdir = self.mkdtemp() + compiler = _msvccompiler.MSVCCompiler() + compiler.initialize() + dll = compiler._vcruntime_redist + self.assertTrue(os.path.isfile(dll)) + + compiler._copy_vcruntime(tempdir) + + self.assertFalse(os.path.isfile(os.path.join( + tempdir, os.path.basename(dll)))) + def test_suite(): return unittest.makeSuite(msvccompilerTestCase) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -98,6 +98,9 @@ Library ------- +- Issue #23144: Make sure that HTMLParser.feed() returns all the data, even + when convert_charrefs is True. + - Issue #24982: shutil.make_archive() with the "zip" format now adds entries for directories (including empty directories) in ZIP file. @@ -143,6 +146,9 @@ Documentation ------------- +- Issue #24952: Clarify the default size argument of stack_size() in + the "threading" and "_thread" modules. Patch from Mattip. + - Issue #23725: Overhaul tempfile docs. Note deprecated status of mktemp. Patch from Zbigniew J?drzejewski-Szmek. @@ -167,6 +173,23 @@ when external libraries are not available. +What's New in Python 3.5.0 release candidate 4? +=============================================== + +Release date: 2015-09-09 + +Library +------- + +- Issue #25029: Fixes MemoryError in test_strptime. + +Build +----- + +- Issue #25027: Reverts partial-static build options and adds + vcruntime140.dll to Windows installation. + + What's New in Python 3.5.0 release candidate 3? =============================================== @@ -186,8 +209,6 @@ ------- - Issue #24917: time_strftime() buffer over-read. -- Issue #23144: Make sure that HTMLParser.feed() returns all the data, even - when convert_charrefs is True. - Issue #24748: To resolve a compatibility problem found with py2exe and pywin32, imp.load_dynamic() once again ignores previously loaded modules @@ -197,7 +218,6 @@ - Issue #24635: Fixed a bug in typing.py where isinstance([], typing.Iterable) would return True once, then False on subsequent calls. - - Issue #24989: Fixed buffer overread in BytesIO.readline() if a position is set beyond size. Based on patch by John Leitch. diff --git a/Modules/timemodule.c b/Modules/timemodule.c --- a/Modules/timemodule.c +++ b/Modules/timemodule.c @@ -648,9 +648,6 @@ * will be ahead of time... */ for (i = 1024; ; i += i) { -#if defined _MSC_VER && _MSC_VER >= 1400 && defined(__STDC_SECURE_LIB__) - int err; -#endif outbuf = (time_char *)PyMem_Malloc(i*sizeof(time_char)); if (outbuf == NULL) { PyErr_NoMemory(); @@ -660,10 +657,14 @@ buflen = format_time(outbuf, i, fmt, &buf); _Py_END_SUPPRESS_IPH #if defined _MSC_VER && _MSC_VER >= 1400 && defined(__STDC_SECURE_LIB__) - err = errno; + /* VisualStudio .NET 2005 does this properly */ + if (buflen == 0 && errno == EINVAL) { + PyErr_SetString(PyExc_ValueError, "Invalid format string"); + PyMem_Free(outbuf); + break; + } #endif - if (buflen > 0 || fmtlen == 0 || - (fmtlen > 4 && i >= 256 * fmtlen)) { + if (buflen > 0 || i >= 256 * fmtlen) { /* If the buffer is 256 times as long as the format, it's probably not failing for lack of room! More likely, the format yields an empty result, @@ -679,13 +680,6 @@ break; } PyMem_Free(outbuf); -#if defined _MSC_VER && _MSC_VER >= 1400 && defined(__STDC_SECURE_LIB__) - /* VisualStudio .NET 2005 does this properly */ - if (buflen == 0 && err == EINVAL) { - PyErr_SetString(PyExc_ValueError, "Invalid format string"); - break; - } -#endif } #ifdef HAVE_WCSFTIME PyMem_Free(format); diff --git a/PCbuild/pyproject.props b/PCbuild/pyproject.props --- a/PCbuild/pyproject.props +++ b/PCbuild/pyproject.props @@ -36,7 +36,7 @@ true true - MultiThreaded + MultiThreadedDLL true Level3 ProgramDatabase @@ -47,7 +47,7 @@ Disabled false - MultiThreadedDebug + MultiThreadedDebugDLL $(OutDir);%(AdditionalLibraryDirectories) @@ -57,9 +57,7 @@ true true true - ucrtd.lib;%(AdditionalDependencies) - ucrt.lib;%(AdditionalDependencies) - LIBC;libucrt.lib;libucrtd.lib;%(IgnoreSpecificDefaultLibraries) + LIBC;%(IgnoreSpecificDefaultLibraries) MachineX86 MachineX64 $(OutDir)$(TargetName).pgd diff --git a/PCbuild/tcl.vcxproj b/PCbuild/tcl.vcxproj --- a/PCbuild/tcl.vcxproj +++ b/PCbuild/tcl.vcxproj @@ -61,8 +61,8 @@ - ucrt - symbols,ucrt + msvcrt + symbols,msvcrt INSTALLDIR="$(OutDir.TrimEnd(`\`))" INSTALL_DIR="$(OutDir.TrimEnd(`\`))" DEBUGFLAGS="-wd4456 -wd4457 -wd4458 -wd4459 -wd4996" setlocal diff --git a/PCbuild/tix.vcxproj b/PCbuild/tix.vcxproj --- a/PCbuild/tix.vcxproj +++ b/PCbuild/tix.vcxproj @@ -57,8 +57,8 @@ BUILDDIRTOP="$(BuildDirTop)" TCL_DIR="$(tclDir.TrimEnd(`\`))" TK_DIR="$(tkDir.TrimEnd(`\`))" INSTALL_DIR="$(OutDir.TrimEnd(`\`))" - DEBUG=1 NODEBUG=0 UCRT=1 TCL_DBGX=g TK_DBGX=g - DEBUG=0 NODEBUG=1 UCRT=1 + DEBUG=1 NODEBUG=0 TCL_DBGX=g TK_DBGX=g + DEBUG=0 NODEBUG=1 setlocal @(ExpectedOutputs->'if not exist "%(FullPath)" goto build',' ') diff --git a/PCbuild/tk.vcxproj b/PCbuild/tk.vcxproj --- a/PCbuild/tk.vcxproj +++ b/PCbuild/tk.vcxproj @@ -60,8 +60,8 @@ - ucrt - symbols,ucrt + msvcrt + symbols,msvcrt TCLDIR="$(tclDir.TrimEnd(`\`))" INSTALLDIR="$(OutDir.TrimEnd(`\`))" DEBUGFLAGS="-wd4456 -wd4457 -wd4458 -wd4459 -wd4996" setlocal diff --git a/Tools/msi/build.bat b/Tools/msi/build.bat --- a/Tools/msi/build.bat +++ b/Tools/msi/build.bat @@ -22,15 +22,15 @@ call "%PCBUILD%env.bat" x86 if defined BUILDX86 ( - call "%PCBUILD%build.bat" -d + call "%PCBUILD%build.bat" -d -e if errorlevel 1 goto :eof - call "%PCBUILD%build.bat" + call "%PCBUILD%build.bat" -e if errorlevel 1 goto :eof ) if defined BUILDX64 ( - call "%PCBUILD%build.bat" -p x64 -d + call "%PCBUILD%build.bat" -p x64 -d -e if errorlevel 1 goto :eof - call "%PCBUILD%build.bat" -p x64 + call "%PCBUILD%build.bat" -p x64 -e if errorlevel 1 goto :eof ) diff --git a/Tools/msi/buildrelease.bat b/Tools/msi/buildrelease.bat --- a/Tools/msi/buildrelease.bat +++ b/Tools/msi/buildrelease.bat @@ -121,7 +121,7 @@ if not "%SKIPBUILD%" EQU "1" ( call "%PCBUILD%build.bat" -e -p %BUILD_PLAT% -d -t %TARGET% %CERTOPTS% if errorlevel 1 exit /B - call "%PCBUILD%build.bat" -p %BUILD_PLAT% -t %TARGET% %CERTOPTS% + call "%PCBUILD%build.bat" -e -p %BUILD_PLAT% -t %TARGET% %CERTOPTS% if errorlevel 1 exit /B @rem build.bat turns echo back on, so we disable it again @echo off diff --git a/Tools/msi/exe/exe_files.wxs b/Tools/msi/exe/exe_files.wxs --- a/Tools/msi/exe/exe_files.wxs +++ b/Tools/msi/exe/exe_files.wxs @@ -29,6 +29,9 @@ + + + diff --git a/Tools/msi/make_zip.proj b/Tools/msi/make_zip.proj --- a/Tools/msi/make_zip.proj +++ b/Tools/msi/make_zip.proj @@ -16,7 +16,8 @@ $(OutputPath)\en-us\$(TargetName)$(TargetExt) "$(PythonExe)" "$(MSBuildThisFileDirectory)\make_zip.py" $(Arguments) -e -o "$(TargetPath)" -t "$(IntermediateOutputPath)\zip_$(ArchName)" -a $(ArchName) - set DOC_FILENAME=python$(PythonVersion).chm + set DOC_FILENAME=python$(PythonVersion).chm +set VCREDIST_PATH=$(VS140COMNTOOLS)\..\..\VC\redist\$(Platform)\Microsoft.VC140.CRT diff --git a/Tools/msi/make_zip.py b/Tools/msi/make_zip.py --- a/Tools/msi/make_zip.py +++ b/Tools/msi/make_zip.py @@ -64,9 +64,6 @@ ('Tools/', 'Tools', '**/*', include_in_tools), ] -if os.getenv('DOC_FILENAME'): - FULL_LAYOUT.append(('Doc/', 'Doc/build/htmlhelp', os.getenv('DOC_FILENAME'), None)) - EMBED_LAYOUT = [ ('/', 'PCBuild/$arch', 'python*.exe', is_not_debug), ('/', 'PCBuild/$arch', '*.pyd', is_not_debug), @@ -74,6 +71,12 @@ ('python35.zip', 'Lib', '**/*', include_in_lib), ] +if os.getenv('DOC_FILENAME'): + FULL_LAYOUT.append(('Doc/', 'Doc/build/htmlhelp', os.getenv('DOC_FILENAME'), None)) +if os.getenv('VCREDIST_PATH'): + FULL_LAYOUT.append(('/', os.getenv('VCREDIST_PATH'), 'vcruntime*.dll', None)) + EMBED_LAYOUT.append(('/', os.getenv('VCREDIST_PATH'), 'vcruntime*.dll', None)) + def copy_to_layout(target, rel_sources): count = 0 diff --git a/Tools/msi/msi.props b/Tools/msi/msi.props --- a/Tools/msi/msi.props +++ b/Tools/msi/msi.props @@ -118,6 +118,9 @@ redist + + redist + -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 9 17:32:59 2015 From: python-checkins at python.org (berker.peksag) Date: Wed, 09 Sep 2015 15:32:59 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Fix_versionchanged_directi?= =?utf-8?q?ve_in_datetime=2Erst?= Message-ID: <20150909153258.11262.20974@psf.io> https://hg.python.org/cpython/rev/816fdd881384 changeset: 97826:816fdd881384 user: Berker Peksag date: Wed Sep 09 18:32:50 2015 +0300 summary: Fix versionchanged directive in datetime.rst files: Doc/library/datetime.rst | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Doc/library/datetime.rst b/Doc/library/datetime.rst --- a/Doc/library/datetime.rst +++ b/Doc/library/datetime.rst @@ -1756,8 +1756,8 @@ ``offset.minutes`` respectively. .. versionchanged:: 3.6 - Name generated from ``offset=timedelta(0)`` is now plain 'UTC', not - 'UTC+00:00'. + Name generated from ``offset=timedelta(0)`` is now plain 'UTC', not + 'UTC+00:00'. .. method:: timezone.dst(dt) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 9 18:25:14 2015 From: python-checkins at python.org (yury.selivanov) Date: Wed, 09 Sep 2015 16:25:14 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy41IC0+IDMuNSk6?= =?utf-8?q?_Merge_3=2E5_heads?= Message-ID: <20150909162514.12019.69933@psf.io> https://hg.python.org/cpython/rev/91573c156b75 changeset: 97829:91573c156b75 branch: 3.5 parent: 97824:7eef1e886138 parent: 97827:97f928f49a54 user: Yury Selivanov date: Wed Sep 09 12:24:16 2015 -0400 summary: Merge 3.5 heads files: Doc/library/collections.rst | 4 + Doc/whatsnew/3.5.rst | 175 ++++++++++++++++++++--- 2 files changed, 152 insertions(+), 27 deletions(-) diff --git a/Doc/library/collections.rst b/Doc/library/collections.rst --- a/Doc/library/collections.rst +++ b/Doc/library/collections.rst @@ -1157,3 +1157,7 @@ be an instance of :class:`bytes`, :class:`str`, :class:`UserString` (or a subclass) or an arbitrary sequence which can be converted into a string using the built-in :func:`str` function. + + .. versionchanged:: 3.5 + New methods ``__getnewargs__``, ``__rmod__``, ``casefold``, + ``format_map``, ``isprintable``, and ``maketrans``. diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -112,11 +112,13 @@ :issue:`16991`. * You may now pass bytes to the :mod:`tempfile` module's APIs and it will - return the temporary pathname as bytes instead of str. It also accepts - a value of ``None`` on parameters where only str was accepted in the past to - do the right thing based on the types of the other inputs. Two functions, - :func:`gettempdirb` and :func:`gettempprefixb`, have been added to go along - with this. This behavior matches that of the :mod:`os` APIs. + return the temporary pathname as :class:`bytes` instead of :class:`str`. + It also accepts a value of ``None`` on parameters where only str was + accepted in the past to do the right thing based on the types of the + other inputs. Two functions, :func:`gettempdirb` and + :func:`gettempprefixb`, have been added to go along with this. + This behavior matches that of the :mod:`os` APIs. + (Contributed by Gregory P. Smith in :issue:`24230`.) * :mod:`ssl` module gained support for Memory BIO, which decouples SSL protocol handling from network IO. (Contributed by Geert Jansen in @@ -127,6 +129,10 @@ :class:`~traceback.StackSummary`, and :class:`traceback.FrameSummary`. (Contributed by Robert Collins in :issue:`17911`.) +* Most of :func:`functools.lru_cache` machinery is now implemented in C. + (Contributed by Matt Joiner, Alexey Kachayev, and Serhiy Storchaka + in :issue:`14373`.) + Security improvements: * SSLv3 is now disabled throughout the standard library. @@ -428,14 +434,14 @@ for the rationale); * :mod:`select` functions: :func:`~select.devpoll.poll`, - :func:`~select.epoll.poll`, :func:`~select.kqueue.control`, - :func:`~select.poll.poll`, :func:`~select.select`; + :func:`~select.epoll.poll`, :func:`~select.kqueue.control`, + :func:`~select.poll.poll`, :func:`~select.select`; * :func:`socket.socket` methods: :meth:`~socket.socket.accept`, :meth:`~socket.socket.connect` (except for non-blocking sockets), - :meth:`~socket.socket.recv`, :meth:`~socket.socket.recvfrom`, - :meth:`~socket.socket.recvmsg`, :meth:`~socket.socket.send`, - :meth:`~socket.socket.sendall`, :meth:`~socket.socket.sendmsg`, + :meth:`~socket.socket.recv`, :meth:`~socket.socket.recvfrom`, + :meth:`~socket.socket.recvmsg`, :meth:`~socket.socket.send`, + :meth:`~socket.socket.sendall`, :meth:`~socket.socket.sendmsg`, :meth:`~socket.socket.sendto`; * :func:`signal.sigtimedwait`, :func:`signal.sigwaitinfo`; @@ -556,6 +562,9 @@ * New Tajik :ref:`codec ` ``koi8_t``. (Contributed by Serhiy Storchaka in :issue:`22681`.) +* Circular imports involving relative imports are now supported. + (Contributed by Brett Cannon and Antoine Pitrou in :issue:`17636`.) + New Modules =========== @@ -589,6 +598,11 @@ :ref:`allow_abbrev` to ``False``. (Contributed by Jonathan Paugh, Steven Bethard, paul j3 and Daniel Eriksson.) +* New *allow_abbrev* parameter for :class:`~argparse.ArgumentParser` to + allow long options to be abbreviated if the abbreviation is unambiguous. + (Contributed by Jonathan Paugh, Steven Bethard, paul j3 and + Daniel Eriksson in :issue:`14910`.) + bz2 --- @@ -628,6 +642,33 @@ (Contributed by Berker Peksag in :issue:`24064`.) +* :class:`~collections.deque` has new methods: + :meth:`~collections.deque.index`, :meth:`~collections.deque.insert`, and + :meth:`~collections.deque.copy`. This allows deques to be registered as a + :class:`~collections.abc.MutableSequence` and improves their + substitutablity for lists. + (Contributed by Raymond Hettinger :issue:`23704`.) + +* :class:`~collections.UserString` now supports ``__getnewargs__``, + ``__rmod__``, ``casefold``, ``format_map``, ``isprintable``, and + ``maketrans`` methods. + (Contributed by Joe Jevnik in :issue:`22189`.) + +* :class:`collections.OrderedDict` is now implemented in C, which improves + its performance between 4x to 100x times. + (Contributed by Eric Snow in :issue:`16991`.) + +collections.abc +--------------- + +* New :class:`~collections.abc.Generator` abstract base class. + (Contributed by Stefan Behnel in :issue:`24018`.) + +* New :class:`~collections.abc.Coroutine`, + :class:`~collections.abc.AsyncIterator`, and + :class:`~collections.abc.AsyncIterable` abstract base classes. + (Contributed by Yury Selivanov in :issue:`24184`.) + compileall ---------- @@ -672,7 +713,8 @@ (Contributed by Berker Peksag in :issue:`2052`.) * It's now possible to compare lists of byte strings with - :func:`difflib.diff_bytes` (fixes a regression from Python 2). + :func:`difflib.diff_bytes`. This fixes a regression from Python 2. + (Contributed by Terry J. Reedy and Greg Ward in :issue:`17445`.) distutils --------- @@ -711,9 +753,26 @@ :rfc:`6532` and used with an SMTP server that supports the :rfc:`6531` ``SMTPUTF8`` extension. (Contributed by R. David Murray in :issue:`24211`.) +faulthandler +------------ + +* :func:`~faulthandler.enable`, :func:`~faulthandler.register`, + :func:`~faulthandler.dump_traceback` and + :func:`~faulthandler.dump_traceback_later` functions now accept file + descriptors. (Contributed by Wei Wu in :issue:`23566`.) + +functools +--------- + +* Most of :func:`~functools.lru_cache` machinery is now implemented in C. + (Contributed by Matt Joiner, Alexey Kachayev, and Serhiy Storchaka + in :issue:`14373`.) + glob ---- +00002-5938667 + * :func:`~glob.iglob` and :func:`~glob.glob` now support recursive search in subdirectories using the "``**``" pattern. (Contributed by Serhiy Storchaka in :issue:`13968`.) @@ -805,6 +864,13 @@ and :func:`~inspect.getinnerframes` now return a list of named tuples. (Contributed by Daniel Shahaf in :issue:`16808`.) +io +-- + +* New Python implementation of :class:`io.FileIO` to make dependency on + ``_io`` module optional. + (Contributed by Serhiy Storchaka in :issue:`21859`.) + ipaddress --------- @@ -845,6 +911,10 @@ for HTTP connection. (Contributed by Alex Gaynor in :issue:`22788`.) +* :class:`~logging.handlers.QueueListener` now takes a *respect_handler_level* + keyword argument which, if set to ``True``, will pass messages to handlers + taking handler levels into account. (Contributed by Vinay Sajip.) + lzma ---- @@ -852,15 +922,26 @@ to limit the maximum size of decompressed data. (Contributed by Martin Panter in :issue:`15955`.) - math ---- * :data:`math.inf` and :data:`math.nan` constants added. (Contributed by Mark Dickinson in :issue:`23185`.) -* :func:`math.isclose` function added. + +* New :func:`~math.isclose` function. (Contributed by Chris Barker and Tal Einat in :issue:`24270`.) +* New :func:`~math.gcd` function. The :func:`fractions.gcd` function now is + deprecated. + (Contributed by Mark Dickinson and Serhiy Storchaka in :issue:`22486`.) + +operator +-------- + +* :func:`~operator.attrgetter`, :func:`~operator.itemgetter`, and + :func:`~operator.methodcaller` objects now support pickling. + (Contributed by Josh Rosenberg and Serhiy Storchaka in :issue:`22955`.) + os -- @@ -877,11 +958,15 @@ now used when available. On OpenBSD 5.6 and newer, the C ``getentropy()`` function is now used. These functions avoid the usage of an internal file descriptor. + (Contributed by Victor Stinner in :issue:`22181`.) * New :func:`os.get_blocking` and :func:`os.set_blocking` functions to get and set the blocking mode of file descriptors. (Contributed by Victor Stinner in :issue:`22054`.) +* :func:`~os.truncate` and :func:`~os.ftruncate` are now supported on + Windows. (Contributed by Steve Dower in :issue:`23668`.) + os.path ------- @@ -1034,6 +1119,9 @@ list of ciphers sent at handshake. (Contributed by Benjamin Peterson in :issue:`23186`.) +* :func:`~ssl.match_hostname` now supports matching of IP addresses. + (Contributed by Antoine Pitrou in :issue:`23239`.) + socket ------ @@ -1099,6 +1187,13 @@ * The :func:`time.monotonic` function is now always available. (Contributed by Victor Stinner in :issue:`22043`.) +timeit +------ + +* New command line option ``-u`` or ``--unit=U`` to specify a time unit for + the timer output. Supported options are ``usec``, ``msec``, or ``sec``. + (Contributed by Julian Gindi in :issue:`18983`.) + tkinter ------- @@ -1118,6 +1213,11 @@ :class:`~traceback.StackSummary`, and :class:`traceback.FrameSummary`. (Contributed by Robert Collins in :issue:`17911`.) +* :func:`~traceback.print_tb` and :func:`~traceback.print_stack` now support + negative values for the *limit* argument. + (Contributed by Dmitry Kazakov in :issue:`22619`.) + + types ----- @@ -1152,6 +1252,12 @@ * The :mod:`unicodedata` module now uses data from `Unicode 8.0.0 `_. +unittest +-------- + +* New command line option ``--locals`` to show local variables in + tracebacks. + (Contributed by Robert Collins in :issue:`22936`.) wsgiref ------- @@ -1176,14 +1282,6 @@ :class:`~xml.sax.xmlreader.InputSource` object. (Contributed by Serhiy Storchaka in :issue:`2175`.) -faulthandler ------------- - -* :func:`~faulthandler.enable`, :func:`~faulthandler.register`, - :func:`~faulthandler.dump_traceback` and - :func:`~faulthandler.dump_traceback_later` functions now accept file - descriptors. (Contributed by Wei Wu in :issue:`23566`.) - zipfile ------- @@ -1194,6 +1292,14 @@ creation) mode. (Contributed by Serhiy Storchaka in :issue:`21717`.) +Other module-level changes +========================== + +* Many functions in modules :mod:`mmap`, :mod:`ossaudiodev`, :mod:`socket`, + :mod:`ssl`, and :mod:`codecs`, now accept writable bytes-like objects. + (Contributed by Serhiy Storchaka in :issue:`23001`.) + + Optimizations ============= @@ -1237,6 +1343,22 @@ as fast as with ``ensure_ascii=True``. (Contributed by Naoki Inada in :issue:`23206`.) +* :c:func:`PyObject_IsInstance` and :c:func:`PyObject_IsSubclass` have + been sped up in the common case that the second argument has metaclass + :class:`type`. + (Contributed Georg Brandl by in :issue:`22540`.) + +* Method caching was slightly improved, yielding up to 5% performance + improvement in some benchmarks. + (Contributed by Antoine Pitrou in :issue:`22847`.) + +* Objects from :mod:`random` module now use 2x less memory on 64-bit + builds. + (Contributed by Serhiy Storchaka in :issue:`23488`.) + +* property() getter calls are up to 25% faster. + (Contributed by Joe Jevnik in :issue:`23910`.) + Build and C API Changes ======================= @@ -1276,6 +1398,9 @@ * The :mod:`formatter` module has now graduated to full deprecation and is still slated for removal in Python 3.6. +* :func:`~asyncio.async` was deprecated in favour of + :func:`~asyncio.ensure_future`. + * :mod:`smtpd` has in the past always decoded the DATA portion of email messages using the ``utf-8`` codec. This can now be controlled by the new *decode_data* keyword to :class:`~smtpd.SMTPServer`. The default value is @@ -1323,12 +1448,6 @@ * None yet. -Deprecated features -------------------- - -* None yet. - - Removed ======= @@ -1486,3 +1605,5 @@ :c:type:`PyTypeObject` was replaced with a :c:member:`tp_as_async` slot. Refer to :ref:`coro-objects` for new types, structures and functions. + +* :c:member:`PyTypeObject.tp_finalize` is now part of stable ABI. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 9 18:25:14 2015 From: python-checkins at python.org (yury.selivanov) Date: Wed, 09 Sep 2015 16:25:14 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogd2hhdHNuZXcvMy41?= =?utf-8?q?=3A_First_pass_over_NEWS_is_done=2E?= Message-ID: <20150909162513.17981.21699@psf.io> https://hg.python.org/cpython/rev/97f928f49a54 changeset: 97827:97f928f49a54 branch: 3.5 parent: 97810:a172a8035a93 user: Yury Selivanov date: Wed Sep 09 12:23:01 2015 -0400 summary: whatsnew/3.5: First pass over NEWS is done. files: Doc/library/collections.rst | 4 + Doc/whatsnew/3.5.rst | 175 ++++++++++++++++++++--- 2 files changed, 152 insertions(+), 27 deletions(-) diff --git a/Doc/library/collections.rst b/Doc/library/collections.rst --- a/Doc/library/collections.rst +++ b/Doc/library/collections.rst @@ -1157,3 +1157,7 @@ be an instance of :class:`bytes`, :class:`str`, :class:`UserString` (or a subclass) or an arbitrary sequence which can be converted into a string using the built-in :func:`str` function. + + .. versionchanged:: 3.5 + New methods ``__getnewargs__``, ``__rmod__``, ``casefold``, + ``format_map``, ``isprintable``, and ``maketrans``. diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -112,11 +112,13 @@ :issue:`16991`. * You may now pass bytes to the :mod:`tempfile` module's APIs and it will - return the temporary pathname as bytes instead of str. It also accepts - a value of ``None`` on parameters where only str was accepted in the past to - do the right thing based on the types of the other inputs. Two functions, - :func:`gettempdirb` and :func:`gettempprefixb`, have been added to go along - with this. This behavior matches that of the :mod:`os` APIs. + return the temporary pathname as :class:`bytes` instead of :class:`str`. + It also accepts a value of ``None`` on parameters where only str was + accepted in the past to do the right thing based on the types of the + other inputs. Two functions, :func:`gettempdirb` and + :func:`gettempprefixb`, have been added to go along with this. + This behavior matches that of the :mod:`os` APIs. + (Contributed by Gregory P. Smith in :issue:`24230`.) * :mod:`ssl` module gained support for Memory BIO, which decouples SSL protocol handling from network IO. (Contributed by Geert Jansen in @@ -127,6 +129,10 @@ :class:`~traceback.StackSummary`, and :class:`traceback.FrameSummary`. (Contributed by Robert Collins in :issue:`17911`.) +* Most of :func:`functools.lru_cache` machinery is now implemented in C. + (Contributed by Matt Joiner, Alexey Kachayev, and Serhiy Storchaka + in :issue:`14373`.) + Security improvements: * SSLv3 is now disabled throughout the standard library. @@ -428,14 +434,14 @@ for the rationale); * :mod:`select` functions: :func:`~select.devpoll.poll`, - :func:`~select.epoll.poll`, :func:`~select.kqueue.control`, - :func:`~select.poll.poll`, :func:`~select.select`; + :func:`~select.epoll.poll`, :func:`~select.kqueue.control`, + :func:`~select.poll.poll`, :func:`~select.select`; * :func:`socket.socket` methods: :meth:`~socket.socket.accept`, :meth:`~socket.socket.connect` (except for non-blocking sockets), - :meth:`~socket.socket.recv`, :meth:`~socket.socket.recvfrom`, - :meth:`~socket.socket.recvmsg`, :meth:`~socket.socket.send`, - :meth:`~socket.socket.sendall`, :meth:`~socket.socket.sendmsg`, + :meth:`~socket.socket.recv`, :meth:`~socket.socket.recvfrom`, + :meth:`~socket.socket.recvmsg`, :meth:`~socket.socket.send`, + :meth:`~socket.socket.sendall`, :meth:`~socket.socket.sendmsg`, :meth:`~socket.socket.sendto`; * :func:`signal.sigtimedwait`, :func:`signal.sigwaitinfo`; @@ -556,6 +562,9 @@ * New Tajik :ref:`codec ` ``koi8_t``. (Contributed by Serhiy Storchaka in :issue:`22681`.) +* Circular imports involving relative imports are now supported. + (Contributed by Brett Cannon and Antoine Pitrou in :issue:`17636`.) + New Modules =========== @@ -589,6 +598,11 @@ :ref:`allow_abbrev` to ``False``. (Contributed by Jonathan Paugh, Steven Bethard, paul j3 and Daniel Eriksson.) +* New *allow_abbrev* parameter for :class:`~argparse.ArgumentParser` to + allow long options to be abbreviated if the abbreviation is unambiguous. + (Contributed by Jonathan Paugh, Steven Bethard, paul j3 and + Daniel Eriksson in :issue:`14910`.) + bz2 --- @@ -628,6 +642,33 @@ (Contributed by Berker Peksag in :issue:`24064`.) +* :class:`~collections.deque` has new methods: + :meth:`~collections.deque.index`, :meth:`~collections.deque.insert`, and + :meth:`~collections.deque.copy`. This allows deques to be registered as a + :class:`~collections.abc.MutableSequence` and improves their + substitutablity for lists. + (Contributed by Raymond Hettinger :issue:`23704`.) + +* :class:`~collections.UserString` now supports ``__getnewargs__``, + ``__rmod__``, ``casefold``, ``format_map``, ``isprintable``, and + ``maketrans`` methods. + (Contributed by Joe Jevnik in :issue:`22189`.) + +* :class:`collections.OrderedDict` is now implemented in C, which improves + its performance between 4x to 100x times. + (Contributed by Eric Snow in :issue:`16991`.) + +collections.abc +--------------- + +* New :class:`~collections.abc.Generator` abstract base class. + (Contributed by Stefan Behnel in :issue:`24018`.) + +* New :class:`~collections.abc.Coroutine`, + :class:`~collections.abc.AsyncIterator`, and + :class:`~collections.abc.AsyncIterable` abstract base classes. + (Contributed by Yury Selivanov in :issue:`24184`.) + compileall ---------- @@ -672,7 +713,8 @@ (Contributed by Berker Peksag in :issue:`2052`.) * It's now possible to compare lists of byte strings with - :func:`difflib.diff_bytes` (fixes a regression from Python 2). + :func:`difflib.diff_bytes`. This fixes a regression from Python 2. + (Contributed by Terry J. Reedy and Greg Ward in :issue:`17445`.) distutils --------- @@ -711,9 +753,26 @@ :rfc:`6532` and used with an SMTP server that supports the :rfc:`6531` ``SMTPUTF8`` extension. (Contributed by R. David Murray in :issue:`24211`.) +faulthandler +------------ + +* :func:`~faulthandler.enable`, :func:`~faulthandler.register`, + :func:`~faulthandler.dump_traceback` and + :func:`~faulthandler.dump_traceback_later` functions now accept file + descriptors. (Contributed by Wei Wu in :issue:`23566`.) + +functools +--------- + +* Most of :func:`~functools.lru_cache` machinery is now implemented in C. + (Contributed by Matt Joiner, Alexey Kachayev, and Serhiy Storchaka + in :issue:`14373`.) + glob ---- +00002-5938667 + * :func:`~glob.iglob` and :func:`~glob.glob` now support recursive search in subdirectories using the "``**``" pattern. (Contributed by Serhiy Storchaka in :issue:`13968`.) @@ -805,6 +864,13 @@ and :func:`~inspect.getinnerframes` now return a list of named tuples. (Contributed by Daniel Shahaf in :issue:`16808`.) +io +-- + +* New Python implementation of :class:`io.FileIO` to make dependency on + ``_io`` module optional. + (Contributed by Serhiy Storchaka in :issue:`21859`.) + ipaddress --------- @@ -845,6 +911,10 @@ for HTTP connection. (Contributed by Alex Gaynor in :issue:`22788`.) +* :class:`~logging.handlers.QueueListener` now takes a *respect_handler_level* + keyword argument which, if set to ``True``, will pass messages to handlers + taking handler levels into account. (Contributed by Vinay Sajip.) + lzma ---- @@ -852,15 +922,26 @@ to limit the maximum size of decompressed data. (Contributed by Martin Panter in :issue:`15955`.) - math ---- * :data:`math.inf` and :data:`math.nan` constants added. (Contributed by Mark Dickinson in :issue:`23185`.) -* :func:`math.isclose` function added. + +* New :func:`~math.isclose` function. (Contributed by Chris Barker and Tal Einat in :issue:`24270`.) +* New :func:`~math.gcd` function. The :func:`fractions.gcd` function now is + deprecated. + (Contributed by Mark Dickinson and Serhiy Storchaka in :issue:`22486`.) + +operator +-------- + +* :func:`~operator.attrgetter`, :func:`~operator.itemgetter`, and + :func:`~operator.methodcaller` objects now support pickling. + (Contributed by Josh Rosenberg and Serhiy Storchaka in :issue:`22955`.) + os -- @@ -877,11 +958,15 @@ now used when available. On OpenBSD 5.6 and newer, the C ``getentropy()`` function is now used. These functions avoid the usage of an internal file descriptor. + (Contributed by Victor Stinner in :issue:`22181`.) * New :func:`os.get_blocking` and :func:`os.set_blocking` functions to get and set the blocking mode of file descriptors. (Contributed by Victor Stinner in :issue:`22054`.) +* :func:`~os.truncate` and :func:`~os.ftruncate` are now supported on + Windows. (Contributed by Steve Dower in :issue:`23668`.) + os.path ------- @@ -1034,6 +1119,9 @@ list of ciphers sent at handshake. (Contributed by Benjamin Peterson in :issue:`23186`.) +* :func:`~ssl.match_hostname` now supports matching of IP addresses. + (Contributed by Antoine Pitrou in :issue:`23239`.) + socket ------ @@ -1099,6 +1187,13 @@ * The :func:`time.monotonic` function is now always available. (Contributed by Victor Stinner in :issue:`22043`.) +timeit +------ + +* New command line option ``-u`` or ``--unit=U`` to specify a time unit for + the timer output. Supported options are ``usec``, ``msec``, or ``sec``. + (Contributed by Julian Gindi in :issue:`18983`.) + tkinter ------- @@ -1118,6 +1213,11 @@ :class:`~traceback.StackSummary`, and :class:`traceback.FrameSummary`. (Contributed by Robert Collins in :issue:`17911`.) +* :func:`~traceback.print_tb` and :func:`~traceback.print_stack` now support + negative values for the *limit* argument. + (Contributed by Dmitry Kazakov in :issue:`22619`.) + + types ----- @@ -1152,6 +1252,12 @@ * The :mod:`unicodedata` module now uses data from `Unicode 8.0.0 `_. +unittest +-------- + +* New command line option ``--locals`` to show local variables in + tracebacks. + (Contributed by Robert Collins in :issue:`22936`.) wsgiref ------- @@ -1176,14 +1282,6 @@ :class:`~xml.sax.xmlreader.InputSource` object. (Contributed by Serhiy Storchaka in :issue:`2175`.) -faulthandler ------------- - -* :func:`~faulthandler.enable`, :func:`~faulthandler.register`, - :func:`~faulthandler.dump_traceback` and - :func:`~faulthandler.dump_traceback_later` functions now accept file - descriptors. (Contributed by Wei Wu in :issue:`23566`.) - zipfile ------- @@ -1194,6 +1292,14 @@ creation) mode. (Contributed by Serhiy Storchaka in :issue:`21717`.) +Other module-level changes +========================== + +* Many functions in modules :mod:`mmap`, :mod:`ossaudiodev`, :mod:`socket`, + :mod:`ssl`, and :mod:`codecs`, now accept writable bytes-like objects. + (Contributed by Serhiy Storchaka in :issue:`23001`.) + + Optimizations ============= @@ -1237,6 +1343,22 @@ as fast as with ``ensure_ascii=True``. (Contributed by Naoki Inada in :issue:`23206`.) +* :c:func:`PyObject_IsInstance` and :c:func:`PyObject_IsSubclass` have + been sped up in the common case that the second argument has metaclass + :class:`type`. + (Contributed Georg Brandl by in :issue:`22540`.) + +* Method caching was slightly improved, yielding up to 5% performance + improvement in some benchmarks. + (Contributed by Antoine Pitrou in :issue:`22847`.) + +* Objects from :mod:`random` module now use 2x less memory on 64-bit + builds. + (Contributed by Serhiy Storchaka in :issue:`23488`.) + +* property() getter calls are up to 25% faster. + (Contributed by Joe Jevnik in :issue:`23910`.) + Build and C API Changes ======================= @@ -1276,6 +1398,9 @@ * The :mod:`formatter` module has now graduated to full deprecation and is still slated for removal in Python 3.6. +* :func:`~asyncio.async` was deprecated in favour of + :func:`~asyncio.ensure_future`. + * :mod:`smtpd` has in the past always decoded the DATA portion of email messages using the ``utf-8`` codec. This can now be controlled by the new *decode_data* keyword to :class:`~smtpd.SMTPServer`. The default value is @@ -1323,12 +1448,6 @@ * None yet. -Deprecated features -------------------- - -* None yet. - - Removed ======= @@ -1486,3 +1605,5 @@ :c:type:`PyTypeObject` was replaced with a :c:member:`tp_as_async` slot. Refer to :ref:`coro-objects` for new types, structures and functions. + +* :c:member:`PyTypeObject.tp_finalize` is now part of stable ABI. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 9 18:25:38 2015 From: python-checkins at python.org (yury.selivanov) Date: Wed, 09 Sep 2015 16:25:38 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogTWVyZ2UgMy41?= Message-ID: <20150909162514.101482.88760@psf.io> https://hg.python.org/cpython/rev/bf1a2af73140 changeset: 97828:bf1a2af73140 parent: 97826:816fdd881384 parent: 97827:97f928f49a54 user: Yury Selivanov date: Wed Sep 09 12:23:18 2015 -0400 summary: Merge 3.5 files: Doc/library/collections.rst | 4 + Doc/whatsnew/3.5.rst | 175 ++++++++++++++++++++--- 2 files changed, 152 insertions(+), 27 deletions(-) diff --git a/Doc/library/collections.rst b/Doc/library/collections.rst --- a/Doc/library/collections.rst +++ b/Doc/library/collections.rst @@ -1160,3 +1160,7 @@ be an instance of :class:`bytes`, :class:`str`, :class:`UserString` (or a subclass) or an arbitrary sequence which can be converted into a string using the built-in :func:`str` function. + + .. versionchanged:: 3.5 + New methods ``__getnewargs__``, ``__rmod__``, ``casefold``, + ``format_map``, ``isprintable``, and ``maketrans``. diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -112,11 +112,13 @@ :issue:`16991`. * You may now pass bytes to the :mod:`tempfile` module's APIs and it will - return the temporary pathname as bytes instead of str. It also accepts - a value of ``None`` on parameters where only str was accepted in the past to - do the right thing based on the types of the other inputs. Two functions, - :func:`gettempdirb` and :func:`gettempprefixb`, have been added to go along - with this. This behavior matches that of the :mod:`os` APIs. + return the temporary pathname as :class:`bytes` instead of :class:`str`. + It also accepts a value of ``None`` on parameters where only str was + accepted in the past to do the right thing based on the types of the + other inputs. Two functions, :func:`gettempdirb` and + :func:`gettempprefixb`, have been added to go along with this. + This behavior matches that of the :mod:`os` APIs. + (Contributed by Gregory P. Smith in :issue:`24230`.) * :mod:`ssl` module gained support for Memory BIO, which decouples SSL protocol handling from network IO. (Contributed by Geert Jansen in @@ -127,6 +129,10 @@ :class:`~traceback.StackSummary`, and :class:`traceback.FrameSummary`. (Contributed by Robert Collins in :issue:`17911`.) +* Most of :func:`functools.lru_cache` machinery is now implemented in C. + (Contributed by Matt Joiner, Alexey Kachayev, and Serhiy Storchaka + in :issue:`14373`.) + Security improvements: * SSLv3 is now disabled throughout the standard library. @@ -428,14 +434,14 @@ for the rationale); * :mod:`select` functions: :func:`~select.devpoll.poll`, - :func:`~select.epoll.poll`, :func:`~select.kqueue.control`, - :func:`~select.poll.poll`, :func:`~select.select`; + :func:`~select.epoll.poll`, :func:`~select.kqueue.control`, + :func:`~select.poll.poll`, :func:`~select.select`; * :func:`socket.socket` methods: :meth:`~socket.socket.accept`, :meth:`~socket.socket.connect` (except for non-blocking sockets), - :meth:`~socket.socket.recv`, :meth:`~socket.socket.recvfrom`, - :meth:`~socket.socket.recvmsg`, :meth:`~socket.socket.send`, - :meth:`~socket.socket.sendall`, :meth:`~socket.socket.sendmsg`, + :meth:`~socket.socket.recv`, :meth:`~socket.socket.recvfrom`, + :meth:`~socket.socket.recvmsg`, :meth:`~socket.socket.send`, + :meth:`~socket.socket.sendall`, :meth:`~socket.socket.sendmsg`, :meth:`~socket.socket.sendto`; * :func:`signal.sigtimedwait`, :func:`signal.sigwaitinfo`; @@ -556,6 +562,9 @@ * New Tajik :ref:`codec ` ``koi8_t``. (Contributed by Serhiy Storchaka in :issue:`22681`.) +* Circular imports involving relative imports are now supported. + (Contributed by Brett Cannon and Antoine Pitrou in :issue:`17636`.) + New Modules =========== @@ -589,6 +598,11 @@ :ref:`allow_abbrev` to ``False``. (Contributed by Jonathan Paugh, Steven Bethard, paul j3 and Daniel Eriksson.) +* New *allow_abbrev* parameter for :class:`~argparse.ArgumentParser` to + allow long options to be abbreviated if the abbreviation is unambiguous. + (Contributed by Jonathan Paugh, Steven Bethard, paul j3 and + Daniel Eriksson in :issue:`14910`.) + bz2 --- @@ -628,6 +642,33 @@ (Contributed by Berker Peksag in :issue:`24064`.) +* :class:`~collections.deque` has new methods: + :meth:`~collections.deque.index`, :meth:`~collections.deque.insert`, and + :meth:`~collections.deque.copy`. This allows deques to be registered as a + :class:`~collections.abc.MutableSequence` and improves their + substitutablity for lists. + (Contributed by Raymond Hettinger :issue:`23704`.) + +* :class:`~collections.UserString` now supports ``__getnewargs__``, + ``__rmod__``, ``casefold``, ``format_map``, ``isprintable``, and + ``maketrans`` methods. + (Contributed by Joe Jevnik in :issue:`22189`.) + +* :class:`collections.OrderedDict` is now implemented in C, which improves + its performance between 4x to 100x times. + (Contributed by Eric Snow in :issue:`16991`.) + +collections.abc +--------------- + +* New :class:`~collections.abc.Generator` abstract base class. + (Contributed by Stefan Behnel in :issue:`24018`.) + +* New :class:`~collections.abc.Coroutine`, + :class:`~collections.abc.AsyncIterator`, and + :class:`~collections.abc.AsyncIterable` abstract base classes. + (Contributed by Yury Selivanov in :issue:`24184`.) + compileall ---------- @@ -672,7 +713,8 @@ (Contributed by Berker Peksag in :issue:`2052`.) * It's now possible to compare lists of byte strings with - :func:`difflib.diff_bytes` (fixes a regression from Python 2). + :func:`difflib.diff_bytes`. This fixes a regression from Python 2. + (Contributed by Terry J. Reedy and Greg Ward in :issue:`17445`.) distutils --------- @@ -711,9 +753,26 @@ :rfc:`6532` and used with an SMTP server that supports the :rfc:`6531` ``SMTPUTF8`` extension. (Contributed by R. David Murray in :issue:`24211`.) +faulthandler +------------ + +* :func:`~faulthandler.enable`, :func:`~faulthandler.register`, + :func:`~faulthandler.dump_traceback` and + :func:`~faulthandler.dump_traceback_later` functions now accept file + descriptors. (Contributed by Wei Wu in :issue:`23566`.) + +functools +--------- + +* Most of :func:`~functools.lru_cache` machinery is now implemented in C. + (Contributed by Matt Joiner, Alexey Kachayev, and Serhiy Storchaka + in :issue:`14373`.) + glob ---- +00002-5938667 + * :func:`~glob.iglob` and :func:`~glob.glob` now support recursive search in subdirectories using the "``**``" pattern. (Contributed by Serhiy Storchaka in :issue:`13968`.) @@ -805,6 +864,13 @@ and :func:`~inspect.getinnerframes` now return a list of named tuples. (Contributed by Daniel Shahaf in :issue:`16808`.) +io +-- + +* New Python implementation of :class:`io.FileIO` to make dependency on + ``_io`` module optional. + (Contributed by Serhiy Storchaka in :issue:`21859`.) + ipaddress --------- @@ -845,6 +911,10 @@ for HTTP connection. (Contributed by Alex Gaynor in :issue:`22788`.) +* :class:`~logging.handlers.QueueListener` now takes a *respect_handler_level* + keyword argument which, if set to ``True``, will pass messages to handlers + taking handler levels into account. (Contributed by Vinay Sajip.) + lzma ---- @@ -852,15 +922,26 @@ to limit the maximum size of decompressed data. (Contributed by Martin Panter in :issue:`15955`.) - math ---- * :data:`math.inf` and :data:`math.nan` constants added. (Contributed by Mark Dickinson in :issue:`23185`.) -* :func:`math.isclose` function added. + +* New :func:`~math.isclose` function. (Contributed by Chris Barker and Tal Einat in :issue:`24270`.) +* New :func:`~math.gcd` function. The :func:`fractions.gcd` function now is + deprecated. + (Contributed by Mark Dickinson and Serhiy Storchaka in :issue:`22486`.) + +operator +-------- + +* :func:`~operator.attrgetter`, :func:`~operator.itemgetter`, and + :func:`~operator.methodcaller` objects now support pickling. + (Contributed by Josh Rosenberg and Serhiy Storchaka in :issue:`22955`.) + os -- @@ -877,11 +958,15 @@ now used when available. On OpenBSD 5.6 and newer, the C ``getentropy()`` function is now used. These functions avoid the usage of an internal file descriptor. + (Contributed by Victor Stinner in :issue:`22181`.) * New :func:`os.get_blocking` and :func:`os.set_blocking` functions to get and set the blocking mode of file descriptors. (Contributed by Victor Stinner in :issue:`22054`.) +* :func:`~os.truncate` and :func:`~os.ftruncate` are now supported on + Windows. (Contributed by Steve Dower in :issue:`23668`.) + os.path ------- @@ -1034,6 +1119,9 @@ list of ciphers sent at handshake. (Contributed by Benjamin Peterson in :issue:`23186`.) +* :func:`~ssl.match_hostname` now supports matching of IP addresses. + (Contributed by Antoine Pitrou in :issue:`23239`.) + socket ------ @@ -1099,6 +1187,13 @@ * The :func:`time.monotonic` function is now always available. (Contributed by Victor Stinner in :issue:`22043`.) +timeit +------ + +* New command line option ``-u`` or ``--unit=U`` to specify a time unit for + the timer output. Supported options are ``usec``, ``msec``, or ``sec``. + (Contributed by Julian Gindi in :issue:`18983`.) + tkinter ------- @@ -1118,6 +1213,11 @@ :class:`~traceback.StackSummary`, and :class:`traceback.FrameSummary`. (Contributed by Robert Collins in :issue:`17911`.) +* :func:`~traceback.print_tb` and :func:`~traceback.print_stack` now support + negative values for the *limit* argument. + (Contributed by Dmitry Kazakov in :issue:`22619`.) + + types ----- @@ -1152,6 +1252,12 @@ * The :mod:`unicodedata` module now uses data from `Unicode 8.0.0 `_. +unittest +-------- + +* New command line option ``--locals`` to show local variables in + tracebacks. + (Contributed by Robert Collins in :issue:`22936`.) wsgiref ------- @@ -1176,14 +1282,6 @@ :class:`~xml.sax.xmlreader.InputSource` object. (Contributed by Serhiy Storchaka in :issue:`2175`.) -faulthandler ------------- - -* :func:`~faulthandler.enable`, :func:`~faulthandler.register`, - :func:`~faulthandler.dump_traceback` and - :func:`~faulthandler.dump_traceback_later` functions now accept file - descriptors. (Contributed by Wei Wu in :issue:`23566`.) - zipfile ------- @@ -1194,6 +1292,14 @@ creation) mode. (Contributed by Serhiy Storchaka in :issue:`21717`.) +Other module-level changes +========================== + +* Many functions in modules :mod:`mmap`, :mod:`ossaudiodev`, :mod:`socket`, + :mod:`ssl`, and :mod:`codecs`, now accept writable bytes-like objects. + (Contributed by Serhiy Storchaka in :issue:`23001`.) + + Optimizations ============= @@ -1237,6 +1343,22 @@ as fast as with ``ensure_ascii=True``. (Contributed by Naoki Inada in :issue:`23206`.) +* :c:func:`PyObject_IsInstance` and :c:func:`PyObject_IsSubclass` have + been sped up in the common case that the second argument has metaclass + :class:`type`. + (Contributed Georg Brandl by in :issue:`22540`.) + +* Method caching was slightly improved, yielding up to 5% performance + improvement in some benchmarks. + (Contributed by Antoine Pitrou in :issue:`22847`.) + +* Objects from :mod:`random` module now use 2x less memory on 64-bit + builds. + (Contributed by Serhiy Storchaka in :issue:`23488`.) + +* property() getter calls are up to 25% faster. + (Contributed by Joe Jevnik in :issue:`23910`.) + Build and C API Changes ======================= @@ -1276,6 +1398,9 @@ * The :mod:`formatter` module has now graduated to full deprecation and is still slated for removal in Python 3.6. +* :func:`~asyncio.async` was deprecated in favour of + :func:`~asyncio.ensure_future`. + * :mod:`smtpd` has in the past always decoded the DATA portion of email messages using the ``utf-8`` codec. This can now be controlled by the new *decode_data* keyword to :class:`~smtpd.SMTPServer`. The default value is @@ -1323,12 +1448,6 @@ * None yet. -Deprecated features -------------------- - -* None yet. - - Removed ======= @@ -1486,3 +1605,5 @@ :c:type:`PyTypeObject` was replaced with a :c:member:`tp_as_async` slot. Refer to :ref:`coro-objects` for new types, structures and functions. + +* :c:member:`PyTypeObject.tp_finalize` is now part of stable ABI. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 9 18:25:39 2015 From: python-checkins at python.org (yury.selivanov) Date: Wed, 09 Sep 2015 16:25:39 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogTWVyZ2UgMy41?= Message-ID: <20150909162514.27695.31489@psf.io> https://hg.python.org/cpython/rev/25883bb02f60 changeset: 97830:25883bb02f60 parent: 97828:bf1a2af73140 parent: 97829:91573c156b75 user: Yury Selivanov date: Wed Sep 09 12:24:33 2015 -0400 summary: Merge 3.5 files: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 9 19:50:28 2015 From: python-checkins at python.org (yury.selivanov) Date: Wed, 09 Sep 2015 17:50:28 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogTWVyZ2UgMy41?= Message-ID: <20150909175026.11246.54544@psf.io> https://hg.python.org/cpython/rev/691cf5eb6a1a changeset: 97832:691cf5eb6a1a parent: 97830:25883bb02f60 parent: 97831:321d06b265cf user: Yury Selivanov date: Wed Sep 09 13:50:12 2015 -0400 summary: Merge 3.5 files: Doc/whatsnew/3.5.rst | 8 ++------ 1 files changed, 2 insertions(+), 6 deletions(-) diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -596,12 +596,8 @@ * :class:`~argparse.ArgumentParser` now allows to disable :ref:`abbreviated usage ` of long options by setting :ref:`allow_abbrev` to ``False``. - (Contributed by Jonathan Paugh, Steven Bethard, paul j3 and Daniel Eriksson.) - -* New *allow_abbrev* parameter for :class:`~argparse.ArgumentParser` to - allow long options to be abbreviated if the abbreviation is unambiguous. - (Contributed by Jonathan Paugh, Steven Bethard, paul j3 and - Daniel Eriksson in :issue:`14910`.) + (Contributed by Jonathan Paugh, Steven Bethard, paul j3 and Daniel Eriksson + in :issue:`14910`.) bz2 --- -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 9 19:50:36 2015 From: python-checkins at python.org (yury.selivanov) Date: Wed, 09 Sep 2015 17:50:36 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogd2hhdHNuZXcvMy41?= =?utf-8?q?=3A_Merge_argparse_entries?= Message-ID: <20150909175026.66868.91334@psf.io> https://hg.python.org/cpython/rev/321d06b265cf changeset: 97831:321d06b265cf branch: 3.5 parent: 97829:91573c156b75 user: Yury Selivanov date: Wed Sep 09 13:49:29 2015 -0400 summary: whatsnew/3.5: Merge argparse entries files: Doc/whatsnew/3.5.rst | 8 ++------ 1 files changed, 2 insertions(+), 6 deletions(-) diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -596,12 +596,8 @@ * :class:`~argparse.ArgumentParser` now allows to disable :ref:`abbreviated usage ` of long options by setting :ref:`allow_abbrev` to ``False``. - (Contributed by Jonathan Paugh, Steven Bethard, paul j3 and Daniel Eriksson.) - -* New *allow_abbrev* parameter for :class:`~argparse.ArgumentParser` to - allow long options to be abbreviated if the abbreviation is unambiguous. - (Contributed by Jonathan Paugh, Steven Bethard, paul j3 and - Daniel Eriksson in :issue:`14910`.) + (Contributed by Jonathan Paugh, Steven Bethard, paul j3 and Daniel Eriksson + in :issue:`14910`.) bz2 --- -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 9 22:40:35 2015 From: python-checkins at python.org (berker.peksag) Date: Wed, 09 Sep 2015 20:40:35 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzI0ODU3?= =?utf-8?q?=3A_Comparing_call=5Fargs_to_a_long_sequence_now_correctly_retu?= =?utf-8?q?rns_a?= Message-ID: <20150909204035.17971.69850@psf.io> https://hg.python.org/cpython/rev/83ea55a1204a changeset: 97833:83ea55a1204a branch: 3.4 parent: 97804:da9b26670e44 user: Berker Peksag date: Wed Sep 09 23:35:25 2015 +0300 summary: Issue #24857: Comparing call_args to a long sequence now correctly returns a boolean result instead of raising an exception. Patch by A Kaptur. files: Lib/unittest/mock.py | 5 +++-- Lib/unittest/test/testmock/testmock.py | 3 +++ Misc/NEWS | 3 +++ 3 files changed, 9 insertions(+), 2 deletions(-) diff --git a/Lib/unittest/mock.py b/Lib/unittest/mock.py --- a/Lib/unittest/mock.py +++ b/Lib/unittest/mock.py @@ -1986,8 +1986,7 @@ else: other_args = () other_kwargs = value - else: - # len 2 + elif len_other == 2: # could be (name, args) or (name, kwargs) or (args, kwargs) first, second = other if isinstance(first, str): @@ -1998,6 +1997,8 @@ other_args, other_kwargs = (), second else: other_args, other_kwargs = first, second + else: + return False if self_name and other_name != self_name: return False diff --git a/Lib/unittest/test/testmock/testmock.py b/Lib/unittest/test/testmock/testmock.py --- a/Lib/unittest/test/testmock/testmock.py +++ b/Lib/unittest/test/testmock/testmock.py @@ -291,6 +291,9 @@ self.assertEqual(mock.call_args, ((sentinel.Arg,), {"kw": sentinel.Kwarg})) + # Comparing call_args to a long sequence should not raise + # an exception. See issue 24857. + self.assertFalse(mock.call_args == "a long sequence") def test_assert_called_with(self): mock = Mock() diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -84,6 +84,9 @@ - Issue #24982: shutil.make_archive() with the "zip" format now adds entries for directories (including empty directories) in ZIP file. +- Issue #24857: Comparing call_args to a long sequence now correctly returns a + boolean result instead of raising an exception. Patch by A Kaptur. + - Issue #25019: Fixed a crash caused by setting non-string key of expat parser. Based on patch by John Leitch. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 9 22:40:36 2015 From: python-checkins at python.org (berker.peksag) Date: Wed, 09 Sep 2015 20:40:36 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2324857=3A_Comparing_call=5Fargs_to_a_long_sequen?= =?utf-8?q?ce_now_correctly_returns_a?= Message-ID: <20150909204035.14859.82483@psf.io> https://hg.python.org/cpython/rev/6853b4bc0b22 changeset: 97835:6853b4bc0b22 parent: 97832:691cf5eb6a1a parent: 97834:df91c1879e56 user: Berker Peksag date: Wed Sep 09 23:40:11 2015 +0300 summary: Issue #24857: Comparing call_args to a long sequence now correctly returns a boolean result instead of raising an exception. Patch by A Kaptur. files: Lib/unittest/mock.py | 5 +++-- Lib/unittest/test/testmock/testmock.py | 3 +++ Misc/NEWS | 3 +++ 3 files changed, 9 insertions(+), 2 deletions(-) diff --git a/Lib/unittest/mock.py b/Lib/unittest/mock.py --- a/Lib/unittest/mock.py +++ b/Lib/unittest/mock.py @@ -2005,8 +2005,7 @@ else: other_args = () other_kwargs = value - else: - # len 2 + elif len_other == 2: # could be (name, args) or (name, kwargs) or (args, kwargs) first, second = other if isinstance(first, str): @@ -2017,6 +2016,8 @@ other_args, other_kwargs = (), second else: other_args, other_kwargs = first, second + else: + return False if self_name and other_name != self_name: return False diff --git a/Lib/unittest/test/testmock/testmock.py b/Lib/unittest/test/testmock/testmock.py --- a/Lib/unittest/test/testmock/testmock.py +++ b/Lib/unittest/test/testmock/testmock.py @@ -300,6 +300,9 @@ self.assertEqual(mock.call_args, ((sentinel.Arg,), {"kw": sentinel.Kwarg})) + # Comparing call_args to a long sequence should not raise + # an exception. See issue 24857. + self.assertFalse(mock.call_args == "a long sequence") def test_assert_called_with(self): mock = Mock() diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -98,6 +98,9 @@ Library ------- +- Issue #24857: Comparing call_args to a long sequence now correctly returns a + boolean result instead of raising an exception. Patch by A Kaptur. + - Issue #23144: Make sure that HTMLParser.feed() returns all the data, even when convert_charrefs is True. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 9 22:40:46 2015 From: python-checkins at python.org (berker.peksag) Date: Wed, 09 Sep 2015 20:40:46 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Issue_=2324857=3A_Comparing_call=5Fargs_to_a_long_sequence_now?= =?utf-8?q?_correctly_returns_a?= Message-ID: <20150909204035.66870.13334@psf.io> https://hg.python.org/cpython/rev/df91c1879e56 changeset: 97834:df91c1879e56 branch: 3.5 parent: 97831:321d06b265cf parent: 97833:83ea55a1204a user: Berker Peksag date: Wed Sep 09 23:39:45 2015 +0300 summary: Issue #24857: Comparing call_args to a long sequence now correctly returns a boolean result instead of raising an exception. Patch by A Kaptur. files: Lib/unittest/mock.py | 5 +++-- Lib/unittest/test/testmock/testmock.py | 3 +++ Misc/NEWS | 3 +++ 3 files changed, 9 insertions(+), 2 deletions(-) diff --git a/Lib/unittest/mock.py b/Lib/unittest/mock.py --- a/Lib/unittest/mock.py +++ b/Lib/unittest/mock.py @@ -2005,8 +2005,7 @@ else: other_args = () other_kwargs = value - else: - # len 2 + elif len_other == 2: # could be (name, args) or (name, kwargs) or (args, kwargs) first, second = other if isinstance(first, str): @@ -2017,6 +2016,8 @@ other_args, other_kwargs = (), second else: other_args, other_kwargs = first, second + else: + return False if self_name and other_name != self_name: return False diff --git a/Lib/unittest/test/testmock/testmock.py b/Lib/unittest/test/testmock/testmock.py --- a/Lib/unittest/test/testmock/testmock.py +++ b/Lib/unittest/test/testmock/testmock.py @@ -300,6 +300,9 @@ self.assertEqual(mock.call_args, ((sentinel.Arg,), {"kw": sentinel.Kwarg})) + # Comparing call_args to a long sequence should not raise + # an exception. See issue 24857. + self.assertFalse(mock.call_args == "a long sequence") def test_assert_called_with(self): mock = Mock() diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -14,6 +14,9 @@ Library ------- +- Issue #24857: Comparing call_args to a long sequence now correctly returns a + boolean result instead of raising an exception. Patch by A Kaptur. + - Issue #23144: Make sure that HTMLParser.feed() returns all the data, even when convert_charrefs is True. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 9 23:48:20 2015 From: python-checkins at python.org (victor.stinner) Date: Wed, 09 Sep 2015 21:48:20 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Make_=5FPyTime=5FRoundHalf?= =?utf-8?q?Even=28=29_private_again?= Message-ID: <20150909214819.17985.23049@psf.io> https://hg.python.org/cpython/rev/dae40b1468fb changeset: 97836:dae40b1468fb user: Victor Stinner date: Wed Sep 09 22:28:09 2015 +0200 summary: Make _PyTime_RoundHalfEven() private again files: Include/pytime.h | 5 ----- Python/pytime.c | 4 +++- 2 files changed, 3 insertions(+), 6 deletions(-) diff --git a/Include/pytime.h b/Include/pytime.h --- a/Include/pytime.h +++ b/Include/pytime.h @@ -44,11 +44,6 @@ PyAPI_FUNC(time_t) _PyLong_AsTime_t( PyObject *obj); -/* Round to nearest with ties going to nearest even integer - (_PyTime_ROUND_HALF_EVEN) */ -PyAPI_FUNC(double) _PyTime_RoundHalfEven( - double x); - /* Convert a number of seconds, int or float, to time_t. */ PyAPI_FUNC(int) _PyTime_ObjectToTime_t( PyObject *obj, diff --git a/Python/pytime.c b/Python/pytime.c --- a/Python/pytime.c +++ b/Python/pytime.c @@ -60,7 +60,9 @@ #endif } -double +/* Round to nearest with ties going to nearest even integer + (_PyTime_ROUND_HALF_EVEN) */ +static double _PyTime_RoundHalfEven(double x) { double rounded = round(x); -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 9 23:48:23 2015 From: python-checkins at python.org (victor.stinner) Date: Wed, 09 Sep 2015 21:48:23 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_pytime=3A_add_=5FPyTime=5F?= =?utf-8?q?Round=28=29_helper_to_factorize_code?= Message-ID: <20150909214819.27691.98026@psf.io> https://hg.python.org/cpython/rev/2b38a6cd010c changeset: 97837:2b38a6cd010c user: Victor Stinner date: Wed Sep 09 22:28:58 2015 +0200 summary: pytime: add _PyTime_Round() helper to factorize code files: Python/pytime.c | 45 ++++++++++++++++-------------------- 1 files changed, 20 insertions(+), 25 deletions(-) diff --git a/Python/pytime.c b/Python/pytime.c --- a/Python/pytime.c +++ b/Python/pytime.c @@ -72,6 +72,17 @@ return rounded; } +static double +_PyTime_Round(double x, _PyTime_round_t round) +{ + if (round == _PyTime_ROUND_HALF_EVEN) + return _PyTime_RoundHalfEven(x); + else if (round == _PyTime_ROUND_CEILING) + return ceil(x); + else + return floor(x); +} + static int _PyTime_DoubleToDenominator(double d, time_t *sec, long *numerator, double denominator, _PyTime_round_t round) @@ -83,12 +94,7 @@ floatpart = modf(d, &intpart); floatpart *= denominator; - if (round == _PyTime_ROUND_HALF_EVEN) - floatpart = _PyTime_RoundHalfEven(floatpart); - else if (round == _PyTime_ROUND_CEILING) - floatpart = ceil(floatpart); - else - floatpart = floor(floatpart); + floatpart = _PyTime_Round(floatpart, round); if (floatpart >= denominator) { floatpart -= denominator; intpart += 1.0; @@ -139,12 +145,7 @@ volatile double d; d = PyFloat_AsDouble(obj); - if (round == _PyTime_ROUND_HALF_EVEN) - d = _PyTime_RoundHalfEven(d); - else if (round == _PyTime_ROUND_CEILING) - d = ceil(d); - else - d = floor(d); + d = _PyTime_Round(d, round); (void)modf(d, &intpart); *sec = (time_t)intpart; @@ -255,7 +256,7 @@ static int _PyTime_FromFloatObject(_PyTime_t *t, double value, _PyTime_round_t round, - long to_nanoseconds) + long unit_to_ns) { double err; /* volatile avoids optimization changing how numbers are rounded */ @@ -263,14 +264,8 @@ /* convert to a number of nanoseconds */ d = value; - d *= to_nanoseconds; - - if (round == _PyTime_ROUND_HALF_EVEN) - d = _PyTime_RoundHalfEven(d); - else if (round == _PyTime_ROUND_CEILING) - d = ceil(d); - else - d = floor(d); + d *= (double)unit_to_ns; + d = _PyTime_Round(d, round); *t = (_PyTime_t)d; err = d - (double)*t; @@ -283,12 +278,12 @@ static int _PyTime_FromObject(_PyTime_t *t, PyObject *obj, _PyTime_round_t round, - long to_nanoseconds) + long unit_to_ns) { if (PyFloat_Check(obj)) { double d; d = PyFloat_AsDouble(obj); - return _PyTime_FromFloatObject(t, d, round, to_nanoseconds); + return _PyTime_FromFloatObject(t, d, round, unit_to_ns); } else { #ifdef HAVE_LONG_LONG @@ -305,8 +300,8 @@ _PyTime_overflow(); return -1; } - *t = sec * to_nanoseconds; - if (*t / to_nanoseconds != sec) { + *t = sec * unit_to_ns; + if (*t / unit_to_ns != sec) { _PyTime_overflow(); return -1; } -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 9 23:48:23 2015 From: python-checkins at python.org (victor.stinner) Date: Wed, 09 Sep 2015 21:48:23 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_test=5Ftime=3A_rewrite_PyT?= =?utf-8?q?ime_API_rounding_tests?= Message-ID: <20150909214820.66856.69636@psf.io> https://hg.python.org/cpython/rev/593f75ccfe11 changeset: 97838:593f75ccfe11 user: Victor Stinner date: Wed Sep 09 22:32:48 2015 +0200 summary: test_time: rewrite PyTime API rounding tests Drop all hardcoded tests. Instead, reimplement each function in Python, usually using decimal.Decimal for the rounding mode. Add much more values to the dataset. Test various timestamp units from picroseconds to seconds, in integer and float. Enhance also _PyTime_AsSecondsDouble(). files: Lib/test/test_time.py | 756 ++++++++--------------------- Python/pytime.c | 16 +- 2 files changed, 224 insertions(+), 548 deletions(-) diff --git a/Lib/test/test_time.py b/Lib/test/test_time.py --- a/Lib/test/test_time.py +++ b/Lib/test/test_time.py @@ -1,6 +1,8 @@ from test import support +import decimal import enum import locale +import math import platform import sys import sysconfig @@ -21,9 +23,11 @@ TIME_MAXYEAR = (1 << 8 * SIZEOF_INT - 1) - 1 TIME_MINYEAR = -TIME_MAXYEAR - 1 +SEC_TO_US = 10 ** 6 US_TO_NS = 10 ** 3 MS_TO_NS = 10 ** 6 SEC_TO_NS = 10 ** 9 +NS_TO_SEC = 10 ** 9 class _PyTime(enum.IntEnum): # Round towards minus infinity (-inf) @@ -33,8 +37,13 @@ # Round to nearest with ties going to nearest even integer ROUND_HALF_EVEN = 2 -ALL_ROUNDING_METHODS = (_PyTime.ROUND_FLOOR, _PyTime.ROUND_CEILING, - _PyTime.ROUND_HALF_EVEN) +# Rounding modes supported by PyTime +ROUNDING_MODES = ( + # (PyTime rounding method, decimal rounding method) + (_PyTime.ROUND_FLOOR, decimal.ROUND_FLOOR), + (_PyTime.ROUND_CEILING, decimal.ROUND_CEILING), + (_PyTime.ROUND_HALF_EVEN, decimal.ROUND_HALF_EVEN), +) class TimeTestCase(unittest.TestCase): @@ -610,126 +619,6 @@ class TestPytime(unittest.TestCase): - def setUp(self): - self.invalid_values = ( - -(2 ** 100), 2 ** 100, - -(2.0 ** 100.0), 2.0 ** 100.0, - ) - - @support.cpython_only - def test_time_t(self): - from _testcapi import pytime_object_to_time_t - - # Conversion giving the same result for all rounding methods - for rnd in ALL_ROUNDING_METHODS: - for obj, seconds in ( - # int - (-1, -1), - (0, 0), - (1, 1), - - # float - (-1.0, -1), - (1.0, 1), - ): - with self.subTest(obj=obj, round=rnd, seconds=seconds): - self.assertEqual(pytime_object_to_time_t(obj, rnd), - seconds) - - # Conversion giving different results depending on the rounding method - FLOOR = _PyTime.ROUND_FLOOR - CEILING = _PyTime.ROUND_CEILING - HALF_EVEN = _PyTime.ROUND_HALF_EVEN - for obj, seconds, rnd in ( - (-1.9, -2, FLOOR), - (-1.9, -1, CEILING), - (-1.9, -2, HALF_EVEN), - - (1.9, 1, FLOOR), - (1.9, 2, CEILING), - (1.9, 2, HALF_EVEN), - - # half even - (-1.5, -2, HALF_EVEN), - (-0.9, -1, HALF_EVEN), - (-0.5, 0, HALF_EVEN), - ( 0.5, 0, HALF_EVEN), - ( 0.9, 1, HALF_EVEN), - ( 1.5, 2, HALF_EVEN), - ): - with self.subTest(obj=obj, round=rnd, seconds=seconds): - self.assertEqual(pytime_object_to_time_t(obj, rnd), seconds) - - # Test OverflowError - rnd = _PyTime.ROUND_FLOOR - for invalid in self.invalid_values: - self.assertRaises(OverflowError, - pytime_object_to_time_t, invalid, rnd) - - @support.cpython_only - def test_object_to_timespec(self): - from _testcapi import pytime_object_to_timespec - - # Conversion giving the same result for all rounding methods - for rnd in ALL_ROUNDING_METHODS: - for obj, timespec in ( - # int - (0, (0, 0)), - (-1, (-1, 0)), - - # float - (-1/2**7, (-1, 992187500)), - (-1.0, (-1, 0)), - (-1e-9, (-1, 999999999)), - (1e-9, (0, 1)), - (1.0, (1, 0)), - ): - with self.subTest(obj=obj, round=rnd, timespec=timespec): - self.assertEqual(pytime_object_to_timespec(obj, rnd), timespec) - - # Conversion giving different results depending on the rounding method - FLOOR = _PyTime.ROUND_FLOOR - CEILING = _PyTime.ROUND_CEILING - HALF_EVEN = _PyTime.ROUND_HALF_EVEN - for obj, timespec, rnd in ( - # Round towards minus infinity (-inf) - (-1e-10, (0, 0), CEILING), - (-1e-10, (-1, 999999999), FLOOR), - (-1e-10, (0, 0), HALF_EVEN), - (1e-10, (0, 0), FLOOR), - (1e-10, (0, 1), CEILING), - (1e-10, (0, 0), HALF_EVEN), - - (0.9999999999, (0, 999999999), FLOOR), - (0.9999999999, (1, 0), CEILING), - - (1.1234567890, (1, 123456789), FLOOR), - (1.1234567899, (1, 123456789), FLOOR), - (-1.1234567890, (-2, 876543210), FLOOR), - (-1.1234567891, (-2, 876543210), FLOOR), - # Round towards infinity (+inf) - (1.1234567890, (1, 123456790), CEILING), - (1.1234567899, (1, 123456790), CEILING), - (-1.1234567890, (-2, 876543211), CEILING), - (-1.1234567891, (-2, 876543211), CEILING), - - # half even - (-1.5e-9, (-1, 999999998), HALF_EVEN), - (-0.9e-9, (-1, 999999999), HALF_EVEN), - (-0.5e-9, (0, 0), HALF_EVEN), - (0.5e-9, (0, 0), HALF_EVEN), - (0.9e-9, (0, 1), HALF_EVEN), - (1.5e-9, (0, 2), HALF_EVEN), - ): - with self.subTest(obj=obj, round=rnd, timespec=timespec): - self.assertEqual(pytime_object_to_timespec(obj, rnd), timespec) - - # Test OverflowError - rnd = _PyTime.ROUND_FLOOR - for invalid in self.invalid_values: - self.assertRaises(OverflowError, - pytime_object_to_timespec, invalid, rnd) - @unittest.skipUnless(time._STRUCT_TM_ITEMS == 11, "needs tm_zone support") def test_localtime_timezone(self): @@ -784,476 +673,259 @@ self.assertIs(lt.tm_zone, None) - at unittest.skipUnless(_testcapi is not None, - 'need the _testcapi module') -class TestPyTime_t(unittest.TestCase): + at unittest.skipIf(_testcapi is None, 'need the _testcapi module') +class CPyTimeTestCase: """ - Test the _PyTime_t API. + Base class to test the C _PyTime_t API. """ + OVERFLOW_SECONDS = None + + def _rounding_values(self, use_float): + "Build timestamps used to test rounding." + + units = [1, US_TO_NS, MS_TO_NS, SEC_TO_NS] + if use_float: + # picoseconds are only tested to pytime_converter accepting floats + units.append(1e-3) + + values = ( + # small values + 1, 2, 5, 7, 123, 456, 1234, + # 10^k - 1 + 9, + 99, + 999, + 9999, + 99999, + 999999, + # test half even rounding near 0.5, 1.5, 2.5, 3.5, 4.5 + 499, 500, 501, + 1499, 1500, 1501, + 2500, + 3500, + 4500, + ) + + ns_timestamps = [0] + for unit in units: + for value in values: + ns = value * unit + ns_timestamps.extend((-ns, ns)) + for pow2 in (0, 5, 10, 15, 22, 23, 24, 30, 33): + ns = (2 ** pow2) * SEC_TO_NS + ns_timestamps.extend(( + -ns-1, -ns, -ns+1, + ns-1, ns, ns+1 + )) + for seconds in (_testcapi.INT_MIN, _testcapi.INT_MAX): + ns_timestamps.append(seconds * SEC_TO_NS) + if use_float: + # numbers with an extract representation in IEEE 754 (base 2) + for pow2 in (3, 7, 10, 15): + ns = 2.0 ** (-pow2) + ns_timestamps.extend((-ns, ns)) + + # seconds close to _PyTime_t type limit + ns = (2 ** 63 // SEC_TO_NS) * SEC_TO_NS + ns_timestamps.extend((-ns, ns)) + + return ns_timestamps + + def _check_rounding(self, pytime_converter, expected_func, + use_float, unit_to_sec, value_filter=None): + + def convert_values(ns_timestamps): + if use_float: + unit_to_ns = SEC_TO_NS / float(unit_to_sec) + values = [ns / unit_to_ns for ns in ns_timestamps] + else: + unit_to_ns = SEC_TO_NS // unit_to_sec + values = [ns // unit_to_ns for ns in ns_timestamps] + + if value_filter: + values = filter(value_filter, values) + + # remove duplicates and sort + return sorted(set(values)) + + # test rounding + ns_timestamps = self._rounding_values(use_float) + valid_values = convert_values(ns_timestamps) + for time_rnd, decimal_rnd in ROUNDING_MODES : + context = decimal.getcontext() + context.rounding = decimal_rnd + + for value in valid_values: + expected = expected_func(value) + self.assertEqual(pytime_converter(value, time_rnd), + expected, + {'value': value, 'rounding': decimal_rnd}) + + # test overflow + ns = self.OVERFLOW_SECONDS * SEC_TO_NS + ns_timestamps = (-ns, ns) + overflow_values = convert_values(ns_timestamps) + for time_rnd, _ in ROUNDING_MODES : + for value in overflow_values: + with self.assertRaises(OverflowError): + pytime_converter(value, time_rnd) + + def check_int_rounding(self, pytime_converter, expected_func, unit_to_sec=1, + value_filter=None): + self._check_rounding(pytime_converter, expected_func, + False, unit_to_sec, value_filter) + + def check_float_rounding(self, pytime_converter, expected_func, unit_to_sec=1): + self._check_rounding(pytime_converter, expected_func, + True, unit_to_sec) + + def decimal_round(self, x): + d = decimal.Decimal(x) + d = d.quantize(1) + return int(d) + + +class TestCPyTime(CPyTimeTestCase, unittest.TestCase): + """ + Test the C _PyTime_t API. + """ + # _PyTime_t is a 64-bit signed integer + OVERFLOW_SECONDS = math.ceil((2**63 + 1) / SEC_TO_NS) + def test_FromSeconds(self): from _testcapi import PyTime_FromSeconds - for seconds in (0, 3, -456, _testcapi.INT_MAX, _testcapi.INT_MIN): - with self.subTest(seconds=seconds): - self.assertEqual(PyTime_FromSeconds(seconds), - seconds * SEC_TO_NS) + + # PyTime_FromSeconds() expects a C int, reject values out of range + def c_int_filter(secs): + return (_testcapi.INT_MIN <= secs <= _testcapi.INT_MAX) + + self.check_int_rounding(lambda secs, rnd: PyTime_FromSeconds(secs), + lambda secs: secs * SEC_TO_NS, + value_filter=c_int_filter) def test_FromSecondsObject(self): from _testcapi import PyTime_FromSecondsObject - # Conversion giving the same result for all rounding methods - for rnd in ALL_ROUNDING_METHODS: - for obj, ts in ( - # integers - (0, 0), - (1, SEC_TO_NS), - (-3, -3 * SEC_TO_NS), + self.check_int_rounding( + PyTime_FromSecondsObject, + lambda secs: secs * SEC_TO_NS) - # float: subseconds - (0.0, 0), - (1e-9, 1), - (1e-6, 10 ** 3), - (1e-3, 10 ** 6), - - # float: seconds - (2.0, 2 * SEC_TO_NS), - (123.0, 123 * SEC_TO_NS), - (-7.0, -7 * SEC_TO_NS), - - # nanosecond are kept for value <= 2^23 seconds, - (2**22 - 1e-9, 4194303999999999), - (2**22, 4194304000000000), - (2**22 + 1e-9, 4194304000000001), - (2**23 - 1e-9, 8388607999999999), - (2**23, 8388608000000000), - - # start loosing precision for value > 2^23 seconds - (2**23 + 1e-9, 8388608000000002), - - # nanoseconds are lost for value > 2^23 seconds - (2**24 - 1e-9, 16777215999999998), - (2**24, 16777216000000000), - (2**24 + 1e-9, 16777216000000000), - (2**25 - 1e-9, 33554432000000000), - (2**25 , 33554432000000000), - (2**25 + 1e-9, 33554432000000000), - - # close to 2^63 nanoseconds (_PyTime_t limit) - (9223372036, 9223372036 * SEC_TO_NS), - (9223372036.0, 9223372036 * SEC_TO_NS), - (-9223372036, -9223372036 * SEC_TO_NS), - (-9223372036.0, -9223372036 * SEC_TO_NS), - ): - with self.subTest(obj=obj, round=rnd, timestamp=ts): - self.assertEqual(PyTime_FromSecondsObject(obj, rnd), ts) - - with self.subTest(round=rnd): - with self.assertRaises(OverflowError): - PyTime_FromSecondsObject(9223372037, rnd) - PyTime_FromSecondsObject(9223372037.0, rnd) - PyTime_FromSecondsObject(-9223372037, rnd) - PyTime_FromSecondsObject(-9223372037.0, rnd) - - # Conversion giving different results depending on the rounding method - FLOOR = _PyTime.ROUND_FLOOR - CEILING = _PyTime.ROUND_CEILING - HALF_EVEN = _PyTime.ROUND_HALF_EVEN - for obj, ts, rnd in ( - # close to zero - ( 1e-10, 0, FLOOR), - ( 1e-10, 1, CEILING), - ( 1e-10, 0, HALF_EVEN), - (-1e-10, -1, FLOOR), - (-1e-10, 0, CEILING), - (-1e-10, 0, HALF_EVEN), - - # test rounding of the last nanosecond - ( 1.1234567899, 1123456789, FLOOR), - ( 1.1234567899, 1123456790, CEILING), - ( 1.1234567899, 1123456790, HALF_EVEN), - (-1.1234567899, -1123456790, FLOOR), - (-1.1234567899, -1123456789, CEILING), - (-1.1234567899, -1123456790, HALF_EVEN), - - # close to 1 second - ( 0.9999999999, 999999999, FLOOR), - ( 0.9999999999, 1000000000, CEILING), - ( 0.9999999999, 1000000000, HALF_EVEN), - (-0.9999999999, -1000000000, FLOOR), - (-0.9999999999, -999999999, CEILING), - (-0.9999999999, -1000000000, HALF_EVEN), - ): - with self.subTest(obj=obj, round=rnd, timestamp=ts): - self.assertEqual(PyTime_FromSecondsObject(obj, rnd), ts) + self.check_float_rounding( + PyTime_FromSecondsObject, + lambda ns: self.decimal_round(ns * SEC_TO_NS)) def test_AsSecondsDouble(self): from _testcapi import PyTime_AsSecondsDouble - for nanoseconds, seconds in ( - # near 1 nanosecond - ( 0, 0.0), - ( 1, 1e-9), - (-1, -1e-9), + def float_converter(ns): + if abs(ns) % SEC_TO_NS == 0: + return float(ns // SEC_TO_NS) + else: + return float(ns) / SEC_TO_NS - # near 1 second - (SEC_TO_NS + 1, 1.0 + 1e-9), - (SEC_TO_NS, 1.0), - (SEC_TO_NS - 1, 1.0 - 1e-9), + self.check_int_rounding(lambda ns, rnd: PyTime_AsSecondsDouble(ns), + float_converter, + NS_TO_SEC) - # a few seconds - (123 * SEC_TO_NS, 123.0), - (-567 * SEC_TO_NS, -567.0), + def create_decimal_converter(self, denominator): + denom = decimal.Decimal(denominator) - # nanosecond are kept for value <= 2^23 seconds - (4194303999999999, 2**22 - 1e-9), - (4194304000000000, 2**22), - (4194304000000001, 2**22 + 1e-9), + def converter(value): + d = decimal.Decimal(value) / denom + return self.decimal_round(d) - # start loosing precision for value > 2^23 seconds - (8388608000000002, 2**23 + 1e-9), + return converter - # nanoseconds are lost for value > 2^23 seconds - (16777215999999998, 2**24 - 1e-9), - (16777215999999999, 2**24 - 1e-9), - (16777216000000000, 2**24 ), - (16777216000000001, 2**24 ), - (16777216000000002, 2**24 + 2e-9), + def test_AsTimeval(self): + from _testcapi import PyTime_AsTimeval - (33554432000000000, 2**25 ), - (33554432000000002, 2**25 ), - (33554432000000004, 2**25 + 4e-9), + us_converter = self.create_decimal_converter(US_TO_NS) - # close to 2^63 nanoseconds (_PyTime_t limit) - (9223372036 * SEC_TO_NS, 9223372036.0), - (-9223372036 * SEC_TO_NS, -9223372036.0), - ): - with self.subTest(nanoseconds=nanoseconds, seconds=seconds): - self.assertEqual(PyTime_AsSecondsDouble(nanoseconds), - seconds) + def timeval_converter(ns): + us = us_converter(ns) + return divmod(us, SEC_TO_US) - def test_timeval(self): - from _testcapi import PyTime_AsTimeval - for rnd in ALL_ROUNDING_METHODS: - for ns, tv in ( - # microseconds - (0, (0, 0)), - (1000, (0, 1)), - (-1000, (-1, 999999)), - - # seconds - (2 * SEC_TO_NS, (2, 0)), - (-3 * SEC_TO_NS, (-3, 0)), - ): - with self.subTest(nanoseconds=ns, timeval=tv, round=rnd): - self.assertEqual(PyTime_AsTimeval(ns, rnd), tv) - - FLOOR = _PyTime.ROUND_FLOOR - CEILING = _PyTime.ROUND_CEILING - HALF_EVEN = _PyTime.ROUND_HALF_EVEN - for ns, tv, rnd in ( - # nanoseconds - (1, (0, 0), FLOOR), - (1, (0, 1), CEILING), - (1, (0, 0), HALF_EVEN), - (-1, (-1, 999999), FLOOR), - (-1, (0, 0), CEILING), - (-1, (0, 0), HALF_EVEN), - - # half even - (-1500, (-1, 999998), HALF_EVEN), - (-999, (-1, 999999), HALF_EVEN), - (-500, (0, 0), HALF_EVEN), - (500, (0, 0), HALF_EVEN), - (999, (0, 1), HALF_EVEN), - (1500, (0, 2), HALF_EVEN), - ): - with self.subTest(nanoseconds=ns, timeval=tv, round=rnd): - self.assertEqual(PyTime_AsTimeval(ns, rnd), tv) + self.check_int_rounding(PyTime_AsTimeval, + timeval_converter, + NS_TO_SEC) @unittest.skipUnless(hasattr(_testcapi, 'PyTime_AsTimespec'), 'need _testcapi.PyTime_AsTimespec') - def test_timespec(self): + def test_AsTimespec(self): from _testcapi import PyTime_AsTimespec - for ns, ts in ( - # nanoseconds - (0, (0, 0)), - (1, (0, 1)), - (-1, (-1, 999999999)), - # seconds - (2 * SEC_TO_NS, (2, 0)), - (-3 * SEC_TO_NS, (-3, 0)), + def timespec_converter(ns): + return divmod(ns, SEC_TO_NS) - # seconds + nanoseconds - (1234567890, (1, 234567890)), - (-1234567890, (-2, 765432110)), - ): - with self.subTest(nanoseconds=ns, timespec=ts): - self.assertEqual(PyTime_AsTimespec(ns), ts) + self.check_int_rounding(lambda ns, rnd: PyTime_AsTimespec(ns), + timespec_converter, + NS_TO_SEC) - def test_milliseconds(self): + def test_AsMilliseconds(self): from _testcapi import PyTime_AsMilliseconds - for rnd in ALL_ROUNDING_METHODS: - for ns, tv in ( - # milliseconds - (1 * MS_TO_NS, 1), - (-2 * MS_TO_NS, -2), - # seconds - (2 * SEC_TO_NS, 2000), - (-3 * SEC_TO_NS, -3000), - ): - with self.subTest(nanoseconds=ns, timeval=tv, round=rnd): - self.assertEqual(PyTime_AsMilliseconds(ns, rnd), tv) + self.check_int_rounding(PyTime_AsMilliseconds, + self.create_decimal_converter(MS_TO_NS), + NS_TO_SEC) - FLOOR = _PyTime.ROUND_FLOOR - CEILING = _PyTime.ROUND_CEILING - HALF_EVEN = _PyTime.ROUND_HALF_EVEN - for ns, ms, rnd in ( - # nanoseconds - (1, 0, FLOOR), - (1, 1, CEILING), - (1, 0, HALF_EVEN), - (-1, -1, FLOOR), - (-1, 0, CEILING), - (-1, 0, HALF_EVEN), + def test_AsMicroseconds(self): + from _testcapi import PyTime_AsMicroseconds - # seconds + nanoseconds - (1234 * MS_TO_NS + 1, 1234, FLOOR), - (1234 * MS_TO_NS + 1, 1235, CEILING), - (1234 * MS_TO_NS + 1, 1234, HALF_EVEN), - (-1234 * MS_TO_NS - 1, -1235, FLOOR), - (-1234 * MS_TO_NS - 1, -1234, CEILING), - (-1234 * MS_TO_NS - 1, -1234, HALF_EVEN), + self.check_int_rounding(PyTime_AsMicroseconds, + self.create_decimal_converter(US_TO_NS), + NS_TO_SEC) - # half up - (-1500000, -2, HALF_EVEN), - (-999999, -1, HALF_EVEN), - (-500000, 0, HALF_EVEN), - (500000, 0, HALF_EVEN), - (999999, 1, HALF_EVEN), - (1500000, 2, HALF_EVEN), - ): - with self.subTest(nanoseconds=ns, milliseconds=ms, round=rnd): - self.assertEqual(PyTime_AsMilliseconds(ns, rnd), ms) - def test_microseconds(self): - from _testcapi import PyTime_AsMicroseconds - for rnd in ALL_ROUNDING_METHODS: - for ns, tv in ( - # microseconds - (1 * US_TO_NS, 1), - (-2 * US_TO_NS, -2), +class TestOldPyTime(CPyTimeTestCase, unittest.TestCase): + """ + Test the old C _PyTime_t API: _PyTime_ObjectToXXX() functions. + """ - # milliseconds - (1 * MS_TO_NS, 1000), - (-2 * MS_TO_NS, -2000), + # time_t is a 32-bit or 64-bit signed integer + OVERFLOW_SECONDS = 2 ** 64 - # seconds - (2 * SEC_TO_NS, 2000000), - (-3 * SEC_TO_NS, -3000000), - ): - with self.subTest(nanoseconds=ns, timeval=tv, round=rnd): - self.assertEqual(PyTime_AsMicroseconds(ns, rnd), tv) - - FLOOR = _PyTime.ROUND_FLOOR - CEILING = _PyTime.ROUND_CEILING - HALF_EVEN = _PyTime.ROUND_HALF_EVEN - for ns, ms, rnd in ( - # nanoseconds - (1, 0, FLOOR), - (1, 1, CEILING), - (1, 0, HALF_EVEN), - (-1, -1, FLOOR), - (-1, 0, CEILING), - (-1, 0, HALF_EVEN), - - # seconds + nanoseconds - (1234 * US_TO_NS + 1, 1234, FLOOR), - (1234 * US_TO_NS + 1, 1235, CEILING), - (1234 * US_TO_NS + 1, 1234, HALF_EVEN), - (-1234 * US_TO_NS - 1, -1235, FLOOR), - (-1234 * US_TO_NS - 1, -1234, CEILING), - (-1234 * US_TO_NS - 1, -1234, HALF_EVEN), - - # half up - (-1500, -2, HALF_EVEN), - (-999, -1, HALF_EVEN), - (-500, 0, HALF_EVEN), - (500, 0, HALF_EVEN), - (999, 1, HALF_EVEN), - (1500, 2, HALF_EVEN), - ): - with self.subTest(nanoseconds=ns, milliseconds=ms, round=rnd): - self.assertEqual(PyTime_AsMicroseconds(ns, rnd), ms) - - - at unittest.skipUnless(_testcapi is not None, - 'need the _testcapi module') -class TestOldPyTime(unittest.TestCase): - """ - Test the old _PyTime_t API: _PyTime_ObjectToXXX() functions. - """ - def setUp(self): - self.invalid_values = ( - -(2 ** 100), 2 ** 100, - -(2.0 ** 100.0), 2.0 ** 100.0, - ) - - @support.cpython_only - def test_time_t(self): + def test_object_to_time_t(self): from _testcapi import pytime_object_to_time_t - # Conversion giving the same result for all rounding methods - for rnd in ALL_ROUNDING_METHODS: - for obj, time_t in ( - # int - (0, 0), - (-1, -1), + self.check_int_rounding(pytime_object_to_time_t, + lambda secs: secs) - # float - (1.0, 1), - (-1.0, -1), - ): - self.assertEqual(pytime_object_to_time_t(obj, rnd), time_t) + self.check_float_rounding(pytime_object_to_time_t, + self.decimal_round) - - # Conversion giving different results depending on the rounding method - FLOOR = _PyTime.ROUND_FLOOR - CEILING = _PyTime.ROUND_CEILING - HALF_EVEN = _PyTime.ROUND_HALF_EVEN - for obj, time_t, rnd in ( - (-1.9, -2, FLOOR), - (-1.9, -2, HALF_EVEN), - (-1.9, -1, CEILING), - - (1.9, 1, FLOOR), - (1.9, 2, HALF_EVEN), - (1.9, 2, CEILING), - - # half even - (-1.5, -2, HALF_EVEN), - (-0.9, -1, HALF_EVEN), - (-0.5, 0, HALF_EVEN), - ( 0.5, 0, HALF_EVEN), - ( 0.9, 1, HALF_EVEN), - ( 1.5, 2, HALF_EVEN), - ): - self.assertEqual(pytime_object_to_time_t(obj, rnd), time_t) - - # Test OverflowError - rnd = _PyTime.ROUND_FLOOR - for invalid in self.invalid_values: - self.assertRaises(OverflowError, - pytime_object_to_time_t, invalid, rnd) + def create_converter(self, sec_to_unit): + def converter(secs): + floatpart, intpart = math.modf(secs) + intpart = int(intpart) + floatpart *= sec_to_unit + floatpart = self.decimal_round(floatpart) + if floatpart < 0: + floatpart += sec_to_unit + intpart -= 1 + elif floatpart >= sec_to_unit: + floatpart -= sec_to_unit + intpart += 1 + return (intpart, floatpart) + return converter def test_object_to_timeval(self): from _testcapi import pytime_object_to_timeval - # Conversion giving the same result for all rounding methods - for rnd in ALL_ROUNDING_METHODS: - for obj, timeval in ( - # int - (0, (0, 0)), - (-1, (-1, 0)), + self.check_int_rounding(pytime_object_to_timeval, + lambda secs: (secs, 0)) - # float - (-1.0, (-1, 0)), - (1/2**6, (0, 15625)), - (-1/2**6, (-1, 984375)), - (-1e-6, (-1, 999999)), - (1e-6, (0, 1)), - ): - with self.subTest(obj=obj, round=rnd, timeval=timeval): - self.assertEqual(pytime_object_to_timeval(obj, rnd), - timeval) + self.check_float_rounding(pytime_object_to_timeval, + self.create_converter(SEC_TO_US)) - # Conversion giving different results depending on the rounding method - FLOOR = _PyTime.ROUND_FLOOR - CEILING = _PyTime.ROUND_CEILING - HALF_EVEN = _PyTime.ROUND_HALF_EVEN - for obj, timeval, rnd in ( - (-1e-7, (-1, 999999), FLOOR), - (-1e-7, (0, 0), CEILING), - (-1e-7, (0, 0), HALF_EVEN), - - (1e-7, (0, 0), FLOOR), - (1e-7, (0, 1), CEILING), - (1e-7, (0, 0), HALF_EVEN), - - (0.9999999, (0, 999999), FLOOR), - (0.9999999, (1, 0), CEILING), - (0.9999999, (1, 0), HALF_EVEN), - - # half even - (-1.5e-6, (-1, 999998), HALF_EVEN), - (-0.9e-6, (-1, 999999), HALF_EVEN), - (-0.5e-6, (0, 0), HALF_EVEN), - (0.5e-6, (0, 0), HALF_EVEN), - (0.9e-6, (0, 1), HALF_EVEN), - (1.5e-6, (0, 2), HALF_EVEN), - ): - with self.subTest(obj=obj, round=rnd, timeval=timeval): - self.assertEqual(pytime_object_to_timeval(obj, rnd), timeval) - - rnd = _PyTime.ROUND_FLOOR - for invalid in self.invalid_values: - self.assertRaises(OverflowError, - pytime_object_to_timeval, invalid, rnd) - - @support.cpython_only - def test_timespec(self): + def test_object_to_timespec(self): from _testcapi import pytime_object_to_timespec - # Conversion giving the same result for all rounding methods - for rnd in ALL_ROUNDING_METHODS: - for obj, timespec in ( - # int - (0, (0, 0)), - (-1, (-1, 0)), + self.check_int_rounding(pytime_object_to_timespec, + lambda secs: (secs, 0)) - # float - (-1.0, (-1, 0)), - (-1e-9, (-1, 999999999)), - (1e-9, (0, 1)), - (-1/2**9, (-1, 998046875)), - ): - with self.subTest(obj=obj, round=rnd, timespec=timespec): - self.assertEqual(pytime_object_to_timespec(obj, rnd), - timespec) + self.check_float_rounding(pytime_object_to_timespec, + self.create_converter(SEC_TO_NS)) - # Conversion giving different results depending on the rounding method - FLOOR = _PyTime.ROUND_FLOOR - CEILING = _PyTime.ROUND_CEILING - HALF_EVEN = _PyTime.ROUND_HALF_EVEN - for obj, timespec, rnd in ( - (-1e-10, (-1, 999999999), FLOOR), - (-1e-10, (0, 0), CEILING), - (-1e-10, (0, 0), HALF_EVEN), - - (1e-10, (0, 0), FLOOR), - (1e-10, (0, 1), CEILING), - (1e-10, (0, 0), HALF_EVEN), - - (0.9999999999, (0, 999999999), FLOOR), - (0.9999999999, (1, 0), CEILING), - (0.9999999999, (1, 0), HALF_EVEN), - - # half even - (-1.5e-9, (-1, 999999998), HALF_EVEN), - (-0.9e-9, (-1, 999999999), HALF_EVEN), - (-0.5e-9, (0, 0), HALF_EVEN), - (0.5e-9, (0, 0), HALF_EVEN), - (0.9e-9, (0, 1), HALF_EVEN), - (1.5e-9, (0, 2), HALF_EVEN), - ): - with self.subTest(obj=obj, round=rnd, timespec=timespec): - self.assertEqual(pytime_object_to_timespec(obj, rnd), timespec) - - # Test OverflowError - rnd = FLOOR - for invalid in self.invalid_values: - self.assertRaises(OverflowError, - pytime_object_to_timespec, invalid, rnd) if __name__ == "__main__": diff --git a/Python/pytime.c b/Python/pytime.c --- a/Python/pytime.c +++ b/Python/pytime.c @@ -324,12 +324,16 @@ double _PyTime_AsSecondsDouble(_PyTime_t t) { - _PyTime_t sec, ns; - /* Divide using integers to avoid rounding issues on the integer part. - 1e-9 cannot be stored exactly in IEEE 64-bit. */ - sec = t / SEC_TO_NS; - ns = t % SEC_TO_NS; - return (double)sec + (double)ns * 1e-9; + if (t % SEC_TO_NS == 0) { + _PyTime_t secs; + /* Divide using integers to avoid rounding issues on the integer part. + 1e-9 cannot be stored exactly in IEEE 64-bit. */ + secs = t / SEC_TO_NS; + return (double)secs; + } + else { + return (double)t / 1e9; + } } PyObject * -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 10 04:33:59 2015 From: python-checkins at python.org (steve.dower) Date: Thu, 10 Sep 2015 02:33:59 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Fixes_handling_of_read-only_files_when_creating_zip_pack?= =?utf-8?q?age=2E?= Message-ID: <20150910023359.66848.20134@psf.io> https://hg.python.org/cpython/rev/a7ac79f066fa changeset: 97840:a7ac79f066fa parent: 97838:593f75ccfe11 parent: 97839:7c72615070af user: Steve Dower date: Wed Sep 09 19:33:06 2015 -0700 summary: Fixes handling of read-only files when creating zip package. files: Tools/msi/make_zip.py | 10 ++++++++-- 1 files changed, 8 insertions(+), 2 deletions(-) diff --git a/Tools/msi/make_zip.py b/Tools/msi/make_zip.py --- a/Tools/msi/make_zip.py +++ b/Tools/msi/make_zip.py @@ -3,6 +3,7 @@ import re import sys import shutil +import stat import os import tempfile @@ -101,11 +102,16 @@ else: for s, rel in rel_sources: + dest = target / rel try: - (target / rel).parent.mkdir(parents=True) + dest.parent.mkdir(parents=True) except FileExistsError: pass - shutil.copy(str(s), str(target / rel)) + if dest.is_file(): + dest.chmod(stat.S_IWRITE) + shutil.copy(str(s), str(dest)) + if dest.is_file(): + dest.chmod(stat.S_IWRITE) count += 1 return count -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 10 04:33:59 2015 From: python-checkins at python.org (steve.dower) Date: Thu, 10 Sep 2015 02:33:59 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_Fixes_handling?= =?utf-8?q?_of_read-only_files_when_creating_zip_package=2E?= Message-ID: <20150910023358.27713.901@psf.io> https://hg.python.org/cpython/rev/7c72615070af changeset: 97839:7c72615070af branch: 3.5 parent: 97834:df91c1879e56 user: Steve Dower date: Wed Sep 09 19:32:45 2015 -0700 summary: Fixes handling of read-only files when creating zip package. files: Tools/msi/make_zip.py | 10 ++++++++-- 1 files changed, 8 insertions(+), 2 deletions(-) diff --git a/Tools/msi/make_zip.py b/Tools/msi/make_zip.py --- a/Tools/msi/make_zip.py +++ b/Tools/msi/make_zip.py @@ -3,6 +3,7 @@ import re import sys import shutil +import stat import os import tempfile @@ -101,11 +102,16 @@ else: for s, rel in rel_sources: + dest = target / rel try: - (target / rel).parent.mkdir(parents=True) + dest.parent.mkdir(parents=True) except FileExistsError: pass - shutil.copy(str(s), str(target / rel)) + if dest.is_file(): + dest.chmod(stat.S_IWRITE) + shutil.copy(str(s), str(dest)) + if dest.is_file(): + dest.chmod(stat.S_IWRITE) count += 1 return count -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 10 04:37:48 2015 From: python-checkins at python.org (guido.van.rossum) Date: Thu, 10 Sep 2015 02:37:48 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy41IC0+IDMuNSk6?= =?utf-8?q?_Merge_typing_doc_updates_from_larry=27s_branch=2E?= Message-ID: <20150910023748.68863.19852@psf.io> https://hg.python.org/cpython/rev/f799482ebd8c changeset: 97844:f799482ebd8c branch: 3.5 parent: 97839:7c72615070af parent: 97843:a9153eca5c72 user: Guido van Rossum date: Wed Sep 09 19:34:36 2015 -0700 summary: Merge typing doc updates from larry's branch. files: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 10 04:37:48 2015 From: python-checkins at python.org (guido.van.rossum) Date: Thu, 10 Sep 2015 02:37:48 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_Update_typing_?= =?utf-8?q?docs_based_on_a_patch_by_Ivan_Levkivskyi_=28but_much_rewritten_?= =?utf-8?q?by?= Message-ID: <20150910023748.66874.53029@psf.io> https://hg.python.org/cpython/rev/e361130782d4 changeset: 97842:e361130782d4 branch: 3.5 user: Guido van Rossum date: Wed Sep 09 11:44:39 2015 -0700 summary: Update typing docs based on a patch by Ivan Levkivskyi (but much rewritten by me). files: Doc/library/typing.rst | 59 ++++++++++++++++++++++++++++- 1 files changed, 56 insertions(+), 3 deletions(-) diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -355,31 +355,84 @@ .. class:: MutableSequence(Sequence[T]) + A generic version of :class:`collections.abc.MutableSequence`. + .. class:: ByteString(Sequence[int]) + A generic version of :class:`collections.abc.ByteString`. + + This type represents the types :class:`bytes`, :class:`bytearray`, + and :class:`memoryview`. + + As a shorthand for this type, :class:`bytes` can be used to + annotate arguments of any of the types mentioned above. + .. class:: List(list, MutableSequence[T]) -.. class:: Set(set, MutableSet[T]) + Generic version of :class:`list`. + Useful for annotating return types. To annotate arguments it is preferred + to use abstract collection types such as :class:`Mapping`, :class:`Sequence`, + or :class:`AbstractSet`. + + This type may be used as follows:: + + T = TypeVar('T', int, float) + + def vec2(x: T, y: T) -> List[T]: + return [x, y] + + def slice__to_4(vector: Sequence[T]) -> List[T]: + return vector[0:4] + +.. class:: AbstractSet(set, MutableSet[T]) + + A generic version of :class:`collections.abc.Set`. .. class:: MappingView(Sized, Iterable[T_co]) + A generic version of :class:`collections.abc.MappingView`. + .. class:: KeysView(MappingView[KT_co], AbstractSet[KT_co]) + A generic version of :class:`collections.abc.KeysView`. + .. class:: ItemsView(MappingView, Generic[KT_co, VT_co]) + A generic version of :class:`collections.abc.ItemsView`. + .. class:: ValuesView(MappingView[VT_co]) + A generic version of :class:`collections.abc.ValuesView`. + .. class:: Dict(dict, MutableMapping[KT, VT]) + A generic version of :class:`dict`. + The usage of this type is as follows:: + + def get_position_in_index(word_list: Dict[str, int], word: str) -> int: + return word_list[word] + .. class:: Generator(Iterator[T_co], Generic[T_co, T_contra, V_co]) .. class:: io - Wrapper namespace for IO generic classes. + Wrapper namespace for I/O stream types. + + This defines the generic type ``IO[AnyStr]`` and aliases ``TextIO`` + and ``BinaryIO`` for respectively ``IO[str]`` and ``IO[bytes]``. + These representing the types of I/O streams such as returned by + :func:`open`. .. class:: re - Wrapper namespace for re type classes. + Wrapper namespace for regular expression matching types. + + This defines the type aliases ``Pattern`` and ``Match`` which + correspond to the return types from :func:`re.compile` and + :func:`re.match`. These types (and the corresponding functions) + are generic in ``AnyStr`` and can be made specific by writing + ``Pattern[str]``, ``Pattern[bytes]``, ``Match[str]``, or + ``Match[bytes]``. .. function:: NamedTuple(typename, fields) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 10 04:37:48 2015 From: python-checkins at python.org (guido.van.rossum) Date: Thu, 10 Sep 2015 02:37:48 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_Update_typing_?= =?utf-8?q?docs_based_on_a_patch_by_Daniel_Andrade_Groppe=2E?= Message-ID: <20150910023748.11266.58783@psf.io> https://hg.python.org/cpython/rev/a9153eca5c72 changeset: 97843:a9153eca5c72 branch: 3.5 user: Guido van Rossum date: Wed Sep 09 12:01:36 2015 -0700 summary: Update typing docs based on a patch by Daniel Andrade Groppe. files: Doc/library/typing.rst | 67 +++++++++++++++++++++++------ 1 files changed, 53 insertions(+), 14 deletions(-) diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -68,7 +68,7 @@ overrides: Mapping[str, str]) -> None: ... Generics can be parametrized by using a new factory available in typing -called TypeVar. +called :class:`TypeVar`. .. code-block:: python @@ -145,7 +145,7 @@ class Pair(Generic[T, T]): # INVALID ... -You can use multiple inheritance with `Generic`:: +You can use multiple inheritance with :class:`Generic`:: from typing import TypeVar, Generic, Sized @@ -154,6 +154,17 @@ class LinkedList(Sized, Generic[T]): ... +When inheriting from generic classes, some type variables could fixed:: + + from typing import TypeVar, Mapping + + T = TypeVar('T') + + class MyDict(Mapping[str, T]): + ... + +In this case ``MyDict`` has a single parameter, ``T``. + Subclassing a generic class without specifying type parameters assumes :class:`Any` for each position. In the following example, ``MyIterable`` is not generic but implicitly inherits from ``Iterable[Any]``:: @@ -162,7 +173,11 @@ class MyIterable(Iterable): # Same as Iterable[Any] -Generic metaclasses are not supported. +The metaclass used by :class:`Generic` is a subclass of :class:`abc.ABCMeta`. +A generic class can be an ABC by including abstract methods or properties, +and generic classes can also have ABCs as base classes without a metaclass +conflict. Generic metaclasses are not supported. + The :class:`Any` type --------------------- @@ -178,15 +193,6 @@ on it, and a value of type :class:`Any` can be assigned to a variable (or used as a return value) of a more constrained type. -Default argument values ------------------------ - -Use a literal ellipsis ``...`` to declare an argument as having a default value:: - - from typing import AnyStr - - def foo(x: AnyStr, y: AnyStr = ...) -> AnyStr: ... - Classes, functions, and decorators ---------------------------------- @@ -236,7 +242,11 @@ Type variables may be marked covariant or contravariant by passing ``covariant=True`` or ``contravariant=True``. See :pep:`484` for more - details. By default type variables are invariant. + details. By default type variables are invariant. Alternatively, + a type variable may specify an upper bound using ``bound=``. + This means that an actual type substituted (explicitly or implictly) + for the type variable must be a subclass of the boundary type, + see :pep:`484`. .. class:: Union @@ -329,30 +339,59 @@ .. class:: Iterable(Generic[T_co]) + A generic version of the :class:`collections.abc.Iterable`. + .. class:: Iterator(Iterable[T_co]) + A generic version of the :class:`collections.abc.Iterator`. + .. class:: SupportsInt + An ABC with one abstract method `__int__`. + .. class:: SupportsFloat + An ABC with one abstract method `__float__`. + .. class:: SupportsAbs + An ABC with one abstract method `__abs__` that is covariant + in its return type. + .. class:: SupportsRound + An ABC with one abstract method `__round__` + that is covariant in its return type. + .. class:: Reversible + An ABC with one abstract method `__reversed__` returning + an `Iterator[T_co]`. + .. class:: Container(Generic[T_co]) + A generic version of :class:`collections.abc.Container`. + .. class:: AbstractSet(Sized, Iterable[T_co], Container[T_co]) + A generic version of :class:`collections.abc.Set`. + .. class:: MutableSet(AbstractSet[T]) -.. class:: Mapping(Sized, Iterable[KT_co], Container[KT_co], Generic[KT_co, VT_co]) + A generic version of :class:`collections.abc.MutableSet`. + +.. class:: Mapping(Sized, Iterable[KT], Container[KT], Generic[VT_co]) + + A generic version of :class:`collections.abc.Mapping`. .. class:: MutableMapping(Mapping[KT, VT]) + A generic version of :class:`collections.abc.MutableMapping`. + .. class:: Sequence(Sized, Iterable[T_co], Container[T_co]) + A generic version of :class:`collections.abc.Sequence`. + .. class:: MutableSequence(Sequence[T]) A generic version of :class:`collections.abc.MutableSequence`. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 10 04:37:48 2015 From: python-checkins at python.org (guido.van.rossum) Date: Thu, 10 Sep 2015 02:37:48 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_Merge_typing_d?= =?utf-8?q?ocs_cleanup_diff_by_Zach_Ware_from_default_back_into_350_branch?= =?utf-8?q?=2E?= Message-ID: <20150910023748.27683.66144@psf.io> https://hg.python.org/cpython/rev/1a8af7ff1847 changeset: 97841:1a8af7ff1847 branch: 3.5 parent: 97822:2d2c84821f2a user: Guido van Rossum date: Wed Sep 09 11:21:18 2015 -0700 summary: Merge typing docs cleanup diff by Zach Ware from default back into 350 branch. files: Doc/library/typing.rst | 97 +++++++++++++++-------------- 1 files changed, 50 insertions(+), 47 deletions(-) diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -20,8 +20,9 @@ def greeting(name: str) -> str: return 'Hello ' + name -In the function `greeting`, the argument `name` is expected to by of type `str` -and the return type `str`. Subtypes are accepted as arguments. +In the function ``greeting``, the argument ``name`` is expected to by of type +:class:`str` and the return type :class:`str`. Subtypes are accepted as +arguments. Type aliases ------------ @@ -49,8 +50,8 @@ It is possible to declare the return type of a callable without specifying the call signature by substituting a literal ellipsis -for the list of arguments in the type hint: `Callable[..., ReturnType]`. -`None` as a type hint is a special case and is replaced by `type(None)`. +for the list of arguments in the type hint: ``Callable[..., ReturnType]``. +``None`` as a type hint is a special case and is replaced by ``type(None)``. Generics -------- @@ -108,11 +109,12 @@ def log(self, message: str) -> None: self.logger.info('{}: {}'.format(self.name, message)) -`Generic[T]` as a base class defines that the class `LoggedVar` takes a single -type parameter `T` . This also makes `T` valid as a type within the class body. +``Generic[T]`` as a base class defines that the class ``LoggedVar`` takes a +single type parameter ``T`` . This also makes ``T`` valid as a type within the +class body. -The `Generic` base class uses a metaclass that defines `__getitem__` so that -`LoggedVar[t]` is valid as a type:: +The :class:`Generic` base class uses a metaclass that defines +:meth:`__getitem__` so that ``LoggedVar[t]`` is valid as a type:: from typing import Iterable @@ -132,7 +134,7 @@ class StrangePair(Generic[T, S]): ... -Each type variable argument to `Generic` must be distinct. +Each type variable argument to :class:`Generic` must be distinct. This is thus invalid:: from typing import TypeVar, Generic @@ -152,9 +154,9 @@ class LinkedList(Sized, Generic[T]): ... -Subclassing a generic class without specifying type parameters assumes `Any` -for each position. In the following example, `MyIterable` is not generic but -implicitly inherits from `Iterable[Any]`:: +Subclassing a generic class without specifying type parameters assumes +:class:`Any` for each position. In the following example, ``MyIterable`` is +not generic but implicitly inherits from ``Iterable[Any]``:: from typing import Iterable @@ -162,24 +164,24 @@ Generic metaclasses are not supported. -The `Any` type --------------- +The :class:`Any` type +--------------------- -A special kind of type is `Any`. Every type is a subtype of `Any`. -This is also true for the builtin type object. However, to the static type -checker these are completely different. +A special kind of type is :class:`Any`. Every type is a subtype of +:class:`Any`. This is also true for the builtin type object. However, to the +static type checker these are completely different. -When the type of a value is `object`, the type checker will reject almost all -operations on it, and assigning it to a variable (or using it as a return value) -of a more specialized type is a type error. On the other hand, when a value has -type `Any`, the type checker will allow all operations on it, and a value of -type `Any` can be assigned to a variable (or used as a return value) of a more -constrained type. +When the type of a value is :class:`object`, the type checker will reject +almost all operations on it, and assigning it to a variable (or using it as a +return value) of a more specialized type is a type error. On the other hand, +when a value has type :class:`Any`, the type checker will allow all operations +on it, and a value of type :class:`Any` can be assigned to a variable (or used +as a return value) of a more constrained type. Default argument values ----------------------- -Use a literal ellipsis `...` to declare an argument as having a default value:: +Use a literal ellipsis ``...`` to declare an argument as having a default value:: from typing import AnyStr @@ -195,9 +197,10 @@ Special type indicating an unconstrained type. - * Any object is an instance of `Any`. - * Any class is a subclass of `Any`. - * As a special case, `Any` and `object` are subclasses of each other. + * Any object is an instance of :class:`Any`. + * Any class is a subclass of :class:`Any`. + * As a special case, :class:`Any` and :class:`object` are subclasses of + each other. .. class:: TypeVar @@ -224,22 +227,22 @@ return x if len(x) >= len(y) else y The latter example's signature is essentially the overloading - of `(str, str) -> str` and `(bytes, bytes) -> bytes`. Also note - that if the arguments are instances of some subclass of `str`, - the return type is still plain `str`. + of ``(str, str) -> str`` and ``(bytes, bytes) -> bytes``. Also note + that if the arguments are instances of some subclass of :class:`str`, + the return type is still plain :class:`str`. - At runtime, `isinstance(x, T)` will raise `TypeError`. In general, - `isinstance` and `issublass` should not be used with types. + At runtime, ``isinstance(x, T)`` will raise :exc:`TypeError`. In general, + :func:`isinstance` and :func:`issublass` should not be used with types. Type variables may be marked covariant or contravariant by passing - `covariant=True` or `contravariant=True`. See :pep:`484` for more + ``covariant=True`` or ``contravariant=True``. See :pep:`484` for more details. By default type variables are invariant. .. class:: Union - Union type; `Union[X, Y]` means either X or Y. + Union type; ``Union[X, Y]`` means either X or Y. - To define a union, use e.g. `Union[int, str]`. Details: + To define a union, use e.g. ``Union[int, str]``. Details: * The arguments must be types and there must be at least one. @@ -259,37 +262,37 @@ Union[int, str] == Union[str, int] - * If `Any` is present it is the sole survivor, e.g.:: + * If :class:`Any` is present it is the sole survivor, e.g.:: Union[int, Any] == Any * You cannot subclass or instantiate a union. - * You cannot write `Union[X][Y]` + * You cannot write ``Union[X][Y]`` - * You can use `Optional[X]` as a shorthand for `Union[X, None]`. + * You can use ``Optional[X]`` as a shorthand for ``Union[X, None]``. .. class:: Optional Optional type. - `Optional[X]` is equivalent to `Union[X, type(None)]`. + ``Optional[X]`` is equivalent to ``Union[X, type(None)]``. .. class:: Tuple - Tuple type; `Tuple[X, Y]` is the is the type of a tuple of two items + Tuple type; ``Tuple[X, Y]`` is the is the type of a tuple of two items with the first item of type X and the second of type Y. - Example: `Tuple[T1, T2]` is a tuple of two elements corresponding - to type variables T1 and T2. `Tuple[int, float, str]` is a tuple + Example: ``Tuple[T1, T2]`` is a tuple of two elements corresponding + to type variables T1 and T2. ``Tuple[int, float, str]`` is a tuple of an int, a float and a string. To specify a variable-length tuple of homogeneous type, - use literal ellipsis, e.g. `Tuple[int, ...]`. + use literal ellipsis, e.g. ``Tuple[int, ...]``. .. class:: Callable - Callable type; `Callable[[int], str]` is a function of (int) -> str. + Callable type; ``Callable[[int], str]`` is a function of (int) -> str. The subscription syntax must always be used with exactly two values: the argument list and the return type. The argument list @@ -297,9 +300,9 @@ There is no syntax to indicate optional or keyword arguments, such function types are rarely used as callback types. - `Callable[..., ReturnType]` could be used to type hint a callable - taking any number of arguments and returning `ReturnType`. - A plain `Callable` is equivalent to `Callable[..., Any]`. + ``Callable[..., ReturnType]`` could be used to type hint a callable + taking any number of arguments and returning ``ReturnType``. + A plain :class:`Callable` is equivalent to ``Callable[..., Any]``. .. class:: Generic -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 10 04:37:48 2015 From: python-checkins at python.org (guido.van.rossum) Date: Thu, 10 Sep 2015 02:37:48 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge_typing_docs_from_3=2E5_branch=2E?= Message-ID: <20150910023748.101480.48371@psf.io> https://hg.python.org/cpython/rev/cea093952b93 changeset: 97845:cea093952b93 parent: 97840:a7ac79f066fa parent: 97844:f799482ebd8c user: Guido van Rossum date: Wed Sep 09 19:36:31 2015 -0700 summary: Merge typing docs from 3.5 branch. files: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 10 04:40:01 2015 From: python-checkins at python.org (raymond.hettinger) Date: Thu, 10 Sep 2015 02:40:01 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Simply_deque_repeat_by_reu?= =?utf-8?q?sing_code_in_in-line_repeat=2E__Avoid_unnecessary?= Message-ID: <20150910024000.11248.85400@psf.io> https://hg.python.org/cpython/rev/eff4f1e29039 changeset: 97846:eff4f1e29039 user: Raymond Hettinger date: Wed Sep 09 22:39:44 2015 -0400 summary: Simply deque repeat by reusing code in in-line repeat. Avoid unnecessary division. files: Modules/_collectionsmodule.c | 48 +++++++++--------------- 1 files changed, 18 insertions(+), 30 deletions(-) diff --git a/Modules/_collectionsmodule.c b/Modules/_collectionsmodule.c --- a/Modules/_collectionsmodule.c +++ b/Modules/_collectionsmodule.c @@ -539,32 +539,6 @@ static void deque_clear(dequeobject *deque); static PyObject * -deque_repeat(dequeobject *deque, Py_ssize_t n) -{ - dequeobject *new_deque; - PyObject *result; - - /* XXX add a special case for when maxlen is defined */ - if (n < 0) - n = 0; - else if (n > 0 && Py_SIZE(deque) > MAX_DEQUE_LEN / n) - return PyErr_NoMemory(); - - new_deque = (dequeobject *)deque_new(&deque_type, (PyObject *)NULL, (PyObject *)NULL); - new_deque->maxlen = deque->maxlen; - - for ( ; n ; n--) { - result = deque_extend(new_deque, (PyObject *)deque); - if (result == NULL) { - Py_DECREF(new_deque); - return NULL; - } - Py_DECREF(result); - } - return (PyObject *)new_deque; -} - -static PyObject * deque_inplace_repeat(dequeobject *deque, Py_ssize_t n) { Py_ssize_t i, size; @@ -583,10 +557,6 @@ return (PyObject *)deque; } - if (size > MAX_DEQUE_LEN / n) { - return PyErr_NoMemory(); - } - if (size == 1) { /* common case, repeating a single element */ PyObject *item = deque->leftblock->data[deque->leftindex]; @@ -594,6 +564,9 @@ if (deque->maxlen != -1 && n > deque->maxlen) n = deque->maxlen; + if (n > MAX_DEQUE_LEN) + return PyErr_NoMemory(); + for (i = 0 ; i < n-1 ; i++) { rv = deque_append(deque, item); if (rv == NULL) @@ -604,6 +577,10 @@ return (PyObject *)deque; } + if ((size_t)size > MAX_DEQUE_LEN / (size_t)n) { + return PyErr_NoMemory(); + } + seq = PySequence_List((PyObject *)deque); if (seq == NULL) return seq; @@ -621,6 +598,17 @@ return (PyObject *)deque; } +static PyObject * +deque_repeat(dequeobject *deque, Py_ssize_t n) +{ + dequeobject *new_deque; + + new_deque = (dequeobject *)deque_copy((PyObject *) deque); + if (new_deque == NULL) + return NULL; + return deque_inplace_repeat(new_deque, n); +} + /* The rotate() method is part of the public API and is used internally as a primitive for other methods. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 10 04:47:10 2015 From: python-checkins at python.org (yury.selivanov) Date: Thu, 10 Sep 2015 02:47:10 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogd2hhdHNuZXcvMy41?= =?utf-8?q?=3A_Mention_issue_22464?= Message-ID: <20150910024710.68885.33702@psf.io> https://hg.python.org/cpython/rev/6c8e2c6e3a99 changeset: 97847:6c8e2c6e3a99 branch: 3.5 parent: 97844:f799482ebd8c user: Yury Selivanov date: Wed Sep 09 22:46:40 2015 -0400 summary: whatsnew/3.5: Mention issue 22464 (About the only new feature that was worth mentioning in whatsnew without a NEWS entry) files: Doc/whatsnew/3.5.rst | 3 +++ 1 files changed, 3 insertions(+), 0 deletions(-) diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -1355,6 +1355,9 @@ * property() getter calls are up to 25% faster. (Contributed by Joe Jevnik in :issue:`23910`.) +* Instantiation of :class:`fractions.Fraction` is now up to 30% faster. + (Contributed by Stefan Behnel in :issue:`22464`.) + Build and C API Changes ======================= -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 10 04:47:10 2015 From: python-checkins at python.org (yury.selivanov) Date: Thu, 10 Sep 2015 02:47:10 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogTWVyZ2UgMy41?= Message-ID: <20150910024710.66848.59515@psf.io> https://hg.python.org/cpython/rev/18802076b52c changeset: 97848:18802076b52c parent: 97846:eff4f1e29039 parent: 97847:6c8e2c6e3a99 user: Yury Selivanov date: Wed Sep 09 22:46:51 2015 -0400 summary: Merge 3.5 files: Doc/whatsnew/3.5.rst | 3 +++ 1 files changed, 3 insertions(+), 0 deletions(-) diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -1355,6 +1355,9 @@ * property() getter calls are up to 25% faster. (Contributed by Joe Jevnik in :issue:`23910`.) +* Instantiation of :class:`fractions.Fraction` is now up to 30% faster. + (Contributed by Stefan Behnel in :issue:`22464`.) + Build and C API Changes ======================= -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 10 09:37:34 2015 From: python-checkins at python.org (victor.stinner) Date: Thu, 10 Sep 2015 07:37:34 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Fix_test=5Ftime_on_Windows?= Message-ID: <20150910073734.11248.18842@psf.io> https://hg.python.org/cpython/rev/ea7e19bc22c9 changeset: 97849:ea7e19bc22c9 user: Victor Stinner date: Thu Sep 10 09:10:14 2015 +0200 summary: Fix test_time on Windows * Filter values which would overflow on conversion to the C long type (for timeval.tv_sec). * Adjust also the message of OverflowError on PyTime conversions * test_time: add debug information if a timestamp conversion fails files: Lib/test/test_time.py | 33 ++++++++++++++++++++++-------- Python/pytime.c | 28 +++++++------------------ 2 files changed, 32 insertions(+), 29 deletions(-) diff --git a/Lib/test/test_time.py b/Lib/test/test_time.py --- a/Lib/test/test_time.py +++ b/Lib/test/test_time.py @@ -756,10 +756,15 @@ context.rounding = decimal_rnd for value in valid_values: - expected = expected_func(value) - self.assertEqual(pytime_converter(value, time_rnd), + debug_info = {'value': value, 'rounding': decimal_rnd} + try: + result = pytime_converter(value, time_rnd) + expected = expected_func(value) + except Exception as exc: + self.fail("Error on timestamp conversion: %s" % debug_info) + self.assertEqual(result, expected, - {'value': value, 'rounding': decimal_rnd}) + debug_info) # test overflow ns = self.OVERFLOW_SECONDS * SEC_TO_NS @@ -770,14 +775,15 @@ with self.assertRaises(OverflowError): pytime_converter(value, time_rnd) - def check_int_rounding(self, pytime_converter, expected_func, unit_to_sec=1, - value_filter=None): + def check_int_rounding(self, pytime_converter, expected_func, + unit_to_sec=1, value_filter=None): self._check_rounding(pytime_converter, expected_func, False, unit_to_sec, value_filter) - def check_float_rounding(self, pytime_converter, expected_func, unit_to_sec=1): + def check_float_rounding(self, pytime_converter, expected_func, + unit_to_sec=1, value_filter=None): self._check_rounding(pytime_converter, expected_func, - True, unit_to_sec) + True, unit_to_sec, value_filter) def decimal_round(self, x): d = decimal.Decimal(x) @@ -845,9 +851,19 @@ us = us_converter(ns) return divmod(us, SEC_TO_US) + if sys.platform == 'win32': + from _testcapi import LONG_MIN, LONG_MAX + + # On Windows, timeval.tv_sec type is a C long + def seconds_filter(secs): + return LONG_MIN <= secs <= LONG_MAX + else: + seconds_filter = None + self.check_int_rounding(PyTime_AsTimeval, timeval_converter, - NS_TO_SEC) + NS_TO_SEC, + value_filter=seconds_filter) @unittest.skipUnless(hasattr(_testcapi, 'PyTime_AsTimespec'), 'need _testcapi.PyTime_AsTimespec') @@ -927,6 +943,5 @@ self.create_converter(SEC_TO_NS)) - if __name__ == "__main__": unittest.main() diff --git a/Python/pytime.c b/Python/pytime.c --- a/Python/pytime.c +++ b/Python/pytime.c @@ -288,18 +288,21 @@ else { #ifdef HAVE_LONG_LONG PY_LONG_LONG sec; + assert(sizeof(PY_LONG_LONG) <= sizeof(_PyTime_t)); + sec = PyLong_AsLongLong(obj); - assert(sizeof(PY_LONG_LONG) <= sizeof(_PyTime_t)); #else long sec; + assert(sizeof(PY_LONG_LONG) <= sizeof(_PyTime_t)); + sec = PyLong_AsLong(obj); - assert(sizeof(PY_LONG_LONG) <= sizeof(_PyTime_t)); #endif if (sec == -1 && PyErr_Occurred()) { if (PyErr_ExceptionMatches(PyExc_OverflowError)) _PyTime_overflow(); return -1; } + *t = sec * unit_to_ns; if (*t / unit_to_ns != sec) { _PyTime_overflow(); @@ -404,27 +407,12 @@ ns = t % SEC_TO_NS; #ifdef MS_WINDOWS - /* On Windows, timeval.tv_sec is a long (32 bit), - whereas time_t can be 64-bit. */ - assert(sizeof(tv->tv_sec) == sizeof(long)); -#if SIZEOF_TIME_T > SIZEOF_LONG - if (secs > LONG_MAX) { - secs = LONG_MAX; - res = -1; - } - else if (secs < LONG_MIN) { - secs = LONG_MIN; - res = -1; - } -#endif tv->tv_sec = (long)secs; #else - /* On OpenBSD 5.4, timeval.tv_sec is a long. - Example: long is 64-bit, whereas time_t is 32-bit. */ tv->tv_sec = secs; +#endif if ((_PyTime_t)tv->tv_sec != secs) res = -1; -#endif usec = (int)_PyTime_Divide(ns, US_TO_NS, round); if (usec < 0) { @@ -440,7 +428,7 @@ tv->tv_usec = usec; if (res && raise) - _PyTime_overflow(); + error_time_t_overflow(); return res; } @@ -473,7 +461,7 @@ ts->tv_nsec = nsec; if ((_PyTime_t)ts->tv_sec != secs) { - _PyTime_overflow(); + error_time_t_overflow(); return -1; } return 0; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 10 10:03:30 2015 From: python-checkins at python.org (eric.smith) Date: Thu, 10 Sep 2015 08:03:30 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_After_discussing_with_Guido?= =?utf-8?b?LCBhbGxvdyBib3RoICdmJyBhbmQgJ0YnLg==?= Message-ID: <20150910080329.66862.52670@psf.io> https://hg.python.org/peps/rev/b64085f15c13 changeset: 6046:b64085f15c13 user: Eric V. Smith date: Thu Sep 10 04:03:33 2015 -0400 summary: After discussing with Guido, allow both 'f' and 'F'. files: pep-0498.txt | 9 +++++---- 1 files changed, 5 insertions(+), 4 deletions(-) diff --git a/pep-0498.txt b/pep-0498.txt --- a/pep-0498.txt +++ b/pep-0498.txt @@ -170,10 +170,11 @@ ============= In source code, f-strings are string literals that are prefixed by the -letter 'f'. 'f' may be combined with 'r', in either order, to produce -raw f-string literals. 'f' may not be combined with 'b': this PEP does -not propose to add binary f-strings. 'f' may also be combined with -'u', in either order, although adding 'u' has no effect. +letter 'f' or 'F'. Everywhere this PEP uses 'f', 'F' may also be +used. 'f' may be combined with 'r', in either order, to produce raw +f-string literals. 'f' may not be combined with 'b': this PEP does not +propose to add binary f-strings. 'f' may also be combined with 'u', in +either order, although adding 'u' has no effect. When tokenizing source files, f-strings use the same rules as normal strings, raw strings, binary strings, and triple quoted strings. That -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Thu Sep 10 10:10:47 2015 From: python-checkins at python.org (victor.stinner) Date: Thu, 10 Sep 2015 08:10:47 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Fix_test=5Ftime_on_platfor?= =?utf-8?q?m_with_32-bit_time=5Ft_type?= Message-ID: <20150910081047.17963.50487@psf.io> https://hg.python.org/cpython/rev/892a91c9c95f changeset: 97850:892a91c9c95f user: Victor Stinner date: Thu Sep 10 10:10:39 2015 +0200 summary: Fix test_time on platform with 32-bit time_t type Filter values which would overflow when converted to a C time_t type. files: Lib/test/test_time.py | 23 ++++++++++++++++++----- Modules/_testcapimodule.c | 1 + 2 files changed, 19 insertions(+), 5 deletions(-) diff --git a/Lib/test/test_time.py b/Lib/test/test_time.py --- a/Lib/test/test_time.py +++ b/Lib/test/test_time.py @@ -680,6 +680,15 @@ """ OVERFLOW_SECONDS = None + def setUp(self): + from _testcapi import SIZEOF_TIME_T + bits = SIZEOF_TIME_T * 8 - 1 + self.time_t_min = -2 ** bits + self.time_t_max = 2 ** bits - 1 + + def time_t_filter(self, seconds): + return (self.time_t_min <= seconds <= self.time_t_max) + def _rounding_values(self, use_float): "Build timestamps used to test rounding." @@ -858,7 +867,7 @@ def seconds_filter(secs): return LONG_MIN <= secs <= LONG_MAX else: - seconds_filter = None + seconds_filter = self.time_t_filter self.check_int_rounding(PyTime_AsTimeval, timeval_converter, @@ -875,7 +884,8 @@ self.check_int_rounding(lambda ns, rnd: PyTime_AsTimespec(ns), timespec_converter, - NS_TO_SEC) + NS_TO_SEC, + value_filter=self.time_t_filter) def test_AsMilliseconds(self): from _testcapi import PyTime_AsMilliseconds @@ -904,7 +914,8 @@ from _testcapi import pytime_object_to_time_t self.check_int_rounding(pytime_object_to_time_t, - lambda secs: secs) + lambda secs: secs, + value_filter=self.time_t_filter) self.check_float_rounding(pytime_object_to_time_t, self.decimal_round) @@ -928,7 +939,8 @@ from _testcapi import pytime_object_to_timeval self.check_int_rounding(pytime_object_to_timeval, - lambda secs: (secs, 0)) + lambda secs: (secs, 0), + value_filter=self.time_t_filter) self.check_float_rounding(pytime_object_to_timeval, self.create_converter(SEC_TO_US)) @@ -937,7 +949,8 @@ from _testcapi import pytime_object_to_timespec self.check_int_rounding(pytime_object_to_timespec, - lambda secs: (secs, 0)) + lambda secs: (secs, 0), + value_filter=self.time_t_filter) self.check_float_rounding(pytime_object_to_timespec, self.create_converter(SEC_TO_NS)) diff --git a/Modules/_testcapimodule.c b/Modules/_testcapimodule.c --- a/Modules/_testcapimodule.c +++ b/Modules/_testcapimodule.c @@ -4114,6 +4114,7 @@ PyModule_AddObject(m, "PY_SSIZE_T_MAX", PyLong_FromSsize_t(PY_SSIZE_T_MAX)); PyModule_AddObject(m, "PY_SSIZE_T_MIN", PyLong_FromSsize_t(PY_SSIZE_T_MIN)); PyModule_AddObject(m, "SIZEOF_PYGC_HEAD", PyLong_FromSsize_t(sizeof(PyGC_Head))); + PyModule_AddObject(m, "SIZEOF_TIME_T", PyLong_FromSsize_t(sizeof(time_t))); Py_INCREF(&PyInstanceMethod_Type); PyModule_AddObject(m, "instancemethod", (PyObject *)&PyInstanceMethod_Type); -- Repository URL: https://hg.python.org/cpython From lp_benchmark_robot at intel.com Thu Sep 10 10:49:15 2015 From: lp_benchmark_robot at intel.com (lp_benchmark_robot) Date: Thu, 10 Sep 2015 08:49:15 +0000 Subject: [Python-checkins] Benchmark Results for Python Default 2015-09-10 Message-ID: <078AA0FFE8C7034097F90205717F504611D7AB5F@IRSMSX102.ger.corp.intel.com> Results for project python_default-nightly, build date 2015-09-10 07:10:19 commit: 18802076b52c1e85222811f2209a288de4d78b0a revision date: 2015-09-10 02:46:51 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v3.4.3, with hash b4cbecbc0781e89a309d03b60a1f75f8499250e6 from 2015-02-25 12:15:33+00:00 ------------------------------------------------------------------------------------------ benchmark relative change since change since current rev with std_dev* last run v3.4.3 regrtest PGO ------------------------------------------------------------------------------------------ :-) django_v2 0.23190% 0.41857% 9.32936% 16.55844% :-( pybench 0.10325% 0.04844% -2.11284% 8.53745% :-( regex_v8 2.91172% 0.17201% -3.63523% 3.68378% :-) nbody 0.12467% 4.43967% -1.71914% 11.69701% :-( json_dump_v2 0.31577% -1.53823% -5.36029% 16.06486% :-| normal_startup 0.71613% 0.43062% -0.36791% 4.87444% ------------------------------------------------------------------------------------------ Note: Benchmark results are measured in seconds. * Relative Standard Deviation (Standard Deviation/Average) Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. From python-checkins at python.org Thu Sep 10 11:48:28 2015 From: python-checkins at python.org (victor.stinner) Date: Thu, 10 Sep 2015 09:48:28 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Fix_test=5Ftime_on_platfor?= =?utf-8?q?m_with_32-bit_time=5Ft_type?= Message-ID: <20150910094828.66864.38331@psf.io> https://hg.python.org/cpython/rev/0456e9379fa7 changeset: 97851:0456e9379fa7 user: Victor Stinner date: Thu Sep 10 11:45:06 2015 +0200 summary: Fix test_time on platform with 32-bit time_t type Filter also values for check_float_rounding(). files: Lib/test/test_time.py | 9 ++++++--- 1 files changed, 6 insertions(+), 3 deletions(-) diff --git a/Lib/test/test_time.py b/Lib/test/test_time.py --- a/Lib/test/test_time.py +++ b/Lib/test/test_time.py @@ -918,7 +918,8 @@ value_filter=self.time_t_filter) self.check_float_rounding(pytime_object_to_time_t, - self.decimal_round) + self.decimal_round, + value_filter=self.time_t_filter) def create_converter(self, sec_to_unit): def converter(secs): @@ -943,7 +944,8 @@ value_filter=self.time_t_filter) self.check_float_rounding(pytime_object_to_timeval, - self.create_converter(SEC_TO_US)) + self.create_converter(SEC_TO_US), + value_filter=self.time_t_filter) def test_object_to_timespec(self): from _testcapi import pytime_object_to_timespec @@ -953,7 +955,8 @@ value_filter=self.time_t_filter) self.check_float_rounding(pytime_object_to_timespec, - self.create_converter(SEC_TO_NS)) + self.create_converter(SEC_TO_NS), + value_filter=self.time_t_filter) if __name__ == "__main__": -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 10 11:48:29 2015 From: python-checkins at python.org (victor.stinner) Date: Thu, 10 Sep 2015 09:48:29 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Try_to_fix_test=5Ftime=2Et?= =?utf-8?q?est=5FAsSecondsDouble=28=29_on_=22x86_Gentoo_Non-Debug_with_X?= Message-ID: <20150910094829.68867.74795@psf.io> https://hg.python.org/cpython/rev/bafae65793a0 changeset: 97852:bafae65793a0 user: Victor Stinner date: Thu Sep 10 11:48:00 2015 +0200 summary: Try to fix test_time.test_AsSecondsDouble() on "x86 Gentoo Non-Debug with X 3.x" buildbot Use volatile keyword in _PyTime_Round() files: Python/pytime.c | 11 ++++++++--- 1 files changed, 8 insertions(+), 3 deletions(-) diff --git a/Python/pytime.c b/Python/pytime.c --- a/Python/pytime.c +++ b/Python/pytime.c @@ -75,12 +75,17 @@ static double _PyTime_Round(double x, _PyTime_round_t round) { + /* volatile avoids optimization changing how numbers are rounded */ + volatile double d; + + d = x; if (round == _PyTime_ROUND_HALF_EVEN) - return _PyTime_RoundHalfEven(x); + d = _PyTime_RoundHalfEven(d); else if (round == _PyTime_ROUND_CEILING) - return ceil(x); + d = ceil(d); else - return floor(x); + d = floor(d); + return d; } static int -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 10 13:25:58 2015 From: python-checkins at python.org (victor.stinner) Date: Thu, 10 Sep 2015 11:25:58 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_New_try_to_fix_test=5Ftime?= =?utf-8?q?=2Etest=5FAsSecondsDouble=28=29_on_x86_buildbots=2E?= Message-ID: <20150910112558.11244.68833@psf.io> https://hg.python.org/cpython/rev/fa66577f6807 changeset: 97853:fa66577f6807 user: Victor Stinner date: Thu Sep 10 13:25:17 2015 +0200 summary: New try to fix test_time.test_AsSecondsDouble() on x86 buildbots. Use volatile keyword in _PyTime_AsSecondsDouble() files: Python/pytime.c | 9 +++++++-- 1 files changed, 7 insertions(+), 2 deletions(-) diff --git a/Python/pytime.c b/Python/pytime.c --- a/Python/pytime.c +++ b/Python/pytime.c @@ -332,16 +332,21 @@ double _PyTime_AsSecondsDouble(_PyTime_t t) { + /* volatile avoids optimization changing how numbers are rounded */ + volatile double d; + if (t % SEC_TO_NS == 0) { _PyTime_t secs; /* Divide using integers to avoid rounding issues on the integer part. 1e-9 cannot be stored exactly in IEEE 64-bit. */ secs = t / SEC_TO_NS; - return (double)secs; + d = (double)secs; } else { - return (double)t / 1e9; + d = (double)t; + d /= 1e9; } + return d; } PyObject * -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 10 15:55:35 2015 From: python-checkins at python.org (victor.stinner) Date: Thu, 10 Sep 2015 13:55:35 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_pytime=3A_add_=5FPyTime=5F?= =?utf-8?q?check=5Fmul=5Foverflow=28=29_macro_to_avoid_undefined_behaviour?= Message-ID: <20150910135535.101486.16364@psf.io> https://hg.python.org/cpython/rev/038db6bb1f6a changeset: 97854:038db6bb1f6a user: Victor Stinner date: Thu Sep 10 15:55:07 2015 +0200 summary: pytime: add _PyTime_check_mul_overflow() macro to avoid undefined behaviour Overflow test in test_FromSecondsObject() fails on FreeBSD 10.0 buildbot which uses clang. clang implements more aggressive optimization which gives different result than GCC on undefined behaviours. Check if a multiplication will overflow, instead of checking if a multiplicatin had overflowed, to avoid undefined behaviour. Add also debug information if the test on overflow fails. files: Lib/test/test_time.py | 3 +- Python/pytime.c | 35 +++++++++++++++++++++--------- 2 files changed, 26 insertions(+), 12 deletions(-) diff --git a/Lib/test/test_time.py b/Lib/test/test_time.py --- a/Lib/test/test_time.py +++ b/Lib/test/test_time.py @@ -781,7 +781,8 @@ overflow_values = convert_values(ns_timestamps) for time_rnd, _ in ROUNDING_MODES : for value in overflow_values: - with self.assertRaises(OverflowError): + debug_info = {'value': value, 'rounding': time_rnd} + with self.assertRaises(OverflowError, msg=debug_info): pytime_converter(value, time_rnd) def check_int_rounding(self, pytime_converter, expected_func, diff --git a/Python/pytime.c b/Python/pytime.c --- a/Python/pytime.c +++ b/Python/pytime.c @@ -7,6 +7,11 @@ #include /* mach_absolute_time(), mach_timebase_info() */ #endif +#define _PyTime_check_mul_overflow(a, b) \ + (assert(b > 0), \ + (_PyTime_t)(a) < _PyTime_MIN / (_PyTime_t)(b) \ + || _PyTime_MAX / (_PyTime_t)(b) < (_PyTime_t)(a)) + /* To millisecond (10^-3) */ #define SEC_TO_MS 1000 @@ -226,12 +231,15 @@ _PyTime_t t; int res = 0; - t = (_PyTime_t)ts->tv_sec * SEC_TO_NS; - if (t / SEC_TO_NS != ts->tv_sec) { + assert(sizeof(ts->tv_sec) <= sizeof(_PyTime_t)); + t = (_PyTime_t)ts->tv_sec; + + if (_PyTime_check_mul_overflow(t, SEC_TO_NS)) { if (raise) _PyTime_overflow(); res = -1; } + t = t * SEC_TO_NS; t += ts->tv_nsec; @@ -245,12 +253,15 @@ _PyTime_t t; int res = 0; - t = (_PyTime_t)tv->tv_sec * SEC_TO_NS; - if (t / SEC_TO_NS != tv->tv_sec) { + assert(sizeof(ts->tv_sec) <= sizeof(_PyTime_t)); + t = (_PyTime_t)tv->tv_sec; + + if (_PyTime_check_mul_overflow(t, SEC_TO_NS)) { if (raise) _PyTime_overflow(); res = -1; } + t = t * SEC_TO_NS; t += (_PyTime_t)tv->tv_usec * US_TO_NS; @@ -308,11 +319,11 @@ return -1; } - *t = sec * unit_to_ns; - if (*t / unit_to_ns != sec) { + if (_PyTime_check_mul_overflow(sec, unit_to_ns)) { _PyTime_overflow(); return -1; } + *t = sec * unit_to_ns; return 0; } } @@ -587,19 +598,20 @@ return pygettimeofday_new(t, info, 1); } - static int pymonotonic(_PyTime_t *tp, _Py_clock_info_t *info, int raise) { #if defined(MS_WINDOWS) - ULONGLONG result; + ULONGLONG ticks; + _PyTime_t t; assert(info == NULL || raise); - result = GetTickCount64(); + ticks = GetTickCount64(); + assert(sizeof(result) <= sizeof(_PyTime_t)); + t = (_PyTime_t)ticks; - *tp = result * MS_TO_NS; - if (*tp / MS_TO_NS != result) { + if (_PyTime_check_mul_overflow(t, MS_TO_NS)) { if (raise) { _PyTime_overflow(); return -1; @@ -607,6 +619,7 @@ /* Hello, time traveler! */ assert(0); } + *tp = t * MS_TO_NS; if (info) { DWORD timeAdjustment, timeIncrement; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 10 16:00:14 2015 From: python-checkins at python.org (victor.stinner) Date: Thu, 10 Sep 2015 14:00:14 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_pytime=3A_oops=2C_fix_typo?= =?utf-8?q?s_on_Windows?= Message-ID: <20150910140011.15724.35926@psf.io> https://hg.python.org/cpython/rev/d433d74e0377 changeset: 97855:d433d74e0377 user: Victor Stinner date: Thu Sep 10 16:00:06 2015 +0200 summary: pytime: oops, fix typos on Windows files: Python/pytime.c | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Python/pytime.c b/Python/pytime.c --- a/Python/pytime.c +++ b/Python/pytime.c @@ -253,7 +253,7 @@ _PyTime_t t; int res = 0; - assert(sizeof(ts->tv_sec) <= sizeof(_PyTime_t)); + assert(sizeof(tv->tv_sec) <= sizeof(_PyTime_t)); t = (_PyTime_t)tv->tv_sec; if (_PyTime_check_mul_overflow(t, SEC_TO_NS)) { @@ -608,7 +608,7 @@ assert(info == NULL || raise); ticks = GetTickCount64(); - assert(sizeof(result) <= sizeof(_PyTime_t)); + assert(sizeof(ticks) <= sizeof(_PyTime_t)); t = (_PyTime_t)ticks; if (_PyTime_check_mul_overflow(t, MS_TO_NS)) { -- Repository URL: https://hg.python.org/cpython From lp_benchmark_robot at intel.com Thu Sep 10 16:20:52 2015 From: lp_benchmark_robot at intel.com (lp_benchmark_robot) Date: Thu, 10 Sep 2015 14:20:52 +0000 Subject: [Python-checkins] Benchmark Results for Python 2.7 2015-09-10 Message-ID: <078AA0FFE8C7034097F90205717F504611D7AC5D@IRSMSX102.ger.corp.intel.com> Results for project python_2.7-nightly, build date 2015-09-10 12:24:47 commit: 32893d8a52a972ed58f3c9fed17a97b65f5f06f1 revision date: 2015-09-09 09:18:36 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v2.7.10, with hash 15c95b7d81dcf821daade360741e00714667653f from 2015-05-23 16:02:14+00:00 ------------------------------------------------------------------------------------------ benchmark relative change since change since current rev with std_dev* last run v2.7.10 regrtest PGO ------------------------------------------------------------------------------------------ :-) django_v2 0.15478% 0.93041% 5.92486% 6.52453% :-) pybench 0.15042% 0.00402% 6.74646% 6.80401% :-| regex_v8 0.54930% 0.00076% -1.09037% 7.55097% :-) nbody 0.11267% 0.04953% 9.13846% 1.87879% :-) json_dump_v2 0.24206% 0.24035% 4.32005% 14.57970% :-| normal_startup 1.88387% 0.92859% -0.73688% 2.50851% :-| ssbench 0.81087% 0.53868% 0.39324% 1.45667% ------------------------------------------------------------------------------------------ Note: Benchmark results for ssbench are measured in requests/second while all other are measured in seconds. * Relative Standard Deviation (Standard Deviation/Average) Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. From python-checkins at python.org Thu Sep 10 19:54:57 2015 From: python-checkins at python.org (guido.van.rossum) Date: Thu, 10 Sep 2015 17:54:57 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Restore_doc_updates_to_typing=2Erst_by_Ivan_Levkivskyi_a?= =?utf-8?q?nd_Daniel_Andrade_Groppe=2E?= Message-ID: <20150910175457.114789.75419@psf.io> https://hg.python.org/cpython/rev/7fc4306d537b changeset: 97857:7fc4306d537b parent: 97855:d433d74e0377 parent: 97856:8a7814a3613b user: Guido van Rossum date: Thu Sep 10 10:54:10 2015 -0700 summary: Restore doc updates to typing.rst by Ivan Levkivskyi and Daniel Andrade Groppe. files: Doc/library/typing.rst | 128 ++++++++++++++++++++++++---- 1 files changed, 110 insertions(+), 18 deletions(-) diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -35,7 +35,7 @@ -------- Frameworks expecting callback functions of specific signatures might be -type hinted using `Callable[[Arg1Type, Arg2Type], ReturnType]`. +type hinted using ``Callable[[Arg1Type, Arg2Type], ReturnType]``. For example:: @@ -68,7 +68,7 @@ overrides: Mapping[str, str]) -> None: ... Generics can be parametrized by using a new factory available in typing -called TypeVar. +called :class:`TypeVar`. .. code-block:: python @@ -145,7 +145,7 @@ class Pair(Generic[T, T]): # INVALID ... -You can use multiple inheritance with `Generic`:: +You can use multiple inheritance with :class:`Generic`:: from typing import TypeVar, Generic, Sized @@ -154,6 +154,17 @@ class LinkedList(Sized, Generic[T]): ... +When inheriting from generic classes, some type variables could fixed:: + + from typing import TypeVar, Mapping + + T = TypeVar('T') + + class MyDict(Mapping[str, T]): + ... + +In this case ``MyDict`` has a single parameter, ``T``. + Subclassing a generic class without specifying type parameters assumes :class:`Any` for each position. In the following example, ``MyIterable`` is not generic but implicitly inherits from ``Iterable[Any]``:: @@ -162,7 +173,11 @@ class MyIterable(Iterable): # Same as Iterable[Any] -Generic metaclasses are not supported. +The metaclass used by :class:`Generic` is a subclass of :class:`abc.ABCMeta`. +A generic class can be an ABC by including abstract methods or properties, +and generic classes can also have ABCs as base classes without a metaclass +conflict. Generic metaclasses are not supported. + The :class:`Any` type --------------------- @@ -178,15 +193,6 @@ on it, and a value of type :class:`Any` can be assigned to a variable (or used as a return value) of a more constrained type. -Default argument values ------------------------ - -Use a literal ellipsis ``...`` to declare an argument as having a default value:: - - from typing import AnyStr - - def foo(x: AnyStr, y: AnyStr = ...) -> AnyStr: ... - Classes, functions, and decorators ---------------------------------- @@ -236,7 +242,11 @@ Type variables may be marked covariant or contravariant by passing ``covariant=True`` or ``contravariant=True``. See :pep:`484` for more - details. By default type variables are invariant. + details. By default type variables are invariant. Alternatively, + a type variable may specify an upper bound using ``bound=``. + This means that an actual type substituted (explicitly or implictly) + for the type variable must be a subclass of the boundary type, + see :pep:`484`. .. class:: Union @@ -329,57 +339,139 @@ .. class:: Iterable(Generic[T_co]) + A generic version of the :class:`collections.abc.Iterable`. + .. class:: Iterator(Iterable[T_co]) + A generic version of the :class:`collections.abc.Iterator`. + .. class:: SupportsInt + An ABC with one abstract method `__int__`. + .. class:: SupportsFloat + An ABC with one abstract method `__float__`. + .. class:: SupportsAbs + An ABC with one abstract method `__abs__` that is covariant + in its return type. + .. class:: SupportsRound + An ABC with one abstract method `__round__` + that is covariant in its return type. + .. class:: Reversible + An ABC with one abstract method `__reversed__` returning + an `Iterator[T_co]`. + .. class:: Container(Generic[T_co]) + A generic version of :class:`collections.abc.Container`. + .. class:: AbstractSet(Sized, Iterable[T_co], Container[T_co]) + A generic version of :class:`collections.abc.Set`. + .. class:: MutableSet(AbstractSet[T]) -.. class:: Mapping(Sized, Iterable[KT_co], Container[KT_co], Generic[KT_co, VT_co]) + A generic version of :class:`collections.abc.MutableSet`. + +.. class:: Mapping(Sized, Iterable[KT], Container[KT], Generic[VT_co]) + + A generic version of :class:`collections.abc.Mapping`. .. class:: MutableMapping(Mapping[KT, VT]) + A generic version of :class:`collections.abc.MutableMapping`. + .. class:: Sequence(Sized, Iterable[T_co], Container[T_co]) + A generic version of :class:`collections.abc.Sequence`. + .. class:: MutableSequence(Sequence[T]) + A generic version of :class:`collections.abc.MutableSequence`. + .. class:: ByteString(Sequence[int]) + A generic version of :class:`collections.abc.ByteString`. + + This type represents the types :class:`bytes`, :class:`bytearray`, + and :class:`memoryview`. + + As a shorthand for this type, :class:`bytes` can be used to + annotate arguments of any of the types mentioned above. + .. class:: List(list, MutableSequence[T]) -.. class:: Set(set, MutableSet[T]) + Generic version of :class:`list`. + Useful for annotating return types. To annotate arguments it is preferred + to use abstract collection types such as :class:`Mapping`, :class:`Sequence`, + or :class:`AbstractSet`. + + This type may be used as follows:: + + T = TypeVar('T', int, float) + + def vec2(x: T, y: T) -> List[T]: + return [x, y] + + def slice__to_4(vector: Sequence[T]) -> List[T]: + return vector[0:4] + +.. class:: AbstractSet(set, MutableSet[T]) + + A generic version of :class:`collections.abc.Set`. .. class:: MappingView(Sized, Iterable[T_co]) + A generic version of :class:`collections.abc.MappingView`. + .. class:: KeysView(MappingView[KT_co], AbstractSet[KT_co]) + A generic version of :class:`collections.abc.KeysView`. + .. class:: ItemsView(MappingView, Generic[KT_co, VT_co]) + A generic version of :class:`collections.abc.ItemsView`. + .. class:: ValuesView(MappingView[VT_co]) + A generic version of :class:`collections.abc.ValuesView`. + .. class:: Dict(dict, MutableMapping[KT, VT]) + A generic version of :class:`dict`. + The usage of this type is as follows:: + + def get_position_in_index(word_list: Dict[str, int], word: str) -> int: + return word_list[word] + .. class:: Generator(Iterator[T_co], Generic[T_co, T_contra, V_co]) .. class:: io - Wrapper namespace for IO generic classes. + Wrapper namespace for I/O stream types. + + This defines the generic type ``IO[AnyStr]`` and aliases ``TextIO`` + and ``BinaryIO`` for respectively ``IO[str]`` and ``IO[bytes]``. + These representing the types of I/O streams such as returned by + :func:`open`. .. class:: re - Wrapper namespace for re type classes. + Wrapper namespace for regular expression matching types. + + This defines the type aliases ``Pattern`` and ``Match`` which + correspond to the return types from :func:`re.compile` and + :func:`re.match`. These types (and the corresponding functions) + are generic in ``AnyStr`` and can be made specific by writing + ``Pattern[str]``, ``Pattern[bytes]``, ``Match[str]``, or + ``Match[bytes]``. .. function:: NamedTuple(typename, fields) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 10 19:54:57 2015 From: python-checkins at python.org (guido.van.rossum) Date: Thu, 10 Sep 2015 17:54:57 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_Restore_doc_up?= =?utf-8?q?dates_to_typing=2Erst_by_Ivan_Levkivskyi_and_Daniel_Andrade_Gro?= =?utf-8?q?ppe=2E?= Message-ID: <20150910175456.27687.26497@psf.io> https://hg.python.org/cpython/rev/8a7814a3613b changeset: 97856:8a7814a3613b branch: 3.5 parent: 97847:6c8e2c6e3a99 user: Guido van Rossum date: Thu Sep 10 10:52:11 2015 -0700 summary: Restore doc updates to typing.rst by Ivan Levkivskyi and Daniel Andrade Groppe. files: Doc/library/typing.rst | 128 ++++++++++++++++++++++++---- 1 files changed, 110 insertions(+), 18 deletions(-) diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -35,7 +35,7 @@ -------- Frameworks expecting callback functions of specific signatures might be -type hinted using `Callable[[Arg1Type, Arg2Type], ReturnType]`. +type hinted using ``Callable[[Arg1Type, Arg2Type], ReturnType]``. For example:: @@ -68,7 +68,7 @@ overrides: Mapping[str, str]) -> None: ... Generics can be parametrized by using a new factory available in typing -called TypeVar. +called :class:`TypeVar`. .. code-block:: python @@ -145,7 +145,7 @@ class Pair(Generic[T, T]): # INVALID ... -You can use multiple inheritance with `Generic`:: +You can use multiple inheritance with :class:`Generic`:: from typing import TypeVar, Generic, Sized @@ -154,6 +154,17 @@ class LinkedList(Sized, Generic[T]): ... +When inheriting from generic classes, some type variables could fixed:: + + from typing import TypeVar, Mapping + + T = TypeVar('T') + + class MyDict(Mapping[str, T]): + ... + +In this case ``MyDict`` has a single parameter, ``T``. + Subclassing a generic class without specifying type parameters assumes :class:`Any` for each position. In the following example, ``MyIterable`` is not generic but implicitly inherits from ``Iterable[Any]``:: @@ -162,7 +173,11 @@ class MyIterable(Iterable): # Same as Iterable[Any] -Generic metaclasses are not supported. +The metaclass used by :class:`Generic` is a subclass of :class:`abc.ABCMeta`. +A generic class can be an ABC by including abstract methods or properties, +and generic classes can also have ABCs as base classes without a metaclass +conflict. Generic metaclasses are not supported. + The :class:`Any` type --------------------- @@ -178,15 +193,6 @@ on it, and a value of type :class:`Any` can be assigned to a variable (or used as a return value) of a more constrained type. -Default argument values ------------------------ - -Use a literal ellipsis ``...`` to declare an argument as having a default value:: - - from typing import AnyStr - - def foo(x: AnyStr, y: AnyStr = ...) -> AnyStr: ... - Classes, functions, and decorators ---------------------------------- @@ -236,7 +242,11 @@ Type variables may be marked covariant or contravariant by passing ``covariant=True`` or ``contravariant=True``. See :pep:`484` for more - details. By default type variables are invariant. + details. By default type variables are invariant. Alternatively, + a type variable may specify an upper bound using ``bound=``. + This means that an actual type substituted (explicitly or implictly) + for the type variable must be a subclass of the boundary type, + see :pep:`484`. .. class:: Union @@ -329,57 +339,139 @@ .. class:: Iterable(Generic[T_co]) + A generic version of the :class:`collections.abc.Iterable`. + .. class:: Iterator(Iterable[T_co]) + A generic version of the :class:`collections.abc.Iterator`. + .. class:: SupportsInt + An ABC with one abstract method `__int__`. + .. class:: SupportsFloat + An ABC with one abstract method `__float__`. + .. class:: SupportsAbs + An ABC with one abstract method `__abs__` that is covariant + in its return type. + .. class:: SupportsRound + An ABC with one abstract method `__round__` + that is covariant in its return type. + .. class:: Reversible + An ABC with one abstract method `__reversed__` returning + an `Iterator[T_co]`. + .. class:: Container(Generic[T_co]) + A generic version of :class:`collections.abc.Container`. + .. class:: AbstractSet(Sized, Iterable[T_co], Container[T_co]) + A generic version of :class:`collections.abc.Set`. + .. class:: MutableSet(AbstractSet[T]) -.. class:: Mapping(Sized, Iterable[KT_co], Container[KT_co], Generic[KT_co, VT_co]) + A generic version of :class:`collections.abc.MutableSet`. + +.. class:: Mapping(Sized, Iterable[KT], Container[KT], Generic[VT_co]) + + A generic version of :class:`collections.abc.Mapping`. .. class:: MutableMapping(Mapping[KT, VT]) + A generic version of :class:`collections.abc.MutableMapping`. + .. class:: Sequence(Sized, Iterable[T_co], Container[T_co]) + A generic version of :class:`collections.abc.Sequence`. + .. class:: MutableSequence(Sequence[T]) + A generic version of :class:`collections.abc.MutableSequence`. + .. class:: ByteString(Sequence[int]) + A generic version of :class:`collections.abc.ByteString`. + + This type represents the types :class:`bytes`, :class:`bytearray`, + and :class:`memoryview`. + + As a shorthand for this type, :class:`bytes` can be used to + annotate arguments of any of the types mentioned above. + .. class:: List(list, MutableSequence[T]) -.. class:: Set(set, MutableSet[T]) + Generic version of :class:`list`. + Useful for annotating return types. To annotate arguments it is preferred + to use abstract collection types such as :class:`Mapping`, :class:`Sequence`, + or :class:`AbstractSet`. + + This type may be used as follows:: + + T = TypeVar('T', int, float) + + def vec2(x: T, y: T) -> List[T]: + return [x, y] + + def slice__to_4(vector: Sequence[T]) -> List[T]: + return vector[0:4] + +.. class:: AbstractSet(set, MutableSet[T]) + + A generic version of :class:`collections.abc.Set`. .. class:: MappingView(Sized, Iterable[T_co]) + A generic version of :class:`collections.abc.MappingView`. + .. class:: KeysView(MappingView[KT_co], AbstractSet[KT_co]) + A generic version of :class:`collections.abc.KeysView`. + .. class:: ItemsView(MappingView, Generic[KT_co, VT_co]) + A generic version of :class:`collections.abc.ItemsView`. + .. class:: ValuesView(MappingView[VT_co]) + A generic version of :class:`collections.abc.ValuesView`. + .. class:: Dict(dict, MutableMapping[KT, VT]) + A generic version of :class:`dict`. + The usage of this type is as follows:: + + def get_position_in_index(word_list: Dict[str, int], word: str) -> int: + return word_list[word] + .. class:: Generator(Iterator[T_co], Generic[T_co, T_contra, V_co]) .. class:: io - Wrapper namespace for IO generic classes. + Wrapper namespace for I/O stream types. + + This defines the generic type ``IO[AnyStr]`` and aliases ``TextIO`` + and ``BinaryIO`` for respectively ``IO[str]`` and ``IO[bytes]``. + These representing the types of I/O streams such as returned by + :func:`open`. .. class:: re - Wrapper namespace for re type classes. + Wrapper namespace for regular expression matching types. + + This defines the type aliases ``Pattern`` and ``Match`` which + correspond to the return types from :func:`re.compile` and + :func:`re.match`. These types (and the corresponding functions) + are generic in ``AnyStr`` and can be made specific by writing + ``Pattern[str]``, ``Pattern[bytes]``, ``Match[str]``, or + ``Match[bytes]``. .. function:: NamedTuple(typename, fields) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 10 20:42:44 2015 From: python-checkins at python.org (berker.peksag) Date: Thu, 10 Sep 2015 18:42:44 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Use_print_function_in_mock_docs=2E?= Message-ID: <20150910184242.11262.97005@psf.io> https://hg.python.org/cpython/rev/f41cdb99202f changeset: 97860:f41cdb99202f parent: 97857:7fc4306d537b parent: 97859:a380841644b2 user: Berker Peksag date: Thu Sep 10 21:42:18 2015 +0300 summary: Use print function in mock docs. files: Doc/library/unittest.mock.rst | 12 ++++++------ 1 files changed, 6 insertions(+), 6 deletions(-) diff --git a/Doc/library/unittest.mock.rst b/Doc/library/unittest.mock.rst --- a/Doc/library/unittest.mock.rst +++ b/Doc/library/unittest.mock.rst @@ -552,7 +552,7 @@ keyword arguments (or an empty dictionary). >>> mock = Mock(return_value=None) - >>> print mock.call_args + >>> print(mock.call_args) None >>> mock() >>> mock.call_args @@ -747,7 +747,7 @@ >>> with patch('__main__.Foo.foo', new_callable=PropertyMock) as mock_foo: ... mock_foo.return_value = 'mockity-mock' ... this_foo = Foo() - ... print this_foo.foo + ... print(this_foo.foo) ... this_foo.foo = 6 ... mockity-mock @@ -1135,7 +1135,7 @@ >>> from io import StringIO >>> def foo(): - ... print 'Something' + ... print('Something') ... >>> @patch('sys.stdout', new_callable=StringIO) ... def test(mock_stdout): @@ -1249,7 +1249,7 @@ >>> import os >>> with patch.dict('os.environ', {'newkey': 'newvalue'}): - ... print os.environ['newkey'] + ... print(os.environ['newkey']) ... newvalue >>> assert 'newkey' not in os.environ @@ -1462,9 +1462,9 @@ >>> @patch('__main__.value', 'not three') ... class Thing: ... def foo_one(self): - ... print value + ... print(value) ... def foo_two(self): - ... print value + ... print(value) ... >>> >>> Thing().foo_one() -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 10 20:42:44 2015 From: python-checkins at python.org (berker.peksag) Date: Thu, 10 Sep 2015 18:42:44 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Use_print_function_in_mock_docs=2E?= Message-ID: <20150910184242.11246.28075@psf.io> https://hg.python.org/cpython/rev/a380841644b2 changeset: 97859:a380841644b2 branch: 3.5 parent: 97856:8a7814a3613b parent: 97858:5093d9151a18 user: Berker Peksag date: Thu Sep 10 21:41:52 2015 +0300 summary: Use print function in mock docs. files: Doc/library/unittest.mock.rst | 12 ++++++------ 1 files changed, 6 insertions(+), 6 deletions(-) diff --git a/Doc/library/unittest.mock.rst b/Doc/library/unittest.mock.rst --- a/Doc/library/unittest.mock.rst +++ b/Doc/library/unittest.mock.rst @@ -552,7 +552,7 @@ keyword arguments (or an empty dictionary). >>> mock = Mock(return_value=None) - >>> print mock.call_args + >>> print(mock.call_args) None >>> mock() >>> mock.call_args @@ -747,7 +747,7 @@ >>> with patch('__main__.Foo.foo', new_callable=PropertyMock) as mock_foo: ... mock_foo.return_value = 'mockity-mock' ... this_foo = Foo() - ... print this_foo.foo + ... print(this_foo.foo) ... this_foo.foo = 6 ... mockity-mock @@ -1135,7 +1135,7 @@ >>> from io import StringIO >>> def foo(): - ... print 'Something' + ... print('Something') ... >>> @patch('sys.stdout', new_callable=StringIO) ... def test(mock_stdout): @@ -1249,7 +1249,7 @@ >>> import os >>> with patch.dict('os.environ', {'newkey': 'newvalue'}): - ... print os.environ['newkey'] + ... print(os.environ['newkey']) ... newvalue >>> assert 'newkey' not in os.environ @@ -1462,9 +1462,9 @@ >>> @patch('__main__.value', 'not three') ... class Thing: ... def foo_one(self): - ... print value + ... print(value) ... def foo_two(self): - ... print value + ... print(value) ... >>> >>> Thing().foo_one() -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 10 20:42:44 2015 From: python-checkins at python.org (berker.peksag) Date: Thu, 10 Sep 2015 18:42:44 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_Use_print_func?= =?utf-8?q?tion_in_mock_docs=2E?= Message-ID: <20150910184242.15708.58807@psf.io> https://hg.python.org/cpython/rev/5093d9151a18 changeset: 97858:5093d9151a18 branch: 3.4 parent: 97833:83ea55a1204a user: Berker Peksag date: Thu Sep 10 21:41:15 2015 +0300 summary: Use print function in mock docs. files: Doc/library/unittest.mock.rst | 12 ++++++------ 1 files changed, 6 insertions(+), 6 deletions(-) diff --git a/Doc/library/unittest.mock.rst b/Doc/library/unittest.mock.rst --- a/Doc/library/unittest.mock.rst +++ b/Doc/library/unittest.mock.rst @@ -532,7 +532,7 @@ keyword arguments (or an empty dictionary). >>> mock = Mock(return_value=None) - >>> print mock.call_args + >>> print(mock.call_args) None >>> mock() >>> mock.call_args @@ -727,7 +727,7 @@ >>> with patch('__main__.Foo.foo', new_callable=PropertyMock) as mock_foo: ... mock_foo.return_value = 'mockity-mock' ... this_foo = Foo() - ... print this_foo.foo + ... print(this_foo.foo) ... this_foo.foo = 6 ... mockity-mock @@ -1109,7 +1109,7 @@ >>> from io import StringIO >>> def foo(): - ... print 'Something' + ... print('Something') ... >>> @patch('sys.stdout', new_callable=StringIO) ... def test(mock_stdout): @@ -1223,7 +1223,7 @@ >>> import os >>> with patch.dict('os.environ', {'newkey': 'newvalue'}): - ... print os.environ['newkey'] + ... print(os.environ['newkey']) ... newvalue >>> assert 'newkey' not in os.environ @@ -1420,9 +1420,9 @@ >>> @patch('__main__.value', 'not three') ... class Thing: ... def foo_one(self): - ... print value + ... print(value) ... def foo_two(self): - ... print value + ... print(value) ... >>> >>> Thing().foo_one() -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 10 20:56:29 2015 From: python-checkins at python.org (berker.peksag) Date: Thu, 10 Sep 2015 18:56:29 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_Fix_typos_and_?= =?utf-8?q?improve_markup_in_typing=2Erst=2E?= Message-ID: <20150910185625.27693.98737@psf.io> https://hg.python.org/cpython/rev/72e696bc480a changeset: 97861:72e696bc480a branch: 3.5 parent: 97859:a380841644b2 user: Berker Peksag date: Thu Sep 10 21:55:50 2015 +0300 summary: Fix typos and improve markup in typing.rst. files: Doc/library/typing.rst | 37 ++++++++++++++--------------- 1 files changed, 18 insertions(+), 19 deletions(-) diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -60,7 +60,7 @@ inferred in a generic way, abstract base classes have been extended to support subscription to denote expected types for container elements. -.. code-block:: python +:: from typing import Mapping, Sequence @@ -70,7 +70,7 @@ Generics can be parametrized by using a new factory available in typing called :class:`TypeVar`. -.. code-block:: python +:: from typing import Sequence, TypeVar @@ -85,7 +85,7 @@ A user-defined class can be defined as a generic class. -.. code-block:: python +:: from typing import TypeVar, Generic from logging import Logger @@ -220,9 +220,7 @@ Type variables exist primarily for the benefit of static type checkers. They serve as the parameters for generic types as well as for generic function definitions. See class Generic for more - information on generic types. Generic functions work as follows: - - .. code-block:: python + information on generic types. Generic functions work as follows:: def repeat(x: T, n: int) -> Sequence[T]: """Return a list containing n references to x.""" @@ -238,13 +236,13 @@ the return type is still plain :class:`str`. At runtime, ``isinstance(x, T)`` will raise :exc:`TypeError`. In general, - :func:`isinstance` and :func:`issublass` should not be used with types. + :func:`isinstance` and :func:`issubclass` should not be used with types. Type variables may be marked covariant or contravariant by passing ``covariant=True`` or ``contravariant=True``. See :pep:`484` for more details. By default type variables are invariant. Alternatively, a type variable may specify an upper bound using ``bound=``. - This means that an actual type substituted (explicitly or implictly) + This means that an actual type substituted (explicitly or implicitly) for the type variable must be a subclass of the boundary type, see :pep:`484`. @@ -278,7 +276,7 @@ * You cannot subclass or instantiate a union. - * You cannot write ``Union[X][Y]`` + * You cannot write ``Union[X][Y]``. * You can use ``Optional[X]`` as a shorthand for ``Union[X, None]``. @@ -331,6 +329,7 @@ X = TypeVar('X') Y = TypeVar('Y') + def lookup_name(mapping: Mapping[X, Y], key: X, default: Y) -> Y: try: return mapping[key] @@ -347,26 +346,26 @@ .. class:: SupportsInt - An ABC with one abstract method `__int__`. + An ABC with one abstract method ``__int__``. .. class:: SupportsFloat - An ABC with one abstract method `__float__`. + An ABC with one abstract method ``__float__``. .. class:: SupportsAbs - An ABC with one abstract method `__abs__` that is covariant + An ABC with one abstract method ``__abs__`` that is covariant in its return type. .. class:: SupportsRound - An ABC with one abstract method `__round__` + An ABC with one abstract method ``__round__`` that is covariant in its return type. .. class:: Reversible - An ABC with one abstract method `__reversed__` returning - an `Iterator[T_co]`. + An ABC with one abstract method ``__reversed__`` returning + an ``Iterator[T_co]``. .. class:: Container(Generic[T_co]) @@ -503,9 +502,9 @@ Return type hints for a function or method object. - This is often the same as obj.__annotations__, but it handles + This is often the same as ``obj.__annotations__``, but it handles forward references encoded as string literals, and if necessary - adds Optional[t] if a default value equal to None is set. + adds ``Optional[t]`` if a default value equal to None is set. .. decorator:: no_type_check(arg) @@ -519,7 +518,7 @@ .. decorator:: no_type_check_decorator(decorator) - Decorator to give another decorator the @no_type_check effect. + Decorator to give another decorator the :func:`no_type_check` effect. This wraps the decorator with something that wraps the decorated - function in @no_type_check. + function in :func:`no_type_check`. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 10 20:56:30 2015 From: python-checkins at python.org (berker.peksag) Date: Thu, 10 Sep 2015 18:56:30 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Fix_typos_and_improve_markup_in_typing=2Erst=2E?= Message-ID: <20150910185625.114948.250@psf.io> https://hg.python.org/cpython/rev/c52194da1ae2 changeset: 97862:c52194da1ae2 parent: 97860:f41cdb99202f parent: 97861:72e696bc480a user: Berker Peksag date: Thu Sep 10 21:56:11 2015 +0300 summary: Fix typos and improve markup in typing.rst. files: Doc/library/typing.rst | 37 ++++++++++++++--------------- 1 files changed, 18 insertions(+), 19 deletions(-) diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -60,7 +60,7 @@ inferred in a generic way, abstract base classes have been extended to support subscription to denote expected types for container elements. -.. code-block:: python +:: from typing import Mapping, Sequence @@ -70,7 +70,7 @@ Generics can be parametrized by using a new factory available in typing called :class:`TypeVar`. -.. code-block:: python +:: from typing import Sequence, TypeVar @@ -85,7 +85,7 @@ A user-defined class can be defined as a generic class. -.. code-block:: python +:: from typing import TypeVar, Generic from logging import Logger @@ -220,9 +220,7 @@ Type variables exist primarily for the benefit of static type checkers. They serve as the parameters for generic types as well as for generic function definitions. See class Generic for more - information on generic types. Generic functions work as follows: - - .. code-block:: python + information on generic types. Generic functions work as follows:: def repeat(x: T, n: int) -> Sequence[T]: """Return a list containing n references to x.""" @@ -238,13 +236,13 @@ the return type is still plain :class:`str`. At runtime, ``isinstance(x, T)`` will raise :exc:`TypeError`. In general, - :func:`isinstance` and :func:`issublass` should not be used with types. + :func:`isinstance` and :func:`issubclass` should not be used with types. Type variables may be marked covariant or contravariant by passing ``covariant=True`` or ``contravariant=True``. See :pep:`484` for more details. By default type variables are invariant. Alternatively, a type variable may specify an upper bound using ``bound=``. - This means that an actual type substituted (explicitly or implictly) + This means that an actual type substituted (explicitly or implicitly) for the type variable must be a subclass of the boundary type, see :pep:`484`. @@ -278,7 +276,7 @@ * You cannot subclass or instantiate a union. - * You cannot write ``Union[X][Y]`` + * You cannot write ``Union[X][Y]``. * You can use ``Optional[X]`` as a shorthand for ``Union[X, None]``. @@ -331,6 +329,7 @@ X = TypeVar('X') Y = TypeVar('Y') + def lookup_name(mapping: Mapping[X, Y], key: X, default: Y) -> Y: try: return mapping[key] @@ -347,26 +346,26 @@ .. class:: SupportsInt - An ABC with one abstract method `__int__`. + An ABC with one abstract method ``__int__``. .. class:: SupportsFloat - An ABC with one abstract method `__float__`. + An ABC with one abstract method ``__float__``. .. class:: SupportsAbs - An ABC with one abstract method `__abs__` that is covariant + An ABC with one abstract method ``__abs__`` that is covariant in its return type. .. class:: SupportsRound - An ABC with one abstract method `__round__` + An ABC with one abstract method ``__round__`` that is covariant in its return type. .. class:: Reversible - An ABC with one abstract method `__reversed__` returning - an `Iterator[T_co]`. + An ABC with one abstract method ``__reversed__`` returning + an ``Iterator[T_co]``. .. class:: Container(Generic[T_co]) @@ -503,9 +502,9 @@ Return type hints for a function or method object. - This is often the same as obj.__annotations__, but it handles + This is often the same as ``obj.__annotations__``, but it handles forward references encoded as string literals, and if necessary - adds Optional[t] if a default value equal to None is set. + adds ``Optional[t]`` if a default value equal to None is set. .. decorator:: no_type_check(arg) @@ -519,7 +518,7 @@ .. decorator:: no_type_check_decorator(decorator) - Decorator to give another decorator the @no_type_check effect. + Decorator to give another decorator the :func:`no_type_check` effect. This wraps the decorator with something that wraps the decorated - function in @no_type_check. + function in :func:`no_type_check`. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 10 21:13:12 2015 From: python-checkins at python.org (guido.van.rossum) Date: Thu, 10 Sep 2015 19:13:12 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Add_the_original_author_of_profile=2Epy_back_to_the_docs?= =?utf-8?q?=2C_at_his_request=2E_=28Merge=29?= Message-ID: <20150910191311.66872.72019@psf.io> https://hg.python.org/cpython/rev/5980aac8be81 changeset: 97865:5980aac8be81 parent: 97862:c52194da1ae2 parent: 97864:47e9d7ddb06a user: Guido van Rossum date: Thu Sep 10 12:12:23 2015 -0700 summary: Add the original author of profile.py back to the docs, at his request. (Merge) files: Doc/library/profile.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/library/profile.rst b/Doc/library/profile.rst --- a/Doc/library/profile.rst +++ b/Doc/library/profile.rst @@ -33,7 +33,7 @@ 2. :mod:`profile`, a pure Python module whose interface is imitated by :mod:`cProfile`, but which adds significant overhead to profiled programs. If you're trying to extend the profiler in some way, the task might be easier - with this module. + with this module. Originally designed and written by Jim Roskind. .. note:: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 10 21:13:12 2015 From: python-checkins at python.org (guido.van.rossum) Date: Thu, 10 Sep 2015 19:13:12 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_Add_the_origin?= =?utf-8?q?al_author_of_profile=2Epy_back_to_the_docs=2C_at_his_request=2E?= Message-ID: <20150910191311.114754.14115@psf.io> https://hg.python.org/cpython/rev/47e9d7ddb06a changeset: 97864:47e9d7ddb06a branch: 3.5 parent: 97861:72e696bc480a user: Guido van Rossum date: Thu Sep 10 12:12:01 2015 -0700 summary: Add the original author of profile.py back to the docs, at his request. files: Doc/library/profile.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/library/profile.rst b/Doc/library/profile.rst --- a/Doc/library/profile.rst +++ b/Doc/library/profile.rst @@ -33,7 +33,7 @@ 2. :mod:`profile`, a pure Python module whose interface is imitated by :mod:`cProfile`, but which adds significant overhead to profiled programs. If you're trying to extend the profiler in some way, the task might be easier - with this module. + with this module. Originally designed and written by Jim Roskind. .. note:: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 10 21:13:12 2015 From: python-checkins at python.org (guido.van.rossum) Date: Thu, 10 Sep 2015 19:13:12 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_Add_the_origin?= =?utf-8?q?al_author_of_profile=2Epy_back_to_the_docs=2C_at_his_request=2E?= Message-ID: <20150910191310.66860.6154@psf.io> https://hg.python.org/cpython/rev/81d3e6fb1957 changeset: 97863:81d3e6fb1957 branch: 2.7 parent: 97807:32893d8a52a9 user: Guido van Rossum date: Thu Sep 10 12:11:17 2015 -0700 summary: Add the original author of profile.py back to the docs, at his request. files: Doc/library/profile.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/library/profile.rst b/Doc/library/profile.rst --- a/Doc/library/profile.rst +++ b/Doc/library/profile.rst @@ -35,7 +35,7 @@ 2. :mod:`profile`, a pure Python module whose interface is imitated by :mod:`cProfile`, but which adds significant overhead to profiled programs. If you're trying to extend the profiler in some way, the task might be easier - with this module. + with this module. Originally designed and written by Jim Roskind. .. versionchanged:: 2.4 Now also reports the time spent in calls to built-in functions -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 10 22:03:47 2015 From: python-checkins at python.org (zach.ware) Date: Thu, 10 Sep 2015 20:03:47 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Closes_=2325022=3A_Merge_with_3=2E5?= Message-ID: <20150910200347.66860.65278@psf.io> https://hg.python.org/cpython/rev/e1ddec83a311 changeset: 97869:e1ddec83a311 parent: 97865:5980aac8be81 parent: 97868:2460fa584323 user: Zachary Ware date: Thu Sep 10 15:03:02 2015 -0500 summary: Closes #25022: Merge with 3.5 files: PC/example_nt/example.c | 32 --- PC/example_nt/example.sln | 21 -- PC/example_nt/example.vcproj | 189 ----------------------- PC/example_nt/readme.txt | 183 ---------------------- PC/example_nt/setup.py | 22 -- PC/readme.txt | 2 - PCbuild/readme.txt | 8 - 7 files changed, 0 insertions(+), 457 deletions(-) diff --git a/PC/example_nt/example.c b/PC/example_nt/example.c deleted file mode 100644 --- a/PC/example_nt/example.c +++ /dev/null @@ -1,32 +0,0 @@ -#include "Python.h" - -static PyObject * -ex_foo(PyObject *self, PyObject *args) -{ - printf("Hello, world\n"); - Py_INCREF(Py_None); - return Py_None; -} - -static PyMethodDef example_methods[] = { - {"foo", ex_foo, METH_VARARGS, "foo() doc string"}, - {NULL, NULL} -}; - -static struct PyModuleDef examplemodule = { - PyModuleDef_HEAD_INIT, - "example", - "example module doc string", - -1, - example_methods, - NULL, - NULL, - NULL, - NULL -}; - -PyMODINIT_FUNC -PyInit_example(void) -{ - return PyModule_Create(&examplemodule); -} diff --git a/PC/example_nt/example.sln b/PC/example_nt/example.sln deleted file mode 100644 --- a/PC/example_nt/example.sln +++ /dev/null @@ -1,21 +0,0 @@ -Microsoft Visual Studio Solution File, Format Version 8.00 -Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "example", "example.vcproj", "{A0608D6F-84ED-44AE-A2A6-A3CC7F4A4030}" - ProjectSection(ProjectDependencies) = postProject - EndProjectSection -EndProject -Global - GlobalSection(SolutionConfiguration) = preSolution - Debug = Debug - Release = Release - EndGlobalSection - GlobalSection(ProjectConfiguration) = postSolution - {A0608D6F-84ED-44AE-A2A6-A3CC7F4A4030}.Debug.ActiveCfg = Debug|Win32 - {A0608D6F-84ED-44AE-A2A6-A3CC7F4A4030}.Debug.Build.0 = Debug|Win32 - {A0608D6F-84ED-44AE-A2A6-A3CC7F4A4030}.Release.ActiveCfg = Release|Win32 - {A0608D6F-84ED-44AE-A2A6-A3CC7F4A4030}.Release.Build.0 = Release|Win32 - EndGlobalSection - GlobalSection(ExtensibilityGlobals) = postSolution - EndGlobalSection - GlobalSection(ExtensibilityAddIns) = postSolution - EndGlobalSection -EndGlobal diff --git a/PC/example_nt/example.vcproj b/PC/example_nt/example.vcproj deleted file mode 100644 --- a/PC/example_nt/example.vcproj +++ /dev/null @@ -1,189 +0,0 @@ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - diff --git a/PC/example_nt/readme.txt b/PC/example_nt/readme.txt deleted file mode 100644 --- a/PC/example_nt/readme.txt +++ /dev/null @@ -1,183 +0,0 @@ -Example Python extension for Windows NT -======================================= - -This directory contains everything needed (except for the Python -distribution!) to build a Python extension module using Microsoft VC++. -Notice that you need to use the same compiler version that was used to build -Python itself. - -The simplest way to build this example is to use the distutils script -'setup.py'. To do this, simply execute: - - % python setup.py install - -after everything builds and installs, you can test it: - - % python -c "import example; example.foo()" - Hello, world - -See setup.py for more details. alternatively, see below for instructions on -how to build inside the Visual Studio environment. - -Visual Studio Build Instructions -================================ - -These are instructions how to build an extension using Visual C++. The -instructions and project files have not been updated to the latest VC -version. In general, it is recommended you use the 'setup.py' instructions -above. - -It has been tested with VC++ 7.1 on Python 2.4. You can also use earlier -versions of VC to build Python extensions, but the sample VC project file -(example.dsw in this directory) is in VC 7.1 format. - -COPY THIS DIRECTORY! --------------------- -This "example_nt" directory is a subdirectory of the PC directory, in order -to keep all the PC-specific files under the same directory. However, the -example_nt directory can't actually be used from this location. You first -need to copy or move it up one level, so that example_nt is a direct -sibling of the PC\ and Include\ directories. Do all your work from within -this new location -- sorry, but you'll be sorry if you don't. - -OPEN THE PROJECT ----------------- -From VC 7.1, use the - File -> Open Solution... -dialog (*not* the "File -> Open..." dialog!). Navigate to and select the -file "example.sln", in the *copy* of the example_nt directory you made -above. -Click Open. - -BUILD THE EXAMPLE DLL ---------------------- -In order to check that everything is set up right, try building: - -1. Select a configuration. This step is optional. Do - Build -> Configuration Manager... -> Active Solution Configuration - and select either "Release" or "Debug". - If you skip this step, you'll use the Debug configuration by default. - -2. Build the DLL. Do - Build -> Build Solution - This creates all intermediate and result files in a subdirectory which - is called either Debug or Release, depending on which configuration you - picked in the preceding step. - -TESTING THE DEBUG-MODE DLL --------------------------- -Once the Debug build has succeeded, bring up a DOS box, and cd to -example_nt\Debug. You should now be able to repeat the following session -("C>" is the DOS prompt, ">>>" is the Python prompt) (note that various -debug output from Python may not match this screen dump exactly): - - C>..\..\PCbuild\python_d - Adding parser accelerators ... - Done. - Python 2.2c1+ (#28, Dec 14 2001, 18:06:39) [MSC 32 bit (Intel)] on win32 - Type "help", "copyright", "credits" or "license" for more information. - >>> import example - [7052 refs] - >>> example.foo() - Hello, world - [7052 refs] - >>> - -TESTING THE RELEASE-MODE DLL ----------------------------- -Once the Release build has succeeded, bring up a DOS box, and cd to -example_nt\Release. You should now be able to repeat the following session -("C>" is the DOS prompt, ">>>" is the Python prompt): - - C>..\..\PCbuild\python - Python 2.2c1+ (#28, Dec 14 2001, 18:06:04) [MSC 32 bit (Intel)] on win32 - Type "help", "copyright", "credits" or "license" for more information. - >>> import example - >>> example.foo() - Hello, world - >>> - -Congratulations! You've successfully built your first Python extension -module. - -CREATING YOUR OWN PROJECT -------------------------- -Choose a name ("spam" is always a winner :-) and create a directory for -it. Copy your C sources into it. Note that the module source file name -does not necessarily have to match the module name, but the "init" function -name should match the module name -- i.e. you can only import a module -"spam" if its init function is called "initspam()", and it should call -Py_InitModule with the string "spam" as its first argument (use the minimal -example.c in this directory as a guide). By convention, it lives in a file -called "spam.c" or "spammodule.c". The output file should be called -"spam.dll" or "spam.pyd" (the latter is supported to avoid confusion with a -system library "spam.dll" to which your module could be a Python interface) -in Release mode, or spam_d.dll or spam_d.pyd in Debug mode. - -Now your options are: - -1) Copy example.sln and example.vcproj, rename them to spam.*, and edit them -by hand. - -or - -2) Create a brand new project; instructions are below. - -In either case, copy example_nt\example.def to spam\spam.def, and edit the -new spam.def so its second line contains the string "initspam". If you -created a new project yourself, add the file spam.def to the project now. -(This is an annoying little file with only two lines. An alternative -approach is to forget about the .def file, and add the option -"/export:initspam" somewhere to the Link settings, by manually editing the -"Project -> Properties -> Linker -> Command Line -> Additional Options" -box). - -You are now all set to build your extension, unless it requires other -external libraries, include files, etc. See Python's Extending and -Embedding manual for instructions on how to write an extension. - - -CREATING A BRAND NEW PROJECT ----------------------------- -Use the - File -> New -> Project... -dialog to create a new Project Workspace. Select "Visual C++ Projects/Win32/ -Win32 Project", enter the name ("spam"), and make sure the "Location" is -set to parent of the spam directory you have created (which should be a direct -subdirectory of the Python build tree, a sibling of Include and PC). -In "Application Settings", select "DLL", and "Empty Project". Click OK. - -You should now create the file spam.def as instructed in the previous -section. Add the source files (including the .def file) to the project, -using "Project", "Add Existing Item". - -Now open the - Project -> spam properties... -dialog. (Impressive, isn't it? :-) You only need to change a few -settings. Make sure "All Configurations" is selected from the "Settings -for:" dropdown list. Select the "C/C++" tab. Choose the "General" -category in the popup menu at the top. Type the following text in the -entry box labeled "Addditional Include Directories:" - - ..\Include,..\PC - -Then, choose the "General" category in the "Linker" tab, and enter - ..\PCbuild -in the "Additional library Directories" box. - -Now you need to add some mode-specific settings (select "Accept" -when asked to confirm your changes): - -Select "Release" in the "Configuration" dropdown list. Click the -"Link" tab, choose the "Input" Category, and append "python24.lib" to the -list in the "Additional Dependencies" box. - -Select "Debug" in the "Settings for:" dropdown list, and append -"python24_d.lib" to the list in the Additional Dependencies" box. Then -click on the C/C++ tab, select "Code Generation", and select -"Multi-threaded Debug DLL" from the "Runtime library" dropdown list. - -Select "Release" again from the "Settings for:" dropdown list. -Select "Multi-threaded DLL" from the "Use run-time library:" dropdown list. - -That's all . diff --git a/PC/example_nt/setup.py b/PC/example_nt/setup.py deleted file mode 100644 --- a/PC/example_nt/setup.py +++ /dev/null @@ -1,22 +0,0 @@ -# This is an example of a distutils 'setup' script for the example_nt -# sample. This provides a simpler way of building your extension -# and means you can avoid keeping MSVC solution files etc in source-control. -# It also means it should magically build with all compilers supported by -# python. - -# USAGE: you probably want 'setup.py install' - but execute 'setup.py --help' -# for all the details. - -# NOTE: This is *not* a sample for distutils - it is just the smallest -# script that can build this. See distutils docs for more info. - -from distutils.core import setup, Extension - -example_mod = Extension('example', sources = ['example.c']) - - -setup(name = "example", - version = "1.0", - description = "A sample extension module", - ext_modules = [example_mod], -) diff --git a/PC/readme.txt b/PC/readme.txt --- a/PC/readme.txt +++ b/PC/readme.txt @@ -71,8 +71,6 @@ dllbase_nt.txt A (manually maintained) list of base addresses for various DLLs, to avoid run-time relocation. -example_nt A subdirectory showing how to build an extension as a - DLL. Note for Windows 3.x and DOS users ================================== diff --git a/PCbuild/readme.txt b/PCbuild/readme.txt --- a/PCbuild/readme.txt +++ b/PCbuild/readme.txt @@ -298,11 +298,3 @@ doesn't always reflect the correct settings and may confuse the user with false information, especially for settings that automatically adapt for diffirent configurations. - - -Your Own Extension DLLs ------------------------ - -If you want to create your own extension module DLL (.pyd), there's an -example with easy-to-follow instructions in ..\PC\example_nt\; read the -file readme.txt there first. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 10 22:03:47 2015 From: python-checkins at python.org (zach.ware) Date: Thu, 10 Sep 2015 20:03:47 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzI1MDIy?= =?utf-8?q?=3A_Remove_PC/example=5Fnt/?= Message-ID: <20150910200347.68869.84433@psf.io> https://hg.python.org/cpython/rev/de3e1c81ecd7 changeset: 97867:de3e1c81ecd7 branch: 3.4 parent: 97858:5093d9151a18 user: Zachary Ware date: Thu Sep 10 14:37:42 2015 -0500 summary: Issue #25022: Remove PC/example_nt/ It was very much outdated, and the topic is better covered elsewhere. files: PC/example_nt/example.c | 32 --- PC/example_nt/example.sln | 21 -- PC/example_nt/example.vcproj | 189 ----------------------- PC/example_nt/readme.txt | 183 ---------------------- PC/example_nt/setup.py | 22 -- PC/readme.txt | 2 - PCbuild/readme.txt | 8 - 7 files changed, 0 insertions(+), 457 deletions(-) diff --git a/PC/example_nt/example.c b/PC/example_nt/example.c deleted file mode 100644 --- a/PC/example_nt/example.c +++ /dev/null @@ -1,32 +0,0 @@ -#include "Python.h" - -static PyObject * -ex_foo(PyObject *self, PyObject *args) -{ - printf("Hello, world\n"); - Py_INCREF(Py_None); - return Py_None; -} - -static PyMethodDef example_methods[] = { - {"foo", ex_foo, METH_VARARGS, "foo() doc string"}, - {NULL, NULL} -}; - -static struct PyModuleDef examplemodule = { - PyModuleDef_HEAD_INIT, - "example", - "example module doc string", - -1, - example_methods, - NULL, - NULL, - NULL, - NULL -}; - -PyMODINIT_FUNC -PyInit_example(void) -{ - return PyModule_Create(&examplemodule); -} diff --git a/PC/example_nt/example.sln b/PC/example_nt/example.sln deleted file mode 100644 --- a/PC/example_nt/example.sln +++ /dev/null @@ -1,21 +0,0 @@ -Microsoft Visual Studio Solution File, Format Version 8.00 -Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "example", "example.vcproj", "{A0608D6F-84ED-44AE-A2A6-A3CC7F4A4030}" - ProjectSection(ProjectDependencies) = postProject - EndProjectSection -EndProject -Global - GlobalSection(SolutionConfiguration) = preSolution - Debug = Debug - Release = Release - EndGlobalSection - GlobalSection(ProjectConfiguration) = postSolution - {A0608D6F-84ED-44AE-A2A6-A3CC7F4A4030}.Debug.ActiveCfg = Debug|Win32 - {A0608D6F-84ED-44AE-A2A6-A3CC7F4A4030}.Debug.Build.0 = Debug|Win32 - {A0608D6F-84ED-44AE-A2A6-A3CC7F4A4030}.Release.ActiveCfg = Release|Win32 - {A0608D6F-84ED-44AE-A2A6-A3CC7F4A4030}.Release.Build.0 = Release|Win32 - EndGlobalSection - GlobalSection(ExtensibilityGlobals) = postSolution - EndGlobalSection - GlobalSection(ExtensibilityAddIns) = postSolution - EndGlobalSection -EndGlobal diff --git a/PC/example_nt/example.vcproj b/PC/example_nt/example.vcproj deleted file mode 100644 --- a/PC/example_nt/example.vcproj +++ /dev/null @@ -1,189 +0,0 @@ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - diff --git a/PC/example_nt/readme.txt b/PC/example_nt/readme.txt deleted file mode 100644 --- a/PC/example_nt/readme.txt +++ /dev/null @@ -1,183 +0,0 @@ -Example Python extension for Windows NT -======================================= - -This directory contains everything needed (except for the Python -distribution!) to build a Python extension module using Microsoft VC++. -Notice that you need to use the same compiler version that was used to build -Python itself. - -The simplest way to build this example is to use the distutils script -'setup.py'. To do this, simply execute: - - % python setup.py install - -after everything builds and installs, you can test it: - - % python -c "import example; example.foo()" - Hello, world - -See setup.py for more details. alternatively, see below for instructions on -how to build inside the Visual Studio environment. - -Visual Studio Build Instructions -================================ - -These are instructions how to build an extension using Visual C++. The -instructions and project files have not been updated to the latest VC -version. In general, it is recommended you use the 'setup.py' instructions -above. - -It has been tested with VC++ 7.1 on Python 2.4. You can also use earlier -versions of VC to build Python extensions, but the sample VC project file -(example.dsw in this directory) is in VC 7.1 format. - -COPY THIS DIRECTORY! --------------------- -This "example_nt" directory is a subdirectory of the PC directory, in order -to keep all the PC-specific files under the same directory. However, the -example_nt directory can't actually be used from this location. You first -need to copy or move it up one level, so that example_nt is a direct -sibling of the PC\ and Include\ directories. Do all your work from within -this new location -- sorry, but you'll be sorry if you don't. - -OPEN THE PROJECT ----------------- -From VC 7.1, use the - File -> Open Solution... -dialog (*not* the "File -> Open..." dialog!). Navigate to and select the -file "example.sln", in the *copy* of the example_nt directory you made -above. -Click Open. - -BUILD THE EXAMPLE DLL ---------------------- -In order to check that everything is set up right, try building: - -1. Select a configuration. This step is optional. Do - Build -> Configuration Manager... -> Active Solution Configuration - and select either "Release" or "Debug". - If you skip this step, you'll use the Debug configuration by default. - -2. Build the DLL. Do - Build -> Build Solution - This creates all intermediate and result files in a subdirectory which - is called either Debug or Release, depending on which configuration you - picked in the preceding step. - -TESTING THE DEBUG-MODE DLL --------------------------- -Once the Debug build has succeeded, bring up a DOS box, and cd to -example_nt\Debug. You should now be able to repeat the following session -("C>" is the DOS prompt, ">>>" is the Python prompt) (note that various -debug output from Python may not match this screen dump exactly): - - C>..\..\PCbuild\python_d - Adding parser accelerators ... - Done. - Python 2.2c1+ (#28, Dec 14 2001, 18:06:39) [MSC 32 bit (Intel)] on win32 - Type "help", "copyright", "credits" or "license" for more information. - >>> import example - [7052 refs] - >>> example.foo() - Hello, world - [7052 refs] - >>> - -TESTING THE RELEASE-MODE DLL ----------------------------- -Once the Release build has succeeded, bring up a DOS box, and cd to -example_nt\Release. You should now be able to repeat the following session -("C>" is the DOS prompt, ">>>" is the Python prompt): - - C>..\..\PCbuild\python - Python 2.2c1+ (#28, Dec 14 2001, 18:06:04) [MSC 32 bit (Intel)] on win32 - Type "help", "copyright", "credits" or "license" for more information. - >>> import example - >>> example.foo() - Hello, world - >>> - -Congratulations! You've successfully built your first Python extension -module. - -CREATING YOUR OWN PROJECT -------------------------- -Choose a name ("spam" is always a winner :-) and create a directory for -it. Copy your C sources into it. Note that the module source file name -does not necessarily have to match the module name, but the "init" function -name should match the module name -- i.e. you can only import a module -"spam" if its init function is called "initspam()", and it should call -Py_InitModule with the string "spam" as its first argument (use the minimal -example.c in this directory as a guide). By convention, it lives in a file -called "spam.c" or "spammodule.c". The output file should be called -"spam.dll" or "spam.pyd" (the latter is supported to avoid confusion with a -system library "spam.dll" to which your module could be a Python interface) -in Release mode, or spam_d.dll or spam_d.pyd in Debug mode. - -Now your options are: - -1) Copy example.sln and example.vcproj, rename them to spam.*, and edit them -by hand. - -or - -2) Create a brand new project; instructions are below. - -In either case, copy example_nt\example.def to spam\spam.def, and edit the -new spam.def so its second line contains the string "initspam". If you -created a new project yourself, add the file spam.def to the project now. -(This is an annoying little file with only two lines. An alternative -approach is to forget about the .def file, and add the option -"/export:initspam" somewhere to the Link settings, by manually editing the -"Project -> Properties -> Linker -> Command Line -> Additional Options" -box). - -You are now all set to build your extension, unless it requires other -external libraries, include files, etc. See Python's Extending and -Embedding manual for instructions on how to write an extension. - - -CREATING A BRAND NEW PROJECT ----------------------------- -Use the - File -> New -> Project... -dialog to create a new Project Workspace. Select "Visual C++ Projects/Win32/ -Win32 Project", enter the name ("spam"), and make sure the "Location" is -set to parent of the spam directory you have created (which should be a direct -subdirectory of the Python build tree, a sibling of Include and PC). -In "Application Settings", select "DLL", and "Empty Project". Click OK. - -You should now create the file spam.def as instructed in the previous -section. Add the source files (including the .def file) to the project, -using "Project", "Add Existing Item". - -Now open the - Project -> spam properties... -dialog. (Impressive, isn't it? :-) You only need to change a few -settings. Make sure "All Configurations" is selected from the "Settings -for:" dropdown list. Select the "C/C++" tab. Choose the "General" -category in the popup menu at the top. Type the following text in the -entry box labeled "Addditional Include Directories:" - - ..\Include,..\PC - -Then, choose the "General" category in the "Linker" tab, and enter - ..\PCbuild -in the "Additional library Directories" box. - -Now you need to add some mode-specific settings (select "Accept" -when asked to confirm your changes): - -Select "Release" in the "Configuration" dropdown list. Click the -"Link" tab, choose the "Input" Category, and append "python24.lib" to the -list in the "Additional Dependencies" box. - -Select "Debug" in the "Settings for:" dropdown list, and append -"python24_d.lib" to the list in the Additional Dependencies" box. Then -click on the C/C++ tab, select "Code Generation", and select -"Multi-threaded Debug DLL" from the "Runtime library" dropdown list. - -Select "Release" again from the "Settings for:" dropdown list. -Select "Multi-threaded DLL" from the "Use run-time library:" dropdown list. - -That's all . diff --git a/PC/example_nt/setup.py b/PC/example_nt/setup.py deleted file mode 100644 --- a/PC/example_nt/setup.py +++ /dev/null @@ -1,22 +0,0 @@ -# This is an example of a distutils 'setup' script for the example_nt -# sample. This provides a simpler way of building your extension -# and means you can avoid keeping MSVC solution files etc in source-control. -# It also means it should magically build with all compilers supported by -# python. - -# USAGE: you probably want 'setup.py install' - but execute 'setup.py --help' -# for all the details. - -# NOTE: This is *not* a sample for distutils - it is just the smallest -# script that can build this. See distutils docs for more info. - -from distutils.core import setup, Extension - -example_mod = Extension('example', sources = ['example.c']) - - -setup(name = "example", - version = "1.0", - description = "A sample extension module", - ext_modules = [example_mod], -) diff --git a/PC/readme.txt b/PC/readme.txt --- a/PC/readme.txt +++ b/PC/readme.txt @@ -71,8 +71,6 @@ dllbase_nt.txt A (manually maintained) list of base addresses for various DLLs, to avoid run-time relocation. -example_nt A subdirectory showing how to build an extension as a - DLL. Legacy support for older versions of Visual Studio ================================================== diff --git a/PCbuild/readme.txt b/PCbuild/readme.txt --- a/PCbuild/readme.txt +++ b/PCbuild/readme.txt @@ -312,11 +312,3 @@ _M_X64 although the macros are set by the compiler, too. The GUI doesn't always know about the macros and confuse the user with false information. - - -Your Own Extension DLLs ------------------------ - -If you want to create your own extension module DLL (.pyd), there's an -example with easy-to-follow instructions in ..\PC\example_nt\; read the -file readme.txt there first. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 10 22:03:47 2015 From: python-checkins at python.org (zach.ware) Date: Thu, 10 Sep 2015 20:03:47 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzI1MDIy?= =?utf-8?q?=3A_Remove_PC/example=5Fnt/?= Message-ID: <20150910200346.68867.83710@psf.io> https://hg.python.org/cpython/rev/a30598e59964 changeset: 97866:a30598e59964 branch: 2.7 parent: 97863:81d3e6fb1957 user: Zachary Ware date: Thu Sep 10 14:37:42 2015 -0500 summary: Issue #25022: Remove PC/example_nt/ It was very much outdated, and the topic is better covered elsewhere. files: PC/VS9.0/readme.txt | 7 - PC/example_nt/example.c | 20 -- PC/example_nt/example.sln | 21 -- PC/example_nt/example.vcproj | 189 ----------------------- PC/example_nt/readme.txt | 183 ---------------------- PC/example_nt/setup.py | 22 -- PC/readme.txt | 2 - PCbuild/readme.txt | 8 - 8 files changed, 0 insertions(+), 452 deletions(-) diff --git a/PC/VS9.0/readme.txt b/PC/VS9.0/readme.txt --- a/PC/VS9.0/readme.txt +++ b/PC/VS9.0/readme.txt @@ -251,10 +251,3 @@ The pyproject propertyfile defines _WIN32 and x64 defines _WIN64 and _M_X64 although the macros are set by the compiler, too. The GUI doesn't always know about the macros and confuse the user with false information. - -YOUR OWN EXTENSION DLLs ------------------------ - -If you want to create your own extension module DLL, there's an example -with easy-to-follow instructions in PC/example/; read the file -readme.txt there first. diff --git a/PC/example_nt/example.c b/PC/example_nt/example.c deleted file mode 100644 --- a/PC/example_nt/example.c +++ /dev/null @@ -1,20 +0,0 @@ -#include "Python.h" - -static PyObject * -ex_foo(PyObject *self, PyObject *args) -{ - printf("Hello, world\n"); - Py_INCREF(Py_None); - return Py_None; -} - -static PyMethodDef example_methods[] = { - {"foo", ex_foo, METH_VARARGS, "foo() doc string"}, - {NULL, NULL} -}; - -PyMODINIT_FUNC -initexample(void) -{ - Py_InitModule("example", example_methods); -} diff --git a/PC/example_nt/example.sln b/PC/example_nt/example.sln deleted file mode 100644 --- a/PC/example_nt/example.sln +++ /dev/null @@ -1,21 +0,0 @@ -Microsoft Visual Studio Solution File, Format Version 8.00 -Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "example", "example.vcproj", "{A0608D6F-84ED-44AE-A2A6-A3CC7F4A4030}" - ProjectSection(ProjectDependencies) = postProject - EndProjectSection -EndProject -Global - GlobalSection(SolutionConfiguration) = preSolution - Debug = Debug - Release = Release - EndGlobalSection - GlobalSection(ProjectConfiguration) = postSolution - {A0608D6F-84ED-44AE-A2A6-A3CC7F4A4030}.Debug.ActiveCfg = Debug|Win32 - {A0608D6F-84ED-44AE-A2A6-A3CC7F4A4030}.Debug.Build.0 = Debug|Win32 - {A0608D6F-84ED-44AE-A2A6-A3CC7F4A4030}.Release.ActiveCfg = Release|Win32 - {A0608D6F-84ED-44AE-A2A6-A3CC7F4A4030}.Release.Build.0 = Release|Win32 - EndGlobalSection - GlobalSection(ExtensibilityGlobals) = postSolution - EndGlobalSection - GlobalSection(ExtensibilityAddIns) = postSolution - EndGlobalSection -EndGlobal diff --git a/PC/example_nt/example.vcproj b/PC/example_nt/example.vcproj deleted file mode 100644 --- a/PC/example_nt/example.vcproj +++ /dev/null @@ -1,189 +0,0 @@ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - diff --git a/PC/example_nt/readme.txt b/PC/example_nt/readme.txt deleted file mode 100644 --- a/PC/example_nt/readme.txt +++ /dev/null @@ -1,183 +0,0 @@ -Example Python extension for Windows NT -======================================= - -This directory contains everything needed (except for the Python -distribution!) to build a Python extension module using Microsoft VC++. -Notice that you need to use the same compiler version that was used to build -Python itself. - -The simplest way to build this example is to use the distutils script -'setup.py'. To do this, simply execute: - - % python setup.py install - -after everything builds and installs, you can test it: - - % python -c "import example; example.foo()" - Hello, world - -See setup.py for more details. alternatively, see below for instructions on -how to build inside the Visual Studio environment. - -Visual Studio Build Instructions -================================ - -These are instructions how to build an extension using Visual C++. The -instructions and project files have not been updated to the latest VC -version. In general, it is recommended you use the 'setup.py' instructions -above. - -It has been tested with VC++ 7.1 on Python 2.4. You can also use earlier -versions of VC to build Python extensions, but the sample VC project file -(example.dsw in this directory) is in VC 7.1 format. - -COPY THIS DIRECTORY! --------------------- -This "example_nt" directory is a subdirectory of the PC directory, in order -to keep all the PC-specific files under the same directory. However, the -example_nt directory can't actually be used from this location. You first -need to copy or move it up one level, so that example_nt is a direct -sibling of the PC\ and Include\ directories. Do all your work from within -this new location -- sorry, but you'll be sorry if you don't. - -OPEN THE PROJECT ----------------- -From VC 7.1, use the - File -> Open Solution... -dialog (*not* the "File -> Open..." dialog!). Navigate to and select the -file "example.sln", in the *copy* of the example_nt directory you made -above. -Click Open. - -BUILD THE EXAMPLE DLL ---------------------- -In order to check that everything is set up right, try building: - -1. Select a configuration. This step is optional. Do - Build -> Configuration Manager... -> Active Solution Configuration - and select either "Release" or "Debug". - If you skip this step, you'll use the Debug configuration by default. - -2. Build the DLL. Do - Build -> Build Solution - This creates all intermediate and result files in a subdirectory which - is called either Debug or Release, depending on which configuration you - picked in the preceding step. - -TESTING THE DEBUG-MODE DLL --------------------------- -Once the Debug build has succeeded, bring up a DOS box, and cd to -example_nt\Debug. You should now be able to repeat the following session -("C>" is the DOS prompt, ">>>" is the Python prompt) (note that various -debug output from Python may not match this screen dump exactly): - - C>..\..\PCbuild\python_d - Adding parser accelerators ... - Done. - Python 2.2c1+ (#28, Dec 14 2001, 18:06:39) [MSC 32 bit (Intel)] on win32 - Type "help", "copyright", "credits" or "license" for more information. - >>> import example - [7052 refs] - >>> example.foo() - Hello, world - [7052 refs] - >>> - -TESTING THE RELEASE-MODE DLL ----------------------------- -Once the Release build has succeeded, bring up a DOS box, and cd to -example_nt\Release. You should now be able to repeat the following session -("C>" is the DOS prompt, ">>>" is the Python prompt): - - C>..\..\PCbuild\python - Python 2.2c1+ (#28, Dec 14 2001, 18:06:04) [MSC 32 bit (Intel)] on win32 - Type "help", "copyright", "credits" or "license" for more information. - >>> import example - >>> example.foo() - Hello, world - >>> - -Congratulations! You've successfully built your first Python extension -module. - -CREATING YOUR OWN PROJECT -------------------------- -Choose a name ("spam" is always a winner :-) and create a directory for -it. Copy your C sources into it. Note that the module source file name -does not necessarily have to match the module name, but the "init" function -name should match the module name -- i.e. you can only import a module -"spam" if its init function is called "initspam()", and it should call -Py_InitModule with the string "spam" as its first argument (use the minimal -example.c in this directory as a guide). By convention, it lives in a file -called "spam.c" or "spammodule.c". The output file should be called -"spam.dll" or "spam.pyd" (the latter is supported to avoid confusion with a -system library "spam.dll" to which your module could be a Python interface) -in Release mode, or spam_d.dll or spam_d.pyd in Debug mode. - -Now your options are: - -1) Copy example.sln and example.vcproj, rename them to spam.*, and edit them -by hand. - -or - -2) Create a brand new project; instructions are below. - -In either case, copy example_nt\example.def to spam\spam.def, and edit the -new spam.def so its second line contains the string "initspam". If you -created a new project yourself, add the file spam.def to the project now. -(This is an annoying little file with only two lines. An alternative -approach is to forget about the .def file, and add the option -"/export:initspam" somewhere to the Link settings, by manually editing the -"Project -> Properties -> Linker -> Command Line -> Additional Options" -box). - -You are now all set to build your extension, unless it requires other -external libraries, include files, etc. See Python's Extending and -Embedding manual for instructions on how to write an extension. - - -CREATING A BRAND NEW PROJECT ----------------------------- -Use the - File -> New -> Project... -dialog to create a new Project Workspace. Select "Visual C++ Projects/Win32/ -Win32 Project", enter the name ("spam"), and make sure the "Location" is -set to parent of the spam directory you have created (which should be a direct -subdirectory of the Python build tree, a sibling of Include and PC). -In "Application Settings", select "DLL", and "Empty Project". Click OK. - -You should now create the file spam.def as instructed in the previous -section. Add the source files (including the .def file) to the project, -using "Project", "Add Existing Item". - -Now open the - Project -> spam properties... -dialog. (Impressive, isn't it? :-) You only need to change a few -settings. Make sure "All Configurations" is selected from the "Settings -for:" dropdown list. Select the "C/C++" tab. Choose the "General" -category in the popup menu at the top. Type the following text in the -entry box labeled "Addditional Include Directories:" - - ..\Include,..\PC - -Then, choose the "General" category in the "Linker" tab, and enter - ..\PCbuild -in the "Additional library Directories" box. - -Now you need to add some mode-specific settings (select "Accept" -when asked to confirm your changes): - -Select "Release" in the "Configuration" dropdown list. Click the -"Link" tab, choose the "Input" Category, and append "python24.lib" to the -list in the "Additional Dependencies" box. - -Select "Debug" in the "Settings for:" dropdown list, and append -"python24_d.lib" to the list in the Additional Dependencies" box. Then -click on the C/C++ tab, select "Code Generation", and select -"Multi-threaded Debug DLL" from the "Runtime library" dropdown list. - -Select "Release" again from the "Settings for:" dropdown list. -Select "Multi-threaded DLL" from the "Use run-time library:" dropdown list. - -That's all . diff --git a/PC/example_nt/setup.py b/PC/example_nt/setup.py deleted file mode 100644 --- a/PC/example_nt/setup.py +++ /dev/null @@ -1,22 +0,0 @@ -# This is an example of a distutils 'setup' script for the example_nt -# sample. This provides a simpler way of building your extension -# and means you can avoid keeping MSVC solution files etc in source-control. -# It also means it should magically build with all compilers supported by -# python. - -# USAGE: you probably want 'setup.py install' - but execute 'setup.py --help' -# for all the details. - -# NOTE: This is *not* a sample for distutils - it is just the smallest -# script that can build this. See distutils docs for more info. - -from distutils.core import setup, Extension - -example_mod = Extension('example', sources = ['example.c']) - - -setup(name = "example", - version = "1.0", - description = "A sample extension module", - ext_modules = [example_mod], -) diff --git a/PC/readme.txt b/PC/readme.txt --- a/PC/readme.txt +++ b/PC/readme.txt @@ -71,8 +71,6 @@ dllbase_nt.txt A (manually maintained) list of base addresses for various DLLs, to avoid run-time relocation. -example_nt A subdirectory showing how to build an extension as a - DLL. Legacy support for older versions of Visual Studio ================================================== diff --git a/PCbuild/readme.txt b/PCbuild/readme.txt --- a/PCbuild/readme.txt +++ b/PCbuild/readme.txt @@ -312,11 +312,3 @@ doesn't always reflect the correct settings and may confuse the user with false information, especially for settings that automatically adapt for diffirent configurations. - - -Your Own Extension DLLs ------------------------ - -If you want to create your own extension module DLL (.pyd), there's an -example with easy-to-follow instructions in ..\PC\example_nt\; read the -file readme.txt there first. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 10 22:03:47 2015 From: python-checkins at python.org (zach.ware) Date: Thu, 10 Sep 2015 20:03:47 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Issue_=2325022=3A_Merge_with_3=2E4?= Message-ID: <20150910200347.15718.42786@psf.io> https://hg.python.org/cpython/rev/2460fa584323 changeset: 97868:2460fa584323 branch: 3.5 parent: 97864:47e9d7ddb06a parent: 97867:de3e1c81ecd7 user: Zachary Ware date: Thu Sep 10 15:02:14 2015 -0500 summary: Issue #25022: Merge with 3.4 files: PC/example_nt/example.c | 32 --- PC/example_nt/example.sln | 21 -- PC/example_nt/example.vcproj | 189 ----------------------- PC/example_nt/readme.txt | 183 ---------------------- PC/example_nt/setup.py | 22 -- PC/readme.txt | 2 - PCbuild/readme.txt | 8 - 7 files changed, 0 insertions(+), 457 deletions(-) diff --git a/PC/example_nt/example.c b/PC/example_nt/example.c deleted file mode 100644 --- a/PC/example_nt/example.c +++ /dev/null @@ -1,32 +0,0 @@ -#include "Python.h" - -static PyObject * -ex_foo(PyObject *self, PyObject *args) -{ - printf("Hello, world\n"); - Py_INCREF(Py_None); - return Py_None; -} - -static PyMethodDef example_methods[] = { - {"foo", ex_foo, METH_VARARGS, "foo() doc string"}, - {NULL, NULL} -}; - -static struct PyModuleDef examplemodule = { - PyModuleDef_HEAD_INIT, - "example", - "example module doc string", - -1, - example_methods, - NULL, - NULL, - NULL, - NULL -}; - -PyMODINIT_FUNC -PyInit_example(void) -{ - return PyModule_Create(&examplemodule); -} diff --git a/PC/example_nt/example.sln b/PC/example_nt/example.sln deleted file mode 100644 --- a/PC/example_nt/example.sln +++ /dev/null @@ -1,21 +0,0 @@ -Microsoft Visual Studio Solution File, Format Version 8.00 -Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "example", "example.vcproj", "{A0608D6F-84ED-44AE-A2A6-A3CC7F4A4030}" - ProjectSection(ProjectDependencies) = postProject - EndProjectSection -EndProject -Global - GlobalSection(SolutionConfiguration) = preSolution - Debug = Debug - Release = Release - EndGlobalSection - GlobalSection(ProjectConfiguration) = postSolution - {A0608D6F-84ED-44AE-A2A6-A3CC7F4A4030}.Debug.ActiveCfg = Debug|Win32 - {A0608D6F-84ED-44AE-A2A6-A3CC7F4A4030}.Debug.Build.0 = Debug|Win32 - {A0608D6F-84ED-44AE-A2A6-A3CC7F4A4030}.Release.ActiveCfg = Release|Win32 - {A0608D6F-84ED-44AE-A2A6-A3CC7F4A4030}.Release.Build.0 = Release|Win32 - EndGlobalSection - GlobalSection(ExtensibilityGlobals) = postSolution - EndGlobalSection - GlobalSection(ExtensibilityAddIns) = postSolution - EndGlobalSection -EndGlobal diff --git a/PC/example_nt/example.vcproj b/PC/example_nt/example.vcproj deleted file mode 100644 --- a/PC/example_nt/example.vcproj +++ /dev/null @@ -1,189 +0,0 @@ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - diff --git a/PC/example_nt/readme.txt b/PC/example_nt/readme.txt deleted file mode 100644 --- a/PC/example_nt/readme.txt +++ /dev/null @@ -1,183 +0,0 @@ -Example Python extension for Windows NT -======================================= - -This directory contains everything needed (except for the Python -distribution!) to build a Python extension module using Microsoft VC++. -Notice that you need to use the same compiler version that was used to build -Python itself. - -The simplest way to build this example is to use the distutils script -'setup.py'. To do this, simply execute: - - % python setup.py install - -after everything builds and installs, you can test it: - - % python -c "import example; example.foo()" - Hello, world - -See setup.py for more details. alternatively, see below for instructions on -how to build inside the Visual Studio environment. - -Visual Studio Build Instructions -================================ - -These are instructions how to build an extension using Visual C++. The -instructions and project files have not been updated to the latest VC -version. In general, it is recommended you use the 'setup.py' instructions -above. - -It has been tested with VC++ 7.1 on Python 2.4. You can also use earlier -versions of VC to build Python extensions, but the sample VC project file -(example.dsw in this directory) is in VC 7.1 format. - -COPY THIS DIRECTORY! --------------------- -This "example_nt" directory is a subdirectory of the PC directory, in order -to keep all the PC-specific files under the same directory. However, the -example_nt directory can't actually be used from this location. You first -need to copy or move it up one level, so that example_nt is a direct -sibling of the PC\ and Include\ directories. Do all your work from within -this new location -- sorry, but you'll be sorry if you don't. - -OPEN THE PROJECT ----------------- -From VC 7.1, use the - File -> Open Solution... -dialog (*not* the "File -> Open..." dialog!). Navigate to and select the -file "example.sln", in the *copy* of the example_nt directory you made -above. -Click Open. - -BUILD THE EXAMPLE DLL ---------------------- -In order to check that everything is set up right, try building: - -1. Select a configuration. This step is optional. Do - Build -> Configuration Manager... -> Active Solution Configuration - and select either "Release" or "Debug". - If you skip this step, you'll use the Debug configuration by default. - -2. Build the DLL. Do - Build -> Build Solution - This creates all intermediate and result files in a subdirectory which - is called either Debug or Release, depending on which configuration you - picked in the preceding step. - -TESTING THE DEBUG-MODE DLL --------------------------- -Once the Debug build has succeeded, bring up a DOS box, and cd to -example_nt\Debug. You should now be able to repeat the following session -("C>" is the DOS prompt, ">>>" is the Python prompt) (note that various -debug output from Python may not match this screen dump exactly): - - C>..\..\PCbuild\python_d - Adding parser accelerators ... - Done. - Python 2.2c1+ (#28, Dec 14 2001, 18:06:39) [MSC 32 bit (Intel)] on win32 - Type "help", "copyright", "credits" or "license" for more information. - >>> import example - [7052 refs] - >>> example.foo() - Hello, world - [7052 refs] - >>> - -TESTING THE RELEASE-MODE DLL ----------------------------- -Once the Release build has succeeded, bring up a DOS box, and cd to -example_nt\Release. You should now be able to repeat the following session -("C>" is the DOS prompt, ">>>" is the Python prompt): - - C>..\..\PCbuild\python - Python 2.2c1+ (#28, Dec 14 2001, 18:06:04) [MSC 32 bit (Intel)] on win32 - Type "help", "copyright", "credits" or "license" for more information. - >>> import example - >>> example.foo() - Hello, world - >>> - -Congratulations! You've successfully built your first Python extension -module. - -CREATING YOUR OWN PROJECT -------------------------- -Choose a name ("spam" is always a winner :-) and create a directory for -it. Copy your C sources into it. Note that the module source file name -does not necessarily have to match the module name, but the "init" function -name should match the module name -- i.e. you can only import a module -"spam" if its init function is called "initspam()", and it should call -Py_InitModule with the string "spam" as its first argument (use the minimal -example.c in this directory as a guide). By convention, it lives in a file -called "spam.c" or "spammodule.c". The output file should be called -"spam.dll" or "spam.pyd" (the latter is supported to avoid confusion with a -system library "spam.dll" to which your module could be a Python interface) -in Release mode, or spam_d.dll or spam_d.pyd in Debug mode. - -Now your options are: - -1) Copy example.sln and example.vcproj, rename them to spam.*, and edit them -by hand. - -or - -2) Create a brand new project; instructions are below. - -In either case, copy example_nt\example.def to spam\spam.def, and edit the -new spam.def so its second line contains the string "initspam". If you -created a new project yourself, add the file spam.def to the project now. -(This is an annoying little file with only two lines. An alternative -approach is to forget about the .def file, and add the option -"/export:initspam" somewhere to the Link settings, by manually editing the -"Project -> Properties -> Linker -> Command Line -> Additional Options" -box). - -You are now all set to build your extension, unless it requires other -external libraries, include files, etc. See Python's Extending and -Embedding manual for instructions on how to write an extension. - - -CREATING A BRAND NEW PROJECT ----------------------------- -Use the - File -> New -> Project... -dialog to create a new Project Workspace. Select "Visual C++ Projects/Win32/ -Win32 Project", enter the name ("spam"), and make sure the "Location" is -set to parent of the spam directory you have created (which should be a direct -subdirectory of the Python build tree, a sibling of Include and PC). -In "Application Settings", select "DLL", and "Empty Project". Click OK. - -You should now create the file spam.def as instructed in the previous -section. Add the source files (including the .def file) to the project, -using "Project", "Add Existing Item". - -Now open the - Project -> spam properties... -dialog. (Impressive, isn't it? :-) You only need to change a few -settings. Make sure "All Configurations" is selected from the "Settings -for:" dropdown list. Select the "C/C++" tab. Choose the "General" -category in the popup menu at the top. Type the following text in the -entry box labeled "Addditional Include Directories:" - - ..\Include,..\PC - -Then, choose the "General" category in the "Linker" tab, and enter - ..\PCbuild -in the "Additional library Directories" box. - -Now you need to add some mode-specific settings (select "Accept" -when asked to confirm your changes): - -Select "Release" in the "Configuration" dropdown list. Click the -"Link" tab, choose the "Input" Category, and append "python24.lib" to the -list in the "Additional Dependencies" box. - -Select "Debug" in the "Settings for:" dropdown list, and append -"python24_d.lib" to the list in the Additional Dependencies" box. Then -click on the C/C++ tab, select "Code Generation", and select -"Multi-threaded Debug DLL" from the "Runtime library" dropdown list. - -Select "Release" again from the "Settings for:" dropdown list. -Select "Multi-threaded DLL" from the "Use run-time library:" dropdown list. - -That's all . diff --git a/PC/example_nt/setup.py b/PC/example_nt/setup.py deleted file mode 100644 --- a/PC/example_nt/setup.py +++ /dev/null @@ -1,22 +0,0 @@ -# This is an example of a distutils 'setup' script for the example_nt -# sample. This provides a simpler way of building your extension -# and means you can avoid keeping MSVC solution files etc in source-control. -# It also means it should magically build with all compilers supported by -# python. - -# USAGE: you probably want 'setup.py install' - but execute 'setup.py --help' -# for all the details. - -# NOTE: This is *not* a sample for distutils - it is just the smallest -# script that can build this. See distutils docs for more info. - -from distutils.core import setup, Extension - -example_mod = Extension('example', sources = ['example.c']) - - -setup(name = "example", - version = "1.0", - description = "A sample extension module", - ext_modules = [example_mod], -) diff --git a/PC/readme.txt b/PC/readme.txt --- a/PC/readme.txt +++ b/PC/readme.txt @@ -71,8 +71,6 @@ dllbase_nt.txt A (manually maintained) list of base addresses for various DLLs, to avoid run-time relocation. -example_nt A subdirectory showing how to build an extension as a - DLL. Note for Windows 3.x and DOS users ================================== diff --git a/PCbuild/readme.txt b/PCbuild/readme.txt --- a/PCbuild/readme.txt +++ b/PCbuild/readme.txt @@ -298,11 +298,3 @@ doesn't always reflect the correct settings and may confuse the user with false information, especially for settings that automatically adapt for diffirent configurations. - - -Your Own Extension DLLs ------------------------ - -If you want to create your own extension module DLL (.pyd), there's an -example with easy-to-follow instructions in ..\PC\example_nt\; read the -file readme.txt there first. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 10 23:14:50 2015 From: python-checkins at python.org (zach.ware) Date: Thu, 10 Sep 2015 21:14:50 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Issue_=2325022=3A_Merge_with_3=2E4?= Message-ID: <20150910211449.27683.38734@psf.io> https://hg.python.org/cpython/rev/70c97a626c41 changeset: 97872:70c97a626c41 branch: 3.5 parent: 97868:2460fa584323 parent: 97871:c535bf72aa60 user: Zachary Ware date: Thu Sep 10 16:08:21 2015 -0500 summary: Issue #25022: Merge with 3.4 files: Doc/extending/windows.rst | 146 +------------------------- Misc/NEWS | 5 + 2 files changed, 8 insertions(+), 143 deletions(-) diff --git a/Doc/extending/windows.rst b/Doc/extending/windows.rst --- a/Doc/extending/windows.rst +++ b/Doc/extending/windows.rst @@ -37,149 +37,9 @@ are on Unix: use the :mod:`distutils` package to control the build process, or do things manually. The distutils approach works well for most extensions; documentation on using :mod:`distutils` to build and package extension modules -is available in :ref:`distutils-index`. This section describes the manual -approach to building Python extensions written in C or C++. - -To build extensions using these instructions, you need to have a copy of the -Python sources of the same version as your installed Python. You will need -Microsoft Visual C++ "Developer Studio"; project files are supplied for VC++ -version 7.1, but you can use older versions of VC++. Notice that you should use -the same version of VC++that was used to build Python itself. The example files -described here are distributed with the Python sources in the -:file:`PC\\example_nt\\` directory. - -#. **Copy the example files** --- The :file:`example_nt` directory is a - subdirectory of the :file:`PC` directory, in order to keep all the PC-specific - files under the same directory in the source distribution. However, the - :file:`example_nt` directory can't actually be used from this location. You - first need to copy or move it up one level, so that :file:`example_nt` is a - sibling of the :file:`PC` and :file:`Include` directories. Do all your work - from within this new location. - -#. **Open the project** --- From VC++, use the :menuselection:`File --> Open - Solution` dialog (not :menuselection:`File --> Open`!). Navigate to and select - the file :file:`example.sln`, in the *copy* of the :file:`example_nt` directory - you made above. Click Open. - -#. **Build the example DLL** --- In order to check that everything is set up - right, try building: - -#. Select a configuration. This step is optional. Choose - :menuselection:`Build --> Configuration Manager --> Active Solution Configuration` - and select either :guilabel:`Release` or :guilabel:`Debug`. If you skip this - step, VC++ will use the Debug configuration by default. - -#. Build the DLL. Choose :menuselection:`Build --> Build Solution`. This - creates all intermediate and result files in a subdirectory called either - :file:`Debug` or :file:`Release`, depending on which configuration you selected - in the preceding step. - -#. **Testing the debug-mode DLL** --- Once the Debug build has succeeded, bring - up a DOS box, and change to the :file:`example_nt\\Debug` directory. You should - now be able to repeat the following session (``C>`` is the DOS prompt, ``>>>`` - is the Python prompt; note that build information and various debug output from - Python may not match this screen dump exactly):: - - C>..\..\PCbuild\python_d - Adding parser accelerators ... - Done. - Python 2.2 (#28, Dec 19 2001, 23:26:37) [MSC 32 bit (Intel)] on win32 - Type "copyright", "credits" or "license" for more information. - >>> import example - [4897 refs] - >>> example.foo() - Hello, world - [4903 refs] - >>> - - Congratulations! You've successfully built your first Python extension module. - -#. **Creating your own project** --- Choose a name and create a directory for - it. Copy your C sources into it. Note that the module source file name does - not necessarily have to match the module name, but the name of the - initialization function should match the module name --- you can only import a - module :mod:`spam` if its initialization function is called :c:func:`PyInit_spam`, - (see :ref:`building`, or use the minimal :file:`Modules/xxmodule.c` as a guide). - By convention, it lives in a file called :file:`spam.c` or :file:`spammodule.c`. - The output file should be called :file:`spam.pyd` (in Release mode) or - :file:`spam_d.pyd` (in Debug mode). The extension :file:`.pyd` was chosen - to avoid confusion with a system library :file:`spam.dll` to which your module - could be a Python interface. - - Now your options are: - -#. Copy :file:`example.sln` and :file:`example.vcproj`, rename them to - :file:`spam.\*`, and edit them by hand, or - -#. Create a brand new project; instructions are below. - - In either case, copy :file:`example_nt\\example.def` to :file:`spam\\spam.def`, - and edit the new :file:`spam.def` so its second line contains the string - '``initspam``'. If you created a new project yourself, add the file - :file:`spam.def` to the project now. (This is an annoying little file with only - two lines. An alternative approach is to forget about the :file:`.def` file, - and add the option :option:`/export:initspam` somewhere to the Link settings, by - manually editing the setting in Project Properties dialog). - -#. **Creating a brand new project** --- Use the :menuselection:`File --> New - --> Project` dialog to create a new Project Workspace. Select :guilabel:`Visual - C++ Projects/Win32/ Win32 Project`, enter the name (``spam``), and make sure the - Location is set to parent of the :file:`spam` directory you have created (which - should be a direct subdirectory of the Python build tree, a sibling of - :file:`Include` and :file:`PC`). Select Win32 as the platform (in my version, - this is the only choice). Make sure the Create new workspace radio button is - selected. Click OK. - - You should now create the file :file:`spam.def` as instructed in the previous - section. Add the source files to the project, using :menuselection:`Project --> - Add Existing Item`. Set the pattern to ``*.*`` and select both :file:`spam.c` - and :file:`spam.def` and click OK. (Inserting them one by one is fine too.) - - Now open the :menuselection:`Project --> spam properties` dialog. You only need - to change a few settings. Make sure :guilabel:`All Configurations` is selected - from the :guilabel:`Settings for:` dropdown list. Select the C/C++ tab. Choose - the General category in the popup menu at the top. Type the following text in - the entry box labeled :guilabel:`Additional Include Directories`:: - - ..\Include,..\PC - - Then, choose the General category in the Linker tab, and enter :: - - ..\PCbuild - - in the text box labelled :guilabel:`Additional library Directories`. - - Now you need to add some mode-specific settings: - - Select :guilabel:`Release` in the :guilabel:`Configuration` dropdown list. - Choose the :guilabel:`Link` tab, choose the :guilabel:`Input` category, and - append ``pythonXY.lib`` to the list in the :guilabel:`Additional Dependencies` - box. - - Select :guilabel:`Debug` in the :guilabel:`Configuration` dropdown list, and - append ``pythonXY_d.lib`` to the list in the :guilabel:`Additional Dependencies` - box. Then click the C/C++ tab, select :guilabel:`Code Generation`, and select - :guilabel:`Multi-threaded Debug DLL` from the :guilabel:`Runtime library` - dropdown list. - - Select :guilabel:`Release` again from the :guilabel:`Configuration` dropdown - list. Select :guilabel:`Multi-threaded DLL` from the :guilabel:`Runtime - library` dropdown list. - -If your module creates a new type, you may have trouble with this line:: - - PyVarObject_HEAD_INIT(&PyType_Type, 0) - -Static type object initializers in extension modules may cause -compiles to fail with an error message like "initializer not a -constant". This shows up when building DLL under MSVC. Change it to:: - - PyVarObject_HEAD_INIT(NULL, 0) - -and add the following to the module initialization function:: - - if (PyType_Ready(&MyObject_Type) < 0) - return NULL; +is available in :ref:`distutils-index`. If you find you really need to do +things manually, it may be instructive to study the project file for the +:source:`winsound ` standard library module. .. _dynamic-linking: diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -91,6 +91,11 @@ - Issue #24986: It is now possible to build Python on Windows without errors when external libraries are not available. +Windows +------- + +- Issue #25022: Removed very outdated PC/example_nt/ directory. + What's New in Python 3.5.0 release candidate 4? =============================================== -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 10 23:14:44 2015 From: python-checkins at python.org (zach.ware) Date: Thu, 10 Sep 2015 21:14:44 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzI1MDIy?= =?utf-8?q?=3A_Add_NEWS=2C_fix_docs_to_not_mention_the_old_example=2E?= Message-ID: <20150910211444.14881.8459@psf.io> https://hg.python.org/cpython/rev/3456af022541 changeset: 97870:3456af022541 branch: 2.7 parent: 97866:a30598e59964 user: Zachary Ware date: Thu Sep 10 15:50:58 2015 -0500 summary: Issue #25022: Add NEWS, fix docs to not mention the old example. files: Doc/extending/windows.rst | 151 +------------------------- Misc/NEWS | 5 + 2 files changed, 8 insertions(+), 148 deletions(-) diff --git a/Doc/extending/windows.rst b/Doc/extending/windows.rst --- a/Doc/extending/windows.rst +++ b/Doc/extending/windows.rst @@ -37,154 +37,9 @@ are on Unix: use the :mod:`distutils` package to control the build process, or do things manually. The distutils approach works well for most extensions; documentation on using :mod:`distutils` to build and package extension modules -is available in :ref:`distutils-index`. This section describes the manual -approach to building Python extensions written in C or C++. - -To build extensions using these instructions, you need to have a copy of the -Python sources of the same version as your installed Python. You will need -Microsoft Visual C++ "Developer Studio"; project files are supplied for VC++ -version 7.1, but you can use older versions of VC++. Notice that you should use -the same version of VC++that was used to build Python itself. The example files -described here are distributed with the Python sources in the -:file:`PC\\example_nt\\` directory. - -#. **Copy the example files** --- The :file:`example_nt` directory is a - subdirectory of the :file:`PC` directory, in order to keep all the PC-specific - files under the same directory in the source distribution. However, the - :file:`example_nt` directory can't actually be used from this location. You - first need to copy or move it up one level, so that :file:`example_nt` is a - sibling of the :file:`PC` and :file:`Include` directories. Do all your work - from within this new location. - -#. **Open the project** --- From VC++, use the :menuselection:`File --> Open - Solution` dialog (not :menuselection:`File --> Open`!). Navigate to and select - the file :file:`example.sln`, in the *copy* of the :file:`example_nt` directory - you made above. Click Open. - -#. **Build the example DLL** --- In order to check that everything is set up - right, try building: - -#. Select a configuration. This step is optional. Choose - :menuselection:`Build --> Configuration Manager --> Active Solution Configuration` - and select either :guilabel:`Release` or :guilabel:`Debug`. If you skip this - step, VC++ will use the Debug configuration by default. - -#. Build the DLL. Choose :menuselection:`Build --> Build Solution`. This - creates all intermediate and result files in a subdirectory called either - :file:`Debug` or :file:`Release`, depending on which configuration you selected - in the preceding step. - -#. **Testing the debug-mode DLL** --- Once the Debug build has succeeded, bring - up a DOS box, and change to the :file:`example_nt\\Debug` directory. You should - now be able to repeat the following session (``C>`` is the DOS prompt, ``>>>`` - is the Python prompt; note that build information and various debug output from - Python may not match this screen dump exactly):: - - C>..\..\PCbuild\python_d - Adding parser accelerators ... - Done. - Python 2.2 (#28, Dec 19 2001, 23:26:37) [MSC 32 bit (Intel)] on win32 - Type "copyright", "credits" or "license" for more information. - >>> import example - [4897 refs] - >>> example.foo() - Hello, world - [4903 refs] - >>> - - Congratulations! You've successfully built your first Python extension module. - -#. **Creating your own project** --- Choose a name and create a directory for - it. Copy your C sources into it. Note that the module source file name does - not necessarily have to match the module name, but the name of the - initialization function should match the module name --- you can only import a - module :mod:`spam` if its initialization function is called :c:func:`initspam`, - and it should call :c:func:`Py_InitModule` with the string ``"spam"`` as its - first argument (use the minimal :file:`example.c` in this directory as a guide). - By convention, it lives in a file called :file:`spam.c` or :file:`spammodule.c`. - The output file should be called :file:`spam.pyd` (in Release mode) or - :file:`spam_d.pyd` (in Debug mode). The extension :file:`.pyd` was chosen - to avoid confusion with a system library :file:`spam.dll` to which your module - could be a Python interface. - - .. versionchanged:: 2.5 - Previously, file names like :file:`spam.dll` (in release mode) or - :file:`spam_d.dll` (in debug mode) were also recognized. - - Now your options are: - -#. Copy :file:`example.sln` and :file:`example.vcproj`, rename them to - :file:`spam.\*`, and edit them by hand, or - -#. Create a brand new project; instructions are below. - - In either case, copy :file:`example_nt\\example.def` to :file:`spam\\spam.def`, - and edit the new :file:`spam.def` so its second line contains the string - '``initspam``'. If you created a new project yourself, add the file - :file:`spam.def` to the project now. (This is an annoying little file with only - two lines. An alternative approach is to forget about the :file:`.def` file, - and add the option :option:`/export:initspam` somewhere to the Link settings, by - manually editing the setting in Project Properties dialog). - -#. **Creating a brand new project** --- Use the :menuselection:`File --> New - --> Project` dialog to create a new Project Workspace. Select :guilabel:`Visual - C++ Projects/Win32/ Win32 Project`, enter the name (``spam``), and make sure the - Location is set to parent of the :file:`spam` directory you have created (which - should be a direct subdirectory of the Python build tree, a sibling of - :file:`Include` and :file:`PC`). Select Win32 as the platform (in my version, - this is the only choice). Make sure the Create new workspace radio button is - selected. Click OK. - - You should now create the file :file:`spam.def` as instructed in the previous - section. Add the source files to the project, using :menuselection:`Project --> - Add Existing Item`. Set the pattern to ``*.*`` and select both :file:`spam.c` - and :file:`spam.def` and click OK. (Inserting them one by one is fine too.) - - Now open the :menuselection:`Project --> spam properties` dialog. You only need - to change a few settings. Make sure :guilabel:`All Configurations` is selected - from the :guilabel:`Settings for:` dropdown list. Select the C/C++ tab. Choose - the General category in the popup menu at the top. Type the following text in - the entry box labeled :guilabel:`Additional Include Directories`:: - - ..\Include,..\PC - - Then, choose the General category in the Linker tab, and enter :: - - ..\PCbuild - - in the text box labelled :guilabel:`Additional library Directories`. - - Now you need to add some mode-specific settings: - - Select :guilabel:`Release` in the :guilabel:`Configuration` dropdown list. - Choose the :guilabel:`Link` tab, choose the :guilabel:`Input` category, and - append ``pythonXY.lib`` to the list in the :guilabel:`Additional Dependencies` - box. - - Select :guilabel:`Debug` in the :guilabel:`Configuration` dropdown list, and - append ``pythonXY_d.lib`` to the list in the :guilabel:`Additional Dependencies` - box. Then click the C/C++ tab, select :guilabel:`Code Generation`, and select - :guilabel:`Multi-threaded Debug DLL` from the :guilabel:`Runtime library` - dropdown list. - - Select :guilabel:`Release` again from the :guilabel:`Configuration` dropdown - list. Select :guilabel:`Multi-threaded DLL` from the :guilabel:`Runtime - library` dropdown list. - -If your module creates a new type, you may have trouble with this line:: - - PyObject_HEAD_INIT(&PyType_Type) - -Static type object initializers in extension modules may cause -compiles to fail with an error message like "initializer not a -constant". This shows up when building DLL under MSVC. Change it to:: - - PyObject_HEAD_INIT(NULL) - -and add the following to the module initialization function:: - - if (PyType_Ready(&MyObject_Type) < 0) - return NULL; +is available in :ref:`distutils-index`. If you find you really need to do +things manually, it may be instructive to study the project file for the +:source:`winsound ` standard library module. .. _dynamic-linking: diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -196,6 +196,11 @@ - PCbuild\rt.bat now accepts an unlimited number of arguments to pass along to regrtest.py. Previously there was a limit of 9. +Windows +------- + +- Issue #25022: Removed very outdated PC/example_nt/ directory. + What's New in Python 2.7.10? ============================ -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 10 23:14:51 2015 From: python-checkins at python.org (zach.ware) Date: Thu, 10 Sep 2015 21:14:51 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Closes_=2325022_=28again=29=3A_Merge_with_3=2E5?= Message-ID: <20150910211449.66864.2737@psf.io> https://hg.python.org/cpython/rev/8c436c36a4a7 changeset: 97873:8c436c36a4a7 parent: 97869:e1ddec83a311 parent: 97872:70c97a626c41 user: Zachary Ware date: Thu Sep 10 16:12:48 2015 -0500 summary: Closes #25022 (again): Merge with 3.5 files: Doc/extending/windows.rst | 146 +------------------------- Misc/NEWS | 5 + 2 files changed, 8 insertions(+), 143 deletions(-) diff --git a/Doc/extending/windows.rst b/Doc/extending/windows.rst --- a/Doc/extending/windows.rst +++ b/Doc/extending/windows.rst @@ -37,149 +37,9 @@ are on Unix: use the :mod:`distutils` package to control the build process, or do things manually. The distutils approach works well for most extensions; documentation on using :mod:`distutils` to build and package extension modules -is available in :ref:`distutils-index`. This section describes the manual -approach to building Python extensions written in C or C++. - -To build extensions using these instructions, you need to have a copy of the -Python sources of the same version as your installed Python. You will need -Microsoft Visual C++ "Developer Studio"; project files are supplied for VC++ -version 7.1, but you can use older versions of VC++. Notice that you should use -the same version of VC++that was used to build Python itself. The example files -described here are distributed with the Python sources in the -:file:`PC\\example_nt\\` directory. - -#. **Copy the example files** --- The :file:`example_nt` directory is a - subdirectory of the :file:`PC` directory, in order to keep all the PC-specific - files under the same directory in the source distribution. However, the - :file:`example_nt` directory can't actually be used from this location. You - first need to copy or move it up one level, so that :file:`example_nt` is a - sibling of the :file:`PC` and :file:`Include` directories. Do all your work - from within this new location. - -#. **Open the project** --- From VC++, use the :menuselection:`File --> Open - Solution` dialog (not :menuselection:`File --> Open`!). Navigate to and select - the file :file:`example.sln`, in the *copy* of the :file:`example_nt` directory - you made above. Click Open. - -#. **Build the example DLL** --- In order to check that everything is set up - right, try building: - -#. Select a configuration. This step is optional. Choose - :menuselection:`Build --> Configuration Manager --> Active Solution Configuration` - and select either :guilabel:`Release` or :guilabel:`Debug`. If you skip this - step, VC++ will use the Debug configuration by default. - -#. Build the DLL. Choose :menuselection:`Build --> Build Solution`. This - creates all intermediate and result files in a subdirectory called either - :file:`Debug` or :file:`Release`, depending on which configuration you selected - in the preceding step. - -#. **Testing the debug-mode DLL** --- Once the Debug build has succeeded, bring - up a DOS box, and change to the :file:`example_nt\\Debug` directory. You should - now be able to repeat the following session (``C>`` is the DOS prompt, ``>>>`` - is the Python prompt; note that build information and various debug output from - Python may not match this screen dump exactly):: - - C>..\..\PCbuild\python_d - Adding parser accelerators ... - Done. - Python 2.2 (#28, Dec 19 2001, 23:26:37) [MSC 32 bit (Intel)] on win32 - Type "copyright", "credits" or "license" for more information. - >>> import example - [4897 refs] - >>> example.foo() - Hello, world - [4903 refs] - >>> - - Congratulations! You've successfully built your first Python extension module. - -#. **Creating your own project** --- Choose a name and create a directory for - it. Copy your C sources into it. Note that the module source file name does - not necessarily have to match the module name, but the name of the - initialization function should match the module name --- you can only import a - module :mod:`spam` if its initialization function is called :c:func:`PyInit_spam`, - (see :ref:`building`, or use the minimal :file:`Modules/xxmodule.c` as a guide). - By convention, it lives in a file called :file:`spam.c` or :file:`spammodule.c`. - The output file should be called :file:`spam.pyd` (in Release mode) or - :file:`spam_d.pyd` (in Debug mode). The extension :file:`.pyd` was chosen - to avoid confusion with a system library :file:`spam.dll` to which your module - could be a Python interface. - - Now your options are: - -#. Copy :file:`example.sln` and :file:`example.vcproj`, rename them to - :file:`spam.\*`, and edit them by hand, or - -#. Create a brand new project; instructions are below. - - In either case, copy :file:`example_nt\\example.def` to :file:`spam\\spam.def`, - and edit the new :file:`spam.def` so its second line contains the string - '``initspam``'. If you created a new project yourself, add the file - :file:`spam.def` to the project now. (This is an annoying little file with only - two lines. An alternative approach is to forget about the :file:`.def` file, - and add the option :option:`/export:initspam` somewhere to the Link settings, by - manually editing the setting in Project Properties dialog). - -#. **Creating a brand new project** --- Use the :menuselection:`File --> New - --> Project` dialog to create a new Project Workspace. Select :guilabel:`Visual - C++ Projects/Win32/ Win32 Project`, enter the name (``spam``), and make sure the - Location is set to parent of the :file:`spam` directory you have created (which - should be a direct subdirectory of the Python build tree, a sibling of - :file:`Include` and :file:`PC`). Select Win32 as the platform (in my version, - this is the only choice). Make sure the Create new workspace radio button is - selected. Click OK. - - You should now create the file :file:`spam.def` as instructed in the previous - section. Add the source files to the project, using :menuselection:`Project --> - Add Existing Item`. Set the pattern to ``*.*`` and select both :file:`spam.c` - and :file:`spam.def` and click OK. (Inserting them one by one is fine too.) - - Now open the :menuselection:`Project --> spam properties` dialog. You only need - to change a few settings. Make sure :guilabel:`All Configurations` is selected - from the :guilabel:`Settings for:` dropdown list. Select the C/C++ tab. Choose - the General category in the popup menu at the top. Type the following text in - the entry box labeled :guilabel:`Additional Include Directories`:: - - ..\Include,..\PC - - Then, choose the General category in the Linker tab, and enter :: - - ..\PCbuild - - in the text box labelled :guilabel:`Additional library Directories`. - - Now you need to add some mode-specific settings: - - Select :guilabel:`Release` in the :guilabel:`Configuration` dropdown list. - Choose the :guilabel:`Link` tab, choose the :guilabel:`Input` category, and - append ``pythonXY.lib`` to the list in the :guilabel:`Additional Dependencies` - box. - - Select :guilabel:`Debug` in the :guilabel:`Configuration` dropdown list, and - append ``pythonXY_d.lib`` to the list in the :guilabel:`Additional Dependencies` - box. Then click the C/C++ tab, select :guilabel:`Code Generation`, and select - :guilabel:`Multi-threaded Debug DLL` from the :guilabel:`Runtime library` - dropdown list. - - Select :guilabel:`Release` again from the :guilabel:`Configuration` dropdown - list. Select :guilabel:`Multi-threaded DLL` from the :guilabel:`Runtime - library` dropdown list. - -If your module creates a new type, you may have trouble with this line:: - - PyVarObject_HEAD_INIT(&PyType_Type, 0) - -Static type object initializers in extension modules may cause -compiles to fail with an error message like "initializer not a -constant". This shows up when building DLL under MSVC. Change it to:: - - PyVarObject_HEAD_INIT(NULL, 0) - -and add the following to the module initialization function:: - - if (PyType_Ready(&MyObject_Type) < 0) - return NULL; +is available in :ref:`distutils-index`. If you find you really need to do +things manually, it may be instructive to study the project file for the +:source:`winsound ` standard library module. .. _dynamic-linking: diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -86,6 +86,11 @@ - Issue #24986: It is now possible to build Python on Windows without errors when external libraries are not available. +Windows +------- + +- Issue #25022: Removed very outdated PC/example_nt/ directory. + What's New in Python 3.5.1 ========================== -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 10 23:14:49 2015 From: python-checkins at python.org (zach.ware) Date: Thu, 10 Sep 2015 21:14:49 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzI1MDIy?= =?utf-8?q?=3A_Add_NEWS=2C_fix_docs_to_not_mention_the_old_example=2E?= Message-ID: <20150910211449.15726.17057@psf.io> https://hg.python.org/cpython/rev/c535bf72aa60 changeset: 97871:c535bf72aa60 branch: 3.4 parent: 97867:de3e1c81ecd7 user: Zachary Ware date: Thu Sep 10 15:50:58 2015 -0500 summary: Issue #25022: Add NEWS, fix docs to not mention the old example. files: Doc/extending/windows.rst | 147 +------------------------- Misc/NEWS | 5 + 2 files changed, 8 insertions(+), 144 deletions(-) diff --git a/Doc/extending/windows.rst b/Doc/extending/windows.rst --- a/Doc/extending/windows.rst +++ b/Doc/extending/windows.rst @@ -37,150 +37,9 @@ are on Unix: use the :mod:`distutils` package to control the build process, or do things manually. The distutils approach works well for most extensions; documentation on using :mod:`distutils` to build and package extension modules -is available in :ref:`distutils-index`. This section describes the manual -approach to building Python extensions written in C or C++. - -To build extensions using these instructions, you need to have a copy of the -Python sources of the same version as your installed Python. You will need -Microsoft Visual C++ "Developer Studio"; project files are supplied for VC++ -version 7.1, but you can use older versions of VC++. Notice that you should use -the same version of VC++that was used to build Python itself. The example files -described here are distributed with the Python sources in the -:file:`PC\\example_nt\\` directory. - -#. **Copy the example files** --- The :file:`example_nt` directory is a - subdirectory of the :file:`PC` directory, in order to keep all the PC-specific - files under the same directory in the source distribution. However, the - :file:`example_nt` directory can't actually be used from this location. You - first need to copy or move it up one level, so that :file:`example_nt` is a - sibling of the :file:`PC` and :file:`Include` directories. Do all your work - from within this new location. - -#. **Open the project** --- From VC++, use the :menuselection:`File --> Open - Solution` dialog (not :menuselection:`File --> Open`!). Navigate to and select - the file :file:`example.sln`, in the *copy* of the :file:`example_nt` directory - you made above. Click Open. - -#. **Build the example DLL** --- In order to check that everything is set up - right, try building: - -#. Select a configuration. This step is optional. Choose - :menuselection:`Build --> Configuration Manager --> Active Solution Configuration` - and select either :guilabel:`Release` or :guilabel:`Debug`. If you skip this - step, VC++ will use the Debug configuration by default. - -#. Build the DLL. Choose :menuselection:`Build --> Build Solution`. This - creates all intermediate and result files in a subdirectory called either - :file:`Debug` or :file:`Release`, depending on which configuration you selected - in the preceding step. - -#. **Testing the debug-mode DLL** --- Once the Debug build has succeeded, bring - up a DOS box, and change to the :file:`example_nt\\Debug` directory. You should - now be able to repeat the following session (``C>`` is the DOS prompt, ``>>>`` - is the Python prompt; note that build information and various debug output from - Python may not match this screen dump exactly):: - - C>..\..\PCbuild\python_d - Adding parser accelerators ... - Done. - Python 2.2 (#28, Dec 19 2001, 23:26:37) [MSC 32 bit (Intel)] on win32 - Type "copyright", "credits" or "license" for more information. - >>> import example - [4897 refs] - >>> example.foo() - Hello, world - [4903 refs] - >>> - - Congratulations! You've successfully built your first Python extension module. - -#. **Creating your own project** --- Choose a name and create a directory for - it. Copy your C sources into it. Note that the module source file name does - not necessarily have to match the module name, but the name of the - initialization function should match the module name --- you can only import a - module :mod:`spam` if its initialization function is called :c:func:`initspam`, - and it should call :c:func:`Py_InitModule` with the string ``"spam"`` as its - first argument (use the minimal :file:`example.c` in this directory as a guide). - By convention, it lives in a file called :file:`spam.c` or :file:`spammodule.c`. - The output file should be called :file:`spam.pyd` (in Release mode) or - :file:`spam_d.pyd` (in Debug mode). The extension :file:`.pyd` was chosen - to avoid confusion with a system library :file:`spam.dll` to which your module - could be a Python interface. - - Now your options are: - -#. Copy :file:`example.sln` and :file:`example.vcproj`, rename them to - :file:`spam.\*`, and edit them by hand, or - -#. Create a brand new project; instructions are below. - - In either case, copy :file:`example_nt\\example.def` to :file:`spam\\spam.def`, - and edit the new :file:`spam.def` so its second line contains the string - '``initspam``'. If you created a new project yourself, add the file - :file:`spam.def` to the project now. (This is an annoying little file with only - two lines. An alternative approach is to forget about the :file:`.def` file, - and add the option :option:`/export:initspam` somewhere to the Link settings, by - manually editing the setting in Project Properties dialog). - -#. **Creating a brand new project** --- Use the :menuselection:`File --> New - --> Project` dialog to create a new Project Workspace. Select :guilabel:`Visual - C++ Projects/Win32/ Win32 Project`, enter the name (``spam``), and make sure the - Location is set to parent of the :file:`spam` directory you have created (which - should be a direct subdirectory of the Python build tree, a sibling of - :file:`Include` and :file:`PC`). Select Win32 as the platform (in my version, - this is the only choice). Make sure the Create new workspace radio button is - selected. Click OK. - - You should now create the file :file:`spam.def` as instructed in the previous - section. Add the source files to the project, using :menuselection:`Project --> - Add Existing Item`. Set the pattern to ``*.*`` and select both :file:`spam.c` - and :file:`spam.def` and click OK. (Inserting them one by one is fine too.) - - Now open the :menuselection:`Project --> spam properties` dialog. You only need - to change a few settings. Make sure :guilabel:`All Configurations` is selected - from the :guilabel:`Settings for:` dropdown list. Select the C/C++ tab. Choose - the General category in the popup menu at the top. Type the following text in - the entry box labeled :guilabel:`Additional Include Directories`:: - - ..\Include,..\PC - - Then, choose the General category in the Linker tab, and enter :: - - ..\PCbuild - - in the text box labelled :guilabel:`Additional library Directories`. - - Now you need to add some mode-specific settings: - - Select :guilabel:`Release` in the :guilabel:`Configuration` dropdown list. - Choose the :guilabel:`Link` tab, choose the :guilabel:`Input` category, and - append ``pythonXY.lib`` to the list in the :guilabel:`Additional Dependencies` - box. - - Select :guilabel:`Debug` in the :guilabel:`Configuration` dropdown list, and - append ``pythonXY_d.lib`` to the list in the :guilabel:`Additional Dependencies` - box. Then click the C/C++ tab, select :guilabel:`Code Generation`, and select - :guilabel:`Multi-threaded Debug DLL` from the :guilabel:`Runtime library` - dropdown list. - - Select :guilabel:`Release` again from the :guilabel:`Configuration` dropdown - list. Select :guilabel:`Multi-threaded DLL` from the :guilabel:`Runtime - library` dropdown list. - -If your module creates a new type, you may have trouble with this line:: - - PyVarObject_HEAD_INIT(&PyType_Type, 0) - -Static type object initializers in extension modules may cause -compiles to fail with an error message like "initializer not a -constant". This shows up when building DLL under MSVC. Change it to:: - - PyVarObject_HEAD_INIT(NULL, 0) - -and add the following to the module initialization function:: - - if (PyType_Ready(&MyObject_Type) < 0) - return NULL; +is available in :ref:`distutils-index`. If you find you really need to do +things manually, it may be instructive to study the project file for the +:source:`winsound ` standard library module. .. _dynamic-linking: diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -517,6 +517,11 @@ - Issue #24031: make patchcheck now supports git checkouts, too. +Windows +------- + +- Issue #25022: Removed very outdated PC/example_nt/ directory. + What's New in Python 3.4.3? =========================== -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 10 23:36:09 2015 From: python-checkins at python.org (yury.selivanov) Date: Thu, 10 Sep 2015 21:36:09 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogd2hhdHNuZXcvMy41?= =?utf-8?q?=3A_Editorialization_pass_on_library_section?= Message-ID: <20150910213608.11999.12766@psf.io> https://hg.python.org/cpython/rev/3265f33df731 changeset: 97874:3265f33df731 branch: 3.5 parent: 97872:70c97a626c41 user: Yury Selivanov date: Thu Sep 10 17:35:38 2015 -0400 summary: whatsnew/3.5: Editorialization pass on library section Patch by Elvis Pranskevichus files: Doc/whatsnew/3.5.rst | 944 +++++++++++++++++------------- 1 files changed, 548 insertions(+), 396 deletions(-) diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -69,8 +69,8 @@ New syntax features: +* :pep:`492`, coroutines with async and await syntax. * :pep:`465`, a new matrix multiplication operator: ``a @ b``. -* :pep:`492`, coroutines with async and await syntax. * :pep:`448`, additional unpacking generalizations. New library modules: @@ -94,22 +94,24 @@ * New :exc:`RecursionError` exception. (Contributed by Georg Brandl in :issue:`19235`.) -Implementation improvements: +CPython implementation improvements: * When the ``LC_TYPE`` locale is the POSIX locale (``C`` locale), :py:data:`sys.stdin` and :py:data:`sys.stdout` are now using the ``surrogateescape`` error handler, instead of the ``strict`` error handler (:issue:`19977`). -* :pep:`488`, the elimination of ``.pyo`` files. +* ``.pyo`` files are no longer used and have been replaced by a more flexible + scheme that inclides the optimization level explicitly in ``.pyc`` name. + (:pep:`488`) -* :pep:`489`, multi-phase initialization of extension modules. +* Builtin and extension modules are now initialized in a multi-phase process, + which is similar to how Python modules are loaded. (:pep:`489`). Significantly Improved Library Modules: -* :class:`collections.OrderedDict` is now implemented in C, which improves - its performance between 4x to 100x times. Contributed by Eric Snow in - :issue:`16991`. +* :class:`collections.OrderedDict` is now implemented in C, which makes it + 4 to 100 times faster. Contributed by Eric Snow in :issue:`16991`. * You may now pass bytes to the :mod:`tempfile` module's APIs and it will return the temporary pathname as :class:`bytes` instead of :class:`str`. @@ -146,8 +148,8 @@ Windows improvements: -* A new installer for Windows has replaced the old MSI. See :ref:`using-on-windows` - for more information. +* A new installer for Windows has replaced the old MSI. + See :ref:`using-on-windows` for more information. * Windows builds now use Microsoft Visual C++ 14.0, and extension modules should use the same. @@ -252,6 +254,8 @@ PEP written and implemented by Yury Selivanov. +.. _whatsnew-pep-465: + PEP 465 - A dedicated infix operator for matrix multiplication -------------------------------------------------------------- @@ -297,6 +301,8 @@ PEP written by Nathaniel J. Smith; implemented by Benjamin Peterson. +.. _whatsnew-pep-448: + PEP 448 - Additional Unpacking Generalizations ---------------------------------------------- @@ -333,6 +339,8 @@ Thomas Wouters, and Joshua Landau. +.. _whatsnew-pep-461: + PEP 461 - % formatting support for bytes and bytearray ------------------------------------------------------ @@ -364,6 +372,8 @@ Ethan Furman. +.. _whatsnew-pep-484: + PEP 484 - Type Hints -------------------- @@ -389,11 +399,13 @@ implemented by Guido van Rossum. +.. _whatsnew-pep-471: + PEP 471 - os.scandir() function -- a better and faster directory iterator ------------------------------------------------------------------------- :pep:`471` adds a new directory iteration function, :func:`os.scandir`, -to the standard library. Additionally, :func:`os.walk` is now +to the standard library. Additionally, :func:`os.walk` is now implemented using :func:`os.scandir`, which speeds it up by 3-5 times on POSIX systems and by 7-20 times on Windows systems. @@ -403,6 +415,8 @@ PEP written and implemented by Ben Hoyt with the help of Victor Stinner. +.. _whatsnew-pep-475: + PEP 475: Retry system calls failing with EINTR ---------------------------------------------- @@ -455,6 +469,8 @@ Victor Stinner, with the help of Antoine Pitrou (the french connection). +.. _whatsnew-pep-479: + PEP 479: Change StopIteration handling inside generators -------------------------------------------------------- @@ -475,6 +491,8 @@ Chris Angelico, Yury Selivanov and Nick Coghlan. +.. _whatsnew-pep-486: + PEP 486: Make the Python Launcher aware of virtual environments --------------------------------------------------------------- @@ -489,6 +507,8 @@ PEP written and implemented by Paul Moore. +.. _whatsnew-pep-488: + PEP 488: Elimination of PYO files --------------------------------- @@ -497,9 +517,10 @@ need to constantly regenerate bytecode files, ``.pyc`` files now have an optional ``opt-`` tag in their name when the bytecode is optimized. This has the side-effect of no more bytecode file name clashes when running under either -``-O`` or ``-OO``. Consequently, bytecode files generated from ``-O``, and -``-OO`` may now exist simultaneously. :func:`importlib.util.cache_from_source` -has an updated API to help with this change. +:option:`-O` or :option:`-OO`. Consequently, bytecode files generated from +:option:`-O`, and :option:`-OO` may now exist simultaneously. +:func:`importlib.util.cache_from_source` has an updated API to help with +this change. .. seealso:: @@ -507,6 +528,8 @@ PEP written and implemented by Brett Cannon. +.. _whatsnew-pep-489: + PEP 489: Multi-phase extension module initialization ---------------------------------------------------- @@ -525,6 +548,8 @@ implemented by Petr Viktorin. +.. _whatsnew-pep-485: + PEP 485: A function for testing approximate equality ---------------------------------------------------- @@ -576,8 +601,8 @@ The new :mod:`zipapp` module (specified in :pep:`441`) provides an API and command line tool for creating executable Python Zip Applications, which -were introduced in Python 2.6 in :issue:`1739468` but which were not well -publicised, either at the time or since. +were introduced in Python 2.6 in :issue:`1739468`, but which were not well +publicized, either at the time or since. With the new module, bundling your application is as simple as putting all the files, including a ``__main__.py`` file, into a directory ``myapp`` @@ -586,6 +611,13 @@ $ python -m zipapp myapp $ python myapp.pyz +The module implementation has been contributed by Paul Moore in +:issue:`23491`. + +.. seealso:: + + :pep:`441` -- Improving Python ZIP Application Support + Improved Modules ================ @@ -593,191 +625,209 @@ argparse -------- -* :class:`~argparse.ArgumentParser` now allows to disable - :ref:`abbreviated usage ` of long options by setting - :ref:`allow_abbrev` to ``False``. - (Contributed by Jonathan Paugh, Steven Bethard, paul j3 and Daniel Eriksson - in :issue:`14910`.) +:class:`~argparse.ArgumentParser` now allows to disable +:ref:`abbreviated usage ` of long options by setting +:ref:`allow_abbrev` to ``False``. (Contributed by Jonathan Paugh, +Steven Bethard, paul j3 and Daniel Eriksson in :issue:`14910`.) + bz2 --- -* New option *max_length* for :meth:`~bz2.BZ2Decompressor.decompress` - to limit the maximum size of decompressed data. - (Contributed by Nikolaus Rath in :issue:`15955`.) +:meth:`~bz2.BZ2Decompressor.decompress` now accepts an optional *max_length* +argument to limit the maximum size of decompressed data. (Contributed by +Nikolaus Rath in :issue:`15955`.) + cgi --- -* :class:`~cgi.FieldStorage` now supports the context management protocol. - (Contributed by Berker Peksag in :issue:`20289`.) +:class:`~cgi.FieldStorage` now supports the context management protocol. +(Contributed by Berker Peksag in :issue:`20289`.) + cmath ----- -* :func:`cmath.isclose` function added. - (Contributed by Chris Barker and Tal Einat in :issue:`24270`.) +A new function :func:`cmath.isclose` provides a way to test for approximate +equality. (Contributed by Chris Barker and Tal Einat in :issue:`24270`.) code ---- -* The :func:`code.InteractiveInterpreter.showtraceback` method now prints - the full chained traceback, just like the interactive interpreter. - (Contributed by Claudiu Popa in :issue:`17442`.) +The :func:`code.InteractiveInterpreter.showtraceback` method now prints +the full chained traceback, just like the interactive interpreter. +(Contributed by Claudiu Popa in :issue:`17442`.) + collections ----------- -* You can now update docstrings produced by :func:`collections.namedtuple`:: +Docstrings produced by :func:`collections.namedtuple` can now be updated:: Point = namedtuple('Point', ['x', 'y']) Point.__doc__ = 'ordered pair' Point.x.__doc__ = 'abscissa' Point.y.__doc__ = 'ordinate' - (Contributed by Berker Peksag in :issue:`24064`.) +(Contributed by Berker Peksag in :issue:`24064`.) -* :class:`~collections.deque` has new methods: - :meth:`~collections.deque.index`, :meth:`~collections.deque.insert`, and - :meth:`~collections.deque.copy`. This allows deques to be registered as a - :class:`~collections.abc.MutableSequence` and improves their - substitutablity for lists. - (Contributed by Raymond Hettinger :issue:`23704`.) +The :class:`~collections.deque` now defines +:meth:`~collections.deque.index`, :meth:`~collections.deque.insert`, and +:meth:`~collections.deque.copy`. This allows deques to be recognized as a +:class:`~collections.abc.MutableSequence` and improves their substitutablity +for lists. (Contributed by Raymond Hettinger :issue:`23704`.) -* :class:`~collections.UserString` now supports ``__getnewargs__``, - ``__rmod__``, ``casefold``, ``format_map``, ``isprintable``, and - ``maketrans`` methods. - (Contributed by Joe Jevnik in :issue:`22189`.) +:class:`~collections.UserString` now implements :meth:`__getnewargs__`, +:meth:`__rmod__`, :meth:`casefold`, :meth:`format_map`, :meth:`isprintable`, and +:meth:`maketrans` methods to match corresponding methods of :class:`str`. +(Contributed by Joe Jevnik in :issue:`22189`.) -* :class:`collections.OrderedDict` is now implemented in C, which improves - its performance between 4x to 100x times. - (Contributed by Eric Snow in :issue:`16991`.) +:class:`collections.OrderedDict` is now implemented in C, which makes it 4x to +100x faster. (Contributed by Eric Snow in :issue:`16991`.) + collections.abc --------------- -* New :class:`~collections.abc.Generator` abstract base class. - (Contributed by Stefan Behnel in :issue:`24018`.) +New :class:`~collections.abc.Generator` abstract base class. (Contributed +by Stefan Behnel in :issue:`24018`.) -* New :class:`~collections.abc.Coroutine`, - :class:`~collections.abc.AsyncIterator`, and - :class:`~collections.abc.AsyncIterable` abstract base classes. - (Contributed by Yury Selivanov in :issue:`24184`.) +New :class:`~collections.abc.Coroutine`, +:class:`~collections.abc.AsyncIterator`, and +:class:`~collections.abc.AsyncIterable` abstract base classes. +(Contributed by Yury Selivanov in :issue:`24184`.) + compileall ---------- -* :func:`compileall.compile_dir` and :mod:`compileall`'s command-line interface - can now do parallel bytecode compilation. - (Contributed by Claudiu Popa in :issue:`16104`.) +A new :mod:`compileall` option, :option:`-j N`, allows to run ``N`` workers +sumultaneously to perform parallel bytecode compilation. :func:`compileall.compile_dir` has a corresponding ``workers`` parameter. (Contributed by +Claudiu Popa in :issue:`16104`.) -* *quiet* parameter of :func:`compileall.compile_dir`, - :func:`compileall.compile_file`, and :func:`compileall.compile_path` - functions now has a multilevel value. New ``-qq`` command line option - is available for suppressing the output. - (Contributed by Thomas Kluyver in :issue:`21338`.) +The :option:`-q` command line option can now be specified more than once, in +which case all output, including errors, will be suppressed. The corresponding +``quiet`` parameter in :func:`compileall.compile_dir`, :func:`compileall. +compile_file`, and :func:`compileall.compile_path` can now accept +an integer value indicating the level of output suppression. +(Contributed by Thomas Kluyver in :issue:`21338`.) + concurrent.futures ------------------ -* :meth:`~concurrent.futures.Executor.map` now takes a *chunksize* - argument to allow batching of tasks in child processes and improve - performance of ProcessPoolExecutor. - (Contributed by Dan O'Reilly in :issue:`11271`.) +:meth:`~concurrent.futures.Executor.map` now accepts a *chunksize* +argument to allow batching of tasks in child processes and improve performance +of ProcessPoolExecutor. (Contributed by Dan O'Reilly in :issue:`11271`.) + contextlib ---------- -* The new :func:`contextlib.redirect_stderr` context manager(similar to - :func:`contextlib.redirect_stdout`) makes it easier for utility scripts to - handle inflexible APIs that write their output to :data:`sys.stderr` and - don't provide any options to redirect it. - (Contributed by Berker Peksag in :issue:`22389`.) +The new :func:`contextlib.redirect_stderr` context manager (similar to +:func:`contextlib.redirect_stdout`) makes it easier for utility scripts to +handle inflexible APIs that write their output to :data:`sys.stderr` and +don't provide any options to redirect it. (Contributed by Berker Peksag in +:issue:`22389`.) + curses ------ -* The new :func:`curses.update_lines_cols` function updates the variables - :envvar:`curses.LINES` and :envvar:`curses.COLS`. + +The new :func:`curses.update_lines_cols` function updates the variables +:envvar:`curses.LINES` and :envvar:`curses.COLS`. + difflib ------- -* The charset of the HTML document generated by :meth:`difflib.HtmlDiff.make_file` - can now be customized by using *charset* keyword-only parameter. The default - charset of HTML document changed from ``'ISO-8859-1'`` to ``'utf-8'``. - (Contributed by Berker Peksag in :issue:`2052`.) +The charset of the HTML document generated by :meth:`difflib.HtmlDiff.make_file` +can now be customized by using *charset* keyword-only parameter. The default +charset of HTML document changed from ``'ISO-8859-1'`` to ``'utf-8'``. +(Contributed by Berker Peksag in :issue:`2052`.) -* It's now possible to compare lists of byte strings with - :func:`difflib.diff_bytes`. This fixes a regression from Python 2. - (Contributed by Terry J. Reedy and Greg Ward in :issue:`17445`.) +It is now possible to compare lists of byte strings with +:func:`difflib.diff_bytes`. This fixes a regression from Python 2. +(Contributed by Terry J. Reedy and Greg Ward in :issue:`17445`.) + distutils --------- -* The ``build`` and ``build_ext`` commands now accept a ``-j`` - option to enable parallel building of extension modules. - (Contributed by Antoine Pitrou in :issue:`5309`.) +The ``build`` and ``build_ext`` commands now accept a :option:`-j` option to +enable parallel building of extension modules. +(Contributed by Antoine Pitrou in :issue:`5309`.) -* Added support for the LZMA compression. - (Contributed by Serhiy Storchaka in :issue:`16314`.) +:mod:`distutils` now supports ``xz`` compression, and can be enabled by +passing ``xztar`` as an argument to ``bdist --format``. +(Contributed by Serhiy Storchaka in :issue:`16314`.) + doctest ------- -* :func:`doctest.DocTestSuite` returns an empty :class:`unittest.TestSuite` if - *module* contains no docstrings instead of raising :exc:`ValueError`. - (Contributed by Glenn Jones in :issue:`15916`.) +:func:`doctest.DocTestSuite` returns an empty :class:`unittest.TestSuite` if +*module* contains no docstrings instead of raising :exc:`ValueError`. +(Contributed by Glenn Jones in :issue:`15916`.) + email ----- -* A new policy option :attr:`~email.policy.Policy.mangle_from_` controls - whether or not lines that start with "From " in email bodies are prefixed with - a '>' character by generators. The default is ``True`` for - :attr:`~email.policy.compat32` and ``False`` for all other policies. - (Contributed by Milan Oberkirch in :issue:`20098`.) +A new policy option :attr:`~email.policy.Policy.mangle_from_` controls +whether or not lines that start with ``"From "`` in email bodies are prefixed +with a ``'>'`` character by generators. The default is ``True`` for +:attr:`~email.policy.compat32` and ``False`` for all other policies. +(Contributed by Milan Oberkirch in :issue:`20098`.) -* A new method :meth:`~email.message.Message.get_content_disposition` provides - easy access to a canonical value for the :mailheader:`Content-Disposition` - header (``None`` if there is no such header). (Contributed by Abhilash Raj - in :issue:`21083`.) +A new method :meth:`~email.message.Message.get_content_disposition` provides +easy access to a canonical value for the :mailheader:`Content-Disposition` +header (``None`` if there is no such header). (Contributed by Abhilash Raj +in :issue:`21083`.) -* A new policy option :attr:`~email.policy.EmailPolicy.utf8` can be set - ``True`` to encode email headers using the utf8 charset instead of using - encoded words. This allows ``Messages`` to be formatted according to - :rfc:`6532` and used with an SMTP server that supports the :rfc:`6531` - ``SMTPUTF8`` extension. (Contributed by R. David Murray in :issue:`24211`.) +A new policy option :attr:`~email.policy.EmailPolicy.utf8` can be set +to ``True`` to encode email headers using the UTF-8 charset instead of using +encoded words. This allows ``Messages`` to be formatted according to +:rfc:`6532` and used with an SMTP server that supports the :rfc:`6531` +``SMTPUTF8`` extension. (Contributed by R. David Murray in :issue:`24211`.) + faulthandler ------------ -* :func:`~faulthandler.enable`, :func:`~faulthandler.register`, - :func:`~faulthandler.dump_traceback` and - :func:`~faulthandler.dump_traceback_later` functions now accept file - descriptors. (Contributed by Wei Wu in :issue:`23566`.) +:func:`~faulthandler.enable`, :func:`~faulthandler.register`, +:func:`~faulthandler.dump_traceback` and +:func:`~faulthandler.dump_traceback_later` functions now accept file +descriptors in addition to file-like objects. +(Contributed by Wei Wu in :issue:`23566`.) + functools --------- -* Most of :func:`~functools.lru_cache` machinery is now implemented in C. - (Contributed by Matt Joiner, Alexey Kachayev, and Serhiy Storchaka - in :issue:`14373`.) +Most of :func:`~functools.lru_cache` machinery is now implemented in C, making +it significantly faster. (Contributed by Matt Joiner, Alexey Kachayev, and +Serhiy Storchaka in :issue:`14373`.) + glob ---- -00002-5938667 +:func:`~glob.iglob` and :func:`~glob.glob` now support recursive search in +subdirectories using the "``**``" pattern. (Contributed by Serhiy Storchaka +in :issue:`13968`.) -* :func:`~glob.iglob` and :func:`~glob.glob` now support recursive search in - subdirectories using the "``**``" pattern. - (Contributed by Serhiy Storchaka in :issue:`13968`.) heapq ----- -* :func:`~heapq.merge` has two new optional parameters ``reverse`` and - ``key``. (Contributed by Raymond Hettinger in :issue:`13742`.) +Element comparison in :func:`~heapq.merge` can now be customized by +passing a :term:`key function` in a new optional ``key`` keyword argument. +A new optional ``reverse`` keyword argument can be used to reverse element +comparison. (Contributed by Raymond Hettinger in :issue:`13742`.) + idlelib and IDLE ---------------- @@ -788,512 +838,613 @@ as well as changes made in future 3.5.x releases. This file is also available from the IDLE Help -> About Idle dialog. + imaplib ------- -* :class:`IMAP4` now supports the context management protocol. When used in a - :keyword:`with` statement, the IMAP4 ``LOGOUT`` command will be called - automatically at the end of the block. (Contributed by Tarek Ziad? and - Serhiy Storchaka in :issue:`4972`.) +:class:`~imaplib.IMAP4` now supports context manager protocol. +When used in a :keyword:`with` statement, the IMAP4 ``LOGOUT`` +command will be called automatically at the end of the block. +(Contributed by Tarek Ziad? and Serhiy Storchaka in :issue:`4972`.) -* :mod:`imaplib` now supports :rfc:`5161`: the :meth:`~imaplib.IMAP4.enable` - extension), and :rfc:`6855`: utf-8 support (internationalized email, via the - ``UTF8=ACCEPT`` argument to :meth:`~imaplib.IMAP4.enable`). A new attribute, - :attr:`~imaplib.IMAP4.utf8_enabled`, tracks whether or not :rfc:`6855` - support is enabled. Milan Oberkirch, R. David Murray, and Maciej Szulik in - :issue:`21800`.) +:mod:`imaplib` now supports :rfc:`5161` (``ENABLE`` extension) via +:meth:`~imaplib.IMAP4.enable`, and :rfc:`6855` (UTF-8 support) via the +``UTF8=ACCEPT`` argument to :meth:`~imaplib.IMAP4.enable`. A new attribute, +:attr:`~imaplib.IMAP4.utf8_enabled`, tracks whether or not :rfc:`6855` +support is enabled. (Contributed by Milan Oberkirch, R. David Murray, +and Maciej Szulik in :issue:`21800`.) -* :mod:`imaplib` now automatically encodes non-ASCII string usernames and - passwords using ``UTF8``, as recommended by the RFCs. (Contributed by Milan - Oberkirch in :issue:`21800`.) +:mod:`imaplib` now automatically encodes non-ASCII string usernames and +passwords using UTF-8, as recommended by the RFCs. (Contributed by Milan +Oberkirch in :issue:`21800`.) + imghdr ------ -* :func:`~imghdr.what` now recognizes the `OpenEXR `_ - format (contributed by Martin Vignali and Claudiu Popa in :issue:`20295`), - and the `WebP `_ format (contributed - by Fabrice Aneche and Claudiu Popa in :issue:`20197`.) +:func:`~imghdr.what` now recognizes the `OpenEXR `_ +format (contributed by Martin Vignali and Claudiu Popa in :issue:`20295`), +and the `WebP `_ format (contributed +by Fabrice Aneche and Claudiu Popa in :issue:`20197`.) + importlib --------- -* :class:`importlib.util.LazyLoader` allows for the lazy loading of modules in - applications where startup time is paramount. - (Contributed by Brett Cannon in :issue:`17621`.) +:class:`importlib.util.LazyLoader` allows for lazy loading of modules in +applications where startup time is important. (Contributed by Brett Cannon +in :issue:`17621`.) -* :func:`importlib.abc.InspectLoader.source_to_code` is now a - static method to make it easier to work with source code in a string. - With a module object that you want to initialize you can then use - ``exec(code, module.__dict__)`` to execute the code in the module. +:func:`importlib.abc.InspectLoader.source_to_code` is now a +static method. This makes it easier to initialize a module object with +code compiled from a string by runnning ``exec(code, module.__dict__)``. +(Contributed by Brett Cannon in :issue:`21156`.) -* :func:`importlib.util.module_from_spec` is now the preferred way to create a - new module. Compared to :class:`types.ModuleType`, this new function will set - the various import-controlled attributes based on the passed-in spec object. - (Contributed by Brett Cannon in :issue:`20383`.) +:func:`importlib.util.module_from_spec` is now the preferred way to create a +new module. Compared to :class:`types.ModuleType`, this new function will set +the various import-controlled attributes based on the passed-in spec object. +(Contributed by Brett Cannon in :issue:`20383`.) + inspect ------- -* :class:`inspect.Signature` and :class:`inspect.Parameter` are now - picklable and hashable. (Contributed by Yury Selivanov in :issue:`20726` - and :issue:`20334`.) +:class:`inspect.Signature` and :class:`inspect.Parameter` are now +picklable and hashable. (Contributed by Yury Selivanov in :issue:`20726` +and :issue:`20334`.) -* New method :meth:`inspect.BoundArguments.apply_defaults`. (Contributed - by Yury Selivanov in :issue:`24190`.) +A new method :meth:`inspect.BoundArguments.apply_defaults` provides a way +to set default values for missing arguments. (Contributed by Yury Selivanov +in :issue:`24190`.) -* New class method :meth:`inspect.Signature.from_callable`, which makes - subclassing of :class:`~inspect.Signature` easier. (Contributed - by Yury Selivanov and Eric Snow in :issue:`17373`.) +A new class method :meth:`inspect.Signature.from_callable` makes +subclassing of :class:`~inspect.Signature` easier. (Contributed +by Yury Selivanov and Eric Snow in :issue:`17373`.) -* New argument ``follow_wrapped`` for :func:`inspect.signature`. - (Contributed by Yury Selivanov in :issue:`20691`.) +:func:`inspect.signature` now accepts a ``follow_wrapped`` optional keyword +argument, which, when set to ``False``, disables automatic following of +``__wrapped__`` links. (Contributed by Yury Selivanov in :issue:`20691`.) -* New :func:`~inspect.iscoroutine`, :func:`~inspect.iscoroutinefunction` - and :func:`~inspect.isawaitable` functions. (Contributed by - Yury Selivanov in :issue:`24017`.) +A set of new functions to inspect +:term:`coroutine functions ` and +``coroutine objects`` as been added: +:func:`~inspect.iscoroutine`, :func:`~inspect.iscoroutinefunction`, +:func:`~inspect.isawaitable`, :func:`~inspect.getcoroutinelocals`, +and :func:`~inspect.getcoroutinestate`. +(Contributed by Yury Selivanov in :issue:`24017` and :issue:`24400`.) -* New :func:`~inspect.getcoroutinelocals` and :func:`~inspect.getcoroutinestate` - functions. (Contributed by Yury Selivanov in :issue:`24400`.) +:func:`~inspect.stack`, :func:`~inspect.trace`, :func:`~inspect.getouterframes`, +and :func:`~inspect.getinnerframes` now return a list of named tuples. +(Contributed by Daniel Shahaf in :issue:`16808`.) -* :func:`~inspect.stack`, :func:`~inspect.trace`, :func:`~inspect.getouterframes`, - and :func:`~inspect.getinnerframes` now return a list of named tuples. - (Contributed by Daniel Shahaf in :issue:`16808`.) io -- -* New Python implementation of :class:`io.FileIO` to make dependency on - ``_io`` module optional. - (Contributed by Serhiy Storchaka in :issue:`21859`.) +:class:`io.FileIO` has been implemented in Python which makes C implementation +of :mod:`io` module entirely optional. (Contributed by Serhiy Storchaka +in :issue:`21859`.) + ipaddress --------- -* :class:`ipaddress.IPv4Network` and :class:`ipaddress.IPv6Network` now - accept an ``(address, netmask)`` tuple argument, so as to easily construct - network objects from existing addresses. (Contributed by Peter Moody - and Antoine Pitrou in :issue:`16531`.) +:class:`ipaddress.IPv4Network` and :class:`ipaddress.IPv6Network` now +accept an ``(address, netmask)`` tuple argument, so as to easily construct +network objects from existing addresses. (Contributed by Peter Moody +and Antoine Pitrou in :issue:`16531`.) + json ---- -* The output of :mod:`json.tool` command line interface is now in the same - order as the input. Use the :option:`--sort-keys` option to sort the output - of dictionaries alphabetically by key. (Contributed by Berker Peksag in - :issue:`21650`.) +:mod:`json.tool` command line interface now preserves the order of keys in +JSON objects passed in input. The new :option:`--sort-keys` option can be used +to sort the keys alphabetically. (Contributed by Berker Peksag +in :issue:`21650`.) -* JSON decoder now raises :exc:`json.JSONDecodeError` instead of - :exc:`ValueError`. (Contributed by Serhiy Storchaka in :issue:`19361`.) +JSON decoder now raises :exc:`json.JSONDecodeError` instead of +:exc:`ValueError`. (Contributed by Serhiy Storchaka in :issue:`19361`.) + locale ------ - * New :func:`~locale.delocalize` function to convert a string into a - normalized number string, following the ``LC_NUMERIC`` settings. - (Contributed by C?dric Krier in :issue:`13918`.) +A new :func:`~locale.delocalize` function can be used to convert a string into +a normalized number string, taking the ``LC_NUMERIC`` settings into account. +(Contributed by C?dric Krier in :issue:`13918`.) + logging ------- -* All logging methods :meth:`~logging.Logger.log`, :meth:`~logging.Logger.exception`, - :meth:`~logging.Logger.critical`, :meth:`~logging.Logger.debug`, etc, - now accept exception instances for ``exc_info`` parameter, in addition - to boolean values and exception tuples. - (Contributed by Yury Selivanov in :issue:`20537`.) +All logging methods (:meth:`~logging.Logger.log`, +:meth:`~logging.Logger.exception`, :meth:`~logging.Logger.critical`, +:meth:`~logging.Logger.debug`, etc.), now accept exception instances +in ``exc_info`` parameter, in addition to boolean values and exception +tuples. (Contributed by Yury Selivanov in :issue:`20537`.) -* :class:`~logging.handlers.HTTPHandler` now accepts optional - :class:`ssl.SSLContext` instance to configure the SSL settings used - for HTTP connection. - (Contributed by Alex Gaynor in :issue:`22788`.) +:class:`~logging.handlers.HTTPHandler` now accepts an optional +:class:`ssl.SSLContext` instance to configure the SSL settings used +in an HTTP connection. (Contributed by Alex Gaynor in :issue:`22788`.) -* :class:`~logging.handlers.QueueListener` now takes a *respect_handler_level* - keyword argument which, if set to ``True``, will pass messages to handlers - taking handler levels into account. (Contributed by Vinay Sajip.) +:class:`~logging.handlers.QueueListener` now takes a *respect_handler_level* +keyword argument which, if set to ``True``, will pass messages to handlers +taking handler levels into account. (Contributed by Vinay Sajip.) + lzma ---- -* New option *max_length* for :meth:`~lzma.LZMADecompressor.decompress` - to limit the maximum size of decompressed data. - (Contributed by Martin Panter in :issue:`15955`.) +:meth:`~lzma.LZMADecompressor.decompress` now accepts an optional *max_length* +argument to limit the maximum size of decompressed data. +(Contributed by Martin Panter in :issue:`15955`.) + math ---- -* :data:`math.inf` and :data:`math.nan` constants added. (Contributed by Mark - Dickinson in :issue:`23185`.) +Two new constants have been added to :mod:`math`: :data:`math.inf` +and :data:`math.nan`. (Contributed by Mark Dickinson in :issue:`23185`.) -* New :func:`~math.isclose` function. - (Contributed by Chris Barker and Tal Einat in :issue:`24270`.) +A new function :func:`math.isclose` provides a way to test for approximate +equality. (Contributed by Chris Barker and Tal Einat in :issue:`24270`.) -* New :func:`~math.gcd` function. The :func:`fractions.gcd` function now is - deprecated. - (Contributed by Mark Dickinson and Serhiy Storchaka in :issue:`22486`.) +A new :func:`~math.gcd` function has been added. The :func:`fractions.gcd` +function is now deprecated. (Contributed by Mark Dickinson and Serhiy +Storchaka in :issue:`22486`.) + operator -------- -* :func:`~operator.attrgetter`, :func:`~operator.itemgetter`, and - :func:`~operator.methodcaller` objects now support pickling. - (Contributed by Josh Rosenberg and Serhiy Storchaka in :issue:`22955`.) +:func:`~operator.attrgetter`, :func:`~operator.itemgetter`, and +:func:`~operator.methodcaller` objects now support pickling. +(Contributed by Josh Rosenberg and Serhiy Storchaka in :issue:`22955`.) + os -- -* New :func:`os.scandir` function that exposes file information from - the operating system when listing a directory. :func:`os.scandir` - returns an iterator of :class:`os.DirEntry` objects corresponding to - the entries in the directory given by *path*. (Contributed by Ben - Hoyt with the help of Victor Stinner in :issue:`22524`.) +The new :func:`os.scandir` returning an iterator of :class:`os.DirEntry` +objects has been added. If possible, :func:`os.scandir` extracts file +attributes while scanning a directory, removing the need to perform +subsequent system calls to determine file type or attributes, which may +significantly improve performance. (Contributed by Ben Hoyt with the help +of Victor Stinner in :issue:`22524`.) -* :class:`os.stat_result` now has a :attr:`~os.stat_result.st_file_attributes` - attribute on Windows. (Contributed by Ben Hoyt in :issue:`21719`.) +On Windows, a new :attr:`~os.stat_result.st_file_attributes` attribute is +now available. It corresponds to ``dwFileAttributes`` member of the +``BY_HANDLE_FILE_INFORMATION`` structure returned by +``GetFileInformationByHandle()``. (Contributed by Ben Hoyt in :issue:`21719`.) -* :func:`os.urandom`: On Linux 3.17 and newer, the ``getrandom()`` syscall is - now used when available. On OpenBSD 5.6 and newer, the C ``getentropy()`` - function is now used. These functions avoid the usage of an internal file - descriptor. - (Contributed by Victor Stinner in :issue:`22181`.) +:func:`os.urandom` now uses ``getrandom()`` syscall on Linux 3.17 or newer, +and ``getentropy()`` on OpenBSD 5.6 and newer, removing the need to use +``/dev/urandom`` and avoiding failures due to potential file descriptor +exhaustion. (Contributed by Victor Stinner in :issue:`22181`.) -* New :func:`os.get_blocking` and :func:`os.set_blocking` functions to - get and set the blocking mode of file descriptors. - (Contributed by Victor Stinner in :issue:`22054`.) +New :func:`os.get_blocking` and :func:`os.set_blocking` functions allow to +get and set the file descriptor blocking mode (:data:`~os.O_NONBLOCK`.) +(Contributed by Victor Stinner in :issue:`22054`.) -* :func:`~os.truncate` and :func:`~os.ftruncate` are now supported on - Windows. (Contributed by Steve Dower in :issue:`23668`.) +:func:`~os.truncate` and :func:`~os.ftruncate` are now supported on +Windows. (Contributed by Steve Dower in :issue:`23668`.) + os.path ------- -* New :func:`~os.path.commonpath` function that extracts common path prefix. - Unlike the :func:`~os.path.commonprefix` function, it always returns a valid - path. (Contributed by Rafik Draoui and Serhiy Storchaka in :issue:`10395`.) +There is a new :func:`~os.path.commonpath` function returning the longest +common sub-path of each passed pathname. Unlike the +:func:`~os.path.commonprefix` function, it always returns a valid +path. (Contributed by Rafik Draoui and Serhiy Storchaka in :issue:`10395`.) + pathlib ------- -* New :meth:`~pathlib.Path.samefile` method to check if other path object - points to the same file. (Contributed by Vajrasky Kok and Antoine Pitrou - in :issue:`19775`.) +The new :meth:`~pathlib.Path.samefile` method can be used to check if the +passed :class:`~pathlib.Path` object, or a string, point to the same file as +the :class:`~pathlib.Path` on which :meth:`~pathlib.Path.samefile` is called. +(Contributed by Vajrasky Kok and Antoine Pitrou in :issue:`19775`.) -* :meth:`~pathlib.Path.mkdir` has a new optional parameter ``exist_ok`` - to mimic ``mkdir -p`` and :func:`os.makrdirs` functionality. - (Contributed by Berker Peksag in :issue:`21539`.) +:meth:`~pathlib.Path.mkdir` how accepts a new optional ``exist_ok`` argument +to match ``mkdir -p`` and :func:`os.makrdirs` functionality. +(Contributed by Berker Peksag in :issue:`21539`.) -* New :meth:`~pathlib.Path.expanduser` to expand ``~`` and ``~user`` - constructs. - (Contributed by Serhiy Storchaka and Claudiu Popa in :issue:`19776`.) +There is a new :meth:`~pathlib.Path.expanduser` method to expand ``~`` +and ``~user`` prefixes. (Contributed by Serhiy Storchaka and Claudiu +Popa in :issue:`19776`.) -* New class method :meth:`~pathlib.Path.home` to get an instance of - :class:`~pathlib.Path` object representing the user?s home directory. - (Contributed by Victor Salgado and Mayank Tripathi in :issue:`19777`.) +A new :meth:`~pathlib.Path.home` class method can be used to get an instance +of :class:`~pathlib.Path` object representing the user?s home directory. +(Contributed by Victor Salgado and Mayank Tripathi in :issue:`19777`.) + pickle ------ -* Serializing more "lookupable" objects (such as unbound methods or nested - classes) now are supported with pickle protocols < 4. - (Contributed by Serhiy Storchaka in :issue:`23611`.) +Nested objects, such as unbound methods or nested classes, can now be pickled using :ref:`pickle protocols ` older than protocol version 4, +which already supported these cases. (Contributed by Serhiy Storchaka in +:issue:`23611`.) + poplib ------ -* A new command :meth:`~poplib.POP3.utf8` enables :rfc:`6856` - (internationalized email) support if the POP server supports it. (Contributed - by Milan OberKirch in :issue:`21804`.) +A new command :meth:`~poplib.POP3.utf8` enables :rfc:`6856` +(internationalized email) support, if the POP server supports it. +(Contributed by Milan OberKirch in :issue:`21804`.) + re -- -* Number of capturing groups in regular expression is no longer limited by 100. - (Contributed by Serhiy Storchaka in :issue:`22437`.) +The number of capturing groups in regular expression is no longer limited by +100. (Contributed by Serhiy Storchaka in :issue:`22437`.) -* Now unmatched groups are replaced with empty strings in :func:`re.sub` - and :func:`re.subn`. (Contributed by Serhiy Storchaka in :issue:`1519638`.) +:func:`re.sub` and :func:`re.subn` now replace unmatched groups with empty +strings instead of rising an exception. (Contributed by Serhiy Storchaka +in :issue:`1519638`.) + readline -------- -* New :func:`~readline.append_history_file` function. - (Contributed by Bruno Cauet in :issue:`22940`.) +The new :func:`~readline.append_history_file` function can be used to append +the specified number of trailing elements in history to a given file. +(Contributed by Bruno Cauet in :issue:`22940`.) + shutil ------ -* :func:`~shutil.move` now accepts a *copy_function* argument, allowing, - for example, :func:`~shutil.copy` to be used instead of the default - :func:`~shutil.copy2` if there is a need to ignore metadata. (Contributed by - Claudiu Popa in :issue:`19840`.) +:func:`~shutil.move` now accepts a *copy_function* argument, allowing, +for example, :func:`~shutil.copy` to be used instead of the default +:func:`~shutil.copy2` there is a need to ignore file metadata when moving. +(Contributed by Claudiu Popa in :issue:`19840`.) -* :func:`~shutil.make_archive` now supports *xztar* format. - (Contributed by Serhiy Storchaka in :issue:`5411`.) +:func:`~shutil.make_archive` now supports *xztar* format. +(Contributed by Serhiy Storchaka in :issue:`5411`.) + signal ------ -* On Windows, :func:`signal.set_wakeup_fd` now also supports socket handles. - (Contributed by Victor Stinner in :issue:`22018`.) +On Windows, :func:`signal.set_wakeup_fd` now also supports socket handles. +(Contributed by Victor Stinner in :issue:`22018`.) -* Different constants of :mod:`signal` module are now enumeration values using - the :mod:`enum` module. This allows meaningful names to be printed during - debugging, instead of integer ?magic numbers?. (Contributed by Giampaolo - Rodola' in :issue:`21076`.) +Various ``SIG*`` constants in :mod:`signal` module have been converted into +:mod:`Enums `. This allows meaningful names to be printed +during debugging, instead of integer "magic numbers". +(Contributed by Giampaolo Rodola' in :issue:`21076`.) + smtpd ----- -* Both :class:`~smtpd.SMTPServer` and :class:`smtpd.SMTPChannel` now accept a - *decode_data* keyword to determine if the DATA portion of the SMTP - transaction is decoded using the ``utf-8`` codec or is instead provided to - :meth:`~smtpd.SMTPServer.process_message` as a byte string. The default - is ``True`` for backward compatibility reasons, but will change to ``False`` - in Python 3.6. If *decode_data* is set to ``False``, the - :meth:`~smtpd.SMTPServer.process_message` method must be prepared to accept - keyword arguments. (Contributed by Maciej Szulik in :issue:`19662`.) +Both :class:`~smtpd.SMTPServer` and :class:`smtpd.SMTPChannel` now accept a +*decode_data* keyword argument to determine if the ``DATA`` portion of the SMTP +transaction is decoded using the ``"utf-8"`` codec or is instead provided to +:meth:`~smtpd.SMTPServer.process_message` as a byte string. The default +is ``True`` for backward compatibility reasons, but will change to ``False`` +in Python 3.6. If *decode_data* is set to ``False``, the +:meth:`~smtpd.SMTPServer.process_message` method must be prepared to accept +keyword arguments. (Contributed by Maciej Szulik in :issue:`19662`.) -* :class:`~smtpd.SMTPServer` now advertises the ``8BITMIME`` extension - (:rfc:`6152`) if if *decode_data* has been set ``True``. If the client - specifies ``BODY=8BITMIME`` on the ``MAIL`` command, it is passed to - :meth:`~smtpd.SMTPServer.process_message` via the ``mail_options`` keyword. - (Contributed by Milan Oberkirch and R. David Murray in :issue:`21795`.) +:class:`~smtpd.SMTPServer` now advertises the ``8BITMIME`` extension +(:rfc:`6152`) if if *decode_data* has been set ``True``. If the client +specifies ``BODY=8BITMIME`` on the ``MAIL`` command, it is passed to +:meth:`~smtpd.SMTPServer.process_message` via the ``mail_options`` keyword. +(Contributed by Milan Oberkirch and R. David Murray in :issue:`21795`.) -* :class:`~smtpd.SMTPServer` now supports the ``SMTPUTF8`` extension - (:rfc:`6531`: Internationalized Email). If the client specified ``SMTPUTF8 - BODY=8BITMIME`` on the ``MAIL`` command, they are passed to - :meth:`~smtpd.SMTPServer.process_message` via the ``mail_options`` keyword. - It is the responsibility of the :meth:`~smtpd.SMTPServer.process_message` - method to correctly handle the ``SMTPUTF8`` data. (Contributed by Milan - Oberkirch in :issue:`21725`.) +:class:`~smtpd.SMTPServer` now supports the ``SMTPUTF8`` extension +(:rfc:`6531`: Internationalized Email). If the client specified ``SMTPUTF8 +BODY=8BITMIME`` on the ``MAIL`` command, they are passed to +:meth:`~smtpd.SMTPServer.process_message` via the ``mail_options`` keyword. +It is the responsibility of the :meth:`~smtpd.SMTPServer.process_message` +method to correctly handle the ``SMTPUTF8`` data. (Contributed by Milan +Oberkirch in :issue:`21725`.) -* It is now possible to provide, directly or via name resolution, IPv6 - addresses in the :class:`~smtpd.SMTPServer` constructor, and have it - successfully connect. (Contributed by Milan Oberkirch in :issue:`14758`.) +It is now possible to provide, directly or via name resolution, IPv6 +addresses in the :class:`~smtpd.SMTPServer` constructor, and have it +successfully connect. (Contributed by Milan Oberkirch in :issue:`14758`.) + smtplib ------- -* A new :meth:`~smtplib.SMTP.auth` method provides a convenient way to - implement custom authentication mechanisms. - (Contributed by Milan Oberkirch in :issue:`15014`.) +A new :meth:`~smtplib.SMTP.auth` method provides a convenient way to +implement custom authentication mechanisms. (Contributed by Milan +Oberkirch in :issue:`15014`.) -* Additional debuglevel (2) shows timestamps for debug messages in - :class:`smtplib.SMTP`. (Contributed by Gavin Chappell and Maciej Szulik in - :issue:`16914`.) +Additional debuglevel (2) shows timestamps for debug messages in +:class:`smtplib.SMTP`. (Contributed by Gavin Chappell and Maciej Szulik in +:issue:`16914`.) -* :mod:`smtplib` now supports :rfc:`6531` (SMTPUTF8) in both the - :meth:`~smtplib.SMTP.sendmail` and :meth:`~smtplib.SMTP.send_message` - commands. (Contributed by Milan Oberkirch and R. David Murray in - :issue:`22027`.) +:mod:`smtplib` now supports :rfc:`6531` (SMTPUTF8) in both the +:meth:`~smtplib.SMTP.sendmail` and :meth:`~smtplib.SMTP.send_message` +commands. (Contributed by Milan Oberkirch and R. David Murray in +:issue:`22027`.) + sndhdr ------ -* :func:`~sndhdr.what` and :func:`~sndhdr.whathdr` now return - :func:`~collections.namedtuple`. - (Contributed by Claudiu Popa in :issue:`18615`.) +:func:`~sndhdr.what` and :func:`~sndhdr.whathdr` now return +:func:`~collections.namedtuple`. (Contributed by Claudiu Popa in +:issue:`18615`.) + ssl --- -* The :meth:`~ssl.SSLSocket.do_handshake`, :meth:`~ssl.SSLSocket.read`, - :meth:`~ssl.SSLSocket.shutdown`, and :meth:`~ssl.SSLSocket.write` methods of - :class:`ssl.SSLSocket` don't reset the socket timeout anymore each time bytes - are received or sent. The socket timeout is now the maximum total duration of - the method. +Memory BIO Support +~~~~~~~~~~~~~~~~~~ -* Memory BIO Support: new classes :class:`~ssl.SSLObject`, - :class:`~ssl.MemoryBIO`, and new - :meth:`SSLContext.wrap_bio ` method. - (Contributed by Geert Jansen in :issue:`21965`.) +(Contributed by Geert Jansen in :issue:`21965`.) -* New :meth:`~ssl.SSLSocket.version` to query the actual protocol version - in use. (Contributed by Antoine Pitrou in :issue:`20421`.) +The new :class:`~ssl.SSLObject` class has been added to provide SSL protocol +support for cases when the network IO capabilities of :class:`~ssl.SSLSocket` +are not necessary or inappropriate. :class:`~ssl.SSLObject` represents +an SSL protocol instance, but does not implement any network IO methods, and +instead provides a memory buffer interface. The new :class:`~ssl.MemoryBIO` +class can be used to pass data between Python and an SSL protocol instance. -* New :meth:`~ssl.SSLObject.shared_ciphers` and - :meth:`~ssl.SSLSocket.shared_ciphers` methods to fetch the client's - list of ciphers sent at handshake. - (Contributed by Benjamin Peterson in :issue:`23186`.) +The memory BIO SSL support is primarily intended to be used in frameworks +implementing asynchronous IO for which :class:`~ssl.SSLObject` IO readiness +model ("select/poll") is inappropriate or inefficient. -* :func:`~ssl.match_hostname` now supports matching of IP addresses. - (Contributed by Antoine Pitrou in :issue:`23239`.) +A new :meth:`~ssl.SSLContext.wrap_bio` method can be used to create a new +:class:`~ssl.SSLObject` instance. + + +Application-Layer Protocol Negotiation Support +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +(Contributed by Benjamin Peterson in :issue:`20188`.) + +Where OpenSSL support is present, :mod:`ssl` module now implements * +Application-Layer Protocol Negotiation* TLS extension as described +in :rfc:`7301`. +The new :meth:`SSLContext.set_alpn_protocols ` +can be used to specify which protocols the socket should advertise during +the TLS handshake. The new +:meth:`SSLSocket.selected_alpn_protocol ` +returns the protocol that was selected during the TLS handshake. :data:`ssl.HAS_ALPN` flag indicates whether APLN support is present. + + +Other Changes +~~~~~~~~~~~~~ + +There is a new :meth:`~ssl.SSLSocket.version` method to query the actual +protocol version in use. (Contributed by Antoine Pitrou in :issue:`20421`.) + +:class:`~ssl.SSLSocket` now implementes :meth:`~ssl.SSLSocket.sendfile` +method. (Contributed by Giampaolo Rodola' in :issue:`17552`.) + +:meth:`ssl.SSLSocket.send()` now raises either :exc:`ssl.SSLWantReadError` +or :exc:`ssl.SSLWantWriteError` on a non-blocking socket if the operation +would block. Previously, it would return 0. (Contributed by Nikolaus Rath +in :issue:`20951`.) + +The :func:`~ssl.cert_time_to_seconds` function now interprets the input time +as UTC and not as local time, per :rfc:`5280`. Additionally, the return +value is always an :class:`int`. (Contributed by Akira Li in :issue:`19940`.) + +New :meth:`~ssl.SSLObject.shared_ciphers` and +:meth:`~ssl.SSLSocket.shared_ciphers` methods return the list of ciphers +sent by the client during the handshake. (Contributed by Benjamin Peterson +in :issue:`23186`.) + +The :meth:`~ssl.SSLSocket.do_handshake`, :meth:`~ssl.SSLSocket.read`, +:meth:`~ssl.SSLSocket.shutdown`, and :meth:`~ssl.SSLSocket.write` methods of +:class:`ssl.SSLSocket` no longer reset the socket timeout every time bytes +are received or sent. The socket timeout is now the maximum total duration of +the method. (Contributed by Victor Stinner in :issue:`23853`.) + +The :func:`~ssl.match_hostname` function now supports matching of IP addresses. +(Contributed by Antoine Pitrou in :issue:`23239`.) + socket ------ -* New :meth:`socket.socket.sendfile` method allows to send a file over a socket - by using high-performance :func:`os.sendfile` function on UNIX resulting in - uploads being from 2x to 3x faster than when using plain - :meth:`socket.socket.send`. - (Contributed by Giampaolo Rodola' in :issue:`17552`.) +A new :meth:`socket.socket.sendfile` method allows to send a file over a +socket by using high-performance :func:`os.sendfile` function on UNIX +resulting in uploads being from 2x to 3x faster than when using plain +:meth:`socket.socket.send`. (Contributed by Giampaolo Rodola' in +:issue:`17552`.) -* The :meth:`socket.socket.sendall` method don't reset the socket timeout - anymore each time bytes are received or sent. The socket timeout is now the - maximum total duration to send all data. +The :meth:`socket.socket.sendall` method no longer resets the socket timeout +every time bytes are received or sent. The socket timeout is now the +maximum total duration to send all data. (Contributed by Victor Stinner in +:issue:`23853`.) -* Functions with timeouts now use a monotonic clock, instead of a - system clock. (Contributed by Victor Stinner in :issue:`22043`.) +Functions with timeouts now use a monotonic clock, instead of a system clock. +(Contributed by Victor Stinner in :issue:`22043`.) + subprocess ---------- -* The new :func:`subprocess.run` function runs subprocesses and returns a - :class:`subprocess.CompletedProcess` object. It Provides a more consistent - API than :func:`~subprocess.call`, :func:`~subprocess.check_call` and - :func:`~subprocess.check_output`. +The new :func:`subprocess.run` function has been added and is the recommended +approach to invoking subprocesses. It runs the specified command and +and returns a :class:`subprocess.CompletedProcess` object. (Contributed by +Thomas Kluyver in :issue:`23342`.) + sys --- -* New :func:`~sys.set_coroutine_wrapper` and :func:`~sys.get_coroutine_wrapper` - functions. (Contributed by Yury Selivanov in :issue:`24017`.) +A new :func:`~sys.set_coroutine_wrapper` function allows setting a global +hook that will be called whenever a :ref:`coro object ` +is created. Essentially, it works like a global coroutine decorator. A +corresponding :func:`~sys.get_coroutine_wrapper` can be used to obtain +a currently set wrapper. Both functions are provisional, and are intended +for debugging purposes only. (Contributed by Yury Selivanov in :issue:`24017`.) -* New :func:`~sys.is_finalizing` to check for :term:`interpreter shutdown`. - (Contributed by Antoine Pitrou in :issue:`22696`.) +There is a new :func:`~sys.is_finalizing` function to check if the Python +interpreter is :term:`shutting down `. +(Contributed by Antoine Pitrou in :issue:`22696`.) + sysconfig --------- -* The user scripts directory on Windows is now versioned. - (Contributed by Paul Moore in :issue:`23437`.) +The name of the user scripts directory on Windows now includes the first +two components of Python version. (Contributed by Paul Moore +in :issue:`23437`.) + tarfile ------- -* The :func:`tarfile.open` function now supports ``'x'`` (exclusive creation) - mode. (Contributed by Berker Peksag in :issue:`21717`.) +The *mode* argument of :func:`tarfile.open` function now accepts ``'x'`` to request exclusive creation. (Contributed by Berker Peksag in :issue:`21717`.) -* The :meth:`~tarfile.TarFile.extractall` and :meth:`~tarfile.TarFile.extract` - methods now take a keyword parameter *numeric_only*. If set to ``True``, - the extracted files and directories will be owned by the numeric uid and gid - from the tarfile. If set to ``False`` (the default, and the behavior in - versions prior to 3.5), they will be owned by the named user and group in the - tarfile. (Contributed by Michael Vogt and Eric Smith in :issue:`23193`.) +The :meth:`~tarfile.TarFile.extractall` and :meth:`~tarfile.TarFile.extract` +methods now take a keyword argument *numeric_only*. If set to ``True``, +the extracted files and directories will be owned by the numeric uid and gid +from the tarfile. If set to ``False`` (the default, and the behavior in +versions prior to 3.5), they will be owned by the named user and group in the +tarfile. (Contributed by Michael Vogt and Eric Smith in :issue:`23193`.) + threading --------- -* :meth:`~threading.Lock.acquire` and :meth:`~threading.RLock.acquire` - now use a monotonic clock for managing timeouts. - (Contributed by Victor Stinner in :issue:`22043`.) +:meth:`~threading.Lock.acquire` and :meth:`~threading.RLock.acquire` +now use a monotonic clock for timeout management. (Contributed by Victor +Stinner in :issue:`22043`.) + time ---- -* The :func:`time.monotonic` function is now always available. (Contributed by - Victor Stinner in :issue:`22043`.) +The :func:`time.monotonic` function is now always available. (Contributed by +Victor Stinner in :issue:`22043`.) + timeit ------ -* New command line option ``-u`` or ``--unit=U`` to specify a time unit for - the timer output. Supported options are ``usec``, ``msec``, or ``sec``. - (Contributed by Julian Gindi in :issue:`18983`.) +New command line option :option:`-u` or :option:`--unit=U` to specify a time +unit for the timer output. Supported options are ``usec``, ``msec``, or ``sec``. +(Contributed by Julian Gindi in :issue:`18983`.) + tkinter ------- -* The :mod:`tkinter._fix` module used for setting up the Tcl/Tk environment - on Windows has been replaced by a private function in the :mod:`_tkinter` - module which makes no permanent changes to environment variables. - (Contributed by Zachary Ware in :issue:`20035`.) +The :mod:`tkinter._fix` module used for setting up the Tcl/Tk environment +on Windows has been replaced by a private function in the :mod:`_tkinter` +module which makes no permanent changes to environment variables. +(Contributed by Zachary Ware in :issue:`20035`.) + traceback --------- -* New :func:`~traceback.walk_stack` and :func:`~traceback.walk_tb` - functions to conveniently traverse frame and traceback objects. - (Contributed by Robert Collins in :issue:`17911`.) +New :func:`~traceback.walk_stack` and :func:`~traceback.walk_tb` +functions to conveniently traverse frame and traceback objects. +(Contributed by Robert Collins in :issue:`17911`.) -* New lightweight classes: :class:`~traceback.TracebackException`, - :class:`~traceback.StackSummary`, and :class:`traceback.FrameSummary`. - (Contributed by Robert Collins in :issue:`17911`.) +New lightweight classes: :class:`~traceback.TracebackException`, +:class:`~traceback.StackSummary`, and :class:`traceback.FrameSummary`. +(Contributed by Robert Collins in :issue:`17911`.) -* :func:`~traceback.print_tb` and :func:`~traceback.print_stack` now support - negative values for the *limit* argument. - (Contributed by Dmitry Kazakov in :issue:`22619`.) +:func:`~traceback.print_tb` and :func:`~traceback.print_stack` now support +negative values for the *limit* argument. +(Contributed by Dmitry Kazakov in :issue:`22619`.) types ----- -* New :func:`~types.coroutine` function. (Contributed by Yury Selivanov - in :issue:`24017`.) +New :func:`~types.coroutine` function. (Contributed by Yury Selivanov +in :issue:`24017`.) -* New :class:`~types.CoroutineType`. (Contributed by Yury Selivanov - in :issue:`24400`.) +New :class:`~types.CoroutineType`. (Contributed by Yury Selivanov +in :issue:`24400`.) + urllib ------ -* A new :class:`~urllib.request.HTTPPasswordMgrWithPriorAuth` allows HTTP Basic - Authentication credentials to be managed so as to eliminate unnecessary - ``401`` response handling, or to unconditionally send credentials - on the first request in order to communicate with servers that return a - ``404`` response instead of a ``401`` if the ``Authorization`` header is not - sent. (Contributed by Matej Cepl in :issue:`19494` and Akshit Khurana in - :issue:`7159`.) +A new :class:`~urllib.request.HTTPPasswordMgrWithPriorAuth` allows HTTP Basic +Authentication credentials to be managed so as to eliminate unnecessary +``401`` response handling, or to unconditionally send credentials +on the first request in order to communicate with servers that return a +``404`` response instead of a ``401`` if the ``Authorization`` header is not +sent. (Contributed by Matej Cepl in :issue:`19494` and Akshit Khurana in +:issue:`7159`.) -* A new :func:`~urllib.parse.urlencode` parameter *quote_via* provides a way to - control the encoding of query parts if needed. (Contributed by Samwyse and - Arnon Yaari in :issue:`13866`.) +A new :func:`~urllib.parse.urlencode` parameter *quote_via* provides a way to +control the encoding of query parts if needed. (Contributed by Samwyse and +Arnon Yaari in :issue:`13866`.) -* :func:`~urllib.request.urlopen` accepts an :class:`ssl.SSLContext` - object as a *context* argument, which will be used for the HTTPS - connection. (Contributed by Alex Gaynor in :issue:`22366`.) +:func:`~urllib.request.urlopen` accepts an :class:`ssl.SSLContext` +object as a *context* argument, which will be used for the HTTPS +connection. (Contributed by Alex Gaynor in :issue:`22366`.) + unicodedata ----------- -* The :mod:`unicodedata` module now uses data from `Unicode 8.0.0 - `_. +The :mod:`unicodedata` module now uses data from `Unicode 8.0.0 +`_. + unittest -------- -* New command line option ``--locals`` to show local variables in - tracebacks. - (Contributed by Robert Collins in :issue:`22936`.) +New command line option :option:`--locals` to show local variables in +tracebacks. (Contributed by Robert Collins in :issue:`22936`.) + wsgiref ------- -* *headers* parameter of :class:`wsgiref.headers.Headers` is now optional. - (Contributed by Pablo Torres Navarrete and SilentGhost in :issue:`5800`.) +*headers* parameter of :class:`wsgiref.headers.Headers` is now optional. +(Contributed by Pablo Torres Navarrete and SilentGhost in :issue:`5800`.) + xmlrpc ------ -* :class:`xmlrpc.client.ServerProxy` is now a :term:`context manager`. - (Contributed by Claudiu Popa in :issue:`20627`.) +:class:`xmlrpc.client.ServerProxy` is now a :term:`context manager`. +(Contributed by Claudiu Popa in :issue:`20627`.) -* :class:`~xmlrpc.client.ServerProxy` constructor now accepts an optional - :class:`ssl.SSLContext` instance. - (Contributed by Alex Gaynor in :issue:`22960`.) +:class:`~xmlrpc.client.ServerProxy` constructor now accepts an optional +:class:`ssl.SSLContext` instance. +(Contributed by Alex Gaynor in :issue:`22960`.) + xml.sax ------- -* SAX parsers now support a character stream of - :class:`~xml.sax.xmlreader.InputSource` object. - (Contributed by Serhiy Storchaka in :issue:`2175`.) +SAX parsers now support a character stream of +:class:`~xml.sax.xmlreader.InputSource` object. +(Contributed by Serhiy Storchaka in :issue:`2175`.) + zipfile ------- -* Added support for writing ZIP files to unseekable streams. - (Contributed by Serhiy Storchaka in :issue:`23252`.) +ZIP output can now be written to unseekable streams. +(Contributed by Serhiy Storchaka in :issue:`23252`.) -* The :func:`zipfile.ZipFile.open` function now supports ``'x'`` (exclusive - creation) mode. (Contributed by Serhiy Storchaka in :issue:`21717`.) +The *mode* argument of :func:`zipfile.ZipFile.open` function now +accepts ``'x'`` to request exclusive creation. +(Contributed by Serhiy Storchaka in :issue:`21717`.) Other module-level changes ========================== -* Many functions in modules :mod:`mmap`, :mod:`ossaudiodev`, :mod:`socket`, - :mod:`ssl`, and :mod:`codecs`, now accept writable bytes-like objects. - (Contributed by Serhiy Storchaka in :issue:`23001`.) +Many functions in :mod:`mmap`, :mod:`ossaudiodev`, :mod:`socket`, +:mod:`ssl`, and :mod:`codecs` modules now accept writable bytes-like objects. +(Contributed by Serhiy Storchaka in :issue:`23001`.) Optimizations @@ -1374,6 +1525,7 @@ * Windows builds now require Microsoft Visual C++ 14.0, which is available as part of `Visual Studio 2015 `_. + Deprecated ========== -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 10 23:36:11 2015 From: python-checkins at python.org (yury.selivanov) Date: Thu, 10 Sep 2015 21:36:11 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogTWVyZ2UgMy41?= Message-ID: <20150910213609.66848.88731@psf.io> https://hg.python.org/cpython/rev/b81713d218a8 changeset: 97875:b81713d218a8 parent: 97873:8c436c36a4a7 parent: 97874:3265f33df731 user: Yury Selivanov date: Thu Sep 10 17:35:51 2015 -0400 summary: Merge 3.5 files: Doc/whatsnew/3.5.rst | 944 +++++++++++++++++------------- 1 files changed, 548 insertions(+), 396 deletions(-) diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -69,8 +69,8 @@ New syntax features: +* :pep:`492`, coroutines with async and await syntax. * :pep:`465`, a new matrix multiplication operator: ``a @ b``. -* :pep:`492`, coroutines with async and await syntax. * :pep:`448`, additional unpacking generalizations. New library modules: @@ -94,22 +94,24 @@ * New :exc:`RecursionError` exception. (Contributed by Georg Brandl in :issue:`19235`.) -Implementation improvements: +CPython implementation improvements: * When the ``LC_TYPE`` locale is the POSIX locale (``C`` locale), :py:data:`sys.stdin` and :py:data:`sys.stdout` are now using the ``surrogateescape`` error handler, instead of the ``strict`` error handler (:issue:`19977`). -* :pep:`488`, the elimination of ``.pyo`` files. +* ``.pyo`` files are no longer used and have been replaced by a more flexible + scheme that inclides the optimization level explicitly in ``.pyc`` name. + (:pep:`488`) -* :pep:`489`, multi-phase initialization of extension modules. +* Builtin and extension modules are now initialized in a multi-phase process, + which is similar to how Python modules are loaded. (:pep:`489`). Significantly Improved Library Modules: -* :class:`collections.OrderedDict` is now implemented in C, which improves - its performance between 4x to 100x times. Contributed by Eric Snow in - :issue:`16991`. +* :class:`collections.OrderedDict` is now implemented in C, which makes it + 4 to 100 times faster. Contributed by Eric Snow in :issue:`16991`. * You may now pass bytes to the :mod:`tempfile` module's APIs and it will return the temporary pathname as :class:`bytes` instead of :class:`str`. @@ -146,8 +148,8 @@ Windows improvements: -* A new installer for Windows has replaced the old MSI. See :ref:`using-on-windows` - for more information. +* A new installer for Windows has replaced the old MSI. + See :ref:`using-on-windows` for more information. * Windows builds now use Microsoft Visual C++ 14.0, and extension modules should use the same. @@ -252,6 +254,8 @@ PEP written and implemented by Yury Selivanov. +.. _whatsnew-pep-465: + PEP 465 - A dedicated infix operator for matrix multiplication -------------------------------------------------------------- @@ -297,6 +301,8 @@ PEP written by Nathaniel J. Smith; implemented by Benjamin Peterson. +.. _whatsnew-pep-448: + PEP 448 - Additional Unpacking Generalizations ---------------------------------------------- @@ -333,6 +339,8 @@ Thomas Wouters, and Joshua Landau. +.. _whatsnew-pep-461: + PEP 461 - % formatting support for bytes and bytearray ------------------------------------------------------ @@ -364,6 +372,8 @@ Ethan Furman. +.. _whatsnew-pep-484: + PEP 484 - Type Hints -------------------- @@ -389,11 +399,13 @@ implemented by Guido van Rossum. +.. _whatsnew-pep-471: + PEP 471 - os.scandir() function -- a better and faster directory iterator ------------------------------------------------------------------------- :pep:`471` adds a new directory iteration function, :func:`os.scandir`, -to the standard library. Additionally, :func:`os.walk` is now +to the standard library. Additionally, :func:`os.walk` is now implemented using :func:`os.scandir`, which speeds it up by 3-5 times on POSIX systems and by 7-20 times on Windows systems. @@ -403,6 +415,8 @@ PEP written and implemented by Ben Hoyt with the help of Victor Stinner. +.. _whatsnew-pep-475: + PEP 475: Retry system calls failing with EINTR ---------------------------------------------- @@ -455,6 +469,8 @@ Victor Stinner, with the help of Antoine Pitrou (the french connection). +.. _whatsnew-pep-479: + PEP 479: Change StopIteration handling inside generators -------------------------------------------------------- @@ -475,6 +491,8 @@ Chris Angelico, Yury Selivanov and Nick Coghlan. +.. _whatsnew-pep-486: + PEP 486: Make the Python Launcher aware of virtual environments --------------------------------------------------------------- @@ -489,6 +507,8 @@ PEP written and implemented by Paul Moore. +.. _whatsnew-pep-488: + PEP 488: Elimination of PYO files --------------------------------- @@ -497,9 +517,10 @@ need to constantly regenerate bytecode files, ``.pyc`` files now have an optional ``opt-`` tag in their name when the bytecode is optimized. This has the side-effect of no more bytecode file name clashes when running under either -``-O`` or ``-OO``. Consequently, bytecode files generated from ``-O``, and -``-OO`` may now exist simultaneously. :func:`importlib.util.cache_from_source` -has an updated API to help with this change. +:option:`-O` or :option:`-OO`. Consequently, bytecode files generated from +:option:`-O`, and :option:`-OO` may now exist simultaneously. +:func:`importlib.util.cache_from_source` has an updated API to help with +this change. .. seealso:: @@ -507,6 +528,8 @@ PEP written and implemented by Brett Cannon. +.. _whatsnew-pep-489: + PEP 489: Multi-phase extension module initialization ---------------------------------------------------- @@ -525,6 +548,8 @@ implemented by Petr Viktorin. +.. _whatsnew-pep-485: + PEP 485: A function for testing approximate equality ---------------------------------------------------- @@ -576,8 +601,8 @@ The new :mod:`zipapp` module (specified in :pep:`441`) provides an API and command line tool for creating executable Python Zip Applications, which -were introduced in Python 2.6 in :issue:`1739468` but which were not well -publicised, either at the time or since. +were introduced in Python 2.6 in :issue:`1739468`, but which were not well +publicized, either at the time or since. With the new module, bundling your application is as simple as putting all the files, including a ``__main__.py`` file, into a directory ``myapp`` @@ -586,6 +611,13 @@ $ python -m zipapp myapp $ python myapp.pyz +The module implementation has been contributed by Paul Moore in +:issue:`23491`. + +.. seealso:: + + :pep:`441` -- Improving Python ZIP Application Support + Improved Modules ================ @@ -593,191 +625,209 @@ argparse -------- -* :class:`~argparse.ArgumentParser` now allows to disable - :ref:`abbreviated usage ` of long options by setting - :ref:`allow_abbrev` to ``False``. - (Contributed by Jonathan Paugh, Steven Bethard, paul j3 and Daniel Eriksson - in :issue:`14910`.) +:class:`~argparse.ArgumentParser` now allows to disable +:ref:`abbreviated usage ` of long options by setting +:ref:`allow_abbrev` to ``False``. (Contributed by Jonathan Paugh, +Steven Bethard, paul j3 and Daniel Eriksson in :issue:`14910`.) + bz2 --- -* New option *max_length* for :meth:`~bz2.BZ2Decompressor.decompress` - to limit the maximum size of decompressed data. - (Contributed by Nikolaus Rath in :issue:`15955`.) +:meth:`~bz2.BZ2Decompressor.decompress` now accepts an optional *max_length* +argument to limit the maximum size of decompressed data. (Contributed by +Nikolaus Rath in :issue:`15955`.) + cgi --- -* :class:`~cgi.FieldStorage` now supports the context management protocol. - (Contributed by Berker Peksag in :issue:`20289`.) +:class:`~cgi.FieldStorage` now supports the context management protocol. +(Contributed by Berker Peksag in :issue:`20289`.) + cmath ----- -* :func:`cmath.isclose` function added. - (Contributed by Chris Barker and Tal Einat in :issue:`24270`.) +A new function :func:`cmath.isclose` provides a way to test for approximate +equality. (Contributed by Chris Barker and Tal Einat in :issue:`24270`.) code ---- -* The :func:`code.InteractiveInterpreter.showtraceback` method now prints - the full chained traceback, just like the interactive interpreter. - (Contributed by Claudiu Popa in :issue:`17442`.) +The :func:`code.InteractiveInterpreter.showtraceback` method now prints +the full chained traceback, just like the interactive interpreter. +(Contributed by Claudiu Popa in :issue:`17442`.) + collections ----------- -* You can now update docstrings produced by :func:`collections.namedtuple`:: +Docstrings produced by :func:`collections.namedtuple` can now be updated:: Point = namedtuple('Point', ['x', 'y']) Point.__doc__ = 'ordered pair' Point.x.__doc__ = 'abscissa' Point.y.__doc__ = 'ordinate' - (Contributed by Berker Peksag in :issue:`24064`.) +(Contributed by Berker Peksag in :issue:`24064`.) -* :class:`~collections.deque` has new methods: - :meth:`~collections.deque.index`, :meth:`~collections.deque.insert`, and - :meth:`~collections.deque.copy`. This allows deques to be registered as a - :class:`~collections.abc.MutableSequence` and improves their - substitutablity for lists. - (Contributed by Raymond Hettinger :issue:`23704`.) +The :class:`~collections.deque` now defines +:meth:`~collections.deque.index`, :meth:`~collections.deque.insert`, and +:meth:`~collections.deque.copy`. This allows deques to be recognized as a +:class:`~collections.abc.MutableSequence` and improves their substitutablity +for lists. (Contributed by Raymond Hettinger :issue:`23704`.) -* :class:`~collections.UserString` now supports ``__getnewargs__``, - ``__rmod__``, ``casefold``, ``format_map``, ``isprintable``, and - ``maketrans`` methods. - (Contributed by Joe Jevnik in :issue:`22189`.) +:class:`~collections.UserString` now implements :meth:`__getnewargs__`, +:meth:`__rmod__`, :meth:`casefold`, :meth:`format_map`, :meth:`isprintable`, and +:meth:`maketrans` methods to match corresponding methods of :class:`str`. +(Contributed by Joe Jevnik in :issue:`22189`.) -* :class:`collections.OrderedDict` is now implemented in C, which improves - its performance between 4x to 100x times. - (Contributed by Eric Snow in :issue:`16991`.) +:class:`collections.OrderedDict` is now implemented in C, which makes it 4x to +100x faster. (Contributed by Eric Snow in :issue:`16991`.) + collections.abc --------------- -* New :class:`~collections.abc.Generator` abstract base class. - (Contributed by Stefan Behnel in :issue:`24018`.) +New :class:`~collections.abc.Generator` abstract base class. (Contributed +by Stefan Behnel in :issue:`24018`.) -* New :class:`~collections.abc.Coroutine`, - :class:`~collections.abc.AsyncIterator`, and - :class:`~collections.abc.AsyncIterable` abstract base classes. - (Contributed by Yury Selivanov in :issue:`24184`.) +New :class:`~collections.abc.Coroutine`, +:class:`~collections.abc.AsyncIterator`, and +:class:`~collections.abc.AsyncIterable` abstract base classes. +(Contributed by Yury Selivanov in :issue:`24184`.) + compileall ---------- -* :func:`compileall.compile_dir` and :mod:`compileall`'s command-line interface - can now do parallel bytecode compilation. - (Contributed by Claudiu Popa in :issue:`16104`.) +A new :mod:`compileall` option, :option:`-j N`, allows to run ``N`` workers +sumultaneously to perform parallel bytecode compilation. :func:`compileall.compile_dir` has a corresponding ``workers`` parameter. (Contributed by +Claudiu Popa in :issue:`16104`.) -* *quiet* parameter of :func:`compileall.compile_dir`, - :func:`compileall.compile_file`, and :func:`compileall.compile_path` - functions now has a multilevel value. New ``-qq`` command line option - is available for suppressing the output. - (Contributed by Thomas Kluyver in :issue:`21338`.) +The :option:`-q` command line option can now be specified more than once, in +which case all output, including errors, will be suppressed. The corresponding +``quiet`` parameter in :func:`compileall.compile_dir`, :func:`compileall. +compile_file`, and :func:`compileall.compile_path` can now accept +an integer value indicating the level of output suppression. +(Contributed by Thomas Kluyver in :issue:`21338`.) + concurrent.futures ------------------ -* :meth:`~concurrent.futures.Executor.map` now takes a *chunksize* - argument to allow batching of tasks in child processes and improve - performance of ProcessPoolExecutor. - (Contributed by Dan O'Reilly in :issue:`11271`.) +:meth:`~concurrent.futures.Executor.map` now accepts a *chunksize* +argument to allow batching of tasks in child processes and improve performance +of ProcessPoolExecutor. (Contributed by Dan O'Reilly in :issue:`11271`.) + contextlib ---------- -* The new :func:`contextlib.redirect_stderr` context manager(similar to - :func:`contextlib.redirect_stdout`) makes it easier for utility scripts to - handle inflexible APIs that write their output to :data:`sys.stderr` and - don't provide any options to redirect it. - (Contributed by Berker Peksag in :issue:`22389`.) +The new :func:`contextlib.redirect_stderr` context manager (similar to +:func:`contextlib.redirect_stdout`) makes it easier for utility scripts to +handle inflexible APIs that write their output to :data:`sys.stderr` and +don't provide any options to redirect it. (Contributed by Berker Peksag in +:issue:`22389`.) + curses ------ -* The new :func:`curses.update_lines_cols` function updates the variables - :envvar:`curses.LINES` and :envvar:`curses.COLS`. + +The new :func:`curses.update_lines_cols` function updates the variables +:envvar:`curses.LINES` and :envvar:`curses.COLS`. + difflib ------- -* The charset of the HTML document generated by :meth:`difflib.HtmlDiff.make_file` - can now be customized by using *charset* keyword-only parameter. The default - charset of HTML document changed from ``'ISO-8859-1'`` to ``'utf-8'``. - (Contributed by Berker Peksag in :issue:`2052`.) +The charset of the HTML document generated by :meth:`difflib.HtmlDiff.make_file` +can now be customized by using *charset* keyword-only parameter. The default +charset of HTML document changed from ``'ISO-8859-1'`` to ``'utf-8'``. +(Contributed by Berker Peksag in :issue:`2052`.) -* It's now possible to compare lists of byte strings with - :func:`difflib.diff_bytes`. This fixes a regression from Python 2. - (Contributed by Terry J. Reedy and Greg Ward in :issue:`17445`.) +It is now possible to compare lists of byte strings with +:func:`difflib.diff_bytes`. This fixes a regression from Python 2. +(Contributed by Terry J. Reedy and Greg Ward in :issue:`17445`.) + distutils --------- -* The ``build`` and ``build_ext`` commands now accept a ``-j`` - option to enable parallel building of extension modules. - (Contributed by Antoine Pitrou in :issue:`5309`.) +The ``build`` and ``build_ext`` commands now accept a :option:`-j` option to +enable parallel building of extension modules. +(Contributed by Antoine Pitrou in :issue:`5309`.) -* Added support for the LZMA compression. - (Contributed by Serhiy Storchaka in :issue:`16314`.) +:mod:`distutils` now supports ``xz`` compression, and can be enabled by +passing ``xztar`` as an argument to ``bdist --format``. +(Contributed by Serhiy Storchaka in :issue:`16314`.) + doctest ------- -* :func:`doctest.DocTestSuite` returns an empty :class:`unittest.TestSuite` if - *module* contains no docstrings instead of raising :exc:`ValueError`. - (Contributed by Glenn Jones in :issue:`15916`.) +:func:`doctest.DocTestSuite` returns an empty :class:`unittest.TestSuite` if +*module* contains no docstrings instead of raising :exc:`ValueError`. +(Contributed by Glenn Jones in :issue:`15916`.) + email ----- -* A new policy option :attr:`~email.policy.Policy.mangle_from_` controls - whether or not lines that start with "From " in email bodies are prefixed with - a '>' character by generators. The default is ``True`` for - :attr:`~email.policy.compat32` and ``False`` for all other policies. - (Contributed by Milan Oberkirch in :issue:`20098`.) +A new policy option :attr:`~email.policy.Policy.mangle_from_` controls +whether or not lines that start with ``"From "`` in email bodies are prefixed +with a ``'>'`` character by generators. The default is ``True`` for +:attr:`~email.policy.compat32` and ``False`` for all other policies. +(Contributed by Milan Oberkirch in :issue:`20098`.) -* A new method :meth:`~email.message.Message.get_content_disposition` provides - easy access to a canonical value for the :mailheader:`Content-Disposition` - header (``None`` if there is no such header). (Contributed by Abhilash Raj - in :issue:`21083`.) +A new method :meth:`~email.message.Message.get_content_disposition` provides +easy access to a canonical value for the :mailheader:`Content-Disposition` +header (``None`` if there is no such header). (Contributed by Abhilash Raj +in :issue:`21083`.) -* A new policy option :attr:`~email.policy.EmailPolicy.utf8` can be set - ``True`` to encode email headers using the utf8 charset instead of using - encoded words. This allows ``Messages`` to be formatted according to - :rfc:`6532` and used with an SMTP server that supports the :rfc:`6531` - ``SMTPUTF8`` extension. (Contributed by R. David Murray in :issue:`24211`.) +A new policy option :attr:`~email.policy.EmailPolicy.utf8` can be set +to ``True`` to encode email headers using the UTF-8 charset instead of using +encoded words. This allows ``Messages`` to be formatted according to +:rfc:`6532` and used with an SMTP server that supports the :rfc:`6531` +``SMTPUTF8`` extension. (Contributed by R. David Murray in :issue:`24211`.) + faulthandler ------------ -* :func:`~faulthandler.enable`, :func:`~faulthandler.register`, - :func:`~faulthandler.dump_traceback` and - :func:`~faulthandler.dump_traceback_later` functions now accept file - descriptors. (Contributed by Wei Wu in :issue:`23566`.) +:func:`~faulthandler.enable`, :func:`~faulthandler.register`, +:func:`~faulthandler.dump_traceback` and +:func:`~faulthandler.dump_traceback_later` functions now accept file +descriptors in addition to file-like objects. +(Contributed by Wei Wu in :issue:`23566`.) + functools --------- -* Most of :func:`~functools.lru_cache` machinery is now implemented in C. - (Contributed by Matt Joiner, Alexey Kachayev, and Serhiy Storchaka - in :issue:`14373`.) +Most of :func:`~functools.lru_cache` machinery is now implemented in C, making +it significantly faster. (Contributed by Matt Joiner, Alexey Kachayev, and +Serhiy Storchaka in :issue:`14373`.) + glob ---- -00002-5938667 +:func:`~glob.iglob` and :func:`~glob.glob` now support recursive search in +subdirectories using the "``**``" pattern. (Contributed by Serhiy Storchaka +in :issue:`13968`.) -* :func:`~glob.iglob` and :func:`~glob.glob` now support recursive search in - subdirectories using the "``**``" pattern. - (Contributed by Serhiy Storchaka in :issue:`13968`.) heapq ----- -* :func:`~heapq.merge` has two new optional parameters ``reverse`` and - ``key``. (Contributed by Raymond Hettinger in :issue:`13742`.) +Element comparison in :func:`~heapq.merge` can now be customized by +passing a :term:`key function` in a new optional ``key`` keyword argument. +A new optional ``reverse`` keyword argument can be used to reverse element +comparison. (Contributed by Raymond Hettinger in :issue:`13742`.) + idlelib and IDLE ---------------- @@ -788,512 +838,613 @@ as well as changes made in future 3.5.x releases. This file is also available from the IDLE Help -> About Idle dialog. + imaplib ------- -* :class:`IMAP4` now supports the context management protocol. When used in a - :keyword:`with` statement, the IMAP4 ``LOGOUT`` command will be called - automatically at the end of the block. (Contributed by Tarek Ziad? and - Serhiy Storchaka in :issue:`4972`.) +:class:`~imaplib.IMAP4` now supports context manager protocol. +When used in a :keyword:`with` statement, the IMAP4 ``LOGOUT`` +command will be called automatically at the end of the block. +(Contributed by Tarek Ziad? and Serhiy Storchaka in :issue:`4972`.) -* :mod:`imaplib` now supports :rfc:`5161`: the :meth:`~imaplib.IMAP4.enable` - extension), and :rfc:`6855`: utf-8 support (internationalized email, via the - ``UTF8=ACCEPT`` argument to :meth:`~imaplib.IMAP4.enable`). A new attribute, - :attr:`~imaplib.IMAP4.utf8_enabled`, tracks whether or not :rfc:`6855` - support is enabled. Milan Oberkirch, R. David Murray, and Maciej Szulik in - :issue:`21800`.) +:mod:`imaplib` now supports :rfc:`5161` (``ENABLE`` extension) via +:meth:`~imaplib.IMAP4.enable`, and :rfc:`6855` (UTF-8 support) via the +``UTF8=ACCEPT`` argument to :meth:`~imaplib.IMAP4.enable`. A new attribute, +:attr:`~imaplib.IMAP4.utf8_enabled`, tracks whether or not :rfc:`6855` +support is enabled. (Contributed by Milan Oberkirch, R. David Murray, +and Maciej Szulik in :issue:`21800`.) -* :mod:`imaplib` now automatically encodes non-ASCII string usernames and - passwords using ``UTF8``, as recommended by the RFCs. (Contributed by Milan - Oberkirch in :issue:`21800`.) +:mod:`imaplib` now automatically encodes non-ASCII string usernames and +passwords using UTF-8, as recommended by the RFCs. (Contributed by Milan +Oberkirch in :issue:`21800`.) + imghdr ------ -* :func:`~imghdr.what` now recognizes the `OpenEXR `_ - format (contributed by Martin Vignali and Claudiu Popa in :issue:`20295`), - and the `WebP `_ format (contributed - by Fabrice Aneche and Claudiu Popa in :issue:`20197`.) +:func:`~imghdr.what` now recognizes the `OpenEXR `_ +format (contributed by Martin Vignali and Claudiu Popa in :issue:`20295`), +and the `WebP `_ format (contributed +by Fabrice Aneche and Claudiu Popa in :issue:`20197`.) + importlib --------- -* :class:`importlib.util.LazyLoader` allows for the lazy loading of modules in - applications where startup time is paramount. - (Contributed by Brett Cannon in :issue:`17621`.) +:class:`importlib.util.LazyLoader` allows for lazy loading of modules in +applications where startup time is important. (Contributed by Brett Cannon +in :issue:`17621`.) -* :func:`importlib.abc.InspectLoader.source_to_code` is now a - static method to make it easier to work with source code in a string. - With a module object that you want to initialize you can then use - ``exec(code, module.__dict__)`` to execute the code in the module. +:func:`importlib.abc.InspectLoader.source_to_code` is now a +static method. This makes it easier to initialize a module object with +code compiled from a string by runnning ``exec(code, module.__dict__)``. +(Contributed by Brett Cannon in :issue:`21156`.) -* :func:`importlib.util.module_from_spec` is now the preferred way to create a - new module. Compared to :class:`types.ModuleType`, this new function will set - the various import-controlled attributes based on the passed-in spec object. - (Contributed by Brett Cannon in :issue:`20383`.) +:func:`importlib.util.module_from_spec` is now the preferred way to create a +new module. Compared to :class:`types.ModuleType`, this new function will set +the various import-controlled attributes based on the passed-in spec object. +(Contributed by Brett Cannon in :issue:`20383`.) + inspect ------- -* :class:`inspect.Signature` and :class:`inspect.Parameter` are now - picklable and hashable. (Contributed by Yury Selivanov in :issue:`20726` - and :issue:`20334`.) +:class:`inspect.Signature` and :class:`inspect.Parameter` are now +picklable and hashable. (Contributed by Yury Selivanov in :issue:`20726` +and :issue:`20334`.) -* New method :meth:`inspect.BoundArguments.apply_defaults`. (Contributed - by Yury Selivanov in :issue:`24190`.) +A new method :meth:`inspect.BoundArguments.apply_defaults` provides a way +to set default values for missing arguments. (Contributed by Yury Selivanov +in :issue:`24190`.) -* New class method :meth:`inspect.Signature.from_callable`, which makes - subclassing of :class:`~inspect.Signature` easier. (Contributed - by Yury Selivanov and Eric Snow in :issue:`17373`.) +A new class method :meth:`inspect.Signature.from_callable` makes +subclassing of :class:`~inspect.Signature` easier. (Contributed +by Yury Selivanov and Eric Snow in :issue:`17373`.) -* New argument ``follow_wrapped`` for :func:`inspect.signature`. - (Contributed by Yury Selivanov in :issue:`20691`.) +:func:`inspect.signature` now accepts a ``follow_wrapped`` optional keyword +argument, which, when set to ``False``, disables automatic following of +``__wrapped__`` links. (Contributed by Yury Selivanov in :issue:`20691`.) -* New :func:`~inspect.iscoroutine`, :func:`~inspect.iscoroutinefunction` - and :func:`~inspect.isawaitable` functions. (Contributed by - Yury Selivanov in :issue:`24017`.) +A set of new functions to inspect +:term:`coroutine functions ` and +``coroutine objects`` as been added: +:func:`~inspect.iscoroutine`, :func:`~inspect.iscoroutinefunction`, +:func:`~inspect.isawaitable`, :func:`~inspect.getcoroutinelocals`, +and :func:`~inspect.getcoroutinestate`. +(Contributed by Yury Selivanov in :issue:`24017` and :issue:`24400`.) -* New :func:`~inspect.getcoroutinelocals` and :func:`~inspect.getcoroutinestate` - functions. (Contributed by Yury Selivanov in :issue:`24400`.) +:func:`~inspect.stack`, :func:`~inspect.trace`, :func:`~inspect.getouterframes`, +and :func:`~inspect.getinnerframes` now return a list of named tuples. +(Contributed by Daniel Shahaf in :issue:`16808`.) -* :func:`~inspect.stack`, :func:`~inspect.trace`, :func:`~inspect.getouterframes`, - and :func:`~inspect.getinnerframes` now return a list of named tuples. - (Contributed by Daniel Shahaf in :issue:`16808`.) io -- -* New Python implementation of :class:`io.FileIO` to make dependency on - ``_io`` module optional. - (Contributed by Serhiy Storchaka in :issue:`21859`.) +:class:`io.FileIO` has been implemented in Python which makes C implementation +of :mod:`io` module entirely optional. (Contributed by Serhiy Storchaka +in :issue:`21859`.) + ipaddress --------- -* :class:`ipaddress.IPv4Network` and :class:`ipaddress.IPv6Network` now - accept an ``(address, netmask)`` tuple argument, so as to easily construct - network objects from existing addresses. (Contributed by Peter Moody - and Antoine Pitrou in :issue:`16531`.) +:class:`ipaddress.IPv4Network` and :class:`ipaddress.IPv6Network` now +accept an ``(address, netmask)`` tuple argument, so as to easily construct +network objects from existing addresses. (Contributed by Peter Moody +and Antoine Pitrou in :issue:`16531`.) + json ---- -* The output of :mod:`json.tool` command line interface is now in the same - order as the input. Use the :option:`--sort-keys` option to sort the output - of dictionaries alphabetically by key. (Contributed by Berker Peksag in - :issue:`21650`.) +:mod:`json.tool` command line interface now preserves the order of keys in +JSON objects passed in input. The new :option:`--sort-keys` option can be used +to sort the keys alphabetically. (Contributed by Berker Peksag +in :issue:`21650`.) -* JSON decoder now raises :exc:`json.JSONDecodeError` instead of - :exc:`ValueError`. (Contributed by Serhiy Storchaka in :issue:`19361`.) +JSON decoder now raises :exc:`json.JSONDecodeError` instead of +:exc:`ValueError`. (Contributed by Serhiy Storchaka in :issue:`19361`.) + locale ------ - * New :func:`~locale.delocalize` function to convert a string into a - normalized number string, following the ``LC_NUMERIC`` settings. - (Contributed by C?dric Krier in :issue:`13918`.) +A new :func:`~locale.delocalize` function can be used to convert a string into +a normalized number string, taking the ``LC_NUMERIC`` settings into account. +(Contributed by C?dric Krier in :issue:`13918`.) + logging ------- -* All logging methods :meth:`~logging.Logger.log`, :meth:`~logging.Logger.exception`, - :meth:`~logging.Logger.critical`, :meth:`~logging.Logger.debug`, etc, - now accept exception instances for ``exc_info`` parameter, in addition - to boolean values and exception tuples. - (Contributed by Yury Selivanov in :issue:`20537`.) +All logging methods (:meth:`~logging.Logger.log`, +:meth:`~logging.Logger.exception`, :meth:`~logging.Logger.critical`, +:meth:`~logging.Logger.debug`, etc.), now accept exception instances +in ``exc_info`` parameter, in addition to boolean values and exception +tuples. (Contributed by Yury Selivanov in :issue:`20537`.) -* :class:`~logging.handlers.HTTPHandler` now accepts optional - :class:`ssl.SSLContext` instance to configure the SSL settings used - for HTTP connection. - (Contributed by Alex Gaynor in :issue:`22788`.) +:class:`~logging.handlers.HTTPHandler` now accepts an optional +:class:`ssl.SSLContext` instance to configure the SSL settings used +in an HTTP connection. (Contributed by Alex Gaynor in :issue:`22788`.) -* :class:`~logging.handlers.QueueListener` now takes a *respect_handler_level* - keyword argument which, if set to ``True``, will pass messages to handlers - taking handler levels into account. (Contributed by Vinay Sajip.) +:class:`~logging.handlers.QueueListener` now takes a *respect_handler_level* +keyword argument which, if set to ``True``, will pass messages to handlers +taking handler levels into account. (Contributed by Vinay Sajip.) + lzma ---- -* New option *max_length* for :meth:`~lzma.LZMADecompressor.decompress` - to limit the maximum size of decompressed data. - (Contributed by Martin Panter in :issue:`15955`.) +:meth:`~lzma.LZMADecompressor.decompress` now accepts an optional *max_length* +argument to limit the maximum size of decompressed data. +(Contributed by Martin Panter in :issue:`15955`.) + math ---- -* :data:`math.inf` and :data:`math.nan` constants added. (Contributed by Mark - Dickinson in :issue:`23185`.) +Two new constants have been added to :mod:`math`: :data:`math.inf` +and :data:`math.nan`. (Contributed by Mark Dickinson in :issue:`23185`.) -* New :func:`~math.isclose` function. - (Contributed by Chris Barker and Tal Einat in :issue:`24270`.) +A new function :func:`math.isclose` provides a way to test for approximate +equality. (Contributed by Chris Barker and Tal Einat in :issue:`24270`.) -* New :func:`~math.gcd` function. The :func:`fractions.gcd` function now is - deprecated. - (Contributed by Mark Dickinson and Serhiy Storchaka in :issue:`22486`.) +A new :func:`~math.gcd` function has been added. The :func:`fractions.gcd` +function is now deprecated. (Contributed by Mark Dickinson and Serhiy +Storchaka in :issue:`22486`.) + operator -------- -* :func:`~operator.attrgetter`, :func:`~operator.itemgetter`, and - :func:`~operator.methodcaller` objects now support pickling. - (Contributed by Josh Rosenberg and Serhiy Storchaka in :issue:`22955`.) +:func:`~operator.attrgetter`, :func:`~operator.itemgetter`, and +:func:`~operator.methodcaller` objects now support pickling. +(Contributed by Josh Rosenberg and Serhiy Storchaka in :issue:`22955`.) + os -- -* New :func:`os.scandir` function that exposes file information from - the operating system when listing a directory. :func:`os.scandir` - returns an iterator of :class:`os.DirEntry` objects corresponding to - the entries in the directory given by *path*. (Contributed by Ben - Hoyt with the help of Victor Stinner in :issue:`22524`.) +The new :func:`os.scandir` returning an iterator of :class:`os.DirEntry` +objects has been added. If possible, :func:`os.scandir` extracts file +attributes while scanning a directory, removing the need to perform +subsequent system calls to determine file type or attributes, which may +significantly improve performance. (Contributed by Ben Hoyt with the help +of Victor Stinner in :issue:`22524`.) -* :class:`os.stat_result` now has a :attr:`~os.stat_result.st_file_attributes` - attribute on Windows. (Contributed by Ben Hoyt in :issue:`21719`.) +On Windows, a new :attr:`~os.stat_result.st_file_attributes` attribute is +now available. It corresponds to ``dwFileAttributes`` member of the +``BY_HANDLE_FILE_INFORMATION`` structure returned by +``GetFileInformationByHandle()``. (Contributed by Ben Hoyt in :issue:`21719`.) -* :func:`os.urandom`: On Linux 3.17 and newer, the ``getrandom()`` syscall is - now used when available. On OpenBSD 5.6 and newer, the C ``getentropy()`` - function is now used. These functions avoid the usage of an internal file - descriptor. - (Contributed by Victor Stinner in :issue:`22181`.) +:func:`os.urandom` now uses ``getrandom()`` syscall on Linux 3.17 or newer, +and ``getentropy()`` on OpenBSD 5.6 and newer, removing the need to use +``/dev/urandom`` and avoiding failures due to potential file descriptor +exhaustion. (Contributed by Victor Stinner in :issue:`22181`.) -* New :func:`os.get_blocking` and :func:`os.set_blocking` functions to - get and set the blocking mode of file descriptors. - (Contributed by Victor Stinner in :issue:`22054`.) +New :func:`os.get_blocking` and :func:`os.set_blocking` functions allow to +get and set the file descriptor blocking mode (:data:`~os.O_NONBLOCK`.) +(Contributed by Victor Stinner in :issue:`22054`.) -* :func:`~os.truncate` and :func:`~os.ftruncate` are now supported on - Windows. (Contributed by Steve Dower in :issue:`23668`.) +:func:`~os.truncate` and :func:`~os.ftruncate` are now supported on +Windows. (Contributed by Steve Dower in :issue:`23668`.) + os.path ------- -* New :func:`~os.path.commonpath` function that extracts common path prefix. - Unlike the :func:`~os.path.commonprefix` function, it always returns a valid - path. (Contributed by Rafik Draoui and Serhiy Storchaka in :issue:`10395`.) +There is a new :func:`~os.path.commonpath` function returning the longest +common sub-path of each passed pathname. Unlike the +:func:`~os.path.commonprefix` function, it always returns a valid +path. (Contributed by Rafik Draoui and Serhiy Storchaka in :issue:`10395`.) + pathlib ------- -* New :meth:`~pathlib.Path.samefile` method to check if other path object - points to the same file. (Contributed by Vajrasky Kok and Antoine Pitrou - in :issue:`19775`.) +The new :meth:`~pathlib.Path.samefile` method can be used to check if the +passed :class:`~pathlib.Path` object, or a string, point to the same file as +the :class:`~pathlib.Path` on which :meth:`~pathlib.Path.samefile` is called. +(Contributed by Vajrasky Kok and Antoine Pitrou in :issue:`19775`.) -* :meth:`~pathlib.Path.mkdir` has a new optional parameter ``exist_ok`` - to mimic ``mkdir -p`` and :func:`os.makrdirs` functionality. - (Contributed by Berker Peksag in :issue:`21539`.) +:meth:`~pathlib.Path.mkdir` how accepts a new optional ``exist_ok`` argument +to match ``mkdir -p`` and :func:`os.makrdirs` functionality. +(Contributed by Berker Peksag in :issue:`21539`.) -* New :meth:`~pathlib.Path.expanduser` to expand ``~`` and ``~user`` - constructs. - (Contributed by Serhiy Storchaka and Claudiu Popa in :issue:`19776`.) +There is a new :meth:`~pathlib.Path.expanduser` method to expand ``~`` +and ``~user`` prefixes. (Contributed by Serhiy Storchaka and Claudiu +Popa in :issue:`19776`.) -* New class method :meth:`~pathlib.Path.home` to get an instance of - :class:`~pathlib.Path` object representing the user?s home directory. - (Contributed by Victor Salgado and Mayank Tripathi in :issue:`19777`.) +A new :meth:`~pathlib.Path.home` class method can be used to get an instance +of :class:`~pathlib.Path` object representing the user?s home directory. +(Contributed by Victor Salgado and Mayank Tripathi in :issue:`19777`.) + pickle ------ -* Serializing more "lookupable" objects (such as unbound methods or nested - classes) now are supported with pickle protocols < 4. - (Contributed by Serhiy Storchaka in :issue:`23611`.) +Nested objects, such as unbound methods or nested classes, can now be pickled using :ref:`pickle protocols ` older than protocol version 4, +which already supported these cases. (Contributed by Serhiy Storchaka in +:issue:`23611`.) + poplib ------ -* A new command :meth:`~poplib.POP3.utf8` enables :rfc:`6856` - (internationalized email) support if the POP server supports it. (Contributed - by Milan OberKirch in :issue:`21804`.) +A new command :meth:`~poplib.POP3.utf8` enables :rfc:`6856` +(internationalized email) support, if the POP server supports it. +(Contributed by Milan OberKirch in :issue:`21804`.) + re -- -* Number of capturing groups in regular expression is no longer limited by 100. - (Contributed by Serhiy Storchaka in :issue:`22437`.) +The number of capturing groups in regular expression is no longer limited by +100. (Contributed by Serhiy Storchaka in :issue:`22437`.) -* Now unmatched groups are replaced with empty strings in :func:`re.sub` - and :func:`re.subn`. (Contributed by Serhiy Storchaka in :issue:`1519638`.) +:func:`re.sub` and :func:`re.subn` now replace unmatched groups with empty +strings instead of rising an exception. (Contributed by Serhiy Storchaka +in :issue:`1519638`.) + readline -------- -* New :func:`~readline.append_history_file` function. - (Contributed by Bruno Cauet in :issue:`22940`.) +The new :func:`~readline.append_history_file` function can be used to append +the specified number of trailing elements in history to a given file. +(Contributed by Bruno Cauet in :issue:`22940`.) + shutil ------ -* :func:`~shutil.move` now accepts a *copy_function* argument, allowing, - for example, :func:`~shutil.copy` to be used instead of the default - :func:`~shutil.copy2` if there is a need to ignore metadata. (Contributed by - Claudiu Popa in :issue:`19840`.) +:func:`~shutil.move` now accepts a *copy_function* argument, allowing, +for example, :func:`~shutil.copy` to be used instead of the default +:func:`~shutil.copy2` there is a need to ignore file metadata when moving. +(Contributed by Claudiu Popa in :issue:`19840`.) -* :func:`~shutil.make_archive` now supports *xztar* format. - (Contributed by Serhiy Storchaka in :issue:`5411`.) +:func:`~shutil.make_archive` now supports *xztar* format. +(Contributed by Serhiy Storchaka in :issue:`5411`.) + signal ------ -* On Windows, :func:`signal.set_wakeup_fd` now also supports socket handles. - (Contributed by Victor Stinner in :issue:`22018`.) +On Windows, :func:`signal.set_wakeup_fd` now also supports socket handles. +(Contributed by Victor Stinner in :issue:`22018`.) -* Different constants of :mod:`signal` module are now enumeration values using - the :mod:`enum` module. This allows meaningful names to be printed during - debugging, instead of integer ?magic numbers?. (Contributed by Giampaolo - Rodola' in :issue:`21076`.) +Various ``SIG*`` constants in :mod:`signal` module have been converted into +:mod:`Enums `. This allows meaningful names to be printed +during debugging, instead of integer "magic numbers". +(Contributed by Giampaolo Rodola' in :issue:`21076`.) + smtpd ----- -* Both :class:`~smtpd.SMTPServer` and :class:`smtpd.SMTPChannel` now accept a - *decode_data* keyword to determine if the DATA portion of the SMTP - transaction is decoded using the ``utf-8`` codec or is instead provided to - :meth:`~smtpd.SMTPServer.process_message` as a byte string. The default - is ``True`` for backward compatibility reasons, but will change to ``False`` - in Python 3.6. If *decode_data* is set to ``False``, the - :meth:`~smtpd.SMTPServer.process_message` method must be prepared to accept - keyword arguments. (Contributed by Maciej Szulik in :issue:`19662`.) +Both :class:`~smtpd.SMTPServer` and :class:`smtpd.SMTPChannel` now accept a +*decode_data* keyword argument to determine if the ``DATA`` portion of the SMTP +transaction is decoded using the ``"utf-8"`` codec or is instead provided to +:meth:`~smtpd.SMTPServer.process_message` as a byte string. The default +is ``True`` for backward compatibility reasons, but will change to ``False`` +in Python 3.6. If *decode_data* is set to ``False``, the +:meth:`~smtpd.SMTPServer.process_message` method must be prepared to accept +keyword arguments. (Contributed by Maciej Szulik in :issue:`19662`.) -* :class:`~smtpd.SMTPServer` now advertises the ``8BITMIME`` extension - (:rfc:`6152`) if if *decode_data* has been set ``True``. If the client - specifies ``BODY=8BITMIME`` on the ``MAIL`` command, it is passed to - :meth:`~smtpd.SMTPServer.process_message` via the ``mail_options`` keyword. - (Contributed by Milan Oberkirch and R. David Murray in :issue:`21795`.) +:class:`~smtpd.SMTPServer` now advertises the ``8BITMIME`` extension +(:rfc:`6152`) if if *decode_data* has been set ``True``. If the client +specifies ``BODY=8BITMIME`` on the ``MAIL`` command, it is passed to +:meth:`~smtpd.SMTPServer.process_message` via the ``mail_options`` keyword. +(Contributed by Milan Oberkirch and R. David Murray in :issue:`21795`.) -* :class:`~smtpd.SMTPServer` now supports the ``SMTPUTF8`` extension - (:rfc:`6531`: Internationalized Email). If the client specified ``SMTPUTF8 - BODY=8BITMIME`` on the ``MAIL`` command, they are passed to - :meth:`~smtpd.SMTPServer.process_message` via the ``mail_options`` keyword. - It is the responsibility of the :meth:`~smtpd.SMTPServer.process_message` - method to correctly handle the ``SMTPUTF8`` data. (Contributed by Milan - Oberkirch in :issue:`21725`.) +:class:`~smtpd.SMTPServer` now supports the ``SMTPUTF8`` extension +(:rfc:`6531`: Internationalized Email). If the client specified ``SMTPUTF8 +BODY=8BITMIME`` on the ``MAIL`` command, they are passed to +:meth:`~smtpd.SMTPServer.process_message` via the ``mail_options`` keyword. +It is the responsibility of the :meth:`~smtpd.SMTPServer.process_message` +method to correctly handle the ``SMTPUTF8`` data. (Contributed by Milan +Oberkirch in :issue:`21725`.) -* It is now possible to provide, directly or via name resolution, IPv6 - addresses in the :class:`~smtpd.SMTPServer` constructor, and have it - successfully connect. (Contributed by Milan Oberkirch in :issue:`14758`.) +It is now possible to provide, directly or via name resolution, IPv6 +addresses in the :class:`~smtpd.SMTPServer` constructor, and have it +successfully connect. (Contributed by Milan Oberkirch in :issue:`14758`.) + smtplib ------- -* A new :meth:`~smtplib.SMTP.auth` method provides a convenient way to - implement custom authentication mechanisms. - (Contributed by Milan Oberkirch in :issue:`15014`.) +A new :meth:`~smtplib.SMTP.auth` method provides a convenient way to +implement custom authentication mechanisms. (Contributed by Milan +Oberkirch in :issue:`15014`.) -* Additional debuglevel (2) shows timestamps for debug messages in - :class:`smtplib.SMTP`. (Contributed by Gavin Chappell and Maciej Szulik in - :issue:`16914`.) +Additional debuglevel (2) shows timestamps for debug messages in +:class:`smtplib.SMTP`. (Contributed by Gavin Chappell and Maciej Szulik in +:issue:`16914`.) -* :mod:`smtplib` now supports :rfc:`6531` (SMTPUTF8) in both the - :meth:`~smtplib.SMTP.sendmail` and :meth:`~smtplib.SMTP.send_message` - commands. (Contributed by Milan Oberkirch and R. David Murray in - :issue:`22027`.) +:mod:`smtplib` now supports :rfc:`6531` (SMTPUTF8) in both the +:meth:`~smtplib.SMTP.sendmail` and :meth:`~smtplib.SMTP.send_message` +commands. (Contributed by Milan Oberkirch and R. David Murray in +:issue:`22027`.) + sndhdr ------ -* :func:`~sndhdr.what` and :func:`~sndhdr.whathdr` now return - :func:`~collections.namedtuple`. - (Contributed by Claudiu Popa in :issue:`18615`.) +:func:`~sndhdr.what` and :func:`~sndhdr.whathdr` now return +:func:`~collections.namedtuple`. (Contributed by Claudiu Popa in +:issue:`18615`.) + ssl --- -* The :meth:`~ssl.SSLSocket.do_handshake`, :meth:`~ssl.SSLSocket.read`, - :meth:`~ssl.SSLSocket.shutdown`, and :meth:`~ssl.SSLSocket.write` methods of - :class:`ssl.SSLSocket` don't reset the socket timeout anymore each time bytes - are received or sent. The socket timeout is now the maximum total duration of - the method. +Memory BIO Support +~~~~~~~~~~~~~~~~~~ -* Memory BIO Support: new classes :class:`~ssl.SSLObject`, - :class:`~ssl.MemoryBIO`, and new - :meth:`SSLContext.wrap_bio ` method. - (Contributed by Geert Jansen in :issue:`21965`.) +(Contributed by Geert Jansen in :issue:`21965`.) -* New :meth:`~ssl.SSLSocket.version` to query the actual protocol version - in use. (Contributed by Antoine Pitrou in :issue:`20421`.) +The new :class:`~ssl.SSLObject` class has been added to provide SSL protocol +support for cases when the network IO capabilities of :class:`~ssl.SSLSocket` +are not necessary or inappropriate. :class:`~ssl.SSLObject` represents +an SSL protocol instance, but does not implement any network IO methods, and +instead provides a memory buffer interface. The new :class:`~ssl.MemoryBIO` +class can be used to pass data between Python and an SSL protocol instance. -* New :meth:`~ssl.SSLObject.shared_ciphers` and - :meth:`~ssl.SSLSocket.shared_ciphers` methods to fetch the client's - list of ciphers sent at handshake. - (Contributed by Benjamin Peterson in :issue:`23186`.) +The memory BIO SSL support is primarily intended to be used in frameworks +implementing asynchronous IO for which :class:`~ssl.SSLObject` IO readiness +model ("select/poll") is inappropriate or inefficient. -* :func:`~ssl.match_hostname` now supports matching of IP addresses. - (Contributed by Antoine Pitrou in :issue:`23239`.) +A new :meth:`~ssl.SSLContext.wrap_bio` method can be used to create a new +:class:`~ssl.SSLObject` instance. + + +Application-Layer Protocol Negotiation Support +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +(Contributed by Benjamin Peterson in :issue:`20188`.) + +Where OpenSSL support is present, :mod:`ssl` module now implements * +Application-Layer Protocol Negotiation* TLS extension as described +in :rfc:`7301`. +The new :meth:`SSLContext.set_alpn_protocols ` +can be used to specify which protocols the socket should advertise during +the TLS handshake. The new +:meth:`SSLSocket.selected_alpn_protocol ` +returns the protocol that was selected during the TLS handshake. :data:`ssl.HAS_ALPN` flag indicates whether APLN support is present. + + +Other Changes +~~~~~~~~~~~~~ + +There is a new :meth:`~ssl.SSLSocket.version` method to query the actual +protocol version in use. (Contributed by Antoine Pitrou in :issue:`20421`.) + +:class:`~ssl.SSLSocket` now implementes :meth:`~ssl.SSLSocket.sendfile` +method. (Contributed by Giampaolo Rodola' in :issue:`17552`.) + +:meth:`ssl.SSLSocket.send()` now raises either :exc:`ssl.SSLWantReadError` +or :exc:`ssl.SSLWantWriteError` on a non-blocking socket if the operation +would block. Previously, it would return 0. (Contributed by Nikolaus Rath +in :issue:`20951`.) + +The :func:`~ssl.cert_time_to_seconds` function now interprets the input time +as UTC and not as local time, per :rfc:`5280`. Additionally, the return +value is always an :class:`int`. (Contributed by Akira Li in :issue:`19940`.) + +New :meth:`~ssl.SSLObject.shared_ciphers` and +:meth:`~ssl.SSLSocket.shared_ciphers` methods return the list of ciphers +sent by the client during the handshake. (Contributed by Benjamin Peterson +in :issue:`23186`.) + +The :meth:`~ssl.SSLSocket.do_handshake`, :meth:`~ssl.SSLSocket.read`, +:meth:`~ssl.SSLSocket.shutdown`, and :meth:`~ssl.SSLSocket.write` methods of +:class:`ssl.SSLSocket` no longer reset the socket timeout every time bytes +are received or sent. The socket timeout is now the maximum total duration of +the method. (Contributed by Victor Stinner in :issue:`23853`.) + +The :func:`~ssl.match_hostname` function now supports matching of IP addresses. +(Contributed by Antoine Pitrou in :issue:`23239`.) + socket ------ -* New :meth:`socket.socket.sendfile` method allows to send a file over a socket - by using high-performance :func:`os.sendfile` function on UNIX resulting in - uploads being from 2x to 3x faster than when using plain - :meth:`socket.socket.send`. - (Contributed by Giampaolo Rodola' in :issue:`17552`.) +A new :meth:`socket.socket.sendfile` method allows to send a file over a +socket by using high-performance :func:`os.sendfile` function on UNIX +resulting in uploads being from 2x to 3x faster than when using plain +:meth:`socket.socket.send`. (Contributed by Giampaolo Rodola' in +:issue:`17552`.) -* The :meth:`socket.socket.sendall` method don't reset the socket timeout - anymore each time bytes are received or sent. The socket timeout is now the - maximum total duration to send all data. +The :meth:`socket.socket.sendall` method no longer resets the socket timeout +every time bytes are received or sent. The socket timeout is now the +maximum total duration to send all data. (Contributed by Victor Stinner in +:issue:`23853`.) -* Functions with timeouts now use a monotonic clock, instead of a - system clock. (Contributed by Victor Stinner in :issue:`22043`.) +Functions with timeouts now use a monotonic clock, instead of a system clock. +(Contributed by Victor Stinner in :issue:`22043`.) + subprocess ---------- -* The new :func:`subprocess.run` function runs subprocesses and returns a - :class:`subprocess.CompletedProcess` object. It Provides a more consistent - API than :func:`~subprocess.call`, :func:`~subprocess.check_call` and - :func:`~subprocess.check_output`. +The new :func:`subprocess.run` function has been added and is the recommended +approach to invoking subprocesses. It runs the specified command and +and returns a :class:`subprocess.CompletedProcess` object. (Contributed by +Thomas Kluyver in :issue:`23342`.) + sys --- -* New :func:`~sys.set_coroutine_wrapper` and :func:`~sys.get_coroutine_wrapper` - functions. (Contributed by Yury Selivanov in :issue:`24017`.) +A new :func:`~sys.set_coroutine_wrapper` function allows setting a global +hook that will be called whenever a :ref:`coro object ` +is created. Essentially, it works like a global coroutine decorator. A +corresponding :func:`~sys.get_coroutine_wrapper` can be used to obtain +a currently set wrapper. Both functions are provisional, and are intended +for debugging purposes only. (Contributed by Yury Selivanov in :issue:`24017`.) -* New :func:`~sys.is_finalizing` to check for :term:`interpreter shutdown`. - (Contributed by Antoine Pitrou in :issue:`22696`.) +There is a new :func:`~sys.is_finalizing` function to check if the Python +interpreter is :term:`shutting down `. +(Contributed by Antoine Pitrou in :issue:`22696`.) + sysconfig --------- -* The user scripts directory on Windows is now versioned. - (Contributed by Paul Moore in :issue:`23437`.) +The name of the user scripts directory on Windows now includes the first +two components of Python version. (Contributed by Paul Moore +in :issue:`23437`.) + tarfile ------- -* The :func:`tarfile.open` function now supports ``'x'`` (exclusive creation) - mode. (Contributed by Berker Peksag in :issue:`21717`.) +The *mode* argument of :func:`tarfile.open` function now accepts ``'x'`` to request exclusive creation. (Contributed by Berker Peksag in :issue:`21717`.) -* The :meth:`~tarfile.TarFile.extractall` and :meth:`~tarfile.TarFile.extract` - methods now take a keyword parameter *numeric_only*. If set to ``True``, - the extracted files and directories will be owned by the numeric uid and gid - from the tarfile. If set to ``False`` (the default, and the behavior in - versions prior to 3.5), they will be owned by the named user and group in the - tarfile. (Contributed by Michael Vogt and Eric Smith in :issue:`23193`.) +The :meth:`~tarfile.TarFile.extractall` and :meth:`~tarfile.TarFile.extract` +methods now take a keyword argument *numeric_only*. If set to ``True``, +the extracted files and directories will be owned by the numeric uid and gid +from the tarfile. If set to ``False`` (the default, and the behavior in +versions prior to 3.5), they will be owned by the named user and group in the +tarfile. (Contributed by Michael Vogt and Eric Smith in :issue:`23193`.) + threading --------- -* :meth:`~threading.Lock.acquire` and :meth:`~threading.RLock.acquire` - now use a monotonic clock for managing timeouts. - (Contributed by Victor Stinner in :issue:`22043`.) +:meth:`~threading.Lock.acquire` and :meth:`~threading.RLock.acquire` +now use a monotonic clock for timeout management. (Contributed by Victor +Stinner in :issue:`22043`.) + time ---- -* The :func:`time.monotonic` function is now always available. (Contributed by - Victor Stinner in :issue:`22043`.) +The :func:`time.monotonic` function is now always available. (Contributed by +Victor Stinner in :issue:`22043`.) + timeit ------ -* New command line option ``-u`` or ``--unit=U`` to specify a time unit for - the timer output. Supported options are ``usec``, ``msec``, or ``sec``. - (Contributed by Julian Gindi in :issue:`18983`.) +New command line option :option:`-u` or :option:`--unit=U` to specify a time +unit for the timer output. Supported options are ``usec``, ``msec``, or ``sec``. +(Contributed by Julian Gindi in :issue:`18983`.) + tkinter ------- -* The :mod:`tkinter._fix` module used for setting up the Tcl/Tk environment - on Windows has been replaced by a private function in the :mod:`_tkinter` - module which makes no permanent changes to environment variables. - (Contributed by Zachary Ware in :issue:`20035`.) +The :mod:`tkinter._fix` module used for setting up the Tcl/Tk environment +on Windows has been replaced by a private function in the :mod:`_tkinter` +module which makes no permanent changes to environment variables. +(Contributed by Zachary Ware in :issue:`20035`.) + traceback --------- -* New :func:`~traceback.walk_stack` and :func:`~traceback.walk_tb` - functions to conveniently traverse frame and traceback objects. - (Contributed by Robert Collins in :issue:`17911`.) +New :func:`~traceback.walk_stack` and :func:`~traceback.walk_tb` +functions to conveniently traverse frame and traceback objects. +(Contributed by Robert Collins in :issue:`17911`.) -* New lightweight classes: :class:`~traceback.TracebackException`, - :class:`~traceback.StackSummary`, and :class:`traceback.FrameSummary`. - (Contributed by Robert Collins in :issue:`17911`.) +New lightweight classes: :class:`~traceback.TracebackException`, +:class:`~traceback.StackSummary`, and :class:`traceback.FrameSummary`. +(Contributed by Robert Collins in :issue:`17911`.) -* :func:`~traceback.print_tb` and :func:`~traceback.print_stack` now support - negative values for the *limit* argument. - (Contributed by Dmitry Kazakov in :issue:`22619`.) +:func:`~traceback.print_tb` and :func:`~traceback.print_stack` now support +negative values for the *limit* argument. +(Contributed by Dmitry Kazakov in :issue:`22619`.) types ----- -* New :func:`~types.coroutine` function. (Contributed by Yury Selivanov - in :issue:`24017`.) +New :func:`~types.coroutine` function. (Contributed by Yury Selivanov +in :issue:`24017`.) -* New :class:`~types.CoroutineType`. (Contributed by Yury Selivanov - in :issue:`24400`.) +New :class:`~types.CoroutineType`. (Contributed by Yury Selivanov +in :issue:`24400`.) + urllib ------ -* A new :class:`~urllib.request.HTTPPasswordMgrWithPriorAuth` allows HTTP Basic - Authentication credentials to be managed so as to eliminate unnecessary - ``401`` response handling, or to unconditionally send credentials - on the first request in order to communicate with servers that return a - ``404`` response instead of a ``401`` if the ``Authorization`` header is not - sent. (Contributed by Matej Cepl in :issue:`19494` and Akshit Khurana in - :issue:`7159`.) +A new :class:`~urllib.request.HTTPPasswordMgrWithPriorAuth` allows HTTP Basic +Authentication credentials to be managed so as to eliminate unnecessary +``401`` response handling, or to unconditionally send credentials +on the first request in order to communicate with servers that return a +``404`` response instead of a ``401`` if the ``Authorization`` header is not +sent. (Contributed by Matej Cepl in :issue:`19494` and Akshit Khurana in +:issue:`7159`.) -* A new :func:`~urllib.parse.urlencode` parameter *quote_via* provides a way to - control the encoding of query parts if needed. (Contributed by Samwyse and - Arnon Yaari in :issue:`13866`.) +A new :func:`~urllib.parse.urlencode` parameter *quote_via* provides a way to +control the encoding of query parts if needed. (Contributed by Samwyse and +Arnon Yaari in :issue:`13866`.) -* :func:`~urllib.request.urlopen` accepts an :class:`ssl.SSLContext` - object as a *context* argument, which will be used for the HTTPS - connection. (Contributed by Alex Gaynor in :issue:`22366`.) +:func:`~urllib.request.urlopen` accepts an :class:`ssl.SSLContext` +object as a *context* argument, which will be used for the HTTPS +connection. (Contributed by Alex Gaynor in :issue:`22366`.) + unicodedata ----------- -* The :mod:`unicodedata` module now uses data from `Unicode 8.0.0 - `_. +The :mod:`unicodedata` module now uses data from `Unicode 8.0.0 +`_. + unittest -------- -* New command line option ``--locals`` to show local variables in - tracebacks. - (Contributed by Robert Collins in :issue:`22936`.) +New command line option :option:`--locals` to show local variables in +tracebacks. (Contributed by Robert Collins in :issue:`22936`.) + wsgiref ------- -* *headers* parameter of :class:`wsgiref.headers.Headers` is now optional. - (Contributed by Pablo Torres Navarrete and SilentGhost in :issue:`5800`.) +*headers* parameter of :class:`wsgiref.headers.Headers` is now optional. +(Contributed by Pablo Torres Navarrete and SilentGhost in :issue:`5800`.) + xmlrpc ------ -* :class:`xmlrpc.client.ServerProxy` is now a :term:`context manager`. - (Contributed by Claudiu Popa in :issue:`20627`.) +:class:`xmlrpc.client.ServerProxy` is now a :term:`context manager`. +(Contributed by Claudiu Popa in :issue:`20627`.) -* :class:`~xmlrpc.client.ServerProxy` constructor now accepts an optional - :class:`ssl.SSLContext` instance. - (Contributed by Alex Gaynor in :issue:`22960`.) +:class:`~xmlrpc.client.ServerProxy` constructor now accepts an optional +:class:`ssl.SSLContext` instance. +(Contributed by Alex Gaynor in :issue:`22960`.) + xml.sax ------- -* SAX parsers now support a character stream of - :class:`~xml.sax.xmlreader.InputSource` object. - (Contributed by Serhiy Storchaka in :issue:`2175`.) +SAX parsers now support a character stream of +:class:`~xml.sax.xmlreader.InputSource` object. +(Contributed by Serhiy Storchaka in :issue:`2175`.) + zipfile ------- -* Added support for writing ZIP files to unseekable streams. - (Contributed by Serhiy Storchaka in :issue:`23252`.) +ZIP output can now be written to unseekable streams. +(Contributed by Serhiy Storchaka in :issue:`23252`.) -* The :func:`zipfile.ZipFile.open` function now supports ``'x'`` (exclusive - creation) mode. (Contributed by Serhiy Storchaka in :issue:`21717`.) +The *mode* argument of :func:`zipfile.ZipFile.open` function now +accepts ``'x'`` to request exclusive creation. +(Contributed by Serhiy Storchaka in :issue:`21717`.) Other module-level changes ========================== -* Many functions in modules :mod:`mmap`, :mod:`ossaudiodev`, :mod:`socket`, - :mod:`ssl`, and :mod:`codecs`, now accept writable bytes-like objects. - (Contributed by Serhiy Storchaka in :issue:`23001`.) +Many functions in :mod:`mmap`, :mod:`ossaudiodev`, :mod:`socket`, +:mod:`ssl`, and :mod:`codecs` modules now accept writable bytes-like objects. +(Contributed by Serhiy Storchaka in :issue:`23001`.) Optimizations @@ -1374,6 +1525,7 @@ * Windows builds now require Microsoft Visual C++ 14.0, which is available as part of `Visual Studio 2015 `_. + Deprecated ========== -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 11 00:05:02 2015 From: python-checkins at python.org (yury.selivanov) Date: Thu, 10 Sep 2015 22:05:02 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogTWVyZ2UgMy41?= Message-ID: <20150910220502.66862.80694@psf.io> https://hg.python.org/cpython/rev/6d18eb9a13ee changeset: 97877:6d18eb9a13ee parent: 97875:b81713d218a8 parent: 97876:63a44a5fa5f7 user: Yury Selivanov date: Thu Sep 10 18:04:45 2015 -0400 summary: Merge 3.5 files: Doc/whatsnew/3.5.rst | 733 ++++++++++++++++-------------- 1 files changed, 381 insertions(+), 352 deletions(-) diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -128,7 +128,7 @@ * :mod:`traceback` has new lightweight and convenient to work with classes :class:`~traceback.TracebackException`, - :class:`~traceback.StackSummary`, and :class:`traceback.FrameSummary`. + :class:`~traceback.StackSummary`, and :class:`~traceback.FrameSummary`. (Contributed by Robert Collins in :issue:`17911`.) * Most of :func:`functools.lru_cache` machinery is now implemented in C. @@ -570,7 +570,7 @@ Some smaller changes made to the core Python language are: -* Added the ``'namereplace'`` error handlers. The ``'backslashreplace'`` +* Added the ``"namereplace"`` error handlers. The ``"backslashreplace"`` error handlers now works with decoding and translating. (Contributed by Serhiy Storchaka in :issue:`19676` and :issue:`22286`.) @@ -625,7 +625,7 @@ argparse -------- -:class:`~argparse.ArgumentParser` now allows to disable +The :class:`~argparse.ArgumentParser` class now allows to disable :ref:`abbreviated usage ` of long options by setting :ref:`allow_abbrev` to ``False``. (Contributed by Jonathan Paugh, Steven Bethard, paul j3 and Daniel Eriksson in :issue:`14910`.) @@ -634,16 +634,16 @@ bz2 --- -:meth:`~bz2.BZ2Decompressor.decompress` now accepts an optional *max_length* -argument to limit the maximum size of decompressed data. (Contributed by -Nikolaus Rath in :issue:`15955`.) +The :meth:`BZ2Decompressor.decompress ` +method now accepts an optional *max_length* argument to limit the maximum +size of decompressed data. (Contributed by Nikolaus Rath in :issue:`15955`.) cgi --- -:class:`~cgi.FieldStorage` now supports the context management protocol. -(Contributed by Berker Peksag in :issue:`20289`.) +The :class:`~cgi.FieldStorage` class now supports the context management +protocol. (Contributed by Berker Peksag in :issue:`20289`.) cmath @@ -656,15 +656,24 @@ code ---- -The :func:`code.InteractiveInterpreter.showtraceback` method now prints -the full chained traceback, just like the interactive interpreter. -(Contributed by Claudiu Popa in :issue:`17442`.) +The :func:`InteractiveInterpreter.showtraceback ` +method now prints the full chained traceback, just like the interactive +interpreter. (Contributed by Claudiu Popa in :issue:`17442`.) collections ----------- -Docstrings produced by :func:`collections.namedtuple` can now be updated:: +The :class:`~collections.OrderedDict` class is now implemented in C, which +makes it 4 to 100 times faster. (Contributed by Eric Snow in :issue:`16991`.) + +The :class:`~collections.deque` class now defines +:meth:`~collections.deque.index`, :meth:`~collections.deque.insert`, and +:meth:`~collections.deque.copy`. This allows deques to be recognized as a +:class:`~collections.abc.MutableSequence` and improves their substitutability +for lists. (Contributed by Raymond Hettinger :issue:`23704`.) + +Docstrings produced by :func:`~collections.namedtuple` can now be updated:: Point = namedtuple('Point', ['x', 'y']) Point.__doc__ = 'ordered pair' @@ -673,28 +682,20 @@ (Contributed by Berker Peksag in :issue:`24064`.) -The :class:`~collections.deque` now defines -:meth:`~collections.deque.index`, :meth:`~collections.deque.insert`, and -:meth:`~collections.deque.copy`. This allows deques to be recognized as a -:class:`~collections.abc.MutableSequence` and improves their substitutablity -for lists. (Contributed by Raymond Hettinger :issue:`23704`.) - -:class:`~collections.UserString` now implements :meth:`__getnewargs__`, -:meth:`__rmod__`, :meth:`casefold`, :meth:`format_map`, :meth:`isprintable`, and -:meth:`maketrans` methods to match corresponding methods of :class:`str`. +The :class:`~collections.UserString` class now implements +:meth:`__getnewargs__`, :meth:`__rmod__`, :meth:`~str.casefold`, +:meth:`~str.format_map`, :meth:`~str.isprintable`, and :meth:`~str.maketrans` +methods to match corresponding methods of :class:`str`. (Contributed by Joe Jevnik in :issue:`22189`.) -:class:`collections.OrderedDict` is now implemented in C, which makes it 4x to -100x faster. (Contributed by Eric Snow in :issue:`16991`.) - collections.abc --------------- -New :class:`~collections.abc.Generator` abstract base class. (Contributed +A new :class:`~collections.abc.Generator` abstract base class. (Contributed by Stefan Behnel in :issue:`24018`.) -New :class:`~collections.abc.Coroutine`, +A new :class:`~collections.abc.Coroutine`, :class:`~collections.abc.AsyncIterator`, and :class:`~collections.abc.AsyncIterable` abstract base classes. (Contributed by Yury Selivanov in :issue:`24184`.) @@ -703,89 +704,93 @@ compileall ---------- -A new :mod:`compileall` option, :option:`-j N`, allows to run ``N`` workers -sumultaneously to perform parallel bytecode compilation. :func:`compileall.compile_dir` has a corresponding ``workers`` parameter. (Contributed by -Claudiu Popa in :issue:`16104`.) +A new :mod:`compileall` option, ``-j N``, allows to run ``N`` workers +sumultaneously to perform parallel bytecode compilation. +The :func:`~compileall.compile_dir` function has a corresponding ``workers`` +parameter. (Contributed by Claudiu Popa in :issue:`16104`.) -The :option:`-q` command line option can now be specified more than once, in +The ``-q`` command line option can now be specified more than once, in which case all output, including errors, will be suppressed. The corresponding -``quiet`` parameter in :func:`compileall.compile_dir`, :func:`compileall. -compile_file`, and :func:`compileall.compile_path` can now accept -an integer value indicating the level of output suppression. +``quiet`` parameter in :func:`~compileall.compile_dir`, +:func:`~compileall.compile_file`, and :func:`~compileall.compile_path` can now +accept an integer value indicating the level of output suppression. (Contributed by Thomas Kluyver in :issue:`21338`.) concurrent.futures ------------------ -:meth:`~concurrent.futures.Executor.map` now accepts a *chunksize* -argument to allow batching of tasks in child processes and improve performance -of ProcessPoolExecutor. (Contributed by Dan O'Reilly in :issue:`11271`.) +The :meth:`Executor.map ` method now accepts a +*chunksize* argument to allow batching of tasks in child processes and improve +performance of :meth:`~concurrent.futures.ProcessPoolExecutor`. +(Contributed by Dan O'Reilly in :issue:`11271`.) contextlib ---------- -The new :func:`contextlib.redirect_stderr` context manager (similar to -:func:`contextlib.redirect_stdout`) makes it easier for utility scripts to +The new :func:`~contextlib.redirect_stderr` context manager (similar to +:func:`~contextlib.redirect_stdout`) makes it easier for utility scripts to handle inflexible APIs that write their output to :data:`sys.stderr` and -don't provide any options to redirect it. (Contributed by Berker Peksag in +don't provide any options to redirect it. (Contributed by Berker Peksag in :issue:`22389`.) curses ------ -The new :func:`curses.update_lines_cols` function updates the variables -:envvar:`curses.LINES` and :envvar:`curses.COLS`. +The new :func:`~curses.update_lines_cols` function updates the variables +:data:`curses.LINES` and :data:`curses.COLS`. difflib ------- -The charset of the HTML document generated by :meth:`difflib.HtmlDiff.make_file` +The charset of the HTML document generated by +:meth:`HtmlDiff.make_file ` can now be customized by using *charset* keyword-only parameter. The default -charset of HTML document changed from ``'ISO-8859-1'`` to ``'utf-8'``. +charset of HTML document changed from ``"ISO-8859-1"`` to ``"utf-8"``. (Contributed by Berker Peksag in :issue:`2052`.) -It is now possible to compare lists of byte strings with -:func:`difflib.diff_bytes`. This fixes a regression from Python 2. +It is now possible to compare lists of byte strings with the +:func:`~difflib.diff_bytes` function. This fixes a regression from Python 2. (Contributed by Terry J. Reedy and Greg Ward in :issue:`17445`.) distutils --------- -The ``build`` and ``build_ext`` commands now accept a :option:`-j` option to +The ``build`` and ``build_ext`` commands now accept a ``-j`` option to enable parallel building of extension modules. (Contributed by Antoine Pitrou in :issue:`5309`.) -:mod:`distutils` now supports ``xz`` compression, and can be enabled by -passing ``xztar`` as an argument to ``bdist --format``. +The :mod:`distutils` module now supports ``xz`` compression, and can be +enabled by passing ``xztar`` as an argument to ``bdist --format``. (Contributed by Serhiy Storchaka in :issue:`16314`.) doctest ------- -:func:`doctest.DocTestSuite` returns an empty :class:`unittest.TestSuite` if -*module* contains no docstrings instead of raising :exc:`ValueError`. -(Contributed by Glenn Jones in :issue:`15916`.) +The :func:`~doctest.DocTestSuite` function returns an empty +:class:`unittest.TestSuite` if *module* contains no docstrings instead of +raising :exc:`ValueError`. (Contributed by Glenn Jones in :issue:`15916`.) email ----- -A new policy option :attr:`~email.policy.Policy.mangle_from_` controls -whether or not lines that start with ``"From "`` in email bodies are prefixed -with a ``'>'`` character by generators. The default is ``True`` for +A new policy option :attr:`Policy.mangle_from_ ` +controls whether or not lines that start with ``"From "`` in email bodies are +prefixed with a ``">"`` character by generators. The default is ``True`` for :attr:`~email.policy.compat32` and ``False`` for all other policies. (Contributed by Milan Oberkirch in :issue:`20098`.) -A new method :meth:`~email.message.Message.get_content_disposition` provides -easy access to a canonical value for the :mailheader:`Content-Disposition` -header (``None`` if there is no such header). (Contributed by Abhilash Raj -in :issue:`21083`.) +A new +:meth:`Message.get_content_disposition ` +method provides easy access to a canonical value for the +:mailheader:`Content-Disposition` header. +(Contributed by Abhilash Raj in :issue:`21083`.) A new policy option :attr:`~email.policy.EmailPolicy.utf8` can be set to ``True`` to encode email headers using the UTF-8 charset instead of using @@ -797,7 +802,7 @@ faulthandler ------------ -:func:`~faulthandler.enable`, :func:`~faulthandler.register`, +The :func:`~faulthandler.enable`, :func:`~faulthandler.register`, :func:`~faulthandler.dump_traceback` and :func:`~faulthandler.dump_traceback_later` functions now accept file descriptors in addition to file-like objects. @@ -815,9 +820,9 @@ glob ---- -:func:`~glob.iglob` and :func:`~glob.glob` now support recursive search in -subdirectories using the "``**``" pattern. (Contributed by Serhiy Storchaka -in :issue:`13968`.) +The :func:`~glob.iglob` and :func:`~glob.glob` functions now support recursive +search in subdirectories using the ``"**"`` pattern. +(Contributed by Serhiy Storchaka in :issue:`13968`.) heapq @@ -826,7 +831,7 @@ Element comparison in :func:`~heapq.merge` can now be customized by passing a :term:`key function` in a new optional ``key`` keyword argument. A new optional ``reverse`` keyword argument can be used to reverse element -comparison. (Contributed by Raymond Hettinger in :issue:`13742`.) +comparison. (Contributed by Raymond Hettinger in :issue:`13742`.) idlelib and IDLE @@ -842,95 +847,101 @@ imaplib ------- -:class:`~imaplib.IMAP4` now supports context manager protocol. +The :class:`~imaplib.IMAP4` class now supports context manager protocol. When used in a :keyword:`with` statement, the IMAP4 ``LOGOUT`` command will be called automatically at the end of the block. (Contributed by Tarek Ziad? and Serhiy Storchaka in :issue:`4972`.) -:mod:`imaplib` now supports :rfc:`5161` (``ENABLE`` extension) via -:meth:`~imaplib.IMAP4.enable`, and :rfc:`6855` (UTF-8 support) via the -``UTF8=ACCEPT`` argument to :meth:`~imaplib.IMAP4.enable`. A new attribute, -:attr:`~imaplib.IMAP4.utf8_enabled`, tracks whether or not :rfc:`6855` -support is enabled. (Contributed by Milan Oberkirch, R. David Murray, -and Maciej Szulik in :issue:`21800`.) +The :mod:`imaplib` module now supports :rfc:`5161` (ENABLE Extension) +and :rfc:`6855` (UTF-8 Support) via the :meth:`IMAP4.enable ` +method. A new :attr:`IMAP4.utf8_enabled ` +attribute, tracks whether or not :rfc:`6855` support is enabled. +(Contributed by Milan Oberkirch, R. David Murray, and Maciej Szulik in +:issue:`21800`.) -:mod:`imaplib` now automatically encodes non-ASCII string usernames and -passwords using UTF-8, as recommended by the RFCs. (Contributed by Milan +The :mod:`imaplib` module now automatically encodes non-ASCII string usernames +and passwords using UTF-8, as recommended by the RFCs. (Contributed by Milan Oberkirch in :issue:`21800`.) imghdr ------ -:func:`~imghdr.what` now recognizes the `OpenEXR `_ -format (contributed by Martin Vignali and Claudiu Popa in :issue:`20295`), -and the `WebP `_ format (contributed -by Fabrice Aneche and Claudiu Popa in :issue:`20197`.) +The :func:`~imghdr.what` function now recognizes the +`OpenEXR `_ format +(contributed by Martin Vignali and Claudiu Popa in :issue:`20295`), +and the `WebP `_ format +(contributed by Fabrice Aneche and Claudiu Popa in :issue:`20197`.) importlib --------- -:class:`importlib.util.LazyLoader` allows for lazy loading of modules in -applications where startup time is important. (Contributed by Brett Cannon +The :class:`importlib.util.LazyLoader` class allows for lazy loading of modules +in applications where startup time is important. (Contributed by Brett Cannon in :issue:`17621`.) -:func:`importlib.abc.InspectLoader.source_to_code` is now a +The :func:`importlib.abc.InspectLoader.source_to_code` method is now a static method. This makes it easier to initialize a module object with -code compiled from a string by runnning ``exec(code, module.__dict__)``. +code compiled from a string by running ``exec(code, module.__dict__)``. (Contributed by Brett Cannon in :issue:`21156`.) -:func:`importlib.util.module_from_spec` is now the preferred way to create a -new module. Compared to :class:`types.ModuleType`, this new function will set -the various import-controlled attributes based on the passed-in spec object. +The new :func:`importlib.util.module_from_spec` function is now the preferred +way to create a new module. Compared to the :class:`types.ModuleType` class, +this new function will set the various import-controlled attributes based +on the passed-in spec object. (Contributed by Brett Cannon in :issue:`20383`.) inspect ------- -:class:`inspect.Signature` and :class:`inspect.Parameter` are now +The :class:`~inspect.Signature` and :class:`~inspect.Parameter` classes are now picklable and hashable. (Contributed by Yury Selivanov in :issue:`20726` and :issue:`20334`.) -A new method :meth:`inspect.BoundArguments.apply_defaults` provides a way -to set default values for missing arguments. (Contributed by Yury Selivanov -in :issue:`24190`.) +A new +:meth:`BoundArguments.apply_defaults ` +method provides a way to set default values for missing arguments. +(Contributed by Yury Selivanov in :issue:`24190`.) -A new class method :meth:`inspect.Signature.from_callable` makes +A new class method +:meth:`Signature.from_callable ` makes subclassing of :class:`~inspect.Signature` easier. (Contributed by Yury Selivanov and Eric Snow in :issue:`17373`.) -:func:`inspect.signature` now accepts a ``follow_wrapped`` optional keyword -argument, which, when set to ``False``, disables automatic following of -``__wrapped__`` links. (Contributed by Yury Selivanov in :issue:`20691`.) +The :func:`~inspect.signature` function now accepts a ``follow_wrapped`` +optional keyword argument, which, when set to ``False``, disables automatic +following of ``__wrapped__`` links. +(Contributed by Yury Selivanov in :issue:`20691`.) A set of new functions to inspect :term:`coroutine functions ` and -``coroutine objects`` as been added: +``coroutine objects`` has been added: :func:`~inspect.iscoroutine`, :func:`~inspect.iscoroutinefunction`, :func:`~inspect.isawaitable`, :func:`~inspect.getcoroutinelocals`, and :func:`~inspect.getcoroutinestate`. (Contributed by Yury Selivanov in :issue:`24017` and :issue:`24400`.) -:func:`~inspect.stack`, :func:`~inspect.trace`, :func:`~inspect.getouterframes`, -and :func:`~inspect.getinnerframes` now return a list of named tuples. +The :func:`~inspect.stack`, :func:`~inspect.trace`, +:func:`~inspect.getouterframes`, and :func:`~inspect.getinnerframes` +functions now return a list of named tuples. (Contributed by Daniel Shahaf in :issue:`16808`.) io -- -:class:`io.FileIO` has been implemented in Python which makes C implementation -of :mod:`io` module entirely optional. (Contributed by Serhiy Storchaka -in :issue:`21859`.) +The :class:`~io.FileIO` class has been implemented in Python which makes +the C implementation of the :mod:`io` module entirely optional. +(Contributed by Serhiy Storchaka in :issue:`21859`.) ipaddress --------- -:class:`ipaddress.IPv4Network` and :class:`ipaddress.IPv6Network` now -accept an ``(address, netmask)`` tuple argument, so as to easily construct +The :class:`~ipaddress.IPv4Network` and :class:`~ipaddress.IPv6Network` classes +now accept an ``(address, netmask)`` tuple argument, so as to easily construct network objects from existing addresses. (Contributed by Peter Moody and Antoine Pitrou in :issue:`16531`.) @@ -938,8 +949,8 @@ json ---- -:mod:`json.tool` command line interface now preserves the order of keys in -JSON objects passed in input. The new :option:`--sort-keys` option can be used +The :mod:`json.tool` command line interface now preserves the order of keys in +JSON objects passed in input. The new ``--sort-keys`` option can be used to sort the keys alphabetically. (Contributed by Berker Peksag in :issue:`21650`.) @@ -958,36 +969,39 @@ logging ------- -All logging methods (:meth:`~logging.Logger.log`, +All logging methods (:class:`~logging.Logger` :meth:`~logging.Logger.log`, :meth:`~logging.Logger.exception`, :meth:`~logging.Logger.critical`, :meth:`~logging.Logger.debug`, etc.), now accept exception instances -in ``exc_info`` parameter, in addition to boolean values and exception +in ``exc_info`` argument, in addition to boolean values and exception tuples. (Contributed by Yury Selivanov in :issue:`20537`.) -:class:`~logging.handlers.HTTPHandler` now accepts an optional -:class:`ssl.SSLContext` instance to configure the SSL settings used -in an HTTP connection. (Contributed by Alex Gaynor in :issue:`22788`.) +The :class:`handlers.HTTPHandler ` classes now +accepts an optional :class:`ssl.SSLContext` instance to configure the SSL +settings used in an HTTP connection. +(Contributed by Alex Gaynor in :issue:`22788`.) -:class:`~logging.handlers.QueueListener` now takes a *respect_handler_level* -keyword argument which, if set to ``True``, will pass messages to handlers -taking handler levels into account. (Contributed by Vinay Sajip.) +The :class:`handlers.QueueListener ` class now +takes a *respect_handler_level* keyword argument which, if set to ``True``, +will pass messages to handlers taking handler levels into account. +(Contributed by Vinay Sajip.) lzma ---- -:meth:`~lzma.LZMADecompressor.decompress` now accepts an optional *max_length* -argument to limit the maximum size of decompressed data. +The :meth:`LZMADecompressor.decompress ` +method now accepts an optional *max_length* argument to limit the maximum +size of decompressed data. (Contributed by Martin Panter in :issue:`15955`.) math ---- -Two new constants have been added to :mod:`math`: :data:`math.inf` -and :data:`math.nan`. (Contributed by Mark Dickinson in :issue:`23185`.) +Two new constants have been added to the :mod:`math` module: :data:`~math.inf` +and :data:`~math.nan`. (Contributed by Mark Dickinson in :issue:`23185`.) -A new function :func:`math.isclose` provides a way to test for approximate +A new function :func:`~math.isclose` provides a way to test for approximate equality. (Contributed by Chris Barker and Tal Einat in :issue:`24270`.) A new :func:`~math.gcd` function has been added. The :func:`fractions.gcd` @@ -998,41 +1012,38 @@ operator -------- -:func:`~operator.attrgetter`, :func:`~operator.itemgetter`, and -:func:`~operator.methodcaller` objects now support pickling. +The :mod:`operator` :func:`~operator.attrgetter`, :func:`~operator.itemgetter`, +and :func:`~operator.methodcaller` objects now support pickling. (Contributed by Josh Rosenberg and Serhiy Storchaka in :issue:`22955`.) os -- -The new :func:`os.scandir` returning an iterator of :class:`os.DirEntry` -objects has been added. If possible, :func:`os.scandir` extracts file -attributes while scanning a directory, removing the need to perform -subsequent system calls to determine file type or attributes, which may +The new :func:`~os.scandir` function returning an iterator of +:class:`~os.DirEntry` objects has been added. If possible, :func:`~os.scandir` +extracts file attributes while scanning a directory, removing the need to +perform subsequent system calls to determine file type or attributes, which may significantly improve performance. (Contributed by Ben Hoyt with the help of Victor Stinner in :issue:`22524`.) -On Windows, a new :attr:`~os.stat_result.st_file_attributes` attribute is -now available. It corresponds to ``dwFileAttributes`` member of the -``BY_HANDLE_FILE_INFORMATION`` structure returned by +On Windows, a new +:attr:`stat_result.st_file_attributes ` +attribute is now available. It corresponds to ``dwFileAttributes`` member of +the ``BY_HANDLE_FILE_INFORMATION`` structure returned by ``GetFileInformationByHandle()``. (Contributed by Ben Hoyt in :issue:`21719`.) -:func:`os.urandom` now uses ``getrandom()`` syscall on Linux 3.17 or newer, -and ``getentropy()`` on OpenBSD 5.6 and newer, removing the need to use -``/dev/urandom`` and avoiding failures due to potential file descriptor +The :func:`~os.urandom` function now uses ``getrandom()`` syscall on Linux 3.17 +or newer, and ``getentropy()`` on OpenBSD 5.6 and newer, removing the need to +use ``/dev/urandom`` and avoiding failures due to potential file descriptor exhaustion. (Contributed by Victor Stinner in :issue:`22181`.) -New :func:`os.get_blocking` and :func:`os.set_blocking` functions allow to +New :func:`~os.get_blocking` and :func:`~os.set_blocking` functions allow to get and set the file descriptor blocking mode (:data:`~os.O_NONBLOCK`.) (Contributed by Victor Stinner in :issue:`22054`.) -:func:`~os.truncate` and :func:`~os.ftruncate` are now supported on -Windows. (Contributed by Steve Dower in :issue:`23668`.) - - -os.path -------- +The :func:`~os.truncate` and :func:`~os.ftruncate` functions are now supported +on Windows. (Contributed by Steve Dower in :issue:`23668`.) There is a new :func:`~os.path.commonpath` function returning the longest common sub-path of each passed pathname. Unlike the @@ -1043,28 +1054,30 @@ pathlib ------- -The new :meth:`~pathlib.Path.samefile` method can be used to check if the -passed :class:`~pathlib.Path` object, or a string, point to the same file as -the :class:`~pathlib.Path` on which :meth:`~pathlib.Path.samefile` is called. +The new :meth:`Path.samefile ` method can be used +to check if the passed :class:`~pathlib.Path` object or a :class:`str` path, +point to the same file. (Contributed by Vajrasky Kok and Antoine Pitrou in :issue:`19775`.) -:meth:`~pathlib.Path.mkdir` how accepts a new optional ``exist_ok`` argument -to match ``mkdir -p`` and :func:`os.makrdirs` functionality. -(Contributed by Berker Peksag in :issue:`21539`.) +The :meth:`Path.mkdir ` method how accepts a new optional +``exist_ok`` argument to match ``mkdir -p`` and :func:`os.makrdirs` +functionality. (Contributed by Berker Peksag in :issue:`21539`.) -There is a new :meth:`~pathlib.Path.expanduser` method to expand ``~`` -and ``~user`` prefixes. (Contributed by Serhiy Storchaka and Claudiu -Popa in :issue:`19776`.) +There is a new :meth:`Path.expanduser ` method to +expand ``~`` and ``~user`` prefixes. (Contributed by Serhiy Storchaka and +Claudiu Popa in :issue:`19776`.) -A new :meth:`~pathlib.Path.home` class method can be used to get an instance -of :class:`~pathlib.Path` object representing the user?s home directory. +A new :meth:`Path.home ` class method can be used to get +an instance of :class:`~pathlib.Path` object representing the user?s home +directory. (Contributed by Victor Salgado and Mayank Tripathi in :issue:`19777`.) pickle ------ -Nested objects, such as unbound methods or nested classes, can now be pickled using :ref:`pickle protocols ` older than protocol version 4, +Nested objects, such as unbound methods or nested classes, can now be pickled +using :ref:`pickle protocols ` older than protocol version 4, which already supported these cases. (Contributed by Serhiy Storchaka in :issue:`23611`.) @@ -1072,7 +1085,7 @@ poplib ------ -A new command :meth:`~poplib.POP3.utf8` enables :rfc:`6856` +A new command :meth:`POP3.utf8 ` enables :rfc:`6856` (internationalized email) support, if the POP server supports it. (Contributed by Milan OberKirch in :issue:`21804`.) @@ -1083,9 +1096,9 @@ The number of capturing groups in regular expression is no longer limited by 100. (Contributed by Serhiy Storchaka in :issue:`22437`.) -:func:`re.sub` and :func:`re.subn` now replace unmatched groups with empty -strings instead of rising an exception. (Contributed by Serhiy Storchaka -in :issue:`1519638`.) +The :func:`~re.sub` and :func:`~re.subn` functions now replace unmatched +groups with empty strings instead of rising an exception. +(Contributed by Serhiy Storchaka in :issue:`1519638`.) readline @@ -1099,20 +1112,21 @@ shutil ------ -:func:`~shutil.move` now accepts a *copy_function* argument, allowing, -for example, :func:`~shutil.copy` to be used instead of the default -:func:`~shutil.copy2` there is a need to ignore file metadata when moving. +The :func:`~shutil.move` function now accepts a *copy_function* argument, +allowing, for example, the :func:`~shutil.copy` function to be used instead of +the default :func:`~shutil.copy2` if there is a need to ignore file metadata +when moving. (Contributed by Claudiu Popa in :issue:`19840`.) -:func:`~shutil.make_archive` now supports *xztar* format. +The :func:`~shutil.make_archive` function now supports *xztar* format. (Contributed by Serhiy Storchaka in :issue:`5411`.) signal ------ -On Windows, :func:`signal.set_wakeup_fd` now also supports socket handles. -(Contributed by Victor Stinner in :issue:`22018`.) +On Windows, the :func:`~signal.set_wakeup_fd` function now also supports +socket handles. (Contributed by Victor Stinner in :issue:`22018`.) Various ``SIG*`` constants in :mod:`signal` module have been converted into :mod:`Enums `. This allows meaningful names to be printed @@ -1123,28 +1137,30 @@ smtpd ----- -Both :class:`~smtpd.SMTPServer` and :class:`smtpd.SMTPChannel` now accept a -*decode_data* keyword argument to determine if the ``DATA`` portion of the SMTP -transaction is decoded using the ``"utf-8"`` codec or is instead provided to -:meth:`~smtpd.SMTPServer.process_message` as a byte string. The default -is ``True`` for backward compatibility reasons, but will change to ``False`` -in Python 3.6. If *decode_data* is set to ``False``, the -:meth:`~smtpd.SMTPServer.process_message` method must be prepared to accept -keyword arguments. (Contributed by Maciej Szulik in :issue:`19662`.) +Both :class:`~smtpd.SMTPServer` and :class:`~smtpd.SMTPChannel` classes now +accept a *decode_data* keyword argument to determine if the ``DATA`` portion of +the SMTP transaction is decoded using the ``"utf-8"`` codec or is instead +provided to :meth:`SMTPServer.process_message ` +method as a byte string. The default is ``True`` for backward compatibility +reasons, but will change to ``False`` in Python 3.6. If *decode_data* is set +to ``False``, the :meth:`~smtpd.SMTPServer.process_message` method must +be prepared to accept keyword arguments. +(Contributed by Maciej Szulik in :issue:`19662`.) -:class:`~smtpd.SMTPServer` now advertises the ``8BITMIME`` extension -(:rfc:`6152`) if if *decode_data* has been set ``True``. If the client +The :class:`~smtpd.SMTPServer` class now advertises the ``8BITMIME`` extension +(:rfc:`6152`) if *decode_data* has been set ``True``. If the client specifies ``BODY=8BITMIME`` on the ``MAIL`` command, it is passed to -:meth:`~smtpd.SMTPServer.process_message` via the ``mail_options`` keyword. +:meth:`SMTPServer.process_message ` +via the ``mail_options`` keyword. (Contributed by Milan Oberkirch and R. David Murray in :issue:`21795`.) -:class:`~smtpd.SMTPServer` now supports the ``SMTPUTF8`` extension -(:rfc:`6531`: Internationalized Email). If the client specified ``SMTPUTF8 -BODY=8BITMIME`` on the ``MAIL`` command, they are passed to -:meth:`~smtpd.SMTPServer.process_message` via the ``mail_options`` keyword. -It is the responsibility of the :meth:`~smtpd.SMTPServer.process_message` -method to correctly handle the ``SMTPUTF8`` data. (Contributed by Milan -Oberkirch in :issue:`21725`.) +The :class:`~smtpd.SMTPServer` class now also supports the ``SMTPUTF8`` +extension (:rfc:`6531`: Internationalized Email). If the client specified +``SMTPUTF8 BODY=8BITMIME`` on the ``MAIL`` command, they are passed to +:meth:`SMTPServer.process_message ` +via the ``mail_options`` keyword. It is the responsibility of the +:meth:`~smtpd.SMTPServer.process_message` method to correctly handle the +``SMTPUTF8`` data. (Contributed by Milan Oberkirch in :issue:`21725`.) It is now possible to provide, directly or via name resolution, IPv6 addresses in the :class:`~smtpd.SMTPServer` constructor, and have it @@ -1154,7 +1170,7 @@ smtplib ------- -A new :meth:`~smtplib.SMTP.auth` method provides a convenient way to +A new :meth:`SMTP.auth ` method provides a convenient way to implement custom authentication mechanisms. (Contributed by Milan Oberkirch in :issue:`15014`.) @@ -1162,17 +1178,17 @@ :class:`smtplib.SMTP`. (Contributed by Gavin Chappell and Maciej Szulik in :issue:`16914`.) -:mod:`smtplib` now supports :rfc:`6531` (SMTPUTF8) in both the -:meth:`~smtplib.SMTP.sendmail` and :meth:`~smtplib.SMTP.send_message` -commands. (Contributed by Milan Oberkirch and R. David Murray in -:issue:`22027`.) +Both :meth:`SMTP.sendmail ` and +:meth:`SMTP.send_message ` methods now +support support :rfc:`6531` (SMTPUTF8). +(Contributed by Milan Oberkirch and R. David Murray in :issue:`22027`.) sndhdr ------ -:func:`~sndhdr.what` and :func:`~sndhdr.whathdr` now return -:func:`~collections.namedtuple`. (Contributed by Claudiu Popa in +The :func:`~sndhdr.what` and :func:`~sndhdr.whathdr` functions now return +a :func:`~collections.namedtuple`. (Contributed by Claudiu Popa in :issue:`18615`.) @@ -1186,17 +1202,17 @@ The new :class:`~ssl.SSLObject` class has been added to provide SSL protocol support for cases when the network IO capabilities of :class:`~ssl.SSLSocket` -are not necessary or inappropriate. :class:`~ssl.SSLObject` represents +are not necessary or suboptimal. :class:`~ssl.SSLObject` represents an SSL protocol instance, but does not implement any network IO methods, and instead provides a memory buffer interface. The new :class:`~ssl.MemoryBIO` class can be used to pass data between Python and an SSL protocol instance. The memory BIO SSL support is primarily intended to be used in frameworks implementing asynchronous IO for which :class:`~ssl.SSLObject` IO readiness -model ("select/poll") is inappropriate or inefficient. +model ("select/poll") is inefficient. -A new :meth:`~ssl.SSLContext.wrap_bio` method can be used to create a new -:class:`~ssl.SSLObject` instance. +A new :meth:`SSLContext.wrap_bio ` method can be used +to create a new :class:`~ssl.SSLObject` instance. Application-Layer Protocol Negotiation Support @@ -1207,41 +1223,49 @@ Where OpenSSL support is present, :mod:`ssl` module now implements * Application-Layer Protocol Negotiation* TLS extension as described in :rfc:`7301`. + The new :meth:`SSLContext.set_alpn_protocols ` can be used to specify which protocols the socket should advertise during -the TLS handshake. The new +the TLS handshake. + +The new :meth:`SSLSocket.selected_alpn_protocol ` -returns the protocol that was selected during the TLS handshake. :data:`ssl.HAS_ALPN` flag indicates whether APLN support is present. +returns the protocol that was selected during the TLS handshake. +:data:`~ssl.HAS_ALPN` flag indicates whether APLN support is present. Other Changes ~~~~~~~~~~~~~ -There is a new :meth:`~ssl.SSLSocket.version` method to query the actual -protocol version in use. (Contributed by Antoine Pitrou in :issue:`20421`.) +There is a new :meth:`SSLSocket.version ` method to query +the actual protocol version in use. +(Contributed by Antoine Pitrou in :issue:`20421`.) -:class:`~ssl.SSLSocket` now implementes :meth:`~ssl.SSLSocket.sendfile` -method. (Contributed by Giampaolo Rodola' in :issue:`17552`.) +The :class:`~ssl.SSLSocket` class now implements +a :meth:`SSLSocket.sendfile ` method. +(Contributed by Giampaolo Rodola in :issue:`17552`.) -:meth:`ssl.SSLSocket.send()` now raises either :exc:`ssl.SSLWantReadError` -or :exc:`ssl.SSLWantWriteError` on a non-blocking socket if the operation -would block. Previously, it would return 0. (Contributed by Nikolaus Rath -in :issue:`20951`.) +The :meth:`SSLSocket.send ` method now raises either +:exc:`ssl.SSLWantReadError` or :exc:`ssl.SSLWantWriteError` exception on a +non-blocking socket if the operation would block. Previously, it would return +``0``. (Contributed by Nikolaus Rath in :issue:`20951`.) The :func:`~ssl.cert_time_to_seconds` function now interprets the input time as UTC and not as local time, per :rfc:`5280`. Additionally, the return value is always an :class:`int`. (Contributed by Akira Li in :issue:`19940`.) -New :meth:`~ssl.SSLObject.shared_ciphers` and -:meth:`~ssl.SSLSocket.shared_ciphers` methods return the list of ciphers -sent by the client during the handshake. (Contributed by Benjamin Peterson -in :issue:`23186`.) +New :meth:`SSLObject.shared_ciphers ` and +:meth:`SSLSocket.shared_ciphers ` methods return +the list of ciphers sent by the client during the handshake. +(Contributed by Benjamin Peterson in :issue:`23186`.) -The :meth:`~ssl.SSLSocket.do_handshake`, :meth:`~ssl.SSLSocket.read`, -:meth:`~ssl.SSLSocket.shutdown`, and :meth:`~ssl.SSLSocket.write` methods of -:class:`ssl.SSLSocket` no longer reset the socket timeout every time bytes -are received or sent. The socket timeout is now the maximum total duration of -the method. (Contributed by Victor Stinner in :issue:`23853`.) +The :meth:`SSLSocket.do_handshake `, +:meth:`SSLSocket.read `, +:meth:`SSLSocket.shutdown `, and +:meth:`SSLSocket.write ` methods of :class:`ssl.SSLSocket` +class no longer reset the socket timeout every time bytes are received or sent. +The socket timeout is now the maximum total duration of the method. +(Contributed by Victor Stinner in :issue:`23853`.) The :func:`~ssl.match_hostname` function now supports matching of IP addresses. (Contributed by Antoine Pitrou in :issue:`23239`.) @@ -1250,28 +1274,28 @@ socket ------ -A new :meth:`socket.socket.sendfile` method allows to send a file over a -socket by using high-performance :func:`os.sendfile` function on UNIX -resulting in uploads being from 2x to 3x faster than when using plain -:meth:`socket.socket.send`. (Contributed by Giampaolo Rodola' in -:issue:`17552`.) - -The :meth:`socket.socket.sendall` method no longer resets the socket timeout -every time bytes are received or sent. The socket timeout is now the -maximum total duration to send all data. (Contributed by Victor Stinner in -:issue:`23853`.) - Functions with timeouts now use a monotonic clock, instead of a system clock. (Contributed by Victor Stinner in :issue:`22043`.) +A new :meth:`socket.sendfile ` method allows to +send a file over a socket by using high-performance :func:`os.sendfile` +function on UNIX resulting in uploads being from 2 to 3 times faster than when +using plain :meth:`socket.send `. +(Contributed by Giampaolo Rodola' in :issue:`17552`.) + +The :meth:`socket.sendall ` method no longer resets the +socket timeout every time bytes are received or sent. The socket timeout is +now the maximum total duration to send all data. +(Contributed by Victor Stinner in :issue:`23853`.) + subprocess ---------- -The new :func:`subprocess.run` function has been added and is the recommended +The new :func:`~subprocess.run` function has been added and is the recommended approach to invoking subprocesses. It runs the specified command and -and returns a :class:`subprocess.CompletedProcess` object. (Contributed by -Thomas Kluyver in :issue:`23342`.) +and returns a :class:`~subprocess.CompletedProcess` object. +(Contributed by Thomas Kluyver in :issue:`23342`.) sys @@ -1300,35 +1324,38 @@ tarfile ------- -The *mode* argument of :func:`tarfile.open` function now accepts ``'x'`` to request exclusive creation. (Contributed by Berker Peksag in :issue:`21717`.) +The *mode* argument of the :func:`~tarfile.open` function now accepts ``"x"`` +to request exclusive creation. (Contributed by Berker Peksag in :issue:`21717`.) -The :meth:`~tarfile.TarFile.extractall` and :meth:`~tarfile.TarFile.extract` -methods now take a keyword argument *numeric_only*. If set to ``True``, -the extracted files and directories will be owned by the numeric uid and gid -from the tarfile. If set to ``False`` (the default, and the behavior in -versions prior to 3.5), they will be owned by the named user and group in the -tarfile. (Contributed by Michael Vogt and Eric Smith in :issue:`23193`.) +The :meth:`TarFile.extractall ` and +:meth:`TarFile.extract ` methods now take a keyword +argument *numeric_only*. If set to ``True``, the extracted files and +directories will be owned by the numeric ``uid`` and ``gid`` from the tarfile. +If set to ``False`` (the default, and the behavior in versions prior to 3.5), +they will be owned by the named user and group in the tarfile. +(Contributed by Michael Vogt and Eric Smith in :issue:`23193`.) threading --------- -:meth:`~threading.Lock.acquire` and :meth:`~threading.RLock.acquire` -now use a monotonic clock for timeout management. (Contributed by Victor -Stinner in :issue:`22043`.) +The :meth:`Lock.acquire ` and +:meth:`RLock.acquire ` methods +now use a monotonic clock for timeout management. +(Contributed by Victor Stinner in :issue:`22043`.) time ---- -The :func:`time.monotonic` function is now always available. (Contributed by -Victor Stinner in :issue:`22043`.) +The :func:`~time.monotonic` function is now always available. +(Contributed by Victor Stinner in :issue:`22043`.) timeit ------ -New command line option :option:`-u` or :option:`--unit=U` to specify a time +New command line option ``-u`` or ``--unit=U`` to specify a time unit for the timer output. Supported options are ``usec``, ``msec``, or ``sec``. (Contributed by Julian Gindi in :issue:`18983`.) @@ -1353,8 +1380,8 @@ :class:`~traceback.StackSummary`, and :class:`traceback.FrameSummary`. (Contributed by Robert Collins in :issue:`17911`.) -:func:`~traceback.print_tb` and :func:`~traceback.print_stack` now support -negative values for the *limit* argument. +The :func:`~traceback.print_tb` and :func:`~traceback.print_stack` functions +now support negative values for the *limit* argument. (Contributed by Dmitry Kazakov in :issue:`22619`.) @@ -1371,21 +1398,23 @@ urllib ------ -A new :class:`~urllib.request.HTTPPasswordMgrWithPriorAuth` allows HTTP Basic -Authentication credentials to be managed so as to eliminate unnecessary -``401`` response handling, or to unconditionally send credentials -on the first request in order to communicate with servers that return a -``404`` response instead of a ``401`` if the ``Authorization`` header is not -sent. (Contributed by Matej Cepl in :issue:`19494` and Akshit Khurana in +A new +:class:`request.HTTPPasswordMgrWithPriorAuth ` +class allows HTTP Basic Authentication credentials to be managed so as to +eliminate unnecessary ``401`` response handling, or to unconditionally send +credentials on the first request in order to communicate with servers that +return a ``404`` response instead of a ``401`` if the ``Authorization`` header +is not sent. (Contributed by Matej Cepl in :issue:`19494` and Akshit Khurana in :issue:`7159`.) -A new :func:`~urllib.parse.urlencode` parameter *quote_via* provides a way to -control the encoding of query parts if needed. (Contributed by Samwyse and -Arnon Yaari in :issue:`13866`.) +A new *quote_via* argument for the +:func:`parse.urlencode ` +function provides a way to control the encoding of query parts if needed. +(Contributed by Samwyse and Arnon Yaari in :issue:`13866`.) -:func:`~urllib.request.urlopen` accepts an :class:`ssl.SSLContext` -object as a *context* argument, which will be used for the HTTPS -connection. (Contributed by Alex Gaynor in :issue:`22366`.) +The :func:`request.urlopen ` function accepts an +:class:`ssl.SSLContext` object as a *context* argument, which will be used for +the HTTPS connection. (Contributed by Alex Gaynor in :issue:`22366`.) unicodedata @@ -1398,33 +1427,35 @@ unittest -------- -New command line option :option:`--locals` to show local variables in +New command line option ``--locals`` to show local variables in tracebacks. (Contributed by Robert Collins in :issue:`22936`.) wsgiref ------- -*headers* parameter of :class:`wsgiref.headers.Headers` is now optional. +The *headers* argument of the :class:`headers.Headers ` +class constructor is now optional. (Contributed by Pablo Torres Navarrete and SilentGhost in :issue:`5800`.) xmlrpc ------ -:class:`xmlrpc.client.ServerProxy` is now a :term:`context manager`. +The :class:`client.ServerProxy ` class is now a +:term:`context manager`. (Contributed by Claudiu Popa in :issue:`20627`.) -:class:`~xmlrpc.client.ServerProxy` constructor now accepts an optional -:class:`ssl.SSLContext` instance. +:class:`client.ServerProxy ` constructor now accepts +an optional :class:`ssl.SSLContext` instance. (Contributed by Alex Gaynor in :issue:`22960`.) xml.sax ------- -SAX parsers now support a character stream of -:class:`~xml.sax.xmlreader.InputSource` object. +SAX parsers now support a character stream of the +:class:`xmlreader.InputSource ` object. (Contributed by Serhiy Storchaka in :issue:`2175`.) @@ -1434,8 +1465,8 @@ ZIP output can now be written to unseekable streams. (Contributed by Serhiy Storchaka in :issue:`23252`.) -The *mode* argument of :func:`zipfile.ZipFile.open` function now -accepts ``'x'`` to request exclusive creation. +The *mode* argument of :meth:`ZipFile.open ` method now +accepts ``"x"`` to request exclusive creation. (Contributed by Serhiy Storchaka in :issue:`21717`.) @@ -1450,80 +1481,76 @@ Optimizations ============= -The following performance enhancements have been added: +The :func:`os.walk` function has been sped up by 3-5 times on POSIX systems, +and by 7-20 times on Windows. This was done using the new :func:`os.scandir` +function, which exposes file information from the underlying ``readdir`` or +``FindFirstFile``/``FindNextFile`` system calls. (Contributed by +Ben Hoyt with help from Victor Stinner in :issue:`23605`.) -* :func:`os.walk` has been sped up by 3-5x on POSIX systems and 7-20x - on Windows. This was done using the new :func:`os.scandir` function, - which exposes file information from the underlying ``readdir`` and - ``FindFirstFile``/``FindNextFile`` system calls. (Contributed by - Ben Hoyt with help from Victor Stinner in :issue:`23605`.) +Construction of ``bytes(int)`` (filled by zero bytes) is faster and uses less +memory for large objects. ``calloc()`` is used instead of ``malloc()`` to +allocate memory for these objects. -* Construction of ``bytes(int)`` (filled by zero bytes) is faster and uses less - memory for large objects. ``calloc()`` is used instead of ``malloc()`` to - allocate memory for these objects. +Some operations on :mod:`ipaddress` :class:`~ipaddress.IPv4Network` and +:class:`~ipaddress.IPv6Network` have been massively sped up, such as +:meth:`~ipaddress.IPv4Network.subnets`, :meth:`~ipaddress.IPv4Network.supernet`, +:func:`~ipaddress.summarize_address_range`, :func:`~ipaddress.collapse_addresses`. +The speed up can range from 3 to 15 times. +(See :issue:`21486`, :issue:`21487`, :issue:`20826`, :issue:`23266`.) -* Some operations on :class:`~ipaddress.IPv4Network` and - :class:`~ipaddress.IPv6Network` have been massively sped up, such as - :meth:`~ipaddress.IPv4Network.subnets`, :meth:`~ipaddress.IPv4Network.supernet`, - :func:`~ipaddress.summarize_address_range`, :func:`~ipaddress.collapse_addresses`. - The speed up can range from 3x to 15x. - (See :issue:`21486`, :issue:`21487`, :issue:`20826`, :issue:`23266`.) +Pickling of :mod:`ipaddress` objects was optimized to produce significantly +smaller output. (Contributed by Serhiy Storchaka in :issue:`23133`.) -* Pickling of :mod:`ipaddress` classes was optimized to produce significantly - smaller output. (Contributed by Serhiy Storchaka in :issue:`23133`.) +Many operations on :class:`io.BytesIO` are now 50% to 100% faster. +(Contributed by Serhiy Storchaka in :issue:`15381` and David Wilson in +:issue:`22003`.) -* Many operations on :class:`io.BytesIO` are now 50% to 100% faster. - (Contributed by Serhiy Storchaka in :issue:`15381` and David Wilson in - :issue:`22003`.) +The :func:`marshal.dumps` function is now faster: 65-85% with versions 3 +and 4, 20-25% with versions 0 to 2 on typical data, and up to 5 times in +best cases. +(Contributed by Serhiy Storchaka in :issue:`20416` and :issue:`23344`.) -* :func:`marshal.dumps` is now faster (65%-85% with versions 3--4, 20-25% with - versions 0--2 on typical data, and up to 5x in best cases). - (Contributed by Serhiy Storchaka in :issue:`20416` and :issue:`23344`.) +The UTF-32 encoder is now 3 to 7 times faster. +(Contributed by Serhiy Storchaka in :issue:`15027`.) -* The UTF-32 encoder is now 3x to 7x faster. (Contributed by Serhiy Storchaka - in :issue:`15027`.) +Regular expressions are now parsed up to 10% faster. +(Contributed by Serhiy Storchaka in :issue:`19380`.) -* Regular expressions are now parsed up to 10% faster. - (Contributed by Serhiy Storchaka in :issue:`19380`.) +The :func:`json.dumps` function was optimized to run with +``ensure_ascii=False`` as fast as with ``ensure_ascii=True``. +(Contributed by Naoki Inada in :issue:`23206`.) -* :func:`json.dumps` was optimized to run with ``ensure_ascii=False`` - as fast as with ``ensure_ascii=True``. - (Contributed by Naoki Inada in :issue:`23206`.) +The :c:func:`PyObject_IsInstance` and :c:func:`PyObject_IsSubclass` +functions have been sped up in the common case that the second argument +has :class:`type` as its metaclass. +(Contributed Georg Brandl by in :issue:`22540`.) -* :c:func:`PyObject_IsInstance` and :c:func:`PyObject_IsSubclass` have - been sped up in the common case that the second argument has metaclass - :class:`type`. - (Contributed Georg Brandl by in :issue:`22540`.) +Method caching was slightly improved, yielding up to 5% performance +improvement in some benchmarks. +(Contributed by Antoine Pitrou in :issue:`22847`.) -* Method caching was slightly improved, yielding up to 5% performance - improvement in some benchmarks. - (Contributed by Antoine Pitrou in :issue:`22847`.) +Objects from :mod:`random` module now use two times less memory on 64-bit +builds. (Contributed by Serhiy Storchaka in :issue:`23488`.) -* Objects from :mod:`random` module now use 2x less memory on 64-bit - builds. - (Contributed by Serhiy Storchaka in :issue:`23488`.) +The :func:`property` getter calls are up to 25% faster. +(Contributed by Joe Jevnik in :issue:`23910`.) -* property() getter calls are up to 25% faster. - (Contributed by Joe Jevnik in :issue:`23910`.) - -* Instantiation of :class:`fractions.Fraction` is now up to 30% faster. - (Contributed by Stefan Behnel in :issue:`22464`.) +Instantiation of :class:`fractions.Fraction` is now up to 30% faster. +(Contributed by Stefan Behnel in :issue:`22464`.) Build and C API Changes ======================= -Changes to Python's build process and to the C API include: - -* New ``calloc`` functions: +New ``calloc`` functions: * :c:func:`PyMem_RawCalloc` * :c:func:`PyMem_Calloc` * :c:func:`PyObject_Calloc` * :c:func:`_PyObject_GC_Calloc` -* Windows builds now require Microsoft Visual C++ 14.0, which - is available as part of `Visual Studio 2015 `_. +Windows builds now require Microsoft Visual C++ 14.0, which +is available as part of `Visual Studio 2015 `_. Deprecated @@ -1540,63 +1567,65 @@ Unsupported Operating Systems ----------------------------- -* Windows XP - Per :PEP:`11`, Microsoft support of Windows XP has ended. +Per :PEP:`11`, Microsoft support of Windows XP has ended. Deprecated Python modules, functions and methods ------------------------------------------------ -* The :mod:`formatter` module has now graduated to full deprecation and is still - slated for removal in Python 3.6. +The :mod:`formatter` module has now graduated to full deprecation and is still +slated for removal in Python 3.6. -* :func:`~asyncio.async` was deprecated in favour of - :func:`~asyncio.ensure_future`. +The :func:`asyncio.async` function is deprecated in favor of +:func:`~asyncio.ensure_future`. -* :mod:`smtpd` has in the past always decoded the DATA portion of email - messages using the ``utf-8`` codec. This can now be controlled by the new - *decode_data* keyword to :class:`~smtpd.SMTPServer`. The default value is - ``True``, but this default is deprecated. Specify the *decode_data* keyword - with an appropriate value to avoid the deprecation warning. +The :mod:`smtpd` module has in the past always decoded the DATA portion of +email messages using the ``utf-8`` codec. This can now be controlled by the +new *decode_data* keyword to :class:`~smtpd.SMTPServer`. The default value is +``True``, but this default is deprecated. Specify the *decode_data* keyword +with an appropriate value to avoid the deprecation warning. -* Directly assigning values to the :attr:`~http.cookies.Morsel.key`, - :attr:`~http.cookies.Morsel.value` and - :attr:`~http.cookies.Morsel.coded_value` of :class:`~http.cookies.Morsel` - objects is deprecated. Use the :func:`~http.cookies.Morsel.set` method - instead. In addition, the undocumented *LegalChars* parameter of - :func:`~http.cookies.Morsel.set` is deprecated, and is now ignored. +Directly assigning values to the :attr:`~http.cookies.Morsel.key`, +:attr:`~http.cookies.Morsel.value` and +:attr:`~http.cookies.Morsel.coded_value` of :class:`~http.cookies.Morsel` +objects is deprecated. Use the :func:`~http.cookies.Morsel.set` method +instead. In addition, the undocumented *LegalChars* parameter of +:func:`~http.cookies.Morsel.set` is deprecated, and is now ignored. -* Passing a format string as keyword argument *format_string* to the - :meth:`~string.Formatter.format` method of the :class:`string.Formatter` - class has been deprecated. +Passing a format string as keyword argument *format_string* to the +:meth:`~string.Formatter.format` method of the :class:`string.Formatter` +class has been deprecated. -* :func:`platform.dist` and :func:`platform.linux_distribution` functions are - now deprecated and will be removed in Python 3.7. Linux distributions use - too many different ways of describing themselves, so the functionality is - left to a package. - (Contributed by Vajrasky Kok and Berker Peksag in :issue:`1322`.) +The :func:`platform.dist` and :func:`platform.linux_distribution` functions +are now deprecated and will be removed in Python 3.7. Linux distributions use +too many different ways of describing themselves, so the functionality is +left to a package. +(Contributed by Vajrasky Kok and Berker Peksag in :issue:`1322`.) -* The previously undocumented ``from_function`` and ``from_builtin`` methods of - :class:`inspect.Signature` are deprecated. Use new - :meth:`inspect.Signature.from_callable` instead. (Contributed by Yury - Selivanov in :issue:`24248`.) +The previously undocumented ``from_function`` and ``from_builtin`` methods of +:class:`inspect.Signature` are deprecated. Use new +:meth:`inspect.Signature.from_callable` instead. (Contributed by Yury +Selivanov in :issue:`24248`.) -* :func:`inspect.getargspec` is deprecated and scheduled to be removed in - Python 3.6. (See :issue:`20438` for details.) +The :func:`inspect.getargspec` function is deprecated and scheduled to be +removed in Python 3.6. (See :issue:`20438` for details.) -* :func:`~inspect.getfullargspec`, :func:`~inspect.getargvalues`, - :func:`~inspect.getcallargs`, :func:`~inspect.getargvalues`, - :func:`~inspect.formatargspec`, and :func:`~inspect.formatargvalues` are - deprecated in favor of :func:`inspect.signature` API. (See :issue:`20438` - for details.) +The :mod:`inspect` :func:`~inspect.getfullargspec`, +:func:`~inspect.getargvalues`, :func:`~inspect.getcallargs`, +:func:`~inspect.getargvalues`, :func:`~inspect.formatargspec`, and +:func:`~inspect.formatargvalues` functions are deprecated in favor of +:func:`inspect.signature` API. +(Contributed by Yury Selivanov in :issue:`20438`.) -* Use of ``re.LOCALE`` flag with str patterns or ``re.ASCII`` is now - deprecated. (Contributed by Serhiy Storchaka in :issue:`22407`.) +Use of ``re.LOCALE`` flag with str patterns or ``re.ASCII`` is now +deprecated. (Contributed by Serhiy Storchaka in :issue:`22407`.) +.. XXX: -Deprecated functions and types of the C API -------------------------------------------- + Deprecated functions and types of the C API + ------------------------------------------- -* None yet. + * None yet. Removed @@ -1640,9 +1669,10 @@ error-prone and has been removed in Python 3.5. See :issue:`13936` for full details. -* :meth:`ssl.SSLSocket.send()` now raises either :exc:`ssl.SSLWantReadError` - or :exc:`ssl.SSLWantWriteError` on a non-blocking socket if the operation - would block. Previously, it would return 0. See :issue:`20951`. +* The :meth:`ssl.SSLSocket.send()` method now raises either + :exc:`ssl.SSLWantReadError` or :exc:`ssl.SSLWantWriteError` + on a non-blocking socket if the operation would block. Previously, + it would return ``0``. See :issue:`20951`. * The ``__name__`` attribute of generator is now set from the function name, instead of being set from the code name. Use ``gen.gi_code.co_name`` to @@ -1681,12 +1711,12 @@ simply define :meth:`~importlib.machinery.Loader.create_module` to return ``None`` (:issue:`23014`). -* :func:`re.split` always ignored empty pattern matches, so the ``'x*'`` - pattern worked the same as ``'x+'``, and the ``'\b'`` pattern never worked. - Now :func:`re.split` raises a warning if the pattern could match +* The :func:`re.split` function always ignored empty pattern matches, so the + ``"x*"`` pattern worked the same as ``"x+"``, and the ``"\b"`` pattern never + worked. Now :func:`re.split` raises a warning if the pattern could match an empty string. For compatibility use patterns that never match an empty - string (e.g. ``'x+'`` instead of ``'x*'``). Patterns that could only match - an empty string (such as ``'\b'``) now raise an error. + string (e.g. ``"x+"`` instead of ``"x*"``). Patterns that could only match + an empty string (such as ``"\b"``) now raise an error. * The :class:`~http.cookies.Morsel` dict-like interface has been made self consistent: morsel comparison now takes the :attr:`~http.cookies.Morsel.key` @@ -1736,7 +1766,6 @@ * The undocumented :c:member:`~PyMemoryViewObject.format` member of the (non-public) :c:type:`PyMemoryViewObject` structure has been removed. - All extensions relying on the relevant parts in ``memoryobject.h`` must be rebuilt. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 11 00:05:02 2015 From: python-checkins at python.org (yury.selivanov) Date: Thu, 10 Sep 2015 22:05:02 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogd2hhdHNuZXcvMy41?= =?utf-8?q?=3A_More_edits_--_use_articles_consistently=3B_fix_refs?= Message-ID: <20150910220501.12006.16299@psf.io> https://hg.python.org/cpython/rev/63a44a5fa5f7 changeset: 97876:63a44a5fa5f7 branch: 3.5 parent: 97874:3265f33df731 user: Yury Selivanov date: Thu Sep 10 18:04:35 2015 -0400 summary: whatsnew/3.5: More edits -- use articles consistently; fix refs files: Doc/whatsnew/3.5.rst | 733 ++++++++++++++++-------------- 1 files changed, 381 insertions(+), 352 deletions(-) diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -128,7 +128,7 @@ * :mod:`traceback` has new lightweight and convenient to work with classes :class:`~traceback.TracebackException`, - :class:`~traceback.StackSummary`, and :class:`traceback.FrameSummary`. + :class:`~traceback.StackSummary`, and :class:`~traceback.FrameSummary`. (Contributed by Robert Collins in :issue:`17911`.) * Most of :func:`functools.lru_cache` machinery is now implemented in C. @@ -570,7 +570,7 @@ Some smaller changes made to the core Python language are: -* Added the ``'namereplace'`` error handlers. The ``'backslashreplace'`` +* Added the ``"namereplace"`` error handlers. The ``"backslashreplace"`` error handlers now works with decoding and translating. (Contributed by Serhiy Storchaka in :issue:`19676` and :issue:`22286`.) @@ -625,7 +625,7 @@ argparse -------- -:class:`~argparse.ArgumentParser` now allows to disable +The :class:`~argparse.ArgumentParser` class now allows to disable :ref:`abbreviated usage ` of long options by setting :ref:`allow_abbrev` to ``False``. (Contributed by Jonathan Paugh, Steven Bethard, paul j3 and Daniel Eriksson in :issue:`14910`.) @@ -634,16 +634,16 @@ bz2 --- -:meth:`~bz2.BZ2Decompressor.decompress` now accepts an optional *max_length* -argument to limit the maximum size of decompressed data. (Contributed by -Nikolaus Rath in :issue:`15955`.) +The :meth:`BZ2Decompressor.decompress ` +method now accepts an optional *max_length* argument to limit the maximum +size of decompressed data. (Contributed by Nikolaus Rath in :issue:`15955`.) cgi --- -:class:`~cgi.FieldStorage` now supports the context management protocol. -(Contributed by Berker Peksag in :issue:`20289`.) +The :class:`~cgi.FieldStorage` class now supports the context management +protocol. (Contributed by Berker Peksag in :issue:`20289`.) cmath @@ -656,15 +656,24 @@ code ---- -The :func:`code.InteractiveInterpreter.showtraceback` method now prints -the full chained traceback, just like the interactive interpreter. -(Contributed by Claudiu Popa in :issue:`17442`.) +The :func:`InteractiveInterpreter.showtraceback ` +method now prints the full chained traceback, just like the interactive +interpreter. (Contributed by Claudiu Popa in :issue:`17442`.) collections ----------- -Docstrings produced by :func:`collections.namedtuple` can now be updated:: +The :class:`~collections.OrderedDict` class is now implemented in C, which +makes it 4 to 100 times faster. (Contributed by Eric Snow in :issue:`16991`.) + +The :class:`~collections.deque` class now defines +:meth:`~collections.deque.index`, :meth:`~collections.deque.insert`, and +:meth:`~collections.deque.copy`. This allows deques to be recognized as a +:class:`~collections.abc.MutableSequence` and improves their substitutability +for lists. (Contributed by Raymond Hettinger :issue:`23704`.) + +Docstrings produced by :func:`~collections.namedtuple` can now be updated:: Point = namedtuple('Point', ['x', 'y']) Point.__doc__ = 'ordered pair' @@ -673,28 +682,20 @@ (Contributed by Berker Peksag in :issue:`24064`.) -The :class:`~collections.deque` now defines -:meth:`~collections.deque.index`, :meth:`~collections.deque.insert`, and -:meth:`~collections.deque.copy`. This allows deques to be recognized as a -:class:`~collections.abc.MutableSequence` and improves their substitutablity -for lists. (Contributed by Raymond Hettinger :issue:`23704`.) - -:class:`~collections.UserString` now implements :meth:`__getnewargs__`, -:meth:`__rmod__`, :meth:`casefold`, :meth:`format_map`, :meth:`isprintable`, and -:meth:`maketrans` methods to match corresponding methods of :class:`str`. +The :class:`~collections.UserString` class now implements +:meth:`__getnewargs__`, :meth:`__rmod__`, :meth:`~str.casefold`, +:meth:`~str.format_map`, :meth:`~str.isprintable`, and :meth:`~str.maketrans` +methods to match corresponding methods of :class:`str`. (Contributed by Joe Jevnik in :issue:`22189`.) -:class:`collections.OrderedDict` is now implemented in C, which makes it 4x to -100x faster. (Contributed by Eric Snow in :issue:`16991`.) - collections.abc --------------- -New :class:`~collections.abc.Generator` abstract base class. (Contributed +A new :class:`~collections.abc.Generator` abstract base class. (Contributed by Stefan Behnel in :issue:`24018`.) -New :class:`~collections.abc.Coroutine`, +A new :class:`~collections.abc.Coroutine`, :class:`~collections.abc.AsyncIterator`, and :class:`~collections.abc.AsyncIterable` abstract base classes. (Contributed by Yury Selivanov in :issue:`24184`.) @@ -703,89 +704,93 @@ compileall ---------- -A new :mod:`compileall` option, :option:`-j N`, allows to run ``N`` workers -sumultaneously to perform parallel bytecode compilation. :func:`compileall.compile_dir` has a corresponding ``workers`` parameter. (Contributed by -Claudiu Popa in :issue:`16104`.) +A new :mod:`compileall` option, ``-j N``, allows to run ``N`` workers +sumultaneously to perform parallel bytecode compilation. +The :func:`~compileall.compile_dir` function has a corresponding ``workers`` +parameter. (Contributed by Claudiu Popa in :issue:`16104`.) -The :option:`-q` command line option can now be specified more than once, in +The ``-q`` command line option can now be specified more than once, in which case all output, including errors, will be suppressed. The corresponding -``quiet`` parameter in :func:`compileall.compile_dir`, :func:`compileall. -compile_file`, and :func:`compileall.compile_path` can now accept -an integer value indicating the level of output suppression. +``quiet`` parameter in :func:`~compileall.compile_dir`, +:func:`~compileall.compile_file`, and :func:`~compileall.compile_path` can now +accept an integer value indicating the level of output suppression. (Contributed by Thomas Kluyver in :issue:`21338`.) concurrent.futures ------------------ -:meth:`~concurrent.futures.Executor.map` now accepts a *chunksize* -argument to allow batching of tasks in child processes and improve performance -of ProcessPoolExecutor. (Contributed by Dan O'Reilly in :issue:`11271`.) +The :meth:`Executor.map ` method now accepts a +*chunksize* argument to allow batching of tasks in child processes and improve +performance of :meth:`~concurrent.futures.ProcessPoolExecutor`. +(Contributed by Dan O'Reilly in :issue:`11271`.) contextlib ---------- -The new :func:`contextlib.redirect_stderr` context manager (similar to -:func:`contextlib.redirect_stdout`) makes it easier for utility scripts to +The new :func:`~contextlib.redirect_stderr` context manager (similar to +:func:`~contextlib.redirect_stdout`) makes it easier for utility scripts to handle inflexible APIs that write their output to :data:`sys.stderr` and -don't provide any options to redirect it. (Contributed by Berker Peksag in +don't provide any options to redirect it. (Contributed by Berker Peksag in :issue:`22389`.) curses ------ -The new :func:`curses.update_lines_cols` function updates the variables -:envvar:`curses.LINES` and :envvar:`curses.COLS`. +The new :func:`~curses.update_lines_cols` function updates the variables +:data:`curses.LINES` and :data:`curses.COLS`. difflib ------- -The charset of the HTML document generated by :meth:`difflib.HtmlDiff.make_file` +The charset of the HTML document generated by +:meth:`HtmlDiff.make_file ` can now be customized by using *charset* keyword-only parameter. The default -charset of HTML document changed from ``'ISO-8859-1'`` to ``'utf-8'``. +charset of HTML document changed from ``"ISO-8859-1"`` to ``"utf-8"``. (Contributed by Berker Peksag in :issue:`2052`.) -It is now possible to compare lists of byte strings with -:func:`difflib.diff_bytes`. This fixes a regression from Python 2. +It is now possible to compare lists of byte strings with the +:func:`~difflib.diff_bytes` function. This fixes a regression from Python 2. (Contributed by Terry J. Reedy and Greg Ward in :issue:`17445`.) distutils --------- -The ``build`` and ``build_ext`` commands now accept a :option:`-j` option to +The ``build`` and ``build_ext`` commands now accept a ``-j`` option to enable parallel building of extension modules. (Contributed by Antoine Pitrou in :issue:`5309`.) -:mod:`distutils` now supports ``xz`` compression, and can be enabled by -passing ``xztar`` as an argument to ``bdist --format``. +The :mod:`distutils` module now supports ``xz`` compression, and can be +enabled by passing ``xztar`` as an argument to ``bdist --format``. (Contributed by Serhiy Storchaka in :issue:`16314`.) doctest ------- -:func:`doctest.DocTestSuite` returns an empty :class:`unittest.TestSuite` if -*module* contains no docstrings instead of raising :exc:`ValueError`. -(Contributed by Glenn Jones in :issue:`15916`.) +The :func:`~doctest.DocTestSuite` function returns an empty +:class:`unittest.TestSuite` if *module* contains no docstrings instead of +raising :exc:`ValueError`. (Contributed by Glenn Jones in :issue:`15916`.) email ----- -A new policy option :attr:`~email.policy.Policy.mangle_from_` controls -whether or not lines that start with ``"From "`` in email bodies are prefixed -with a ``'>'`` character by generators. The default is ``True`` for +A new policy option :attr:`Policy.mangle_from_ ` +controls whether or not lines that start with ``"From "`` in email bodies are +prefixed with a ``">"`` character by generators. The default is ``True`` for :attr:`~email.policy.compat32` and ``False`` for all other policies. (Contributed by Milan Oberkirch in :issue:`20098`.) -A new method :meth:`~email.message.Message.get_content_disposition` provides -easy access to a canonical value for the :mailheader:`Content-Disposition` -header (``None`` if there is no such header). (Contributed by Abhilash Raj -in :issue:`21083`.) +A new +:meth:`Message.get_content_disposition ` +method provides easy access to a canonical value for the +:mailheader:`Content-Disposition` header. +(Contributed by Abhilash Raj in :issue:`21083`.) A new policy option :attr:`~email.policy.EmailPolicy.utf8` can be set to ``True`` to encode email headers using the UTF-8 charset instead of using @@ -797,7 +802,7 @@ faulthandler ------------ -:func:`~faulthandler.enable`, :func:`~faulthandler.register`, +The :func:`~faulthandler.enable`, :func:`~faulthandler.register`, :func:`~faulthandler.dump_traceback` and :func:`~faulthandler.dump_traceback_later` functions now accept file descriptors in addition to file-like objects. @@ -815,9 +820,9 @@ glob ---- -:func:`~glob.iglob` and :func:`~glob.glob` now support recursive search in -subdirectories using the "``**``" pattern. (Contributed by Serhiy Storchaka -in :issue:`13968`.) +The :func:`~glob.iglob` and :func:`~glob.glob` functions now support recursive +search in subdirectories using the ``"**"`` pattern. +(Contributed by Serhiy Storchaka in :issue:`13968`.) heapq @@ -826,7 +831,7 @@ Element comparison in :func:`~heapq.merge` can now be customized by passing a :term:`key function` in a new optional ``key`` keyword argument. A new optional ``reverse`` keyword argument can be used to reverse element -comparison. (Contributed by Raymond Hettinger in :issue:`13742`.) +comparison. (Contributed by Raymond Hettinger in :issue:`13742`.) idlelib and IDLE @@ -842,95 +847,101 @@ imaplib ------- -:class:`~imaplib.IMAP4` now supports context manager protocol. +The :class:`~imaplib.IMAP4` class now supports context manager protocol. When used in a :keyword:`with` statement, the IMAP4 ``LOGOUT`` command will be called automatically at the end of the block. (Contributed by Tarek Ziad? and Serhiy Storchaka in :issue:`4972`.) -:mod:`imaplib` now supports :rfc:`5161` (``ENABLE`` extension) via -:meth:`~imaplib.IMAP4.enable`, and :rfc:`6855` (UTF-8 support) via the -``UTF8=ACCEPT`` argument to :meth:`~imaplib.IMAP4.enable`. A new attribute, -:attr:`~imaplib.IMAP4.utf8_enabled`, tracks whether or not :rfc:`6855` -support is enabled. (Contributed by Milan Oberkirch, R. David Murray, -and Maciej Szulik in :issue:`21800`.) +The :mod:`imaplib` module now supports :rfc:`5161` (ENABLE Extension) +and :rfc:`6855` (UTF-8 Support) via the :meth:`IMAP4.enable ` +method. A new :attr:`IMAP4.utf8_enabled ` +attribute, tracks whether or not :rfc:`6855` support is enabled. +(Contributed by Milan Oberkirch, R. David Murray, and Maciej Szulik in +:issue:`21800`.) -:mod:`imaplib` now automatically encodes non-ASCII string usernames and -passwords using UTF-8, as recommended by the RFCs. (Contributed by Milan +The :mod:`imaplib` module now automatically encodes non-ASCII string usernames +and passwords using UTF-8, as recommended by the RFCs. (Contributed by Milan Oberkirch in :issue:`21800`.) imghdr ------ -:func:`~imghdr.what` now recognizes the `OpenEXR `_ -format (contributed by Martin Vignali and Claudiu Popa in :issue:`20295`), -and the `WebP `_ format (contributed -by Fabrice Aneche and Claudiu Popa in :issue:`20197`.) +The :func:`~imghdr.what` function now recognizes the +`OpenEXR `_ format +(contributed by Martin Vignali and Claudiu Popa in :issue:`20295`), +and the `WebP `_ format +(contributed by Fabrice Aneche and Claudiu Popa in :issue:`20197`.) importlib --------- -:class:`importlib.util.LazyLoader` allows for lazy loading of modules in -applications where startup time is important. (Contributed by Brett Cannon +The :class:`importlib.util.LazyLoader` class allows for lazy loading of modules +in applications where startup time is important. (Contributed by Brett Cannon in :issue:`17621`.) -:func:`importlib.abc.InspectLoader.source_to_code` is now a +The :func:`importlib.abc.InspectLoader.source_to_code` method is now a static method. This makes it easier to initialize a module object with -code compiled from a string by runnning ``exec(code, module.__dict__)``. +code compiled from a string by running ``exec(code, module.__dict__)``. (Contributed by Brett Cannon in :issue:`21156`.) -:func:`importlib.util.module_from_spec` is now the preferred way to create a -new module. Compared to :class:`types.ModuleType`, this new function will set -the various import-controlled attributes based on the passed-in spec object. +The new :func:`importlib.util.module_from_spec` function is now the preferred +way to create a new module. Compared to the :class:`types.ModuleType` class, +this new function will set the various import-controlled attributes based +on the passed-in spec object. (Contributed by Brett Cannon in :issue:`20383`.) inspect ------- -:class:`inspect.Signature` and :class:`inspect.Parameter` are now +The :class:`~inspect.Signature` and :class:`~inspect.Parameter` classes are now picklable and hashable. (Contributed by Yury Selivanov in :issue:`20726` and :issue:`20334`.) -A new method :meth:`inspect.BoundArguments.apply_defaults` provides a way -to set default values for missing arguments. (Contributed by Yury Selivanov -in :issue:`24190`.) +A new +:meth:`BoundArguments.apply_defaults ` +method provides a way to set default values for missing arguments. +(Contributed by Yury Selivanov in :issue:`24190`.) -A new class method :meth:`inspect.Signature.from_callable` makes +A new class method +:meth:`Signature.from_callable ` makes subclassing of :class:`~inspect.Signature` easier. (Contributed by Yury Selivanov and Eric Snow in :issue:`17373`.) -:func:`inspect.signature` now accepts a ``follow_wrapped`` optional keyword -argument, which, when set to ``False``, disables automatic following of -``__wrapped__`` links. (Contributed by Yury Selivanov in :issue:`20691`.) +The :func:`~inspect.signature` function now accepts a ``follow_wrapped`` +optional keyword argument, which, when set to ``False``, disables automatic +following of ``__wrapped__`` links. +(Contributed by Yury Selivanov in :issue:`20691`.) A set of new functions to inspect :term:`coroutine functions ` and -``coroutine objects`` as been added: +``coroutine objects`` has been added: :func:`~inspect.iscoroutine`, :func:`~inspect.iscoroutinefunction`, :func:`~inspect.isawaitable`, :func:`~inspect.getcoroutinelocals`, and :func:`~inspect.getcoroutinestate`. (Contributed by Yury Selivanov in :issue:`24017` and :issue:`24400`.) -:func:`~inspect.stack`, :func:`~inspect.trace`, :func:`~inspect.getouterframes`, -and :func:`~inspect.getinnerframes` now return a list of named tuples. +The :func:`~inspect.stack`, :func:`~inspect.trace`, +:func:`~inspect.getouterframes`, and :func:`~inspect.getinnerframes` +functions now return a list of named tuples. (Contributed by Daniel Shahaf in :issue:`16808`.) io -- -:class:`io.FileIO` has been implemented in Python which makes C implementation -of :mod:`io` module entirely optional. (Contributed by Serhiy Storchaka -in :issue:`21859`.) +The :class:`~io.FileIO` class has been implemented in Python which makes +the C implementation of the :mod:`io` module entirely optional. +(Contributed by Serhiy Storchaka in :issue:`21859`.) ipaddress --------- -:class:`ipaddress.IPv4Network` and :class:`ipaddress.IPv6Network` now -accept an ``(address, netmask)`` tuple argument, so as to easily construct +The :class:`~ipaddress.IPv4Network` and :class:`~ipaddress.IPv6Network` classes +now accept an ``(address, netmask)`` tuple argument, so as to easily construct network objects from existing addresses. (Contributed by Peter Moody and Antoine Pitrou in :issue:`16531`.) @@ -938,8 +949,8 @@ json ---- -:mod:`json.tool` command line interface now preserves the order of keys in -JSON objects passed in input. The new :option:`--sort-keys` option can be used +The :mod:`json.tool` command line interface now preserves the order of keys in +JSON objects passed in input. The new ``--sort-keys`` option can be used to sort the keys alphabetically. (Contributed by Berker Peksag in :issue:`21650`.) @@ -958,36 +969,39 @@ logging ------- -All logging methods (:meth:`~logging.Logger.log`, +All logging methods (:class:`~logging.Logger` :meth:`~logging.Logger.log`, :meth:`~logging.Logger.exception`, :meth:`~logging.Logger.critical`, :meth:`~logging.Logger.debug`, etc.), now accept exception instances -in ``exc_info`` parameter, in addition to boolean values and exception +in ``exc_info`` argument, in addition to boolean values and exception tuples. (Contributed by Yury Selivanov in :issue:`20537`.) -:class:`~logging.handlers.HTTPHandler` now accepts an optional -:class:`ssl.SSLContext` instance to configure the SSL settings used -in an HTTP connection. (Contributed by Alex Gaynor in :issue:`22788`.) +The :class:`handlers.HTTPHandler ` classes now +accepts an optional :class:`ssl.SSLContext` instance to configure the SSL +settings used in an HTTP connection. +(Contributed by Alex Gaynor in :issue:`22788`.) -:class:`~logging.handlers.QueueListener` now takes a *respect_handler_level* -keyword argument which, if set to ``True``, will pass messages to handlers -taking handler levels into account. (Contributed by Vinay Sajip.) +The :class:`handlers.QueueListener ` class now +takes a *respect_handler_level* keyword argument which, if set to ``True``, +will pass messages to handlers taking handler levels into account. +(Contributed by Vinay Sajip.) lzma ---- -:meth:`~lzma.LZMADecompressor.decompress` now accepts an optional *max_length* -argument to limit the maximum size of decompressed data. +The :meth:`LZMADecompressor.decompress ` +method now accepts an optional *max_length* argument to limit the maximum +size of decompressed data. (Contributed by Martin Panter in :issue:`15955`.) math ---- -Two new constants have been added to :mod:`math`: :data:`math.inf` -and :data:`math.nan`. (Contributed by Mark Dickinson in :issue:`23185`.) +Two new constants have been added to the :mod:`math` module: :data:`~math.inf` +and :data:`~math.nan`. (Contributed by Mark Dickinson in :issue:`23185`.) -A new function :func:`math.isclose` provides a way to test for approximate +A new function :func:`~math.isclose` provides a way to test for approximate equality. (Contributed by Chris Barker and Tal Einat in :issue:`24270`.) A new :func:`~math.gcd` function has been added. The :func:`fractions.gcd` @@ -998,41 +1012,38 @@ operator -------- -:func:`~operator.attrgetter`, :func:`~operator.itemgetter`, and -:func:`~operator.methodcaller` objects now support pickling. +The :mod:`operator` :func:`~operator.attrgetter`, :func:`~operator.itemgetter`, +and :func:`~operator.methodcaller` objects now support pickling. (Contributed by Josh Rosenberg and Serhiy Storchaka in :issue:`22955`.) os -- -The new :func:`os.scandir` returning an iterator of :class:`os.DirEntry` -objects has been added. If possible, :func:`os.scandir` extracts file -attributes while scanning a directory, removing the need to perform -subsequent system calls to determine file type or attributes, which may +The new :func:`~os.scandir` function returning an iterator of +:class:`~os.DirEntry` objects has been added. If possible, :func:`~os.scandir` +extracts file attributes while scanning a directory, removing the need to +perform subsequent system calls to determine file type or attributes, which may significantly improve performance. (Contributed by Ben Hoyt with the help of Victor Stinner in :issue:`22524`.) -On Windows, a new :attr:`~os.stat_result.st_file_attributes` attribute is -now available. It corresponds to ``dwFileAttributes`` member of the -``BY_HANDLE_FILE_INFORMATION`` structure returned by +On Windows, a new +:attr:`stat_result.st_file_attributes ` +attribute is now available. It corresponds to ``dwFileAttributes`` member of +the ``BY_HANDLE_FILE_INFORMATION`` structure returned by ``GetFileInformationByHandle()``. (Contributed by Ben Hoyt in :issue:`21719`.) -:func:`os.urandom` now uses ``getrandom()`` syscall on Linux 3.17 or newer, -and ``getentropy()`` on OpenBSD 5.6 and newer, removing the need to use -``/dev/urandom`` and avoiding failures due to potential file descriptor +The :func:`~os.urandom` function now uses ``getrandom()`` syscall on Linux 3.17 +or newer, and ``getentropy()`` on OpenBSD 5.6 and newer, removing the need to +use ``/dev/urandom`` and avoiding failures due to potential file descriptor exhaustion. (Contributed by Victor Stinner in :issue:`22181`.) -New :func:`os.get_blocking` and :func:`os.set_blocking` functions allow to +New :func:`~os.get_blocking` and :func:`~os.set_blocking` functions allow to get and set the file descriptor blocking mode (:data:`~os.O_NONBLOCK`.) (Contributed by Victor Stinner in :issue:`22054`.) -:func:`~os.truncate` and :func:`~os.ftruncate` are now supported on -Windows. (Contributed by Steve Dower in :issue:`23668`.) - - -os.path -------- +The :func:`~os.truncate` and :func:`~os.ftruncate` functions are now supported +on Windows. (Contributed by Steve Dower in :issue:`23668`.) There is a new :func:`~os.path.commonpath` function returning the longest common sub-path of each passed pathname. Unlike the @@ -1043,28 +1054,30 @@ pathlib ------- -The new :meth:`~pathlib.Path.samefile` method can be used to check if the -passed :class:`~pathlib.Path` object, or a string, point to the same file as -the :class:`~pathlib.Path` on which :meth:`~pathlib.Path.samefile` is called. +The new :meth:`Path.samefile ` method can be used +to check if the passed :class:`~pathlib.Path` object or a :class:`str` path, +point to the same file. (Contributed by Vajrasky Kok and Antoine Pitrou in :issue:`19775`.) -:meth:`~pathlib.Path.mkdir` how accepts a new optional ``exist_ok`` argument -to match ``mkdir -p`` and :func:`os.makrdirs` functionality. -(Contributed by Berker Peksag in :issue:`21539`.) +The :meth:`Path.mkdir ` method how accepts a new optional +``exist_ok`` argument to match ``mkdir -p`` and :func:`os.makrdirs` +functionality. (Contributed by Berker Peksag in :issue:`21539`.) -There is a new :meth:`~pathlib.Path.expanduser` method to expand ``~`` -and ``~user`` prefixes. (Contributed by Serhiy Storchaka and Claudiu -Popa in :issue:`19776`.) +There is a new :meth:`Path.expanduser ` method to +expand ``~`` and ``~user`` prefixes. (Contributed by Serhiy Storchaka and +Claudiu Popa in :issue:`19776`.) -A new :meth:`~pathlib.Path.home` class method can be used to get an instance -of :class:`~pathlib.Path` object representing the user?s home directory. +A new :meth:`Path.home ` class method can be used to get +an instance of :class:`~pathlib.Path` object representing the user?s home +directory. (Contributed by Victor Salgado and Mayank Tripathi in :issue:`19777`.) pickle ------ -Nested objects, such as unbound methods or nested classes, can now be pickled using :ref:`pickle protocols ` older than protocol version 4, +Nested objects, such as unbound methods or nested classes, can now be pickled +using :ref:`pickle protocols ` older than protocol version 4, which already supported these cases. (Contributed by Serhiy Storchaka in :issue:`23611`.) @@ -1072,7 +1085,7 @@ poplib ------ -A new command :meth:`~poplib.POP3.utf8` enables :rfc:`6856` +A new command :meth:`POP3.utf8 ` enables :rfc:`6856` (internationalized email) support, if the POP server supports it. (Contributed by Milan OberKirch in :issue:`21804`.) @@ -1083,9 +1096,9 @@ The number of capturing groups in regular expression is no longer limited by 100. (Contributed by Serhiy Storchaka in :issue:`22437`.) -:func:`re.sub` and :func:`re.subn` now replace unmatched groups with empty -strings instead of rising an exception. (Contributed by Serhiy Storchaka -in :issue:`1519638`.) +The :func:`~re.sub` and :func:`~re.subn` functions now replace unmatched +groups with empty strings instead of rising an exception. +(Contributed by Serhiy Storchaka in :issue:`1519638`.) readline @@ -1099,20 +1112,21 @@ shutil ------ -:func:`~shutil.move` now accepts a *copy_function* argument, allowing, -for example, :func:`~shutil.copy` to be used instead of the default -:func:`~shutil.copy2` there is a need to ignore file metadata when moving. +The :func:`~shutil.move` function now accepts a *copy_function* argument, +allowing, for example, the :func:`~shutil.copy` function to be used instead of +the default :func:`~shutil.copy2` if there is a need to ignore file metadata +when moving. (Contributed by Claudiu Popa in :issue:`19840`.) -:func:`~shutil.make_archive` now supports *xztar* format. +The :func:`~shutil.make_archive` function now supports *xztar* format. (Contributed by Serhiy Storchaka in :issue:`5411`.) signal ------ -On Windows, :func:`signal.set_wakeup_fd` now also supports socket handles. -(Contributed by Victor Stinner in :issue:`22018`.) +On Windows, the :func:`~signal.set_wakeup_fd` function now also supports +socket handles. (Contributed by Victor Stinner in :issue:`22018`.) Various ``SIG*`` constants in :mod:`signal` module have been converted into :mod:`Enums `. This allows meaningful names to be printed @@ -1123,28 +1137,30 @@ smtpd ----- -Both :class:`~smtpd.SMTPServer` and :class:`smtpd.SMTPChannel` now accept a -*decode_data* keyword argument to determine if the ``DATA`` portion of the SMTP -transaction is decoded using the ``"utf-8"`` codec or is instead provided to -:meth:`~smtpd.SMTPServer.process_message` as a byte string. The default -is ``True`` for backward compatibility reasons, but will change to ``False`` -in Python 3.6. If *decode_data* is set to ``False``, the -:meth:`~smtpd.SMTPServer.process_message` method must be prepared to accept -keyword arguments. (Contributed by Maciej Szulik in :issue:`19662`.) +Both :class:`~smtpd.SMTPServer` and :class:`~smtpd.SMTPChannel` classes now +accept a *decode_data* keyword argument to determine if the ``DATA`` portion of +the SMTP transaction is decoded using the ``"utf-8"`` codec or is instead +provided to :meth:`SMTPServer.process_message ` +method as a byte string. The default is ``True`` for backward compatibility +reasons, but will change to ``False`` in Python 3.6. If *decode_data* is set +to ``False``, the :meth:`~smtpd.SMTPServer.process_message` method must +be prepared to accept keyword arguments. +(Contributed by Maciej Szulik in :issue:`19662`.) -:class:`~smtpd.SMTPServer` now advertises the ``8BITMIME`` extension -(:rfc:`6152`) if if *decode_data* has been set ``True``. If the client +The :class:`~smtpd.SMTPServer` class now advertises the ``8BITMIME`` extension +(:rfc:`6152`) if *decode_data* has been set ``True``. If the client specifies ``BODY=8BITMIME`` on the ``MAIL`` command, it is passed to -:meth:`~smtpd.SMTPServer.process_message` via the ``mail_options`` keyword. +:meth:`SMTPServer.process_message ` +via the ``mail_options`` keyword. (Contributed by Milan Oberkirch and R. David Murray in :issue:`21795`.) -:class:`~smtpd.SMTPServer` now supports the ``SMTPUTF8`` extension -(:rfc:`6531`: Internationalized Email). If the client specified ``SMTPUTF8 -BODY=8BITMIME`` on the ``MAIL`` command, they are passed to -:meth:`~smtpd.SMTPServer.process_message` via the ``mail_options`` keyword. -It is the responsibility of the :meth:`~smtpd.SMTPServer.process_message` -method to correctly handle the ``SMTPUTF8`` data. (Contributed by Milan -Oberkirch in :issue:`21725`.) +The :class:`~smtpd.SMTPServer` class now also supports the ``SMTPUTF8`` +extension (:rfc:`6531`: Internationalized Email). If the client specified +``SMTPUTF8 BODY=8BITMIME`` on the ``MAIL`` command, they are passed to +:meth:`SMTPServer.process_message ` +via the ``mail_options`` keyword. It is the responsibility of the +:meth:`~smtpd.SMTPServer.process_message` method to correctly handle the +``SMTPUTF8`` data. (Contributed by Milan Oberkirch in :issue:`21725`.) It is now possible to provide, directly or via name resolution, IPv6 addresses in the :class:`~smtpd.SMTPServer` constructor, and have it @@ -1154,7 +1170,7 @@ smtplib ------- -A new :meth:`~smtplib.SMTP.auth` method provides a convenient way to +A new :meth:`SMTP.auth ` method provides a convenient way to implement custom authentication mechanisms. (Contributed by Milan Oberkirch in :issue:`15014`.) @@ -1162,17 +1178,17 @@ :class:`smtplib.SMTP`. (Contributed by Gavin Chappell and Maciej Szulik in :issue:`16914`.) -:mod:`smtplib` now supports :rfc:`6531` (SMTPUTF8) in both the -:meth:`~smtplib.SMTP.sendmail` and :meth:`~smtplib.SMTP.send_message` -commands. (Contributed by Milan Oberkirch and R. David Murray in -:issue:`22027`.) +Both :meth:`SMTP.sendmail ` and +:meth:`SMTP.send_message ` methods now +support support :rfc:`6531` (SMTPUTF8). +(Contributed by Milan Oberkirch and R. David Murray in :issue:`22027`.) sndhdr ------ -:func:`~sndhdr.what` and :func:`~sndhdr.whathdr` now return -:func:`~collections.namedtuple`. (Contributed by Claudiu Popa in +The :func:`~sndhdr.what` and :func:`~sndhdr.whathdr` functions now return +a :func:`~collections.namedtuple`. (Contributed by Claudiu Popa in :issue:`18615`.) @@ -1186,17 +1202,17 @@ The new :class:`~ssl.SSLObject` class has been added to provide SSL protocol support for cases when the network IO capabilities of :class:`~ssl.SSLSocket` -are not necessary or inappropriate. :class:`~ssl.SSLObject` represents +are not necessary or suboptimal. :class:`~ssl.SSLObject` represents an SSL protocol instance, but does not implement any network IO methods, and instead provides a memory buffer interface. The new :class:`~ssl.MemoryBIO` class can be used to pass data between Python and an SSL protocol instance. The memory BIO SSL support is primarily intended to be used in frameworks implementing asynchronous IO for which :class:`~ssl.SSLObject` IO readiness -model ("select/poll") is inappropriate or inefficient. +model ("select/poll") is inefficient. -A new :meth:`~ssl.SSLContext.wrap_bio` method can be used to create a new -:class:`~ssl.SSLObject` instance. +A new :meth:`SSLContext.wrap_bio ` method can be used +to create a new :class:`~ssl.SSLObject` instance. Application-Layer Protocol Negotiation Support @@ -1207,41 +1223,49 @@ Where OpenSSL support is present, :mod:`ssl` module now implements * Application-Layer Protocol Negotiation* TLS extension as described in :rfc:`7301`. + The new :meth:`SSLContext.set_alpn_protocols ` can be used to specify which protocols the socket should advertise during -the TLS handshake. The new +the TLS handshake. + +The new :meth:`SSLSocket.selected_alpn_protocol ` -returns the protocol that was selected during the TLS handshake. :data:`ssl.HAS_ALPN` flag indicates whether APLN support is present. +returns the protocol that was selected during the TLS handshake. +:data:`~ssl.HAS_ALPN` flag indicates whether APLN support is present. Other Changes ~~~~~~~~~~~~~ -There is a new :meth:`~ssl.SSLSocket.version` method to query the actual -protocol version in use. (Contributed by Antoine Pitrou in :issue:`20421`.) +There is a new :meth:`SSLSocket.version ` method to query +the actual protocol version in use. +(Contributed by Antoine Pitrou in :issue:`20421`.) -:class:`~ssl.SSLSocket` now implementes :meth:`~ssl.SSLSocket.sendfile` -method. (Contributed by Giampaolo Rodola' in :issue:`17552`.) +The :class:`~ssl.SSLSocket` class now implements +a :meth:`SSLSocket.sendfile ` method. +(Contributed by Giampaolo Rodola in :issue:`17552`.) -:meth:`ssl.SSLSocket.send()` now raises either :exc:`ssl.SSLWantReadError` -or :exc:`ssl.SSLWantWriteError` on a non-blocking socket if the operation -would block. Previously, it would return 0. (Contributed by Nikolaus Rath -in :issue:`20951`.) +The :meth:`SSLSocket.send ` method now raises either +:exc:`ssl.SSLWantReadError` or :exc:`ssl.SSLWantWriteError` exception on a +non-blocking socket if the operation would block. Previously, it would return +``0``. (Contributed by Nikolaus Rath in :issue:`20951`.) The :func:`~ssl.cert_time_to_seconds` function now interprets the input time as UTC and not as local time, per :rfc:`5280`. Additionally, the return value is always an :class:`int`. (Contributed by Akira Li in :issue:`19940`.) -New :meth:`~ssl.SSLObject.shared_ciphers` and -:meth:`~ssl.SSLSocket.shared_ciphers` methods return the list of ciphers -sent by the client during the handshake. (Contributed by Benjamin Peterson -in :issue:`23186`.) +New :meth:`SSLObject.shared_ciphers ` and +:meth:`SSLSocket.shared_ciphers ` methods return +the list of ciphers sent by the client during the handshake. +(Contributed by Benjamin Peterson in :issue:`23186`.) -The :meth:`~ssl.SSLSocket.do_handshake`, :meth:`~ssl.SSLSocket.read`, -:meth:`~ssl.SSLSocket.shutdown`, and :meth:`~ssl.SSLSocket.write` methods of -:class:`ssl.SSLSocket` no longer reset the socket timeout every time bytes -are received or sent. The socket timeout is now the maximum total duration of -the method. (Contributed by Victor Stinner in :issue:`23853`.) +The :meth:`SSLSocket.do_handshake `, +:meth:`SSLSocket.read `, +:meth:`SSLSocket.shutdown `, and +:meth:`SSLSocket.write ` methods of :class:`ssl.SSLSocket` +class no longer reset the socket timeout every time bytes are received or sent. +The socket timeout is now the maximum total duration of the method. +(Contributed by Victor Stinner in :issue:`23853`.) The :func:`~ssl.match_hostname` function now supports matching of IP addresses. (Contributed by Antoine Pitrou in :issue:`23239`.) @@ -1250,28 +1274,28 @@ socket ------ -A new :meth:`socket.socket.sendfile` method allows to send a file over a -socket by using high-performance :func:`os.sendfile` function on UNIX -resulting in uploads being from 2x to 3x faster than when using plain -:meth:`socket.socket.send`. (Contributed by Giampaolo Rodola' in -:issue:`17552`.) - -The :meth:`socket.socket.sendall` method no longer resets the socket timeout -every time bytes are received or sent. The socket timeout is now the -maximum total duration to send all data. (Contributed by Victor Stinner in -:issue:`23853`.) - Functions with timeouts now use a monotonic clock, instead of a system clock. (Contributed by Victor Stinner in :issue:`22043`.) +A new :meth:`socket.sendfile ` method allows to +send a file over a socket by using high-performance :func:`os.sendfile` +function on UNIX resulting in uploads being from 2 to 3 times faster than when +using plain :meth:`socket.send `. +(Contributed by Giampaolo Rodola' in :issue:`17552`.) + +The :meth:`socket.sendall ` method no longer resets the +socket timeout every time bytes are received or sent. The socket timeout is +now the maximum total duration to send all data. +(Contributed by Victor Stinner in :issue:`23853`.) + subprocess ---------- -The new :func:`subprocess.run` function has been added and is the recommended +The new :func:`~subprocess.run` function has been added and is the recommended approach to invoking subprocesses. It runs the specified command and -and returns a :class:`subprocess.CompletedProcess` object. (Contributed by -Thomas Kluyver in :issue:`23342`.) +and returns a :class:`~subprocess.CompletedProcess` object. +(Contributed by Thomas Kluyver in :issue:`23342`.) sys @@ -1300,35 +1324,38 @@ tarfile ------- -The *mode* argument of :func:`tarfile.open` function now accepts ``'x'`` to request exclusive creation. (Contributed by Berker Peksag in :issue:`21717`.) +The *mode* argument of the :func:`~tarfile.open` function now accepts ``"x"`` +to request exclusive creation. (Contributed by Berker Peksag in :issue:`21717`.) -The :meth:`~tarfile.TarFile.extractall` and :meth:`~tarfile.TarFile.extract` -methods now take a keyword argument *numeric_only*. If set to ``True``, -the extracted files and directories will be owned by the numeric uid and gid -from the tarfile. If set to ``False`` (the default, and the behavior in -versions prior to 3.5), they will be owned by the named user and group in the -tarfile. (Contributed by Michael Vogt and Eric Smith in :issue:`23193`.) +The :meth:`TarFile.extractall ` and +:meth:`TarFile.extract ` methods now take a keyword +argument *numeric_only*. If set to ``True``, the extracted files and +directories will be owned by the numeric ``uid`` and ``gid`` from the tarfile. +If set to ``False`` (the default, and the behavior in versions prior to 3.5), +they will be owned by the named user and group in the tarfile. +(Contributed by Michael Vogt and Eric Smith in :issue:`23193`.) threading --------- -:meth:`~threading.Lock.acquire` and :meth:`~threading.RLock.acquire` -now use a monotonic clock for timeout management. (Contributed by Victor -Stinner in :issue:`22043`.) +The :meth:`Lock.acquire ` and +:meth:`RLock.acquire ` methods +now use a monotonic clock for timeout management. +(Contributed by Victor Stinner in :issue:`22043`.) time ---- -The :func:`time.monotonic` function is now always available. (Contributed by -Victor Stinner in :issue:`22043`.) +The :func:`~time.monotonic` function is now always available. +(Contributed by Victor Stinner in :issue:`22043`.) timeit ------ -New command line option :option:`-u` or :option:`--unit=U` to specify a time +New command line option ``-u`` or ``--unit=U`` to specify a time unit for the timer output. Supported options are ``usec``, ``msec``, or ``sec``. (Contributed by Julian Gindi in :issue:`18983`.) @@ -1353,8 +1380,8 @@ :class:`~traceback.StackSummary`, and :class:`traceback.FrameSummary`. (Contributed by Robert Collins in :issue:`17911`.) -:func:`~traceback.print_tb` and :func:`~traceback.print_stack` now support -negative values for the *limit* argument. +The :func:`~traceback.print_tb` and :func:`~traceback.print_stack` functions +now support negative values for the *limit* argument. (Contributed by Dmitry Kazakov in :issue:`22619`.) @@ -1371,21 +1398,23 @@ urllib ------ -A new :class:`~urllib.request.HTTPPasswordMgrWithPriorAuth` allows HTTP Basic -Authentication credentials to be managed so as to eliminate unnecessary -``401`` response handling, or to unconditionally send credentials -on the first request in order to communicate with servers that return a -``404`` response instead of a ``401`` if the ``Authorization`` header is not -sent. (Contributed by Matej Cepl in :issue:`19494` and Akshit Khurana in +A new +:class:`request.HTTPPasswordMgrWithPriorAuth ` +class allows HTTP Basic Authentication credentials to be managed so as to +eliminate unnecessary ``401`` response handling, or to unconditionally send +credentials on the first request in order to communicate with servers that +return a ``404`` response instead of a ``401`` if the ``Authorization`` header +is not sent. (Contributed by Matej Cepl in :issue:`19494` and Akshit Khurana in :issue:`7159`.) -A new :func:`~urllib.parse.urlencode` parameter *quote_via* provides a way to -control the encoding of query parts if needed. (Contributed by Samwyse and -Arnon Yaari in :issue:`13866`.) +A new *quote_via* argument for the +:func:`parse.urlencode ` +function provides a way to control the encoding of query parts if needed. +(Contributed by Samwyse and Arnon Yaari in :issue:`13866`.) -:func:`~urllib.request.urlopen` accepts an :class:`ssl.SSLContext` -object as a *context* argument, which will be used for the HTTPS -connection. (Contributed by Alex Gaynor in :issue:`22366`.) +The :func:`request.urlopen ` function accepts an +:class:`ssl.SSLContext` object as a *context* argument, which will be used for +the HTTPS connection. (Contributed by Alex Gaynor in :issue:`22366`.) unicodedata @@ -1398,33 +1427,35 @@ unittest -------- -New command line option :option:`--locals` to show local variables in +New command line option ``--locals`` to show local variables in tracebacks. (Contributed by Robert Collins in :issue:`22936`.) wsgiref ------- -*headers* parameter of :class:`wsgiref.headers.Headers` is now optional. +The *headers* argument of the :class:`headers.Headers ` +class constructor is now optional. (Contributed by Pablo Torres Navarrete and SilentGhost in :issue:`5800`.) xmlrpc ------ -:class:`xmlrpc.client.ServerProxy` is now a :term:`context manager`. +The :class:`client.ServerProxy ` class is now a +:term:`context manager`. (Contributed by Claudiu Popa in :issue:`20627`.) -:class:`~xmlrpc.client.ServerProxy` constructor now accepts an optional -:class:`ssl.SSLContext` instance. +:class:`client.ServerProxy ` constructor now accepts +an optional :class:`ssl.SSLContext` instance. (Contributed by Alex Gaynor in :issue:`22960`.) xml.sax ------- -SAX parsers now support a character stream of -:class:`~xml.sax.xmlreader.InputSource` object. +SAX parsers now support a character stream of the +:class:`xmlreader.InputSource ` object. (Contributed by Serhiy Storchaka in :issue:`2175`.) @@ -1434,8 +1465,8 @@ ZIP output can now be written to unseekable streams. (Contributed by Serhiy Storchaka in :issue:`23252`.) -The *mode* argument of :func:`zipfile.ZipFile.open` function now -accepts ``'x'`` to request exclusive creation. +The *mode* argument of :meth:`ZipFile.open ` method now +accepts ``"x"`` to request exclusive creation. (Contributed by Serhiy Storchaka in :issue:`21717`.) @@ -1450,80 +1481,76 @@ Optimizations ============= -The following performance enhancements have been added: +The :func:`os.walk` function has been sped up by 3-5 times on POSIX systems, +and by 7-20 times on Windows. This was done using the new :func:`os.scandir` +function, which exposes file information from the underlying ``readdir`` or +``FindFirstFile``/``FindNextFile`` system calls. (Contributed by +Ben Hoyt with help from Victor Stinner in :issue:`23605`.) -* :func:`os.walk` has been sped up by 3-5x on POSIX systems and 7-20x - on Windows. This was done using the new :func:`os.scandir` function, - which exposes file information from the underlying ``readdir`` and - ``FindFirstFile``/``FindNextFile`` system calls. (Contributed by - Ben Hoyt with help from Victor Stinner in :issue:`23605`.) +Construction of ``bytes(int)`` (filled by zero bytes) is faster and uses less +memory for large objects. ``calloc()`` is used instead of ``malloc()`` to +allocate memory for these objects. -* Construction of ``bytes(int)`` (filled by zero bytes) is faster and uses less - memory for large objects. ``calloc()`` is used instead of ``malloc()`` to - allocate memory for these objects. +Some operations on :mod:`ipaddress` :class:`~ipaddress.IPv4Network` and +:class:`~ipaddress.IPv6Network` have been massively sped up, such as +:meth:`~ipaddress.IPv4Network.subnets`, :meth:`~ipaddress.IPv4Network.supernet`, +:func:`~ipaddress.summarize_address_range`, :func:`~ipaddress.collapse_addresses`. +The speed up can range from 3 to 15 times. +(See :issue:`21486`, :issue:`21487`, :issue:`20826`, :issue:`23266`.) -* Some operations on :class:`~ipaddress.IPv4Network` and - :class:`~ipaddress.IPv6Network` have been massively sped up, such as - :meth:`~ipaddress.IPv4Network.subnets`, :meth:`~ipaddress.IPv4Network.supernet`, - :func:`~ipaddress.summarize_address_range`, :func:`~ipaddress.collapse_addresses`. - The speed up can range from 3x to 15x. - (See :issue:`21486`, :issue:`21487`, :issue:`20826`, :issue:`23266`.) +Pickling of :mod:`ipaddress` objects was optimized to produce significantly +smaller output. (Contributed by Serhiy Storchaka in :issue:`23133`.) -* Pickling of :mod:`ipaddress` classes was optimized to produce significantly - smaller output. (Contributed by Serhiy Storchaka in :issue:`23133`.) +Many operations on :class:`io.BytesIO` are now 50% to 100% faster. +(Contributed by Serhiy Storchaka in :issue:`15381` and David Wilson in +:issue:`22003`.) -* Many operations on :class:`io.BytesIO` are now 50% to 100% faster. - (Contributed by Serhiy Storchaka in :issue:`15381` and David Wilson in - :issue:`22003`.) +The :func:`marshal.dumps` function is now faster: 65-85% with versions 3 +and 4, 20-25% with versions 0 to 2 on typical data, and up to 5 times in +best cases. +(Contributed by Serhiy Storchaka in :issue:`20416` and :issue:`23344`.) -* :func:`marshal.dumps` is now faster (65%-85% with versions 3--4, 20-25% with - versions 0--2 on typical data, and up to 5x in best cases). - (Contributed by Serhiy Storchaka in :issue:`20416` and :issue:`23344`.) +The UTF-32 encoder is now 3 to 7 times faster. +(Contributed by Serhiy Storchaka in :issue:`15027`.) -* The UTF-32 encoder is now 3x to 7x faster. (Contributed by Serhiy Storchaka - in :issue:`15027`.) +Regular expressions are now parsed up to 10% faster. +(Contributed by Serhiy Storchaka in :issue:`19380`.) -* Regular expressions are now parsed up to 10% faster. - (Contributed by Serhiy Storchaka in :issue:`19380`.) +The :func:`json.dumps` function was optimized to run with +``ensure_ascii=False`` as fast as with ``ensure_ascii=True``. +(Contributed by Naoki Inada in :issue:`23206`.) -* :func:`json.dumps` was optimized to run with ``ensure_ascii=False`` - as fast as with ``ensure_ascii=True``. - (Contributed by Naoki Inada in :issue:`23206`.) +The :c:func:`PyObject_IsInstance` and :c:func:`PyObject_IsSubclass` +functions have been sped up in the common case that the second argument +has :class:`type` as its metaclass. +(Contributed Georg Brandl by in :issue:`22540`.) -* :c:func:`PyObject_IsInstance` and :c:func:`PyObject_IsSubclass` have - been sped up in the common case that the second argument has metaclass - :class:`type`. - (Contributed Georg Brandl by in :issue:`22540`.) +Method caching was slightly improved, yielding up to 5% performance +improvement in some benchmarks. +(Contributed by Antoine Pitrou in :issue:`22847`.) -* Method caching was slightly improved, yielding up to 5% performance - improvement in some benchmarks. - (Contributed by Antoine Pitrou in :issue:`22847`.) +Objects from :mod:`random` module now use two times less memory on 64-bit +builds. (Contributed by Serhiy Storchaka in :issue:`23488`.) -* Objects from :mod:`random` module now use 2x less memory on 64-bit - builds. - (Contributed by Serhiy Storchaka in :issue:`23488`.) +The :func:`property` getter calls are up to 25% faster. +(Contributed by Joe Jevnik in :issue:`23910`.) -* property() getter calls are up to 25% faster. - (Contributed by Joe Jevnik in :issue:`23910`.) - -* Instantiation of :class:`fractions.Fraction` is now up to 30% faster. - (Contributed by Stefan Behnel in :issue:`22464`.) +Instantiation of :class:`fractions.Fraction` is now up to 30% faster. +(Contributed by Stefan Behnel in :issue:`22464`.) Build and C API Changes ======================= -Changes to Python's build process and to the C API include: - -* New ``calloc`` functions: +New ``calloc`` functions: * :c:func:`PyMem_RawCalloc` * :c:func:`PyMem_Calloc` * :c:func:`PyObject_Calloc` * :c:func:`_PyObject_GC_Calloc` -* Windows builds now require Microsoft Visual C++ 14.0, which - is available as part of `Visual Studio 2015 `_. +Windows builds now require Microsoft Visual C++ 14.0, which +is available as part of `Visual Studio 2015 `_. Deprecated @@ -1540,63 +1567,65 @@ Unsupported Operating Systems ----------------------------- -* Windows XP - Per :PEP:`11`, Microsoft support of Windows XP has ended. +Per :PEP:`11`, Microsoft support of Windows XP has ended. Deprecated Python modules, functions and methods ------------------------------------------------ -* The :mod:`formatter` module has now graduated to full deprecation and is still - slated for removal in Python 3.6. +The :mod:`formatter` module has now graduated to full deprecation and is still +slated for removal in Python 3.6. -* :func:`~asyncio.async` was deprecated in favour of - :func:`~asyncio.ensure_future`. +The :func:`asyncio.async` function is deprecated in favor of +:func:`~asyncio.ensure_future`. -* :mod:`smtpd` has in the past always decoded the DATA portion of email - messages using the ``utf-8`` codec. This can now be controlled by the new - *decode_data* keyword to :class:`~smtpd.SMTPServer`. The default value is - ``True``, but this default is deprecated. Specify the *decode_data* keyword - with an appropriate value to avoid the deprecation warning. +The :mod:`smtpd` module has in the past always decoded the DATA portion of +email messages using the ``utf-8`` codec. This can now be controlled by the +new *decode_data* keyword to :class:`~smtpd.SMTPServer`. The default value is +``True``, but this default is deprecated. Specify the *decode_data* keyword +with an appropriate value to avoid the deprecation warning. -* Directly assigning values to the :attr:`~http.cookies.Morsel.key`, - :attr:`~http.cookies.Morsel.value` and - :attr:`~http.cookies.Morsel.coded_value` of :class:`~http.cookies.Morsel` - objects is deprecated. Use the :func:`~http.cookies.Morsel.set` method - instead. In addition, the undocumented *LegalChars* parameter of - :func:`~http.cookies.Morsel.set` is deprecated, and is now ignored. +Directly assigning values to the :attr:`~http.cookies.Morsel.key`, +:attr:`~http.cookies.Morsel.value` and +:attr:`~http.cookies.Morsel.coded_value` of :class:`~http.cookies.Morsel` +objects is deprecated. Use the :func:`~http.cookies.Morsel.set` method +instead. In addition, the undocumented *LegalChars* parameter of +:func:`~http.cookies.Morsel.set` is deprecated, and is now ignored. -* Passing a format string as keyword argument *format_string* to the - :meth:`~string.Formatter.format` method of the :class:`string.Formatter` - class has been deprecated. +Passing a format string as keyword argument *format_string* to the +:meth:`~string.Formatter.format` method of the :class:`string.Formatter` +class has been deprecated. -* :func:`platform.dist` and :func:`platform.linux_distribution` functions are - now deprecated and will be removed in Python 3.7. Linux distributions use - too many different ways of describing themselves, so the functionality is - left to a package. - (Contributed by Vajrasky Kok and Berker Peksag in :issue:`1322`.) +The :func:`platform.dist` and :func:`platform.linux_distribution` functions +are now deprecated and will be removed in Python 3.7. Linux distributions use +too many different ways of describing themselves, so the functionality is +left to a package. +(Contributed by Vajrasky Kok and Berker Peksag in :issue:`1322`.) -* The previously undocumented ``from_function`` and ``from_builtin`` methods of - :class:`inspect.Signature` are deprecated. Use new - :meth:`inspect.Signature.from_callable` instead. (Contributed by Yury - Selivanov in :issue:`24248`.) +The previously undocumented ``from_function`` and ``from_builtin`` methods of +:class:`inspect.Signature` are deprecated. Use new +:meth:`inspect.Signature.from_callable` instead. (Contributed by Yury +Selivanov in :issue:`24248`.) -* :func:`inspect.getargspec` is deprecated and scheduled to be removed in - Python 3.6. (See :issue:`20438` for details.) +The :func:`inspect.getargspec` function is deprecated and scheduled to be +removed in Python 3.6. (See :issue:`20438` for details.) -* :func:`~inspect.getfullargspec`, :func:`~inspect.getargvalues`, - :func:`~inspect.getcallargs`, :func:`~inspect.getargvalues`, - :func:`~inspect.formatargspec`, and :func:`~inspect.formatargvalues` are - deprecated in favor of :func:`inspect.signature` API. (See :issue:`20438` - for details.) +The :mod:`inspect` :func:`~inspect.getfullargspec`, +:func:`~inspect.getargvalues`, :func:`~inspect.getcallargs`, +:func:`~inspect.getargvalues`, :func:`~inspect.formatargspec`, and +:func:`~inspect.formatargvalues` functions are deprecated in favor of +:func:`inspect.signature` API. +(Contributed by Yury Selivanov in :issue:`20438`.) -* Use of ``re.LOCALE`` flag with str patterns or ``re.ASCII`` is now - deprecated. (Contributed by Serhiy Storchaka in :issue:`22407`.) +Use of ``re.LOCALE`` flag with str patterns or ``re.ASCII`` is now +deprecated. (Contributed by Serhiy Storchaka in :issue:`22407`.) +.. XXX: -Deprecated functions and types of the C API -------------------------------------------- + Deprecated functions and types of the C API + ------------------------------------------- -* None yet. + * None yet. Removed @@ -1640,9 +1669,10 @@ error-prone and has been removed in Python 3.5. See :issue:`13936` for full details. -* :meth:`ssl.SSLSocket.send()` now raises either :exc:`ssl.SSLWantReadError` - or :exc:`ssl.SSLWantWriteError` on a non-blocking socket if the operation - would block. Previously, it would return 0. See :issue:`20951`. +* The :meth:`ssl.SSLSocket.send()` method now raises either + :exc:`ssl.SSLWantReadError` or :exc:`ssl.SSLWantWriteError` + on a non-blocking socket if the operation would block. Previously, + it would return ``0``. See :issue:`20951`. * The ``__name__`` attribute of generator is now set from the function name, instead of being set from the code name. Use ``gen.gi_code.co_name`` to @@ -1681,12 +1711,12 @@ simply define :meth:`~importlib.machinery.Loader.create_module` to return ``None`` (:issue:`23014`). -* :func:`re.split` always ignored empty pattern matches, so the ``'x*'`` - pattern worked the same as ``'x+'``, and the ``'\b'`` pattern never worked. - Now :func:`re.split` raises a warning if the pattern could match +* The :func:`re.split` function always ignored empty pattern matches, so the + ``"x*"`` pattern worked the same as ``"x+"``, and the ``"\b"`` pattern never + worked. Now :func:`re.split` raises a warning if the pattern could match an empty string. For compatibility use patterns that never match an empty - string (e.g. ``'x+'`` instead of ``'x*'``). Patterns that could only match - an empty string (such as ``'\b'``) now raise an error. + string (e.g. ``"x+"`` instead of ``"x*"``). Patterns that could only match + an empty string (such as ``"\b"``) now raise an error. * The :class:`~http.cookies.Morsel` dict-like interface has been made self consistent: morsel comparison now takes the :attr:`~http.cookies.Morsel.key` @@ -1736,7 +1766,6 @@ * The undocumented :c:member:`~PyMemoryViewObject.format` member of the (non-public) :c:type:`PyMemoryViewObject` structure has been removed. - All extensions relying on the relevant parts in ``memoryobject.h`` must be rebuilt. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 11 00:27:56 2015 From: python-checkins at python.org (yury.selivanov) Date: Thu, 10 Sep 2015 22:27:56 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogd2hhdHNuZXcvMy41?= =?utf-8?q?=3A_Describe_changes_in_issue_=2322980?= Message-ID: <20150910222756.114948.15911@psf.io> https://hg.python.org/cpython/rev/1744d65705d0 changeset: 97878:1744d65705d0 branch: 3.5 parent: 97876:63a44a5fa5f7 user: Yury Selivanov date: Thu Sep 10 18:26:44 2015 -0400 summary: whatsnew/3.5: Describe changes in issue #22980 Initial patch by Larry Hastings. files: Doc/whatsnew/3.5.rst | 45 +++++++++++++++++++++++++++++++- 1 files changed, 44 insertions(+), 1 deletions(-) diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -1542,7 +1542,7 @@ Build and C API Changes ======================= -New ``calloc`` functions: +New ``calloc`` functions were added: * :c:func:`PyMem_RawCalloc` * :c:func:`PyMem_Calloc` @@ -1552,6 +1552,49 @@ Windows builds now require Microsoft Visual C++ 14.0, which is available as part of `Visual Studio 2015 `_. +Extension modules now include platform information tag in their filename on +some platforms (the tag is optional, and CPython will import extensions without +it; although if the tag is present and mismatched, the extension won't be +loaded): + +* On Linux, extension module filenames end with + ``.cpython-m--.pyd``: + + * ```` is the major number of the Python version; + for Python 3.5 this is ``3``. + + * ```` is the minor number of the Python version; + for Python 3.5 this is ``5``. + + * ```` is the hardware architecture the extension module + was built to run on. It's most commonly either ``i386`` for 32-bit Intel + platforms or ``x86_64`` for 64-bit Intel (and AMD) platforms. + + * ```` is always ``linux-gnu``, except for extensions built to + talk to the 32-bit ABI on 64-bit platforms, in which case it is + ``linux-gnu32`` (and ```` will be ``x86_64``). + +* On Windows, extension module filenames end with + ``..cp-.pyd``: + + * ```` is the major number of the Python version; + for Python 3.5 this is ``3``. + + * ```` is the minor number of the Python version; + for Python 3.5 this is ``5``. + + * ```` is the platform the extension module was built for, + either ``win32`` for Win32, ``win_amd64`` for Win64, ``win_ia64`` for + Windows Itanium 64, and ``win_arm`` for Windows on ARM. + + * If built in debug mode, ```` will be ``_d``, + otherwise it will be blank. + +* On OS X platforms, extension module filenames now end with ``-darwin.so``. + +* On all other platforms, extension module filenames are the same as they were + with Python 3.4. + Deprecated ========== -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 11 00:27:56 2015 From: python-checkins at python.org (yury.selivanov) Date: Thu, 10 Sep 2015 22:27:56 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogTWVyZ2UgMy41IChpc3N1ZSAjMjI5ODAsIHdoYXRzbmV3LzMuNSk=?= Message-ID: <20150910222756.15724.69347@psf.io> https://hg.python.org/cpython/rev/cfbcb3a6a848 changeset: 97879:cfbcb3a6a848 parent: 97877:6d18eb9a13ee parent: 97878:1744d65705d0 user: Yury Selivanov date: Thu Sep 10 18:27:17 2015 -0400 summary: Merge 3.5 (issue #22980, whatsnew/3.5) files: Doc/whatsnew/3.5.rst | 45 +++++++++++++++++++++++++++++++- 1 files changed, 44 insertions(+), 1 deletions(-) diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -1542,7 +1542,7 @@ Build and C API Changes ======================= -New ``calloc`` functions: +New ``calloc`` functions were added: * :c:func:`PyMem_RawCalloc` * :c:func:`PyMem_Calloc` @@ -1552,6 +1552,49 @@ Windows builds now require Microsoft Visual C++ 14.0, which is available as part of `Visual Studio 2015 `_. +Extension modules now include platform information tag in their filename on +some platforms (the tag is optional, and CPython will import extensions without +it; although if the tag is present and mismatched, the extension won't be +loaded): + +* On Linux, extension module filenames end with + ``.cpython-m--.pyd``: + + * ```` is the major number of the Python version; + for Python 3.5 this is ``3``. + + * ```` is the minor number of the Python version; + for Python 3.5 this is ``5``. + + * ```` is the hardware architecture the extension module + was built to run on. It's most commonly either ``i386`` for 32-bit Intel + platforms or ``x86_64`` for 64-bit Intel (and AMD) platforms. + + * ```` is always ``linux-gnu``, except for extensions built to + talk to the 32-bit ABI on 64-bit platforms, in which case it is + ``linux-gnu32`` (and ```` will be ``x86_64``). + +* On Windows, extension module filenames end with + ``..cp-.pyd``: + + * ```` is the major number of the Python version; + for Python 3.5 this is ``3``. + + * ```` is the minor number of the Python version; + for Python 3.5 this is ``5``. + + * ```` is the platform the extension module was built for, + either ``win32`` for Win32, ``win_amd64`` for Win64, ``win_ia64`` for + Windows Itanium 64, and ``win_arm`` for Windows on ARM. + + * If built in debug mode, ```` will be ``_d``, + otherwise it will be blank. + +* On OS X platforms, extension module filenames now end with ``-darwin.so``. + +* On all other platforms, extension module filenames are the same as they were + with Python 3.4. + Deprecated ========== -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 11 00:32:04 2015 From: python-checkins at python.org (yury.selivanov) Date: Thu, 10 Sep 2015 22:32:04 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogTWVyZ2UgMy41?= Message-ID: <20150910223204.12006.66897@psf.io> https://hg.python.org/cpython/rev/247945814be1 changeset: 97881:247945814be1 parent: 97879:cfbcb3a6a848 parent: 97880:da84aedc5fdb user: Yury Selivanov date: Thu Sep 10 18:32:00 2015 -0400 summary: Merge 3.5 files: Doc/whatsnew/3.5.rst | 8 -------- 1 files changed, 0 insertions(+), 8 deletions(-) diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -929,14 +929,6 @@ (Contributed by Daniel Shahaf in :issue:`16808`.) -io --- - -The :class:`~io.FileIO` class has been implemented in Python which makes -the C implementation of the :mod:`io` module entirely optional. -(Contributed by Serhiy Storchaka in :issue:`21859`.) - - ipaddress --------- -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 11 00:32:05 2015 From: python-checkins at python.org (yury.selivanov) Date: Thu, 10 Sep 2015 22:32:05 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogd2hhdHNuZXcvMy41?= =?utf-8?q?=3A_Don=27t_mention_pyio=2EFileIO?= Message-ID: <20150910223204.68863.87347@psf.io> https://hg.python.org/cpython/rev/da84aedc5fdb changeset: 97880:da84aedc5fdb branch: 3.5 parent: 97878:1744d65705d0 user: Yury Selivanov date: Thu Sep 10 18:31:49 2015 -0400 summary: whatsnew/3.5: Don't mention pyio.FileIO files: Doc/whatsnew/3.5.rst | 8 -------- 1 files changed, 0 insertions(+), 8 deletions(-) diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -929,14 +929,6 @@ (Contributed by Daniel Shahaf in :issue:`16808`.) -io --- - -The :class:`~io.FileIO` class has been implemented in Python which makes -the C implementation of the :mod:`io` module entirely optional. -(Contributed by Serhiy Storchaka in :issue:`21859`.) - - ipaddress --------- -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 11 01:00:00 2015 From: python-checkins at python.org (yury.selivanov) Date: Thu, 10 Sep 2015 23:00:00 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogd2hhdHNuZXcvMy41?= =?utf-8?q?=3A_Clarify_types=2Ecoroutine_=26_types=2ECoroutineType?= Message-ID: <20150910230000.17965.75619@psf.io> https://hg.python.org/cpython/rev/8750ef16c75d changeset: 97882:8750ef16c75d branch: 3.5 parent: 97880:da84aedc5fdb user: Yury Selivanov date: Thu Sep 10 18:59:42 2015 -0400 summary: whatsnew/3.5: Clarify types.coroutine & types.CoroutineType files: Doc/whatsnew/3.5.rst | 12 ++++++++---- 1 files changed, 8 insertions(+), 4 deletions(-) diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -1380,11 +1380,15 @@ types ----- -New :func:`~types.coroutine` function. (Contributed by Yury Selivanov -in :issue:`24017`.) +A new :func:`~types.coroutine` function to transform +:term:`generator ` and +:class:`generator-like ` objects into +:term:`awaitables `. +(Contributed by Yury Selivanov in :issue:`24017`.) -New :class:`~types.CoroutineType`. (Contributed by Yury Selivanov -in :issue:`24400`.) +A new :class:`~types.CoroutineType` is the type of :term:`coroutine` objects, +produced by calling a function defined with an :keyword:`async def` statement. +(Contributed by Yury Selivanov in :issue:`24400`.) urllib -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 11 01:00:03 2015 From: python-checkins at python.org (yury.selivanov) Date: Thu, 10 Sep 2015 23:00:03 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogTWVyZ2UgMy41?= Message-ID: <20150910230000.27701.91073@psf.io> https://hg.python.org/cpython/rev/c4b889d0bf93 changeset: 97883:c4b889d0bf93 parent: 97881:247945814be1 parent: 97882:8750ef16c75d user: Yury Selivanov date: Thu Sep 10 18:59:52 2015 -0400 summary: Merge 3.5 files: Doc/whatsnew/3.5.rst | 12 ++++++++---- 1 files changed, 8 insertions(+), 4 deletions(-) diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -1380,11 +1380,15 @@ types ----- -New :func:`~types.coroutine` function. (Contributed by Yury Selivanov -in :issue:`24017`.) +A new :func:`~types.coroutine` function to transform +:term:`generator ` and +:class:`generator-like ` objects into +:term:`awaitables `. +(Contributed by Yury Selivanov in :issue:`24017`.) -New :class:`~types.CoroutineType`. (Contributed by Yury Selivanov -in :issue:`24400`.) +A new :class:`~types.CoroutineType` is the type of :term:`coroutine` objects, +produced by calling a function defined with an :keyword:`async def` statement. +(Contributed by Yury Selivanov in :issue:`24400`.) urllib -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 11 01:02:55 2015 From: python-checkins at python.org (yury.selivanov) Date: Thu, 10 Sep 2015 23:02:55 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogd2hhdHNuZXcvMy41?= =?utf-8?q?=3A_Fix_refs_in_the_importlib_section?= Message-ID: <20150910230255.14885.20490@psf.io> https://hg.python.org/cpython/rev/4b8033347e18 changeset: 97884:4b8033347e18 branch: 3.5 parent: 97882:8750ef16c75d user: Yury Selivanov date: Thu Sep 10 19:02:24 2015 -0400 summary: whatsnew/3.5: Fix refs in the importlib section files: Doc/whatsnew/3.5.rst | 21 +++++++++++---------- 1 files changed, 11 insertions(+), 10 deletions(-) diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -877,19 +877,20 @@ importlib --------- -The :class:`importlib.util.LazyLoader` class allows for lazy loading of modules -in applications where startup time is important. (Contributed by Brett Cannon -in :issue:`17621`.) +The :class:`util.LazyLoader ` class allows for +lazy loading of modules in applications where startup time is important. +(Contributed by Brett Cannon in :issue:`17621`.) -The :func:`importlib.abc.InspectLoader.source_to_code` method is now a -static method. This makes it easier to initialize a module object with -code compiled from a string by running ``exec(code, module.__dict__)``. +The :func:`abc.InspectLoader.source_to_code ` +method is now a static method. This makes it easier to initialize a module +object with code compiled from a string by running +``exec(code, module.__dict__)``. (Contributed by Brett Cannon in :issue:`21156`.) -The new :func:`importlib.util.module_from_spec` function is now the preferred -way to create a new module. Compared to the :class:`types.ModuleType` class, -this new function will set the various import-controlled attributes based -on the passed-in spec object. +The new :func:`util.module_from_spec ` +function is now the preferred way to create a new module. Compared to the +:class:`types.ModuleType` class, this new function will set the various +import-controlled attributes based on the passed-in spec object. (Contributed by Brett Cannon in :issue:`20383`.) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 11 01:02:55 2015 From: python-checkins at python.org (yury.selivanov) Date: Thu, 10 Sep 2015 23:02:55 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogTWVyZ2UgMy41?= Message-ID: <20150910230255.15724.88455@psf.io> https://hg.python.org/cpython/rev/482ec940b413 changeset: 97885:482ec940b413 parent: 97883:c4b889d0bf93 parent: 97884:4b8033347e18 user: Yury Selivanov date: Thu Sep 10 19:02:34 2015 -0400 summary: Merge 3.5 files: Doc/whatsnew/3.5.rst | 21 +++++++++++---------- 1 files changed, 11 insertions(+), 10 deletions(-) diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -877,19 +877,20 @@ importlib --------- -The :class:`importlib.util.LazyLoader` class allows for lazy loading of modules -in applications where startup time is important. (Contributed by Brett Cannon -in :issue:`17621`.) +The :class:`util.LazyLoader ` class allows for +lazy loading of modules in applications where startup time is important. +(Contributed by Brett Cannon in :issue:`17621`.) -The :func:`importlib.abc.InspectLoader.source_to_code` method is now a -static method. This makes it easier to initialize a module object with -code compiled from a string by running ``exec(code, module.__dict__)``. +The :func:`abc.InspectLoader.source_to_code ` +method is now a static method. This makes it easier to initialize a module +object with code compiled from a string by running +``exec(code, module.__dict__)``. (Contributed by Brett Cannon in :issue:`21156`.) -The new :func:`importlib.util.module_from_spec` function is now the preferred -way to create a new module. Compared to the :class:`types.ModuleType` class, -this new function will set the various import-controlled attributes based -on the passed-in spec object. +The new :func:`util.module_from_spec ` +function is now the preferred way to create a new module. Compared to the +:class:`types.ModuleType` class, this new function will set the various +import-controlled attributes based on the passed-in spec object. (Contributed by Brett Cannon in :issue:`20383`.) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 11 03:27:20 2015 From: python-checkins at python.org (yury.selivanov) Date: Fri, 11 Sep 2015 01:27:20 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogTWVyZ2UgMy41?= Message-ID: <20150911012720.101478.38097@psf.io> https://hg.python.org/cpython/rev/3af318476d0f changeset: 97887:3af318476d0f parent: 97885:482ec940b413 parent: 97886:2034c31eb158 user: Yury Selivanov date: Thu Sep 10 21:27:04 2015 -0400 summary: Merge 3.5 files: Doc/library/types.rst | 8 +- Doc/whatsnew/3.5.rst | 190 ++++++++++++++++------------- 2 files changed, 106 insertions(+), 92 deletions(-) diff --git a/Doc/library/types.rst b/Doc/library/types.rst --- a/Doc/library/types.rst +++ b/Doc/library/types.rst @@ -86,14 +86,14 @@ .. data:: GeneratorType - The type of :term:`generator`-iterator objects, produced by calling a - generator function. + The type of :term:`generator`-iterator objects, created by + generator functions. .. data:: CoroutineType - The type of :term:`coroutine` objects, produced by calling a - function defined with an :keyword:`async def` statement. + The type of :term:`coroutine` objects, created by + :keyword:`async def` functions. .. versionadded:: 3.5 diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -178,7 +178,7 @@ >>> async def coro(): ... return 'spam' -Inside a coroutine function, a new :keyword:`await` expression can be used +Inside a coroutine function, the new :keyword:`await` expression can be used to suspend coroutine execution until the result is available. Any object can be *awaited*, as long as it implements the :term:`awaitable` protocol by defining the :meth:`__await__` method. @@ -245,7 +245,7 @@ Note that both :keyword:`async for` and :keyword:`async with` can only be used inside a coroutine function declared with :keyword:`async def`. -Coroutine functions are intended to be ran inside a compatible event loop, +Coroutine functions are intended to be run inside a compatible event loop, such as :class:`asyncio.Loop`. .. seealso:: @@ -627,7 +627,7 @@ The :class:`~argparse.ArgumentParser` class now allows to disable :ref:`abbreviated usage ` of long options by setting -:ref:`allow_abbrev` to ``False``. (Contributed by Jonathan Paugh, +:ref:`allow_abbrev` to ``False``. (Contributed by Jonathan Paugh, Steven Bethard, paul j3 and Daniel Eriksson in :issue:`14910`.) @@ -649,7 +649,7 @@ cmath ----- -A new function :func:`cmath.isclose` provides a way to test for approximate +A new function :func:`~cmath.isclose` provides a way to test for approximate equality. (Contributed by Chris Barker and Tal Einat in :issue:`24270`.) @@ -695,7 +695,7 @@ A new :class:`~collections.abc.Generator` abstract base class. (Contributed by Stefan Behnel in :issue:`24018`.) -A new :class:`~collections.abc.Coroutine`, +New :class:`~collections.abc.Coroutine`, :class:`~collections.abc.AsyncIterator`, and :class:`~collections.abc.AsyncIterable` abstract base classes. (Contributed by Yury Selivanov in :issue:`24184`.) @@ -721,8 +721,8 @@ ------------------ The :meth:`Executor.map ` method now accepts a -*chunksize* argument to allow batching of tasks in child processes and improve -performance of :meth:`~concurrent.futures.ProcessPoolExecutor`. +*chunksize* argument to allow batching of tasks to improve performance when +:meth:`~concurrent.futures.ProcessPoolExecutor` is used. (Contributed by Dan O'Reilly in :issue:`11271`.) @@ -739,28 +739,30 @@ curses ------ -The new :func:`~curses.update_lines_cols` function updates the variables -:data:`curses.LINES` and :data:`curses.COLS`. +The new :func:`~curses.update_lines_cols` function updates :envvar:`LINES` +and :envvar:`COLS` environment variables. This is useful for detecting +manual screen resize. (Contributed by Arnon Yaari in :issue:`4254`.) difflib ------- -The charset of the HTML document generated by +The charset of HTML documents generated by :meth:`HtmlDiff.make_file ` -can now be customized by using *charset* keyword-only parameter. The default -charset of HTML document changed from ``"ISO-8859-1"`` to ``"utf-8"``. +can now be customized by using a new *charset* keyword-only argument. +The default charset of HTML document changed from ``"ISO-8859-1"`` +to ``"utf-8"``. (Contributed by Berker Peksag in :issue:`2052`.) -It is now possible to compare lists of byte strings with the -:func:`~difflib.diff_bytes` function. This fixes a regression from Python 2. +The :func:`~difflib.diff_bytes` function can now compare lists of byte +strings. This fixes a regression from Python 2. (Contributed by Terry J. Reedy and Greg Ward in :issue:`17445`.) distutils --------- -The ``build`` and ``build_ext`` commands now accept a ``-j`` option to +Both ``build`` and ``build_ext`` commands now accept a ``-j`` option to enable parallel building of extension modules. (Contributed by Antoine Pitrou in :issue:`5309`.) @@ -792,17 +794,18 @@ :mailheader:`Content-Disposition` header. (Contributed by Abhilash Raj in :issue:`21083`.) -A new policy option :attr:`~email.policy.EmailPolicy.utf8` can be set -to ``True`` to encode email headers using the UTF-8 charset instead of using -encoded words. This allows ``Messages`` to be formatted according to +A new policy option :attr:`EmailPolicy.utf8 ` +can be set to ``True`` to encode email headers using the UTF-8 charset instead +of using encoded words. This allows ``Messages`` to be formatted according to :rfc:`6532` and used with an SMTP server that supports the :rfc:`6531` -``SMTPUTF8`` extension. (Contributed by R. David Murray in :issue:`24211`.) +``SMTPUTF8`` extension. (Contributed by R. David Murray in +:issue:`24211`.) faulthandler ------------ -The :func:`~faulthandler.enable`, :func:`~faulthandler.register`, +:func:`~faulthandler.enable`, :func:`~faulthandler.register`, :func:`~faulthandler.dump_traceback` and :func:`~faulthandler.dump_traceback_later` functions now accept file descriptors in addition to file-like objects. @@ -813,14 +816,14 @@ --------- Most of :func:`~functools.lru_cache` machinery is now implemented in C, making -it significantly faster. (Contributed by Matt Joiner, Alexey Kachayev, and +it significantly faster. (Contributed by Matt Joiner, Alexey Kachayev, and Serhiy Storchaka in :issue:`14373`.) glob ---- -The :func:`~glob.iglob` and :func:`~glob.glob` functions now support recursive +:func:`~glob.iglob` and :func:`~glob.glob` functions now support recursive search in subdirectories using the ``"**"`` pattern. (Contributed by Serhiy Storchaka in :issue:`13968`.) @@ -838,7 +841,7 @@ ---------------- Since idlelib implements the IDLE shell and editor and is not intended for -import by other programs, it gets improvements with every release. See +import by other programs, it gets improvements with every release. See :file:`Lib/idlelib/NEWS.txt` for a cumulative list of changes since 3.4.0, as well as changes made in future 3.5.x releases. This file is also available from the IDLE Help -> About Idle dialog. @@ -888,17 +891,17 @@ (Contributed by Brett Cannon in :issue:`21156`.) The new :func:`util.module_from_spec ` -function is now the preferred way to create a new module. Compared to the -:class:`types.ModuleType` class, this new function will set the various -import-controlled attributes based on the passed-in spec object. -(Contributed by Brett Cannon in :issue:`20383`.) +function is now the preferred way to create a new module. As opposed to +creating a :class:`types.ModuleType` instance directly, this new function +will set the various import-controlled attributes based on the passed-in +spec object. (Contributed by Brett Cannon in :issue:`20383`.) inspect ------- -The :class:`~inspect.Signature` and :class:`~inspect.Parameter` classes are now -picklable and hashable. (Contributed by Yury Selivanov in :issue:`20726` +Both :class:`~inspect.Signature` and :class:`~inspect.Parameter` classes are +now picklable and hashable. (Contributed by Yury Selivanov in :issue:`20726` and :issue:`20334`.) A new @@ -918,13 +921,13 @@ A set of new functions to inspect :term:`coroutine functions ` and -``coroutine objects`` has been added: +:term:`coroutine objects ` has been added: :func:`~inspect.iscoroutine`, :func:`~inspect.iscoroutinefunction`, :func:`~inspect.isawaitable`, :func:`~inspect.getcoroutinelocals`, and :func:`~inspect.getcoroutinestate`. (Contributed by Yury Selivanov in :issue:`24017` and :issue:`24400`.) -The :func:`~inspect.stack`, :func:`~inspect.trace`, +:func:`~inspect.stack`, :func:`~inspect.trace`, :func:`~inspect.getouterframes`, and :func:`~inspect.getinnerframes` functions now return a list of named tuples. (Contributed by Daniel Shahaf in :issue:`16808`.) @@ -933,7 +936,7 @@ ipaddress --------- -The :class:`~ipaddress.IPv4Network` and :class:`~ipaddress.IPv6Network` classes +Both :class:`~ipaddress.IPv4Network` and :class:`~ipaddress.IPv6Network` classes now accept an ``(address, netmask)`` tuple argument, so as to easily construct network objects from existing addresses. (Contributed by Peter Moody and Antoine Pitrou in :issue:`16531`.) @@ -965,11 +968,11 @@ All logging methods (:class:`~logging.Logger` :meth:`~logging.Logger.log`, :meth:`~logging.Logger.exception`, :meth:`~logging.Logger.critical`, :meth:`~logging.Logger.debug`, etc.), now accept exception instances -in ``exc_info`` argument, in addition to boolean values and exception +as an ``exc_info`` argument, in addition to boolean values and exception tuples. (Contributed by Yury Selivanov in :issue:`20537`.) -The :class:`handlers.HTTPHandler ` classes now -accepts an optional :class:`ssl.SSLContext` instance to configure the SSL +The :class:`handlers.HTTPHandler ` class now +accepts an optional :class:`ssl.SSLContext` instance to configure SSL settings used in an HTTP connection. (Contributed by Alex Gaynor in :issue:`22788`.) @@ -983,7 +986,7 @@ ---- The :meth:`LZMADecompressor.decompress ` -method now accepts an optional *max_length* argument to limit the maximum +method now accepts an optional *max_length* argument to limit the maximum size of decompressed data. (Contributed by Martin Panter in :issue:`15955`.) @@ -1005,7 +1008,7 @@ operator -------- -The :mod:`operator` :func:`~operator.attrgetter`, :func:`~operator.itemgetter`, +:func:`~operator.attrgetter`, :func:`~operator.itemgetter`, and :func:`~operator.methodcaller` objects now support pickling. (Contributed by Josh Rosenberg and Serhiy Storchaka in :issue:`22955`.) @@ -1032,15 +1035,15 @@ exhaustion. (Contributed by Victor Stinner in :issue:`22181`.) New :func:`~os.get_blocking` and :func:`~os.set_blocking` functions allow to -get and set the file descriptor blocking mode (:data:`~os.O_NONBLOCK`.) +get and set a file descriptor blocking mode (:data:`~os.O_NONBLOCK`.) (Contributed by Victor Stinner in :issue:`22054`.) The :func:`~os.truncate` and :func:`~os.ftruncate` functions are now supported on Windows. (Contributed by Steve Dower in :issue:`23668`.) -There is a new :func:`~os.path.commonpath` function returning the longest +There is a new :func:`os.path.commonpath` function returning the longest common sub-path of each passed pathname. Unlike the -:func:`~os.path.commonprefix` function, it always returns a valid +:func:`os.path.commonprefix` function, it always returns a valid path. (Contributed by Rafik Draoui and Serhiy Storchaka in :issue:`10395`.) @@ -1048,8 +1051,8 @@ ------- The new :meth:`Path.samefile ` method can be used -to check if the passed :class:`~pathlib.Path` object or a :class:`str` path, -point to the same file. +to check whether the path points to the same file as other path, which can be +either an another :class:`~pathlib.Path` object, or a string. (Contributed by Vajrasky Kok and Antoine Pitrou in :issue:`19775`.) The :meth:`Path.mkdir ` method how accepts a new optional @@ -1070,16 +1073,16 @@ ------ Nested objects, such as unbound methods or nested classes, can now be pickled -using :ref:`pickle protocols ` older than protocol version 4, -which already supported these cases. (Contributed by Serhiy Storchaka in -:issue:`23611`.) +using :ref:`pickle protocols ` older than protocol version 4. +Protocol version 4 already supports these cases. (Contributed by Serhiy +Storchaka in :issue:`23611`.) poplib ------ -A new command :meth:`POP3.utf8 ` enables :rfc:`6856` -(internationalized email) support, if the POP server supports it. +A new :meth:`POP3.utf8 ` command enables :rfc:`6856` +(Internationalized Email) support, if a POP server supports it. (Contributed by Milan OberKirch in :issue:`21804`.) @@ -1090,15 +1093,15 @@ 100. (Contributed by Serhiy Storchaka in :issue:`22437`.) The :func:`~re.sub` and :func:`~re.subn` functions now replace unmatched -groups with empty strings instead of rising an exception. +groups with empty strings instead of raising an exception. (Contributed by Serhiy Storchaka in :issue:`1519638`.) readline -------- -The new :func:`~readline.append_history_file` function can be used to append -the specified number of trailing elements in history to a given file. +A new :func:`~readline.append_history_file` function can be used to append +the specified number of trailing elements in history to the given file. (Contributed by Bruno Cauet in :issue:`22940`.) @@ -1111,7 +1114,7 @@ when moving. (Contributed by Claudiu Popa in :issue:`19840`.) -The :func:`~shutil.make_archive` function now supports *xztar* format. +The :func:`~shutil.make_archive` function now supports the *xztar* format. (Contributed by Serhiy Storchaka in :issue:`5411`.) @@ -1121,7 +1124,7 @@ On Windows, the :func:`~signal.set_wakeup_fd` function now also supports socket handles. (Contributed by Victor Stinner in :issue:`22018`.) -Various ``SIG*`` constants in :mod:`signal` module have been converted into +Various ``SIG*`` constants in the :mod:`signal` module have been converted into :mod:`Enums `. This allows meaningful names to be printed during debugging, instead of integer "magic numbers". (Contributed by Giampaolo Rodola' in :issue:`21076`.) @@ -1133,7 +1136,8 @@ Both :class:`~smtpd.SMTPServer` and :class:`~smtpd.SMTPChannel` classes now accept a *decode_data* keyword argument to determine if the ``DATA`` portion of the SMTP transaction is decoded using the ``"utf-8"`` codec or is instead -provided to :meth:`SMTPServer.process_message ` +provided to the +:meth:`SMTPServer.process_message ` method as a byte string. The default is ``True`` for backward compatibility reasons, but will change to ``False`` in Python 3.6. If *decode_data* is set to ``False``, the :meth:`~smtpd.SMTPServer.process_message` method must @@ -1167,9 +1171,9 @@ implement custom authentication mechanisms. (Contributed by Milan Oberkirch in :issue:`15014`.) -Additional debuglevel (2) shows timestamps for debug messages in -:class:`smtplib.SMTP`. (Contributed by Gavin Chappell and Maciej Szulik in -:issue:`16914`.) +The :meth:`SMTP.set_debuglevel ` method now +accepts an additional debuglevel (2), which enables timestamps in debug +messages. (Contributed by Gavin Chappell and Maciej Szulik in :issue:`16914`.) Both :meth:`SMTP.sendmail ` and :meth:`SMTP.send_message ` methods now @@ -1180,8 +1184,8 @@ sndhdr ------ -The :func:`~sndhdr.what` and :func:`~sndhdr.whathdr` functions now return -a :func:`~collections.namedtuple`. (Contributed by Claudiu Popa in +:func:`~sndhdr.what` and :func:`~sndhdr.whathdr` functions now return +a :func:`~collections.namedtuple`. (Contributed by Claudiu Popa in :issue:`18615`.) @@ -1194,14 +1198,14 @@ (Contributed by Geert Jansen in :issue:`21965`.) The new :class:`~ssl.SSLObject` class has been added to provide SSL protocol -support for cases when the network IO capabilities of :class:`~ssl.SSLSocket` +support for cases when the network I/O capabilities of :class:`~ssl.SSLSocket` are not necessary or suboptimal. :class:`~ssl.SSLObject` represents -an SSL protocol instance, but does not implement any network IO methods, and +an SSL protocol instance, but does not implement any network I/O methods, and instead provides a memory buffer interface. The new :class:`~ssl.MemoryBIO` class can be used to pass data between Python and an SSL protocol instance. The memory BIO SSL support is primarily intended to be used in frameworks -implementing asynchronous IO for which :class:`~ssl.SSLObject` IO readiness +implementing asynchronous I/O for which :class:`~ssl.SSLSocket`'s readiness model ("select/poll") is inefficient. A new :meth:`SSLContext.wrap_bio ` method can be used @@ -1213,12 +1217,12 @@ (Contributed by Benjamin Peterson in :issue:`20188`.) -Where OpenSSL support is present, :mod:`ssl` module now implements * -Application-Layer Protocol Negotiation* TLS extension as described +Where OpenSSL support is present, :mod:`ssl` module now implements +*Application-Layer Protocol Negotiation* TLS extension as described in :rfc:`7301`. The new :meth:`SSLContext.set_alpn_protocols ` -can be used to specify which protocols the socket should advertise during +can be used to specify which protocols a socket should advertise during the TLS handshake. The new @@ -1271,7 +1275,7 @@ (Contributed by Victor Stinner in :issue:`22043`.) A new :meth:`socket.sendfile ` method allows to -send a file over a socket by using high-performance :func:`os.sendfile` +send a file over a socket by using the high-performance :func:`os.sendfile` function on UNIX resulting in uploads being from 2 to 3 times faster than when using plain :meth:`socket.send `. (Contributed by Giampaolo Rodola' in :issue:`17552`.) @@ -1285,9 +1289,12 @@ subprocess ---------- -The new :func:`~subprocess.run` function has been added and is the recommended -approach to invoking subprocesses. It runs the specified command and -and returns a :class:`~subprocess.CompletedProcess` object. +The new :func:`~subprocess.run` function has been added. +It runs the specified command and and returns a +:class:`~subprocess.CompletedProcess` object, which describes a finished +process. The new API is more consistent and is the recommended approach +to invoking subprocesses in Python code that does not need to maintain +compatibility with earlier Python versions. (Contributed by Thomas Kluyver in :issue:`23342`.) @@ -1295,13 +1302,13 @@ --- A new :func:`~sys.set_coroutine_wrapper` function allows setting a global -hook that will be called whenever a :ref:`coro object ` -is created. Essentially, it works like a global coroutine decorator. A -corresponding :func:`~sys.get_coroutine_wrapper` can be used to obtain -a currently set wrapper. Both functions are provisional, and are intended -for debugging purposes only. (Contributed by Yury Selivanov in :issue:`24017`.) +hook that will be called whenever a :term:`coroutine object ` +is created by an :keyword:`async def` function. A corresponding +:func:`~sys.get_coroutine_wrapper` can be used to obtain a currently set +wrapper. Both functions are provisional, and are intended for debugging +purposes only. (Contributed by Yury Selivanov in :issue:`24017`.) -There is a new :func:`~sys.is_finalizing` function to check if the Python +A new :func:`~sys.is_finalizing` function can be used to check if the Python interpreter is :term:`shutting down `. (Contributed by Antoine Pitrou in :issue:`22696`.) @@ -1320,7 +1327,7 @@ The *mode* argument of the :func:`~tarfile.open` function now accepts ``"x"`` to request exclusive creation. (Contributed by Berker Peksag in :issue:`21717`.) -The :meth:`TarFile.extractall ` and +:meth:`TarFile.extractall ` and :meth:`TarFile.extract ` methods now take a keyword argument *numeric_only*. If set to ``True``, the extracted files and directories will be owned by the numeric ``uid`` and ``gid`` from the tarfile. @@ -1332,7 +1339,7 @@ threading --------- -The :meth:`Lock.acquire ` and +Both :meth:`Lock.acquire ` and :meth:`RLock.acquire ` methods now use a monotonic clock for timeout management. (Contributed by Victor Stinner in :issue:`22043`.) @@ -1348,9 +1355,9 @@ timeit ------ -New command line option ``-u`` or ``--unit=U`` to specify a time -unit for the timer output. Supported options are ``usec``, ``msec``, or ``sec``. -(Contributed by Julian Gindi in :issue:`18983`.) +New command line option ``-u`` or ``--unit=U`` can be used to specify the time +unit for the timer output. Supported options are ``usec``, ``msec``, +or ``sec``. (Contributed by Julian Gindi in :issue:`18983`.) tkinter @@ -1373,7 +1380,7 @@ :class:`~traceback.StackSummary`, and :class:`traceback.FrameSummary`. (Contributed by Robert Collins in :issue:`17911`.) -The :func:`~traceback.print_tb` and :func:`~traceback.print_stack` functions +Both :func:`~traceback.print_tb` and :func:`~traceback.print_stack` functions now support negative values for the *limit* argument. (Contributed by Dmitry Kazakov in :issue:`22619`.) @@ -1387,8 +1394,8 @@ :term:`awaitables `. (Contributed by Yury Selivanov in :issue:`24017`.) -A new :class:`~types.CoroutineType` is the type of :term:`coroutine` objects, -produced by calling a function defined with an :keyword:`async def` statement. +A new :class:`~types.CoroutineType` is the type of :term:`coroutine` objects +created by :keyword:`async def` functions. (Contributed by Yury Selivanov in :issue:`24400`.) @@ -1478,22 +1485,24 @@ Optimizations ============= -The :func:`os.walk` function has been sped up by 3-5 times on POSIX systems, -and by 7-20 times on Windows. This was done using the new :func:`os.scandir` +The :func:`os.walk` function has been sped up by 3 to 5 times on POSIX systems, +and by 7 to 20 times on Windows. This was done using the new :func:`os.scandir` function, which exposes file information from the underlying ``readdir`` or -``FindFirstFile``/``FindNextFile`` system calls. (Contributed by +``FindFirstFile``/``FindNextFile`` system calls. (Contributed by Ben Hoyt with help from Victor Stinner in :issue:`23605`.) Construction of ``bytes(int)`` (filled by zero bytes) is faster and uses less memory for large objects. ``calloc()`` is used instead of ``malloc()`` to allocate memory for these objects. +(Contributed by Victor Stinner in :issue:`21233`.) Some operations on :mod:`ipaddress` :class:`~ipaddress.IPv4Network` and :class:`~ipaddress.IPv6Network` have been massively sped up, such as :meth:`~ipaddress.IPv4Network.subnets`, :meth:`~ipaddress.IPv4Network.supernet`, :func:`~ipaddress.summarize_address_range`, :func:`~ipaddress.collapse_addresses`. The speed up can range from 3 to 15 times. -(See :issue:`21486`, :issue:`21487`, :issue:`20826`, :issue:`23266`.) +(Contributed by Antoine Pitrou, Michel Albert, and Markus in +:issue:`21486`, :issue:`21487`, :issue:`20826`, :issue:`23266`.) Pickling of :mod:`ipaddress` objects was optimized to produce significantly smaller output. (Contributed by Serhiy Storchaka in :issue:`23133`.) @@ -1572,7 +1581,7 @@ ``linux-gnu32`` (and ```` will be ``x86_64``). * On Windows, extension module filenames end with - ``..cp-.pyd``: + ``.cp-.pyd``: * ```` is the major number of the Python version; for Python 3.5 this is ``3``. @@ -1607,7 +1616,8 @@ Unsupported Operating Systems ----------------------------- -Per :PEP:`11`, Microsoft support of Windows XP has ended. +Windows XP is no longer supported by Microsoft, thus, per :PEP:`11`, CPython +3.5 is no longer officially supported on this OS. Deprecated Python modules, functions and methods @@ -1782,6 +1792,10 @@ * The :mod:`socket` module now exports the CAN_RAW_FD_FRAMES constant on linux 3.6 and greater. +* The :func:`~ssl.cert_time_to_seconds` function now interprets the input time + as UTC and not as local time, per :rfc:`5280`. Additionally, the return + value is always an :class:`int`. (Contributed by Akira Li in :issue:`19940`.) + * The ``pygettext.py`` Tool now uses the standard +NNNN format for timezones in the POT-Creation-Date header. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 11 03:27:20 2015 From: python-checkins at python.org (yury.selivanov) Date: Fri, 11 Sep 2015 01:27:20 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogd2hhdHNuZXcvMy41?= =?utf-8?q?=3A_Another_editing_pass?= Message-ID: <20150911012719.114658.67563@psf.io> https://hg.python.org/cpython/rev/2034c31eb158 changeset: 97886:2034c31eb158 branch: 3.5 parent: 97884:4b8033347e18 user: Yury Selivanov date: Thu Sep 10 21:26:54 2015 -0400 summary: whatsnew/3.5: Another editing pass Patch by Elvis Pranskevichus. files: Doc/library/types.rst | 8 +- Doc/whatsnew/3.5.rst | 190 ++++++++++++++++------------- 2 files changed, 106 insertions(+), 92 deletions(-) diff --git a/Doc/library/types.rst b/Doc/library/types.rst --- a/Doc/library/types.rst +++ b/Doc/library/types.rst @@ -86,14 +86,14 @@ .. data:: GeneratorType - The type of :term:`generator`-iterator objects, produced by calling a - generator function. + The type of :term:`generator`-iterator objects, created by + generator functions. .. data:: CoroutineType - The type of :term:`coroutine` objects, produced by calling a - function defined with an :keyword:`async def` statement. + The type of :term:`coroutine` objects, created by + :keyword:`async def` functions. .. versionadded:: 3.5 diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -178,7 +178,7 @@ >>> async def coro(): ... return 'spam' -Inside a coroutine function, a new :keyword:`await` expression can be used +Inside a coroutine function, the new :keyword:`await` expression can be used to suspend coroutine execution until the result is available. Any object can be *awaited*, as long as it implements the :term:`awaitable` protocol by defining the :meth:`__await__` method. @@ -245,7 +245,7 @@ Note that both :keyword:`async for` and :keyword:`async with` can only be used inside a coroutine function declared with :keyword:`async def`. -Coroutine functions are intended to be ran inside a compatible event loop, +Coroutine functions are intended to be run inside a compatible event loop, such as :class:`asyncio.Loop`. .. seealso:: @@ -627,7 +627,7 @@ The :class:`~argparse.ArgumentParser` class now allows to disable :ref:`abbreviated usage ` of long options by setting -:ref:`allow_abbrev` to ``False``. (Contributed by Jonathan Paugh, +:ref:`allow_abbrev` to ``False``. (Contributed by Jonathan Paugh, Steven Bethard, paul j3 and Daniel Eriksson in :issue:`14910`.) @@ -649,7 +649,7 @@ cmath ----- -A new function :func:`cmath.isclose` provides a way to test for approximate +A new function :func:`~cmath.isclose` provides a way to test for approximate equality. (Contributed by Chris Barker and Tal Einat in :issue:`24270`.) @@ -695,7 +695,7 @@ A new :class:`~collections.abc.Generator` abstract base class. (Contributed by Stefan Behnel in :issue:`24018`.) -A new :class:`~collections.abc.Coroutine`, +New :class:`~collections.abc.Coroutine`, :class:`~collections.abc.AsyncIterator`, and :class:`~collections.abc.AsyncIterable` abstract base classes. (Contributed by Yury Selivanov in :issue:`24184`.) @@ -721,8 +721,8 @@ ------------------ The :meth:`Executor.map ` method now accepts a -*chunksize* argument to allow batching of tasks in child processes and improve -performance of :meth:`~concurrent.futures.ProcessPoolExecutor`. +*chunksize* argument to allow batching of tasks to improve performance when +:meth:`~concurrent.futures.ProcessPoolExecutor` is used. (Contributed by Dan O'Reilly in :issue:`11271`.) @@ -739,28 +739,30 @@ curses ------ -The new :func:`~curses.update_lines_cols` function updates the variables -:data:`curses.LINES` and :data:`curses.COLS`. +The new :func:`~curses.update_lines_cols` function updates :envvar:`LINES` +and :envvar:`COLS` environment variables. This is useful for detecting +manual screen resize. (Contributed by Arnon Yaari in :issue:`4254`.) difflib ------- -The charset of the HTML document generated by +The charset of HTML documents generated by :meth:`HtmlDiff.make_file ` -can now be customized by using *charset* keyword-only parameter. The default -charset of HTML document changed from ``"ISO-8859-1"`` to ``"utf-8"``. +can now be customized by using a new *charset* keyword-only argument. +The default charset of HTML document changed from ``"ISO-8859-1"`` +to ``"utf-8"``. (Contributed by Berker Peksag in :issue:`2052`.) -It is now possible to compare lists of byte strings with the -:func:`~difflib.diff_bytes` function. This fixes a regression from Python 2. +The :func:`~difflib.diff_bytes` function can now compare lists of byte +strings. This fixes a regression from Python 2. (Contributed by Terry J. Reedy and Greg Ward in :issue:`17445`.) distutils --------- -The ``build`` and ``build_ext`` commands now accept a ``-j`` option to +Both ``build`` and ``build_ext`` commands now accept a ``-j`` option to enable parallel building of extension modules. (Contributed by Antoine Pitrou in :issue:`5309`.) @@ -792,17 +794,18 @@ :mailheader:`Content-Disposition` header. (Contributed by Abhilash Raj in :issue:`21083`.) -A new policy option :attr:`~email.policy.EmailPolicy.utf8` can be set -to ``True`` to encode email headers using the UTF-8 charset instead of using -encoded words. This allows ``Messages`` to be formatted according to +A new policy option :attr:`EmailPolicy.utf8 ` +can be set to ``True`` to encode email headers using the UTF-8 charset instead +of using encoded words. This allows ``Messages`` to be formatted according to :rfc:`6532` and used with an SMTP server that supports the :rfc:`6531` -``SMTPUTF8`` extension. (Contributed by R. David Murray in :issue:`24211`.) +``SMTPUTF8`` extension. (Contributed by R. David Murray in +:issue:`24211`.) faulthandler ------------ -The :func:`~faulthandler.enable`, :func:`~faulthandler.register`, +:func:`~faulthandler.enable`, :func:`~faulthandler.register`, :func:`~faulthandler.dump_traceback` and :func:`~faulthandler.dump_traceback_later` functions now accept file descriptors in addition to file-like objects. @@ -813,14 +816,14 @@ --------- Most of :func:`~functools.lru_cache` machinery is now implemented in C, making -it significantly faster. (Contributed by Matt Joiner, Alexey Kachayev, and +it significantly faster. (Contributed by Matt Joiner, Alexey Kachayev, and Serhiy Storchaka in :issue:`14373`.) glob ---- -The :func:`~glob.iglob` and :func:`~glob.glob` functions now support recursive +:func:`~glob.iglob` and :func:`~glob.glob` functions now support recursive search in subdirectories using the ``"**"`` pattern. (Contributed by Serhiy Storchaka in :issue:`13968`.) @@ -838,7 +841,7 @@ ---------------- Since idlelib implements the IDLE shell and editor and is not intended for -import by other programs, it gets improvements with every release. See +import by other programs, it gets improvements with every release. See :file:`Lib/idlelib/NEWS.txt` for a cumulative list of changes since 3.4.0, as well as changes made in future 3.5.x releases. This file is also available from the IDLE Help -> About Idle dialog. @@ -888,17 +891,17 @@ (Contributed by Brett Cannon in :issue:`21156`.) The new :func:`util.module_from_spec ` -function is now the preferred way to create a new module. Compared to the -:class:`types.ModuleType` class, this new function will set the various -import-controlled attributes based on the passed-in spec object. -(Contributed by Brett Cannon in :issue:`20383`.) +function is now the preferred way to create a new module. As opposed to +creating a :class:`types.ModuleType` instance directly, this new function +will set the various import-controlled attributes based on the passed-in +spec object. (Contributed by Brett Cannon in :issue:`20383`.) inspect ------- -The :class:`~inspect.Signature` and :class:`~inspect.Parameter` classes are now -picklable and hashable. (Contributed by Yury Selivanov in :issue:`20726` +Both :class:`~inspect.Signature` and :class:`~inspect.Parameter` classes are +now picklable and hashable. (Contributed by Yury Selivanov in :issue:`20726` and :issue:`20334`.) A new @@ -918,13 +921,13 @@ A set of new functions to inspect :term:`coroutine functions ` and -``coroutine objects`` has been added: +:term:`coroutine objects ` has been added: :func:`~inspect.iscoroutine`, :func:`~inspect.iscoroutinefunction`, :func:`~inspect.isawaitable`, :func:`~inspect.getcoroutinelocals`, and :func:`~inspect.getcoroutinestate`. (Contributed by Yury Selivanov in :issue:`24017` and :issue:`24400`.) -The :func:`~inspect.stack`, :func:`~inspect.trace`, +:func:`~inspect.stack`, :func:`~inspect.trace`, :func:`~inspect.getouterframes`, and :func:`~inspect.getinnerframes` functions now return a list of named tuples. (Contributed by Daniel Shahaf in :issue:`16808`.) @@ -933,7 +936,7 @@ ipaddress --------- -The :class:`~ipaddress.IPv4Network` and :class:`~ipaddress.IPv6Network` classes +Both :class:`~ipaddress.IPv4Network` and :class:`~ipaddress.IPv6Network` classes now accept an ``(address, netmask)`` tuple argument, so as to easily construct network objects from existing addresses. (Contributed by Peter Moody and Antoine Pitrou in :issue:`16531`.) @@ -965,11 +968,11 @@ All logging methods (:class:`~logging.Logger` :meth:`~logging.Logger.log`, :meth:`~logging.Logger.exception`, :meth:`~logging.Logger.critical`, :meth:`~logging.Logger.debug`, etc.), now accept exception instances -in ``exc_info`` argument, in addition to boolean values and exception +as an ``exc_info`` argument, in addition to boolean values and exception tuples. (Contributed by Yury Selivanov in :issue:`20537`.) -The :class:`handlers.HTTPHandler ` classes now -accepts an optional :class:`ssl.SSLContext` instance to configure the SSL +The :class:`handlers.HTTPHandler ` class now +accepts an optional :class:`ssl.SSLContext` instance to configure SSL settings used in an HTTP connection. (Contributed by Alex Gaynor in :issue:`22788`.) @@ -983,7 +986,7 @@ ---- The :meth:`LZMADecompressor.decompress ` -method now accepts an optional *max_length* argument to limit the maximum +method now accepts an optional *max_length* argument to limit the maximum size of decompressed data. (Contributed by Martin Panter in :issue:`15955`.) @@ -1005,7 +1008,7 @@ operator -------- -The :mod:`operator` :func:`~operator.attrgetter`, :func:`~operator.itemgetter`, +:func:`~operator.attrgetter`, :func:`~operator.itemgetter`, and :func:`~operator.methodcaller` objects now support pickling. (Contributed by Josh Rosenberg and Serhiy Storchaka in :issue:`22955`.) @@ -1032,15 +1035,15 @@ exhaustion. (Contributed by Victor Stinner in :issue:`22181`.) New :func:`~os.get_blocking` and :func:`~os.set_blocking` functions allow to -get and set the file descriptor blocking mode (:data:`~os.O_NONBLOCK`.) +get and set a file descriptor blocking mode (:data:`~os.O_NONBLOCK`.) (Contributed by Victor Stinner in :issue:`22054`.) The :func:`~os.truncate` and :func:`~os.ftruncate` functions are now supported on Windows. (Contributed by Steve Dower in :issue:`23668`.) -There is a new :func:`~os.path.commonpath` function returning the longest +There is a new :func:`os.path.commonpath` function returning the longest common sub-path of each passed pathname. Unlike the -:func:`~os.path.commonprefix` function, it always returns a valid +:func:`os.path.commonprefix` function, it always returns a valid path. (Contributed by Rafik Draoui and Serhiy Storchaka in :issue:`10395`.) @@ -1048,8 +1051,8 @@ ------- The new :meth:`Path.samefile ` method can be used -to check if the passed :class:`~pathlib.Path` object or a :class:`str` path, -point to the same file. +to check whether the path points to the same file as other path, which can be +either an another :class:`~pathlib.Path` object, or a string. (Contributed by Vajrasky Kok and Antoine Pitrou in :issue:`19775`.) The :meth:`Path.mkdir ` method how accepts a new optional @@ -1070,16 +1073,16 @@ ------ Nested objects, such as unbound methods or nested classes, can now be pickled -using :ref:`pickle protocols ` older than protocol version 4, -which already supported these cases. (Contributed by Serhiy Storchaka in -:issue:`23611`.) +using :ref:`pickle protocols ` older than protocol version 4. +Protocol version 4 already supports these cases. (Contributed by Serhiy +Storchaka in :issue:`23611`.) poplib ------ -A new command :meth:`POP3.utf8 ` enables :rfc:`6856` -(internationalized email) support, if the POP server supports it. +A new :meth:`POP3.utf8 ` command enables :rfc:`6856` +(Internationalized Email) support, if a POP server supports it. (Contributed by Milan OberKirch in :issue:`21804`.) @@ -1090,15 +1093,15 @@ 100. (Contributed by Serhiy Storchaka in :issue:`22437`.) The :func:`~re.sub` and :func:`~re.subn` functions now replace unmatched -groups with empty strings instead of rising an exception. +groups with empty strings instead of raising an exception. (Contributed by Serhiy Storchaka in :issue:`1519638`.) readline -------- -The new :func:`~readline.append_history_file` function can be used to append -the specified number of trailing elements in history to a given file. +A new :func:`~readline.append_history_file` function can be used to append +the specified number of trailing elements in history to the given file. (Contributed by Bruno Cauet in :issue:`22940`.) @@ -1111,7 +1114,7 @@ when moving. (Contributed by Claudiu Popa in :issue:`19840`.) -The :func:`~shutil.make_archive` function now supports *xztar* format. +The :func:`~shutil.make_archive` function now supports the *xztar* format. (Contributed by Serhiy Storchaka in :issue:`5411`.) @@ -1121,7 +1124,7 @@ On Windows, the :func:`~signal.set_wakeup_fd` function now also supports socket handles. (Contributed by Victor Stinner in :issue:`22018`.) -Various ``SIG*`` constants in :mod:`signal` module have been converted into +Various ``SIG*`` constants in the :mod:`signal` module have been converted into :mod:`Enums `. This allows meaningful names to be printed during debugging, instead of integer "magic numbers". (Contributed by Giampaolo Rodola' in :issue:`21076`.) @@ -1133,7 +1136,8 @@ Both :class:`~smtpd.SMTPServer` and :class:`~smtpd.SMTPChannel` classes now accept a *decode_data* keyword argument to determine if the ``DATA`` portion of the SMTP transaction is decoded using the ``"utf-8"`` codec or is instead -provided to :meth:`SMTPServer.process_message ` +provided to the +:meth:`SMTPServer.process_message ` method as a byte string. The default is ``True`` for backward compatibility reasons, but will change to ``False`` in Python 3.6. If *decode_data* is set to ``False``, the :meth:`~smtpd.SMTPServer.process_message` method must @@ -1167,9 +1171,9 @@ implement custom authentication mechanisms. (Contributed by Milan Oberkirch in :issue:`15014`.) -Additional debuglevel (2) shows timestamps for debug messages in -:class:`smtplib.SMTP`. (Contributed by Gavin Chappell and Maciej Szulik in -:issue:`16914`.) +The :meth:`SMTP.set_debuglevel ` method now +accepts an additional debuglevel (2), which enables timestamps in debug +messages. (Contributed by Gavin Chappell and Maciej Szulik in :issue:`16914`.) Both :meth:`SMTP.sendmail ` and :meth:`SMTP.send_message ` methods now @@ -1180,8 +1184,8 @@ sndhdr ------ -The :func:`~sndhdr.what` and :func:`~sndhdr.whathdr` functions now return -a :func:`~collections.namedtuple`. (Contributed by Claudiu Popa in +:func:`~sndhdr.what` and :func:`~sndhdr.whathdr` functions now return +a :func:`~collections.namedtuple`. (Contributed by Claudiu Popa in :issue:`18615`.) @@ -1194,14 +1198,14 @@ (Contributed by Geert Jansen in :issue:`21965`.) The new :class:`~ssl.SSLObject` class has been added to provide SSL protocol -support for cases when the network IO capabilities of :class:`~ssl.SSLSocket` +support for cases when the network I/O capabilities of :class:`~ssl.SSLSocket` are not necessary or suboptimal. :class:`~ssl.SSLObject` represents -an SSL protocol instance, but does not implement any network IO methods, and +an SSL protocol instance, but does not implement any network I/O methods, and instead provides a memory buffer interface. The new :class:`~ssl.MemoryBIO` class can be used to pass data between Python and an SSL protocol instance. The memory BIO SSL support is primarily intended to be used in frameworks -implementing asynchronous IO for which :class:`~ssl.SSLObject` IO readiness +implementing asynchronous I/O for which :class:`~ssl.SSLSocket`'s readiness model ("select/poll") is inefficient. A new :meth:`SSLContext.wrap_bio ` method can be used @@ -1213,12 +1217,12 @@ (Contributed by Benjamin Peterson in :issue:`20188`.) -Where OpenSSL support is present, :mod:`ssl` module now implements * -Application-Layer Protocol Negotiation* TLS extension as described +Where OpenSSL support is present, :mod:`ssl` module now implements +*Application-Layer Protocol Negotiation* TLS extension as described in :rfc:`7301`. The new :meth:`SSLContext.set_alpn_protocols ` -can be used to specify which protocols the socket should advertise during +can be used to specify which protocols a socket should advertise during the TLS handshake. The new @@ -1271,7 +1275,7 @@ (Contributed by Victor Stinner in :issue:`22043`.) A new :meth:`socket.sendfile ` method allows to -send a file over a socket by using high-performance :func:`os.sendfile` +send a file over a socket by using the high-performance :func:`os.sendfile` function on UNIX resulting in uploads being from 2 to 3 times faster than when using plain :meth:`socket.send `. (Contributed by Giampaolo Rodola' in :issue:`17552`.) @@ -1285,9 +1289,12 @@ subprocess ---------- -The new :func:`~subprocess.run` function has been added and is the recommended -approach to invoking subprocesses. It runs the specified command and -and returns a :class:`~subprocess.CompletedProcess` object. +The new :func:`~subprocess.run` function has been added. +It runs the specified command and and returns a +:class:`~subprocess.CompletedProcess` object, which describes a finished +process. The new API is more consistent and is the recommended approach +to invoking subprocesses in Python code that does not need to maintain +compatibility with earlier Python versions. (Contributed by Thomas Kluyver in :issue:`23342`.) @@ -1295,13 +1302,13 @@ --- A new :func:`~sys.set_coroutine_wrapper` function allows setting a global -hook that will be called whenever a :ref:`coro object ` -is created. Essentially, it works like a global coroutine decorator. A -corresponding :func:`~sys.get_coroutine_wrapper` can be used to obtain -a currently set wrapper. Both functions are provisional, and are intended -for debugging purposes only. (Contributed by Yury Selivanov in :issue:`24017`.) +hook that will be called whenever a :term:`coroutine object ` +is created by an :keyword:`async def` function. A corresponding +:func:`~sys.get_coroutine_wrapper` can be used to obtain a currently set +wrapper. Both functions are provisional, and are intended for debugging +purposes only. (Contributed by Yury Selivanov in :issue:`24017`.) -There is a new :func:`~sys.is_finalizing` function to check if the Python +A new :func:`~sys.is_finalizing` function can be used to check if the Python interpreter is :term:`shutting down `. (Contributed by Antoine Pitrou in :issue:`22696`.) @@ -1320,7 +1327,7 @@ The *mode* argument of the :func:`~tarfile.open` function now accepts ``"x"`` to request exclusive creation. (Contributed by Berker Peksag in :issue:`21717`.) -The :meth:`TarFile.extractall ` and +:meth:`TarFile.extractall ` and :meth:`TarFile.extract ` methods now take a keyword argument *numeric_only*. If set to ``True``, the extracted files and directories will be owned by the numeric ``uid`` and ``gid`` from the tarfile. @@ -1332,7 +1339,7 @@ threading --------- -The :meth:`Lock.acquire ` and +Both :meth:`Lock.acquire ` and :meth:`RLock.acquire ` methods now use a monotonic clock for timeout management. (Contributed by Victor Stinner in :issue:`22043`.) @@ -1348,9 +1355,9 @@ timeit ------ -New command line option ``-u`` or ``--unit=U`` to specify a time -unit for the timer output. Supported options are ``usec``, ``msec``, or ``sec``. -(Contributed by Julian Gindi in :issue:`18983`.) +New command line option ``-u`` or ``--unit=U`` can be used to specify the time +unit for the timer output. Supported options are ``usec``, ``msec``, +or ``sec``. (Contributed by Julian Gindi in :issue:`18983`.) tkinter @@ -1373,7 +1380,7 @@ :class:`~traceback.StackSummary`, and :class:`traceback.FrameSummary`. (Contributed by Robert Collins in :issue:`17911`.) -The :func:`~traceback.print_tb` and :func:`~traceback.print_stack` functions +Both :func:`~traceback.print_tb` and :func:`~traceback.print_stack` functions now support negative values for the *limit* argument. (Contributed by Dmitry Kazakov in :issue:`22619`.) @@ -1387,8 +1394,8 @@ :term:`awaitables `. (Contributed by Yury Selivanov in :issue:`24017`.) -A new :class:`~types.CoroutineType` is the type of :term:`coroutine` objects, -produced by calling a function defined with an :keyword:`async def` statement. +A new :class:`~types.CoroutineType` is the type of :term:`coroutine` objects +created by :keyword:`async def` functions. (Contributed by Yury Selivanov in :issue:`24400`.) @@ -1478,22 +1485,24 @@ Optimizations ============= -The :func:`os.walk` function has been sped up by 3-5 times on POSIX systems, -and by 7-20 times on Windows. This was done using the new :func:`os.scandir` +The :func:`os.walk` function has been sped up by 3 to 5 times on POSIX systems, +and by 7 to 20 times on Windows. This was done using the new :func:`os.scandir` function, which exposes file information from the underlying ``readdir`` or -``FindFirstFile``/``FindNextFile`` system calls. (Contributed by +``FindFirstFile``/``FindNextFile`` system calls. (Contributed by Ben Hoyt with help from Victor Stinner in :issue:`23605`.) Construction of ``bytes(int)`` (filled by zero bytes) is faster and uses less memory for large objects. ``calloc()`` is used instead of ``malloc()`` to allocate memory for these objects. +(Contributed by Victor Stinner in :issue:`21233`.) Some operations on :mod:`ipaddress` :class:`~ipaddress.IPv4Network` and :class:`~ipaddress.IPv6Network` have been massively sped up, such as :meth:`~ipaddress.IPv4Network.subnets`, :meth:`~ipaddress.IPv4Network.supernet`, :func:`~ipaddress.summarize_address_range`, :func:`~ipaddress.collapse_addresses`. The speed up can range from 3 to 15 times. -(See :issue:`21486`, :issue:`21487`, :issue:`20826`, :issue:`23266`.) +(Contributed by Antoine Pitrou, Michel Albert, and Markus in +:issue:`21486`, :issue:`21487`, :issue:`20826`, :issue:`23266`.) Pickling of :mod:`ipaddress` objects was optimized to produce significantly smaller output. (Contributed by Serhiy Storchaka in :issue:`23133`.) @@ -1572,7 +1581,7 @@ ``linux-gnu32`` (and ```` will be ``x86_64``). * On Windows, extension module filenames end with - ``..cp-.pyd``: + ``.cp-.pyd``: * ```` is the major number of the Python version; for Python 3.5 this is ``3``. @@ -1607,7 +1616,8 @@ Unsupported Operating Systems ----------------------------- -Per :PEP:`11`, Microsoft support of Windows XP has ended. +Windows XP is no longer supported by Microsoft, thus, per :PEP:`11`, CPython +3.5 is no longer officially supported on this OS. Deprecated Python modules, functions and methods @@ -1782,6 +1792,10 @@ * The :mod:`socket` module now exports the CAN_RAW_FD_FRAMES constant on linux 3.6 and greater. +* The :func:`~ssl.cert_time_to_seconds` function now interprets the input time + as UTC and not as local time, per :rfc:`5280`. Additionally, the return + value is always an :class:`int`. (Contributed by Akira Li in :issue:`19940`.) + * The ``pygettext.py`` Tool now uses the standard +NNNN format for timezones in the POT-Creation-Date header. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 11 03:45:27 2015 From: python-checkins at python.org (yury.selivanov) Date: Fri, 11 Sep 2015 01:45:27 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogd2hhdHNuZXcvMy41?= =?utf-8?q?=3A_Reformat_code_examples?= Message-ID: <20150911014527.15714.20388@psf.io> https://hg.python.org/cpython/rev/0fcb1039a260 changeset: 97888:0fcb1039a260 branch: 3.5 parent: 97886:2034c31eb158 user: Yury Selivanov date: Thu Sep 10 21:44:59 2015 -0400 summary: whatsnew/3.5: Reformat code examples files: Doc/whatsnew/3.5.rst | 62 +++++++++++++++++-------------- 1 files changed, 33 insertions(+), 29 deletions(-) diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -111,7 +111,7 @@ Significantly Improved Library Modules: * :class:`collections.OrderedDict` is now implemented in C, which makes it - 4 to 100 times faster. Contributed by Eric Snow in :issue:`16991`. + 4 to 100 times faster. (Contributed by Eric Snow in :issue:`16991`.) * You may now pass bytes to the :mod:`tempfile` module's APIs and it will return the temporary pathname as :class:`bytes` instead of :class:`str`. @@ -213,34 +213,33 @@ Similarly to asynchronous iteration, there is a new syntax for asynchronous -context managers:: - - >>> import asyncio - >>> async def coro1(lock): - ... print('coro1: waiting for lock') - ... async with lock: - ... print('coro1: holding the lock') - ... await asyncio.sleep(1) - ... print('coro1: releasing the lock') - ... - >>> async def coro2(lock): - ... print('coro2: waiting for lock') - ... async with lock: - ... print('coro2: holding the lock') - ... await asyncio.sleep(1) - ... print('coro2: releasing the lock') - ... - >>> loop = asyncio.get_event_loop() - >>> lock = asyncio.Lock() - >>> coros = asyncio.gather(coro1(lock), coro2(lock), loop=loop) - >>> loop.run_until_complete(coros) - coro1: waiting for lock - coro1: holding the lock - coro2: waiting for lock - coro1: releasing the lock - coro2: holding the lock - coro2: releasing the lock - >>> loop.close() +context managers. The following script:: + + import asyncio + + async def coro(name, lock): + print('coro {}: waiting for lock'.format(name)) + async with lock: + print('coro {}: holding the lock'.format(name)) + await asyncio.sleep(1) + print('coro {}: releasing the lock'.format(name)) + + loop = asyncio.get_event_loop() + lock = asyncio.Lock() + coros = asyncio.gather(coro(1, lock), coro(2, lock)) + try: + loop.run_until_complete(coros) + finally: + loop.close() + +will print:: + + coro 2: waiting for lock + coro 2: holding the lock + coro 1: waiting for lock + coro 2: releasing the lock + coro 1: holding the lock + coro 1: releasing the lock Note that both :keyword:`async for` and :keyword:`async with` can only be used inside a coroutine function declared with :keyword:`async def`. @@ -325,10 +324,13 @@ >>> *range(4), 4 (0, 1, 2, 3, 4) + >>> [*range(4), 4] [0, 1, 2, 3, 4] + >>> {*range(4), 4, *(5, 6, 7)} {0, 1, 2, 3, 4, 5, 6, 7} + >>> {'x': 1, **{'y': 2}} {'x': 1, 'y': 2} @@ -352,6 +354,7 @@ >>> b'Hello %s!' % b'World' b'Hello World!' + >>> b'x=%i y=%f' % (1, 2.5) b'x=1 y=2.500000' @@ -362,6 +365,7 @@ Traceback (most recent call last): File "", line 1, in TypeError: %b requires bytes, or an object that implements __bytes__, not 'str' + >>> b'price: %a' % '10?' b"price: '10\\u20ac'" -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 11 03:45:27 2015 From: python-checkins at python.org (yury.selivanov) Date: Fri, 11 Sep 2015 01:45:27 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogTWVyZ2UgMy41?= Message-ID: <20150911014527.15724.94024@psf.io> https://hg.python.org/cpython/rev/f58f93b99bbc changeset: 97889:f58f93b99bbc parent: 97887:3af318476d0f parent: 97888:0fcb1039a260 user: Yury Selivanov date: Thu Sep 10 21:45:08 2015 -0400 summary: Merge 3.5 files: Doc/whatsnew/3.5.rst | 62 +++++++++++++++++-------------- 1 files changed, 33 insertions(+), 29 deletions(-) diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -111,7 +111,7 @@ Significantly Improved Library Modules: * :class:`collections.OrderedDict` is now implemented in C, which makes it - 4 to 100 times faster. Contributed by Eric Snow in :issue:`16991`. + 4 to 100 times faster. (Contributed by Eric Snow in :issue:`16991`.) * You may now pass bytes to the :mod:`tempfile` module's APIs and it will return the temporary pathname as :class:`bytes` instead of :class:`str`. @@ -213,34 +213,33 @@ Similarly to asynchronous iteration, there is a new syntax for asynchronous -context managers:: - - >>> import asyncio - >>> async def coro1(lock): - ... print('coro1: waiting for lock') - ... async with lock: - ... print('coro1: holding the lock') - ... await asyncio.sleep(1) - ... print('coro1: releasing the lock') - ... - >>> async def coro2(lock): - ... print('coro2: waiting for lock') - ... async with lock: - ... print('coro2: holding the lock') - ... await asyncio.sleep(1) - ... print('coro2: releasing the lock') - ... - >>> loop = asyncio.get_event_loop() - >>> lock = asyncio.Lock() - >>> coros = asyncio.gather(coro1(lock), coro2(lock), loop=loop) - >>> loop.run_until_complete(coros) - coro1: waiting for lock - coro1: holding the lock - coro2: waiting for lock - coro1: releasing the lock - coro2: holding the lock - coro2: releasing the lock - >>> loop.close() +context managers. The following script:: + + import asyncio + + async def coro(name, lock): + print('coro {}: waiting for lock'.format(name)) + async with lock: + print('coro {}: holding the lock'.format(name)) + await asyncio.sleep(1) + print('coro {}: releasing the lock'.format(name)) + + loop = asyncio.get_event_loop() + lock = asyncio.Lock() + coros = asyncio.gather(coro(1, lock), coro(2, lock)) + try: + loop.run_until_complete(coros) + finally: + loop.close() + +will print:: + + coro 2: waiting for lock + coro 2: holding the lock + coro 1: waiting for lock + coro 2: releasing the lock + coro 1: holding the lock + coro 1: releasing the lock Note that both :keyword:`async for` and :keyword:`async with` can only be used inside a coroutine function declared with :keyword:`async def`. @@ -325,10 +324,13 @@ >>> *range(4), 4 (0, 1, 2, 3, 4) + >>> [*range(4), 4] [0, 1, 2, 3, 4] + >>> {*range(4), 4, *(5, 6, 7)} {0, 1, 2, 3, 4, 5, 6, 7} + >>> {'x': 1, **{'y': 2}} {'x': 1, 'y': 2} @@ -352,6 +354,7 @@ >>> b'Hello %s!' % b'World' b'Hello World!' + >>> b'x=%i y=%f' % (1, 2.5) b'x=1 y=2.500000' @@ -362,6 +365,7 @@ Traceback (most recent call last): File "", line 1, in TypeError: %b requires bytes, or an object that implements __bytes__, not 'str' + >>> b'price: %a' % '10?' b"price: '10\\u20ac'" -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 11 04:36:36 2015 From: python-checkins at python.org (martin.panter) Date: Fri, 11 Sep 2015 02:36:36 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2324984=3A_Merge_BTPROTO=5FSCO_doc_fix_from_3=2E5?= Message-ID: <20150911023636.68885.85692@psf.io> https://hg.python.org/cpython/rev/7e4391155d9c changeset: 97892:7e4391155d9c parent: 97889:f58f93b99bbc parent: 97891:b507a7e79154 user: Martin Panter date: Fri Sep 11 02:30:08 2015 +0000 summary: Issue #24984: Merge BTPROTO_SCO doc fix from 3.5 files: Doc/library/socket.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/library/socket.rst b/Doc/library/socket.rst --- a/Doc/library/socket.rst +++ b/Doc/library/socket.rst @@ -124,7 +124,7 @@ NetBSD and DragonFlyBSD support added. - :const:`BTPROTO_SCO` accepts ``bdaddr`` where ``bdaddr`` is a - :term:`bytes-like object` containing the Bluetooth address in a + :class:`bytes` object containing the Bluetooth address in a string format. (ex. ``b'12:23:34:45:56:67'``) This protocol is not supported under FreeBSD. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 11 04:36:36 2015 From: python-checkins at python.org (martin.panter) Date: Fri, 11 Sep 2015 02:36:36 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzI0OTg0?= =?utf-8?q?=3A_BTPROTO=5FSCO_supports_only_bytes_objects?= Message-ID: <20150911023636.27685.33569@psf.io> https://hg.python.org/cpython/rev/00db99149cdd changeset: 97890:00db99149cdd branch: 3.4 parent: 97871:c535bf72aa60 user: Martin Panter date: Fri Sep 11 02:23:41 2015 +0000 summary: Issue #24984: BTPROTO_SCO supports only bytes objects files: Doc/library/socket.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/library/socket.rst b/Doc/library/socket.rst --- a/Doc/library/socket.rst +++ b/Doc/library/socket.rst @@ -121,7 +121,7 @@ NetBSD and DragonFlyBSD support added. - :const:`BTPROTO_SCO` accepts ``bdaddr`` where ``bdaddr`` is a - :term:`bytes-like object` containing the Bluetooth address in a + :class:`bytes` object containing the Bluetooth address in a string format. (ex. ``b'12:23:34:45:56:67'``) This protocol is not supported under FreeBSD. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 11 04:36:36 2015 From: python-checkins at python.org (martin.panter) Date: Fri, 11 Sep 2015 02:36:36 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Issue_=2324984=3A_Merge_BTPROTO=5FSCO_doc_fix_from_3=2E4_into_?= =?utf-8?b?My41?= Message-ID: <20150911023636.14881.1236@psf.io> https://hg.python.org/cpython/rev/b507a7e79154 changeset: 97891:b507a7e79154 branch: 3.5 parent: 97888:0fcb1039a260 parent: 97890:00db99149cdd user: Martin Panter date: Fri Sep 11 02:29:35 2015 +0000 summary: Issue #24984: Merge BTPROTO_SCO doc fix from 3.4 into 3.5 files: Doc/library/socket.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/library/socket.rst b/Doc/library/socket.rst --- a/Doc/library/socket.rst +++ b/Doc/library/socket.rst @@ -124,7 +124,7 @@ NetBSD and DragonFlyBSD support added. - :const:`BTPROTO_SCO` accepts ``bdaddr`` where ``bdaddr`` is a - :term:`bytes-like object` containing the Bluetooth address in a + :class:`bytes` object containing the Bluetooth address in a string format. (ex. ``b'12:23:34:45:56:67'``) This protocol is not supported under FreeBSD. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 11 04:59:57 2015 From: python-checkins at python.org (martin.panter) Date: Fri, 11 Sep 2015 02:59:57 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzI1MDIy?= =?utf-8?q?=3A_Avoid_warning_about_unused_suspicious_rule?= Message-ID: <20150911025957.12011.62359@psf.io> https://hg.python.org/cpython/rev/8ba735b018d2 changeset: 97893:8ba735b018d2 branch: 3.4 parent: 97890:00db99149cdd user: Martin Panter date: Fri Sep 11 02:45:10 2015 +0000 summary: Issue #25022: Avoid warning about unused suspicious rule files: Doc/tools/susp-ignored.csv | 1 - 1 files changed, 0 insertions(+), 1 deletions(-) diff --git a/Doc/tools/susp-ignored.csv b/Doc/tools/susp-ignored.csv --- a/Doc/tools/susp-ignored.csv +++ b/Doc/tools/susp-ignored.csv @@ -10,7 +10,6 @@ extending/extending,,:myfunction,"PyArg_ParseTuple(args, ""D:myfunction"", &c);" extending/extending,,:set,"if (PyArg_ParseTuple(args, ""O:set_callback"", &temp)) {" extending/newtypes,,:call,"if (!PyArg_ParseTuple(args, ""sss:call"", &arg1, &arg2, &arg3)) {" -extending/windows,,:initspam,/export:initspam faq/programming,,:chr,">=4.0) or 1+f(xc,yc,x*x-y*y+xc,2.0*x*y+yc,k-1,f):f(xc,yc,x,y,k,f):chr(" faq/programming,,::,for x in sequence[::-1]: faq/programming,,:reduce,"print((lambda Ru,Ro,Iu,Io,IM,Sx,Sy:reduce(lambda x,y:x+y,map(lambda y," -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 11 05:00:02 2015 From: python-checkins at python.org (martin.panter) Date: Fri, 11 Sep 2015 03:00:02 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzI1MDIy?= =?utf-8?q?=3A_Avoid_warning_about_unused_suspicious_rule?= Message-ID: <20150911030002.14879.14728@psf.io> https://hg.python.org/cpython/rev/1a52db3ef0e8 changeset: 97896:1a52db3ef0e8 branch: 2.7 parent: 97870:3456af022541 user: Martin Panter date: Fri Sep 11 02:45:10 2015 +0000 summary: Issue #25022: Avoid warning about unused suspicious rule files: Doc/tools/susp-ignored.csv | 1 - 1 files changed, 0 insertions(+), 1 deletions(-) diff --git a/Doc/tools/susp-ignored.csv b/Doc/tools/susp-ignored.csv --- a/Doc/tools/susp-ignored.csv +++ b/Doc/tools/susp-ignored.csv @@ -7,7 +7,6 @@ extending/extending,,:set,"if (PyArg_ParseTuple(args, ""O:set_callback"", &temp)) {" extending/extending,,:myfunction,"PyArg_ParseTuple(args, ""D:myfunction"", &c);" extending/newtypes,,:call,"if (!PyArg_ParseTuple(args, ""sss:call"", &arg1, &arg2, &arg3)) {" -extending/windows,,:initspam,/export:initspam faq/programming,,:reduce,"print (lambda Ru,Ro,Iu,Io,IM,Sx,Sy:reduce(lambda x,y:x+y,map(lambda y," faq/programming,,:reduce,"Sx=Sx,Sy=Sy:reduce(lambda x,y:x+y,map(lambda x,xc=Ru,yc=yc,Ru=Ru,Ro=Ro," faq/programming,,:chr,">=4.0) or 1+f(xc,yc,x*x-y*y+xc,2.0*x*y+yc,k-1,f):f(xc,yc,x,y,k,f):chr(" -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 11 05:00:02 2015 From: python-checkins at python.org (martin.panter) Date: Fri, 11 Sep 2015 03:00:02 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Issue_=2325022=3A_Merge_susp-ignored=2Ecsv_from_3=2E4_into_3?= =?utf-8?q?=2E5?= Message-ID: <20150911030002.12009.83090@psf.io> https://hg.python.org/cpython/rev/b75b4a6e1c9b changeset: 97894:b75b4a6e1c9b branch: 3.5 parent: 97891:b507a7e79154 parent: 97893:8ba735b018d2 user: Martin Panter date: Fri Sep 11 02:46:54 2015 +0000 summary: Issue #25022: Merge susp-ignored.csv from 3.4 into 3.5 files: Doc/tools/susp-ignored.csv | 1 - 1 files changed, 0 insertions(+), 1 deletions(-) diff --git a/Doc/tools/susp-ignored.csv b/Doc/tools/susp-ignored.csv --- a/Doc/tools/susp-ignored.csv +++ b/Doc/tools/susp-ignored.csv @@ -10,7 +10,6 @@ extending/extending,,:myfunction,"PyArg_ParseTuple(args, ""D:myfunction"", &c);" extending/extending,,:set,"if (PyArg_ParseTuple(args, ""O:set_callback"", &temp)) {" extending/newtypes,,:call,"if (!PyArg_ParseTuple(args, ""sss:call"", &arg1, &arg2, &arg3)) {" -extending/windows,,:initspam,/export:initspam faq/programming,,:chr,">=4.0) or 1+f(xc,yc,x*x-y*y+xc,2.0*x*y+yc,k-1,f):f(xc,yc,x,y,k,f):chr(" faq/programming,,::,for x in sequence[::-1]: faq/programming,,:reduce,"print((lambda Ru,Ro,Iu,Io,IM,Sx,Sy:reduce(lambda x,y:x+y,map(lambda y," -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 11 05:00:02 2015 From: python-checkins at python.org (martin.panter) Date: Fri, 11 Sep 2015 03:00:02 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2325022=3A_Merge_susp-ignored=2Ecsv_from_3=2E5?= Message-ID: <20150911030002.11258.77081@psf.io> https://hg.python.org/cpython/rev/8902bcea59be changeset: 97895:8902bcea59be parent: 97892:7e4391155d9c parent: 97894:b75b4a6e1c9b user: Martin Panter date: Fri Sep 11 02:47:13 2015 +0000 summary: Issue #25022: Merge susp-ignored.csv from 3.5 files: Doc/tools/susp-ignored.csv | 1 - 1 files changed, 0 insertions(+), 1 deletions(-) diff --git a/Doc/tools/susp-ignored.csv b/Doc/tools/susp-ignored.csv --- a/Doc/tools/susp-ignored.csv +++ b/Doc/tools/susp-ignored.csv @@ -10,7 +10,6 @@ extending/extending,,:myfunction,"PyArg_ParseTuple(args, ""D:myfunction"", &c);" extending/extending,,:set,"if (PyArg_ParseTuple(args, ""O:set_callback"", &temp)) {" extending/newtypes,,:call,"if (!PyArg_ParseTuple(args, ""sss:call"", &arg1, &arg2, &arg3)) {" -extending/windows,,:initspam,/export:initspam faq/programming,,:chr,">=4.0) or 1+f(xc,yc,x*x-y*y+xc,2.0*x*y+yc,k-1,f):f(xc,yc,x,y,k,f):chr(" faq/programming,,::,for x in sequence[::-1]: faq/programming,,:reduce,"print((lambda Ru,Ro,Iu,Io,IM,Sx,Sy:reduce(lambda x,y:x+y,map(lambda y," -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 11 05:37:19 2015 From: python-checkins at python.org (yury.selivanov) Date: Fri, 11 Sep 2015 03:37:19 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogd2hhdHNuZXcvMy41?= =?utf-8?q?=3A_Second_pass_over_NEWS_entries?= Message-ID: <20150911033719.17983.82579@psf.io> https://hg.python.org/cpython/rev/c9543b3ad827 changeset: 97897:c9543b3ad827 branch: 3.5 parent: 97894:b75b4a6e1c9b user: Yury Selivanov date: Thu Sep 10 23:37:06 2015 -0400 summary: whatsnew/3.5: Second pass over NEWS entries files: Doc/whatsnew/3.5.rst | 105 ++++++++++++++++++++++++++++-- 1 files changed, 95 insertions(+), 10 deletions(-) diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -4,7 +4,7 @@ :Release: |release| :Date: |today| -:Editor: Elvis Pranskevichus +:Editors: Elvis Pranskevichus , Yury Selivanov .. Rules for maintenance: @@ -671,11 +671,18 @@ The :class:`~collections.OrderedDict` class is now implemented in C, which makes it 4 to 100 times faster. (Contributed by Eric Snow in :issue:`16991`.) +:meth:`OrderedDict.items `, +:meth:`OrderedDict.items `, +:meth:`OrderedDict.items ` views now support +:func:`reversed` iteration. +(Contributed by Serhiy Storchaka in :issue:`19505`.) + The :class:`~collections.deque` class now defines :meth:`~collections.deque.index`, :meth:`~collections.deque.insert`, and -:meth:`~collections.deque.copy`. This allows deques to be recognized as a -:class:`~collections.abc.MutableSequence` and improves their substitutability -for lists. (Contributed by Raymond Hettinger :issue:`23704`.) +:meth:`~collections.deque.copy`, as well as supports ``+`` and ``*`` operators. +This allows deques to be recognized as a :class:`~collections.abc.MutableSequence` +and improves their substitutability for lists. +(Contributed by Raymond Hettinger :issue:`23704`.) Docstrings produced by :func:`~collections.namedtuple` can now be updated:: @@ -945,6 +952,11 @@ network objects from existing addresses. (Contributed by Peter Moody and Antoine Pitrou in :issue:`16531`.) +New :attr:`~ipaddress.IPv4Network.reverse_pointer>` attribute for +:class:`~ipaddress.IPv4Network` and :class:`~ipaddress.IPv6Network` classes +returns the name of the reverse DNS PTR record. +(Contributed by Leon Weber in :issue:`20480`.) + json ---- @@ -1100,6 +1112,13 @@ groups with empty strings instead of raising an exception. (Contributed by Serhiy Storchaka in :issue:`1519638`.) +The :class:`re.error` exceptions have new attributes: +:attr:`~re.error.msg`, :attr:`~re.error.pattern`, +:attr:`~re.error.pos`, :attr:`~re.error.lineno`, +and :attr:`~re.error.colno` that provide better context +information about the error. +(Contributed by Serhiy Storchaka in :issue:`22578`.) + readline -------- @@ -1109,6 +1128,13 @@ (Contributed by Bruno Cauet in :issue:`22940`.) +selectors +--------- + +The module now supports efficient ``/dev/poll`` on Solaris. +(Contributed by Giampaolo Rodola' in :issue:`18931`.) + + shutil ------ @@ -1244,7 +1270,7 @@ The :class:`~ssl.SSLSocket` class now implements a :meth:`SSLSocket.sendfile ` method. -(Contributed by Giampaolo Rodola in :issue:`17552`.) +(Contributed by Giampaolo Rodola' in :issue:`17552`.) The :meth:`SSLSocket.send ` method now raises either :exc:`ssl.SSLWantReadError` or :exc:`ssl.SSLWantWriteError` exception on a @@ -1289,6 +1315,20 @@ now the maximum total duration to send all data. (Contributed by Victor Stinner in :issue:`23853`.) +The *backlog* argument of the :meth:`socket.listen ` +method is now optional. By default it is set to +:data:`SOMAXCONN ` or to ``128`` whichever is less. +(Contributed by Charles-Fran?ois Natali in :issue:`21455`.) + + +sqlite3 +------- + +The :class:`~sqlite3.Row` class now fully supports sequence protocol, +in particular :func:`reverse` and slice indexing. +(Contributed by Claudiu Popa in :issue:`10203`; by Lucas Sinclair, +Jessica McKellar, and Serhiy Storchaka in :issue:`13583`.) + subprocess ---------- @@ -1363,6 +1403,10 @@ unit for the timer output. Supported options are ``usec``, ``msec``, or ``sec``. (Contributed by Julian Gindi in :issue:`18983`.) +The :func:`~timeit.timeit` function has a new *globals* parameter for +specifying the namespace in which the code will be running. +(Contributed by Ben Roberts in :issue:`2527`.) + tkinter ------- @@ -1424,6 +1468,11 @@ :class:`ssl.SSLContext` object as a *context* argument, which will be used for the HTTPS connection. (Contributed by Alex Gaynor in :issue:`22366`.) +The :func:`parse.urljoin ` was updated to use the +:rfc:`3986` semantics for the resolution of relative URLs, rather than +:rfc:`1808` and :rfc:`2396`. +(Contributed by Demian Brecht and Senthil Kumaran in :issue:`22118`.) + unicodedata ----------- @@ -1439,6 +1488,26 @@ tracebacks. (Contributed by Robert Collins in :issue:`22936`.) +unittest.mock +------------- + +The :class:`~unittest.mock.Mock` has the following improvements: + +* Class constructor has a new *unsafe* parameter, which causes mock + objects to raise :exc:`AttributeError` on attribute names starting + with ``"assert"``. + (Contributed by Kushal Das in :issue:`21238`.) + +* A new :meth:`Mock.assert_not_called ` + method to check if the mock object was called. + (Contributed by Kushal Das in :issue:`21262`.) + +The :class:`~unittest.mock.MagicMock` class now supports :meth:`__truediv__`, +:meth:`__divmod__` and :meth:`__matmul__` operators. +(Contributed by Johannes Baiter in :issue:`20968`, and H?kan L?vdahl +in :issue:`23581` and :issue:`23568`.) + + wsgiref ------- @@ -1548,16 +1617,32 @@ Instantiation of :class:`fractions.Fraction` is now up to 30% faster. (Contributed by Stefan Behnel in :issue:`22464`.) +String methods :meth:`~str.find`, :meth:`~str.rfind`, :meth:`~str.split`, +:meth:`~str.partition` and :keyword:`in` string operator are now significantly +faster for searching 1-character substrings. +(Contributed by Serhiy Storchaka in :issue:`23573`.) + Build and C API Changes ======================= New ``calloc`` functions were added: - * :c:func:`PyMem_RawCalloc` - * :c:func:`PyMem_Calloc` - * :c:func:`PyObject_Calloc` - * :c:func:`_PyObject_GC_Calloc` + * :c:func:`PyMem_RawCalloc`, + * :c:func:`PyMem_Calloc`, + * :c:func:`PyObject_Calloc`, + * :c:func:`_PyObject_GC_Calloc`. + +(Contributed by Victor Stinner in :issue:`21233`.) + +New encoding/decoding helper functions: + + * :c:func:`Py_DecodeLocale` (replaced ``_Py_char2wchar()``), + * :c:func:`Py_EncodeLocale` (replaced ``_Py_wchar2char()``). + +(Contributed by Victor Stinner in :issue:`18395`.) + +The :c:member:`PyTypeObject.tp_finalize` slot is now part of stable ABI. Windows builds now require Microsoft Visual C++ 14.0, which is available as part of `Visual Studio 2015 `_. @@ -1833,6 +1918,7 @@ * Removed non-documented macro :c:macro:`PyObject_REPR` which leaked references. Use format character ``%R`` in :c:func:`PyUnicode_FromFormat`-like functions to format the :func:`repr` of the object. + (Contributed by Serhiy Storchaka in :issue:`22453`.) * Because the lack of the :attr:`__module__` attribute breaks pickling and introspection, a deprecation warning now is raised for builtin type without @@ -1844,4 +1930,3 @@ :c:member:`tp_as_async` slot. Refer to :ref:`coro-objects` for new types, structures and functions. -* :c:member:`PyTypeObject.tp_finalize` is now part of stable ABI. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 11 05:37:19 2015 From: python-checkins at python.org (yury.selivanov) Date: Fri, 11 Sep 2015 03:37:19 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogTWVyZ2UgMy41?= Message-ID: <20150911033719.14865.48526@psf.io> https://hg.python.org/cpython/rev/72736bd18062 changeset: 97898:72736bd18062 parent: 97895:8902bcea59be parent: 97897:c9543b3ad827 user: Yury Selivanov date: Thu Sep 10 23:37:16 2015 -0400 summary: Merge 3.5 files: Doc/whatsnew/3.5.rst | 105 ++++++++++++++++++++++++++++-- 1 files changed, 95 insertions(+), 10 deletions(-) diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -4,7 +4,7 @@ :Release: |release| :Date: |today| -:Editor: Elvis Pranskevichus +:Editors: Elvis Pranskevichus , Yury Selivanov .. Rules for maintenance: @@ -671,11 +671,18 @@ The :class:`~collections.OrderedDict` class is now implemented in C, which makes it 4 to 100 times faster. (Contributed by Eric Snow in :issue:`16991`.) +:meth:`OrderedDict.items `, +:meth:`OrderedDict.items `, +:meth:`OrderedDict.items ` views now support +:func:`reversed` iteration. +(Contributed by Serhiy Storchaka in :issue:`19505`.) + The :class:`~collections.deque` class now defines :meth:`~collections.deque.index`, :meth:`~collections.deque.insert`, and -:meth:`~collections.deque.copy`. This allows deques to be recognized as a -:class:`~collections.abc.MutableSequence` and improves their substitutability -for lists. (Contributed by Raymond Hettinger :issue:`23704`.) +:meth:`~collections.deque.copy`, as well as supports ``+`` and ``*`` operators. +This allows deques to be recognized as a :class:`~collections.abc.MutableSequence` +and improves their substitutability for lists. +(Contributed by Raymond Hettinger :issue:`23704`.) Docstrings produced by :func:`~collections.namedtuple` can now be updated:: @@ -945,6 +952,11 @@ network objects from existing addresses. (Contributed by Peter Moody and Antoine Pitrou in :issue:`16531`.) +New :attr:`~ipaddress.IPv4Network.reverse_pointer>` attribute for +:class:`~ipaddress.IPv4Network` and :class:`~ipaddress.IPv6Network` classes +returns the name of the reverse DNS PTR record. +(Contributed by Leon Weber in :issue:`20480`.) + json ---- @@ -1100,6 +1112,13 @@ groups with empty strings instead of raising an exception. (Contributed by Serhiy Storchaka in :issue:`1519638`.) +The :class:`re.error` exceptions have new attributes: +:attr:`~re.error.msg`, :attr:`~re.error.pattern`, +:attr:`~re.error.pos`, :attr:`~re.error.lineno`, +and :attr:`~re.error.colno` that provide better context +information about the error. +(Contributed by Serhiy Storchaka in :issue:`22578`.) + readline -------- @@ -1109,6 +1128,13 @@ (Contributed by Bruno Cauet in :issue:`22940`.) +selectors +--------- + +The module now supports efficient ``/dev/poll`` on Solaris. +(Contributed by Giampaolo Rodola' in :issue:`18931`.) + + shutil ------ @@ -1244,7 +1270,7 @@ The :class:`~ssl.SSLSocket` class now implements a :meth:`SSLSocket.sendfile ` method. -(Contributed by Giampaolo Rodola in :issue:`17552`.) +(Contributed by Giampaolo Rodola' in :issue:`17552`.) The :meth:`SSLSocket.send ` method now raises either :exc:`ssl.SSLWantReadError` or :exc:`ssl.SSLWantWriteError` exception on a @@ -1289,6 +1315,20 @@ now the maximum total duration to send all data. (Contributed by Victor Stinner in :issue:`23853`.) +The *backlog* argument of the :meth:`socket.listen ` +method is now optional. By default it is set to +:data:`SOMAXCONN ` or to ``128`` whichever is less. +(Contributed by Charles-Fran?ois Natali in :issue:`21455`.) + + +sqlite3 +------- + +The :class:`~sqlite3.Row` class now fully supports sequence protocol, +in particular :func:`reverse` and slice indexing. +(Contributed by Claudiu Popa in :issue:`10203`; by Lucas Sinclair, +Jessica McKellar, and Serhiy Storchaka in :issue:`13583`.) + subprocess ---------- @@ -1363,6 +1403,10 @@ unit for the timer output. Supported options are ``usec``, ``msec``, or ``sec``. (Contributed by Julian Gindi in :issue:`18983`.) +The :func:`~timeit.timeit` function has a new *globals* parameter for +specifying the namespace in which the code will be running. +(Contributed by Ben Roberts in :issue:`2527`.) + tkinter ------- @@ -1424,6 +1468,11 @@ :class:`ssl.SSLContext` object as a *context* argument, which will be used for the HTTPS connection. (Contributed by Alex Gaynor in :issue:`22366`.) +The :func:`parse.urljoin ` was updated to use the +:rfc:`3986` semantics for the resolution of relative URLs, rather than +:rfc:`1808` and :rfc:`2396`. +(Contributed by Demian Brecht and Senthil Kumaran in :issue:`22118`.) + unicodedata ----------- @@ -1439,6 +1488,26 @@ tracebacks. (Contributed by Robert Collins in :issue:`22936`.) +unittest.mock +------------- + +The :class:`~unittest.mock.Mock` has the following improvements: + +* Class constructor has a new *unsafe* parameter, which causes mock + objects to raise :exc:`AttributeError` on attribute names starting + with ``"assert"``. + (Contributed by Kushal Das in :issue:`21238`.) + +* A new :meth:`Mock.assert_not_called ` + method to check if the mock object was called. + (Contributed by Kushal Das in :issue:`21262`.) + +The :class:`~unittest.mock.MagicMock` class now supports :meth:`__truediv__`, +:meth:`__divmod__` and :meth:`__matmul__` operators. +(Contributed by Johannes Baiter in :issue:`20968`, and H?kan L?vdahl +in :issue:`23581` and :issue:`23568`.) + + wsgiref ------- @@ -1548,16 +1617,32 @@ Instantiation of :class:`fractions.Fraction` is now up to 30% faster. (Contributed by Stefan Behnel in :issue:`22464`.) +String methods :meth:`~str.find`, :meth:`~str.rfind`, :meth:`~str.split`, +:meth:`~str.partition` and :keyword:`in` string operator are now significantly +faster for searching 1-character substrings. +(Contributed by Serhiy Storchaka in :issue:`23573`.) + Build and C API Changes ======================= New ``calloc`` functions were added: - * :c:func:`PyMem_RawCalloc` - * :c:func:`PyMem_Calloc` - * :c:func:`PyObject_Calloc` - * :c:func:`_PyObject_GC_Calloc` + * :c:func:`PyMem_RawCalloc`, + * :c:func:`PyMem_Calloc`, + * :c:func:`PyObject_Calloc`, + * :c:func:`_PyObject_GC_Calloc`. + +(Contributed by Victor Stinner in :issue:`21233`.) + +New encoding/decoding helper functions: + + * :c:func:`Py_DecodeLocale` (replaced ``_Py_char2wchar()``), + * :c:func:`Py_EncodeLocale` (replaced ``_Py_wchar2char()``). + +(Contributed by Victor Stinner in :issue:`18395`.) + +The :c:member:`PyTypeObject.tp_finalize` slot is now part of stable ABI. Windows builds now require Microsoft Visual C++ 14.0, which is available as part of `Visual Studio 2015 `_. @@ -1833,6 +1918,7 @@ * Removed non-documented macro :c:macro:`PyObject_REPR` which leaked references. Use format character ``%R`` in :c:func:`PyUnicode_FromFormat`-like functions to format the :func:`repr` of the object. + (Contributed by Serhiy Storchaka in :issue:`22453`.) * Because the lack of the :attr:`__module__` attribute breaks pickling and introspection, a deprecation warning now is raised for builtin type without @@ -1844,4 +1930,3 @@ :c:member:`tp_as_async` slot. Refer to :ref:`coro-objects` for new types, structures and functions. -* :c:member:`PyTypeObject.tp_finalize` is now part of stable ABI. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 11 06:11:32 2015 From: python-checkins at python.org (benjamin.peterson) Date: Fri, 11 Sep 2015 04:11:32 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogbWVyZ2UgMy41ICgjMjUwNjAp?= Message-ID: <20150911041132.11248.74743@psf.io> https://hg.python.org/cpython/rev/7d76df6dafde changeset: 97900:7d76df6dafde parent: 97898:72736bd18062 parent: 97899:6c1b0dd07340 user: Benjamin Peterson date: Thu Sep 10 21:11:26 2015 -0700 summary: merge 3.5 (#25060) files: Misc/NEWS | 2 ++ Python/compile.c | 2 +- Python/importlib_external.h | 2 +- 3 files changed, 4 insertions(+), 2 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -103,6 +103,8 @@ Library ------- +- Issue #25060: Correctly compute stack usage of the BUILD_MAP opcode. + - Issue #24857: Comparing call_args to a long sequence now correctly returns a boolean result instead of raising an exception. Patch by A Kaptur. diff --git a/Python/compile.c b/Python/compile.c --- a/Python/compile.c +++ b/Python/compile.c @@ -985,7 +985,7 @@ case BUILD_MAP_UNPACK_WITH_CALL: return 1 - (oparg & 0xFF); case BUILD_MAP: - return 1; + return 1 - 2*oparg; case LOAD_ATTR: return 0; case COMPARE_OP: diff --git a/Python/importlib_external.h b/Python/importlib_external.h --- a/Python/importlib_external.h +++ b/Python/importlib_external.h @@ -1395,7 +1395,7 @@ 32,83,111,117,114,99,101,76,111,97,100,101,114,32,117,115, 105,110,103,32,116,104,101,32,102,105,108,101,32,115,121,115, 116,101,109,46,99,2,0,0,0,0,0,0,0,3,0,0, - 0,5,0,0,0,67,0,0,0,115,34,0,0,0,116,0, + 0,4,0,0,0,67,0,0,0,115,34,0,0,0,116,0, 0,124,1,0,131,1,0,125,2,0,100,1,0,124,2,0, 106,1,0,100,2,0,124,2,0,106,2,0,105,2,0,83, 41,3,122,33,82,101,116,117,114,110,32,116,104,101,32,109, -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 11 06:11:32 2015 From: python-checkins at python.org (benjamin.peterson) Date: Fri, 11 Sep 2015 04:11:32 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_compute_stack_?= =?utf-8?q?effect_of_BUILD=5FMAP_correctly_=28closes_=2325060=29?= Message-ID: <20150911041132.14859.56040@psf.io> https://hg.python.org/cpython/rev/6c1b0dd07340 changeset: 97899:6c1b0dd07340 branch: 3.5 parent: 97897:c9543b3ad827 user: Benjamin Peterson date: Thu Sep 10 21:02:39 2015 -0700 summary: compute stack effect of BUILD_MAP correctly (closes #25060) files: Misc/NEWS | 2 ++ Python/compile.c | 2 +- Python/importlib_external.h | 2 +- 3 files changed, 4 insertions(+), 2 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -14,6 +14,8 @@ Library ------- +- Issue #25060: Correctly compute stack usage of the BUILD_MAP opcode. + - Issue #24857: Comparing call_args to a long sequence now correctly returns a boolean result instead of raising an exception. Patch by A Kaptur. diff --git a/Python/compile.c b/Python/compile.c --- a/Python/compile.c +++ b/Python/compile.c @@ -985,7 +985,7 @@ case BUILD_MAP_UNPACK_WITH_CALL: return 1 - (oparg & 0xFF); case BUILD_MAP: - return 1; + return 1 - 2*oparg; case LOAD_ATTR: return 0; case COMPARE_OP: diff --git a/Python/importlib_external.h b/Python/importlib_external.h --- a/Python/importlib_external.h +++ b/Python/importlib_external.h @@ -1395,7 +1395,7 @@ 32,83,111,117,114,99,101,76,111,97,100,101,114,32,117,115, 105,110,103,32,116,104,101,32,102,105,108,101,32,115,121,115, 116,101,109,46,99,2,0,0,0,0,0,0,0,3,0,0, - 0,5,0,0,0,67,0,0,0,115,34,0,0,0,116,0, + 0,4,0,0,0,67,0,0,0,115,34,0,0,0,116,0, 0,124,1,0,131,1,0,125,2,0,100,1,0,124,2,0, 106,1,0,100,2,0,124,2,0,106,2,0,105,2,0,83, 41,3,122,33,82,101,116,117,114,110,32,116,104,101,32,109, -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 11 06:40:48 2015 From: python-checkins at python.org (martin.panter) Date: Fri, 11 Sep 2015 04:40:48 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2325030=3A_Merge_seek=28=29_doc_fixes_from_3=2E5?= Message-ID: <20150911044048.14871.86613@psf.io> https://hg.python.org/cpython/rev/c4bb0da8b45e changeset: 97904:c4bb0da8b45e parent: 97900:7d76df6dafde parent: 97903:a0c6f5358029 user: Martin Panter date: Fri Sep 11 04:39:42 2015 +0000 summary: Issue #25030: Merge seek() doc fixes from 3.5 files: Doc/library/io.rst | 13 +++++++------ Misc/ACKS | 1 + 2 files changed, 8 insertions(+), 6 deletions(-) diff --git a/Doc/library/io.rst b/Doc/library/io.rst --- a/Doc/library/io.rst +++ b/Doc/library/io.rst @@ -301,11 +301,11 @@ Note that it's already possible to iterate on file objects using ``for line in file: ...`` without calling ``file.readlines()``. - .. method:: seek(offset, whence=SEEK_SET) + .. method:: seek(offset[, whence]) Change the stream position to the given byte *offset*. *offset* is - interpreted relative to the position indicated by *whence*. Values for - *whence* are: + interpreted relative to the position indicated by *whence*. The default + value for *whence* is :data:`SEEK_SET`. Values for *whence* are: * :data:`SEEK_SET` or ``0`` -- start of the stream (the default); *offset* should be zero or positive @@ -783,10 +783,11 @@ If *size* is specified, at most *size* characters will be read. - .. method:: seek(offset, whence=SEEK_SET) + .. method:: seek(offset[, whence]) - Change the stream position to the given *offset*. Behaviour depends - on the *whence* parameter: + Change the stream position to the given *offset*. Behaviour depends on + the *whence* parameter. The default value for *whence* is + :data:`SEEK_SET`. * :data:`SEEK_SET` or ``0``: seek from the start of the stream (the default); *offset* must either be a number returned by diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -1312,6 +1312,7 @@ Pete Shinners Michael Shiplett John W. Shipman +Shiyao Ma Alex Shkop Joel Shprentz Yue Shuaijie -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 11 06:40:48 2015 From: python-checkins at python.org (martin.panter) Date: Fri, 11 Sep 2015 04:40:48 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzI1MDMw?= =?utf-8?q?=3A_Do_not_document_seek=28=29_as_if_it_accepts_keyword_argumen?= =?utf-8?q?ts?= Message-ID: <20150911044048.68887.9708@psf.io> https://hg.python.org/cpython/rev/77784422da4d changeset: 97902:77784422da4d branch: 2.7 parent: 97896:1a52db3ef0e8 user: Martin Panter date: Fri Sep 11 03:58:30 2015 +0000 summary: Issue #25030: Do not document seek() as if it accepts keyword arguments Patch from Shiyao Ma. files: Doc/library/io.rst | 13 +++++++------ Misc/ACKS | 1 + Modules/_io/bytesio.c | 2 +- 3 files changed, 9 insertions(+), 7 deletions(-) diff --git a/Doc/library/io.rst b/Doc/library/io.rst --- a/Doc/library/io.rst +++ b/Doc/library/io.rst @@ -299,11 +299,11 @@ Note that it's already possible to iterate on file objects using ``for line in file: ...`` without calling ``file.readlines()``. - .. method:: seek(offset, whence=SEEK_SET) + .. method:: seek(offset[, whence]) Change the stream position to the given byte *offset*. *offset* is - interpreted relative to the position indicated by *whence*. Values for - *whence* are: + interpreted relative to the position indicated by *whence*. The default + value for *whence* is :data:`SEEK_SET`. Values for *whence* are: * :data:`SEEK_SET` or ``0`` -- start of the stream (the default); *offset* should be zero or positive @@ -716,10 +716,11 @@ If *limit* is specified, at most *limit* characters will be read. - .. method:: seek(offset, whence=SEEK_SET) + .. method:: seek(offset[, whence]) - Change the stream position to the given *offset*. Behaviour depends - on the *whence* parameter: + Change the stream position to the given *offset*. Behaviour depends on + the *whence* parameter. The default value for *whence* is + :data:`SEEK_SET`. * :data:`SEEK_SET` or ``0``: seek from the start of the stream (the default); *offset* must either be a number returned by diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -1257,6 +1257,7 @@ Pete Shinners Michael Shiplett John W. Shipman +Shiyao Ma Joel Shprentz Yue Shuaijie Terrel Shumway diff --git a/Modules/_io/bytesio.c b/Modules/_io/bytesio.c --- a/Modules/_io/bytesio.c +++ b/Modules/_io/bytesio.c @@ -490,7 +490,7 @@ } PyDoc_STRVAR(seek_doc, -"seek(pos, whence=0) -> int. Change stream position.\n" +"seek(pos[, whence]) -> int. Change stream position.\n" "\n" "Seek to byte offset pos relative to position indicated by whence:\n" " 0 Start of stream (the default). pos should be >= 0;\n" -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 11 06:40:48 2015 From: python-checkins at python.org (martin.panter) Date: Fri, 11 Sep 2015 04:40:48 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzI1MDMw?= =?utf-8?q?=3A_Do_not_document_seek=28=29_as_if_it_accepts_keyword_argumen?= =?utf-8?q?ts?= Message-ID: <20150911044048.114719.48352@psf.io> https://hg.python.org/cpython/rev/0ca216f5276e changeset: 97901:0ca216f5276e branch: 3.4 parent: 97893:8ba735b018d2 user: Martin Panter date: Fri Sep 11 03:58:30 2015 +0000 summary: Issue #25030: Do not document seek() as if it accepts keyword arguments Patch from Shiyao Ma. files: Doc/library/io.rst | 13 +++++++------ Misc/ACKS | 1 + Modules/_io/bytesio.c | 2 +- 3 files changed, 9 insertions(+), 7 deletions(-) diff --git a/Doc/library/io.rst b/Doc/library/io.rst --- a/Doc/library/io.rst +++ b/Doc/library/io.rst @@ -301,11 +301,11 @@ Note that it's already possible to iterate on file objects using ``for line in file: ...`` without calling ``file.readlines()``. - .. method:: seek(offset, whence=SEEK_SET) + .. method:: seek(offset[, whence]) Change the stream position to the given byte *offset*. *offset* is - interpreted relative to the position indicated by *whence*. Values for - *whence* are: + interpreted relative to the position indicated by *whence*. The default + value for *whence* is :data:`SEEK_SET`. Values for *whence* are: * :data:`SEEK_SET` or ``0`` -- start of the stream (the default); *offset* should be zero or positive @@ -760,10 +760,11 @@ If *size* is specified, at most *size* characters will be read. - .. method:: seek(offset, whence=SEEK_SET) + .. method:: seek(offset[, whence]) - Change the stream position to the given *offset*. Behaviour depends - on the *whence* parameter: + Change the stream position to the given *offset*. Behaviour depends on + the *whence* parameter. The default value for *whence* is + :data:`SEEK_SET`. * :data:`SEEK_SET` or ``0``: seek from the start of the stream (the default); *offset* must either be a number returned by diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -1276,6 +1276,7 @@ Pete Shinners Michael Shiplett John W. Shipman +Shiyao Ma Alex Shkop Joel Shprentz Yue Shuaijie diff --git a/Modules/_io/bytesio.c b/Modules/_io/bytesio.c --- a/Modules/_io/bytesio.c +++ b/Modules/_io/bytesio.c @@ -530,7 +530,7 @@ } PyDoc_STRVAR(seek_doc, -"seek(pos, whence=0) -> int. Change stream position.\n" +"seek(pos[, whence]) -> int. Change stream position.\n" "\n" "Seek to byte offset pos relative to position indicated by whence:\n" " 0 Start of stream (the default). pos should be >= 0;\n" -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 11 06:40:48 2015 From: python-checkins at python.org (martin.panter) Date: Fri, 11 Sep 2015 04:40:48 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Issue_=2325030=3A_Merge_seek=28=29_doc_fixes_from_3=2E4_into_3?= =?utf-8?q?=2E5?= Message-ID: <20150911044048.12019.29532@psf.io> https://hg.python.org/cpython/rev/a0c6f5358029 changeset: 97903:a0c6f5358029 branch: 3.5 parent: 97899:6c1b0dd07340 parent: 97901:0ca216f5276e user: Martin Panter date: Fri Sep 11 04:39:31 2015 +0000 summary: Issue #25030: Merge seek() doc fixes from 3.4 into 3.5 files: Doc/library/io.rst | 13 +++++++------ Misc/ACKS | 1 + 2 files changed, 8 insertions(+), 6 deletions(-) diff --git a/Doc/library/io.rst b/Doc/library/io.rst --- a/Doc/library/io.rst +++ b/Doc/library/io.rst @@ -301,11 +301,11 @@ Note that it's already possible to iterate on file objects using ``for line in file: ...`` without calling ``file.readlines()``. - .. method:: seek(offset, whence=SEEK_SET) + .. method:: seek(offset[, whence]) Change the stream position to the given byte *offset*. *offset* is - interpreted relative to the position indicated by *whence*. Values for - *whence* are: + interpreted relative to the position indicated by *whence*. The default + value for *whence* is :data:`SEEK_SET`. Values for *whence* are: * :data:`SEEK_SET` or ``0`` -- start of the stream (the default); *offset* should be zero or positive @@ -783,10 +783,11 @@ If *size* is specified, at most *size* characters will be read. - .. method:: seek(offset, whence=SEEK_SET) + .. method:: seek(offset[, whence]) - Change the stream position to the given *offset*. Behaviour depends - on the *whence* parameter: + Change the stream position to the given *offset*. Behaviour depends on + the *whence* parameter. The default value for *whence* is + :data:`SEEK_SET`. * :data:`SEEK_SET` or ``0``: seek from the start of the stream (the default); *offset* must either be a number returned by diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -1312,6 +1312,7 @@ Pete Shinners Michael Shiplett John W. Shipman +Shiyao Ma Alex Shkop Joel Shprentz Yue Shuaijie -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 11 06:49:23 2015 From: python-checkins at python.org (yury.selivanov) Date: Fri, 11 Sep 2015 04:49:23 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogd2hhdHNuZXcvMy41?= =?utf-8?q?=3A_Sync_whatsnew_with_versionadded/versionchanged_doc_tags?= Message-ID: <20150911044923.15726.4518@psf.io> https://hg.python.org/cpython/rev/d6436b84f3a4 changeset: 97905:d6436b84f3a4 branch: 3.5 parent: 97903:a0c6f5358029 user: Yury Selivanov date: Fri Sep 11 00:48:21 2015 -0400 summary: whatsnew/3.5: Sync whatsnew with versionadded/versionchanged doc tags files: Doc/library/enum.rst | 2 +- Doc/library/io.rst | 2 +- Doc/whatsnew/3.5.rst | 81 ++++++++++++++++++++++++++++++- 3 files changed, 79 insertions(+), 6 deletions(-) diff --git a/Doc/library/enum.rst b/Doc/library/enum.rst --- a/Doc/library/enum.rst +++ b/Doc/library/enum.rst @@ -466,7 +466,7 @@ :type: type to mix in to new Enum class. -:start: number to start counting at if only names are passed in +:start: number to start counting at if only names are passed in. .. versionchanged:: 3.5 The *start* parameter was added. diff --git a/Doc/library/io.rst b/Doc/library/io.rst --- a/Doc/library/io.rst +++ b/Doc/library/io.rst @@ -487,7 +487,7 @@ .. method:: readinto1(b) - Read up to ``len(b)`` bytes into bytearray *b*, ,using at most one call to + Read up to ``len(b)`` bytes into bytearray *b*, using at most one call to the underlying raw stream's :meth:`~RawIOBase.read` (or :meth:`~RawIOBase.readinto`) method. Return the number of bytes read. diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -87,13 +87,19 @@ ``memoryview(b'\xf0\x9f\x90\x8d').hex()``: :issue:`9951` - A ``hex`` method has been added to bytes, bytearray, and memoryview. +* :class:`memoryview` (including multi-dimensional) now supports tuple indexing. + (Contributed by Antoine Pitrou in :issue:`23632`.) + * Generators have new ``gi_yieldfrom`` attribute, which returns the object being iterated by ``yield from`` expressions. (Contributed by Benno Leslie and Yury Selivanov in :issue:`24450`.) -* New :exc:`RecursionError` exception. (Contributed by Georg Brandl +* New :exc:`RecursionError` exception. (Contributed by Georg Brandl in :issue:`19235`.) +* New :exc:`StopAsyncIteration` exception. (Contributed by + Yury Selivanov in :issue:`24017`. See also :pep:`492`.) + CPython implementation improvements: * When the ``LC_TYPE`` locale is the POSIX locale (``C`` locale), @@ -813,6 +819,21 @@ :issue:`24211`.) +enum +---- + +The :class:`~enum.Enum` callable has a new parameter *start* to +specify the initial number of enum values if only *names* are provided:: + + >>> Animal = enum.Enum('Animal', 'cat dog', start=10) + >>> Animal.cat + + >>> Animal.dog + + +(Contributed by Ethan Furman in :issue:`21706`.) + + faulthandler ------------ @@ -848,6 +869,14 @@ comparison. (Contributed by Raymond Hettinger in :issue:`13742`.) +http +---- + +A new :class:`HTTPStatus ` enum that defines a set of +HTTP status codes, reason phrases and long descriptions written in English. +(Contributed by Demian Brecht in :issue:`21793`.) + + idlelib and IDLE ---------------- @@ -944,6 +973,16 @@ (Contributed by Daniel Shahaf in :issue:`16808`.) +io +-- + +A new :meth:`BufferedIOBase.readinto1 ` +method, that uses at most one call to the underlying raw stream's +:meth:`RawIOBase.read ` (or +:meth:`RawIOBase.readinto `) method. +(Contributed by Nikolaus Rath in :issue:`20578`.) + + ipaddress --------- @@ -1028,6 +1067,10 @@ and :func:`~operator.methodcaller` objects now support pickling. (Contributed by Josh Rosenberg and Serhiy Storchaka in :issue:`22955`.) +New :func:`~operator.matmul` and :func:`~operator.imatmul` functions +to perform matrix multiplication. +(Contributed by Benjamin Peterson in :issue:`21176`.) + os -- @@ -1084,6 +1127,13 @@ directory. (Contributed by Victor Salgado and Mayank Tripathi in :issue:`19777`.) +New :meth:`Path.write_text `, +:meth:`Path.read_text `, +:meth:`Path.write_bytes `, +:meth:`Path.read_bytes ` methods to simplify +read/write operations on files. +(Contributed by Christopher Welborn in :issue:`20218`.) + pickle ------ @@ -1131,7 +1181,8 @@ selectors --------- -The module now supports efficient ``/dev/poll`` on Solaris. +The new :class:`~selectors.DevpollSelector` supports efficient +``/dev/poll`` polling on Solaris. (Contributed by Giampaolo Rodola' in :issue:`18931`.) @@ -1642,6 +1693,28 @@ (Contributed by Victor Stinner in :issue:`18395`.) +New :c:func:`PyCodec_NameReplaceErrors` function to replace the unicode +encode error with ``\N{...}`` escapes. +(Contributed by Serhiy Storchaka in :issue:`19676`.) + +New :c:func:`PyErr_FormatV` similar to :c:func:`PyErr_Format`, +but accepts a ``va_list`` argument. +(Contributed by Antoine Pitrou in :issue:`18711`.) + +New :c:data:`PyExc_RecursionError` exception. +(Contributed by Georg Brandl in :issue:`19235`.) + +New :c:func:`PyModule_FromDefAndSpec`, :c:func:`PyModule_FromDefAndSpec2`, +and :c:func:`PyModule_ExecDef` introduced by :pep:`489` -- multi-phase +extension module initialization. +(Contributed by Petr Viktorin in :issue:`24268`.) + +New :c:func:`PyNumber_MatrixMultiply` and +:c:func:`PyNumber_InPlaceMatrixMultiply` functions to perform matrix +multiplication. +(Contributed by Benjamin Peterson in :issue:`21176`. See also :pep:`465` +for details.) + The :c:member:`PyTypeObject.tp_finalize` slot is now part of stable ABI. Windows builds now require Microsoft Visual C++ 14.0, which @@ -1878,8 +1951,8 @@ in Python 3.5, all old `.pyo` files from previous versions of Python are invalid regardless of this PEP. -* The :mod:`socket` module now exports the CAN_RAW_FD_FRAMES constant on linux - 3.6 and greater. +* The :mod:`socket` module now exports the :data:`~socket.CAN_RAW_FD_FRAMES` + constant on linux 3.6 and greater. * The :func:`~ssl.cert_time_to_seconds` function now interprets the input time as UTC and not as local time, per :rfc:`5280`. Additionally, the return -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 11 06:49:23 2015 From: python-checkins at python.org (yury.selivanov) Date: Fri, 11 Sep 2015 04:49:23 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogTWVyZ2UgMy41?= Message-ID: <20150911044923.15724.49984@psf.io> https://hg.python.org/cpython/rev/c69553840474 changeset: 97906:c69553840474 parent: 97904:c4bb0da8b45e parent: 97905:d6436b84f3a4 user: Yury Selivanov date: Fri Sep 11 00:48:45 2015 -0400 summary: Merge 3.5 files: Doc/library/enum.rst | 2 +- Doc/library/io.rst | 2 +- Doc/whatsnew/3.5.rst | 81 ++++++++++++++++++++++++++++++- 3 files changed, 79 insertions(+), 6 deletions(-) diff --git a/Doc/library/enum.rst b/Doc/library/enum.rst --- a/Doc/library/enum.rst +++ b/Doc/library/enum.rst @@ -466,7 +466,7 @@ :type: type to mix in to new Enum class. -:start: number to start counting at if only names are passed in +:start: number to start counting at if only names are passed in. .. versionchanged:: 3.5 The *start* parameter was added. diff --git a/Doc/library/io.rst b/Doc/library/io.rst --- a/Doc/library/io.rst +++ b/Doc/library/io.rst @@ -487,7 +487,7 @@ .. method:: readinto1(b) - Read up to ``len(b)`` bytes into bytearray *b*, ,using at most one call to + Read up to ``len(b)`` bytes into bytearray *b*, using at most one call to the underlying raw stream's :meth:`~RawIOBase.read` (or :meth:`~RawIOBase.readinto`) method. Return the number of bytes read. diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -87,13 +87,19 @@ ``memoryview(b'\xf0\x9f\x90\x8d').hex()``: :issue:`9951` - A ``hex`` method has been added to bytes, bytearray, and memoryview. +* :class:`memoryview` (including multi-dimensional) now supports tuple indexing. + (Contributed by Antoine Pitrou in :issue:`23632`.) + * Generators have new ``gi_yieldfrom`` attribute, which returns the object being iterated by ``yield from`` expressions. (Contributed by Benno Leslie and Yury Selivanov in :issue:`24450`.) -* New :exc:`RecursionError` exception. (Contributed by Georg Brandl +* New :exc:`RecursionError` exception. (Contributed by Georg Brandl in :issue:`19235`.) +* New :exc:`StopAsyncIteration` exception. (Contributed by + Yury Selivanov in :issue:`24017`. See also :pep:`492`.) + CPython implementation improvements: * When the ``LC_TYPE`` locale is the POSIX locale (``C`` locale), @@ -813,6 +819,21 @@ :issue:`24211`.) +enum +---- + +The :class:`~enum.Enum` callable has a new parameter *start* to +specify the initial number of enum values if only *names* are provided:: + + >>> Animal = enum.Enum('Animal', 'cat dog', start=10) + >>> Animal.cat + + >>> Animal.dog + + +(Contributed by Ethan Furman in :issue:`21706`.) + + faulthandler ------------ @@ -848,6 +869,14 @@ comparison. (Contributed by Raymond Hettinger in :issue:`13742`.) +http +---- + +A new :class:`HTTPStatus ` enum that defines a set of +HTTP status codes, reason phrases and long descriptions written in English. +(Contributed by Demian Brecht in :issue:`21793`.) + + idlelib and IDLE ---------------- @@ -944,6 +973,16 @@ (Contributed by Daniel Shahaf in :issue:`16808`.) +io +-- + +A new :meth:`BufferedIOBase.readinto1 ` +method, that uses at most one call to the underlying raw stream's +:meth:`RawIOBase.read ` (or +:meth:`RawIOBase.readinto `) method. +(Contributed by Nikolaus Rath in :issue:`20578`.) + + ipaddress --------- @@ -1028,6 +1067,10 @@ and :func:`~operator.methodcaller` objects now support pickling. (Contributed by Josh Rosenberg and Serhiy Storchaka in :issue:`22955`.) +New :func:`~operator.matmul` and :func:`~operator.imatmul` functions +to perform matrix multiplication. +(Contributed by Benjamin Peterson in :issue:`21176`.) + os -- @@ -1084,6 +1127,13 @@ directory. (Contributed by Victor Salgado and Mayank Tripathi in :issue:`19777`.) +New :meth:`Path.write_text `, +:meth:`Path.read_text `, +:meth:`Path.write_bytes `, +:meth:`Path.read_bytes ` methods to simplify +read/write operations on files. +(Contributed by Christopher Welborn in :issue:`20218`.) + pickle ------ @@ -1131,7 +1181,8 @@ selectors --------- -The module now supports efficient ``/dev/poll`` on Solaris. +The new :class:`~selectors.DevpollSelector` supports efficient +``/dev/poll`` polling on Solaris. (Contributed by Giampaolo Rodola' in :issue:`18931`.) @@ -1642,6 +1693,28 @@ (Contributed by Victor Stinner in :issue:`18395`.) +New :c:func:`PyCodec_NameReplaceErrors` function to replace the unicode +encode error with ``\N{...}`` escapes. +(Contributed by Serhiy Storchaka in :issue:`19676`.) + +New :c:func:`PyErr_FormatV` similar to :c:func:`PyErr_Format`, +but accepts a ``va_list`` argument. +(Contributed by Antoine Pitrou in :issue:`18711`.) + +New :c:data:`PyExc_RecursionError` exception. +(Contributed by Georg Brandl in :issue:`19235`.) + +New :c:func:`PyModule_FromDefAndSpec`, :c:func:`PyModule_FromDefAndSpec2`, +and :c:func:`PyModule_ExecDef` introduced by :pep:`489` -- multi-phase +extension module initialization. +(Contributed by Petr Viktorin in :issue:`24268`.) + +New :c:func:`PyNumber_MatrixMultiply` and +:c:func:`PyNumber_InPlaceMatrixMultiply` functions to perform matrix +multiplication. +(Contributed by Benjamin Peterson in :issue:`21176`. See also :pep:`465` +for details.) + The :c:member:`PyTypeObject.tp_finalize` slot is now part of stable ABI. Windows builds now require Microsoft Visual C++ 14.0, which @@ -1878,8 +1951,8 @@ in Python 3.5, all old `.pyo` files from previous versions of Python are invalid regardless of this PEP. -* The :mod:`socket` module now exports the CAN_RAW_FD_FRAMES constant on linux - 3.6 and greater. +* The :mod:`socket` module now exports the :data:`~socket.CAN_RAW_FD_FRAMES` + constant on linux 3.6 and greater. * The :func:`~ssl.cert_time_to_seconds` function now interprets the input time as UTC and not as local time, per :rfc:`5280`. Additionally, the return -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 11 06:51:30 2015 From: python-checkins at python.org (yury.selivanov) Date: Fri, 11 Sep 2015 04:51:30 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogd2hhdHNuZXcvMy41?= =?utf-8?q?=3A_Drop_empty_section?= Message-ID: <20150911045130.17963.18060@psf.io> https://hg.python.org/cpython/rev/6e93dd5e880e changeset: 97907:6e93dd5e880e branch: 3.5 parent: 97905:d6436b84f3a4 user: Yury Selivanov date: Fri Sep 11 00:50:39 2015 -0400 summary: whatsnew/3.5: Drop empty section files: Doc/whatsnew/3.5.rst | 7 ------- 1 files changed, 0 insertions(+), 7 deletions(-) diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -1832,13 +1832,6 @@ Use of ``re.LOCALE`` flag with str patterns or ``re.ASCII`` is now deprecated. (Contributed by Serhiy Storchaka in :issue:`22407`.) -.. XXX: - - Deprecated functions and types of the C API - ------------------------------------------- - - * None yet. - Removed ======= -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 11 06:51:30 2015 From: python-checkins at python.org (yury.selivanov) Date: Fri, 11 Sep 2015 04:51:30 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogTWVyZ2UgMy41?= Message-ID: <20150911045130.17983.48419@psf.io> https://hg.python.org/cpython/rev/f7725d83cdb6 changeset: 97908:f7725d83cdb6 parent: 97906:c69553840474 parent: 97907:6e93dd5e880e user: Yury Selivanov date: Fri Sep 11 00:50:54 2015 -0400 summary: Merge 3.5 files: Doc/whatsnew/3.5.rst | 7 ------- 1 files changed, 0 insertions(+), 7 deletions(-) diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -1832,13 +1832,6 @@ Use of ``re.LOCALE`` flag with str patterns or ``re.ASCII`` is now deprecated. (Contributed by Serhiy Storchaka in :issue:`22407`.) -.. XXX: - - Deprecated functions and types of the C API - ------------------------------------------- - - * None yet. - Removed ======= -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 11 07:23:51 2015 From: python-checkins at python.org (yury.selivanov) Date: Fri, 11 Sep 2015 05:23:51 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogTWVyZ2UgMy41?= Message-ID: <20150911052351.68861.81924@psf.io> https://hg.python.org/cpython/rev/1c94ab9c5ecf changeset: 97910:1c94ab9c5ecf parent: 97908:f7725d83cdb6 parent: 97909:1639f5c1ac87 user: Yury Selivanov date: Fri Sep 11 01:23:31 2015 -0400 summary: Merge 3.5 files: Doc/whatsnew/3.5.rst | 16 ++++++++-------- 1 files changed, 8 insertions(+), 8 deletions(-) diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -678,8 +678,8 @@ makes it 4 to 100 times faster. (Contributed by Eric Snow in :issue:`16991`.) :meth:`OrderedDict.items `, -:meth:`OrderedDict.items `, -:meth:`OrderedDict.items ` views now support +:meth:`OrderedDict.keys `, +:meth:`OrderedDict.values ` views now support :func:`reversed` iteration. (Contributed by Serhiy Storchaka in :issue:`19505`.) @@ -991,7 +991,7 @@ network objects from existing addresses. (Contributed by Peter Moody and Antoine Pitrou in :issue:`16531`.) -New :attr:`~ipaddress.IPv4Network.reverse_pointer>` attribute for +A new :attr:`~ipaddress.IPv4Network.reverse_pointer>` attribute for :class:`~ipaddress.IPv4Network` and :class:`~ipaddress.IPv6Network` classes returns the name of the reverse DNS PTR record. (Contributed by Leon Weber in :issue:`20480`.) @@ -1450,7 +1450,7 @@ timeit ------ -New command line option ``-u`` or ``--unit=U`` can be used to specify the time +A new command line option ``-u`` or ``--unit=U`` can be used to specify the time unit for the timer output. Supported options are ``usec``, ``msec``, or ``sec``. (Contributed by Julian Gindi in :issue:`18983`.) @@ -1535,7 +1535,7 @@ unittest -------- -New command line option ``--locals`` to show local variables in +A new command line option ``--locals`` to show local variables in tracebacks. (Contributed by Robert Collins in :issue:`22936`.) @@ -1693,15 +1693,15 @@ (Contributed by Victor Stinner in :issue:`18395`.) -New :c:func:`PyCodec_NameReplaceErrors` function to replace the unicode +A new :c:func:`PyCodec_NameReplaceErrors` function to replace the unicode encode error with ``\N{...}`` escapes. (Contributed by Serhiy Storchaka in :issue:`19676`.) -New :c:func:`PyErr_FormatV` similar to :c:func:`PyErr_Format`, +A new :c:func:`PyErr_FormatV` function similar to :c:func:`PyErr_Format`, but accepts a ``va_list`` argument. (Contributed by Antoine Pitrou in :issue:`18711`.) -New :c:data:`PyExc_RecursionError` exception. +A new :c:data:`PyExc_RecursionError` exception. (Contributed by Georg Brandl in :issue:`19235`.) New :c:func:`PyModule_FromDefAndSpec`, :c:func:`PyModule_FromDefAndSpec2`, -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 11 07:23:51 2015 From: python-checkins at python.org (yury.selivanov) Date: Fri, 11 Sep 2015 05:23:51 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogd2hhdHNuZXcvMy41?= =?utf-8?q?=3A_Fix_nits?= Message-ID: <20150911052350.68879.41217@psf.io> https://hg.python.org/cpython/rev/1639f5c1ac87 changeset: 97909:1639f5c1ac87 branch: 3.5 parent: 97907:6e93dd5e880e user: Yury Selivanov date: Fri Sep 11 01:23:10 2015 -0400 summary: whatsnew/3.5: Fix nits files: Doc/whatsnew/3.5.rst | 16 ++++++++-------- 1 files changed, 8 insertions(+), 8 deletions(-) diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -678,8 +678,8 @@ makes it 4 to 100 times faster. (Contributed by Eric Snow in :issue:`16991`.) :meth:`OrderedDict.items `, -:meth:`OrderedDict.items `, -:meth:`OrderedDict.items ` views now support +:meth:`OrderedDict.keys `, +:meth:`OrderedDict.values ` views now support :func:`reversed` iteration. (Contributed by Serhiy Storchaka in :issue:`19505`.) @@ -991,7 +991,7 @@ network objects from existing addresses. (Contributed by Peter Moody and Antoine Pitrou in :issue:`16531`.) -New :attr:`~ipaddress.IPv4Network.reverse_pointer>` attribute for +A new :attr:`~ipaddress.IPv4Network.reverse_pointer>` attribute for :class:`~ipaddress.IPv4Network` and :class:`~ipaddress.IPv6Network` classes returns the name of the reverse DNS PTR record. (Contributed by Leon Weber in :issue:`20480`.) @@ -1450,7 +1450,7 @@ timeit ------ -New command line option ``-u`` or ``--unit=U`` can be used to specify the time +A new command line option ``-u`` or ``--unit=U`` can be used to specify the time unit for the timer output. Supported options are ``usec``, ``msec``, or ``sec``. (Contributed by Julian Gindi in :issue:`18983`.) @@ -1535,7 +1535,7 @@ unittest -------- -New command line option ``--locals`` to show local variables in +A new command line option ``--locals`` to show local variables in tracebacks. (Contributed by Robert Collins in :issue:`22936`.) @@ -1693,15 +1693,15 @@ (Contributed by Victor Stinner in :issue:`18395`.) -New :c:func:`PyCodec_NameReplaceErrors` function to replace the unicode +A new :c:func:`PyCodec_NameReplaceErrors` function to replace the unicode encode error with ``\N{...}`` escapes. (Contributed by Serhiy Storchaka in :issue:`19676`.) -New :c:func:`PyErr_FormatV` similar to :c:func:`PyErr_Format`, +A new :c:func:`PyErr_FormatV` function similar to :c:func:`PyErr_Format`, but accepts a ``va_list`` argument. (Contributed by Antoine Pitrou in :issue:`18711`.) -New :c:data:`PyExc_RecursionError` exception. +A new :c:data:`PyExc_RecursionError` exception. (Contributed by Georg Brandl in :issue:`19235`.) New :c:func:`PyModule_FromDefAndSpec`, :c:func:`PyModule_FromDefAndSpec2`, -- Repository URL: https://hg.python.org/cpython From solipsis at pitrou.net Fri Sep 11 10:47:15 2015 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Fri, 11 Sep 2015 08:47:15 +0000 Subject: [Python-checkins] Daily reference leaks (1c94ab9c5ecf): sum=18942 Message-ID: <20150911084715.15718.84096@psf.io> results for 1c94ab9c5ecf on branch "default" -------------------------------------------- test_capi leaked [1598, 1598, 1598] references, sum=4794 test_capi leaked [387, 389, 389] memory blocks, sum=1165 test_deque leaked [257, 257, 257] references, sum=771 test_deque leaked [79, 80, 80] memory blocks, sum=239 test_functools leaked [0, 2, 2] memory blocks, sum=4 test_multiprocessing_forkserver leaked [0, 38, 0] references, sum=38 test_multiprocessing_forkserver leaked [0, 17, 0] memory blocks, sum=17 test_threading leaked [3196, 3196, 3196] references, sum=9588 test_threading leaked [774, 776, 776] memory blocks, sum=2326 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/psf-users/antoine/refleaks/reflogxAVMGq', '--timeout', '7200'] From python-checkins at python.org Fri Sep 11 12:43:07 2015 From: python-checkins at python.org (victor.stinner) Date: Fri, 11 Sep 2015 10:43:07 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogTWVyZ2UgMy41?= Message-ID: <20150911104307.27703.86119@psf.io> https://hg.python.org/cpython/rev/60e4ae9e2e01 changeset: 97913:60e4ae9e2e01 parent: 97910:1c94ab9c5ecf parent: 97912:7bf325941636 user: Victor Stinner date: Fri Sep 11 12:38:27 2015 +0200 summary: Merge 3.5 files: Misc/NEWS | 6 ++++++ Modules/socketmodule.c | 4 +--- 2 files changed, 7 insertions(+), 3 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -103,6 +103,12 @@ Library ------- +- Issue #24684: socket.socket.getaddrinfo() now calls + PyUnicode_AsEncodedString() instead of calling the encode() method of the + host, to handle correctly custom string with an encode() method which doesn't + return a byte string. The encoder of the IDNA codec is now called directly + instead of calling the encode() method of the string. + - Issue #25060: Correctly compute stack usage of the BUILD_MAP opcode. - Issue #24857: Comparing call_args to a long sequence now correctly returns a diff --git a/Modules/socketmodule.c b/Modules/socketmodule.c --- a/Modules/socketmodule.c +++ b/Modules/socketmodule.c @@ -5513,9 +5513,7 @@ if (hobj == Py_None) { hptr = NULL; } else if (PyUnicode_Check(hobj)) { - _Py_IDENTIFIER(encode); - - idna = _PyObject_CallMethodId(hobj, &PyId_encode, "s", "idna"); + idna = PyUnicode_AsEncodedString(hobj, "idna", NULL); if (!idna) return NULL; assert(PyBytes_Check(idna)); -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 11 12:43:07 2015 From: python-checkins at python.org (victor.stinner) Date: Fri, 11 Sep 2015 10:43:07 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Merge_3=2E4?= Message-ID: <20150911104307.27707.16952@psf.io> https://hg.python.org/cpython/rev/7bf325941636 changeset: 97912:7bf325941636 branch: 3.5 parent: 97909:1639f5c1ac87 parent: 97911:2bff115e6ba0 user: Victor Stinner date: Fri Sep 11 12:38:17 2015 +0200 summary: Merge 3.4 files: Misc/NEWS | 6 ++++++ Modules/socketmodule.c | 4 +--- 2 files changed, 7 insertions(+), 3 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -14,6 +14,12 @@ Library ------- +- Issue #24684: socket.socket.getaddrinfo() now calls + PyUnicode_AsEncodedString() instead of calling the encode() method of the + host, to handle correctly custom string with an encode() method which doesn't + return a byte string. The encoder of the IDNA codec is now called directly + instead of calling the encode() method of the string. + - Issue #25060: Correctly compute stack usage of the BUILD_MAP opcode. - Issue #24857: Comparing call_args to a long sequence now correctly returns a diff --git a/Modules/socketmodule.c b/Modules/socketmodule.c --- a/Modules/socketmodule.c +++ b/Modules/socketmodule.c @@ -5513,9 +5513,7 @@ if (hobj == Py_None) { hptr = NULL; } else if (PyUnicode_Check(hobj)) { - _Py_IDENTIFIER(encode); - - idna = _PyObject_CallMethodId(hobj, &PyId_encode, "s", "idna"); + idna = PyUnicode_AsEncodedString(hobj, "idna", NULL); if (!idna) return NULL; assert(PyBytes_Check(idna)); -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 11 12:43:07 2015 From: python-checkins at python.org (victor.stinner) Date: Fri, 11 Sep 2015 10:43:07 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzI0Njg0?= =?utf-8?q?=3A_socket=2Esocket=2Egetaddrinfo=28=29_now_calls?= Message-ID: <20150911104306.11260.4169@psf.io> https://hg.python.org/cpython/rev/2bff115e6ba0 changeset: 97911:2bff115e6ba0 branch: 3.4 parent: 97901:0ca216f5276e user: Victor Stinner date: Fri Sep 11 12:37:30 2015 +0200 summary: Issue #24684: socket.socket.getaddrinfo() now calls PyUnicode_AsEncodedString() instead of calling the encode() method of the host, to handle correctly custom string with an encode() method which doesn't return a byte string. The encoder of the IDNA codec is now called directly instead of calling the encode() method of the string. files: Misc/NEWS | 6 ++++++ Modules/socketmodule.c | 4 +--- 2 files changed, 7 insertions(+), 3 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -81,6 +81,12 @@ Library ------- +- Issue #24684: socket.socket.getaddrinfo() now calls + PyUnicode_AsEncodedString() instead of calling the encode() method of the + host, to handle correctly custom string with an encode() method which doesn't + return a byte string. The encoder of the IDNA codec is now called directly + instead of calling the encode() method of the string. + - Issue #24982: shutil.make_archive() with the "zip" format now adds entries for directories (including empty directories) in ZIP file. diff --git a/Modules/socketmodule.c b/Modules/socketmodule.c --- a/Modules/socketmodule.c +++ b/Modules/socketmodule.c @@ -5213,9 +5213,7 @@ if (hobj == Py_None) { hptr = NULL; } else if (PyUnicode_Check(hobj)) { - _Py_IDENTIFIER(encode); - - idna = _PyObject_CallMethodId(hobj, &PyId_encode, "s", "idna"); + idna = PyUnicode_AsEncodedString(hobj, "idna", NULL); if (!idna) return NULL; assert(PyBytes_Check(idna)); -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 11 12:43:08 2015 From: python-checkins at python.org (victor.stinner) Date: Fri, 11 Sep 2015 10:43:08 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzI0Njg0?= =?utf-8?q?=3A_socket=2Esocket=2Egetaddrinfo=28=29_now_calls?= Message-ID: <20150911104307.15732.24250@psf.io> https://hg.python.org/cpython/rev/0c13674cf8b5 changeset: 97914:0c13674cf8b5 branch: 2.7 parent: 97902:77784422da4d user: Victor Stinner date: Fri Sep 11 12:42:13 2015 +0200 summary: Issue #24684: socket.socket.getaddrinfo() now calls PyUnicode_AsEncodedString() instead of calling the encode() method of the host, to handle correctly custom unicode string with an encode() method which doesn't return a byte string. The encoder of the IDNA codec is now called directly instead of calling the encode() method of the string. files: Misc/NEWS | 6 ++++++ Modules/socketmodule.c | 2 +- 2 files changed, 7 insertions(+), 1 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -37,6 +37,12 @@ Library ------- +- Issue #24684: socket.socket.getaddrinfo() now calls + PyUnicode_AsEncodedString() instead of calling the encode() method of the + host, to handle correctly custom unicode string with an encode() method + which doesn't return a byte string. The encoder of the IDNA codec is now + called directly instead of calling the encode() method of the string. + - Issue #24982: shutil.make_archive() with the "zip" format now adds entries for directories (including empty directories) in ZIP file. diff --git a/Modules/socketmodule.c b/Modules/socketmodule.c --- a/Modules/socketmodule.c +++ b/Modules/socketmodule.c @@ -4158,7 +4158,7 @@ if (hobj == Py_None) { hptr = NULL; } else if (PyUnicode_Check(hobj)) { - idna = PyObject_CallMethod(hobj, "encode", "s", "idna"); + idna = PyUnicode_AsEncodedString(hobj, "idna", NULL); if (!idna) return NULL; hptr = PyString_AsString(idna); -- Repository URL: https://hg.python.org/cpython From lp_benchmark_robot at intel.com Fri Sep 11 15:55:15 2015 From: lp_benchmark_robot at intel.com (lp_benchmark_robot) Date: Fri, 11 Sep 2015 13:55:15 +0000 Subject: [Python-checkins] Benchmark Results for Python 2.7 2015-09-11 Message-ID: <078AA0FFE8C7034097F90205717F504611D7BE52@IRSMSX102.ger.corp.intel.com> Results for project python_2.7-nightly, build date 2015-09-11 12:13:33 commit: 0c13674cf8b5347914b6b323c57b167e55906825 revision date: 2015-09-11 10:42:13 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v2.7.10, with hash 15c95b7d81dcf821daade360741e00714667653f from 2015-05-23 16:02:14+00:00 ------------------------------------------------------------------------------------------ benchmark relative change since change since current rev with std_dev* last run v2.7.10 regrtest PGO ------------------------------------------------------------------------------------------ :-) django_v2 0.19705% -0.45617% 4.33531% 8.45204% :-) pybench 0.15930% -0.04829% 6.70894% 6.86160% :-| regex_v8 0.56380% -0.08838% -1.16266% 7.97516% :-) nbody 0.13309% -0.05918% 9.05320% 4.55772% :-) json_dump_v2 0.25869% -1.18928% 3.32973% 13.29297% :-| normal_startup 2.08229% -0.76787% -1.85290% 3.14314% :-) ssbench 0.38215% 0.02539% 2.33334% 1.59678% ------------------------------------------------------------------------------------------ Note: Benchmark results for ssbench are measured in requests/second while all other are measured in seconds. * Relative Standard Deviation (Standard Deviation/Average) Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. From lp_benchmark_robot at intel.com Fri Sep 11 15:55:33 2015 From: lp_benchmark_robot at intel.com (lp_benchmark_robot) Date: Fri, 11 Sep 2015 13:55:33 +0000 Subject: [Python-checkins] Benchmark Results for Python Default 2015-09-11 Message-ID: <078AA0FFE8C7034097F90205717F504611D7BE69@IRSMSX102.ger.corp.intel.com> Results for project python_default-nightly, build date 2015-09-11 03:02:03 commit: 8902bcea59be7ee89a12d959ae8df7c245082a95 revision date: 2015-09-11 02:47:13 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v3.4.3, with hash b4cbecbc0781e89a309d03b60a1f75f8499250e6 from 2015-02-25 12:15:33+00:00 ------------------------------------------------------------------------------------------ benchmark relative change since change since current rev with std_dev* last run v3.4.3 regrtest PGO ------------------------------------------------------------------------------------------ :-) django_v2 0.29190% -0.95468% 8.46375% 17.57823% :-( pybench 0.11173% -0.11930% -2.23466% 8.29641% :-( regex_v8 2.91802% 0.14752% -3.48234% 4.73095% :-| nbody 0.23719% 0.34997% -1.36316% 10.94635% :-( json_dump_v2 0.30808% 2.40906% -2.82210% 13.49787% :-| normal_startup 0.86128% 0.20999% -0.05027% 5.52596% ------------------------------------------------------------------------------------------ Note: Benchmark results are measured in seconds. * Relative Standard Deviation (Standard Deviation/Average) Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. From python-checkins at python.org Fri Sep 11 17:48:22 2015 From: python-checkins at python.org (steve.dower) Date: Fri, 11 Sep 2015 15:48:22 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge_with_3=2E5?= Message-ID: <20150911154822.11991.36315@psf.io> https://hg.python.org/cpython/rev/0d1a052e42ee changeset: 97916:0d1a052e42ee parent: 97913:60e4ae9e2e01 parent: 97915:df60218959fc user: Steve Dower date: Fri Sep 11 08:47:55 2015 -0700 summary: Merge with 3.5 files: Doc/using/windows.rst | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Doc/using/windows.rst b/Doc/using/windows.rst --- a/Doc/using/windows.rst +++ b/Doc/using/windows.rst @@ -77,8 +77,8 @@ suppressing the UI in order to change some of the defaults. To completely hide the installer UI and install Python silently, pass the -``/quiet`` (``/q``) option. To skip past the user interaction but still display -progress and errors, pass the ``/passive`` (``/p``) option. The ``/uninstall`` +``/quiet`` option. To skip past the user interaction but still display +progress and errors, pass the ``/passive`` option. The ``/uninstall`` option may be passed to immediately begin removing Python - no prompt will be displayed. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 11 17:48:22 2015 From: python-checkins at python.org (steve.dower) Date: Fri, 11 Sep 2015 15:48:22 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_Removes_invali?= =?utf-8?q?d_installer_options_from_documentation=2E?= Message-ID: <20150911154822.11250.79145@psf.io> https://hg.python.org/cpython/rev/df60218959fc changeset: 97915:df60218959fc branch: 3.5 parent: 97912:7bf325941636 user: Steve Dower date: Fri Sep 11 08:47:42 2015 -0700 summary: Removes invalid installer options from documentation. files: Doc/using/windows.rst | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Doc/using/windows.rst b/Doc/using/windows.rst --- a/Doc/using/windows.rst +++ b/Doc/using/windows.rst @@ -77,8 +77,8 @@ suppressing the UI in order to change some of the defaults. To completely hide the installer UI and install Python silently, pass the -``/quiet`` (``/q``) option. To skip past the user interaction but still display -progress and errors, pass the ``/passive`` (``/p``) option. The ``/uninstall`` +``/quiet`` option. To skip past the user interaction but still display +progress and errors, pass the ``/passive`` option. The ``/uninstall`` option may be passed to immediately begin removing Python - no prompt will be displayed. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 11 17:53:32 2015 From: python-checkins at python.org (zach.ware) Date: Fri, 11 Sep 2015 15:53:32 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_Fix_grammatica?= =?utf-8?q?l_error_in_csv_docs=2E?= Message-ID: <20150911155332.66858.46521@psf.io> https://hg.python.org/cpython/rev/64d6130f9e24 changeset: 97917:64d6130f9e24 branch: 2.7 parent: 97914:0c13674cf8b5 user: Zachary Ware date: Fri Sep 11 10:51:47 2015 -0500 summary: Fix grammatical error in csv docs. Reported by Nat Dunn on docs@ files: Doc/library/csv.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/library/csv.rst b/Doc/library/csv.rst --- a/Doc/library/csv.rst +++ b/Doc/library/csv.rst @@ -338,7 +338,7 @@ .. attribute:: Dialect.doublequote - Controls how instances of *quotechar* appearing inside a field should be + Controls how instances of *quotechar* appearing inside a field should themselves be quoted. When :const:`True`, the character is doubled. When :const:`False`, the *escapechar* is used as a prefix to the *quotechar*. It defaults to :const:`True`. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 11 17:53:33 2015 From: python-checkins at python.org (zach.ware) Date: Fri, 11 Sep 2015 15:53:33 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Merge_with_3=2E4?= Message-ID: <20150911155333.68869.61221@psf.io> https://hg.python.org/cpython/rev/6063fba0e28a changeset: 97919:6063fba0e28a branch: 3.5 parent: 97915:df60218959fc parent: 97918:4488e321fe85 user: Zachary Ware date: Fri Sep 11 10:52:36 2015 -0500 summary: Merge with 3.4 files: Doc/library/csv.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/library/csv.rst b/Doc/library/csv.rst --- a/Doc/library/csv.rst +++ b/Doc/library/csv.rst @@ -325,7 +325,7 @@ .. attribute:: Dialect.doublequote - Controls how instances of *quotechar* appearing inside a field should be + Controls how instances of *quotechar* appearing inside a field should themselves be quoted. When :const:`True`, the character is doubled. When :const:`False`, the *escapechar* is used as a prefix to the *quotechar*. It defaults to :const:`True`. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 11 17:53:33 2015 From: python-checkins at python.org (zach.ware) Date: Fri, 11 Sep 2015 15:53:33 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_Fix_grammatica?= =?utf-8?q?l_error_in_csv_docs=2E?= Message-ID: <20150911155332.27699.77855@psf.io> https://hg.python.org/cpython/rev/4488e321fe85 changeset: 97918:4488e321fe85 branch: 3.4 parent: 97911:2bff115e6ba0 user: Zachary Ware date: Fri Sep 11 10:51:47 2015 -0500 summary: Fix grammatical error in csv docs. Reported by Nat Dunn on docs@ files: Doc/library/csv.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/library/csv.rst b/Doc/library/csv.rst --- a/Doc/library/csv.rst +++ b/Doc/library/csv.rst @@ -324,7 +324,7 @@ .. attribute:: Dialect.doublequote - Controls how instances of *quotechar* appearing inside a field should be + Controls how instances of *quotechar* appearing inside a field should themselves be quoted. When :const:`True`, the character is doubled. When :const:`False`, the *escapechar* is used as a prefix to the *quotechar*. It defaults to :const:`True`. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 11 17:53:34 2015 From: python-checkins at python.org (zach.ware) Date: Fri, 11 Sep 2015 15:53:34 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge_with_3=2E5?= Message-ID: <20150911155333.68875.63486@psf.io> https://hg.python.org/cpython/rev/91fbeed05d7a changeset: 97920:91fbeed05d7a parent: 97916:0d1a052e42ee parent: 97919:6063fba0e28a user: Zachary Ware date: Fri Sep 11 10:52:51 2015 -0500 summary: Merge with 3.5 files: Doc/library/csv.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/library/csv.rst b/Doc/library/csv.rst --- a/Doc/library/csv.rst +++ b/Doc/library/csv.rst @@ -325,7 +325,7 @@ .. attribute:: Dialect.doublequote - Controls how instances of *quotechar* appearing inside a field should be + Controls how instances of *quotechar* appearing inside a field should themselves be quoted. When :const:`True`, the character is doubled. When :const:`False`, the *escapechar* is used as a prefix to the *quotechar*. It defaults to :const:`True`. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 11 19:03:00 2015 From: python-checkins at python.org (brett.cannon) Date: Fri, 11 Sep 2015 17:03:00 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_Make_the_ignoring_of_envs_mor?= =?utf-8?q?e_broad=2E?= Message-ID: <20150911170257.12021.27078@psf.io> https://hg.python.org/peps/rev/2434111428f4 changeset: 6047:2434111428f4 user: Brett Cannon date: Fri Sep 11 10:02:51 2015 -0700 summary: Make the ignoring of envs more broad. files: .hgignore | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/.hgignore b/.hgignore --- a/.hgignore +++ b/.hgignore @@ -4,4 +4,4 @@ *.pyc *.pyo *~ -venv +*env -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Fri Sep 11 19:56:54 2015 From: python-checkins at python.org (serhiy.storchaka) Date: Fri, 11 Sep 2015 17:56:54 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogd2hhdHNuZXcvMy41?= =?utf-8?q?=3A_Added_missed_author_names=2E?= Message-ID: <20150911175654.11252.12209@psf.io> https://hg.python.org/cpython/rev/33925c19516e changeset: 97921:33925c19516e branch: 3.5 parent: 97919:6063fba0e28a user: Serhiy Storchaka date: Fri Sep 11 20:55:28 2015 +0300 summary: whatsnew/3.5: Added missed author names. files: Doc/whatsnew/3.5.rst | 36 +++++++++++++++++-------------- 1 files changed, 20 insertions(+), 16 deletions(-) diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -104,8 +104,8 @@ * When the ``LC_TYPE`` locale is the POSIX locale (``C`` locale), :py:data:`sys.stdin` and :py:data:`sys.stdout` are now using the - ``surrogateescape`` error handler, instead of the ``strict`` error handler - (:issue:`19977`). + ``surrogateescape`` error handler, instead of the ``strict`` error handler. + (Contributed by Victor Stinner in :issue:`19977`.) * ``.pyo`` files are no longer used and have been replaced by a more flexible scheme that inclides the optimization level explicitly in ``.pyc`` name. @@ -585,7 +585,7 @@ (Contributed by Serhiy Storchaka in :issue:`19676` and :issue:`22286`.) * The :option:`-b` option now affects comparisons of :class:`bytes` with - :class:`int`. (Contributed by Serhiy Storchaka in :issue:`23681`) + :class:`int`. (Contributed by Serhiy Storchaka in :issue:`23681`.) * New Kazakh :ref:`codec ` ``kz1048``. (Contributed by Serhiy Storchaka in :issue:`22682`.) @@ -1376,7 +1376,7 @@ ------- The :class:`~sqlite3.Row` class now fully supports sequence protocol, -in particular :func:`reverse` and slice indexing. +in particular :func:`reversed` iteration and slice indexing. (Contributed by Claudiu Popa in :issue:`10203`; by Lucas Sinclair, Jessica McKellar, and Serhiy Storchaka in :issue:`13583`.) @@ -1602,7 +1602,8 @@ ========================== Many functions in :mod:`mmap`, :mod:`ossaudiodev`, :mod:`socket`, -:mod:`ssl`, and :mod:`codecs` modules now accept writable bytes-like objects. +:mod:`ssl`, and :mod:`codecs` modules now accept writable +:term:`bytes-like objects `. (Contributed by Serhiy Storchaka in :issue:`23001`.) @@ -1853,7 +1854,8 @@ * The concept of ``.pyo`` files has been removed. * The JoinableQueue class in the provisional asyncio module was deprecated - in 3.4.4 and is now removed (:issue:`23464`). + in 3.4.4 and is now removed. + (Contributed by A. Jesse Jiryu Davis in :issue:`23464`.) Porting to Python 3.5 @@ -1876,14 +1878,15 @@ * The :meth:`ssl.SSLSocket.send()` method now raises either :exc:`ssl.SSLWantReadError` or :exc:`ssl.SSLWantWriteError` - on a non-blocking socket if the operation would block. Previously, - it would return ``0``. See :issue:`20951`. + on a non-blocking socket if the operation would block. Previously, + it would return ``0``. (Contributed by Nikolaus Rath in :issue:`20951`.) * The ``__name__`` attribute of generator is now set from the function name, instead of being set from the code name. Use ``gen.gi_code.co_name`` to retrieve the code name. Generators also have a new ``__qualname__`` attribute, the qualified name, which is now used for the representation - of a generator (``repr(gen)``). See :issue:`21205`. + of a generator (``repr(gen)``). + (Contributed by Victor Stinner in :issue:`21205`.) * The deprecated "strict" mode and argument of :class:`~html.parser.HTMLParser`, :meth:`HTMLParser.error`, and the :exc:`HTMLParserError` exception have been @@ -1894,8 +1897,8 @@ * Although it is not formally part of the API, it is worth noting for porting purposes (ie: fixing tests) that error messages that were previously of the form "'sometype' does not support the buffer protocol" are now of the form "a - bytes-like object is required, not 'sometype'". (Contributed by Ezio Melotti - in :issue:`16518`.) + :term:`bytes-like object` is required, not 'sometype'". + (Contributed by Ezio Melotti in :issue:`16518`.) * If the current directory is set to a directory that no longer exists then :exc:`FileNotFoundError` will no longer be raised and instead @@ -1914,7 +1917,7 @@ :exc:`DeprecationWarning` now, will be an error in Python 3.6). If the loader inherits from :class:`importlib.abc.Loader` then there is nothing to do, else simply define :meth:`~importlib.machinery.Loader.create_module` to return - ``None`` (:issue:`23014`). + ``None``. (Contributed by Brett Cannon in :issue:`23014`.) * The :func:`re.split` function always ignored empty pattern matches, so the ``"x*"`` pattern worked the same as ``"x+"``, and the ``"\b"`` pattern never @@ -1931,7 +1934,7 @@ :meth:`~http.cookies.Morsel.update` will now raise an exception if any of the keys in the update dictionary are invalid. In addition, the undocumented *LegalChars* parameter of :func:`~http.cookies.Morsel.set` is deprecated and - is now ignored. (:issue:`2211`) + is now ignored. (Contributed by Demian Brecht in :issue:`2211`.) * :pep:`488` has removed ``.pyo`` files from Python and introduced the optional ``opt-`` tag in ``.pyc`` file names. The @@ -1961,14 +1964,15 @@ * The :meth:`str.startswith` and :meth:`str.endswith` methods no longer return ``True`` when finding the empty string and the indexes are completely out of - range. See :issue:`24284`. + range. (Contributed by Serhiy Storchaka in :issue:`24284`.) * The :func:`inspect.getdoc` function now returns documentation strings inherited from base classes. Documentation strings no longer need to be duplicated if the inherited documentation is appropriate. To suppress an inherited string, an empty string must be specified (or the documentation may be filled in). This change affects the output of the :mod:`pydoc` - module and the :func:`help` function. See :issue:`15582`. + module and the :func:`help` function. + (Contributed by Serhiy Storchaka in :issue:`15582`.) Changes in the C API -------------------- @@ -1989,7 +1993,7 @@ * Because the lack of the :attr:`__module__` attribute breaks pickling and introspection, a deprecation warning now is raised for builtin type without the :attr:`__module__` attribute. Would be an AttributeError in future. - (:issue:`20204`) + (Contributed by Serhiy Storchaka in :issue:`20204`.) * As part of :pep:`492` implementation, ``tp_reserved`` slot of :c:type:`PyTypeObject` was replaced with a -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 11 19:56:57 2015 From: python-checkins at python.org (serhiy.storchaka) Date: Fri, 11 Sep 2015 17:56:57 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogTWVyZ2UgMy41?= Message-ID: <20150911175654.27707.65402@psf.io> https://hg.python.org/cpython/rev/e4ebbed37092 changeset: 97922:e4ebbed37092 parent: 97920:91fbeed05d7a parent: 97921:33925c19516e user: Serhiy Storchaka date: Fri Sep 11 20:56:05 2015 +0300 summary: Merge 3.5 files: Doc/whatsnew/3.5.rst | 36 +++++++++++++++++-------------- 1 files changed, 20 insertions(+), 16 deletions(-) diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -104,8 +104,8 @@ * When the ``LC_TYPE`` locale is the POSIX locale (``C`` locale), :py:data:`sys.stdin` and :py:data:`sys.stdout` are now using the - ``surrogateescape`` error handler, instead of the ``strict`` error handler - (:issue:`19977`). + ``surrogateescape`` error handler, instead of the ``strict`` error handler. + (Contributed by Victor Stinner in :issue:`19977`.) * ``.pyo`` files are no longer used and have been replaced by a more flexible scheme that inclides the optimization level explicitly in ``.pyc`` name. @@ -585,7 +585,7 @@ (Contributed by Serhiy Storchaka in :issue:`19676` and :issue:`22286`.) * The :option:`-b` option now affects comparisons of :class:`bytes` with - :class:`int`. (Contributed by Serhiy Storchaka in :issue:`23681`) + :class:`int`. (Contributed by Serhiy Storchaka in :issue:`23681`.) * New Kazakh :ref:`codec ` ``kz1048``. (Contributed by Serhiy Storchaka in :issue:`22682`.) @@ -1376,7 +1376,7 @@ ------- The :class:`~sqlite3.Row` class now fully supports sequence protocol, -in particular :func:`reverse` and slice indexing. +in particular :func:`reversed` iteration and slice indexing. (Contributed by Claudiu Popa in :issue:`10203`; by Lucas Sinclair, Jessica McKellar, and Serhiy Storchaka in :issue:`13583`.) @@ -1602,7 +1602,8 @@ ========================== Many functions in :mod:`mmap`, :mod:`ossaudiodev`, :mod:`socket`, -:mod:`ssl`, and :mod:`codecs` modules now accept writable bytes-like objects. +:mod:`ssl`, and :mod:`codecs` modules now accept writable +:term:`bytes-like objects `. (Contributed by Serhiy Storchaka in :issue:`23001`.) @@ -1853,7 +1854,8 @@ * The concept of ``.pyo`` files has been removed. * The JoinableQueue class in the provisional asyncio module was deprecated - in 3.4.4 and is now removed (:issue:`23464`). + in 3.4.4 and is now removed. + (Contributed by A. Jesse Jiryu Davis in :issue:`23464`.) Porting to Python 3.5 @@ -1876,14 +1878,15 @@ * The :meth:`ssl.SSLSocket.send()` method now raises either :exc:`ssl.SSLWantReadError` or :exc:`ssl.SSLWantWriteError` - on a non-blocking socket if the operation would block. Previously, - it would return ``0``. See :issue:`20951`. + on a non-blocking socket if the operation would block. Previously, + it would return ``0``. (Contributed by Nikolaus Rath in :issue:`20951`.) * The ``__name__`` attribute of generator is now set from the function name, instead of being set from the code name. Use ``gen.gi_code.co_name`` to retrieve the code name. Generators also have a new ``__qualname__`` attribute, the qualified name, which is now used for the representation - of a generator (``repr(gen)``). See :issue:`21205`. + of a generator (``repr(gen)``). + (Contributed by Victor Stinner in :issue:`21205`.) * The deprecated "strict" mode and argument of :class:`~html.parser.HTMLParser`, :meth:`HTMLParser.error`, and the :exc:`HTMLParserError` exception have been @@ -1894,8 +1897,8 @@ * Although it is not formally part of the API, it is worth noting for porting purposes (ie: fixing tests) that error messages that were previously of the form "'sometype' does not support the buffer protocol" are now of the form "a - bytes-like object is required, not 'sometype'". (Contributed by Ezio Melotti - in :issue:`16518`.) + :term:`bytes-like object` is required, not 'sometype'". + (Contributed by Ezio Melotti in :issue:`16518`.) * If the current directory is set to a directory that no longer exists then :exc:`FileNotFoundError` will no longer be raised and instead @@ -1914,7 +1917,7 @@ :exc:`DeprecationWarning` now, will be an error in Python 3.6). If the loader inherits from :class:`importlib.abc.Loader` then there is nothing to do, else simply define :meth:`~importlib.machinery.Loader.create_module` to return - ``None`` (:issue:`23014`). + ``None``. (Contributed by Brett Cannon in :issue:`23014`.) * The :func:`re.split` function always ignored empty pattern matches, so the ``"x*"`` pattern worked the same as ``"x+"``, and the ``"\b"`` pattern never @@ -1931,7 +1934,7 @@ :meth:`~http.cookies.Morsel.update` will now raise an exception if any of the keys in the update dictionary are invalid. In addition, the undocumented *LegalChars* parameter of :func:`~http.cookies.Morsel.set` is deprecated and - is now ignored. (:issue:`2211`) + is now ignored. (Contributed by Demian Brecht in :issue:`2211`.) * :pep:`488` has removed ``.pyo`` files from Python and introduced the optional ``opt-`` tag in ``.pyc`` file names. The @@ -1961,14 +1964,15 @@ * The :meth:`str.startswith` and :meth:`str.endswith` methods no longer return ``True`` when finding the empty string and the indexes are completely out of - range. See :issue:`24284`. + range. (Contributed by Serhiy Storchaka in :issue:`24284`.) * The :func:`inspect.getdoc` function now returns documentation strings inherited from base classes. Documentation strings no longer need to be duplicated if the inherited documentation is appropriate. To suppress an inherited string, an empty string must be specified (or the documentation may be filled in). This change affects the output of the :mod:`pydoc` - module and the :func:`help` function. See :issue:`15582`. + module and the :func:`help` function. + (Contributed by Serhiy Storchaka in :issue:`15582`.) Changes in the C API -------------------- @@ -1989,7 +1993,7 @@ * Because the lack of the :attr:`__module__` attribute breaks pickling and introspection, a deprecation warning now is raised for builtin type without the :attr:`__module__` attribute. Would be an AttributeError in future. - (:issue:`20204`) + (Contributed by Serhiy Storchaka in :issue:`20204`.) * As part of :pep:`492` implementation, ``tp_reserved`` slot of :c:type:`PyTypeObject` was replaced with a -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 11 20:31:46 2015 From: python-checkins at python.org (steve.dower) Date: Fri, 11 Sep 2015 18:31:46 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogQWRkcyAzLjUuMCBo?= =?utf-8?q?eader_to_Misc/NEWS?= Message-ID: <20150911183145.14867.22563@psf.io> https://hg.python.org/cpython/rev/ee87dae3b4ea changeset: 97925:ee87dae3b4ea branch: 3.5 user: Steve Dower date: Fri Sep 11 11:29:07 2015 -0700 summary: Adds 3.5.0 header to Misc/NEWS files: Misc/NEWS | 5 +++++ 1 files changed, 5 insertions(+), 0 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -104,6 +104,11 @@ - Issue #25022: Removed very outdated PC/example_nt/ directory. +What's New in Python 3.5.0 final? +================================= + +Release date: 2015-09-13 + Build ----- -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 11 20:31:46 2015 From: python-checkins at python.org (steve.dower) Date: Fri, 11 Sep 2015 18:31:46 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI1MDcx?= =?utf-8?q?=3A_Windows_installer_should_not_require_TargetDir_parameter_wh?= =?utf-8?q?en?= Message-ID: <20150911183145.14885.34571@psf.io> https://hg.python.org/cpython/rev/da8f2767b6cc changeset: 97923:da8f2767b6cc branch: 3.5 parent: 97843:a9153eca5c72 user: Steve Dower date: Fri Sep 11 10:56:59 2015 -0700 summary: Issue #25071: Windows installer should not require TargetDir parameter when installing quietly files: Misc/NEWS | 5 + Tools/msi/bundle/bootstrap/PythonBootstrapperApplication.cpp | 30 ++++++++++ 2 files changed, 35 insertions(+), 0 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -7,6 +7,11 @@ Release date: 2015-09-13 +Build +----- + +- Issue #25071: Windows installer should not require TargetDir + parameter when installing quietly What's New in Python 3.5.0 release candidate 4? =============================================== diff --git a/Tools/msi/bundle/bootstrap/PythonBootstrapperApplication.cpp b/Tools/msi/bundle/bootstrap/PythonBootstrapperApplication.cpp --- a/Tools/msi/bundle/bootstrap/PythonBootstrapperApplication.cpp +++ b/Tools/msi/bundle/bootstrap/PythonBootstrapperApplication.cpp @@ -723,6 +723,36 @@ hrStatus = EvaluateConditions(); } + if (SUCCEEDED(hrStatus)) { + // Ensure the default path has been set + LONGLONG installAll; + LPWSTR targetDir = nullptr; + LPWSTR defaultTargetDir = nullptr; + + hrStatus = BalGetStringVariable(L"TargetDir", &targetDir); + if (FAILED(hrStatus) || !targetDir || !targetDir[0]) { + ReleaseStr(targetDir); + targetDir = nullptr; + + if (FAILED(BalGetNumericVariable(L"InstallAllUsers", &installAll))) { + installAll = 0; + } + + hrStatus = BalGetStringVariable( + installAll ? L"DefaultAllUsersTargetDir" : L"DefaultJustForMeTargetDir", + &defaultTargetDir + ); + + if (SUCCEEDED(hrStatus) && defaultTargetDir) { + if (defaultTargetDir[0] && SUCCEEDED(BalFormatString(defaultTargetDir, &targetDir))) { + hrStatus = _engine->SetVariableString(L"TargetDir", targetDir); + ReleaseStr(targetDir); + } + ReleaseStr(defaultTargetDir); + } + } + } + SetState(PYBA_STATE_DETECTED, hrStatus); // If we're not interacting with the user or we're doing a layout or we're just after a force restart -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 11 20:31:46 2015 From: python-checkins at python.org (steve.dower) Date: Fri, 11 Sep 2015 18:31:46 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy41IC0+IDMuNSk6?= =?utf-8?q?_Merge_from_3=2E5=2E0?= Message-ID: <20150911183145.14863.38856@psf.io> https://hg.python.org/cpython/rev/7cb5d8133d00 changeset: 97924:7cb5d8133d00 branch: 3.5 parent: 97921:33925c19516e parent: 97923:da8f2767b6cc user: Steve Dower date: Fri Sep 11 11:27:45 2015 -0700 summary: Merge from 3.5.0 files: Misc/NEWS | 5 + Tools/msi/bundle/bootstrap/PythonBootstrapperApplication.cpp | 30 ++++++++++ 2 files changed, 35 insertions(+), 0 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -104,6 +104,11 @@ - Issue #25022: Removed very outdated PC/example_nt/ directory. +Build +----- + +- Issue #25071: Windows installer should not require TargetDir + parameter when installing quietly What's New in Python 3.5.0 release candidate 4? =============================================== diff --git a/Tools/msi/bundle/bootstrap/PythonBootstrapperApplication.cpp b/Tools/msi/bundle/bootstrap/PythonBootstrapperApplication.cpp --- a/Tools/msi/bundle/bootstrap/PythonBootstrapperApplication.cpp +++ b/Tools/msi/bundle/bootstrap/PythonBootstrapperApplication.cpp @@ -723,6 +723,36 @@ hrStatus = EvaluateConditions(); } + if (SUCCEEDED(hrStatus)) { + // Ensure the default path has been set + LONGLONG installAll; + LPWSTR targetDir = nullptr; + LPWSTR defaultTargetDir = nullptr; + + hrStatus = BalGetStringVariable(L"TargetDir", &targetDir); + if (FAILED(hrStatus) || !targetDir || !targetDir[0]) { + ReleaseStr(targetDir); + targetDir = nullptr; + + if (FAILED(BalGetNumericVariable(L"InstallAllUsers", &installAll))) { + installAll = 0; + } + + hrStatus = BalGetStringVariable( + installAll ? L"DefaultAllUsersTargetDir" : L"DefaultJustForMeTargetDir", + &defaultTargetDir + ); + + if (SUCCEEDED(hrStatus) && defaultTargetDir) { + if (defaultTargetDir[0] && SUCCEEDED(BalFormatString(defaultTargetDir, &targetDir))) { + hrStatus = _engine->SetVariableString(L"TargetDir", targetDir); + ReleaseStr(targetDir); + } + ReleaseStr(defaultTargetDir); + } + } + } + SetState(PYBA_STATE_DETECTED, hrStatus); // If we're not interacting with the user or we're doing a layout or we're just after a force restart -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 11 20:31:49 2015 From: python-checkins at python.org (steve.dower) Date: Fri, 11 Sep 2015 18:31:49 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2325071=3A_Windows_installer_should_not_require_T?= =?utf-8?q?argetDir_parameter_when?= Message-ID: <20150911183146.66866.55839@psf.io> https://hg.python.org/cpython/rev/bb7363b8b50e changeset: 97926:bb7363b8b50e parent: 97922:e4ebbed37092 parent: 97925:ee87dae3b4ea user: Steve Dower date: Fri Sep 11 11:31:07 2015 -0700 summary: Issue #25071: Windows installer should not require TargetDir parameter when installing quietly files: Misc/NEWS | 16 +++++ Tools/msi/bundle/bootstrap/PythonBootstrapperApplication.cpp | 30 ++++++++++ 2 files changed, 46 insertions(+), 0 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -189,6 +189,22 @@ when external libraries are not available. +Windows +------- + +- Issue #25022: Removed very outdated PC/example_nt/ directory. + +What's New in Python 3.5.0 final? +================================= + +Release date: 2015-09-13 + +Build +----- + +- Issue #25071: Windows installer should not require TargetDir + parameter when installing quietly + What's New in Python 3.5.0 release candidate 4? =============================================== diff --git a/Tools/msi/bundle/bootstrap/PythonBootstrapperApplication.cpp b/Tools/msi/bundle/bootstrap/PythonBootstrapperApplication.cpp --- a/Tools/msi/bundle/bootstrap/PythonBootstrapperApplication.cpp +++ b/Tools/msi/bundle/bootstrap/PythonBootstrapperApplication.cpp @@ -723,6 +723,36 @@ hrStatus = EvaluateConditions(); } + if (SUCCEEDED(hrStatus)) { + // Ensure the default path has been set + LONGLONG installAll; + LPWSTR targetDir = nullptr; + LPWSTR defaultTargetDir = nullptr; + + hrStatus = BalGetStringVariable(L"TargetDir", &targetDir); + if (FAILED(hrStatus) || !targetDir || !targetDir[0]) { + ReleaseStr(targetDir); + targetDir = nullptr; + + if (FAILED(BalGetNumericVariable(L"InstallAllUsers", &installAll))) { + installAll = 0; + } + + hrStatus = BalGetStringVariable( + installAll ? L"DefaultAllUsersTargetDir" : L"DefaultJustForMeTargetDir", + &defaultTargetDir + ); + + if (SUCCEEDED(hrStatus) && defaultTargetDir) { + if (defaultTargetDir[0] && SUCCEEDED(BalFormatString(defaultTargetDir, &targetDir))) { + hrStatus = _engine->SetVariableString(L"TargetDir", targetDir); + ReleaseStr(targetDir); + } + ReleaseStr(defaultTargetDir); + } + } + } + SetState(PYBA_STATE_DETECTED, hrStatus); // If we're not interacting with the user or we're doing a layout or we're just after a force restart -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 12 01:32:11 2015 From: python-checkins at python.org (martin.panter) Date: Fri, 11 Sep 2015 23:32:11 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Issue_=2325043=3A_Merge_Bluetooth_doc_from_3=2E4_into_3=2E5?= Message-ID: <20150911233210.17989.67810@psf.io> https://hg.python.org/cpython/rev/3f475417eadd changeset: 97928:3f475417eadd branch: 3.5 parent: 97925:ee87dae3b4ea parent: 97927:152568976062 user: Martin Panter date: Fri Sep 11 23:19:10 2015 +0000 summary: Issue #25043: Merge Bluetooth doc from 3.4 into 3.5 files: Doc/library/socket.rst | 16 ++++++++++++++++ 1 files changed, 16 insertions(+), 0 deletions(-) diff --git a/Doc/library/socket.rst b/Doc/library/socket.rst --- a/Doc/library/socket.rst +++ b/Doc/library/socket.rst @@ -349,6 +349,22 @@ This constant contains a boolean value which indicates if IPv6 is supported on this platform. +.. data:: BDADDR_ANY + BDADDR_LOCAL + + These are string constants containing Bluetooth addresses with special + meanings. For example, :const:`BDADDR_ANY` can be used to indicate + any address when specifying the binding socket with + :const:`BTPROTO_RFCOMM`. + +.. data:: HCI_FILTER + HCI_TIME_STAMP + HCI_DATA_DIR + + For use with :const:`BTPROTO_HCI`. :const:`HCI_FILTER` is not + available for NetBSD or DragonFlyBSD. :const:`HCI_TIME_STAMP` and + :const:`HCI_DATA_DIR` are not available for FreeBSD, NetBSD, or + DragonFlyBSD. Functions ^^^^^^^^^ -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 12 01:32:11 2015 From: python-checkins at python.org (martin.panter) Date: Fri, 11 Sep 2015 23:32:11 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2325043=3A_Merge_Bluetooth_doc_from_3=2E5?= Message-ID: <20150911233210.17965.52611@psf.io> https://hg.python.org/cpython/rev/5325233f117d changeset: 97929:5325233f117d parent: 97926:bb7363b8b50e parent: 97928:3f475417eadd user: Martin Panter date: Fri Sep 11 23:19:30 2015 +0000 summary: Issue #25043: Merge Bluetooth doc from 3.5 files: Doc/library/socket.rst | 16 ++++++++++++++++ 1 files changed, 16 insertions(+), 0 deletions(-) diff --git a/Doc/library/socket.rst b/Doc/library/socket.rst --- a/Doc/library/socket.rst +++ b/Doc/library/socket.rst @@ -349,6 +349,22 @@ This constant contains a boolean value which indicates if IPv6 is supported on this platform. +.. data:: BDADDR_ANY + BDADDR_LOCAL + + These are string constants containing Bluetooth addresses with special + meanings. For example, :const:`BDADDR_ANY` can be used to indicate + any address when specifying the binding socket with + :const:`BTPROTO_RFCOMM`. + +.. data:: HCI_FILTER + HCI_TIME_STAMP + HCI_DATA_DIR + + For use with :const:`BTPROTO_HCI`. :const:`HCI_FILTER` is not + available for NetBSD or DragonFlyBSD. :const:`HCI_TIME_STAMP` and + :const:`HCI_DATA_DIR` are not available for FreeBSD, NetBSD, or + DragonFlyBSD. Functions ^^^^^^^^^ -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 12 01:32:14 2015 From: python-checkins at python.org (martin.panter) Date: Fri, 11 Sep 2015 23:32:14 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzI1MDQz?= =?utf-8?q?=3A_Document_BDADDR=5F_and_HCI=5F_Bluetooth_socket_constants?= Message-ID: <20150911233210.66878.3288@psf.io> https://hg.python.org/cpython/rev/152568976062 changeset: 97927:152568976062 branch: 3.4 parent: 97918:4488e321fe85 user: Martin Panter date: Fri Sep 11 23:14:57 2015 +0000 summary: Issue #25043: Document BDADDR_ and HCI_ Bluetooth socket constants Patch from Tim Tisdall. files: Doc/library/socket.rst | 16 ++++++++++++++++ 1 files changed, 16 insertions(+), 0 deletions(-) diff --git a/Doc/library/socket.rst b/Doc/library/socket.rst --- a/Doc/library/socket.rst +++ b/Doc/library/socket.rst @@ -334,6 +334,22 @@ This constant contains a boolean value which indicates if IPv6 is supported on this platform. +.. data:: BDADDR_ANY + BDADDR_LOCAL + + These are string constants containing Bluetooth addresses with special + meanings. For example, :const:`BDADDR_ANY` can be used to indicate + any address when specifying the binding socket with + :const:`BTPROTO_RFCOMM`. + +.. data:: HCI_FILTER + HCI_TIME_STAMP + HCI_DATA_DIR + + For use with :const:`BTPROTO_HCI`. :const:`HCI_FILTER` is not + available for NetBSD or DragonFlyBSD. :const:`HCI_TIME_STAMP` and + :const:`HCI_DATA_DIR` are not available for FreeBSD, NetBSD, or + DragonFlyBSD. Functions ^^^^^^^^^ -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 12 01:56:34 2015 From: python-checkins at python.org (martin.panter) Date: Fri, 11 Sep 2015 23:56:34 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Dummy_merge_3=2E4_into_3=2E5=3B_socket=2Esendfile=28=29_exists?= =?utf-8?q?_in_3=2E5?= Message-ID: <20150911235634.11991.65182@psf.io> https://hg.python.org/cpython/rev/0a83dab7327f changeset: 97931:0a83dab7327f branch: 3.5 parent: 97928:3f475417eadd parent: 97930:3d9cbbad8a04 user: Martin Panter date: Fri Sep 11 23:43:42 2015 +0000 summary: Dummy merge 3.4 into 3.5; socket.sendfile() exists in 3.5 files: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 12 01:56:34 2015 From: python-checkins at python.org (martin.panter) Date: Fri, 11 Sep 2015 23:56:34 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzI1MDYz?= =?utf-8?q?=3A_socket=2Esendfile=28=29_does_not_exist_in_3=2E4?= Message-ID: <20150911235633.114719.11517@psf.io> https://hg.python.org/cpython/rev/3d9cbbad8a04 changeset: 97930:3d9cbbad8a04 branch: 3.4 parent: 97927:152568976062 user: Martin Panter date: Fri Sep 11 23:39:34 2015 +0000 summary: Issue #25063: socket.sendfile() does not exist in 3.4 Remove notice that was backported in revision 50527a1b769c. files: Doc/library/os.rst | 5 ----- 1 files changed, 0 insertions(+), 5 deletions(-) diff --git a/Doc/library/os.rst b/Doc/library/os.rst --- a/Doc/library/os.rst +++ b/Doc/library/os.rst @@ -1099,11 +1099,6 @@ Availability: Unix. - .. note:: - - For a higher-level wrapper of :func:`sendfile`, see - :mod:`socket.socket.sendfile`. - .. versionadded:: 3.3 -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 12 01:56:34 2015 From: python-checkins at python.org (martin.panter) Date: Fri, 11 Sep 2015 23:56:34 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogc29ja2V0LnNlbmRm?= =?utf-8?q?ile=28=29_is_a_method_not_a_module?= Message-ID: <20150911235634.66864.83376@psf.io> https://hg.python.org/cpython/rev/857ae7ab0955 changeset: 97932:857ae7ab0955 branch: 3.5 user: Martin Panter date: Fri Sep 11 23:44:18 2015 +0000 summary: socket.sendfile() is a method not a module files: Doc/library/os.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/library/os.rst b/Doc/library/os.rst --- a/Doc/library/os.rst +++ b/Doc/library/os.rst @@ -1102,7 +1102,7 @@ .. note:: For a higher-level wrapper of :func:`sendfile`, see - :mod:`socket.socket.sendfile`. + :meth:`socket.socket.sendfile`. .. versionadded:: 3.3 -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 12 01:56:34 2015 From: python-checkins at python.org (martin.panter) Date: Fri, 11 Sep 2015 23:56:34 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge_sendfile=28=29_doc_fixes_from_3=2E5?= Message-ID: <20150911235634.114658.64399@psf.io> https://hg.python.org/cpython/rev/62347c5c1e35 changeset: 97933:62347c5c1e35 parent: 97929:5325233f117d parent: 97932:857ae7ab0955 user: Martin Panter date: Fri Sep 11 23:54:58 2015 +0000 summary: Merge sendfile() doc fixes from 3.5 files: Doc/library/os.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/library/os.rst b/Doc/library/os.rst --- a/Doc/library/os.rst +++ b/Doc/library/os.rst @@ -1102,7 +1102,7 @@ .. note:: For a higher-level wrapper of :func:`sendfile`, see - :mod:`socket.socket.sendfile`. + :meth:`socket.socket.sendfile`. .. versionadded:: 3.3 -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 12 03:44:40 2015 From: python-checkins at python.org (martin.panter) Date: Sat, 12 Sep 2015 01:44:40 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Issue_=2316473=3A_Merge_codecs_doc_and_test_from_3=2E4_into_3?= =?utf-8?q?=2E5?= Message-ID: <20150912014440.66854.60151@psf.io> https://hg.python.org/cpython/rev/28cd11dc2915 changeset: 97935:28cd11dc2915 branch: 3.5 parent: 97932:857ae7ab0955 parent: 97934:de82f41d6669 user: Martin Panter date: Sat Sep 12 01:22:17 2015 +0000 summary: Issue #16473: Merge codecs doc and test from 3.4 into 3.5 files: Doc/library/codecs.rst | 16 ++++++++-------- Lib/encodings/quopri_codec.py | 2 +- Lib/test/test_codecs.py | 8 ++++++++ 3 files changed, 17 insertions(+), 9 deletions(-) diff --git a/Doc/library/codecs.rst b/Doc/library/codecs.rst --- a/Doc/library/codecs.rst +++ b/Doc/library/codecs.rst @@ -1310,9 +1310,9 @@ +----------------------+------------------+------------------------------+------------------------------+ | Codec | Aliases | Purpose | Encoder / decoder | +======================+==================+==============================+==============================+ -| base64_codec [#b64]_ | base64, base_64 | Convert operand to MIME | :meth:`base64.b64encode` / | -| | | base64 (the result always | :meth:`base64.b64decode` | -| | | includes a trailing | | +| base64_codec [#b64]_ | base64, base_64 | Convert operand to multiline | :meth:`base64.encodebytes` / | +| | | MIME base64 (the result | :meth:`base64.decodebytes` | +| | | always includes a trailing | | | | | ``'\n'``) | | | | | | | | | | .. versionchanged:: 3.4 | | @@ -1324,14 +1324,14 @@ | bz2_codec | bz2 | Compress the operand | :meth:`bz2.compress` / | | | | using bz2 | :meth:`bz2.decompress` | +----------------------+------------------+------------------------------+------------------------------+ -| hex_codec | hex | Convert operand to | :meth:`base64.b16encode` / | -| | | hexadecimal | :meth:`base64.b16decode` | +| hex_codec | hex | Convert operand to | :meth:`binascii.b2a_hex` / | +| | | hexadecimal | :meth:`binascii.a2b_hex` | | | | representation, with two | | | | | digits per byte | | +----------------------+------------------+------------------------------+------------------------------+ -| quopri_codec | quopri, | Convert operand to MIME | :meth:`quopri.encodestring` /| -| | quotedprintable, | quoted printable | :meth:`quopri.decodestring` | -| | quoted_printable | | | +| quopri_codec | quopri, | Convert operand to MIME | :meth:`quopri.encode` with | +| | quotedprintable, | quoted printable | ``quotetabs=True`` / | +| | quoted_printable | | :meth:`quopri.decode` | +----------------------+------------------+------------------------------+------------------------------+ | uu_codec | uu | Convert the operand using | :meth:`uu.encode` / | | | | uuencode | :meth:`uu.decode` | diff --git a/Lib/encodings/quopri_codec.py b/Lib/encodings/quopri_codec.py --- a/Lib/encodings/quopri_codec.py +++ b/Lib/encodings/quopri_codec.py @@ -11,7 +11,7 @@ assert errors == 'strict' f = BytesIO(input) g = BytesIO() - quopri.encode(f, g, 1) + quopri.encode(f, g, quotetabs=True) return (g.getvalue(), len(input)) def quopri_decode(input, errors='strict'): diff --git a/Lib/test/test_codecs.py b/Lib/test/test_codecs.py --- a/Lib/test/test_codecs.py +++ b/Lib/test/test_codecs.py @@ -2684,6 +2684,14 @@ info = codecs.lookup(alias) self.assertEqual(info.name, expected_name) + def test_quopri_stateless(self): + # Should encode with quotetabs=True + encoded = codecs.encode(b"space tab\teol \n", "quopri-codec") + self.assertEqual(encoded, b"space=20tab=09eol=20\n") + # But should still support unescaped tabs and spaces + unescaped = b"space tab eol\n" + self.assertEqual(codecs.decode(unescaped, "quopri-codec"), unescaped) + def test_uu_invalid(self): # Missing "begin" line self.assertRaises(ValueError, codecs.decode, b"", "uu-codec") -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 12 03:44:39 2015 From: python-checkins at python.org (martin.panter) Date: Sat, 12 Sep 2015 01:44:39 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzE2NDcz?= =?utf-8?q?=3A_Fix_byte_transform_codec_documentation=3B_test_quotetabs=3D?= =?utf-8?q?True?= Message-ID: <20150912014439.11252.90584@psf.io> https://hg.python.org/cpython/rev/de82f41d6669 changeset: 97934:de82f41d6669 branch: 3.4 parent: 97930:3d9cbbad8a04 user: Martin Panter date: Sat Sep 12 00:34:28 2015 +0000 summary: Issue #16473: Fix byte transform codec documentation; test quotetabs=True This changes the equivalent functions listed for the Base-64, hex and Quoted- Printable codecs to reflect the functions actually used. Also mention and test the "quotetabs" setting for Quoted-Printable encoding. files: Doc/library/codecs.rst | 16 ++++++++-------- Lib/encodings/quopri_codec.py | 2 +- Lib/test/test_codecs.py | 8 ++++++++ 3 files changed, 17 insertions(+), 9 deletions(-) diff --git a/Doc/library/codecs.rst b/Doc/library/codecs.rst --- a/Doc/library/codecs.rst +++ b/Doc/library/codecs.rst @@ -1284,9 +1284,9 @@ +----------------------+------------------+------------------------------+------------------------------+ | Codec | Aliases | Purpose | Encoder / decoder | +======================+==================+==============================+==============================+ -| base64_codec [#b64]_ | base64, base_64 | Convert operand to MIME | :meth:`base64.b64encode` / | -| | | base64 (the result always | :meth:`base64.b64decode` | -| | | includes a trailing | | +| base64_codec [#b64]_ | base64, base_64 | Convert operand to multiline | :meth:`base64.encodebytes` / | +| | | MIME base64 (the result | :meth:`base64.decodebytes` | +| | | always includes a trailing | | | | | ``'\n'``) | | | | | | | | | | .. versionchanged:: 3.4 | | @@ -1298,14 +1298,14 @@ | bz2_codec | bz2 | Compress the operand | :meth:`bz2.compress` / | | | | using bz2 | :meth:`bz2.decompress` | +----------------------+------------------+------------------------------+------------------------------+ -| hex_codec | hex | Convert operand to | :meth:`base64.b16encode` / | -| | | hexadecimal | :meth:`base64.b16decode` | +| hex_codec | hex | Convert operand to | :meth:`binascii.b2a_hex` / | +| | | hexadecimal | :meth:`binascii.a2b_hex` | | | | representation, with two | | | | | digits per byte | | +----------------------+------------------+------------------------------+------------------------------+ -| quopri_codec | quopri, | Convert operand to MIME | :meth:`quopri.encodestring` /| -| | quotedprintable, | quoted printable | :meth:`quopri.decodestring` | -| | quoted_printable | | | +| quopri_codec | quopri, | Convert operand to MIME | :meth:`quopri.encode` with | +| | quotedprintable, | quoted printable | ``quotetabs=True`` / | +| | quoted_printable | | :meth:`quopri.decode` | +----------------------+------------------+------------------------------+------------------------------+ | uu_codec | uu | Convert the operand using | :meth:`uu.encode` / | | | | uuencode | :meth:`uu.decode` | diff --git a/Lib/encodings/quopri_codec.py b/Lib/encodings/quopri_codec.py --- a/Lib/encodings/quopri_codec.py +++ b/Lib/encodings/quopri_codec.py @@ -11,7 +11,7 @@ assert errors == 'strict' f = BytesIO(input) g = BytesIO() - quopri.encode(f, g, 1) + quopri.encode(f, g, quotetabs=True) return (g.getvalue(), len(input)) def quopri_decode(input, errors='strict'): diff --git a/Lib/test/test_codecs.py b/Lib/test/test_codecs.py --- a/Lib/test/test_codecs.py +++ b/Lib/test/test_codecs.py @@ -2613,6 +2613,14 @@ info = codecs.lookup(alias) self.assertEqual(info.name, expected_name) + def test_quopri_stateless(self): + # Should encode with quotetabs=True + encoded = codecs.encode(b"space tab\teol \n", "quopri-codec") + self.assertEqual(encoded, b"space=20tab=09eol=20\n") + # But should still support unescaped tabs and spaces + unescaped = b"space tab eol\n" + self.assertEqual(codecs.decode(unescaped, "quopri-codec"), unescaped) + def test_uu_invalid(self): # Missing "begin" line self.assertRaises(ValueError, codecs.decode, b"", "uu-codec") -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 12 03:44:40 2015 From: python-checkins at python.org (martin.panter) Date: Sat, 12 Sep 2015 01:44:40 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2316473=3A_Merge_codecs_doc_and_test_from_3=2E5?= Message-ID: <20150912014440.17973.53582@psf.io> https://hg.python.org/cpython/rev/3ecb5766ba15 changeset: 97936:3ecb5766ba15 parent: 97933:62347c5c1e35 parent: 97935:28cd11dc2915 user: Martin Panter date: Sat Sep 12 01:24:33 2015 +0000 summary: Issue #16473: Merge codecs doc and test from 3.5 files: Doc/library/codecs.rst | 16 ++++++++-------- Lib/encodings/quopri_codec.py | 2 +- Lib/test/test_codecs.py | 8 ++++++++ 3 files changed, 17 insertions(+), 9 deletions(-) diff --git a/Doc/library/codecs.rst b/Doc/library/codecs.rst --- a/Doc/library/codecs.rst +++ b/Doc/library/codecs.rst @@ -1310,9 +1310,9 @@ +----------------------+------------------+------------------------------+------------------------------+ | Codec | Aliases | Purpose | Encoder / decoder | +======================+==================+==============================+==============================+ -| base64_codec [#b64]_ | base64, base_64 | Convert operand to MIME | :meth:`base64.b64encode` / | -| | | base64 (the result always | :meth:`base64.b64decode` | -| | | includes a trailing | | +| base64_codec [#b64]_ | base64, base_64 | Convert operand to multiline | :meth:`base64.encodebytes` / | +| | | MIME base64 (the result | :meth:`base64.decodebytes` | +| | | always includes a trailing | | | | | ``'\n'``) | | | | | | | | | | .. versionchanged:: 3.4 | | @@ -1324,14 +1324,14 @@ | bz2_codec | bz2 | Compress the operand | :meth:`bz2.compress` / | | | | using bz2 | :meth:`bz2.decompress` | +----------------------+------------------+------------------------------+------------------------------+ -| hex_codec | hex | Convert operand to | :meth:`base64.b16encode` / | -| | | hexadecimal | :meth:`base64.b16decode` | +| hex_codec | hex | Convert operand to | :meth:`binascii.b2a_hex` / | +| | | hexadecimal | :meth:`binascii.a2b_hex` | | | | representation, with two | | | | | digits per byte | | +----------------------+------------------+------------------------------+------------------------------+ -| quopri_codec | quopri, | Convert operand to MIME | :meth:`quopri.encodestring` /| -| | quotedprintable, | quoted printable | :meth:`quopri.decodestring` | -| | quoted_printable | | | +| quopri_codec | quopri, | Convert operand to MIME | :meth:`quopri.encode` with | +| | quotedprintable, | quoted printable | ``quotetabs=True`` / | +| | quoted_printable | | :meth:`quopri.decode` | +----------------------+------------------+------------------------------+------------------------------+ | uu_codec | uu | Convert the operand using | :meth:`uu.encode` / | | | | uuencode | :meth:`uu.decode` | diff --git a/Lib/encodings/quopri_codec.py b/Lib/encodings/quopri_codec.py --- a/Lib/encodings/quopri_codec.py +++ b/Lib/encodings/quopri_codec.py @@ -11,7 +11,7 @@ assert errors == 'strict' f = BytesIO(input) g = BytesIO() - quopri.encode(f, g, 1) + quopri.encode(f, g, quotetabs=True) return (g.getvalue(), len(input)) def quopri_decode(input, errors='strict'): diff --git a/Lib/test/test_codecs.py b/Lib/test/test_codecs.py --- a/Lib/test/test_codecs.py +++ b/Lib/test/test_codecs.py @@ -2684,6 +2684,14 @@ info = codecs.lookup(alias) self.assertEqual(info.name, expected_name) + def test_quopri_stateless(self): + # Should encode with quotetabs=True + encoded = codecs.encode(b"space tab\teol \n", "quopri-codec") + self.assertEqual(encoded, b"space=20tab=09eol=20\n") + # But should still support unescaped tabs and spaces + unescaped = b"space tab eol\n" + self.assertEqual(codecs.decode(unescaped, "quopri-codec"), unescaped) + def test_uu_invalid(self): # Missing "begin" line self.assertRaises(ValueError, codecs.decode, b"", "uu-codec") -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 12 04:36:26 2015 From: python-checkins at python.org (martin.panter) Date: Sat, 12 Sep 2015 02:36:26 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_Remove_and_ena?= =?utf-8?q?ble_misplaced_codecs_tests_ported_from_Python_3?= Message-ID: <20150912023626.14881.69896@psf.io> https://hg.python.org/cpython/rev/9abe48b0f944 changeset: 97937:9abe48b0f944 branch: 2.7 parent: 97917:64d6130f9e24 user: Martin Panter date: Sat Sep 12 02:20:06 2015 +0000 summary: Remove and enable misplaced codecs tests ported from Python 3 Most of these tests are about blacklisted non-text codecs, which are not relevant in Python 2. The only one remaining is TransformCodecTest.test_uu_ invalid(). files: Lib/test/test_codecs.py | 53 ++++------------------------ 1 files changed, 8 insertions(+), 45 deletions(-) diff --git a/Lib/test/test_codecs.py b/Lib/test/test_codecs.py --- a/Lib/test/test_codecs.py +++ b/Lib/test/test_codecs.py @@ -2101,6 +2101,13 @@ self.assertEqual(f.read(), data * 2) +class TransformCodecTest(unittest.TestCase): + + def test_uu_invalid(self): + # Missing "begin" line + self.assertRaises(ValueError, codecs.decode, "", "uu-codec") + + def test_main(): test_support.run_unittest( UTF32Test, @@ -2132,53 +2139,9 @@ UnicodeEscapeTest, RawUnicodeEscapeTest, BomTest, + TransformCodecTest, ) - def test_uu_invalid(self): - # Missing "begin" line - self.assertRaises(ValueError, codecs.decode, "", "uu-codec") - - def test_text_to_binary_blacklists_binary_transforms(self): - # Check binary -> binary codecs give a good error for str input - bad_input = "bad input type" - for encoding in bytes_transform_encodings: - fmt = (r"{!r} is not a text encoding; " - r"use codecs.encode\(\) to handle arbitrary codecs") - msg = fmt.format(encoding) - with self.assertRaisesRegex(LookupError, msg) as failure: - bad_input.encode(encoding) - self.assertIsNone(failure.exception.__cause__) - - def test_text_to_binary_blacklists_text_transforms(self): - # Check str.encode gives a good error message for str -> str codecs - msg = (r"^'rot_13' is not a text encoding; " - r"use codecs.encode\(\) to handle arbitrary codecs") - with self.assertRaisesRegex(LookupError, msg): - "just an example message".encode("rot_13") - - def test_binary_to_text_blacklists_binary_transforms(self): - # Check bytes.decode and bytearray.decode give a good error - # message for binary -> binary codecs - data = b"encode first to ensure we meet any format restrictions" - for encoding in bytes_transform_encodings: - encoded_data = codecs.encode(data, encoding) - fmt = (r"{!r} is not a text encoding; " - r"use codecs.decode\(\) to handle arbitrary codecs") - msg = fmt.format(encoding) - with self.assertRaisesRegex(LookupError, msg): - encoded_data.decode(encoding) - with self.assertRaisesRegex(LookupError, msg): - bytearray(encoded_data).decode(encoding) - - def test_binary_to_text_blacklists_text_transforms(self): - # Check str -> str codec gives a good error for binary input - for bad_input in (b"immutable", bytearray(b"mutable")): - msg = (r"^'rot_13' is not a text encoding; " - r"use codecs.decode\(\) to handle arbitrary codecs") - with self.assertRaisesRegex(LookupError, msg) as failure: - bad_input.decode("rot_13") - self.assertIsNone(failure.exception.__cause__) - if __name__ == "__main__": test_main() -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 12 04:56:28 2015 From: python-checkins at python.org (martin.panter) Date: Sat, 12 Sep 2015 02:56:28 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzE2NDcz?= =?utf-8?q?=3A_Fix_byte_transform_codec_documentation=3B_test_quotetabs=3D?= =?utf-8?q?True?= Message-ID: <20150912025628.68883.16361@psf.io> https://hg.python.org/cpython/rev/cfb0481c89d7 changeset: 97938:cfb0481c89d7 branch: 2.7 user: Martin Panter date: Sat Sep 12 00:34:28 2015 +0000 summary: Issue #16473: Fix byte transform codec documentation; test quotetabs=True This changes the equivalent functions listed for the Base-64, hex and Quoted- Printable codecs to reflect the functions actually used. Also mention and test the "quotetabs" setting for Quoted-Printable encoding. files: Doc/library/codecs.rst | 17 +++++++++-------- Lib/encodings/quopri_codec.py | 2 +- Lib/test/test_codecs.py | 8 ++++++++ 3 files changed, 18 insertions(+), 9 deletions(-) diff --git a/Doc/library/codecs.rst b/Doc/library/codecs.rst --- a/Doc/library/codecs.rst +++ b/Doc/library/codecs.rst @@ -1193,21 +1193,22 @@ +--------------------+---------------------------+---------------------------+------------------------------+ | Codec | Aliases | Purpose | Encoder/decoder | +====================+===========================+===========================+==============================+ -| base64_codec | base64, base-64 | Convert operand to MIME | :meth:`base64.b64encode`, | -| | | base64 (the result always | :meth:`base64.b64decode` | -| | | includes a trailing | | -| | | ``'\n'``) | | +| base64_codec | base64, base-64 | Convert operand to | :meth:`base64.encodestring`, | +| | | multiline MIME base64 (the| :meth:`base64.decodestring` | +| | | result always includes a | | +| | | trailing ``'\n'``) | | +--------------------+---------------------------+---------------------------+------------------------------+ | bz2_codec | bz2 | Compress the operand | :meth:`bz2.compress`, | | | | using bz2 | :meth:`bz2.decompress` | +--------------------+---------------------------+---------------------------+------------------------------+ -| hex_codec | hex | Convert operand to | :meth:`base64.b16encode`, | -| | | hexadecimal | :meth:`base64.b16decode` | +| hex_codec | hex | Convert operand to | :meth:`binascii.b2a_hex`, | +| | | hexadecimal | :meth:`binascii.a2b_hex` | | | | representation, with two | | | | | digits per byte | | +--------------------+---------------------------+---------------------------+------------------------------+ -| quopri_codec | quopri, quoted-printable, | Convert operand to MIME | :meth:`quopri.encodestring`, | -| | quotedprintable | quoted printable | :meth:`quopri.decodestring` | +| quopri_codec | quopri, quoted-printable, | Convert operand to MIME | :meth:`quopri.encode` with | +| | quotedprintable | quoted printable | ``quotetabs=True``, | +| | | | :meth:`quopri.decode` | +--------------------+---------------------------+---------------------------+------------------------------+ | string_escape | | Produce a string that is | | | | | suitable as string | | diff --git a/Lib/encodings/quopri_codec.py b/Lib/encodings/quopri_codec.py --- a/Lib/encodings/quopri_codec.py +++ b/Lib/encodings/quopri_codec.py @@ -21,7 +21,7 @@ # using str() because of cStringIO's Unicode undesired Unicode behavior. f = StringIO(str(input)) g = StringIO() - quopri.encode(f, g, 1) + quopri.encode(f, g, quotetabs=True) output = g.getvalue() return (output, len(input)) diff --git a/Lib/test/test_codecs.py b/Lib/test/test_codecs.py --- a/Lib/test/test_codecs.py +++ b/Lib/test/test_codecs.py @@ -2103,6 +2103,14 @@ class TransformCodecTest(unittest.TestCase): + def test_quopri_stateless(self): + # Should encode with quotetabs=True + encoded = codecs.encode(b"space tab\teol \n", "quopri-codec") + self.assertEqual(encoded, b"space=20tab=09eol=20\n") + # But should still support unescaped tabs and spaces + unescaped = b"space tab eol\n" + self.assertEqual(codecs.decode(unescaped, "quopri-codec"), unescaped) + def test_uu_invalid(self): # Missing "begin" line self.assertRaises(ValueError, codecs.decode, "", "uu-codec") -- Repository URL: https://hg.python.org/cpython From solipsis at pitrou.net Sat Sep 12 10:45:16 2015 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Sat, 12 Sep 2015 08:45:16 +0000 Subject: [Python-checkins] Daily reference leaks (3ecb5766ba15): sum=18887 Message-ID: <20150912084515.68865.8554@psf.io> results for 3ecb5766ba15 on branch "default" -------------------------------------------- test_capi leaked [1598, 1598, 1598] references, sum=4794 test_capi leaked [387, 389, 389] memory blocks, sum=1165 test_deque leaked [257, 257, 257] references, sum=771 test_deque leaked [79, 80, 80] memory blocks, sum=239 test_functools leaked [0, 2, 2] memory blocks, sum=4 test_threading leaked [3196, 3196, 3196] references, sum=9588 test_threading leaked [774, 776, 776] memory blocks, sum=2326 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/psf-users/antoine/refleaks/reflogrY6Khr', '--timeout', '7200'] From python-checkins at python.org Sat Sep 12 12:48:25 2015 From: python-checkins at python.org (eric.smith) Date: Sat, 12 Sep 2015 10:48:25 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_Fix_2_typos=2E_Thanks_Johan_M?= =?utf-8?q?usaeus_Bruun=2E?= Message-ID: <20150912104825.15722.22189@psf.io> https://hg.python.org/peps/rev/ef508976b305 changeset: 6048:ef508976b305 user: Eric V. Smith date: Sat Sep 12 06:48:29 2015 -0400 summary: Fix 2 typos. Thanks Johan Musaeus Bruun. files: pep-0498.txt | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/pep-0498.txt b/pep-0498.txt --- a/pep-0498.txt +++ b/pep-0498.txt @@ -294,7 +294,7 @@ Might be be evaluated as:: - 'abc' + expr1.__format__(spec1) + repr(expr2).__format__(spec2) + 'def' + str(spec3).__format__('') + 'ghi' + 'abc' + expr1.__format__(spec1) + repr(expr2).__format__(spec2) + 'def' + str(expr3).__format__('') + 'ghi' Expression evaluation --------------------- @@ -372,7 +372,7 @@ While the exact method of this run time concatenation is unspecified, the above code might evaluate to:: - 'ab' + x.__format__('') + '{c}' + 'str<' + y.__format__('^4') + 'de' + 'ab' + x.__format__('') + '{c}' + 'str<' + y.__format__('^4') + '>de' Error handling -------------- -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Sat Sep 12 12:55:37 2015 From: python-checkins at python.org (eric.smith) Date: Sat, 12 Sep 2015 10:55:37 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_Added_a_note_about_concatenat?= =?utf-8?q?ing_adjacent_f-strings=2E?= Message-ID: <20150912105537.14861.56714@psf.io> https://hg.python.org/peps/rev/eecc65fd0c06 changeset: 6049:eecc65fd0c06 user: Eric V. Smith date: Sat Sep 12 06:55:44 2015 -0400 summary: Added a note about concatenating adjacent f-strings. files: pep-0498.txt | 8 ++++++++ 1 files changed, 8 insertions(+), 0 deletions(-) diff --git a/pep-0498.txt b/pep-0498.txt --- a/pep-0498.txt +++ b/pep-0498.txt @@ -374,6 +374,14 @@ 'ab' + x.__format__('') + '{c}' + 'str<' + y.__format__('^4') + '>de' +Each f-string is entirely evaluated before being concatenated to +adjacent f-strings. That means that this:: + + >>> f'{x' f'}' + +Is a syntax error, because the first f-string does not contain a +closing brace. + Error handling -------------- -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Sat Sep 12 16:48:37 2015 From: python-checkins at python.org (serhiy.storchaka) Date: Sat, 12 Sep 2015 14:48:37 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_Marked_keystro?= =?utf-8?q?kes_with_the_=3Akbd=3A_role=2E?= Message-ID: <20150912144837.17961.82206@psf.io> https://hg.python.org/cpython/rev/39e2300f6267 changeset: 97942:39e2300f6267 branch: 2.7 parent: 97938:cfb0481c89d7 user: Serhiy Storchaka date: Sat Sep 12 17:47:12 2015 +0300 summary: Marked keystrokes with the :kbd: role. Fixed the case of the "Ctrl-" prefixes. files: Doc/faq/extending.rst | 2 +- Doc/faq/windows.rst | 12 ++++++------ Doc/library/idle.rst | 4 ++-- Doc/library/multiprocessing.rst | 2 +- Doc/library/signal.rst | 4 ++-- Doc/library/ttk.rst | 6 +++--- Doc/library/unittest.rst | 4 ++-- Doc/tutorial/appendix.rst | 2 +- Doc/tutorial/interpreter.rst | 2 +- Doc/using/cmdline.rst | 2 +- Doc/whatsnew/2.0.rst | 4 ++-- Doc/whatsnew/2.5.rst | 2 +- Doc/whatsnew/2.7.rst | 2 +- Lib/test/test_os.py | 4 ++-- Lib/unittest/main.py | 2 +- 15 files changed, 27 insertions(+), 27 deletions(-) diff --git a/Doc/faq/extending.rst b/Doc/faq/extending.rst --- a/Doc/faq/extending.rst +++ b/Doc/faq/extending.rst @@ -345,7 +345,7 @@ { line = readline (prompt); - if (NULL == line) /* CTRL-D pressed */ + if (NULL == line) /* Ctrl-D pressed */ { done = 1; } diff --git a/Doc/faq/windows.rst b/Doc/faq/windows.rst --- a/Doc/faq/windows.rst +++ b/Doc/faq/windows.rst @@ -77,14 +77,14 @@ 'HelloHelloHello' Many people use the interactive mode as a convenient yet highly programmable -calculator. When you want to end your interactive Python session, hold the Ctrl -key down while you enter a Z, then hit the "Enter" key to get back to your +calculator. When you want to end your interactive Python session, hold the :kbd:`Ctrl` +key down while you enter a :kbd:`Z`, then hit the ":kbd:`Enter`" key to get back to your Windows command prompt. You may also find that you have a Start-menu entry such as :menuselection:`Start --> Programs --> Python 2.7 --> Python (command line)` that results in you seeing the ``>>>`` prompt in a new window. If so, the window will disappear -after you enter the Ctrl-Z character; Windows is running a single "python" +after you enter the :kbd:`Ctrl-Z` character; Windows is running a single "python" command in the window, and closes it when you terminate the interpreter. If the ``python`` command, instead of displaying the interpreter prompt ``>>>``, @@ -127,8 +127,8 @@ c:\Python27\python -starts up the interpreter as above (and don't forget you'll need a "CTRL-Z" and -an "Enter" to get out of it). Once you have verified the directory, you can +starts up the interpreter as above (and don't forget you'll need a ":kbd:`Ctrl-Z`" and +an ":kbd:`Enter`" to get out of it). Once you have verified the directory, you can add it to the system path to make it easier to start Python by just running the ``python`` command. This is currently an option in the installer as of CPython 2.7. @@ -321,7 +321,7 @@ return (0 != kernel32.TerminateProcess(handle, 0)) In 2.7 and 3.2, :func:`os.kill` is implemented similar to the above function, -with the additional feature of being able to send CTRL+C and CTRL+BREAK +with the additional feature of being able to send :kbd:`Ctrl+C` and :kbd:`Ctrl+Break` to console subprocesses which are designed to handle those signals. See :func:`os.kill` for further details. diff --git a/Doc/library/idle.rst b/Doc/library/idle.rst --- a/Doc/library/idle.rst +++ b/Doc/library/idle.rst @@ -330,8 +330,8 @@ Editing and navigation ---------------------- -In this section, 'C' refers to the Control key on Windows and Unix and -the Command key on Mac OSX. +In this section, 'C' refers to the :kbd:`Control` key on Windows and Unix and +the :kbd:`Command` key on Mac OSX. * :kbd:`Backspace` deletes to the left; :kbd:`Del` deletes to the right diff --git a/Doc/library/multiprocessing.rst b/Doc/library/multiprocessing.rst --- a/Doc/library/multiprocessing.rst +++ b/Doc/library/multiprocessing.rst @@ -957,7 +957,7 @@ .. note:: - If the SIGINT signal generated by Ctrl-C arrives while the main thread is + If the SIGINT signal generated by :kbd:`Ctrl-C` arrives while the main thread is blocked by a call to :meth:`BoundedSemaphore.acquire`, :meth:`Lock.acquire`, :meth:`RLock.acquire`, :meth:`Semaphore.acquire`, :meth:`Condition.acquire` or :meth:`Condition.wait` then the call will be immediately interrupted and diff --git a/Doc/library/signal.rst b/Doc/library/signal.rst --- a/Doc/library/signal.rst +++ b/Doc/library/signal.rst @@ -77,7 +77,7 @@ .. data:: CTRL_C_EVENT - The signal corresponding to the CTRL+C keystroke event. This signal can + The signal corresponding to the :kbd:`Ctrl+C` keystroke event. This signal can only be used with :func:`os.kill`. Availability: Windows. @@ -87,7 +87,7 @@ .. data:: CTRL_BREAK_EVENT - The signal corresponding to the CTRL+BREAK keystroke event. This signal can + The signal corresponding to the :kbd:`Ctrl+Break` keystroke event. This signal can only be used with :func:`os.kill`. Availability: Windows. diff --git a/Doc/library/ttk.rst b/Doc/library/ttk.rst --- a/Doc/library/ttk.rst +++ b/Doc/library/ttk.rst @@ -544,9 +544,9 @@ This will extend the bindings for the toplevel window containing the notebook as follows: - * Control-Tab: selects the tab following the currently selected one. - * Shift-Control-Tab: selects the tab preceding the currently selected one. - * Alt-K: where K is the mnemonic (underlined) character of any tab, will + * :kbd:`Control-Tab`: selects the tab following the currently selected one. + * :kbd:`Shift-Control-Tab`: selects the tab preceding the currently selected one. + * :kbd:`Alt-K`: where *K* is the mnemonic (underlined) character of any tab, will select that tab. Multiple notebooks in a single toplevel may be enabled for traversal, diff --git a/Doc/library/unittest.rst b/Doc/library/unittest.rst --- a/Doc/library/unittest.rst +++ b/Doc/library/unittest.rst @@ -226,8 +226,8 @@ .. cmdoption:: -c, --catch - Control-C during the test run waits for the current test to end and then - reports all the results so far. A second control-C raises the normal + :kbd:`Control-C` during the test run waits for the current test to end and then + reports all the results so far. A second :kbd:`Control-C` raises the normal :exc:`KeyboardInterrupt` exception. See `Signal Handling`_ for the functions that provide this functionality. diff --git a/Doc/tutorial/appendix.rst b/Doc/tutorial/appendix.rst --- a/Doc/tutorial/appendix.rst +++ b/Doc/tutorial/appendix.rst @@ -25,7 +25,7 @@ standard error stream; normal output from executed commands is written to standard output. -Typing the interrupt character (usually Control-C or DEL) to the primary or +Typing the interrupt character (usually :kbd:`Control-C` or :kbd:`Delete`) to the primary or secondary prompt cancels the input and returns to the primary prompt. [#]_ Typing an interrupt while a command is executing raises the :exc:`KeyboardInterrupt` exception, which may be handled by a :keyword:`try` diff --git a/Doc/tutorial/interpreter.rst b/Doc/tutorial/interpreter.rst --- a/Doc/tutorial/interpreter.rst +++ b/Doc/tutorial/interpreter.rst @@ -37,7 +37,7 @@ Unix, whoever installed the interpreter may have enabled support for the GNU readline library, which adds more elaborate interactive editing and history features. Perhaps the quickest check to see whether command line editing is -supported is typing Control-P to the first Python prompt you get. If it beeps, +supported is typing :kbd:`Control-P` to the first Python prompt you get. If it beeps, you have command line editing; see Appendix :ref:`tut-interacting` for an introduction to the keys. If nothing appears to happen, or if ``^P`` is echoed, command line editing isn't available; you'll only be able to use backspace to diff --git a/Doc/using/cmdline.rst b/Doc/using/cmdline.rst --- a/Doc/using/cmdline.rst +++ b/Doc/using/cmdline.rst @@ -41,7 +41,7 @@ * When called with standard input connected to a tty device, it prompts for commands and executes them until an EOF (an end-of-file character, you can - produce that with *Ctrl-D* on UNIX or *Ctrl-Z, Enter* on Windows) is read. + produce that with :kbd:`Ctrl-D` on UNIX or :kbd:`Ctrl-Z, Enter` on Windows) is read. * When called with a file name argument or with a file as standard input, it reads and executes a script from that file. * When called with a directory name argument, it reads and executes an diff --git a/Doc/whatsnew/2.0.rst b/Doc/whatsnew/2.0.rst --- a/Doc/whatsnew/2.0.rst +++ b/Doc/whatsnew/2.0.rst @@ -1174,8 +1174,8 @@ * In the editor window, there is now a line/column bar at the bottom. -* Three new keystroke commands: Check module (Alt-F5), Import module (F5) and - Run script (Ctrl-F5). +* Three new keystroke commands: Check module (:kbd:`Alt-F5`), Import module (:kbd:`F5`) and + Run script (:kbd:`Ctrl-F5`). .. ====================================================================== diff --git a/Doc/whatsnew/2.5.rst b/Doc/whatsnew/2.5.rst --- a/Doc/whatsnew/2.5.rst +++ b/Doc/whatsnew/2.5.rst @@ -827,7 +827,7 @@ This rearrangement was done because people often want to catch all exceptions that indicate program errors. :exc:`KeyboardInterrupt` and :exc:`SystemExit` aren't errors, though, and usually represent an explicit action such as the user -hitting Control-C or code calling :func:`sys.exit`. A bare ``except:`` will +hitting :kbd:`Control-C` or code calling :func:`sys.exit`. A bare ``except:`` will catch all exceptions, so you commonly need to list :exc:`KeyboardInterrupt` and :exc:`SystemExit` in order to re-raise them. The usual pattern is:: diff --git a/Doc/whatsnew/2.7.rst b/Doc/whatsnew/2.7.rst --- a/Doc/whatsnew/2.7.rst +++ b/Doc/whatsnew/2.7.rst @@ -2320,7 +2320,7 @@ * The :func:`os.kill` function now works on Windows. The signal value can be the constants :const:`CTRL_C_EVENT`, :const:`CTRL_BREAK_EVENT`, or any integer. The first two constants - will send Control-C and Control-Break keystroke events to + will send :kbd:`Control-C` and :kbd:`Control-Break` keystroke events to subprocesses; any other value will use the :c:func:`TerminateProcess` API. (Contributed by Miki Tebeka; :issue:`1220212`.) diff --git a/Lib/test/test_os.py b/Lib/test/test_os.py --- a/Lib/test/test_os.py +++ b/Lib/test/test_os.py @@ -856,7 +856,7 @@ os.kill(proc.pid, signal.SIGINT) self.fail("subprocess did not stop on {}".format(name)) - @unittest.skip("subprocesses aren't inheriting CTRL+C property") + @unittest.skip("subprocesses aren't inheriting Ctrl+C property") def test_CTRL_C_EVENT(self): from ctypes import wintypes import ctypes @@ -869,7 +869,7 @@ SetConsoleCtrlHandler.restype = wintypes.BOOL # Calling this with NULL and FALSE causes the calling process to - # handle CTRL+C, rather than ignore it. This property is inherited + # handle Ctrl+C, rather than ignore it. This property is inherited # by subprocesses. SetConsoleCtrlHandler(NULL, 0) diff --git a/Lib/unittest/main.py b/Lib/unittest/main.py --- a/Lib/unittest/main.py +++ b/Lib/unittest/main.py @@ -174,7 +174,7 @@ action='store_true') if self.catchbreak != False: parser.add_option('-c', '--catch', dest='catchbreak', default=False, - help='Catch ctrl-C and display results so far', + help='Catch Ctrl-C and display results so far', action='store_true') if self.buffer != False: parser.add_option('-b', '--buffer', dest='buffer', default=False, -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 12 16:48:37 2015 From: python-checkins at python.org (serhiy.storchaka) Date: Sat, 12 Sep 2015 14:48:37 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Marked_keystrokes_with_the_=3Akbd=3A_role=2E?= Message-ID: <20150912144837.15730.89451@psf.io> https://hg.python.org/cpython/rev/cdd1679e666d changeset: 97940:cdd1679e666d branch: 3.5 parent: 97935:28cd11dc2915 parent: 97939:452805163bd5 user: Serhiy Storchaka date: Sat Sep 12 17:46:20 2015 +0300 summary: Marked keystrokes with the :kbd: role. Fixed the case of the "Ctrl-" prefixes. files: Doc/faq/extending.rst | 2 +- Doc/faq/windows.rst | 12 ++++++------ Doc/library/asyncio-eventloop.rst | 2 +- Doc/library/asyncio-protocol.rst | 2 +- Doc/library/asyncio-stream.rst | 2 +- Doc/library/idle.rst | 4 ++-- Doc/library/multiprocessing.rst | 2 +- Doc/library/pdb.rst | 4 ++-- Doc/library/signal.rst | 4 ++-- Doc/library/tkinter.ttk.rst | 6 +++--- Doc/library/unittest.rst | 4 ++-- Doc/tutorial/appendix.rst | 2 +- Doc/tutorial/interpreter.rst | 2 +- Doc/using/cmdline.rst | 2 +- Doc/whatsnew/2.0.rst | 4 ++-- Doc/whatsnew/2.5.rst | 2 +- Doc/whatsnew/2.7.rst | 2 +- Lib/test/test_os.py | 4 ++-- Lib/unittest/main.py | 2 +- 19 files changed, 32 insertions(+), 32 deletions(-) diff --git a/Doc/faq/extending.rst b/Doc/faq/extending.rst --- a/Doc/faq/extending.rst +++ b/Doc/faq/extending.rst @@ -348,7 +348,7 @@ { line = readline (prompt); - if (NULL == line) /* CTRL-D pressed */ + if (NULL == line) /* Ctrl-D pressed */ { done = 1; } diff --git a/Doc/faq/windows.rst b/Doc/faq/windows.rst --- a/Doc/faq/windows.rst +++ b/Doc/faq/windows.rst @@ -81,14 +81,14 @@ 'HelloHelloHello' Many people use the interactive mode as a convenient yet highly programmable -calculator. When you want to end your interactive Python session, hold the Ctrl -key down while you enter a Z, then hit the "Enter" key to get back to your +calculator. When you want to end your interactive Python session, hold the :kbd:`Ctrl` +key down while you enter a :kbd:`Z`, then hit the ":kbd:`Enter`" key to get back to your Windows command prompt. You may also find that you have a Start-menu entry such as :menuselection:`Start --> Programs --> Python 3.3 --> Python (command line)` that results in you seeing the ``>>>`` prompt in a new window. If so, the window will disappear -after you enter the Ctrl-Z character; Windows is running a single "python" +after you enter the :kbd:`Ctrl-Z` character; Windows is running a single "python" command in the window, and closes it when you terminate the interpreter. If the ``python`` command, instead of displaying the interpreter prompt ``>>>``, @@ -131,8 +131,8 @@ c:\Python33\python -starts up the interpreter as above (and don't forget you'll need a "CTRL-Z" and -an "Enter" to get out of it). Once you have verified the directory, you can +starts up the interpreter as above (and don't forget you'll need a ":kbd:`Ctrl-Z`" and +an ":kbd:`Enter`" to get out of it). Once you have verified the directory, you can add it to the system path to make it easier to start Python by just running the ``python`` command. This is currently an option in the installer as of CPython 3.3. @@ -327,7 +327,7 @@ return (0 != kernel32.TerminateProcess(handle, 0)) In 2.7 and 3.2, :func:`os.kill` is implemented similar to the above function, -with the additional feature of being able to send CTRL+C and CTRL+BREAK +with the additional feature of being able to send :kbd:`Ctrl+C` and :kbd:`Ctrl+Break` to console subprocesses which are designed to handle those signals. See :func:`os.kill` for further details. diff --git a/Doc/library/asyncio-eventloop.rst b/Doc/library/asyncio-eventloop.rst --- a/Doc/library/asyncio-eventloop.rst +++ b/Doc/library/asyncio-eventloop.rst @@ -853,7 +853,7 @@ loop.add_signal_handler(getattr(signal, signame), functools.partial(ask_exit, signame)) - print("Event loop running forever, press CTRL+c to interrupt.") + print("Event loop running forever, press Ctrl+C to interrupt.") print("pid %s: send SIGINT or SIGTERM to exit." % os.getpid()) try: loop.run_forever() diff --git a/Doc/library/asyncio-protocol.rst b/Doc/library/asyncio-protocol.rst --- a/Doc/library/asyncio-protocol.rst +++ b/Doc/library/asyncio-protocol.rst @@ -539,7 +539,7 @@ coro = loop.create_server(EchoServerClientProtocol, '127.0.0.1', 8888) server = loop.run_until_complete(coro) - # Serve requests until CTRL+c is pressed + # Serve requests until Ctrl+C is pressed print('Serving on {}'.format(server.sockets[0].getsockname())) try: loop.run_forever() diff --git a/Doc/library/asyncio-stream.rst b/Doc/library/asyncio-stream.rst --- a/Doc/library/asyncio-stream.rst +++ b/Doc/library/asyncio-stream.rst @@ -312,7 +312,7 @@ coro = asyncio.start_server(handle_echo, '127.0.0.1', 8888, loop=loop) server = loop.run_until_complete(coro) - # Serve requests until CTRL+c is pressed + # Serve requests until Ctrl+C is pressed print('Serving on {}'.format(server.sockets[0].getsockname())) try: loop.run_forever() diff --git a/Doc/library/idle.rst b/Doc/library/idle.rst --- a/Doc/library/idle.rst +++ b/Doc/library/idle.rst @@ -330,8 +330,8 @@ Editing and navigation ---------------------- -In this section, 'C' refers to the Control key on Windows and Unix and -the Command key on Mac OSX. +In this section, 'C' refers to the :kbd:`Control` key on Windows and Unix and +the :kbd:`Command` key on Mac OSX. * :kbd:`Backspace` deletes to the left; :kbd:`Del` deletes to the right diff --git a/Doc/library/multiprocessing.rst b/Doc/library/multiprocessing.rst --- a/Doc/library/multiprocessing.rst +++ b/Doc/library/multiprocessing.rst @@ -1174,7 +1174,7 @@ .. note:: - If the SIGINT signal generated by Ctrl-C arrives while the main thread is + If the SIGINT signal generated by :kbd:`Ctrl-C` arrives while the main thread is blocked by a call to :meth:`BoundedSemaphore.acquire`, :meth:`Lock.acquire`, :meth:`RLock.acquire`, :meth:`Semaphore.acquire`, :meth:`Condition.acquire` or :meth:`Condition.wait` then the call will be immediately interrupted and diff --git a/Doc/library/pdb.rst b/Doc/library/pdb.rst --- a/Doc/library/pdb.rst +++ b/Doc/library/pdb.rst @@ -156,8 +156,8 @@ that matches one of these patterns. [1]_ By default, Pdb sets a handler for the SIGINT signal (which is sent when the - user presses Ctrl-C on the console) when you give a ``continue`` command. - This allows you to break into the debugger again by pressing Ctrl-C. If you + user presses :kbd:`Ctrl-C` on the console) when you give a ``continue`` command. + This allows you to break into the debugger again by pressing :kbd:`Ctrl-C`. If you want Pdb not to touch the SIGINT handler, set *nosigint* tot true. Example call to enable tracing with *skip*:: diff --git a/Doc/library/signal.rst b/Doc/library/signal.rst --- a/Doc/library/signal.rst +++ b/Doc/library/signal.rst @@ -105,7 +105,7 @@ .. data:: CTRL_C_EVENT - The signal corresponding to the CTRL+C keystroke event. This signal can + The signal corresponding to the :kbd:`Ctrl+C` keystroke event. This signal can only be used with :func:`os.kill`. Availability: Windows. @@ -115,7 +115,7 @@ .. data:: CTRL_BREAK_EVENT - The signal corresponding to the CTRL+BREAK keystroke event. This signal can + The signal corresponding to the :kbd:`Ctrl+Break` keystroke event. This signal can only be used with :func:`os.kill`. Availability: Windows. diff --git a/Doc/library/tkinter.ttk.rst b/Doc/library/tkinter.ttk.rst --- a/Doc/library/tkinter.ttk.rst +++ b/Doc/library/tkinter.ttk.rst @@ -554,9 +554,9 @@ This will extend the bindings for the toplevel window containing the notebook as follows: - * Control-Tab: selects the tab following the currently selected one. - * Shift-Control-Tab: selects the tab preceding the currently selected one. - * Alt-K: where K is the mnemonic (underlined) character of any tab, will + * :kbd:`Control-Tab`: selects the tab following the currently selected one. + * :kbd:`Shift-Control-Tab`: selects the tab preceding the currently selected one. + * :kbd:`Alt-K`: where *K* is the mnemonic (underlined) character of any tab, will select that tab. Multiple notebooks in a single toplevel may be enabled for traversal, diff --git a/Doc/library/unittest.rst b/Doc/library/unittest.rst --- a/Doc/library/unittest.rst +++ b/Doc/library/unittest.rst @@ -204,8 +204,8 @@ .. cmdoption:: -c, --catch - Control-C during the test run waits for the current test to end and then - reports all the results so far. A second control-C raises the normal + :kbd:`Control-C` during the test run waits for the current test to end and then + reports all the results so far. A second :kbd:`Control-C` raises the normal :exc:`KeyboardInterrupt` exception. See `Signal Handling`_ for the functions that provide this functionality. diff --git a/Doc/tutorial/appendix.rst b/Doc/tutorial/appendix.rst --- a/Doc/tutorial/appendix.rst +++ b/Doc/tutorial/appendix.rst @@ -25,7 +25,7 @@ standard error stream; normal output from executed commands is written to standard output. -Typing the interrupt character (usually Control-C or DEL) to the primary or +Typing the interrupt character (usually :kbd:`Control-C` or :kbd:`Delete`) to the primary or secondary prompt cancels the input and returns to the primary prompt. [#]_ Typing an interrupt while a command is executing raises the :exc:`KeyboardInterrupt` exception, which may be handled by a :keyword:`try` diff --git a/Doc/tutorial/interpreter.rst b/Doc/tutorial/interpreter.rst --- a/Doc/tutorial/interpreter.rst +++ b/Doc/tutorial/interpreter.rst @@ -38,7 +38,7 @@ The interpreter's line-editing features include interactive editing, history substitution and code completion on systems that support readline. Perhaps the quickest check to see whether command line editing is supported is typing -Control-P to the first Python prompt you get. If it beeps, you have command +:kbd:`Control-P` to the first Python prompt you get. If it beeps, you have command line editing; see Appendix :ref:`tut-interacting` for an introduction to the keys. If nothing appears to happen, or if ``^P`` is echoed, command line editing isn't available; you'll only be able to use backspace to remove diff --git a/Doc/using/cmdline.rst b/Doc/using/cmdline.rst --- a/Doc/using/cmdline.rst +++ b/Doc/using/cmdline.rst @@ -41,7 +41,7 @@ * When called with standard input connected to a tty device, it prompts for commands and executes them until an EOF (an end-of-file character, you can - produce that with *Ctrl-D* on UNIX or *Ctrl-Z, Enter* on Windows) is read. + produce that with :kbd:`Ctrl-D` on UNIX or :kbd:`Ctrl-Z, Enter` on Windows) is read. * When called with a file name argument or with a file as standard input, it reads and executes a script from that file. * When called with a directory name argument, it reads and executes an diff --git a/Doc/whatsnew/2.0.rst b/Doc/whatsnew/2.0.rst --- a/Doc/whatsnew/2.0.rst +++ b/Doc/whatsnew/2.0.rst @@ -1174,8 +1174,8 @@ * In the editor window, there is now a line/column bar at the bottom. -* Three new keystroke commands: Check module (Alt-F5), Import module (F5) and - Run script (Ctrl-F5). +* Three new keystroke commands: Check module (:kbd:`Alt-F5`), Import module (:kbd:`F5`) and + Run script (:kbd:`Ctrl-F5`). .. ====================================================================== diff --git a/Doc/whatsnew/2.5.rst b/Doc/whatsnew/2.5.rst --- a/Doc/whatsnew/2.5.rst +++ b/Doc/whatsnew/2.5.rst @@ -827,7 +827,7 @@ This rearrangement was done because people often want to catch all exceptions that indicate program errors. :exc:`KeyboardInterrupt` and :exc:`SystemExit` aren't errors, though, and usually represent an explicit action such as the user -hitting Control-C or code calling :func:`sys.exit`. A bare ``except:`` will +hitting :kbd:`Control-C` or code calling :func:`sys.exit`. A bare ``except:`` will catch all exceptions, so you commonly need to list :exc:`KeyboardInterrupt` and :exc:`SystemExit` in order to re-raise them. The usual pattern is:: diff --git a/Doc/whatsnew/2.7.rst b/Doc/whatsnew/2.7.rst --- a/Doc/whatsnew/2.7.rst +++ b/Doc/whatsnew/2.7.rst @@ -2320,7 +2320,7 @@ * The :func:`os.kill` function now works on Windows. The signal value can be the constants :const:`CTRL_C_EVENT`, :const:`CTRL_BREAK_EVENT`, or any integer. The first two constants - will send Control-C and Control-Break keystroke events to + will send :kbd:`Control-C` and :kbd:`Control-Break` keystroke events to subprocesses; any other value will use the :c:func:`TerminateProcess` API. (Contributed by Miki Tebeka; :issue:`1220212`.) diff --git a/Lib/test/test_os.py b/Lib/test/test_os.py --- a/Lib/test/test_os.py +++ b/Lib/test/test_os.py @@ -1786,7 +1786,7 @@ os.kill(proc.pid, signal.SIGINT) self.fail("subprocess did not stop on {}".format(name)) - @unittest.skip("subprocesses aren't inheriting CTRL+C property") + @unittest.skip("subprocesses aren't inheriting Ctrl+C property") def test_CTRL_C_EVENT(self): from ctypes import wintypes import ctypes @@ -1799,7 +1799,7 @@ SetConsoleCtrlHandler.restype = wintypes.BOOL # Calling this with NULL and FALSE causes the calling process to - # handle CTRL+C, rather than ignore it. This property is inherited + # handle Ctrl+C, rather than ignore it. This property is inherited # by subprocesses. SetConsoleCtrlHandler(NULL, 0) diff --git a/Lib/unittest/main.py b/Lib/unittest/main.py --- a/Lib/unittest/main.py +++ b/Lib/unittest/main.py @@ -171,7 +171,7 @@ if self.catchbreak is None: parser.add_argument('-c', '--catch', dest='catchbreak', action='store_true', - help='Catch ctrl-C and display results so far') + help='Catch Ctrl-C and display results so far') self.catchbreak = False if self.buffer is None: parser.add_argument('-b', '--buffer', dest='buffer', -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 12 16:48:37 2015 From: python-checkins at python.org (serhiy.storchaka) Date: Sat, 12 Sep 2015 14:48:37 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Marked_keystrokes_with_the_=3Akbd=3A_role=2E?= Message-ID: <20150912144837.12009.94247@psf.io> https://hg.python.org/cpython/rev/b8f3a01937be changeset: 97941:b8f3a01937be parent: 97936:3ecb5766ba15 parent: 97940:cdd1679e666d user: Serhiy Storchaka date: Sat Sep 12 17:46:56 2015 +0300 summary: Marked keystrokes with the :kbd: role. Fixed the case of the "Ctrl-" prefixes. files: Doc/faq/extending.rst | 2 +- Doc/faq/windows.rst | 12 ++++++------ Doc/library/asyncio-eventloop.rst | 2 +- Doc/library/asyncio-protocol.rst | 2 +- Doc/library/asyncio-stream.rst | 2 +- Doc/library/idle.rst | 4 ++-- Doc/library/multiprocessing.rst | 2 +- Doc/library/pdb.rst | 4 ++-- Doc/library/signal.rst | 4 ++-- Doc/library/tkinter.ttk.rst | 6 +++--- Doc/library/unittest.rst | 4 ++-- Doc/tutorial/appendix.rst | 2 +- Doc/tutorial/interpreter.rst | 2 +- Doc/using/cmdline.rst | 2 +- Doc/whatsnew/2.0.rst | 4 ++-- Doc/whatsnew/2.5.rst | 2 +- Doc/whatsnew/2.7.rst | 2 +- Lib/test/test_os.py | 4 ++-- Lib/unittest/main.py | 2 +- 19 files changed, 32 insertions(+), 32 deletions(-) diff --git a/Doc/faq/extending.rst b/Doc/faq/extending.rst --- a/Doc/faq/extending.rst +++ b/Doc/faq/extending.rst @@ -348,7 +348,7 @@ { line = readline (prompt); - if (NULL == line) /* CTRL-D pressed */ + if (NULL == line) /* Ctrl-D pressed */ { done = 1; } diff --git a/Doc/faq/windows.rst b/Doc/faq/windows.rst --- a/Doc/faq/windows.rst +++ b/Doc/faq/windows.rst @@ -81,14 +81,14 @@ 'HelloHelloHello' Many people use the interactive mode as a convenient yet highly programmable -calculator. When you want to end your interactive Python session, hold the Ctrl -key down while you enter a Z, then hit the "Enter" key to get back to your +calculator. When you want to end your interactive Python session, hold the :kbd:`Ctrl` +key down while you enter a :kbd:`Z`, then hit the ":kbd:`Enter`" key to get back to your Windows command prompt. You may also find that you have a Start-menu entry such as :menuselection:`Start --> Programs --> Python 3.3 --> Python (command line)` that results in you seeing the ``>>>`` prompt in a new window. If so, the window will disappear -after you enter the Ctrl-Z character; Windows is running a single "python" +after you enter the :kbd:`Ctrl-Z` character; Windows is running a single "python" command in the window, and closes it when you terminate the interpreter. If the ``python`` command, instead of displaying the interpreter prompt ``>>>``, @@ -131,8 +131,8 @@ c:\Python33\python -starts up the interpreter as above (and don't forget you'll need a "CTRL-Z" and -an "Enter" to get out of it). Once you have verified the directory, you can +starts up the interpreter as above (and don't forget you'll need a ":kbd:`Ctrl-Z`" and +an ":kbd:`Enter`" to get out of it). Once you have verified the directory, you can add it to the system path to make it easier to start Python by just running the ``python`` command. This is currently an option in the installer as of CPython 3.3. @@ -327,7 +327,7 @@ return (0 != kernel32.TerminateProcess(handle, 0)) In 2.7 and 3.2, :func:`os.kill` is implemented similar to the above function, -with the additional feature of being able to send CTRL+C and CTRL+BREAK +with the additional feature of being able to send :kbd:`Ctrl+C` and :kbd:`Ctrl+Break` to console subprocesses which are designed to handle those signals. See :func:`os.kill` for further details. diff --git a/Doc/library/asyncio-eventloop.rst b/Doc/library/asyncio-eventloop.rst --- a/Doc/library/asyncio-eventloop.rst +++ b/Doc/library/asyncio-eventloop.rst @@ -853,7 +853,7 @@ loop.add_signal_handler(getattr(signal, signame), functools.partial(ask_exit, signame)) - print("Event loop running forever, press CTRL+c to interrupt.") + print("Event loop running forever, press Ctrl+C to interrupt.") print("pid %s: send SIGINT or SIGTERM to exit." % os.getpid()) try: loop.run_forever() diff --git a/Doc/library/asyncio-protocol.rst b/Doc/library/asyncio-protocol.rst --- a/Doc/library/asyncio-protocol.rst +++ b/Doc/library/asyncio-protocol.rst @@ -539,7 +539,7 @@ coro = loop.create_server(EchoServerClientProtocol, '127.0.0.1', 8888) server = loop.run_until_complete(coro) - # Serve requests until CTRL+c is pressed + # Serve requests until Ctrl+C is pressed print('Serving on {}'.format(server.sockets[0].getsockname())) try: loop.run_forever() diff --git a/Doc/library/asyncio-stream.rst b/Doc/library/asyncio-stream.rst --- a/Doc/library/asyncio-stream.rst +++ b/Doc/library/asyncio-stream.rst @@ -312,7 +312,7 @@ coro = asyncio.start_server(handle_echo, '127.0.0.1', 8888, loop=loop) server = loop.run_until_complete(coro) - # Serve requests until CTRL+c is pressed + # Serve requests until Ctrl+C is pressed print('Serving on {}'.format(server.sockets[0].getsockname())) try: loop.run_forever() diff --git a/Doc/library/idle.rst b/Doc/library/idle.rst --- a/Doc/library/idle.rst +++ b/Doc/library/idle.rst @@ -330,8 +330,8 @@ Editing and navigation ---------------------- -In this section, 'C' refers to the Control key on Windows and Unix and -the Command key on Mac OSX. +In this section, 'C' refers to the :kbd:`Control` key on Windows and Unix and +the :kbd:`Command` key on Mac OSX. * :kbd:`Backspace` deletes to the left; :kbd:`Del` deletes to the right diff --git a/Doc/library/multiprocessing.rst b/Doc/library/multiprocessing.rst --- a/Doc/library/multiprocessing.rst +++ b/Doc/library/multiprocessing.rst @@ -1179,7 +1179,7 @@ .. note:: - If the SIGINT signal generated by Ctrl-C arrives while the main thread is + If the SIGINT signal generated by :kbd:`Ctrl-C` arrives while the main thread is blocked by a call to :meth:`BoundedSemaphore.acquire`, :meth:`Lock.acquire`, :meth:`RLock.acquire`, :meth:`Semaphore.acquire`, :meth:`Condition.acquire` or :meth:`Condition.wait` then the call will be immediately interrupted and diff --git a/Doc/library/pdb.rst b/Doc/library/pdb.rst --- a/Doc/library/pdb.rst +++ b/Doc/library/pdb.rst @@ -156,8 +156,8 @@ that matches one of these patterns. [1]_ By default, Pdb sets a handler for the SIGINT signal (which is sent when the - user presses Ctrl-C on the console) when you give a ``continue`` command. - This allows you to break into the debugger again by pressing Ctrl-C. If you + user presses :kbd:`Ctrl-C` on the console) when you give a ``continue`` command. + This allows you to break into the debugger again by pressing :kbd:`Ctrl-C`. If you want Pdb not to touch the SIGINT handler, set *nosigint* tot true. Example call to enable tracing with *skip*:: diff --git a/Doc/library/signal.rst b/Doc/library/signal.rst --- a/Doc/library/signal.rst +++ b/Doc/library/signal.rst @@ -105,7 +105,7 @@ .. data:: CTRL_C_EVENT - The signal corresponding to the CTRL+C keystroke event. This signal can + The signal corresponding to the :kbd:`Ctrl+C` keystroke event. This signal can only be used with :func:`os.kill`. Availability: Windows. @@ -115,7 +115,7 @@ .. data:: CTRL_BREAK_EVENT - The signal corresponding to the CTRL+BREAK keystroke event. This signal can + The signal corresponding to the :kbd:`Ctrl+Break` keystroke event. This signal can only be used with :func:`os.kill`. Availability: Windows. diff --git a/Doc/library/tkinter.ttk.rst b/Doc/library/tkinter.ttk.rst --- a/Doc/library/tkinter.ttk.rst +++ b/Doc/library/tkinter.ttk.rst @@ -554,9 +554,9 @@ This will extend the bindings for the toplevel window containing the notebook as follows: - * Control-Tab: selects the tab following the currently selected one. - * Shift-Control-Tab: selects the tab preceding the currently selected one. - * Alt-K: where K is the mnemonic (underlined) character of any tab, will + * :kbd:`Control-Tab`: selects the tab following the currently selected one. + * :kbd:`Shift-Control-Tab`: selects the tab preceding the currently selected one. + * :kbd:`Alt-K`: where *K* is the mnemonic (underlined) character of any tab, will select that tab. Multiple notebooks in a single toplevel may be enabled for traversal, diff --git a/Doc/library/unittest.rst b/Doc/library/unittest.rst --- a/Doc/library/unittest.rst +++ b/Doc/library/unittest.rst @@ -204,8 +204,8 @@ .. cmdoption:: -c, --catch - Control-C during the test run waits for the current test to end and then - reports all the results so far. A second control-C raises the normal + :kbd:`Control-C` during the test run waits for the current test to end and then + reports all the results so far. A second :kbd:`Control-C` raises the normal :exc:`KeyboardInterrupt` exception. See `Signal Handling`_ for the functions that provide this functionality. diff --git a/Doc/tutorial/appendix.rst b/Doc/tutorial/appendix.rst --- a/Doc/tutorial/appendix.rst +++ b/Doc/tutorial/appendix.rst @@ -25,7 +25,7 @@ standard error stream; normal output from executed commands is written to standard output. -Typing the interrupt character (usually Control-C or DEL) to the primary or +Typing the interrupt character (usually :kbd:`Control-C` or :kbd:`Delete`) to the primary or secondary prompt cancels the input and returns to the primary prompt. [#]_ Typing an interrupt while a command is executing raises the :exc:`KeyboardInterrupt` exception, which may be handled by a :keyword:`try` diff --git a/Doc/tutorial/interpreter.rst b/Doc/tutorial/interpreter.rst --- a/Doc/tutorial/interpreter.rst +++ b/Doc/tutorial/interpreter.rst @@ -38,7 +38,7 @@ The interpreter's line-editing features include interactive editing, history substitution and code completion on systems that support readline. Perhaps the quickest check to see whether command line editing is supported is typing -Control-P to the first Python prompt you get. If it beeps, you have command +:kbd:`Control-P` to the first Python prompt you get. If it beeps, you have command line editing; see Appendix :ref:`tut-interacting` for an introduction to the keys. If nothing appears to happen, or if ``^P`` is echoed, command line editing isn't available; you'll only be able to use backspace to remove diff --git a/Doc/using/cmdline.rst b/Doc/using/cmdline.rst --- a/Doc/using/cmdline.rst +++ b/Doc/using/cmdline.rst @@ -41,7 +41,7 @@ * When called with standard input connected to a tty device, it prompts for commands and executes them until an EOF (an end-of-file character, you can - produce that with *Ctrl-D* on UNIX or *Ctrl-Z, Enter* on Windows) is read. + produce that with :kbd:`Ctrl-D` on UNIX or :kbd:`Ctrl-Z, Enter` on Windows) is read. * When called with a file name argument or with a file as standard input, it reads and executes a script from that file. * When called with a directory name argument, it reads and executes an diff --git a/Doc/whatsnew/2.0.rst b/Doc/whatsnew/2.0.rst --- a/Doc/whatsnew/2.0.rst +++ b/Doc/whatsnew/2.0.rst @@ -1174,8 +1174,8 @@ * In the editor window, there is now a line/column bar at the bottom. -* Three new keystroke commands: Check module (Alt-F5), Import module (F5) and - Run script (Ctrl-F5). +* Three new keystroke commands: Check module (:kbd:`Alt-F5`), Import module (:kbd:`F5`) and + Run script (:kbd:`Ctrl-F5`). .. ====================================================================== diff --git a/Doc/whatsnew/2.5.rst b/Doc/whatsnew/2.5.rst --- a/Doc/whatsnew/2.5.rst +++ b/Doc/whatsnew/2.5.rst @@ -827,7 +827,7 @@ This rearrangement was done because people often want to catch all exceptions that indicate program errors. :exc:`KeyboardInterrupt` and :exc:`SystemExit` aren't errors, though, and usually represent an explicit action such as the user -hitting Control-C or code calling :func:`sys.exit`. A bare ``except:`` will +hitting :kbd:`Control-C` or code calling :func:`sys.exit`. A bare ``except:`` will catch all exceptions, so you commonly need to list :exc:`KeyboardInterrupt` and :exc:`SystemExit` in order to re-raise them. The usual pattern is:: diff --git a/Doc/whatsnew/2.7.rst b/Doc/whatsnew/2.7.rst --- a/Doc/whatsnew/2.7.rst +++ b/Doc/whatsnew/2.7.rst @@ -2320,7 +2320,7 @@ * The :func:`os.kill` function now works on Windows. The signal value can be the constants :const:`CTRL_C_EVENT`, :const:`CTRL_BREAK_EVENT`, or any integer. The first two constants - will send Control-C and Control-Break keystroke events to + will send :kbd:`Control-C` and :kbd:`Control-Break` keystroke events to subprocesses; any other value will use the :c:func:`TerminateProcess` API. (Contributed by Miki Tebeka; :issue:`1220212`.) diff --git a/Lib/test/test_os.py b/Lib/test/test_os.py --- a/Lib/test/test_os.py +++ b/Lib/test/test_os.py @@ -1785,7 +1785,7 @@ os.kill(proc.pid, signal.SIGINT) self.fail("subprocess did not stop on {}".format(name)) - @unittest.skip("subprocesses aren't inheriting CTRL+C property") + @unittest.skip("subprocesses aren't inheriting Ctrl+C property") def test_CTRL_C_EVENT(self): from ctypes import wintypes import ctypes @@ -1798,7 +1798,7 @@ SetConsoleCtrlHandler.restype = wintypes.BOOL # Calling this with NULL and FALSE causes the calling process to - # handle CTRL+C, rather than ignore it. This property is inherited + # handle Ctrl+C, rather than ignore it. This property is inherited # by subprocesses. SetConsoleCtrlHandler(NULL, 0) diff --git a/Lib/unittest/main.py b/Lib/unittest/main.py --- a/Lib/unittest/main.py +++ b/Lib/unittest/main.py @@ -171,7 +171,7 @@ if self.catchbreak is None: parser.add_argument('-c', '--catch', dest='catchbreak', action='store_true', - help='Catch ctrl-C and display results so far') + help='Catch Ctrl-C and display results so far') self.catchbreak = False if self.buffer is None: parser.add_argument('-b', '--buffer', dest='buffer', -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 12 16:48:37 2015 From: python-checkins at python.org (serhiy.storchaka) Date: Sat, 12 Sep 2015 14:48:37 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_Marked_keystro?= =?utf-8?q?kes_with_the_=3Akbd=3A_role=2E?= Message-ID: <20150912144836.114789.90529@psf.io> https://hg.python.org/cpython/rev/452805163bd5 changeset: 97939:452805163bd5 branch: 3.4 parent: 97934:de82f41d6669 user: Serhiy Storchaka date: Sat Sep 12 17:45:25 2015 +0300 summary: Marked keystrokes with the :kbd: role. Fixed the case of the "Ctrl-" prefixes. files: Doc/faq/extending.rst | 2 +- Doc/faq/windows.rst | 12 ++++++------ Doc/library/asyncio-eventloop.rst | 2 +- Doc/library/asyncio-protocol.rst | 2 +- Doc/library/asyncio-stream.rst | 2 +- Doc/library/idle.rst | 4 ++-- Doc/library/multiprocessing.rst | 2 +- Doc/library/pdb.rst | 4 ++-- Doc/library/signal.rst | 4 ++-- Doc/library/tkinter.ttk.rst | 6 +++--- Doc/library/unittest.rst | 4 ++-- Doc/tutorial/appendix.rst | 2 +- Doc/tutorial/interpreter.rst | 2 +- Doc/using/cmdline.rst | 2 +- Doc/whatsnew/2.0.rst | 4 ++-- Doc/whatsnew/2.5.rst | 2 +- Doc/whatsnew/2.7.rst | 2 +- Lib/test/test_os.py | 4 ++-- Lib/unittest/main.py | 2 +- 19 files changed, 32 insertions(+), 32 deletions(-) diff --git a/Doc/faq/extending.rst b/Doc/faq/extending.rst --- a/Doc/faq/extending.rst +++ b/Doc/faq/extending.rst @@ -348,7 +348,7 @@ { line = readline (prompt); - if (NULL == line) /* CTRL-D pressed */ + if (NULL == line) /* Ctrl-D pressed */ { done = 1; } diff --git a/Doc/faq/windows.rst b/Doc/faq/windows.rst --- a/Doc/faq/windows.rst +++ b/Doc/faq/windows.rst @@ -81,14 +81,14 @@ 'HelloHelloHello' Many people use the interactive mode as a convenient yet highly programmable -calculator. When you want to end your interactive Python session, hold the Ctrl -key down while you enter a Z, then hit the "Enter" key to get back to your +calculator. When you want to end your interactive Python session, hold the :kbd:`Ctrl` +key down while you enter a :kbd:`Z`, then hit the ":kbd:`Enter`" key to get back to your Windows command prompt. You may also find that you have a Start-menu entry such as :menuselection:`Start --> Programs --> Python 3.3 --> Python (command line)` that results in you seeing the ``>>>`` prompt in a new window. If so, the window will disappear -after you enter the Ctrl-Z character; Windows is running a single "python" +after you enter the :kbd:`Ctrl-Z` character; Windows is running a single "python" command in the window, and closes it when you terminate the interpreter. If the ``python`` command, instead of displaying the interpreter prompt ``>>>``, @@ -131,8 +131,8 @@ c:\Python33\python -starts up the interpreter as above (and don't forget you'll need a "CTRL-Z" and -an "Enter" to get out of it). Once you have verified the directory, you can +starts up the interpreter as above (and don't forget you'll need a ":kbd:`Ctrl-Z`" and +an ":kbd:`Enter`" to get out of it). Once you have verified the directory, you can add it to the system path to make it easier to start Python by just running the ``python`` command. This is currently an option in the installer as of CPython 3.3. @@ -327,7 +327,7 @@ return (0 != kernel32.TerminateProcess(handle, 0)) In 2.7 and 3.2, :func:`os.kill` is implemented similar to the above function, -with the additional feature of being able to send CTRL+C and CTRL+BREAK +with the additional feature of being able to send :kbd:`Ctrl+C` and :kbd:`Ctrl+Break` to console subprocesses which are designed to handle those signals. See :func:`os.kill` for further details. diff --git a/Doc/library/asyncio-eventloop.rst b/Doc/library/asyncio-eventloop.rst --- a/Doc/library/asyncio-eventloop.rst +++ b/Doc/library/asyncio-eventloop.rst @@ -853,7 +853,7 @@ loop.add_signal_handler(getattr(signal, signame), functools.partial(ask_exit, signame)) - print("Event loop running forever, press CTRL+c to interrupt.") + print("Event loop running forever, press Ctrl+C to interrupt.") print("pid %s: send SIGINT or SIGTERM to exit." % os.getpid()) try: loop.run_forever() diff --git a/Doc/library/asyncio-protocol.rst b/Doc/library/asyncio-protocol.rst --- a/Doc/library/asyncio-protocol.rst +++ b/Doc/library/asyncio-protocol.rst @@ -539,7 +539,7 @@ coro = loop.create_server(EchoServerClientProtocol, '127.0.0.1', 8888) server = loop.run_until_complete(coro) - # Serve requests until CTRL+c is pressed + # Serve requests until Ctrl+C is pressed print('Serving on {}'.format(server.sockets[0].getsockname())) try: loop.run_forever() diff --git a/Doc/library/asyncio-stream.rst b/Doc/library/asyncio-stream.rst --- a/Doc/library/asyncio-stream.rst +++ b/Doc/library/asyncio-stream.rst @@ -312,7 +312,7 @@ coro = asyncio.start_server(handle_echo, '127.0.0.1', 8888, loop=loop) server = loop.run_until_complete(coro) - # Serve requests until CTRL+c is pressed + # Serve requests until Ctrl+C is pressed print('Serving on {}'.format(server.sockets[0].getsockname())) try: loop.run_forever() diff --git a/Doc/library/idle.rst b/Doc/library/idle.rst --- a/Doc/library/idle.rst +++ b/Doc/library/idle.rst @@ -330,8 +330,8 @@ Editing and navigation ---------------------- -In this section, 'C' refers to the Control key on Windows and Unix and -the Command key on Mac OSX. +In this section, 'C' refers to the :kbd:`Control` key on Windows and Unix and +the :kbd:`Command` key on Mac OSX. * :kbd:`Backspace` deletes to the left; :kbd:`Del` deletes to the right diff --git a/Doc/library/multiprocessing.rst b/Doc/library/multiprocessing.rst --- a/Doc/library/multiprocessing.rst +++ b/Doc/library/multiprocessing.rst @@ -1174,7 +1174,7 @@ .. note:: - If the SIGINT signal generated by Ctrl-C arrives while the main thread is + If the SIGINT signal generated by :kbd:`Ctrl-C` arrives while the main thread is blocked by a call to :meth:`BoundedSemaphore.acquire`, :meth:`Lock.acquire`, :meth:`RLock.acquire`, :meth:`Semaphore.acquire`, :meth:`Condition.acquire` or :meth:`Condition.wait` then the call will be immediately interrupted and diff --git a/Doc/library/pdb.rst b/Doc/library/pdb.rst --- a/Doc/library/pdb.rst +++ b/Doc/library/pdb.rst @@ -156,8 +156,8 @@ that matches one of these patterns. [1]_ By default, Pdb sets a handler for the SIGINT signal (which is sent when the - user presses Ctrl-C on the console) when you give a ``continue`` command. - This allows you to break into the debugger again by pressing Ctrl-C. If you + user presses :kbd:`Ctrl-C` on the console) when you give a ``continue`` command. + This allows you to break into the debugger again by pressing :kbd:`Ctrl-C`. If you want Pdb not to touch the SIGINT handler, set *nosigint* tot true. Example call to enable tracing with *skip*:: diff --git a/Doc/library/signal.rst b/Doc/library/signal.rst --- a/Doc/library/signal.rst +++ b/Doc/library/signal.rst @@ -95,7 +95,7 @@ .. data:: CTRL_C_EVENT - The signal corresponding to the CTRL+C keystroke event. This signal can + The signal corresponding to the :kbd:`Ctrl+C` keystroke event. This signal can only be used with :func:`os.kill`. Availability: Windows. @@ -105,7 +105,7 @@ .. data:: CTRL_BREAK_EVENT - The signal corresponding to the CTRL+BREAK keystroke event. This signal can + The signal corresponding to the :kbd:`Ctrl+Break` keystroke event. This signal can only be used with :func:`os.kill`. Availability: Windows. diff --git a/Doc/library/tkinter.ttk.rst b/Doc/library/tkinter.ttk.rst --- a/Doc/library/tkinter.ttk.rst +++ b/Doc/library/tkinter.ttk.rst @@ -554,9 +554,9 @@ This will extend the bindings for the toplevel window containing the notebook as follows: - * Control-Tab: selects the tab following the currently selected one. - * Shift-Control-Tab: selects the tab preceding the currently selected one. - * Alt-K: where K is the mnemonic (underlined) character of any tab, will + * :kbd:`Control-Tab`: selects the tab following the currently selected one. + * :kbd:`Shift-Control-Tab`: selects the tab preceding the currently selected one. + * :kbd:`Alt-K`: where *K* is the mnemonic (underlined) character of any tab, will select that tab. Multiple notebooks in a single toplevel may be enabled for traversal, diff --git a/Doc/library/unittest.rst b/Doc/library/unittest.rst --- a/Doc/library/unittest.rst +++ b/Doc/library/unittest.rst @@ -204,8 +204,8 @@ .. cmdoption:: -c, --catch - Control-C during the test run waits for the current test to end and then - reports all the results so far. A second control-C raises the normal + :kbd:`Control-C` during the test run waits for the current test to end and then + reports all the results so far. A second :kbd:`Control-C` raises the normal :exc:`KeyboardInterrupt` exception. See `Signal Handling`_ for the functions that provide this functionality. diff --git a/Doc/tutorial/appendix.rst b/Doc/tutorial/appendix.rst --- a/Doc/tutorial/appendix.rst +++ b/Doc/tutorial/appendix.rst @@ -25,7 +25,7 @@ standard error stream; normal output from executed commands is written to standard output. -Typing the interrupt character (usually Control-C or DEL) to the primary or +Typing the interrupt character (usually :kbd:`Control-C` or :kbd:`Delete`) to the primary or secondary prompt cancels the input and returns to the primary prompt. [#]_ Typing an interrupt while a command is executing raises the :exc:`KeyboardInterrupt` exception, which may be handled by a :keyword:`try` diff --git a/Doc/tutorial/interpreter.rst b/Doc/tutorial/interpreter.rst --- a/Doc/tutorial/interpreter.rst +++ b/Doc/tutorial/interpreter.rst @@ -38,7 +38,7 @@ The interpreter's line-editing features include interactive editing, history substitution and code completion on systems that support readline. Perhaps the quickest check to see whether command line editing is supported is typing -Control-P to the first Python prompt you get. If it beeps, you have command +:kbd:`Control-P` to the first Python prompt you get. If it beeps, you have command line editing; see Appendix :ref:`tut-interacting` for an introduction to the keys. If nothing appears to happen, or if ``^P`` is echoed, command line editing isn't available; you'll only be able to use backspace to remove diff --git a/Doc/using/cmdline.rst b/Doc/using/cmdline.rst --- a/Doc/using/cmdline.rst +++ b/Doc/using/cmdline.rst @@ -41,7 +41,7 @@ * When called with standard input connected to a tty device, it prompts for commands and executes them until an EOF (an end-of-file character, you can - produce that with *Ctrl-D* on UNIX or *Ctrl-Z, Enter* on Windows) is read. + produce that with :kbd:`Ctrl-D` on UNIX or :kbd:`Ctrl-Z, Enter` on Windows) is read. * When called with a file name argument or with a file as standard input, it reads and executes a script from that file. * When called with a directory name argument, it reads and executes an diff --git a/Doc/whatsnew/2.0.rst b/Doc/whatsnew/2.0.rst --- a/Doc/whatsnew/2.0.rst +++ b/Doc/whatsnew/2.0.rst @@ -1174,8 +1174,8 @@ * In the editor window, there is now a line/column bar at the bottom. -* Three new keystroke commands: Check module (Alt-F5), Import module (F5) and - Run script (Ctrl-F5). +* Three new keystroke commands: Check module (:kbd:`Alt-F5`), Import module (:kbd:`F5`) and + Run script (:kbd:`Ctrl-F5`). .. ====================================================================== diff --git a/Doc/whatsnew/2.5.rst b/Doc/whatsnew/2.5.rst --- a/Doc/whatsnew/2.5.rst +++ b/Doc/whatsnew/2.5.rst @@ -827,7 +827,7 @@ This rearrangement was done because people often want to catch all exceptions that indicate program errors. :exc:`KeyboardInterrupt` and :exc:`SystemExit` aren't errors, though, and usually represent an explicit action such as the user -hitting Control-C or code calling :func:`sys.exit`. A bare ``except:`` will +hitting :kbd:`Control-C` or code calling :func:`sys.exit`. A bare ``except:`` will catch all exceptions, so you commonly need to list :exc:`KeyboardInterrupt` and :exc:`SystemExit` in order to re-raise them. The usual pattern is:: diff --git a/Doc/whatsnew/2.7.rst b/Doc/whatsnew/2.7.rst --- a/Doc/whatsnew/2.7.rst +++ b/Doc/whatsnew/2.7.rst @@ -2320,7 +2320,7 @@ * The :func:`os.kill` function now works on Windows. The signal value can be the constants :const:`CTRL_C_EVENT`, :const:`CTRL_BREAK_EVENT`, or any integer. The first two constants - will send Control-C and Control-Break keystroke events to + will send :kbd:`Control-C` and :kbd:`Control-Break` keystroke events to subprocesses; any other value will use the :c:func:`TerminateProcess` API. (Contributed by Miki Tebeka; :issue:`1220212`.) diff --git a/Lib/test/test_os.py b/Lib/test/test_os.py --- a/Lib/test/test_os.py +++ b/Lib/test/test_os.py @@ -1657,7 +1657,7 @@ os.kill(proc.pid, signal.SIGINT) self.fail("subprocess did not stop on {}".format(name)) - @unittest.skip("subprocesses aren't inheriting CTRL+C property") + @unittest.skip("subprocesses aren't inheriting Ctrl+C property") def test_CTRL_C_EVENT(self): from ctypes import wintypes import ctypes @@ -1670,7 +1670,7 @@ SetConsoleCtrlHandler.restype = wintypes.BOOL # Calling this with NULL and FALSE causes the calling process to - # handle CTRL+C, rather than ignore it. This property is inherited + # handle Ctrl+C, rather than ignore it. This property is inherited # by subprocesses. SetConsoleCtrlHandler(NULL, 0) diff --git a/Lib/unittest/main.py b/Lib/unittest/main.py --- a/Lib/unittest/main.py +++ b/Lib/unittest/main.py @@ -168,7 +168,7 @@ if self.catchbreak is None: parser.add_argument('-c', '--catch', dest='catchbreak', action='store_true', - help='Catch ctrl-C and display results so far') + help='Catch Ctrl-C and display results so far') self.catchbreak = False if self.buffer is None: parser.add_argument('-b', '--buffer', dest='buffer', -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 12 17:00:39 2015 From: python-checkins at python.org (raymond.hettinger) Date: Sat, 12 Sep 2015 15:00:39 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_In-line_the_append_operati?= =?utf-8?q?ons_inside_deque=5Finplace=5Frepeat=28=29=2E?= Message-ID: <20150912150038.15706.90493@psf.io> https://hg.python.org/cpython/rev/cb96ffe6ff10 changeset: 97943:cb96ffe6ff10 parent: 97941:b8f3a01937be user: Raymond Hettinger date: Sat Sep 12 11:00:20 2015 -0400 summary: In-line the append operations inside deque_inplace_repeat(). files: Modules/_collectionsmodule.c | 22 ++++++++++++++++++---- 1 files changed, 18 insertions(+), 4 deletions(-) diff --git a/Modules/_collectionsmodule.c b/Modules/_collectionsmodule.c --- a/Modules/_collectionsmodule.c +++ b/Modules/_collectionsmodule.c @@ -567,12 +567,26 @@ if (n > MAX_DEQUE_LEN) return PyErr_NoMemory(); + deque->state++; for (i = 0 ; i < n-1 ; i++) { - rv = deque_append(deque, item); - if (rv == NULL) - return NULL; - Py_DECREF(rv); + if (deque->rightindex == BLOCKLEN - 1) { + block *b = newblock(Py_SIZE(deque) + i); + if (b == NULL) { + Py_SIZE(deque) += i; + return NULL; + } + b->leftlink = deque->rightblock; + CHECK_END(deque->rightblock->rightlink); + deque->rightblock->rightlink = b; + deque->rightblock = b; + MARK_END(b->rightlink); + deque->rightindex = -1; + } + deque->rightindex++; + Py_INCREF(item); + deque->rightblock->data[deque->rightindex] = item; } + Py_SIZE(deque) += i; Py_INCREF(deque); return (PyObject *)deque; } -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 12 17:59:45 2015 From: python-checkins at python.org (eric.smith) Date: Sat, 12 Sep 2015 15:59:45 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_Removed_being_able_to_combine?= =?utf-8?b?ICdmJyB3aXRoICd1Jy4=?= Message-ID: <20150912155945.12001.44027@psf.io> https://hg.python.org/peps/rev/29f40cf868a8 changeset: 6050:29f40cf868a8 user: Eric V. Smith date: Sat Sep 12 11:59:53 2015 -0400 summary: Removed being able to combine 'f' with 'u'. files: pep-0498.txt | 11 +++++++++-- 1 files changed, 9 insertions(+), 2 deletions(-) diff --git a/pep-0498.txt b/pep-0498.txt --- a/pep-0498.txt +++ b/pep-0498.txt @@ -173,8 +173,7 @@ letter 'f' or 'F'. Everywhere this PEP uses 'f', 'F' may also be used. 'f' may be combined with 'r', in either order, to produce raw f-string literals. 'f' may not be combined with 'b': this PEP does not -propose to add binary f-strings. 'f' may also be combined with 'u', in -either order, although adding 'u' has no effect. +propose to add binary f-strings. 'f' may not be combined with 'u'. When tokenizing source files, f-strings use the same rules as normal strings, raw strings, binary strings, and triple quoted strings. That @@ -661,6 +660,14 @@ >>> f'{(lambda x: x*2)(3)}' '6' +Can't combine with 'u' +-------------------------- + +The 'u' prefix was added to Python 3.3 in PEP 414 as a means to ease +source compatibility with Python 2.7. Because Python 2.7 will never +support f-strings, there is nothing to be gained by being able to +combine the 'f' prefix with 'u'. + Examples from Python's source code ================================== -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Sat Sep 12 18:42:17 2015 From: python-checkins at python.org (kristjan.jonsson) Date: Sat, 12 Sep 2015 16:42:17 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2325021=3A_Merge_3=2E5_to_default?= Message-ID: <20150912164217.11993.47218@psf.io> https://hg.python.org/cpython/rev/92e7ac79c681 changeset: 97947:92e7ac79c681 parent: 97941:b8f3a01937be parent: 97946:ca628ccec846 user: Kristj?n Valur J?nsson date: Sat Sep 12 16:36:15 2015 +0000 summary: Issue #25021: Merge 3.5 to default files: Lib/test/test_itertools.py | 10 ++++++++++ Modules/itertoolsmodule.c | 12 ++++++++++-- 2 files changed, 20 insertions(+), 2 deletions(-) diff --git a/Lib/test/test_itertools.py b/Lib/test/test_itertools.py --- a/Lib/test/test_itertools.py +++ b/Lib/test/test_itertools.py @@ -1035,6 +1035,16 @@ for proto in range(pickle.HIGHEST_PROTOCOL + 1): self.pickletest(proto, product(*args)) + def test_product_issue_25021(self): + # test that indices are properly clamped to the length of the tuples + p = product((1, 2),(3,)) + p.__setstate__((0, 0x1000)) # will access tuple element 1 if not clamped + self.assertEqual(next(p), (2, 3)) + # test that empty tuple in the list will result in an immediate StopIteration + p = product((1, 2), (), (3,)) + p.__setstate__((0, 0, 0x1000)) # will access tuple element 1 if not clamped + self.assertRaises(StopIteration, next, p) + def test_repeat(self): self.assertEqual(list(repeat(object='a', times=3)), ['a', 'a', 'a']) self.assertEqual(lzip(range(3),repeat('a')), diff --git a/Modules/itertoolsmodule.c b/Modules/itertoolsmodule.c --- a/Modules/itertoolsmodule.c +++ b/Modules/itertoolsmodule.c @@ -2256,13 +2256,21 @@ { PyObject* indexObject = PyTuple_GET_ITEM(state, i); Py_ssize_t index = PyLong_AsSsize_t(indexObject); + PyObject* pool; + Py_ssize_t poolsize; if (index < 0 && PyErr_Occurred()) return NULL; /* not an integer */ + pool = PyTuple_GET_ITEM(lz->pools, i); + poolsize = PyTuple_GET_SIZE(pool); + if (poolsize == 0) { + lz->stopped = 1; + Py_RETURN_NONE; + } /* clamp the index */ if (index < 0) index = 0; - else if (index > n-1) - index = n-1; + else if (index > poolsize-1) + index = poolsize-1; lz->indices[i] = index; } -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 12 18:42:17 2015 From: python-checkins at python.org (kristjan.jonsson) Date: Sat, 12 Sep 2015 16:42:17 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Issue_=2325021=3A_Merge_3=2E4_to_3=2E5?= Message-ID: <20150912164217.15720.67442@psf.io> https://hg.python.org/cpython/rev/ca628ccec846 changeset: 97946:ca628ccec846 branch: 3.5 parent: 97940:cdd1679e666d parent: 97945:4f85b6228697 user: Kristj?n Valur J?nsson date: Sat Sep 12 16:34:33 2015 +0000 summary: Issue #25021: Merge 3.4 to 3.5 files: Lib/test/test_itertools.py | 10 ++++++++++ Modules/itertoolsmodule.c | 12 ++++++++++-- 2 files changed, 20 insertions(+), 2 deletions(-) diff --git a/Lib/test/test_itertools.py b/Lib/test/test_itertools.py --- a/Lib/test/test_itertools.py +++ b/Lib/test/test_itertools.py @@ -985,6 +985,16 @@ for proto in range(pickle.HIGHEST_PROTOCOL + 1): self.pickletest(proto, product(*args)) + def test_product_issue_25021(self): + # test that indices are properly clamped to the length of the tuples + p = product((1, 2),(3,)) + p.__setstate__((0, 0x1000)) # will access tuple element 1 if not clamped + self.assertEqual(next(p), (2, 3)) + # test that empty tuple in the list will result in an immediate StopIteration + p = product((1, 2), (), (3,)) + p.__setstate__((0, 0, 0x1000)) # will access tuple element 1 if not clamped + self.assertRaises(StopIteration, next, p) + def test_repeat(self): self.assertEqual(list(repeat(object='a', times=3)), ['a', 'a', 'a']) self.assertEqual(lzip(range(3),repeat('a')), diff --git a/Modules/itertoolsmodule.c b/Modules/itertoolsmodule.c --- a/Modules/itertoolsmodule.c +++ b/Modules/itertoolsmodule.c @@ -2236,13 +2236,21 @@ { PyObject* indexObject = PyTuple_GET_ITEM(state, i); Py_ssize_t index = PyLong_AsSsize_t(indexObject); + PyObject* pool; + Py_ssize_t poolsize; if (index < 0 && PyErr_Occurred()) return NULL; /* not an integer */ + pool = PyTuple_GET_ITEM(lz->pools, i); + poolsize = PyTuple_GET_SIZE(pool); + if (poolsize == 0) { + lz->stopped = 1; + Py_RETURN_NONE; + } /* clamp the index */ if (index < 0) index = 0; - else if (index > n-1) - index = n-1; + else if (index > poolsize-1) + index = poolsize-1; lz->indices[i] = index; } -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 12 18:42:17 2015 From: python-checkins at python.org (kristjan.jonsson) Date: Sat, 12 Sep 2015 16:42:17 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy4zIC0+IDMuNCk6?= =?utf-8?q?_Issue_=2325021=3A_Merge_from_3=2E3_to_3=2E4?= Message-ID: <20150912164217.11260.12527@psf.io> https://hg.python.org/cpython/rev/4f85b6228697 changeset: 97945:4f85b6228697 branch: 3.4 parent: 97939:452805163bd5 parent: 97944:8cc052c28910 user: Kristj?n Valur J?nsson date: Sat Sep 12 15:30:23 2015 +0000 summary: Issue #25021: Merge from 3.3 to 3.4 files: Lib/test/test_itertools.py | 10 ++++++++++ Modules/itertoolsmodule.c | 12 ++++++++++-- 2 files changed, 20 insertions(+), 2 deletions(-) diff --git a/Lib/test/test_itertools.py b/Lib/test/test_itertools.py --- a/Lib/test/test_itertools.py +++ b/Lib/test/test_itertools.py @@ -985,6 +985,16 @@ for proto in range(pickle.HIGHEST_PROTOCOL + 1): self.pickletest(proto, product(*args)) + def test_product_issue_25021(self): + # test that indices are properly clamped to the length of the tuples + p = product((1, 2),(3,)) + p.__setstate__((0, 0x1000)) # will access tuple element 1 if not clamped + self.assertEqual(next(p), (2, 3)) + # test that empty tuple in the list will result in an immediate StopIteration + p = product((1, 2), (), (3,)) + p.__setstate__((0, 0, 0x1000)) # will access tuple element 1 if not clamped + self.assertRaises(StopIteration, next, p) + def test_repeat(self): self.assertEqual(list(repeat(object='a', times=3)), ['a', 'a', 'a']) self.assertEqual(lzip(range(3),repeat('a')), diff --git a/Modules/itertoolsmodule.c b/Modules/itertoolsmodule.c --- a/Modules/itertoolsmodule.c +++ b/Modules/itertoolsmodule.c @@ -2236,13 +2236,21 @@ { PyObject* indexObject = PyTuple_GET_ITEM(state, i); Py_ssize_t index = PyLong_AsSsize_t(indexObject); + PyObject* pool; + Py_ssize_t poolsize; if (index < 0 && PyErr_Occurred()) return NULL; /* not an integer */ + pool = PyTuple_GET_ITEM(lz->pools, i); + poolsize = PyTuple_GET_SIZE(pool); + if (poolsize == 0) { + lz->stopped = 1; + Py_RETURN_NONE; + } /* clamp the index */ if (index < 0) index = 0; - else if (index > n-1) - index = n-1; + else if (index > poolsize-1) + index = poolsize-1; lz->indices[i] = index; } -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 12 18:42:17 2015 From: python-checkins at python.org (kristjan.jonsson) Date: Sat, 12 Sep 2015 16:42:17 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy4zKTogSXNzdWUgIzI1MDIx?= =?utf-8?q?=3A_Correctly_make_sure_that_product=2E=5F=5Fsetstate=5F=5F_doe?= =?utf-8?q?s_not_access?= Message-ID: <20150912164217.68879.64040@psf.io> https://hg.python.org/cpython/rev/8cc052c28910 changeset: 97944:8cc052c28910 branch: 3.3 parent: 96811:37fed8b02f00 user: Kristj?n Valur J?nsson date: Sat Sep 12 15:20:54 2015 +0000 summary: Issue #25021: Correctly make sure that product.__setstate__ does not access invalid memory. files: Lib/test/test_itertools.py | 10 ++++++++++ Modules/itertoolsmodule.c | 12 ++++++++++-- 2 files changed, 20 insertions(+), 2 deletions(-) diff --git a/Lib/test/test_itertools.py b/Lib/test/test_itertools.py --- a/Lib/test/test_itertools.py +++ b/Lib/test/test_itertools.py @@ -959,6 +959,16 @@ self.assertEqual(list(copy.deepcopy(product(*args))), result) self.pickletest(product(*args)) + def test_product_issue_25021(self): + # test that indices are properly clamped to the length of the tuples + p = product((1, 2),(3,)) + p.__setstate__((0, 0x1000)) # will access tuple element 1 if not clamped + self.assertEqual(next(p), (2, 3)) + # test that empty tuple in the list will result in an immediate StopIteration + p = product((1, 2), (), (3,)) + p.__setstate__((0, 0, 0x1000)) # will access tuple element 1 if not clamped + self.assertRaises(StopIteration, next, p) + def test_repeat(self): self.assertEqual(list(repeat(object='a', times=3)), ['a', 'a', 'a']) self.assertEqual(lzip(range(3),repeat('a')), diff --git a/Modules/itertoolsmodule.c b/Modules/itertoolsmodule.c --- a/Modules/itertoolsmodule.c +++ b/Modules/itertoolsmodule.c @@ -2205,13 +2205,21 @@ { PyObject* indexObject = PyTuple_GET_ITEM(state, i); Py_ssize_t index = PyLong_AsSsize_t(indexObject); + PyObject* pool; + Py_ssize_t poolsize; if (index < 0 && PyErr_Occurred()) return NULL; /* not an integer */ + pool = PyTuple_GET_ITEM(lz->pools, i); + poolsize = PyTuple_GET_SIZE(pool); + if (poolsize == 0) { + lz->stopped = 1; + Py_RETURN_NONE; + } /* clamp the index */ if (index < 0) index = 0; - else if (index > n-1) - index = n-1; + else if (index > poolsize-1) + index = poolsize-1; lz->indices[i] = index; } -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 12 18:42:18 2015 From: python-checkins at python.org (kristjan.jonsson) Date: Sat, 12 Sep 2015 16:42:18 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_default_-=3E_default?= =?utf-8?q?=29=3A_Merge?= Message-ID: <20150912164217.14865.5740@psf.io> https://hg.python.org/cpython/rev/062536a59240 changeset: 97948:062536a59240 parent: 97943:cb96ffe6ff10 parent: 97947:92e7ac79c681 user: Kristj?n Valur J?nsson date: Sat Sep 12 16:41:22 2015 +0000 summary: Merge files: Lib/test/test_itertools.py | 10 ++++++++++ Modules/itertoolsmodule.c | 12 ++++++++++-- 2 files changed, 20 insertions(+), 2 deletions(-) diff --git a/Lib/test/test_itertools.py b/Lib/test/test_itertools.py --- a/Lib/test/test_itertools.py +++ b/Lib/test/test_itertools.py @@ -1035,6 +1035,16 @@ for proto in range(pickle.HIGHEST_PROTOCOL + 1): self.pickletest(proto, product(*args)) + def test_product_issue_25021(self): + # test that indices are properly clamped to the length of the tuples + p = product((1, 2),(3,)) + p.__setstate__((0, 0x1000)) # will access tuple element 1 if not clamped + self.assertEqual(next(p), (2, 3)) + # test that empty tuple in the list will result in an immediate StopIteration + p = product((1, 2), (), (3,)) + p.__setstate__((0, 0, 0x1000)) # will access tuple element 1 if not clamped + self.assertRaises(StopIteration, next, p) + def test_repeat(self): self.assertEqual(list(repeat(object='a', times=3)), ['a', 'a', 'a']) self.assertEqual(lzip(range(3),repeat('a')), diff --git a/Modules/itertoolsmodule.c b/Modules/itertoolsmodule.c --- a/Modules/itertoolsmodule.c +++ b/Modules/itertoolsmodule.c @@ -2256,13 +2256,21 @@ { PyObject* indexObject = PyTuple_GET_ITEM(state, i); Py_ssize_t index = PyLong_AsSsize_t(indexObject); + PyObject* pool; + Py_ssize_t poolsize; if (index < 0 && PyErr_Occurred()) return NULL; /* not an integer */ + pool = PyTuple_GET_ITEM(lz->pools, i); + poolsize = PyTuple_GET_SIZE(pool); + if (poolsize == 0) { + lz->stopped = 1; + Py_RETURN_NONE; + } /* clamp the index */ if (index < 0) index = 0; - else if (index > n-1) - index = n-1; + else if (index > poolsize-1) + index = poolsize-1; lz->indices[i] = index; } -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 12 21:52:08 2015 From: python-checkins at python.org (yury.selivanov) Date: Sat, 12 Sep 2015 19:52:08 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogd2hhdHNuZXcvMy41?= =?utf-8?q?=3A_Edits?= Message-ID: <20150912195208.11250.79707@psf.io> https://hg.python.org/cpython/rev/5c055d8f4328 changeset: 97949:5c055d8f4328 branch: 3.5 parent: 97946:ca628ccec846 user: Yury Selivanov date: Sat Sep 12 15:52:04 2015 -0400 summary: whatsnew/3.5: Edits Patch by Elvis Pranskevichus files: Doc/whatsnew/3.5.rst | 139 ++++++++++++++++++++++++++++-- 1 files changed, 129 insertions(+), 10 deletions(-) diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -114,7 +114,7 @@ * Builtin and extension modules are now initialized in a multi-phase process, which is similar to how Python modules are loaded. (:pep:`489`). -Significantly Improved Library Modules: +Significantly improved library modules: * :class:`collections.OrderedDict` is now implemented in C, which makes it 4 to 100 times faster. (Contributed by Eric Snow in :issue:`16991`.) @@ -432,7 +432,7 @@ :pep:`475` adds support for automatic retry of system calls failing with :py:data:`~errno.EINTR`: this means that user code doesn't have to deal with -EINTR or :exc:`InterruptedError` manually, and should make it more robust +``EINTR`` or :exc:`InterruptedError` manually, and should make it more robust against asynchronous signal reception. Examples of functions which are now retried when interrupted by a signal @@ -656,6 +656,13 @@ protocol. (Contributed by Berker Peksag in :issue:`20289`.) +csv +--- + +:meth:`Writer.writerow ` now supports arbitrary iterables, +not just sequences. (Contributed by Serhiy Storchaka in :issue:`23171`.) + + cmath ----- @@ -709,10 +716,15 @@ collections.abc --------------- +The :meth:`Sequence.index ` method now +accepts *start* and *stop* arguments to match the corresponding methods +of :class:`tuple`, :class:`list`, etc. +(Contributed by Devin Jeanpierre in :issue:`23086`.) + A new :class:`~collections.abc.Generator` abstract base class. (Contributed by Stefan Behnel in :issue:`24018`.) -New :class:`~collections.abc.Coroutine`, +New :class:`~collections.abc.Awaitable` :class:`~collections.abc.Coroutine`, :class:`~collections.abc.AsyncIterator`, and :class:`~collections.abc.AsyncIterable` abstract base classes. (Contributed by Yury Selivanov in :issue:`24184`.) @@ -726,6 +738,9 @@ The :func:`~compileall.compile_dir` function has a corresponding ``workers`` parameter. (Contributed by Claudiu Popa in :issue:`16104`.) +Another new option, ``-r``, allows to control the maximum recursion +level for subdirectories. (Contributed by Claudiu Popa in :issue:`19628`.) + The ``-q`` command line option can now be specified more than once, in which case all output, including errors, will be suppressed. The corresponding ``quiet`` parameter in :func:`~compileall.compile_dir`, @@ -742,6 +757,19 @@ :meth:`~concurrent.futures.ProcessPoolExecutor` is used. (Contributed by Dan O'Reilly in :issue:`11271`.) +A number of workers in :class:`~concurrent.futures.ThreadPoolExecutor` is +optional now. The default value equals to 5 times the number of CPUs. +(Contributed by Claudiu Popa in :issue:`21527`.) + + +configparser +------------ + +Config parsers can be customized by providing a dictionary of converters in the +constructor. All converters defined in config parser (either by subclassing or +by providing in a constructor) will be available on all section proxies. +(Contributed by ?ukasz Langa in :issue:`18159`.) + contextlib ---------- @@ -761,6 +789,13 @@ manual screen resize. (Contributed by Arnon Yaari in :issue:`4254`.) +dbm +--- + +:func:`dumb.open ` always creates a new database when the flag +has the value ``'n'``. (Contributed by Claudiu Popa in :issue:`18039`.) + + difflib ------- @@ -818,6 +853,10 @@ ``SMTPUTF8`` extension. (Contributed by R. David Murray in :issue:`24211`.) +:class:`email.mime.text.MIMEText` constructor now accepts a +:class:`~email.charset.Charset` instance. +(Contributed by Claude Paroz and Berker Peksag in :issue:`16324`.) + enum ---- @@ -860,12 +899,20 @@ (Contributed by Serhiy Storchaka in :issue:`13968`.) +gzip +---- + +The *mode* argument of :class:`~gzip.GzipFile` constructor now +accepts ``"x"`` to request exclusive creation. +(Contributed by Tim Heaney in :issue:`19222`.) + + heapq ----- Element comparison in :func:`~heapq.merge` can now be customized by -passing a :term:`key function` in a new optional ``key`` keyword argument. -A new optional ``reverse`` keyword argument can be used to reverse element +passing a :term:`key function` in a new optional *key* keyword argument. +A new optional *reverse* keyword argument can be used to reverse element comparison. (Contributed by Raymond Hettinger in :issue:`13742`.) @@ -877,6 +924,18 @@ (Contributed by Demian Brecht in :issue:`21793`.) +http.client +----------- + +:meth:`HTTPConnection.getresponse() ` +now raises a :exc:`~http.client.RemoteDisconnected` exception when a +remote server connection is closed unexpectedly. Additionally, if a +:exc:`ConnectionError` (of which ``RemoteDisconnected`` +is a subclass) is raised, the client socket is now closed automatically, +and will reconnect on the next request. +(Contributed by Martin Panter in :issue:`3566`.) + + idlelib and IDLE ---------------- @@ -1006,7 +1065,18 @@ in :issue:`21650`.) JSON decoder now raises :exc:`json.JSONDecodeError` instead of -:exc:`ValueError`. (Contributed by Serhiy Storchaka in :issue:`19361`.) +:exc:`ValueError` to provide better context information about the error. +(Contributed by Serhiy Storchaka in :issue:`19361`.) + + +linecache +--------- + +A new :func:`~linecache.lazycache` function can be used to capture information +about a non-file based module to permit getting its lines later via +:func:`~linecache.getline`. This avoids doing I/O until a line is actually +needed, without having to carry the module globals around indefinitely. +(Contributed by Robert Collins in :issue:`17911`.) locale @@ -1060,6 +1130,14 @@ Storchaka in :issue:`22486`.) +multiprocessing +--------------- + +:func:`~multiprocessing.synchronized` objects now support the +:term:`context manager` protocol. (Contributed by Charles-Fran?ois Natali in +:issue:`21565`.) + + operator -------- @@ -1155,6 +1233,10 @@ re -- +References and conditional references to groups with fixed length are now +allowed in lookbehind assertions. +(Contributed by Serhiy Storchaka in :issue:`9179`.) + The number of capturing groups in regular expression is no longer limited by 100. (Contributed by Serhiy Storchaka in :issue:`22437`.) @@ -1400,8 +1482,9 @@ hook that will be called whenever a :term:`coroutine object ` is created by an :keyword:`async def` function. A corresponding :func:`~sys.get_coroutine_wrapper` can be used to obtain a currently set -wrapper. Both functions are provisional, and are intended for debugging -purposes only. (Contributed by Yury Selivanov in :issue:`24017`.) +wrapper. Both functions are :term:`provisional `, +and are intended for debugging purposes only. (Contributed by Yury Selivanov +in :issue:`24017`.) A new :func:`~sys.is_finalizing` function can be used to check if the Python interpreter is :term:`shutting down `. @@ -1430,6 +1513,11 @@ they will be owned by the named user and group in the tarfile. (Contributed by Michael Vogt and Eric Smith in :issue:`23193`.) +The :meth:`TarFile.list ` now accepts an optional +*members* keyword argument that can be set to a subset of the list returned +by :meth:`TarFile.getmembers `. +(Contributed by Serhiy Storchaka in :issue:`21549`.) + threading --------- @@ -1535,6 +1623,18 @@ unittest -------- +The :meth:`TestLoader.loadTestsFromModule ` +method now accepts a keyword-only argument *pattern* which is passed to +``load_tests`` as the third argument. Found packages are now checked for +``load_tests`` regardless of whether their path matches *pattern*, because it +is impossible for a package name to match the default pattern. +(Contributed by Robert Collins and Barry A. Warsaw in :issue:`16662`.) + +Unittest discovery errors now are exposed in +:data:`TestLoader.errors ` attribute of the +:class:`~unittest.TestLoader` instance. +(Contributed by Robert Collins in :issue:`19746`.) + A new command line option ``--locals`` to show local variables in tracebacks. (Contributed by Robert Collins in :issue:`22936`.) @@ -1558,6 +1658,10 @@ (Contributed by Johannes Baiter in :issue:`20968`, and H?kan L?vdahl in :issue:`23581` and :issue:`23568`.) +It is no longer necessary to explicitly pass ``create=True`` to the +:func:`~unittest.mock.patch` function when patching builtin names. +(Contributed by Kushal Das in :issue:`17660`.) + wsgiref ------- @@ -1586,6 +1690,9 @@ :class:`xmlreader.InputSource ` object. (Contributed by Serhiy Storchaka in :issue:`2175`.) +:func:`~xml.sax.parseString` now accepts a :class:`str` instance. +(Contributed by Serhiy Storchaka in :issue:`10590`.) + zipfile ------- @@ -1808,6 +1915,7 @@ Passing a format string as keyword argument *format_string* to the :meth:`~string.Formatter.format` method of the :class:`string.Formatter` class has been deprecated. +(Contributed by Serhiy Storchaka in :issue:`23671`.) The :func:`platform.dist` and :func:`platform.linux_distribution` functions are now deprecated and will be removed in Python 3.7. Linux distributions use @@ -1833,6 +1941,16 @@ Use of ``re.LOCALE`` flag with str patterns or ``re.ASCII`` is now deprecated. (Contributed by Serhiy Storchaka in :issue:`22407`.) +Use of unrecognized special sequences consisting of ``'\'`` and an ASCII letter +in regular expression patterns and replacement patterns now raises a +deprecation warning and will be forbidden in Python 3.6. +(Contributed by Serhiy Storchaka in :issue:`23622`.) + +The undocumented and unofficial *use_load_tests* default argument of the +:meth:`unittest.TestLoader.loadTestsFromModule` method now is +deprecated and ignored. +(Contributed by Robert Collins and Barry A. Warsaw in :issue:`16662`.) + Removed ======= @@ -1853,8 +1971,8 @@ * The concept of ``.pyo`` files has been removed. -* The JoinableQueue class in the provisional asyncio module was deprecated - in 3.4.4 and is now removed. +* The JoinableQueue class in the provisional :mod:`asyncio` module was + deprecated in 3.4.4 and is now removed. (Contributed by A. Jesse Jiryu Davis in :issue:`23464`.) @@ -1925,6 +2043,7 @@ an empty string. For compatibility use patterns that never match an empty string (e.g. ``"x+"`` instead of ``"x*"``). Patterns that could only match an empty string (such as ``"\b"``) now raise an error. + (Contributed by Serhiy Storchaka in :issue:`22818`.) * The :class:`~http.cookies.Morsel` dict-like interface has been made self consistent: morsel comparison now takes the :attr:`~http.cookies.Morsel.key` -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 12 21:52:23 2015 From: python-checkins at python.org (yury.selivanov) Date: Sat, 12 Sep 2015 19:52:23 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogTWVyZ2UgMy41?= Message-ID: <20150912195223.14883.5544@psf.io> https://hg.python.org/cpython/rev/7e3392eb34aa changeset: 97950:7e3392eb34aa parent: 97948:062536a59240 parent: 97949:5c055d8f4328 user: Yury Selivanov date: Sat Sep 12 15:52:20 2015 -0400 summary: Merge 3.5 files: Doc/whatsnew/3.5.rst | 139 ++++++++++++++++++++++++++++-- 1 files changed, 129 insertions(+), 10 deletions(-) diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -114,7 +114,7 @@ * Builtin and extension modules are now initialized in a multi-phase process, which is similar to how Python modules are loaded. (:pep:`489`). -Significantly Improved Library Modules: +Significantly improved library modules: * :class:`collections.OrderedDict` is now implemented in C, which makes it 4 to 100 times faster. (Contributed by Eric Snow in :issue:`16991`.) @@ -432,7 +432,7 @@ :pep:`475` adds support for automatic retry of system calls failing with :py:data:`~errno.EINTR`: this means that user code doesn't have to deal with -EINTR or :exc:`InterruptedError` manually, and should make it more robust +``EINTR`` or :exc:`InterruptedError` manually, and should make it more robust against asynchronous signal reception. Examples of functions which are now retried when interrupted by a signal @@ -656,6 +656,13 @@ protocol. (Contributed by Berker Peksag in :issue:`20289`.) +csv +--- + +:meth:`Writer.writerow ` now supports arbitrary iterables, +not just sequences. (Contributed by Serhiy Storchaka in :issue:`23171`.) + + cmath ----- @@ -709,10 +716,15 @@ collections.abc --------------- +The :meth:`Sequence.index ` method now +accepts *start* and *stop* arguments to match the corresponding methods +of :class:`tuple`, :class:`list`, etc. +(Contributed by Devin Jeanpierre in :issue:`23086`.) + A new :class:`~collections.abc.Generator` abstract base class. (Contributed by Stefan Behnel in :issue:`24018`.) -New :class:`~collections.abc.Coroutine`, +New :class:`~collections.abc.Awaitable` :class:`~collections.abc.Coroutine`, :class:`~collections.abc.AsyncIterator`, and :class:`~collections.abc.AsyncIterable` abstract base classes. (Contributed by Yury Selivanov in :issue:`24184`.) @@ -726,6 +738,9 @@ The :func:`~compileall.compile_dir` function has a corresponding ``workers`` parameter. (Contributed by Claudiu Popa in :issue:`16104`.) +Another new option, ``-r``, allows to control the maximum recursion +level for subdirectories. (Contributed by Claudiu Popa in :issue:`19628`.) + The ``-q`` command line option can now be specified more than once, in which case all output, including errors, will be suppressed. The corresponding ``quiet`` parameter in :func:`~compileall.compile_dir`, @@ -742,6 +757,19 @@ :meth:`~concurrent.futures.ProcessPoolExecutor` is used. (Contributed by Dan O'Reilly in :issue:`11271`.) +A number of workers in :class:`~concurrent.futures.ThreadPoolExecutor` is +optional now. The default value equals to 5 times the number of CPUs. +(Contributed by Claudiu Popa in :issue:`21527`.) + + +configparser +------------ + +Config parsers can be customized by providing a dictionary of converters in the +constructor. All converters defined in config parser (either by subclassing or +by providing in a constructor) will be available on all section proxies. +(Contributed by ?ukasz Langa in :issue:`18159`.) + contextlib ---------- @@ -761,6 +789,13 @@ manual screen resize. (Contributed by Arnon Yaari in :issue:`4254`.) +dbm +--- + +:func:`dumb.open ` always creates a new database when the flag +has the value ``'n'``. (Contributed by Claudiu Popa in :issue:`18039`.) + + difflib ------- @@ -818,6 +853,10 @@ ``SMTPUTF8`` extension. (Contributed by R. David Murray in :issue:`24211`.) +:class:`email.mime.text.MIMEText` constructor now accepts a +:class:`~email.charset.Charset` instance. +(Contributed by Claude Paroz and Berker Peksag in :issue:`16324`.) + enum ---- @@ -860,12 +899,20 @@ (Contributed by Serhiy Storchaka in :issue:`13968`.) +gzip +---- + +The *mode* argument of :class:`~gzip.GzipFile` constructor now +accepts ``"x"`` to request exclusive creation. +(Contributed by Tim Heaney in :issue:`19222`.) + + heapq ----- Element comparison in :func:`~heapq.merge` can now be customized by -passing a :term:`key function` in a new optional ``key`` keyword argument. -A new optional ``reverse`` keyword argument can be used to reverse element +passing a :term:`key function` in a new optional *key* keyword argument. +A new optional *reverse* keyword argument can be used to reverse element comparison. (Contributed by Raymond Hettinger in :issue:`13742`.) @@ -877,6 +924,18 @@ (Contributed by Demian Brecht in :issue:`21793`.) +http.client +----------- + +:meth:`HTTPConnection.getresponse() ` +now raises a :exc:`~http.client.RemoteDisconnected` exception when a +remote server connection is closed unexpectedly. Additionally, if a +:exc:`ConnectionError` (of which ``RemoteDisconnected`` +is a subclass) is raised, the client socket is now closed automatically, +and will reconnect on the next request. +(Contributed by Martin Panter in :issue:`3566`.) + + idlelib and IDLE ---------------- @@ -1006,7 +1065,18 @@ in :issue:`21650`.) JSON decoder now raises :exc:`json.JSONDecodeError` instead of -:exc:`ValueError`. (Contributed by Serhiy Storchaka in :issue:`19361`.) +:exc:`ValueError` to provide better context information about the error. +(Contributed by Serhiy Storchaka in :issue:`19361`.) + + +linecache +--------- + +A new :func:`~linecache.lazycache` function can be used to capture information +about a non-file based module to permit getting its lines later via +:func:`~linecache.getline`. This avoids doing I/O until a line is actually +needed, without having to carry the module globals around indefinitely. +(Contributed by Robert Collins in :issue:`17911`.) locale @@ -1060,6 +1130,14 @@ Storchaka in :issue:`22486`.) +multiprocessing +--------------- + +:func:`~multiprocessing.synchronized` objects now support the +:term:`context manager` protocol. (Contributed by Charles-Fran?ois Natali in +:issue:`21565`.) + + operator -------- @@ -1155,6 +1233,10 @@ re -- +References and conditional references to groups with fixed length are now +allowed in lookbehind assertions. +(Contributed by Serhiy Storchaka in :issue:`9179`.) + The number of capturing groups in regular expression is no longer limited by 100. (Contributed by Serhiy Storchaka in :issue:`22437`.) @@ -1400,8 +1482,9 @@ hook that will be called whenever a :term:`coroutine object ` is created by an :keyword:`async def` function. A corresponding :func:`~sys.get_coroutine_wrapper` can be used to obtain a currently set -wrapper. Both functions are provisional, and are intended for debugging -purposes only. (Contributed by Yury Selivanov in :issue:`24017`.) +wrapper. Both functions are :term:`provisional `, +and are intended for debugging purposes only. (Contributed by Yury Selivanov +in :issue:`24017`.) A new :func:`~sys.is_finalizing` function can be used to check if the Python interpreter is :term:`shutting down `. @@ -1430,6 +1513,11 @@ they will be owned by the named user and group in the tarfile. (Contributed by Michael Vogt and Eric Smith in :issue:`23193`.) +The :meth:`TarFile.list ` now accepts an optional +*members* keyword argument that can be set to a subset of the list returned +by :meth:`TarFile.getmembers `. +(Contributed by Serhiy Storchaka in :issue:`21549`.) + threading --------- @@ -1535,6 +1623,18 @@ unittest -------- +The :meth:`TestLoader.loadTestsFromModule ` +method now accepts a keyword-only argument *pattern* which is passed to +``load_tests`` as the third argument. Found packages are now checked for +``load_tests`` regardless of whether their path matches *pattern*, because it +is impossible for a package name to match the default pattern. +(Contributed by Robert Collins and Barry A. Warsaw in :issue:`16662`.) + +Unittest discovery errors now are exposed in +:data:`TestLoader.errors ` attribute of the +:class:`~unittest.TestLoader` instance. +(Contributed by Robert Collins in :issue:`19746`.) + A new command line option ``--locals`` to show local variables in tracebacks. (Contributed by Robert Collins in :issue:`22936`.) @@ -1558,6 +1658,10 @@ (Contributed by Johannes Baiter in :issue:`20968`, and H?kan L?vdahl in :issue:`23581` and :issue:`23568`.) +It is no longer necessary to explicitly pass ``create=True`` to the +:func:`~unittest.mock.patch` function when patching builtin names. +(Contributed by Kushal Das in :issue:`17660`.) + wsgiref ------- @@ -1586,6 +1690,9 @@ :class:`xmlreader.InputSource ` object. (Contributed by Serhiy Storchaka in :issue:`2175`.) +:func:`~xml.sax.parseString` now accepts a :class:`str` instance. +(Contributed by Serhiy Storchaka in :issue:`10590`.) + zipfile ------- @@ -1808,6 +1915,7 @@ Passing a format string as keyword argument *format_string* to the :meth:`~string.Formatter.format` method of the :class:`string.Formatter` class has been deprecated. +(Contributed by Serhiy Storchaka in :issue:`23671`.) The :func:`platform.dist` and :func:`platform.linux_distribution` functions are now deprecated and will be removed in Python 3.7. Linux distributions use @@ -1833,6 +1941,16 @@ Use of ``re.LOCALE`` flag with str patterns or ``re.ASCII`` is now deprecated. (Contributed by Serhiy Storchaka in :issue:`22407`.) +Use of unrecognized special sequences consisting of ``'\'`` and an ASCII letter +in regular expression patterns and replacement patterns now raises a +deprecation warning and will be forbidden in Python 3.6. +(Contributed by Serhiy Storchaka in :issue:`23622`.) + +The undocumented and unofficial *use_load_tests* default argument of the +:meth:`unittest.TestLoader.loadTestsFromModule` method now is +deprecated and ignored. +(Contributed by Robert Collins and Barry A. Warsaw in :issue:`16662`.) + Removed ======= @@ -1853,8 +1971,8 @@ * The concept of ``.pyo`` files has been removed. -* The JoinableQueue class in the provisional asyncio module was deprecated - in 3.4.4 and is now removed. +* The JoinableQueue class in the provisional :mod:`asyncio` module was + deprecated in 3.4.4 and is now removed. (Contributed by A. Jesse Jiryu Davis in :issue:`23464`.) @@ -1925,6 +2043,7 @@ an empty string. For compatibility use patterns that never match an empty string (e.g. ``"x+"`` instead of ``"x*"``). Patterns that could only match an empty string (such as ``"\b"``) now raise an error. + (Contributed by Serhiy Storchaka in :issue:`22818`.) * The :class:`~http.cookies.Morsel` dict-like interface has been made self consistent: morsel comparison now takes the :attr:`~http.cookies.Morsel.key` -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 12 23:51:20 2015 From: python-checkins at python.org (yury.selivanov) Date: Sat, 12 Sep 2015 21:51:20 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogd2hhdHNuZXcvMy41?= =?utf-8?q?_More_edits?= Message-ID: <20150912215120.17985.19131@psf.io> https://hg.python.org/cpython/rev/10a9d4acd9cb changeset: 97951:10a9d4acd9cb branch: 3.5 parent: 97949:5c055d8f4328 user: Yury Selivanov date: Sat Sep 12 17:50:58 2015 -0400 summary: whatsnew/3.5 More edits Patch by Elvis Praskevichus. (+ issue #25070) files: Doc/whatsnew/3.5.rst | 228 +++++++++++++++++++++++------- 1 files changed, 174 insertions(+), 54 deletions(-) diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -73,11 +73,14 @@ * :pep:`465`, a new matrix multiplication operator: ``a @ b``. * :pep:`448`, additional unpacking generalizations. + New library modules: +* :mod:`typing`: :ref:`Type Hints ` (:pep:`484`). * :mod:`zipapp`: :ref:`Improving Python ZIP Application Support ` (:pep:`441`). + New built-in features: * ``bytes % args``, ``bytearray % args``: :pep:`461` - Adding ``%`` formatting @@ -100,6 +103,7 @@ * New :exc:`StopAsyncIteration` exception. (Contributed by Yury Selivanov in :issue:`24017`. See also :pep:`492`.) + CPython implementation improvements: * When the ``LC_TYPE`` locale is the POSIX locale (``C`` locale), @@ -114,32 +118,32 @@ * Builtin and extension modules are now initialized in a multi-phase process, which is similar to how Python modules are loaded. (:pep:`489`). -Significantly improved library modules: - -* :class:`collections.OrderedDict` is now implemented in C, which makes it - 4 to 100 times faster. (Contributed by Eric Snow in :issue:`16991`.) - -* You may now pass bytes to the :mod:`tempfile` module's APIs and it will - return the temporary pathname as :class:`bytes` instead of :class:`str`. - It also accepts a value of ``None`` on parameters where only str was - accepted in the past to do the right thing based on the types of the - other inputs. Two functions, :func:`gettempdirb` and - :func:`gettempprefixb`, have been added to go along with this. - This behavior matches that of the :mod:`os` APIs. - (Contributed by Gregory P. Smith in :issue:`24230`.) - -* :mod:`ssl` module gained support for Memory BIO, which decouples SSL - protocol handling from network IO. (Contributed by Geert Jansen in - :issue:`21965`.) - -* :mod:`traceback` has new lightweight and convenient to work with - classes :class:`~traceback.TracebackException`, - :class:`~traceback.StackSummary`, and :class:`~traceback.FrameSummary`. - (Contributed by Robert Collins in :issue:`17911`.) - -* Most of :func:`functools.lru_cache` machinery is now implemented in C. - (Contributed by Matt Joiner, Alexey Kachayev, and Serhiy Storchaka - in :issue:`14373`.) + +Significant improvements in standard library: + +* :class:`collections.OrderedDict` is now + :ref:`implemented in C `, which makes it + 4 to 100 times faster. + +* :mod:`ssl` module gained + :ref:`support for Memory BIO `, which decouples SSL + protocol handling from network IO. + +* :mod:`traceback` module has been significantly + :ref:`enhanced ` for improved + performance and developer convenience. + +* The new :func:`os.scandir` function provides a + :ref:`better and significantly faster way ` + of directory traversal. + +* :func:`functools.lru_cache` has been largely + :ref:`reimplemented in C `, yielding much better + performance. + +* The new :func:`subprocess.run` function provides a + :ref:`streamlined way to run subprocesses `. + Security improvements: @@ -152,6 +156,7 @@ against potential injection attacks. (Contributed by Antoine Pitrou in :issue:`22796`.) + Windows improvements: * A new installer for Windows has replaced the old MSI. @@ -160,6 +165,7 @@ * Windows builds now use Microsoft Visual C++ 14.0, and extension modules should use the same. + Please read on for a comprehensive list of user-facing changes, including many other smaller improvements, CPython optimizations, deprecations, and potential porting issues. @@ -238,7 +244,7 @@ finally: loop.close() -will print:: +will output:: coro 2: waiting for lock coro 2: holding the lock @@ -352,22 +358,29 @@ PEP 461 - % formatting support for bytes and bytearray ------------------------------------------------------ -PEP 461 adds % formatting to :class:`bytes` and :class:`bytearray`, aiding in -handling data that is a mixture of binary and ASCII compatible text. This -feature also eases porting such code from Python 2. +PEP 461 adds support for ``%`` +:ref:`interpolation operator ` to :class:`bytes` +and :class:`bytearray`. + +While interpolation is usually thought of as a string operation, there are +cases where interpolation on ``bytes`` or ``bytearrays`` make sense, and the +work needed to make up for this missing functionality detracts from the +overall readability of the code. This issue is particularly important when +dealing with wire format protocols, which are often a mixture of binary +and ASCII compatible text. Examples:: - >>> b'Hello %s!' % b'World' + >>> b'Hello %b!' % b'World' b'Hello World!' >>> b'x=%i y=%f' % (1, 2.5) b'x=1 y=2.500000' -Unicode is not allowed for ``%s``, but it is accepted by ``%a`` (equivalent of +Unicode is not allowed for ``%b``, but it is accepted by ``%a`` (equivalent of ``repr(obj).encode('ascii', 'backslashreplace')``):: - >>> b'Hello %s!' % 'World' + >>> b'Hello %b!' % 'World' Traceback (most recent call last): File "", line 1, in TypeError: %b requires bytes, or an object that implements __bytes__, not 'str' @@ -375,6 +388,9 @@ >>> b'price: %a' % '10?' b"price: '10\\u20ac'" +Note that ``%s`` and ``%r`` conversion types, although supported, should +only be used in codebases that need compatibility with Python 2. + .. seealso:: :pep:`461` -- Adding % formatting to bytes and bytearray @@ -387,9 +403,17 @@ PEP 484 - Type Hints -------------------- -This PEP introduces a provisional module to provide these standard -definitions and tools, along with some conventions for situations -where annotations are not available. +Function annotation syntax has been a Python feature since version 3.0 +(:pep:`3107`), however the semantics of annotations has been left undefined. + +Experience has shown that the majority of function annotation +uses were to provide type hints to function parameters and return values. It +became evident that it would be beneficial for Python users, if the +standard library included the base definitions and tools for type annotations. + +:pep:`484` introduces a :term:`provisional module ` to +provide these standard definitions and tools, along with some conventions +for situations where annotations are not available. For example, here is a simple function whose argument and return type are declared in the annotations:: @@ -397,9 +421,14 @@ def greeting(name: str) -> str: return 'Hello ' + name +While these annotations are available at runtime through the usual +:attr:`__annotations__` attribute, *no automatic type checking happens at +runtime*. Instead, it is assumed that a separate off-line type checker will +be used for on-demand source code analysis. + The type system supports unions, generic types, and a special type -named ``Any`` which is consistent with (i.e. assignable to and from) all -types. +named :class:`~typing.Any` which is consistent with (i.e. assignable to +and from) all types. .. seealso:: @@ -407,6 +436,8 @@ * :pep:`484` -- Type Hints PEP written by Guido van Rossum, Jukka Lehtosalo, and ?ukasz Langa; implemented by Guido van Rossum. + * :pep:`483` -- The Theory of Type Hints + PEP written by Guido van Rossum .. _whatsnew-pep-471: @@ -416,8 +447,14 @@ :pep:`471` adds a new directory iteration function, :func:`os.scandir`, to the standard library. Additionally, :func:`os.walk` is now -implemented using :func:`os.scandir`, which speeds it up by 3-5 times -on POSIX systems and by 7-20 times on Windows systems. +implemented using ``os.scandir()``, which makes it 3 to 5 times faster +on POSIX systems and 7 to 20 times faster on Windows systems. This is +largely achieved by greatly reducing the number of calls to :func:`os.stat` +required to walk a directory tree. + +Additionally, ``os.scandir()`` returns an iterator, as opposed to returning +a list of file names, which improves memory efficiency when iterating +over very large directories. .. seealso:: @@ -430,14 +467,39 @@ PEP 475: Retry system calls failing with EINTR ---------------------------------------------- -:pep:`475` adds support for automatic retry of system calls failing with -:py:data:`~errno.EINTR`: this means that user code doesn't have to deal with -``EINTR`` or :exc:`InterruptedError` manually, and should make it more robust -against asynchronous signal reception. - -Examples of functions which are now retried when interrupted by a signal -instead of raising :exc:`InterruptedError` if the Python signal handler does -not raise an exception: +A :py:data:`~errno.EINTR` error code is returned whenever a system call, that +is waiting for I/O, is interrupted by a signal. Previously, Python would +raise :exc:`InterruptedError` in such case. This meant that, when writing a +Python application, the developer had two choices: + +#. Ignore the ``InterruptedError``. +#. Handle the ``InterruptedError`` and attempt to restart the interrupted + system call at every call site. + +The first option makes an application fail intermittently. +The second option adds a large amount of boilerplate that makes the +code nearly unreadable. Compare:: + + print("Hello World") + +and:: + + while True: + try: + print("Hello World") + break + except InterruptedError: + continue + +:pep:`475` implements automatic retry of system calls on +``EINTR``. This removes the burden of dealing with ``EINTR`` +or :exc:`InterruptedError` in user code in most situations and makes +Python programs, including the standard library, more robust. Note that +the system call is only retried if the signal handler does not raise an +exception. + +Below is a list of functions which are now retried when interrupted +by a signal: * :func:`open`, :func:`os.open`, :func:`io.open`; @@ -476,7 +538,7 @@ :pep:`475` -- Retry system calls failing with EINTR PEP and implementation written by Charles-Fran?ois Natali and - Victor Stinner, with the help of Antoine Pitrou (the french connection). + Victor Stinner, with the help of Antoine Pitrou (the French connection). .. _whatsnew-pep-479: @@ -484,15 +546,27 @@ PEP 479: Change StopIteration handling inside generators -------------------------------------------------------- -:pep:`479` changes the behavior of generators: when a :exc:`StopIteration` +The interaction of generators and :exc:`StopIteration` in Python 3.4 and +earlier was somewhat surprising, and could conceal obscure bugs. Previously, +``StopIteration`` raised accidentally inside a generator function was +interpreted as the end of the iteration by the loop construct driving the +generator. + +:pep:`479` changes the behavior of generators: when a ``StopIteration`` exception is raised inside a generator, it is replaced with a -:exc:`RuntimeError`. To enable the feature a ``__future__`` import should -be used:: +:exc:`RuntimeError` before it exits the generator frame. The main goal of +this change is to ease debugging in the situation where an unguarded +:func:`next` call raises ``StopIteration`` and causes the iteration controlled +by the generator to terminate silently. This is particularly pernicious in +combination with the ``yield from`` construct. + +This is a backwards incompatible change, so to enable the new behavior, +a :term:`__future__` import is necessary:: from __future__ import generator_stop Without a ``__future__`` import, a :exc:`PendingDeprecationWarning` will be -raised. +raised whenever a ``StopIteration`` exception is raised inside a generator. .. seealso:: @@ -641,6 +715,18 @@ Steven Bethard, paul j3 and Daniel Eriksson in :issue:`14910`.) +asyncio +------- + +Since :mod:`asyncio` module is :term:`provisional `, +all changes introduced in Python 3.5 have also been backported to Python 3.4.x. + +Notable changes in :mod:`asyncio` module since Python 3.4.0: + +* The proactor event loop now supports SSL. + (Contributed by Antoine Pitrou and Victor Stinner in :issue:`22560`.) + + bz2 --- @@ -681,6 +767,8 @@ collections ----------- +.. _whatsnew-ordereddict: + The :class:`~collections.OrderedDict` class is now implemented in C, which makes it 4 to 100 times faster. (Contributed by Eric Snow in :issue:`16991`.) @@ -886,6 +974,8 @@ functools --------- +.. _whatsnew-lrucache: + Most of :func:`~functools.lru_cache` machinery is now implemented in C, making it significantly faster. (Contributed by Matt Joiner, Alexey Kachayev, and Serhiy Storchaka in :issue:`14373`.) @@ -1355,6 +1445,8 @@ ssl --- +.. _whatsnew-sslmemorybio: + Memory BIO Support ~~~~~~~~~~~~~~~~~~ @@ -1463,6 +1555,8 @@ Jessica McKellar, and Serhiy Storchaka in :issue:`13583`.) +.. _whatsnew-subprocess: + subprocess ---------- @@ -1482,7 +1576,7 @@ hook that will be called whenever a :term:`coroutine object ` is created by an :keyword:`async def` function. A corresponding :func:`~sys.get_coroutine_wrapper` can be used to obtain a currently set -wrapper. Both functions are :term:`provisional `, +wrapper. Both functions are :term:`provisional `, and are intended for debugging purposes only. (Contributed by Yury Selivanov in :issue:`24017`.) @@ -1556,6 +1650,8 @@ (Contributed by Zachary Ware in :issue:`20035`.) +.. _whatsnew-traceback: + traceback --------- @@ -1883,6 +1979,16 @@ become proper keywords in Python 3.7. +Deprecated Python Behavior +-------------------------- + +Raising :exc:`StopIteration` inside a generator will now generate a silent +:exc:`PendingDeprecationWarning`, which will become a non-silent deprecation +warning in Python 3.6 and will trigger a :exc:`RuntimeError` in Python 3.7. +See :ref:`PEP 479: Change StopIteration handling inside generators ` +for details. + + Unsupported Operating Systems ----------------------------- @@ -1982,6 +2088,20 @@ This section lists previously described changes and other bugfixes that may require changes to your code. + +Changes in Python behavior +-------------------------- + +* Due to an oversight, earlier Python versions erroneously accepted the + following syntax:: + + f(1 for x in [1], *args) + f(1 for x in [1], **kwargs) + + Python 3.5 now correctly raises a :exc:`SyntaxError`, as generator + expressions must be put in parentheses if not a sole argument to a function. + + Changes in the Python API ------------------------- -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 12 23:51:20 2015 From: python-checkins at python.org (yury.selivanov) Date: Sat, 12 Sep 2015 21:51:20 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogTWVyZ2UgMy41?= Message-ID: <20150912215120.101480.42560@psf.io> https://hg.python.org/cpython/rev/d9bf044898be changeset: 97952:d9bf044898be parent: 97950:7e3392eb34aa parent: 97951:10a9d4acd9cb user: Yury Selivanov date: Sat Sep 12 17:51:16 2015 -0400 summary: Merge 3.5 files: Doc/whatsnew/3.5.rst | 228 +++++++++++++++++++++++------- 1 files changed, 174 insertions(+), 54 deletions(-) diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -73,11 +73,14 @@ * :pep:`465`, a new matrix multiplication operator: ``a @ b``. * :pep:`448`, additional unpacking generalizations. + New library modules: +* :mod:`typing`: :ref:`Type Hints ` (:pep:`484`). * :mod:`zipapp`: :ref:`Improving Python ZIP Application Support ` (:pep:`441`). + New built-in features: * ``bytes % args``, ``bytearray % args``: :pep:`461` - Adding ``%`` formatting @@ -100,6 +103,7 @@ * New :exc:`StopAsyncIteration` exception. (Contributed by Yury Selivanov in :issue:`24017`. See also :pep:`492`.) + CPython implementation improvements: * When the ``LC_TYPE`` locale is the POSIX locale (``C`` locale), @@ -114,32 +118,32 @@ * Builtin and extension modules are now initialized in a multi-phase process, which is similar to how Python modules are loaded. (:pep:`489`). -Significantly improved library modules: - -* :class:`collections.OrderedDict` is now implemented in C, which makes it - 4 to 100 times faster. (Contributed by Eric Snow in :issue:`16991`.) - -* You may now pass bytes to the :mod:`tempfile` module's APIs and it will - return the temporary pathname as :class:`bytes` instead of :class:`str`. - It also accepts a value of ``None`` on parameters where only str was - accepted in the past to do the right thing based on the types of the - other inputs. Two functions, :func:`gettempdirb` and - :func:`gettempprefixb`, have been added to go along with this. - This behavior matches that of the :mod:`os` APIs. - (Contributed by Gregory P. Smith in :issue:`24230`.) - -* :mod:`ssl` module gained support for Memory BIO, which decouples SSL - protocol handling from network IO. (Contributed by Geert Jansen in - :issue:`21965`.) - -* :mod:`traceback` has new lightweight and convenient to work with - classes :class:`~traceback.TracebackException`, - :class:`~traceback.StackSummary`, and :class:`~traceback.FrameSummary`. - (Contributed by Robert Collins in :issue:`17911`.) - -* Most of :func:`functools.lru_cache` machinery is now implemented in C. - (Contributed by Matt Joiner, Alexey Kachayev, and Serhiy Storchaka - in :issue:`14373`.) + +Significant improvements in standard library: + +* :class:`collections.OrderedDict` is now + :ref:`implemented in C `, which makes it + 4 to 100 times faster. + +* :mod:`ssl` module gained + :ref:`support for Memory BIO `, which decouples SSL + protocol handling from network IO. + +* :mod:`traceback` module has been significantly + :ref:`enhanced ` for improved + performance and developer convenience. + +* The new :func:`os.scandir` function provides a + :ref:`better and significantly faster way ` + of directory traversal. + +* :func:`functools.lru_cache` has been largely + :ref:`reimplemented in C `, yielding much better + performance. + +* The new :func:`subprocess.run` function provides a + :ref:`streamlined way to run subprocesses `. + Security improvements: @@ -152,6 +156,7 @@ against potential injection attacks. (Contributed by Antoine Pitrou in :issue:`22796`.) + Windows improvements: * A new installer for Windows has replaced the old MSI. @@ -160,6 +165,7 @@ * Windows builds now use Microsoft Visual C++ 14.0, and extension modules should use the same. + Please read on for a comprehensive list of user-facing changes, including many other smaller improvements, CPython optimizations, deprecations, and potential porting issues. @@ -238,7 +244,7 @@ finally: loop.close() -will print:: +will output:: coro 2: waiting for lock coro 2: holding the lock @@ -352,22 +358,29 @@ PEP 461 - % formatting support for bytes and bytearray ------------------------------------------------------ -PEP 461 adds % formatting to :class:`bytes` and :class:`bytearray`, aiding in -handling data that is a mixture of binary and ASCII compatible text. This -feature also eases porting such code from Python 2. +PEP 461 adds support for ``%`` +:ref:`interpolation operator ` to :class:`bytes` +and :class:`bytearray`. + +While interpolation is usually thought of as a string operation, there are +cases where interpolation on ``bytes`` or ``bytearrays`` make sense, and the +work needed to make up for this missing functionality detracts from the +overall readability of the code. This issue is particularly important when +dealing with wire format protocols, which are often a mixture of binary +and ASCII compatible text. Examples:: - >>> b'Hello %s!' % b'World' + >>> b'Hello %b!' % b'World' b'Hello World!' >>> b'x=%i y=%f' % (1, 2.5) b'x=1 y=2.500000' -Unicode is not allowed for ``%s``, but it is accepted by ``%a`` (equivalent of +Unicode is not allowed for ``%b``, but it is accepted by ``%a`` (equivalent of ``repr(obj).encode('ascii', 'backslashreplace')``):: - >>> b'Hello %s!' % 'World' + >>> b'Hello %b!' % 'World' Traceback (most recent call last): File "", line 1, in TypeError: %b requires bytes, or an object that implements __bytes__, not 'str' @@ -375,6 +388,9 @@ >>> b'price: %a' % '10?' b"price: '10\\u20ac'" +Note that ``%s`` and ``%r`` conversion types, although supported, should +only be used in codebases that need compatibility with Python 2. + .. seealso:: :pep:`461` -- Adding % formatting to bytes and bytearray @@ -387,9 +403,17 @@ PEP 484 - Type Hints -------------------- -This PEP introduces a provisional module to provide these standard -definitions and tools, along with some conventions for situations -where annotations are not available. +Function annotation syntax has been a Python feature since version 3.0 +(:pep:`3107`), however the semantics of annotations has been left undefined. + +Experience has shown that the majority of function annotation +uses were to provide type hints to function parameters and return values. It +became evident that it would be beneficial for Python users, if the +standard library included the base definitions and tools for type annotations. + +:pep:`484` introduces a :term:`provisional module ` to +provide these standard definitions and tools, along with some conventions +for situations where annotations are not available. For example, here is a simple function whose argument and return type are declared in the annotations:: @@ -397,9 +421,14 @@ def greeting(name: str) -> str: return 'Hello ' + name +While these annotations are available at runtime through the usual +:attr:`__annotations__` attribute, *no automatic type checking happens at +runtime*. Instead, it is assumed that a separate off-line type checker will +be used for on-demand source code analysis. + The type system supports unions, generic types, and a special type -named ``Any`` which is consistent with (i.e. assignable to and from) all -types. +named :class:`~typing.Any` which is consistent with (i.e. assignable to +and from) all types. .. seealso:: @@ -407,6 +436,8 @@ * :pep:`484` -- Type Hints PEP written by Guido van Rossum, Jukka Lehtosalo, and ?ukasz Langa; implemented by Guido van Rossum. + * :pep:`483` -- The Theory of Type Hints + PEP written by Guido van Rossum .. _whatsnew-pep-471: @@ -416,8 +447,14 @@ :pep:`471` adds a new directory iteration function, :func:`os.scandir`, to the standard library. Additionally, :func:`os.walk` is now -implemented using :func:`os.scandir`, which speeds it up by 3-5 times -on POSIX systems and by 7-20 times on Windows systems. +implemented using ``os.scandir()``, which makes it 3 to 5 times faster +on POSIX systems and 7 to 20 times faster on Windows systems. This is +largely achieved by greatly reducing the number of calls to :func:`os.stat` +required to walk a directory tree. + +Additionally, ``os.scandir()`` returns an iterator, as opposed to returning +a list of file names, which improves memory efficiency when iterating +over very large directories. .. seealso:: @@ -430,14 +467,39 @@ PEP 475: Retry system calls failing with EINTR ---------------------------------------------- -:pep:`475` adds support for automatic retry of system calls failing with -:py:data:`~errno.EINTR`: this means that user code doesn't have to deal with -``EINTR`` or :exc:`InterruptedError` manually, and should make it more robust -against asynchronous signal reception. - -Examples of functions which are now retried when interrupted by a signal -instead of raising :exc:`InterruptedError` if the Python signal handler does -not raise an exception: +A :py:data:`~errno.EINTR` error code is returned whenever a system call, that +is waiting for I/O, is interrupted by a signal. Previously, Python would +raise :exc:`InterruptedError` in such case. This meant that, when writing a +Python application, the developer had two choices: + +#. Ignore the ``InterruptedError``. +#. Handle the ``InterruptedError`` and attempt to restart the interrupted + system call at every call site. + +The first option makes an application fail intermittently. +The second option adds a large amount of boilerplate that makes the +code nearly unreadable. Compare:: + + print("Hello World") + +and:: + + while True: + try: + print("Hello World") + break + except InterruptedError: + continue + +:pep:`475` implements automatic retry of system calls on +``EINTR``. This removes the burden of dealing with ``EINTR`` +or :exc:`InterruptedError` in user code in most situations and makes +Python programs, including the standard library, more robust. Note that +the system call is only retried if the signal handler does not raise an +exception. + +Below is a list of functions which are now retried when interrupted +by a signal: * :func:`open`, :func:`os.open`, :func:`io.open`; @@ -476,7 +538,7 @@ :pep:`475` -- Retry system calls failing with EINTR PEP and implementation written by Charles-Fran?ois Natali and - Victor Stinner, with the help of Antoine Pitrou (the french connection). + Victor Stinner, with the help of Antoine Pitrou (the French connection). .. _whatsnew-pep-479: @@ -484,15 +546,27 @@ PEP 479: Change StopIteration handling inside generators -------------------------------------------------------- -:pep:`479` changes the behavior of generators: when a :exc:`StopIteration` +The interaction of generators and :exc:`StopIteration` in Python 3.4 and +earlier was somewhat surprising, and could conceal obscure bugs. Previously, +``StopIteration`` raised accidentally inside a generator function was +interpreted as the end of the iteration by the loop construct driving the +generator. + +:pep:`479` changes the behavior of generators: when a ``StopIteration`` exception is raised inside a generator, it is replaced with a -:exc:`RuntimeError`. To enable the feature a ``__future__`` import should -be used:: +:exc:`RuntimeError` before it exits the generator frame. The main goal of +this change is to ease debugging in the situation where an unguarded +:func:`next` call raises ``StopIteration`` and causes the iteration controlled +by the generator to terminate silently. This is particularly pernicious in +combination with the ``yield from`` construct. + +This is a backwards incompatible change, so to enable the new behavior, +a :term:`__future__` import is necessary:: from __future__ import generator_stop Without a ``__future__`` import, a :exc:`PendingDeprecationWarning` will be -raised. +raised whenever a ``StopIteration`` exception is raised inside a generator. .. seealso:: @@ -641,6 +715,18 @@ Steven Bethard, paul j3 and Daniel Eriksson in :issue:`14910`.) +asyncio +------- + +Since :mod:`asyncio` module is :term:`provisional `, +all changes introduced in Python 3.5 have also been backported to Python 3.4.x. + +Notable changes in :mod:`asyncio` module since Python 3.4.0: + +* The proactor event loop now supports SSL. + (Contributed by Antoine Pitrou and Victor Stinner in :issue:`22560`.) + + bz2 --- @@ -681,6 +767,8 @@ collections ----------- +.. _whatsnew-ordereddict: + The :class:`~collections.OrderedDict` class is now implemented in C, which makes it 4 to 100 times faster. (Contributed by Eric Snow in :issue:`16991`.) @@ -886,6 +974,8 @@ functools --------- +.. _whatsnew-lrucache: + Most of :func:`~functools.lru_cache` machinery is now implemented in C, making it significantly faster. (Contributed by Matt Joiner, Alexey Kachayev, and Serhiy Storchaka in :issue:`14373`.) @@ -1355,6 +1445,8 @@ ssl --- +.. _whatsnew-sslmemorybio: + Memory BIO Support ~~~~~~~~~~~~~~~~~~ @@ -1463,6 +1555,8 @@ Jessica McKellar, and Serhiy Storchaka in :issue:`13583`.) +.. _whatsnew-subprocess: + subprocess ---------- @@ -1482,7 +1576,7 @@ hook that will be called whenever a :term:`coroutine object ` is created by an :keyword:`async def` function. A corresponding :func:`~sys.get_coroutine_wrapper` can be used to obtain a currently set -wrapper. Both functions are :term:`provisional `, +wrapper. Both functions are :term:`provisional `, and are intended for debugging purposes only. (Contributed by Yury Selivanov in :issue:`24017`.) @@ -1556,6 +1650,8 @@ (Contributed by Zachary Ware in :issue:`20035`.) +.. _whatsnew-traceback: + traceback --------- @@ -1883,6 +1979,16 @@ become proper keywords in Python 3.7. +Deprecated Python Behavior +-------------------------- + +Raising :exc:`StopIteration` inside a generator will now generate a silent +:exc:`PendingDeprecationWarning`, which will become a non-silent deprecation +warning in Python 3.6 and will trigger a :exc:`RuntimeError` in Python 3.7. +See :ref:`PEP 479: Change StopIteration handling inside generators ` +for details. + + Unsupported Operating Systems ----------------------------- @@ -1982,6 +2088,20 @@ This section lists previously described changes and other bugfixes that may require changes to your code. + +Changes in Python behavior +-------------------------- + +* Due to an oversight, earlier Python versions erroneously accepted the + following syntax:: + + f(1 for x in [1], *args) + f(1 for x in [1], **kwargs) + + Python 3.5 now correctly raises a :exc:`SyntaxError`, as generator + expressions must be put in parentheses if not a sole argument to a function. + + Changes in the Python API ------------------------- -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 12 23:53:46 2015 From: python-checkins at python.org (yury.selivanov) Date: Sat, 12 Sep 2015 21:53:46 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogd2hhdHNuZXcvMy41?= =?utf-8?q?=3A_Update_editor=27s_email_addresses?= Message-ID: <20150912215346.15704.42449@psf.io> https://hg.python.org/cpython/rev/84fb57098a28 changeset: 97953:84fb57098a28 branch: 3.5 parent: 97951:10a9d4acd9cb user: Yury Selivanov date: Sat Sep 12 17:53:33 2015 -0400 summary: whatsnew/3.5: Update editor's email addresses files: Doc/whatsnew/3.5.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -4,7 +4,7 @@ :Release: |release| :Date: |today| -:Editors: Elvis Pranskevichus , Yury Selivanov +:Editors: Elvis Pranskevichus , Yury Selivanov .. Rules for maintenance: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 12 23:53:46 2015 From: python-checkins at python.org (yury.selivanov) Date: Sat, 12 Sep 2015 21:53:46 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogTWVyZ2UgMy41?= Message-ID: <20150912215346.27685.43267@psf.io> https://hg.python.org/cpython/rev/f733d7ec03a9 changeset: 97954:f733d7ec03a9 parent: 97952:d9bf044898be parent: 97953:84fb57098a28 user: Yury Selivanov date: Sat Sep 12 17:53:42 2015 -0400 summary: Merge 3.5 files: Doc/whatsnew/3.5.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -4,7 +4,7 @@ :Release: |release| :Date: |today| -:Editors: Elvis Pranskevichus , Yury Selivanov +:Editors: Elvis Pranskevichus , Yury Selivanov .. Rules for maintenance: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 12 23:55:19 2015 From: python-checkins at python.org (yury.selivanov) Date: Sat, 12 Sep 2015 21:55:19 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_Update_email_address_for_Yury?= =?utf-8?q?_Selivanov?= Message-ID: <20150912215519.101478.72358@psf.io> https://hg.python.org/peps/rev/236f66167b92 changeset: 6051:236f66167b92 user: Yury Selivanov date: Sat Sep 12 17:55:15 2015 -0400 summary: Update email address for Yury Selivanov files: pep-0362.txt | 2 +- pep-0492.txt | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/pep-0362.txt b/pep-0362.txt --- a/pep-0362.txt +++ b/pep-0362.txt @@ -3,7 +3,7 @@ Version: $Revision$ Last-Modified: $Date$ Author: Brett Cannon , Jiwon Seo , - Yury Selivanov , Larry Hastings + Yury Selivanov , Larry Hastings Status: Final Type: Standards Track Content-Type: text/x-rst diff --git a/pep-0492.txt b/pep-0492.txt --- a/pep-0492.txt +++ b/pep-0492.txt @@ -2,7 +2,7 @@ Title: Coroutines with async and await syntax Version: $Revision$ Last-Modified: $Date$ -Author: Yury Selivanov +Author: Yury Selivanov Discussions-To: Status: Final Type: Standards Track -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Sun Sep 13 00:53:38 2015 From: python-checkins at python.org (eric.smith) Date: Sat, 12 Sep 2015 22:53:38 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Fixed_indentation=2E?= Message-ID: <20150912225338.68875.41337@psf.io> https://hg.python.org/cpython/rev/8baa2f2d60eb changeset: 97955:8baa2f2d60eb user: Eric V. Smith date: Sat Sep 12 18:53:36 2015 -0400 summary: Fixed indentation. files: Parser/tokenizer.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Parser/tokenizer.c b/Parser/tokenizer.c --- a/Parser/tokenizer.c +++ b/Parser/tokenizer.c @@ -1741,7 +1741,7 @@ else { end_quote_size = 0; if (c == '\\') - c = tok_nextc(tok); /* skip escaped char */ + c = tok_nextc(tok); /* skip escaped char */ } } -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 13 02:08:10 2015 From: python-checkins at python.org (eric.smith) Date: Sun, 13 Sep 2015 00:08:10 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_Some_grammar_and_typo_fixes?= =?utf-8?q?=2E?= Message-ID: <20150913000809.12021.6010@psf.io> https://hg.python.org/peps/rev/3956c944f394 changeset: 6052:3956c944f394 user: Eric V. Smith date: Sat Sep 12 20:08:18 2015 -0400 summary: Some grammar and typo fixes. files: pep-0498.txt | 19 ++++++++++--------- 1 files changed, 10 insertions(+), 9 deletions(-) diff --git a/pep-0498.txt b/pep-0498.txt --- a/pep-0498.txt +++ b/pep-0498.txt @@ -197,9 +197,9 @@ expressions are evaluated, formatted with the existing __format__ protocol, then the results are concatenated together with the string literals. While scanning the string for expressions, any doubled -braces ``'{{'`` or ``'}}'`` are replaced by the corresponding single -brace. Doubled opening braces do not signify the start of an -expression. +braces ``'{{'`` or ``'}}'`` inside literal portions of an f-string are +replaced by the corresponding single brace. Doubled opening braces do +not signify the start of an expression. Comments, using the ``'#'`` character, are not allowed inside an expression. @@ -220,11 +220,11 @@ So, an f-string looks like:: - f ' { } text ... ' + f ' { } ... ' The resulting expression's ``__format__`` method is called with the -format specifier. The resulting value is used when building the value -of the f-string. +format specifier as an argument. The resulting value is used when +building the value of the f-string. Expressions cannot contain ``':'`` or ``'!'`` outside of strings or parentheses, brackets, or braces. The exception is that the ``'!='`` @@ -432,7 +432,8 @@ --------------------------------------------------------- For ease of readability, leading and trailing whitespace in -expressions is ignored. +expressions is ignored. This is a by-product of enclosing the +expression in parentheses before evaluation. Evaluation order of expressions ------------------------------- @@ -584,8 +585,8 @@ Triple quoted f-strings are allowed. These strings are parsed just as normal triple-quoted strings are. After parsing and decoding, the -normal f-string logic is applied, and ``__format__()`` on each value -is called. +normal f-string logic is applied, and ``__format__()`` is called on +each value. Raw f-strings ------------- -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Sun Sep 13 02:21:54 2015 From: python-checkins at python.org (benjamin.peterson) Date: Sun, 13 Sep 2015 00:21:54 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_fix_name_of_ar?= =?utf-8?q?gument_in_docstring_and_the_docs_=28closes_=2325076=29?= Message-ID: <20150913002154.27685.96104@psf.io> https://hg.python.org/cpython/rev/1208c85af6d5 changeset: 97956:1208c85af6d5 branch: 3.4 parent: 97945:4f85b6228697 user: Benjamin Peterson date: Sat Sep 12 17:20:47 2015 -0700 summary: fix name of argument in docstring and the docs (closes #25076) Patch by TAKASE Arihiro. files: Doc/distutils/apiref.rst | 2 +- Lib/distutils/ccompiler.py | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Doc/distutils/apiref.rst b/Doc/distutils/apiref.rst --- a/Doc/distutils/apiref.rst +++ b/Doc/distutils/apiref.rst @@ -521,7 +521,7 @@ .. method:: CCompiler.library_option(lib) - Return the compiler option to add *dir* to the list of libraries linked into the + Return the compiler option to add *lib* to the list of libraries linked into the shared library or executable. diff --git a/Lib/distutils/ccompiler.py b/Lib/distutils/ccompiler.py --- a/Lib/distutils/ccompiler.py +++ b/Lib/distutils/ccompiler.py @@ -752,7 +752,7 @@ raise NotImplementedError def library_option(self, lib): - """Return the compiler option to add 'dir' to the list of libraries + """Return the compiler option to add 'lib' to the list of libraries linked into the shared library or executable. """ raise NotImplementedError -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 13 02:21:54 2015 From: python-checkins at python.org (benjamin.peterson) Date: Sun, 13 Sep 2015 00:21:54 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_fix_name_of_ar?= =?utf-8?q?gument_in_docstring_and_the_docs_=28closes_=2325076=29?= Message-ID: <20150913002154.12006.17703@psf.io> https://hg.python.org/cpython/rev/63bbe9f80909 changeset: 97957:63bbe9f80909 branch: 2.7 parent: 97942:39e2300f6267 user: Benjamin Peterson date: Sat Sep 12 17:20:47 2015 -0700 summary: fix name of argument in docstring and the docs (closes #25076) Patch by TAKASE Arihiro. files: Doc/distutils/apiref.rst | 2 +- Lib/distutils/ccompiler.py | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Doc/distutils/apiref.rst b/Doc/distutils/apiref.rst --- a/Doc/distutils/apiref.rst +++ b/Doc/distutils/apiref.rst @@ -516,7 +516,7 @@ .. method:: CCompiler.library_option(lib) - Return the compiler option to add *dir* to the list of libraries linked into the + Return the compiler option to add *lib* to the list of libraries linked into the shared library or executable. diff --git a/Lib/distutils/ccompiler.py b/Lib/distutils/ccompiler.py --- a/Lib/distutils/ccompiler.py +++ b/Lib/distutils/ccompiler.py @@ -718,7 +718,7 @@ raise NotImplementedError def library_option(self, lib): - """Return the compiler option to add 'dir' to the list of libraries + """Return the compiler option to add 'lib' to the list of libraries linked into the shared library or executable. """ raise NotImplementedError -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 13 02:21:54 2015 From: python-checkins at python.org (benjamin.peterson) Date: Sun, 13 Sep 2015 00:21:54 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_merge_3=2E4?= Message-ID: <20150913002154.115052.81329@psf.io> https://hg.python.org/cpython/rev/d1748935d4d1 changeset: 97958:d1748935d4d1 branch: 3.5 parent: 97953:84fb57098a28 parent: 97956:1208c85af6d5 user: Benjamin Peterson date: Sat Sep 12 17:21:16 2015 -0700 summary: merge 3.4 files: Doc/distutils/apiref.rst | 2 +- Lib/distutils/ccompiler.py | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Doc/distutils/apiref.rst b/Doc/distutils/apiref.rst --- a/Doc/distutils/apiref.rst +++ b/Doc/distutils/apiref.rst @@ -521,7 +521,7 @@ .. method:: CCompiler.library_option(lib) - Return the compiler option to add *dir* to the list of libraries linked into the + Return the compiler option to add *lib* to the list of libraries linked into the shared library or executable. diff --git a/Lib/distutils/ccompiler.py b/Lib/distutils/ccompiler.py --- a/Lib/distutils/ccompiler.py +++ b/Lib/distutils/ccompiler.py @@ -752,7 +752,7 @@ raise NotImplementedError def library_option(self, lib): - """Return the compiler option to add 'dir' to the list of libraries + """Return the compiler option to add 'lib' to the list of libraries linked into the shared library or executable. """ raise NotImplementedError -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 13 02:21:54 2015 From: python-checkins at python.org (benjamin.peterson) Date: Sun, 13 Sep 2015 00:21:54 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogbWVyZ2UgMy41ICgjMjUwNzYp?= Message-ID: <20150913002154.68867.77882@psf.io> https://hg.python.org/cpython/rev/f4dc1b8bb4b6 changeset: 97959:f4dc1b8bb4b6 parent: 97955:8baa2f2d60eb parent: 97958:d1748935d4d1 user: Benjamin Peterson date: Sat Sep 12 17:21:24 2015 -0700 summary: merge 3.5 (#25076) files: Doc/distutils/apiref.rst | 2 +- Lib/distutils/ccompiler.py | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Doc/distutils/apiref.rst b/Doc/distutils/apiref.rst --- a/Doc/distutils/apiref.rst +++ b/Doc/distutils/apiref.rst @@ -521,7 +521,7 @@ .. method:: CCompiler.library_option(lib) - Return the compiler option to add *dir* to the list of libraries linked into the + Return the compiler option to add *lib* to the list of libraries linked into the shared library or executable. diff --git a/Lib/distutils/ccompiler.py b/Lib/distutils/ccompiler.py --- a/Lib/distutils/ccompiler.py +++ b/Lib/distutils/ccompiler.py @@ -752,7 +752,7 @@ raise NotImplementedError def library_option(self, lib): - """Return the compiler option to add 'dir' to the list of libraries + """Return the compiler option to add 'lib' to the list of libraries linked into the shared library or executable. """ raise NotImplementedError -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 13 05:23:47 2015 From: python-checkins at python.org (alexander.belopolsky) Date: Sun, 13 Sep 2015 03:23:47 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_PEP_495=3A_Restore_the_transi?= =?utf-8?q?tivity_of_=3D=3D_comparisons=2E?= Message-ID: <20150913032347.66868.79081@psf.io> https://hg.python.org/peps/rev/3dc0382326de changeset: 6053:3dc0382326de user: Alexander Belopolsky date: Sat Sep 12 23:23:39 2015 -0400 summary: PEP 495: Restore the transitivity of == comparisons. files: pep-0495.txt | 89 +++++++++++++++++++++++++++++++++------ 1 files changed, 75 insertions(+), 14 deletions(-) diff --git a/pep-0495.txt b/pep-0495.txt --- a/pep-0495.txt +++ b/pep-0495.txt @@ -397,27 +397,86 @@ where ``delta`` is the size of the fold or the gap. -Temporal Arithmetic -=================== +Temporal Arithmetic and Comparison Operators +============================================ -The value of "fold" will be ignored in all operations except those -that involve conversion between timezones. [#]_ As a consequence, +The value of the ``fold`` attribute will be ignored in all operations +with naive datetime instances. As a consequence, naive ``datetime.datetime`` or ``datetime.time`` instances that differ only by the value of ``fold`` will compare as equal. Applications that need to differentiate between such instances should check the value of -``fold`` or convert them to a timezone that does not have ambiguous -times. +``fold`` explicitly or convert those instances to a timezone that does +not have ambiguous times. -The result of addition (subtraction) of a timedelta to (from) a -datetime will always have ``fold`` set to 0 even if the +The value of ``fold`` will also be ignored whenever a timedelta is +added to or subtracted from a datetime instance which may be either +aware or naive. The result of addition (subtraction) of a timedelta +to (from) a datetime will always have ``fold`` set to 0 even if the original datetime instance had ``fold=1``. -.. [#] Computing a difference between two aware datetime instances - with different values of ``tzinfo`` involves an implicit timezone - conversion. In this case, the result may depend on the value of - the ``fold`` attribute in either of the instances, but only if the - instance has ``tzinfo`` that accounts for the value of ``fold`` - in its ``utcoffset()`` method. +No changes are proposed to the way the difference ``t - s`` is +computed for datetime instances ``t`` and ``s``. If both instances +are naive or ``t.tzinfo`` is the same instance as ``s.tzinfo`` +(``t.tzinfo is s.tzinfo`` evaluates to ``True``) then ``t - s`` is a +timedelta ``d`` such that ``s + d == t``. As explained in the +previous paragraph, timedelta addition ignores both ``fold`` and +``tzinfo`` attributes and so does intra-zone or naive datetime +subtraction. + +Naive and intra-zone comparisons will ignore the value of ``fold`` and +return the same results as they do now. + +The inter-zone subtraction will be defined as it is now: ``t - s`` is +computed as ``(t - t.utcoffset()) - (s - +s.utcoffset()).replace(tzinfo=t.tzinfo)``, but the result may now +depend on the values of ``t.fold`` and ``s.fold`` when either +``t.tzinfo`` or ``s.tzinfo`` is post-PEP. [#]_ + +.. [#] Note that the new rules may result in a paradoxical situation + when ``s == t`` but ``s - t != timedelta(0)``. Such paradoxes are + not really new and are inherent in the overloading of the minus + operator as two different intra- and inter-zone operations. For + example, one can easily construct datetime instances ``t`` and ``s`` + with some variable offset ``tzinfo`` and a datetime ``u`` with + ``tzinfo=timezone.utc`` such that ``(t - u) - (s - u) != t - s``. + The explanation for this paradox is that the minuses inside the + parentheses and the two other minuses are really three different + operations: inter-zone datetime subtraction, timedelta subtraction + and intra-zone datetime subtraction which have the mathematical + properties of subtraction separately, but not when combined in a + single expression. + + +Aware datetime Equality Comparison +---------------------------------- + +The aware datetime comparison operators will work the same as they do +now with results indirectly affected by the value of ``fold`` whenever +``utcoffset()`` value of one of the operands depends on it, with one +exception. Whenever one of the operands in inter-zone comparison is +such that its ``utcoffset()`` depends on the value of its ``fold`` +fold attribute, the result is ``False``. [#]_ + +.. [#] This exception is designed to preserve the hash and equivalence + invariants in the face of paradoxes of inter-zone arithmetic. + +Formally, ``t == s`` when ``t.tzinfo is s.tzinfo`` evaluates to +``False`` can be defined as follows. Let ``toutc(t, fold)`` be a +function that takes an aware datetime instance ``t`` and returns a +naive instance representing the same time in UTC assuming a given +value of ``fold``: + +.. code:: + + def toutc(t, fold): + u = t - t.replace(fold=fold).utcoffset() + return u.replace(tzinfo=None) + +Then ``t == s`` is equivalent to + +.. code:: + + toutc(t, fold=0) == toutc(t, fold=1) == toutc(s, fold=0) == toutc(s, fold=1) Backward and Forward Compatibility @@ -725,3 +784,5 @@ employee, taken or made as part of that person's official duties. As a work of the U.S. federal government, the image is in the public domain. + + LocalWords: isdst Py tm -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Sun Sep 13 05:46:52 2015 From: python-checkins at python.org (yury.selivanov) Date: Sun, 13 Sep 2015 03:46:52 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogd2hhdHNuZXcvMy41?= =?utf-8?q?=3A_Add_some_examples?= Message-ID: <20150913034652.101500.95518@psf.io> https://hg.python.org/cpython/rev/c471f57097fb changeset: 97960:c471f57097fb branch: 3.5 parent: 97958:d1748935d4d1 user: Yury Selivanov date: Sat Sep 12 23:46:39 2015 -0400 summary: whatsnew/3.5: Add some examples Patch by Elvis Pranskevichus files: Doc/whatsnew/3.5.rst | 198 +++++++++++++++++++++++++++--- 1 files changed, 177 insertions(+), 21 deletions(-) diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -358,7 +358,7 @@ PEP 461 - % formatting support for bytes and bytearray ------------------------------------------------------ -PEP 461 adds support for ``%`` +:pep:`461` adds support for ``%`` :ref:`interpolation operator ` to :class:`bytes` and :class:`bytearray`. @@ -447,15 +447,24 @@ :pep:`471` adds a new directory iteration function, :func:`os.scandir`, to the standard library. Additionally, :func:`os.walk` is now -implemented using ``os.scandir()``, which makes it 3 to 5 times faster +implemented using ``scandir``, which makes it 3 to 5 times faster on POSIX systems and 7 to 20 times faster on Windows systems. This is largely achieved by greatly reducing the number of calls to :func:`os.stat` required to walk a directory tree. -Additionally, ``os.scandir()`` returns an iterator, as opposed to returning +Additionally, ``scandir`` returns an iterator, as opposed to returning a list of file names, which improves memory efficiency when iterating over very large directories. +The following example shows a simple use of :func:`os.scandir` to display all +the files (excluding directories) in the given *path* that don't start with +``'.'``. The :meth:`entry.is_file ` call will generally +not make an additional system call:: + + for entry in os.scandir(path): + if not entry.name.startswith('.') and entry.is_file(): + print(entry.name) + .. seealso:: :pep:`471` -- os.scandir() function -- a better and faster directory iterator @@ -641,6 +650,27 @@ functions which tell whether two values are approximately equal or "close" to each other. Whether or not two values are considered close is determined according to given absolute and relative tolerances. +Relative tolerance is the maximum allowed difference between ``isclose()`` +arguments, relative to the larger absolute value:: + + >>> import math + >>> a = 5.0 + >>> b = 4.99998 + >>> math.isclose(a, b, rel_tol=1e-5) + True + >>> math.isclose(a, b, rel_tol=1e-6) + False + +It is also possible to compare two values using absolute tolerance, which +must be a non-negative value:: + + >>> import math + >>> a = 5.0 + >>> b = 4.99998 + >>> math.isclose(a, b, abs_tol=0.00003) + True + >>> math.isclose(a, b, abs_tol=0.00001) + False .. seealso:: @@ -678,6 +708,13 @@ New Modules =========== +typing +------ + +The new :mod:`typing` :term:`provisional ` module +provides standard definitions and tools for function type annotations. +See :ref:`Type Hints ` for more information. + .. _whatsnew-zipapp: zipapp @@ -854,8 +891,25 @@ ------------ Config parsers can be customized by providing a dictionary of converters in the -constructor. All converters defined in config parser (either by subclassing or +constructor, or All converters defined in config parser (either by subclassing or by providing in a constructor) will be available on all section proxies. + +Example:: + + >>> import configparser + >>> conv = {} + >>> conv['list'] = lambda v: [e.strip() for e in v.split() if e.strip()] + >>> cfg = configparser.ConfigParser(converters=conv) + >>> cfg.read_string(""" + ... [s] + ... list = a b c d e f g + ... """) + >>> cfg.get('s', 'list') + 'a b c d e f g' + >>> cfg.getlist('s', 'list') + ['a', 'b', 'c', 'd', 'e', 'f', 'g'] + + (Contributed by ?ukasz Langa in :issue:`18159`.) @@ -865,8 +919,17 @@ The new :func:`~contextlib.redirect_stderr` context manager (similar to :func:`~contextlib.redirect_stdout`) makes it easier for utility scripts to handle inflexible APIs that write their output to :data:`sys.stderr` and -don't provide any options to redirect it. (Contributed by Berker Peksag in -:issue:`22389`.) +don't provide any options to redirect it:: + + >>> import contextlib, io, logging + >>> f = io.StringIO() + >>> with contextlib.redirect_stderr(f): + ... logging.warning('warning') + ... + >>> f.getvalue() + 'WARNING:root:warning\n' + +(Contributed by Berker Peksag in :issue:`22389`.) curses @@ -1001,9 +1064,19 @@ ----- Element comparison in :func:`~heapq.merge` can now be customized by -passing a :term:`key function` in a new optional *key* keyword argument. -A new optional *reverse* keyword argument can be used to reverse element -comparison. (Contributed by Raymond Hettinger in :issue:`13742`.) +passing a :term:`key function` in a new optional *key* keyword argument, +and a new optional *reverse* keyword argument can be used to reverse element +comparison:: + + >>> import heapq + >>> a = ['9', '777', '55555'] + >>> b = ['88', '6666'] + >>> list(heapq.merge(a, b, key=len)) + ['9', '88', '777', '6666', '55555'] + >>> list(heapq.merge(reversed(a), reversed(b), key=len, reverse=True)) + ['55555', '6666', '777', '88', '9'] + +(Contributed by Raymond Hettinger in :issue:`13742`.) http @@ -1022,7 +1095,17 @@ remote server connection is closed unexpectedly. Additionally, if a :exc:`ConnectionError` (of which ``RemoteDisconnected`` is a subclass) is raised, the client socket is now closed automatically, -and will reconnect on the next request. +and will reconnect on the next request:: + + import http.client + conn = http.client.HTTPConnection('www.python.org') + for retries in range(3): + try: + conn.request('GET', '/') + resp = conn.getresponse() + except http.client.RemoteDisconnected: + pass + (Contributed by Martin Panter in :issue:`3566`.) @@ -1095,7 +1178,14 @@ A new :meth:`BoundArguments.apply_defaults ` -method provides a way to set default values for missing arguments. +method provides a way to set default values for missing arguments:: + + >>> def foo(a, b='ham', *args): pass + >>> ba = inspect.signature(foo).bind('spam') + >>> ba.apply_defaults() + >>> ba.arguments + OrderedDict([('a', 'spam'), ('b', 'ham'), ('args', ())]) + (Contributed by Yury Selivanov in :issue:`24190`.) A new class method @@ -1137,12 +1227,28 @@ Both :class:`~ipaddress.IPv4Network` and :class:`~ipaddress.IPv6Network` classes now accept an ``(address, netmask)`` tuple argument, so as to easily construct -network objects from existing addresses. (Contributed by Peter Moody -and Antoine Pitrou in :issue:`16531`.) +network objects from existing addresses:: + + >>> import ipaddress + >>> ipaddress.IPv4Network(('127.0.0.0', 8)) + IPv4Network('127.0.0.0/8') + >>> ipaddress.IPv4Network(('127.0.0.0', '255.0.0.0')) + IPv4Network('127.0.0.0/8') + +(Contributed by Peter Moody and Antoine Pitrou in :issue:`16531`.) A new :attr:`~ipaddress.IPv4Network.reverse_pointer>` attribute for :class:`~ipaddress.IPv4Network` and :class:`~ipaddress.IPv6Network` classes -returns the name of the reverse DNS PTR record. +returns the name of the reverse DNS PTR record:: + + >>> import ipaddress + >>> addr = ipaddress.IPv4Address('127.0.0.1') + >>> addr.reverse_pointer + '1.0.0.127.in-addr.arpa' + >>> addr6 = ipaddress.IPv6Address('::1') + >>> addr6.reverse_pointer + '1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.ip6.arpa' + (Contributed by Leon Weber in :issue:`20480`.) @@ -1173,7 +1279,18 @@ ------ A new :func:`~locale.delocalize` function can be used to convert a string into -a normalized number string, taking the ``LC_NUMERIC`` settings into account. +a normalized number string, taking the ``LC_NUMERIC`` settings into account:: + + >>> import locale + >>> locale.setlocale(locale.LC_NUMERIC, 'de_DE.UTF-8') + 'de_DE.UTF-8' + >>> locale.delocalize('1.234,56') + '1234.56' + >>> locale.setlocale(locale.LC_NUMERIC, 'en_US.UTF-8') + 'en_US.UTF-8' + >>> locale.delocalize('1,234.56') + '1234.56' + (Contributed by C?dric Krier in :issue:`13918`.) @@ -1223,9 +1340,9 @@ multiprocessing --------------- -:func:`~multiprocessing.synchronized` objects now support the -:term:`context manager` protocol. (Contributed by Charles-Fran?ois Natali in -:issue:`21565`.) +:func:`sharedctypes.synchronized ` +objects now support the :term:`context manager` protocol. +(Contributed by Charles-Fran?ois Natali in :issue:`21565`.) operator @@ -1271,7 +1388,15 @@ There is a new :func:`os.path.commonpath` function returning the longest common sub-path of each passed pathname. Unlike the :func:`os.path.commonprefix` function, it always returns a valid -path. (Contributed by Rafik Draoui and Serhiy Storchaka in :issue:`10395`.) +path:: + + >>> os.path.commonprefix(['/usr/lib', '/usr/local/lib']) + '/usr/l' + + >>> os.path.commonpath(['/usr/lib', '/usr/local/lib']) + '/usr' + +(Contributed by Rafik Draoui and Serhiy Storchaka in :issue:`10395`.) pathlib @@ -1324,7 +1449,15 @@ -- References and conditional references to groups with fixed length are now -allowed in lookbehind assertions. +allowed in lookbehind assertions:: + + >>> import re + >>> pat = re.compile(r'(a|b).(?<=\1)c') + >>> pat.match('aac') + <_sre.SRE_Match object; span=(0, 3), match='aac'> + >>> pat.match('bbc') + <_sre.SRE_Match object; span=(0, 3), match='bbc'> + (Contributed by Serhiy Storchaka in :issue:`9179`.) The number of capturing groups in regular expression is no longer limited by @@ -1338,7 +1471,16 @@ :attr:`~re.error.msg`, :attr:`~re.error.pattern`, :attr:`~re.error.pos`, :attr:`~re.error.lineno`, and :attr:`~re.error.colno` that provide better context -information about the error. +information about the error:: + + >>> re.compile(""" + ... (?x) + ... .++ + ... """) + Traceback (most recent call last): + ... + sre_constants.error: multiple repeat at position 16 (line 3, column 7) + (Contributed by Serhiy Storchaka in :issue:`22578`.) @@ -1568,6 +1710,20 @@ compatibility with earlier Python versions. (Contributed by Thomas Kluyver in :issue:`23342`.) +Examples:: + + >>> subprocess.run(["ls", "-l"]) # doesn't capture output + CompletedProcess(args=['ls', '-l'], returncode=0) + + >>> subprocess.run("exit 1", shell=True, check=True) + Traceback (most recent call last): + ... + subprocess.CalledProcessError: Command 'exit 1' returned non-zero exit status 1 + + >>> subprocess.run(["ls", "-l", "/dev/null"], stdout=subprocess.PIPE) + CompletedProcess(args=['ls', '-l', '/dev/null'], returncode=0, + stdout=b'crw-rw-rw- 1 root root 1, 3 Jan 23 16:23 /dev/null\n') + sys --- -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 13 05:46:53 2015 From: python-checkins at python.org (yury.selivanov) Date: Sun, 13 Sep 2015 03:46:53 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogTWVyZ2UgMy41?= Message-ID: <20150913034653.66852.21178@psf.io> https://hg.python.org/cpython/rev/34f941df3476 changeset: 97961:34f941df3476 parent: 97959:f4dc1b8bb4b6 parent: 97960:c471f57097fb user: Yury Selivanov date: Sat Sep 12 23:46:50 2015 -0400 summary: Merge 3.5 files: Doc/whatsnew/3.5.rst | 198 +++++++++++++++++++++++++++--- 1 files changed, 177 insertions(+), 21 deletions(-) diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -358,7 +358,7 @@ PEP 461 - % formatting support for bytes and bytearray ------------------------------------------------------ -PEP 461 adds support for ``%`` +:pep:`461` adds support for ``%`` :ref:`interpolation operator ` to :class:`bytes` and :class:`bytearray`. @@ -447,15 +447,24 @@ :pep:`471` adds a new directory iteration function, :func:`os.scandir`, to the standard library. Additionally, :func:`os.walk` is now -implemented using ``os.scandir()``, which makes it 3 to 5 times faster +implemented using ``scandir``, which makes it 3 to 5 times faster on POSIX systems and 7 to 20 times faster on Windows systems. This is largely achieved by greatly reducing the number of calls to :func:`os.stat` required to walk a directory tree. -Additionally, ``os.scandir()`` returns an iterator, as opposed to returning +Additionally, ``scandir`` returns an iterator, as opposed to returning a list of file names, which improves memory efficiency when iterating over very large directories. +The following example shows a simple use of :func:`os.scandir` to display all +the files (excluding directories) in the given *path* that don't start with +``'.'``. The :meth:`entry.is_file ` call will generally +not make an additional system call:: + + for entry in os.scandir(path): + if not entry.name.startswith('.') and entry.is_file(): + print(entry.name) + .. seealso:: :pep:`471` -- os.scandir() function -- a better and faster directory iterator @@ -641,6 +650,27 @@ functions which tell whether two values are approximately equal or "close" to each other. Whether or not two values are considered close is determined according to given absolute and relative tolerances. +Relative tolerance is the maximum allowed difference between ``isclose()`` +arguments, relative to the larger absolute value:: + + >>> import math + >>> a = 5.0 + >>> b = 4.99998 + >>> math.isclose(a, b, rel_tol=1e-5) + True + >>> math.isclose(a, b, rel_tol=1e-6) + False + +It is also possible to compare two values using absolute tolerance, which +must be a non-negative value:: + + >>> import math + >>> a = 5.0 + >>> b = 4.99998 + >>> math.isclose(a, b, abs_tol=0.00003) + True + >>> math.isclose(a, b, abs_tol=0.00001) + False .. seealso:: @@ -678,6 +708,13 @@ New Modules =========== +typing +------ + +The new :mod:`typing` :term:`provisional ` module +provides standard definitions and tools for function type annotations. +See :ref:`Type Hints ` for more information. + .. _whatsnew-zipapp: zipapp @@ -854,8 +891,25 @@ ------------ Config parsers can be customized by providing a dictionary of converters in the -constructor. All converters defined in config parser (either by subclassing or +constructor, or All converters defined in config parser (either by subclassing or by providing in a constructor) will be available on all section proxies. + +Example:: + + >>> import configparser + >>> conv = {} + >>> conv['list'] = lambda v: [e.strip() for e in v.split() if e.strip()] + >>> cfg = configparser.ConfigParser(converters=conv) + >>> cfg.read_string(""" + ... [s] + ... list = a b c d e f g + ... """) + >>> cfg.get('s', 'list') + 'a b c d e f g' + >>> cfg.getlist('s', 'list') + ['a', 'b', 'c', 'd', 'e', 'f', 'g'] + + (Contributed by ?ukasz Langa in :issue:`18159`.) @@ -865,8 +919,17 @@ The new :func:`~contextlib.redirect_stderr` context manager (similar to :func:`~contextlib.redirect_stdout`) makes it easier for utility scripts to handle inflexible APIs that write their output to :data:`sys.stderr` and -don't provide any options to redirect it. (Contributed by Berker Peksag in -:issue:`22389`.) +don't provide any options to redirect it:: + + >>> import contextlib, io, logging + >>> f = io.StringIO() + >>> with contextlib.redirect_stderr(f): + ... logging.warning('warning') + ... + >>> f.getvalue() + 'WARNING:root:warning\n' + +(Contributed by Berker Peksag in :issue:`22389`.) curses @@ -1001,9 +1064,19 @@ ----- Element comparison in :func:`~heapq.merge` can now be customized by -passing a :term:`key function` in a new optional *key* keyword argument. -A new optional *reverse* keyword argument can be used to reverse element -comparison. (Contributed by Raymond Hettinger in :issue:`13742`.) +passing a :term:`key function` in a new optional *key* keyword argument, +and a new optional *reverse* keyword argument can be used to reverse element +comparison:: + + >>> import heapq + >>> a = ['9', '777', '55555'] + >>> b = ['88', '6666'] + >>> list(heapq.merge(a, b, key=len)) + ['9', '88', '777', '6666', '55555'] + >>> list(heapq.merge(reversed(a), reversed(b), key=len, reverse=True)) + ['55555', '6666', '777', '88', '9'] + +(Contributed by Raymond Hettinger in :issue:`13742`.) http @@ -1022,7 +1095,17 @@ remote server connection is closed unexpectedly. Additionally, if a :exc:`ConnectionError` (of which ``RemoteDisconnected`` is a subclass) is raised, the client socket is now closed automatically, -and will reconnect on the next request. +and will reconnect on the next request:: + + import http.client + conn = http.client.HTTPConnection('www.python.org') + for retries in range(3): + try: + conn.request('GET', '/') + resp = conn.getresponse() + except http.client.RemoteDisconnected: + pass + (Contributed by Martin Panter in :issue:`3566`.) @@ -1095,7 +1178,14 @@ A new :meth:`BoundArguments.apply_defaults ` -method provides a way to set default values for missing arguments. +method provides a way to set default values for missing arguments:: + + >>> def foo(a, b='ham', *args): pass + >>> ba = inspect.signature(foo).bind('spam') + >>> ba.apply_defaults() + >>> ba.arguments + OrderedDict([('a', 'spam'), ('b', 'ham'), ('args', ())]) + (Contributed by Yury Selivanov in :issue:`24190`.) A new class method @@ -1137,12 +1227,28 @@ Both :class:`~ipaddress.IPv4Network` and :class:`~ipaddress.IPv6Network` classes now accept an ``(address, netmask)`` tuple argument, so as to easily construct -network objects from existing addresses. (Contributed by Peter Moody -and Antoine Pitrou in :issue:`16531`.) +network objects from existing addresses:: + + >>> import ipaddress + >>> ipaddress.IPv4Network(('127.0.0.0', 8)) + IPv4Network('127.0.0.0/8') + >>> ipaddress.IPv4Network(('127.0.0.0', '255.0.0.0')) + IPv4Network('127.0.0.0/8') + +(Contributed by Peter Moody and Antoine Pitrou in :issue:`16531`.) A new :attr:`~ipaddress.IPv4Network.reverse_pointer>` attribute for :class:`~ipaddress.IPv4Network` and :class:`~ipaddress.IPv6Network` classes -returns the name of the reverse DNS PTR record. +returns the name of the reverse DNS PTR record:: + + >>> import ipaddress + >>> addr = ipaddress.IPv4Address('127.0.0.1') + >>> addr.reverse_pointer + '1.0.0.127.in-addr.arpa' + >>> addr6 = ipaddress.IPv6Address('::1') + >>> addr6.reverse_pointer + '1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.ip6.arpa' + (Contributed by Leon Weber in :issue:`20480`.) @@ -1173,7 +1279,18 @@ ------ A new :func:`~locale.delocalize` function can be used to convert a string into -a normalized number string, taking the ``LC_NUMERIC`` settings into account. +a normalized number string, taking the ``LC_NUMERIC`` settings into account:: + + >>> import locale + >>> locale.setlocale(locale.LC_NUMERIC, 'de_DE.UTF-8') + 'de_DE.UTF-8' + >>> locale.delocalize('1.234,56') + '1234.56' + >>> locale.setlocale(locale.LC_NUMERIC, 'en_US.UTF-8') + 'en_US.UTF-8' + >>> locale.delocalize('1,234.56') + '1234.56' + (Contributed by C?dric Krier in :issue:`13918`.) @@ -1223,9 +1340,9 @@ multiprocessing --------------- -:func:`~multiprocessing.synchronized` objects now support the -:term:`context manager` protocol. (Contributed by Charles-Fran?ois Natali in -:issue:`21565`.) +:func:`sharedctypes.synchronized ` +objects now support the :term:`context manager` protocol. +(Contributed by Charles-Fran?ois Natali in :issue:`21565`.) operator @@ -1271,7 +1388,15 @@ There is a new :func:`os.path.commonpath` function returning the longest common sub-path of each passed pathname. Unlike the :func:`os.path.commonprefix` function, it always returns a valid -path. (Contributed by Rafik Draoui and Serhiy Storchaka in :issue:`10395`.) +path:: + + >>> os.path.commonprefix(['/usr/lib', '/usr/local/lib']) + '/usr/l' + + >>> os.path.commonpath(['/usr/lib', '/usr/local/lib']) + '/usr' + +(Contributed by Rafik Draoui and Serhiy Storchaka in :issue:`10395`.) pathlib @@ -1324,7 +1449,15 @@ -- References and conditional references to groups with fixed length are now -allowed in lookbehind assertions. +allowed in lookbehind assertions:: + + >>> import re + >>> pat = re.compile(r'(a|b).(?<=\1)c') + >>> pat.match('aac') + <_sre.SRE_Match object; span=(0, 3), match='aac'> + >>> pat.match('bbc') + <_sre.SRE_Match object; span=(0, 3), match='bbc'> + (Contributed by Serhiy Storchaka in :issue:`9179`.) The number of capturing groups in regular expression is no longer limited by @@ -1338,7 +1471,16 @@ :attr:`~re.error.msg`, :attr:`~re.error.pattern`, :attr:`~re.error.pos`, :attr:`~re.error.lineno`, and :attr:`~re.error.colno` that provide better context -information about the error. +information about the error:: + + >>> re.compile(""" + ... (?x) + ... .++ + ... """) + Traceback (most recent call last): + ... + sre_constants.error: multiple repeat at position 16 (line 3, column 7) + (Contributed by Serhiy Storchaka in :issue:`22578`.) @@ -1568,6 +1710,20 @@ compatibility with earlier Python versions. (Contributed by Thomas Kluyver in :issue:`23342`.) +Examples:: + + >>> subprocess.run(["ls", "-l"]) # doesn't capture output + CompletedProcess(args=['ls', '-l'], returncode=0) + + >>> subprocess.run("exit 1", shell=True, check=True) + Traceback (most recent call last): + ... + subprocess.CalledProcessError: Command 'exit 1' returned non-zero exit status 1 + + >>> subprocess.run(["ls", "-l", "/dev/null"], stdout=subprocess.PIPE) + CompletedProcess(args=['ls', '-l', '/dev/null'], returncode=0, + stdout=b'crw-rw-rw- 1 root root 1, 3 Jan 23 16:23 /dev/null\n') + sys --- -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 13 05:47:45 2015 From: python-checkins at python.org (alexander.belopolsky) Date: Sun, 13 Sep 2015 03:47:45 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_PEP_495=3A_Corrected_the_desc?= =?utf-8?q?ription_of_an_interzone_arithmetic_paradox=2E?= Message-ID: <20150913034745.114754.45261@psf.io> https://hg.python.org/peps/rev/447ed890b416 changeset: 6054:447ed890b416 user: Alexander Belopolsky date: Sat Sep 12 23:47:36 2015 -0400 summary: PEP 495: Corrected the description of an interzone arithmetic paradox. files: pep-0495.txt | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/pep-0495.txt b/pep-0495.txt --- a/pep-0495.txt +++ b/pep-0495.txt @@ -433,7 +433,7 @@ ``t.tzinfo`` or ``s.tzinfo`` is post-PEP. [#]_ .. [#] Note that the new rules may result in a paradoxical situation - when ``s == t`` but ``s - t != timedelta(0)``. Such paradoxes are + when ``s == t`` but ``s - u != t - u``. Such paradoxes are not really new and are inherent in the overloading of the minus operator as two different intra- and inter-zone operations. For example, one can easily construct datetime instances ``t`` and ``s`` -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Sun Sep 13 05:54:39 2015 From: python-checkins at python.org (alexander.belopolsky) Date: Sun, 13 Sep 2015 03:54:39 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_PEP_495=3A_Minor_rewording=2E?= Message-ID: <20150913035439.68857.87768@psf.io> https://hg.python.org/peps/rev/5967673690e8 changeset: 6055:5967673690e8 user: Alexander Belopolsky date: Sat Sep 12 23:54:36 2015 -0400 summary: PEP 495: Minor rewording. files: pep-0495.txt | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/pep-0495.txt b/pep-0495.txt --- a/pep-0495.txt +++ b/pep-0495.txt @@ -428,7 +428,7 @@ The inter-zone subtraction will be defined as it is now: ``t - s`` is computed as ``(t - t.utcoffset()) - (s - -s.utcoffset()).replace(tzinfo=t.tzinfo)``, but the result may now +s.utcoffset()).replace(tzinfo=t.tzinfo)``, but the result will depend on the values of ``t.fold`` and ``s.fold`` when either ``t.tzinfo`` or ``s.tzinfo`` is post-PEP. [#]_ -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Sun Sep 13 06:29:16 2015 From: python-checkins at python.org (yury.selivanov) Date: Sun, 13 Sep 2015 04:29:16 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogTWVyZ2UgMy41?= Message-ID: <20150913042916.15710.46052@psf.io> https://hg.python.org/cpython/rev/9faede0c33a8 changeset: 97963:9faede0c33a8 parent: 97961:34f941df3476 parent: 97962:26104a2e5cc4 user: Yury Selivanov date: Sun Sep 13 00:29:14 2015 -0400 summary: Merge 3.5 files: Doc/whatsnew/3.5.rst | 38 ++++++++++++++++++++++++++++++++ 1 files changed, 38 insertions(+), 0 deletions(-) diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -760,9 +760,47 @@ Notable changes in :mod:`asyncio` module since Python 3.4.0: +* A new debugging APIs: :meth:`loop.set_debug() ` + and :meth:`loop.get_debug() `. + (Contributed by Victor Stinner.) + * The proactor event loop now supports SSL. (Contributed by Antoine Pitrou and Victor Stinner in :issue:`22560`.) +* A new :meth:`loop.is_closed() ` to + check if the event loop is closed. + (Contributed by Victor Stinner in :issue:`21326`.) + +* A new :meth:`loop.create_task() ` + to conveniently create and schedule a new :class:`~asyncio.Task` + for a coroutine. The ``create_task`` method is also used by all + asyncio functions that wrap coroutines into tasks: :func:`asyncio.wait`, + :func:`asyncio.gather`, etc. + (Contributed by Victor Stinner.) + +* A new :meth:`WriteTransport.get_write_buffer_limits ` + method to inquire for *high-* and *low-* water limits of the flow + control. + (Contributed by Victor Stinner.) + +* The :func:`~asyncio.async` function is deprecated in favor of + :func:`~asyncio.ensure_future`. + (Contributed by Yury Selivanov.) + +* New :meth:`loop.set_task_factory ` + and :meth:`loop.set_task_factory ` + to customize the task factory that + :meth:`loop.create_task() ` method uses. + (Contributed by Yury Selivanov.) + +* New :meth:`Queue.join ` and + :meth:`Queue.task_done ` queue methods. + (Contributed by Victor Stinner.) + +* The ``JoinableQueue`` class was removed, in favor of the + :class:`asyncio.Queue` class. + (Contributed by Victor Stinner.) + bz2 --- -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 13 06:29:16 2015 From: python-checkins at python.org (yury.selivanov) Date: Sun, 13 Sep 2015 04:29:16 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogd2hhdHNuZXcvMy41?= =?utf-8?q?=3A_Cover_asyncio_changes_relative_to_3=2E4=2E0?= Message-ID: <20150913042916.14857.91475@psf.io> https://hg.python.org/cpython/rev/26104a2e5cc4 changeset: 97962:26104a2e5cc4 branch: 3.5 parent: 97960:c471f57097fb user: Yury Selivanov date: Sun Sep 13 00:29:02 2015 -0400 summary: whatsnew/3.5: Cover asyncio changes relative to 3.4.0 files: Doc/whatsnew/3.5.rst | 38 ++++++++++++++++++++++++++++++++ 1 files changed, 38 insertions(+), 0 deletions(-) diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -760,9 +760,47 @@ Notable changes in :mod:`asyncio` module since Python 3.4.0: +* A new debugging APIs: :meth:`loop.set_debug() ` + and :meth:`loop.get_debug() `. + (Contributed by Victor Stinner.) + * The proactor event loop now supports SSL. (Contributed by Antoine Pitrou and Victor Stinner in :issue:`22560`.) +* A new :meth:`loop.is_closed() ` to + check if the event loop is closed. + (Contributed by Victor Stinner in :issue:`21326`.) + +* A new :meth:`loop.create_task() ` + to conveniently create and schedule a new :class:`~asyncio.Task` + for a coroutine. The ``create_task`` method is also used by all + asyncio functions that wrap coroutines into tasks: :func:`asyncio.wait`, + :func:`asyncio.gather`, etc. + (Contributed by Victor Stinner.) + +* A new :meth:`WriteTransport.get_write_buffer_limits ` + method to inquire for *high-* and *low-* water limits of the flow + control. + (Contributed by Victor Stinner.) + +* The :func:`~asyncio.async` function is deprecated in favor of + :func:`~asyncio.ensure_future`. + (Contributed by Yury Selivanov.) + +* New :meth:`loop.set_task_factory ` + and :meth:`loop.set_task_factory ` + to customize the task factory that + :meth:`loop.create_task() ` method uses. + (Contributed by Yury Selivanov.) + +* New :meth:`Queue.join ` and + :meth:`Queue.task_done ` queue methods. + (Contributed by Victor Stinner.) + +* The ``JoinableQueue`` class was removed, in favor of the + :class:`asyncio.Queue` class. + (Contributed by Victor Stinner.) + bz2 --- -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 13 07:14:19 2015 From: python-checkins at python.org (yury.selivanov) Date: Sun, 13 Sep 2015 05:14:19 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogd2hhdHNuZXcvMy41?= =?utf-8?q?=3A_More_examples?= Message-ID: <20150913051046.27697.39500@psf.io> https://hg.python.org/cpython/rev/ef2423d18c1c changeset: 97964:ef2423d18c1c branch: 3.5 parent: 97962:26104a2e5cc4 user: Yury Selivanov date: Sun Sep 13 01:10:19 2015 -0400 summary: whatsnew/3.5: More examples files: Doc/whatsnew/3.5.rst | 29 +++++++++++++++++++++++++++-- 1 files changed, 27 insertions(+), 2 deletions(-) diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -1339,7 +1339,16 @@ :meth:`~logging.Logger.exception`, :meth:`~logging.Logger.critical`, :meth:`~logging.Logger.debug`, etc.), now accept exception instances as an ``exc_info`` argument, in addition to boolean values and exception -tuples. (Contributed by Yury Selivanov in :issue:`20537`.) +tuples:: + + >>> import logging + >>> try: + ... 1/0 + ... except ZeroDivisionError as ex: + ... logging.error('exception', exc_info=ex) + ERROR:root:exception + +(Contributed by Yury Selivanov in :issue:`20537`.) The :class:`handlers.HTTPHandler ` class now accepts an optional :class:`ssl.SSLContext` instance to configure SSL @@ -1442,7 +1451,14 @@ The new :meth:`Path.samefile ` method can be used to check whether the path points to the same file as other path, which can be -either an another :class:`~pathlib.Path` object, or a string. +either an another :class:`~pathlib.Path` object, or a string:: + + >>> import pathlib + >>> p1 = pathlib.Path('/etc/hosts') + >>> p2 = pathlib.Path('/etc/../etc/hosts') + >>> p1.samefile(p2) + True + (Contributed by Vajrasky Kok and Antoine Pitrou in :issue:`19775`.) The :meth:`Path.mkdir ` method how accepts a new optional @@ -1463,6 +1479,15 @@ :meth:`Path.write_bytes `, :meth:`Path.read_bytes ` methods to simplify read/write operations on files. + +The following code snippet will create or rewrite existing file +``~/spam42``:: + + >>> import pathlib + >>> p = pathlib.Path('~/spam42') + >>> p.expanduser().write_text('ham') + 3 + (Contributed by Christopher Welborn in :issue:`20218`.) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 13 07:39:32 2015 From: python-checkins at python.org (yury.selivanov) Date: Sun, 13 Sep 2015 05:39:32 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogd2hhdHNuZXcvMy41?= =?utf-8?q?=3A_Tweak_asyncio_module_section?= Message-ID: <20150913053932.66856.52085@psf.io> https://hg.python.org/cpython/rev/cd74d760f334 changeset: 97966:cd74d760f334 branch: 3.5 parent: 97964:ef2423d18c1c user: Yury Selivanov date: Sun Sep 13 01:39:05 2015 -0400 summary: whatsnew/3.5: Tweak asyncio module section files: Doc/whatsnew/3.5.rst | 14 +++++++------- 1 files changed, 7 insertions(+), 7 deletions(-) diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -761,13 +761,13 @@ Notable changes in :mod:`asyncio` module since Python 3.4.0: * A new debugging APIs: :meth:`loop.set_debug() ` - and :meth:`loop.get_debug() `. + and :meth:`loop.get_debug() ` methods. (Contributed by Victor Stinner.) * The proactor event loop now supports SSL. (Contributed by Antoine Pitrou and Victor Stinner in :issue:`22560`.) -* A new :meth:`loop.is_closed() ` to +* A new :meth:`loop.is_closed() ` method to check if the event loop is closed. (Contributed by Victor Stinner in :issue:`21326`.) @@ -787,14 +787,14 @@ :func:`~asyncio.ensure_future`. (Contributed by Yury Selivanov.) -* New :meth:`loop.set_task_factory ` - and :meth:`loop.set_task_factory ` - to customize the task factory that +* New :meth:`loop.set_task_factory() ` + and :meth:`loop.set_task_factory() ` + methods to customize the task factory that :meth:`loop.create_task() ` method uses. (Contributed by Yury Selivanov.) -* New :meth:`Queue.join ` and - :meth:`Queue.task_done ` queue methods. +* New :meth:`Queue.join() ` and + :meth:`Queue.task_done() ` queue methods. (Contributed by Victor Stinner.) * The ``JoinableQueue`` class was removed, in favor of the -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 13 07:39:32 2015 From: python-checkins at python.org (yury.selivanov) Date: Sun, 13 Sep 2015 05:39:32 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogTWVyZ2UgMy41?= Message-ID: <20150913053932.11268.37965@psf.io> https://hg.python.org/cpython/rev/e788c1141c09 changeset: 97967:e788c1141c09 parent: 97965:c8171af00966 parent: 97966:cd74d760f334 user: Yury Selivanov date: Sun Sep 13 01:39:16 2015 -0400 summary: Merge 3.5 files: Doc/whatsnew/3.5.rst | 14 +++++++------- 1 files changed, 7 insertions(+), 7 deletions(-) diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -761,13 +761,13 @@ Notable changes in :mod:`asyncio` module since Python 3.4.0: * A new debugging APIs: :meth:`loop.set_debug() ` - and :meth:`loop.get_debug() `. + and :meth:`loop.get_debug() ` methods. (Contributed by Victor Stinner.) * The proactor event loop now supports SSL. (Contributed by Antoine Pitrou and Victor Stinner in :issue:`22560`.) -* A new :meth:`loop.is_closed() ` to +* A new :meth:`loop.is_closed() ` method to check if the event loop is closed. (Contributed by Victor Stinner in :issue:`21326`.) @@ -787,14 +787,14 @@ :func:`~asyncio.ensure_future`. (Contributed by Yury Selivanov.) -* New :meth:`loop.set_task_factory ` - and :meth:`loop.set_task_factory ` - to customize the task factory that +* New :meth:`loop.set_task_factory() ` + and :meth:`loop.set_task_factory() ` + methods to customize the task factory that :meth:`loop.create_task() ` method uses. (Contributed by Yury Selivanov.) -* New :meth:`Queue.join ` and - :meth:`Queue.task_done ` queue methods. +* New :meth:`Queue.join() ` and + :meth:`Queue.task_done() ` queue methods. (Contributed by Victor Stinner.) * The ``JoinableQueue`` class was removed, in favor of the -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 13 07:40:56 2015 From: python-checkins at python.org (yury.selivanov) Date: Sun, 13 Sep 2015 05:40:56 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogd2hhdHNuZXcvMy41?= =?utf-8?q?=3A_Fix_formatting?= Message-ID: <20150913054056.11248.87523@psf.io> https://hg.python.org/cpython/rev/adc30928b25b changeset: 97968:adc30928b25b branch: 3.5 parent: 97966:cd74d760f334 user: Yury Selivanov date: Sun Sep 13 01:40:36 2015 -0400 summary: whatsnew/3.5: Fix formatting files: Doc/whatsnew/3.5.rst | 12 ++++++------ 1 files changed, 6 insertions(+), 6 deletions(-) diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -2102,17 +2102,17 @@ New ``calloc`` functions were added: - * :c:func:`PyMem_RawCalloc`, - * :c:func:`PyMem_Calloc`, - * :c:func:`PyObject_Calloc`, - * :c:func:`_PyObject_GC_Calloc`. +* :c:func:`PyMem_RawCalloc`, +* :c:func:`PyMem_Calloc`, +* :c:func:`PyObject_Calloc`, +* :c:func:`_PyObject_GC_Calloc`. (Contributed by Victor Stinner in :issue:`21233`.) New encoding/decoding helper functions: - * :c:func:`Py_DecodeLocale` (replaced ``_Py_char2wchar()``), - * :c:func:`Py_EncodeLocale` (replaced ``_Py_wchar2char()``). +* :c:func:`Py_DecodeLocale` (replaced ``_Py_char2wchar()``), +* :c:func:`Py_EncodeLocale` (replaced ``_Py_wchar2char()``). (Contributed by Victor Stinner in :issue:`18395`.) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 13 07:40:57 2015 From: python-checkins at python.org (yury.selivanov) Date: Sun, 13 Sep 2015 05:40:57 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogTWVyZ2UgMy41?= Message-ID: <20150913054056.17971.5548@psf.io> https://hg.python.org/cpython/rev/0790c2e00eca changeset: 97969:0790c2e00eca parent: 97967:e788c1141c09 parent: 97968:adc30928b25b user: Yury Selivanov date: Sun Sep 13 01:40:46 2015 -0400 summary: Merge 3.5 files: Doc/whatsnew/3.5.rst | 12 ++++++------ 1 files changed, 6 insertions(+), 6 deletions(-) diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -2102,17 +2102,17 @@ New ``calloc`` functions were added: - * :c:func:`PyMem_RawCalloc`, - * :c:func:`PyMem_Calloc`, - * :c:func:`PyObject_Calloc`, - * :c:func:`_PyObject_GC_Calloc`. +* :c:func:`PyMem_RawCalloc`, +* :c:func:`PyMem_Calloc`, +* :c:func:`PyObject_Calloc`, +* :c:func:`_PyObject_GC_Calloc`. (Contributed by Victor Stinner in :issue:`21233`.) New encoding/decoding helper functions: - * :c:func:`Py_DecodeLocale` (replaced ``_Py_char2wchar()``), - * :c:func:`Py_EncodeLocale` (replaced ``_Py_wchar2char()``). +* :c:func:`Py_DecodeLocale` (replaced ``_Py_char2wchar()``), +* :c:func:`Py_EncodeLocale` (replaced ``_Py_wchar2char()``). (Contributed by Victor Stinner in :issue:`18395`.) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 13 07:42:13 2015 From: python-checkins at python.org (yury.selivanov) Date: Sun, 13 Sep 2015 05:42:13 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogTWVyZ2UgMy41?= Message-ID: <20150913051046.114864.26505@psf.io> https://hg.python.org/cpython/rev/c8171af00966 changeset: 97965:c8171af00966 parent: 97963:9faede0c33a8 parent: 97964:ef2423d18c1c user: Yury Selivanov date: Sun Sep 13 01:10:29 2015 -0400 summary: Merge 3.5 files: Doc/whatsnew/3.5.rst | 29 +++++++++++++++++++++++++++-- 1 files changed, 27 insertions(+), 2 deletions(-) diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -1339,7 +1339,16 @@ :meth:`~logging.Logger.exception`, :meth:`~logging.Logger.critical`, :meth:`~logging.Logger.debug`, etc.), now accept exception instances as an ``exc_info`` argument, in addition to boolean values and exception -tuples. (Contributed by Yury Selivanov in :issue:`20537`.) +tuples:: + + >>> import logging + >>> try: + ... 1/0 + ... except ZeroDivisionError as ex: + ... logging.error('exception', exc_info=ex) + ERROR:root:exception + +(Contributed by Yury Selivanov in :issue:`20537`.) The :class:`handlers.HTTPHandler ` class now accepts an optional :class:`ssl.SSLContext` instance to configure SSL @@ -1442,7 +1451,14 @@ The new :meth:`Path.samefile ` method can be used to check whether the path points to the same file as other path, which can be -either an another :class:`~pathlib.Path` object, or a string. +either an another :class:`~pathlib.Path` object, or a string:: + + >>> import pathlib + >>> p1 = pathlib.Path('/etc/hosts') + >>> p2 = pathlib.Path('/etc/../etc/hosts') + >>> p1.samefile(p2) + True + (Contributed by Vajrasky Kok and Antoine Pitrou in :issue:`19775`.) The :meth:`Path.mkdir ` method how accepts a new optional @@ -1463,6 +1479,15 @@ :meth:`Path.write_bytes `, :meth:`Path.read_bytes ` methods to simplify read/write operations on files. + +The following code snippet will create or rewrite existing file +``~/spam42``:: + + >>> import pathlib + >>> p = pathlib.Path('~/spam42') + >>> p.expanduser().write_text('ham') + 3 + (Contributed by Christopher Welborn in :issue:`20218`.) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 13 07:51:33 2015 From: python-checkins at python.org (larry.hastings) Date: Sun, 13 Sep 2015 05:51:33 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_Minor_typo_fix=2E__=28=22Widn?= =?utf-8?b?b3dzIiAtPiAiV2luZG93cyIp?= Message-ID: <20150913055133.17967.47768@psf.io> https://hg.python.org/peps/rev/a26d291b320c changeset: 6056:a26d291b320c user: Larry Hastings date: Sun Sep 13 06:51:13 2015 +0100 summary: Minor typo fix. ("Widnows" -> "Windows") files: pep-0478.txt | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/pep-0478.txt b/pep-0478.txt --- a/pep-0478.txt +++ b/pep-0478.txt @@ -66,7 +66,7 @@ * PEP 479, change StopIteration handling inside generators * PEP 484, the typing module, a new standard for type annotations * PEP 485, math.isclose(), a function for testing approximate equality -* PEP 486, making the Widnows Python launcher aware of virtual environments +* PEP 486, making the Windows Python launcher aware of virtual environments * PEP 488, eliminating .pyo files * PEP 489, a new and improved mechanism for loading extension modules * PEP 492, coroutines with async and await syntax -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Sun Sep 13 07:58:18 2015 From: python-checkins at python.org (yury.selivanov) Date: Sun, 13 Sep 2015 05:58:18 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogTWVyZ2UgMy41?= Message-ID: <20150913055818.17983.32984@psf.io> https://hg.python.org/cpython/rev/8d703eb80d41 changeset: 97971:8d703eb80d41 parent: 97969:0790c2e00eca parent: 97970:2c192ceb39f5 user: Yury Selivanov date: Sun Sep 13 01:58:09 2015 -0400 summary: Merge 3.5 files: Doc/whatsnew/3.5.rst | 11 ----------- 1 files changed, 0 insertions(+), 11 deletions(-) diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -2,8 +2,6 @@ What's New In Python 3.5 **************************** -:Release: |release| -:Date: |today| :Editors: Elvis Pranskevichus , Yury Selivanov .. Rules for maintenance: @@ -49,12 +47,6 @@ This article explains the new features in Python 3.5, compared to 3.4. For full details, see the :source:`Misc/NEWS` file. -.. note:: - - Prerelease users should be aware that this document is currently in draft - form. It will be updated substantially as Python 3.5 moves towards release, - so it's worth checking back even after reading earlier versions. - .. seealso:: @@ -64,9 +56,6 @@ Summary -- Release highlights ============================= -.. This section singles out the most important changes in Python 3.5. - Brevity is key. - New syntax features: * :pep:`492`, coroutines with async and await syntax. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 13 07:58:18 2015 From: python-checkins at python.org (yury.selivanov) Date: Sun, 13 Sep 2015 05:58:18 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogd2hhdHNuZXcvMy41?= =?utf-8?q?=3A_Delete_prerelease_warning_note=2E?= Message-ID: <20150913055818.11258.29205@psf.io> https://hg.python.org/cpython/rev/2c192ceb39f5 changeset: 97970:2c192ceb39f5 branch: 3.5 parent: 97968:adc30928b25b user: Yury Selivanov date: Sun Sep 13 01:57:57 2015 -0400 summary: whatsnew/3.5: Delete prerelease warning note. (we'll make a couple more commits tomorrow before release) files: Doc/whatsnew/3.5.rst | 11 ----------- 1 files changed, 0 insertions(+), 11 deletions(-) diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -2,8 +2,6 @@ What's New In Python 3.5 **************************** -:Release: |release| -:Date: |today| :Editors: Elvis Pranskevichus , Yury Selivanov .. Rules for maintenance: @@ -49,12 +47,6 @@ This article explains the new features in Python 3.5, compared to 3.4. For full details, see the :source:`Misc/NEWS` file. -.. note:: - - Prerelease users should be aware that this document is currently in draft - form. It will be updated substantially as Python 3.5 moves towards release, - so it's worth checking back even after reading earlier versions. - .. seealso:: @@ -64,9 +56,6 @@ Summary -- Release highlights ============================= -.. This section singles out the most important changes in Python 3.5. - Brevity is key. - New syntax features: * :pep:`492`, coroutines with async and await syntax. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 13 08:41:28 2015 From: python-checkins at python.org (raymond.hettinger) Date: Sun, 13 Sep 2015 06:41:28 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Fix_refcount=2E?= Message-ID: <20150913064128.14857.79645@psf.io> https://hg.python.org/cpython/rev/3eaeb280d8e5 changeset: 97972:3eaeb280d8e5 user: Raymond Hettinger date: Sun Sep 13 02:41:18 2015 -0400 summary: Fix refcount. files: Modules/_collectionsmodule.c | 5 ++++- 1 files changed, 4 insertions(+), 1 deletions(-) diff --git a/Modules/_collectionsmodule.c b/Modules/_collectionsmodule.c --- a/Modules/_collectionsmodule.c +++ b/Modules/_collectionsmodule.c @@ -616,11 +616,14 @@ deque_repeat(dequeobject *deque, Py_ssize_t n) { dequeobject *new_deque; + PyObject *rv; new_deque = (dequeobject *)deque_copy((PyObject *) deque); if (new_deque == NULL) return NULL; - return deque_inplace_repeat(new_deque, n); + rv = deque_inplace_repeat(new_deque, n); + Py_DECREF(new_deque); + return rv; } /* The rotate() method is part of the public API and is used internally -- Repository URL: https://hg.python.org/cpython From solipsis at pitrou.net Sun Sep 13 10:47:03 2015 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Sun, 13 Sep 2015 08:47:03 +0000 Subject: [Python-checkins] Daily reference leaks (0790c2e00eca): sum=33569 Message-ID: <20150913084702.68861.97908@psf.io> results for 0790c2e00eca on branch "default" -------------------------------------------- test_capi leaked [5446, 5446, 5446] references, sum=16338 test_capi leaked [1433, 1435, 1435] memory blocks, sum=4303 test_deque leaked [257, 257, 257] references, sum=771 test_deque leaked [79, 80, 80] memory blocks, sum=239 test_functools leaked [0, 2, 2] memory blocks, sum=4 test_threading leaked [3196, 3196, 3196] references, sum=9588 test_threading leaked [774, 776, 776] memory blocks, sum=2326 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/psf-users/antoine/refleaks/reflog2q4fBm', '--timeout', '7200'] From python-checkins at python.org Sun Sep 13 11:18:32 2015 From: python-checkins at python.org (serhiy.storchaka) Date: Sun, 13 Sep 2015 09:18:32 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogd2hhdHNuZXcvMy41?= =?utf-8?q?=3A_Fix_formatting=2E_More_minor_edits=2E?= Message-ID: <20150913091832.101502.20015@psf.io> https://hg.python.org/cpython/rev/334a988f744d changeset: 97973:334a988f744d branch: 3.5 parent: 97970:2c192ceb39f5 user: Serhiy Storchaka date: Sun Sep 13 12:07:54 2015 +0300 summary: whatsnew/3.5: Fix formatting. More minor edits. files: Doc/whatsnew/3.5.rst | 46 ++++++++++++++++---------------- 1 files changed, 23 insertions(+), 23 deletions(-) diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -683,13 +683,13 @@ * New Kazakh :ref:`codec ` ``kz1048``. (Contributed by Serhiy Storchaka in :issue:`22682`.) +* New Tajik :ref:`codec ` ``koi8_t``. (Contributed by + Serhiy Storchaka in :issue:`22681`.) + * Property docstrings are now writable. This is especially useful for :func:`collections.namedtuple` docstrings. (Contributed by Berker Peksag in :issue:`24064`.) -* New Tajik :ref:`codec ` ``koi8_t``. (Contributed by - Serhiy Storchaka in :issue:`22681`.) - * Circular imports involving relative imports are now supported. (Contributed by Brett Cannon and Antoine Pitrou in :issue:`17636`.) @@ -802,14 +802,14 @@ cgi --- -The :class:`~cgi.FieldStorage` class now supports the context management +The :class:`~cgi.FieldStorage` class now supports the :term:`context manager` protocol. (Contributed by Berker Peksag in :issue:`20289`.) csv --- -:meth:`Writer.writerow ` now supports arbitrary iterables, +The :meth:`~csv.csvwriter.writerow` method now supports arbitrary iterables, not just sequences. (Contributed by Serhiy Storchaka in :issue:`23171`.) @@ -885,7 +885,7 @@ compileall ---------- -A new :mod:`compileall` option, ``-j N``, allows to run ``N`` workers +A new :mod:`compileall` option, :samp:`-j {N}`, allows to run *N* workers sumultaneously to perform parallel bytecode compilation. The :func:`~compileall.compile_dir` function has a corresponding ``workers`` parameter. (Contributed by Claudiu Popa in :issue:`16104`.) @@ -909,7 +909,7 @@ :meth:`~concurrent.futures.ProcessPoolExecutor` is used. (Contributed by Dan O'Reilly in :issue:`11271`.) -A number of workers in :class:`~concurrent.futures.ThreadPoolExecutor` is +A number of workers in :class:`~concurrent.futures.ThreadPoolExecutor` constructor is optional now. The default value equals to 5 times the number of CPUs. (Contributed by Claudiu Popa in :issue:`21527`.) @@ -943,7 +943,7 @@ contextlib ---------- -The new :func:`~contextlib.redirect_stderr` context manager (similar to +The new :func:`~contextlib.redirect_stderr` :term:`context manager` (similar to :func:`~contextlib.redirect_stdout`) makes it easier for utility scripts to handle inflexible APIs that write their output to :data:`sys.stderr` and don't provide any options to redirect it:: @@ -1031,7 +1031,7 @@ ``SMTPUTF8`` extension. (Contributed by R. David Murray in :issue:`24211`.) -:class:`email.mime.text.MIMEText` constructor now accepts a +:class:`~email.mime.text.MIMEText` constructor now accepts a :class:`~email.charset.Charset` instance. (Contributed by Claude Paroz and Berker Peksag in :issue:`16324`.) @@ -1143,13 +1143,13 @@ import by other programs, it gets improvements with every release. See :file:`Lib/idlelib/NEWS.txt` for a cumulative list of changes since 3.4.0, as well as changes made in future 3.5.x releases. This file is also available -from the IDLE Help -> About Idle dialog. +from the IDLE :menuselection:`Help --> About IDLE` dialog. imaplib ------- -The :class:`~imaplib.IMAP4` class now supports context manager protocol. +The :class:`~imaplib.IMAP4` class now supports :term:`context manager` protocol. When used in a :keyword:`with` statement, the IMAP4 ``LOGOUT`` command will be called automatically at the end of the block. (Contributed by Tarek Ziad? and Serhiy Storchaka in :issue:`4972`.) @@ -1220,7 +1220,7 @@ subclassing of :class:`~inspect.Signature` easier. (Contributed by Yury Selivanov and Eric Snow in :issue:`17373`.) -The :func:`~inspect.signature` function now accepts a ``follow_wrapped`` +The :func:`~inspect.signature` function now accepts a *follow_wrapped* optional keyword argument, which, when set to ``False``, disables automatic following of ``__wrapped__`` links. (Contributed by Yury Selivanov in :issue:`20691`.) @@ -1264,7 +1264,7 @@ (Contributed by Peter Moody and Antoine Pitrou in :issue:`16531`.) -A new :attr:`~ipaddress.IPv4Network.reverse_pointer>` attribute for +A new :attr:`~ipaddress.IPv4Network.reverse_pointer` attribute for :class:`~ipaddress.IPv4Network` and :class:`~ipaddress.IPv6Network` classes returns the name of the reverse DNS PTR record:: @@ -1287,7 +1287,7 @@ to sort the keys alphabetically. (Contributed by Berker Peksag in :issue:`21650`.) -JSON decoder now raises :exc:`json.JSONDecodeError` instead of +JSON decoder now raises :exc:`~json.JSONDecodeError` instead of :exc:`ValueError` to provide better context information about the error. (Contributed by Serhiy Storchaka in :issue:`19361`.) @@ -1327,7 +1327,7 @@ All logging methods (:class:`~logging.Logger` :meth:`~logging.Logger.log`, :meth:`~logging.Logger.exception`, :meth:`~logging.Logger.critical`, :meth:`~logging.Logger.debug`, etc.), now accept exception instances -as an ``exc_info`` argument, in addition to boolean values and exception +as an *exc_info* argument, in addition to boolean values and exception tuples:: >>> import logging @@ -1451,7 +1451,7 @@ (Contributed by Vajrasky Kok and Antoine Pitrou in :issue:`19775`.) The :meth:`Path.mkdir ` method how accepts a new optional -``exist_ok`` argument to match ``mkdir -p`` and :func:`os.makrdirs` +*exist_ok* argument to match ``mkdir -p`` and :func:`os.makrdirs` functionality. (Contributed by Berker Peksag in :issue:`21539`.) There is a new :meth:`Path.expanduser ` method to @@ -1595,14 +1595,14 @@ (:rfc:`6152`) if *decode_data* has been set ``True``. If the client specifies ``BODY=8BITMIME`` on the ``MAIL`` command, it is passed to :meth:`SMTPServer.process_message ` -via the ``mail_options`` keyword. +via the *mail_options* keyword. (Contributed by Milan Oberkirch and R. David Murray in :issue:`21795`.) The :class:`~smtpd.SMTPServer` class now also supports the ``SMTPUTF8`` extension (:rfc:`6531`: Internationalized Email). If the client specified ``SMTPUTF8 BODY=8BITMIME`` on the ``MAIL`` command, they are passed to :meth:`SMTPServer.process_message ` -via the ``mail_options`` keyword. It is the responsibility of the +via the *mail_options* keyword. It is the responsibility of the :meth:`~smtpd.SMTPServer.process_message` method to correctly handle the ``SMTPUTF8`` data. (Contributed by Milan Oberkirch in :issue:`21725`.) @@ -1840,7 +1840,7 @@ timeit ------ -A new command line option ``-u`` or ``--unit=U`` can be used to specify the time +A new command line option ``-u`` or :samp:`--unit={U}` can be used to specify the time unit for the timer output. Supported options are ``usec``, ``msec``, or ``sec``. (Contributed by Julian Gindi in :issue:`18983`.) @@ -1868,7 +1868,7 @@ (Contributed by Robert Collins in :issue:`17911`.) New lightweight classes: :class:`~traceback.TracebackException`, -:class:`~traceback.StackSummary`, and :class:`traceback.FrameSummary`. +:class:`~traceback.StackSummary`, and :class:`~traceback.FrameSummary`. (Contributed by Robert Collins in :issue:`17911`.) Both :func:`~traceback.print_tb` and :func:`~traceback.print_stack` functions @@ -1978,8 +1978,8 @@ xmlrpc ------ -The :class:`client.ServerProxy ` class is now a -:term:`context manager`. +The :class:`client.ServerProxy ` class now supports +:term:`context manager` protocol. (Contributed by Claudiu Popa in :issue:`20627`.) :class:`client.ServerProxy ` constructor now accepts @@ -2252,7 +2252,7 @@ :func:`inspect.signature` API. (Contributed by Yury Selivanov in :issue:`20438`.) -Use of ``re.LOCALE`` flag with str patterns or ``re.ASCII`` is now +Use of :const:`re.LOCALE` flag with str patterns or :const:`re.ASCII` is now deprecated. (Contributed by Serhiy Storchaka in :issue:`22407`.) Use of unrecognized special sequences consisting of ``'\'`` and an ASCII letter -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 13 11:18:33 2015 From: python-checkins at python.org (serhiy.storchaka) Date: Sun, 13 Sep 2015 09:18:33 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogTWVyZ2UgMy41?= Message-ID: <20150913091832.15726.64308@psf.io> https://hg.python.org/cpython/rev/76913f1eea0f changeset: 97974:76913f1eea0f parent: 97972:3eaeb280d8e5 parent: 97973:334a988f744d user: Serhiy Storchaka date: Sun Sep 13 12:08:19 2015 +0300 summary: Merge 3.5 files: Doc/whatsnew/3.5.rst | 46 ++++++++++++++++---------------- 1 files changed, 23 insertions(+), 23 deletions(-) diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -683,13 +683,13 @@ * New Kazakh :ref:`codec ` ``kz1048``. (Contributed by Serhiy Storchaka in :issue:`22682`.) +* New Tajik :ref:`codec ` ``koi8_t``. (Contributed by + Serhiy Storchaka in :issue:`22681`.) + * Property docstrings are now writable. This is especially useful for :func:`collections.namedtuple` docstrings. (Contributed by Berker Peksag in :issue:`24064`.) -* New Tajik :ref:`codec ` ``koi8_t``. (Contributed by - Serhiy Storchaka in :issue:`22681`.) - * Circular imports involving relative imports are now supported. (Contributed by Brett Cannon and Antoine Pitrou in :issue:`17636`.) @@ -802,14 +802,14 @@ cgi --- -The :class:`~cgi.FieldStorage` class now supports the context management +The :class:`~cgi.FieldStorage` class now supports the :term:`context manager` protocol. (Contributed by Berker Peksag in :issue:`20289`.) csv --- -:meth:`Writer.writerow ` now supports arbitrary iterables, +The :meth:`~csv.csvwriter.writerow` method now supports arbitrary iterables, not just sequences. (Contributed by Serhiy Storchaka in :issue:`23171`.) @@ -885,7 +885,7 @@ compileall ---------- -A new :mod:`compileall` option, ``-j N``, allows to run ``N`` workers +A new :mod:`compileall` option, :samp:`-j {N}`, allows to run *N* workers sumultaneously to perform parallel bytecode compilation. The :func:`~compileall.compile_dir` function has a corresponding ``workers`` parameter. (Contributed by Claudiu Popa in :issue:`16104`.) @@ -909,7 +909,7 @@ :meth:`~concurrent.futures.ProcessPoolExecutor` is used. (Contributed by Dan O'Reilly in :issue:`11271`.) -A number of workers in :class:`~concurrent.futures.ThreadPoolExecutor` is +A number of workers in :class:`~concurrent.futures.ThreadPoolExecutor` constructor is optional now. The default value equals to 5 times the number of CPUs. (Contributed by Claudiu Popa in :issue:`21527`.) @@ -943,7 +943,7 @@ contextlib ---------- -The new :func:`~contextlib.redirect_stderr` context manager (similar to +The new :func:`~contextlib.redirect_stderr` :term:`context manager` (similar to :func:`~contextlib.redirect_stdout`) makes it easier for utility scripts to handle inflexible APIs that write their output to :data:`sys.stderr` and don't provide any options to redirect it:: @@ -1031,7 +1031,7 @@ ``SMTPUTF8`` extension. (Contributed by R. David Murray in :issue:`24211`.) -:class:`email.mime.text.MIMEText` constructor now accepts a +:class:`~email.mime.text.MIMEText` constructor now accepts a :class:`~email.charset.Charset` instance. (Contributed by Claude Paroz and Berker Peksag in :issue:`16324`.) @@ -1143,13 +1143,13 @@ import by other programs, it gets improvements with every release. See :file:`Lib/idlelib/NEWS.txt` for a cumulative list of changes since 3.4.0, as well as changes made in future 3.5.x releases. This file is also available -from the IDLE Help -> About Idle dialog. +from the IDLE :menuselection:`Help --> About IDLE` dialog. imaplib ------- -The :class:`~imaplib.IMAP4` class now supports context manager protocol. +The :class:`~imaplib.IMAP4` class now supports :term:`context manager` protocol. When used in a :keyword:`with` statement, the IMAP4 ``LOGOUT`` command will be called automatically at the end of the block. (Contributed by Tarek Ziad? and Serhiy Storchaka in :issue:`4972`.) @@ -1220,7 +1220,7 @@ subclassing of :class:`~inspect.Signature` easier. (Contributed by Yury Selivanov and Eric Snow in :issue:`17373`.) -The :func:`~inspect.signature` function now accepts a ``follow_wrapped`` +The :func:`~inspect.signature` function now accepts a *follow_wrapped* optional keyword argument, which, when set to ``False``, disables automatic following of ``__wrapped__`` links. (Contributed by Yury Selivanov in :issue:`20691`.) @@ -1264,7 +1264,7 @@ (Contributed by Peter Moody and Antoine Pitrou in :issue:`16531`.) -A new :attr:`~ipaddress.IPv4Network.reverse_pointer>` attribute for +A new :attr:`~ipaddress.IPv4Network.reverse_pointer` attribute for :class:`~ipaddress.IPv4Network` and :class:`~ipaddress.IPv6Network` classes returns the name of the reverse DNS PTR record:: @@ -1287,7 +1287,7 @@ to sort the keys alphabetically. (Contributed by Berker Peksag in :issue:`21650`.) -JSON decoder now raises :exc:`json.JSONDecodeError` instead of +JSON decoder now raises :exc:`~json.JSONDecodeError` instead of :exc:`ValueError` to provide better context information about the error. (Contributed by Serhiy Storchaka in :issue:`19361`.) @@ -1327,7 +1327,7 @@ All logging methods (:class:`~logging.Logger` :meth:`~logging.Logger.log`, :meth:`~logging.Logger.exception`, :meth:`~logging.Logger.critical`, :meth:`~logging.Logger.debug`, etc.), now accept exception instances -as an ``exc_info`` argument, in addition to boolean values and exception +as an *exc_info* argument, in addition to boolean values and exception tuples:: >>> import logging @@ -1451,7 +1451,7 @@ (Contributed by Vajrasky Kok and Antoine Pitrou in :issue:`19775`.) The :meth:`Path.mkdir ` method how accepts a new optional -``exist_ok`` argument to match ``mkdir -p`` and :func:`os.makrdirs` +*exist_ok* argument to match ``mkdir -p`` and :func:`os.makrdirs` functionality. (Contributed by Berker Peksag in :issue:`21539`.) There is a new :meth:`Path.expanduser ` method to @@ -1595,14 +1595,14 @@ (:rfc:`6152`) if *decode_data* has been set ``True``. If the client specifies ``BODY=8BITMIME`` on the ``MAIL`` command, it is passed to :meth:`SMTPServer.process_message ` -via the ``mail_options`` keyword. +via the *mail_options* keyword. (Contributed by Milan Oberkirch and R. David Murray in :issue:`21795`.) The :class:`~smtpd.SMTPServer` class now also supports the ``SMTPUTF8`` extension (:rfc:`6531`: Internationalized Email). If the client specified ``SMTPUTF8 BODY=8BITMIME`` on the ``MAIL`` command, they are passed to :meth:`SMTPServer.process_message ` -via the ``mail_options`` keyword. It is the responsibility of the +via the *mail_options* keyword. It is the responsibility of the :meth:`~smtpd.SMTPServer.process_message` method to correctly handle the ``SMTPUTF8`` data. (Contributed by Milan Oberkirch in :issue:`21725`.) @@ -1840,7 +1840,7 @@ timeit ------ -A new command line option ``-u`` or ``--unit=U`` can be used to specify the time +A new command line option ``-u`` or :samp:`--unit={U}` can be used to specify the time unit for the timer output. Supported options are ``usec``, ``msec``, or ``sec``. (Contributed by Julian Gindi in :issue:`18983`.) @@ -1868,7 +1868,7 @@ (Contributed by Robert Collins in :issue:`17911`.) New lightweight classes: :class:`~traceback.TracebackException`, -:class:`~traceback.StackSummary`, and :class:`traceback.FrameSummary`. +:class:`~traceback.StackSummary`, and :class:`~traceback.FrameSummary`. (Contributed by Robert Collins in :issue:`17911`.) Both :func:`~traceback.print_tb` and :func:`~traceback.print_stack` functions @@ -1978,8 +1978,8 @@ xmlrpc ------ -The :class:`client.ServerProxy ` class is now a -:term:`context manager`. +The :class:`client.ServerProxy ` class now supports +:term:`context manager` protocol. (Contributed by Claudiu Popa in :issue:`20627`.) :class:`client.ServerProxy ` constructor now accepts @@ -2252,7 +2252,7 @@ :func:`inspect.signature` API. (Contributed by Yury Selivanov in :issue:`20438`.) -Use of ``re.LOCALE`` flag with str patterns or ``re.ASCII`` is now +Use of :const:`re.LOCALE` flag with str patterns or :const:`re.ASCII` is now deprecated. (Contributed by Serhiy Storchaka in :issue:`22407`.) Use of unrecognized special sequences consisting of ``'\'`` and an ASCII letter -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 13 14:29:43 2015 From: python-checkins at python.org (yury.selivanov) Date: Sun, 13 Sep 2015 12:29:43 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogTWVyZ2UgMy41?= Message-ID: <20150913122943.14885.19551@psf.io> https://hg.python.org/cpython/rev/56a01af0a869 changeset: 97976:56a01af0a869 parent: 97974:76913f1eea0f parent: 97975:59f7007fbf3c user: Yury Selivanov date: Sun Sep 13 08:29:30 2015 -0400 summary: Merge 3.5 files: Doc/whatsnew/3.5.rst | 182 +++++++++++++++--------------- 1 files changed, 91 insertions(+), 91 deletions(-) diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -118,21 +118,21 @@ :ref:`support for Memory BIO `, which decouples SSL protocol handling from network IO. +* The new :func:`os.scandir` function provides a + :ref:`better and significantly faster way ` + of directory traversal. + +* :func:`functools.lru_cache` has been largely + :ref:`reimplemented in C `, yielding much better + performance. + +* The new :func:`subprocess.run` function provides a + :ref:`streamlined way to run subprocesses `. + * :mod:`traceback` module has been significantly :ref:`enhanced ` for improved performance and developer convenience. -* The new :func:`os.scandir` function provides a - :ref:`better and significantly faster way ` - of directory traversal. - -* :func:`functools.lru_cache` has been largely - :ref:`reimplemented in C `, yielding much better - performance. - -* The new :func:`subprocess.run` function provides a - :ref:`streamlined way to run subprocesses `. - Security improvements: @@ -806,13 +806,6 @@ protocol. (Contributed by Berker Peksag in :issue:`20289`.) -csv ---- - -The :meth:`~csv.csvwriter.writerow` method now supports arbitrary iterables, -not just sequences. (Contributed by Serhiy Storchaka in :issue:`23171`.) - - cmath ----- @@ -959,6 +952,13 @@ (Contributed by Berker Peksag in :issue:`22389`.) +csv +--- + +The :meth:`~csv.csvwriter.writerow` method now supports arbitrary iterables, +not just sequences. (Contributed by Serhiy Storchaka in :issue:`23171`.) + + curses ------ @@ -1636,6 +1636,29 @@ :issue:`18615`.) +socket +------ + +Functions with timeouts now use a monotonic clock, instead of a system clock. +(Contributed by Victor Stinner in :issue:`22043`.) + +A new :meth:`socket.sendfile ` method allows to +send a file over a socket by using the high-performance :func:`os.sendfile` +function on UNIX resulting in uploads being from 2 to 3 times faster than when +using plain :meth:`socket.send `. +(Contributed by Giampaolo Rodola' in :issue:`17552`.) + +The :meth:`socket.sendall ` method no longer resets the +socket timeout every time bytes are received or sent. The socket timeout is +now the maximum total duration to send all data. +(Contributed by Victor Stinner in :issue:`23853`.) + +The *backlog* argument of the :meth:`socket.listen ` +method is now optional. By default it is set to +:data:`SOMAXCONN ` or to ``128`` whichever is less. +(Contributed by Charles-Fran?ois Natali in :issue:`21455`.) + + ssl --- @@ -1717,29 +1740,6 @@ (Contributed by Antoine Pitrou in :issue:`23239`.) -socket ------- - -Functions with timeouts now use a monotonic clock, instead of a system clock. -(Contributed by Victor Stinner in :issue:`22043`.) - -A new :meth:`socket.sendfile ` method allows to -send a file over a socket by using the high-performance :func:`os.sendfile` -function on UNIX resulting in uploads being from 2 to 3 times faster than when -using plain :meth:`socket.send `. -(Contributed by Giampaolo Rodola' in :issue:`17552`.) - -The :meth:`socket.sendall ` method no longer resets the -socket timeout every time bytes are received or sent. The socket timeout is -now the maximum total duration to send all data. -(Contributed by Victor Stinner in :issue:`23853`.) - -The *backlog* argument of the :meth:`socket.listen ` -method is now optional. By default it is set to -:data:`SOMAXCONN ` or to ``128`` whichever is less. -(Contributed by Charles-Fran?ois Natali in :issue:`21455`.) - - sqlite3 ------- @@ -1890,6 +1890,56 @@ (Contributed by Yury Selivanov in :issue:`24400`.) +unicodedata +----------- + +The :mod:`unicodedata` module now uses data from `Unicode 8.0.0 +`_. + + +unittest +-------- + +The :meth:`TestLoader.loadTestsFromModule ` +method now accepts a keyword-only argument *pattern* which is passed to +``load_tests`` as the third argument. Found packages are now checked for +``load_tests`` regardless of whether their path matches *pattern*, because it +is impossible for a package name to match the default pattern. +(Contributed by Robert Collins and Barry A. Warsaw in :issue:`16662`.) + +Unittest discovery errors now are exposed in +:data:`TestLoader.errors ` attribute of the +:class:`~unittest.TestLoader` instance. +(Contributed by Robert Collins in :issue:`19746`.) + +A new command line option ``--locals`` to show local variables in +tracebacks. (Contributed by Robert Collins in :issue:`22936`.) + + +unittest.mock +------------- + +The :class:`~unittest.mock.Mock` has the following improvements: + +* Class constructor has a new *unsafe* parameter, which causes mock + objects to raise :exc:`AttributeError` on attribute names starting + with ``"assert"``. + (Contributed by Kushal Das in :issue:`21238`.) + +* A new :meth:`Mock.assert_not_called ` + method to check if the mock object was called. + (Contributed by Kushal Das in :issue:`21262`.) + +The :class:`~unittest.mock.MagicMock` class now supports :meth:`__truediv__`, +:meth:`__divmod__` and :meth:`__matmul__` operators. +(Contributed by Johannes Baiter in :issue:`20968`, and H?kan L?vdahl +in :issue:`23581` and :issue:`23568`.) + +It is no longer necessary to explicitly pass ``create=True`` to the +:func:`~unittest.mock.patch` function when patching builtin names. +(Contributed by Kushal Das in :issue:`17660`.) + + urllib ------ @@ -1917,56 +1967,6 @@ (Contributed by Demian Brecht and Senthil Kumaran in :issue:`22118`.) -unicodedata ------------ - -The :mod:`unicodedata` module now uses data from `Unicode 8.0.0 -`_. - - -unittest --------- - -The :meth:`TestLoader.loadTestsFromModule ` -method now accepts a keyword-only argument *pattern* which is passed to -``load_tests`` as the third argument. Found packages are now checked for -``load_tests`` regardless of whether their path matches *pattern*, because it -is impossible for a package name to match the default pattern. -(Contributed by Robert Collins and Barry A. Warsaw in :issue:`16662`.) - -Unittest discovery errors now are exposed in -:data:`TestLoader.errors ` attribute of the -:class:`~unittest.TestLoader` instance. -(Contributed by Robert Collins in :issue:`19746`.) - -A new command line option ``--locals`` to show local variables in -tracebacks. (Contributed by Robert Collins in :issue:`22936`.) - - -unittest.mock -------------- - -The :class:`~unittest.mock.Mock` has the following improvements: - -* Class constructor has a new *unsafe* parameter, which causes mock - objects to raise :exc:`AttributeError` on attribute names starting - with ``"assert"``. - (Contributed by Kushal Das in :issue:`21238`.) - -* A new :meth:`Mock.assert_not_called ` - method to check if the mock object was called. - (Contributed by Kushal Das in :issue:`21262`.) - -The :class:`~unittest.mock.MagicMock` class now supports :meth:`__truediv__`, -:meth:`__divmod__` and :meth:`__matmul__` operators. -(Contributed by Johannes Baiter in :issue:`20968`, and H?kan L?vdahl -in :issue:`23581` and :issue:`23568`.) - -It is no longer necessary to explicitly pass ``create=True`` to the -:func:`~unittest.mock.patch` function when patching builtin names. -(Contributed by Kushal Das in :issue:`17660`.) - - wsgiref ------- -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 13 14:29:43 2015 From: python-checkins at python.org (yury.selivanov) Date: Sun, 13 Sep 2015 12:29:43 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogd2hhdHNuZXcvMy41?= =?utf-8?q?=3A_Reorder_stuff_=28issue_=2325082=29=2E?= Message-ID: <20150913122943.17991.78093@psf.io> https://hg.python.org/cpython/rev/59f7007fbf3c changeset: 97975:59f7007fbf3c branch: 3.5 parent: 97973:334a988f744d user: Yury Selivanov date: Sun Sep 13 08:29:19 2015 -0400 summary: whatsnew/3.5: Reorder stuff (issue #25082). files: Doc/whatsnew/3.5.rst | 182 +++++++++++++++--------------- 1 files changed, 91 insertions(+), 91 deletions(-) diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -118,21 +118,21 @@ :ref:`support for Memory BIO `, which decouples SSL protocol handling from network IO. +* The new :func:`os.scandir` function provides a + :ref:`better and significantly faster way ` + of directory traversal. + +* :func:`functools.lru_cache` has been largely + :ref:`reimplemented in C `, yielding much better + performance. + +* The new :func:`subprocess.run` function provides a + :ref:`streamlined way to run subprocesses `. + * :mod:`traceback` module has been significantly :ref:`enhanced ` for improved performance and developer convenience. -* The new :func:`os.scandir` function provides a - :ref:`better and significantly faster way ` - of directory traversal. - -* :func:`functools.lru_cache` has been largely - :ref:`reimplemented in C `, yielding much better - performance. - -* The new :func:`subprocess.run` function provides a - :ref:`streamlined way to run subprocesses `. - Security improvements: @@ -806,13 +806,6 @@ protocol. (Contributed by Berker Peksag in :issue:`20289`.) -csv ---- - -The :meth:`~csv.csvwriter.writerow` method now supports arbitrary iterables, -not just sequences. (Contributed by Serhiy Storchaka in :issue:`23171`.) - - cmath ----- @@ -959,6 +952,13 @@ (Contributed by Berker Peksag in :issue:`22389`.) +csv +--- + +The :meth:`~csv.csvwriter.writerow` method now supports arbitrary iterables, +not just sequences. (Contributed by Serhiy Storchaka in :issue:`23171`.) + + curses ------ @@ -1636,6 +1636,29 @@ :issue:`18615`.) +socket +------ + +Functions with timeouts now use a monotonic clock, instead of a system clock. +(Contributed by Victor Stinner in :issue:`22043`.) + +A new :meth:`socket.sendfile ` method allows to +send a file over a socket by using the high-performance :func:`os.sendfile` +function on UNIX resulting in uploads being from 2 to 3 times faster than when +using plain :meth:`socket.send `. +(Contributed by Giampaolo Rodola' in :issue:`17552`.) + +The :meth:`socket.sendall ` method no longer resets the +socket timeout every time bytes are received or sent. The socket timeout is +now the maximum total duration to send all data. +(Contributed by Victor Stinner in :issue:`23853`.) + +The *backlog* argument of the :meth:`socket.listen ` +method is now optional. By default it is set to +:data:`SOMAXCONN ` or to ``128`` whichever is less. +(Contributed by Charles-Fran?ois Natali in :issue:`21455`.) + + ssl --- @@ -1717,29 +1740,6 @@ (Contributed by Antoine Pitrou in :issue:`23239`.) -socket ------- - -Functions with timeouts now use a monotonic clock, instead of a system clock. -(Contributed by Victor Stinner in :issue:`22043`.) - -A new :meth:`socket.sendfile ` method allows to -send a file over a socket by using the high-performance :func:`os.sendfile` -function on UNIX resulting in uploads being from 2 to 3 times faster than when -using plain :meth:`socket.send `. -(Contributed by Giampaolo Rodola' in :issue:`17552`.) - -The :meth:`socket.sendall ` method no longer resets the -socket timeout every time bytes are received or sent. The socket timeout is -now the maximum total duration to send all data. -(Contributed by Victor Stinner in :issue:`23853`.) - -The *backlog* argument of the :meth:`socket.listen ` -method is now optional. By default it is set to -:data:`SOMAXCONN ` or to ``128`` whichever is less. -(Contributed by Charles-Fran?ois Natali in :issue:`21455`.) - - sqlite3 ------- @@ -1890,6 +1890,56 @@ (Contributed by Yury Selivanov in :issue:`24400`.) +unicodedata +----------- + +The :mod:`unicodedata` module now uses data from `Unicode 8.0.0 +`_. + + +unittest +-------- + +The :meth:`TestLoader.loadTestsFromModule ` +method now accepts a keyword-only argument *pattern* which is passed to +``load_tests`` as the third argument. Found packages are now checked for +``load_tests`` regardless of whether their path matches *pattern*, because it +is impossible for a package name to match the default pattern. +(Contributed by Robert Collins and Barry A. Warsaw in :issue:`16662`.) + +Unittest discovery errors now are exposed in +:data:`TestLoader.errors ` attribute of the +:class:`~unittest.TestLoader` instance. +(Contributed by Robert Collins in :issue:`19746`.) + +A new command line option ``--locals`` to show local variables in +tracebacks. (Contributed by Robert Collins in :issue:`22936`.) + + +unittest.mock +------------- + +The :class:`~unittest.mock.Mock` has the following improvements: + +* Class constructor has a new *unsafe* parameter, which causes mock + objects to raise :exc:`AttributeError` on attribute names starting + with ``"assert"``. + (Contributed by Kushal Das in :issue:`21238`.) + +* A new :meth:`Mock.assert_not_called ` + method to check if the mock object was called. + (Contributed by Kushal Das in :issue:`21262`.) + +The :class:`~unittest.mock.MagicMock` class now supports :meth:`__truediv__`, +:meth:`__divmod__` and :meth:`__matmul__` operators. +(Contributed by Johannes Baiter in :issue:`20968`, and H?kan L?vdahl +in :issue:`23581` and :issue:`23568`.) + +It is no longer necessary to explicitly pass ``create=True`` to the +:func:`~unittest.mock.patch` function when patching builtin names. +(Contributed by Kushal Das in :issue:`17660`.) + + urllib ------ @@ -1917,56 +1967,6 @@ (Contributed by Demian Brecht and Senthil Kumaran in :issue:`22118`.) -unicodedata ------------ - -The :mod:`unicodedata` module now uses data from `Unicode 8.0.0 -`_. - - -unittest --------- - -The :meth:`TestLoader.loadTestsFromModule ` -method now accepts a keyword-only argument *pattern* which is passed to -``load_tests`` as the third argument. Found packages are now checked for -``load_tests`` regardless of whether their path matches *pattern*, because it -is impossible for a package name to match the default pattern. -(Contributed by Robert Collins and Barry A. Warsaw in :issue:`16662`.) - -Unittest discovery errors now are exposed in -:data:`TestLoader.errors ` attribute of the -:class:`~unittest.TestLoader` instance. -(Contributed by Robert Collins in :issue:`19746`.) - -A new command line option ``--locals`` to show local variables in -tracebacks. (Contributed by Robert Collins in :issue:`22936`.) - - -unittest.mock -------------- - -The :class:`~unittest.mock.Mock` has the following improvements: - -* Class constructor has a new *unsafe* parameter, which causes mock - objects to raise :exc:`AttributeError` on attribute names starting - with ``"assert"``. - (Contributed by Kushal Das in :issue:`21238`.) - -* A new :meth:`Mock.assert_not_called ` - method to check if the mock object was called. - (Contributed by Kushal Das in :issue:`21262`.) - -The :class:`~unittest.mock.MagicMock` class now supports :meth:`__truediv__`, -:meth:`__divmod__` and :meth:`__matmul__` operators. -(Contributed by Johannes Baiter in :issue:`20968`, and H?kan L?vdahl -in :issue:`23581` and :issue:`23568`.) - -It is no longer necessary to explicitly pass ``create=True`` to the -:func:`~unittest.mock.patch` function when patching builtin names. -(Contributed by Kushal Das in :issue:`17660`.) - - wsgiref ------- -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 13 14:31:18 2015 From: python-checkins at python.org (yury.selivanov) Date: Sun, 13 Sep 2015 12:31:18 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogTWVyZ2UgMy41?= Message-ID: <20150913123117.11256.29528@psf.io> https://hg.python.org/cpython/rev/c31b1b63a0de changeset: 97978:c31b1b63a0de parent: 97976:56a01af0a869 parent: 97977:aa288ad94089 user: Yury Selivanov date: Sun Sep 13 08:31:07 2015 -0400 summary: Merge 3.5 files: Doc/whatsnew/3.5.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -911,7 +911,7 @@ ------------ Config parsers can be customized by providing a dictionary of converters in the -constructor, or All converters defined in config parser (either by subclassing or +constructor. All converters defined in config parser (either by subclassing or by providing in a constructor) will be available on all section proxies. Example:: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 13 14:31:18 2015 From: python-checkins at python.org (yury.selivanov) Date: Sun, 13 Sep 2015 12:31:18 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogd2hhdHNuZXcvMy41?= =?utf-8?q?=3A_Fix_typo_=28issue_=2325082=29?= Message-ID: <20150913123117.12019.72730@psf.io> https://hg.python.org/cpython/rev/aa288ad94089 changeset: 97977:aa288ad94089 branch: 3.5 parent: 97975:59f7007fbf3c user: Yury Selivanov date: Sun Sep 13 08:30:58 2015 -0400 summary: whatsnew/3.5: Fix typo (issue #25082) files: Doc/whatsnew/3.5.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -911,7 +911,7 @@ ------------ Config parsers can be customized by providing a dictionary of converters in the -constructor, or All converters defined in config parser (either by subclassing or +constructor. All converters defined in config parser (either by subclassing or by providing in a constructor) will be available on all section proxies. Example:: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 13 16:17:47 2015 From: python-checkins at python.org (larry.hastings) Date: Sun, 13 Sep 2015 14:17:47 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_Minor_PEP_101_tweak=2E?= Message-ID: <20150913141747.14869.22650@psf.io> https://hg.python.org/peps/rev/a76e36a00590 changeset: 6057:a76e36a00590 user: Larry Hastings date: Sun Sep 13 15:17:21 2015 +0100 summary: Minor PEP 101 tweak. files: pep-0101.txt | 4 ++++ 1 files changed, 4 insertions(+), 0 deletions(-) diff --git a/pep-0101.txt b/pep-0101.txt --- a/pep-0101.txt +++ b/pep-0101.txt @@ -484,6 +484,10 @@ Note that the easiest thing is probably to copy fields from an existing Python release "page", editing as you go. + There should only be one "page" for a release (e.g. 3.5.0, 3.5.1). + Reuse the same page for all pre-releases, changing the version + number and the documentation as you go. + ___ If this isn't the first release for a version, open the existing "page" for editing and update it to the new release. Don't save yet! -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Sun Sep 13 16:46:04 2015 From: python-checkins at python.org (larry.hastings) Date: Sun, 13 Sep 2015 14:46:04 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_Added_tag_v3?= =?utf-8?q?=2E5=2E0_for_changeset_374f501f4567?= Message-ID: <20150913144604.17991.41088@psf.io> https://hg.python.org/cpython/rev/48ea62b61203 changeset: 97983:48ea62b61203 branch: 3.5 user: Larry Hastings date: Sat Sep 12 17:36:53 2015 +0100 summary: Added tag v3.5.0 for changeset 374f501f4567 files: .hgtags | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diff --git a/.hgtags b/.hgtags --- a/.hgtags +++ b/.hgtags @@ -156,3 +156,4 @@ cc15d736d860303b9da90d43cd32db39bab048df v3.5.0rc2 66ed52375df802f9d0a34480daaa8ce79fc41313 v3.5.0rc3 2d033fedfa7f1e325fd14ccdaa9cb42155da206f v3.5.0rc4 +374f501f4567b7595f2ad7798aa09afa2456bb28 v3.5.0 -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 13 16:46:04 2015 From: python-checkins at python.org (larry.hastings) Date: Sun, 13 Sep 2015 14:46:04 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_Post-release_u?= =?utf-8?q?pdates_for_Python_3=2E5=2E0=2E?= Message-ID: <20150913144604.17985.3985@psf.io> https://hg.python.org/cpython/rev/5a30d334fffc changeset: 97984:5a30d334fffc branch: 3.5 user: Larry Hastings date: Sun Sep 13 15:36:07 2015 +0100 summary: Post-release updates for Python 3.5.0. files: Include/patchlevel.h | 2 +- Misc/NEWS | 12 ++++++++++++ 2 files changed, 13 insertions(+), 1 deletions(-) diff --git a/Include/patchlevel.h b/Include/patchlevel.h --- a/Include/patchlevel.h +++ b/Include/patchlevel.h @@ -23,7 +23,7 @@ #define PY_RELEASE_SERIAL 0 /* Version as a string */ -#define PY_VERSION "3.5.0" +#define PY_VERSION "3.5.0+" /*--end constants--*/ /* Version as a single 4-byte hex number, e.g. 0x010502B2 == 1.5.2b2. diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -2,6 +2,18 @@ Python News +++++++++++ +What's New in Python 3.5.1 release candidate 1? +=============================================== + +Release date: TBA + +Core and Builtins +----------------- + +Library +------- + + What's New in Python 3.5.0 final? ================================= -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 13 16:46:05 2015 From: python-checkins at python.org (larry.hastings) Date: Sun, 13 Sep 2015 14:46:05 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_Version_bump_f?= =?utf-8?q?or_Python_3=2E5=2E0_final=2E?= Message-ID: <20150913144604.12017.815@psf.io> https://hg.python.org/cpython/rev/af05eb9d4ad0 changeset: 97981:af05eb9d4ad0 branch: 3.5 user: Larry Hastings date: Sat Sep 12 17:28:39 2015 +0100 summary: Version bump for Python 3.5.0 final. files: Include/patchlevel.h | 6 +- Misc/NEWS | 3 +- README | 72 ++++++++++++++++--------------- 3 files changed, 42 insertions(+), 39 deletions(-) diff --git a/Include/patchlevel.h b/Include/patchlevel.h --- a/Include/patchlevel.h +++ b/Include/patchlevel.h @@ -19,11 +19,11 @@ #define PY_MAJOR_VERSION 3 #define PY_MINOR_VERSION 5 #define PY_MICRO_VERSION 0 -#define PY_RELEASE_LEVEL PY_RELEASE_LEVEL_GAMMA -#define PY_RELEASE_SERIAL 4 +#define PY_RELEASE_LEVEL PY_RELEASE_LEVEL_FINAL +#define PY_RELEASE_SERIAL 0 /* Version as a string */ -#define PY_VERSION "3.5.0rc4+" +#define PY_VERSION "3.5.0" /*--end constants--*/ /* Version as a single 4-byte hex number, e.g. 0x010502B2 == 1.5.2b2. diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -11,7 +11,8 @@ ----- - Issue #25071: Windows installer should not require TargetDir - parameter when installing quietly + parameter when installing quietly. + What's New in Python 3.5.0 release candidate 4? =============================================== diff --git a/README b/README --- a/README +++ b/README @@ -1,13 +1,14 @@ -This is Python version 3.5.0 release candidate 4 -================================================ +This is Python version 3.5.0 +============================ Copyright (c) 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014, 2015 Python Software Foundation. All rights reserved. -Python 3.x is a new version of the language, which is incompatible with the 2.x -line of releases. The language is mostly the same, but many details, especially -how built-in objects like dictionaries and strings work, have changed -considerably, and a lot of deprecated features have finally been removed. +Python 3.x is a new version of the language, which is incompatible with the +2.x line of releases. The language is mostly the same, but many details, +especially how built-in objects like dictionaries and strings work, +have changed considerably, and a lot of deprecated features have finally +been removed. Build Instructions @@ -27,14 +28,14 @@ elsewhere it's just python. On Mac OS X, if you have configured Python with --enable-framework, you should -use "make frameworkinstall" to do the installation. Note that this installs the -Python executable in a place that is not normally on your PATH, you may want to -set up a symlink in /usr/local/bin. +use "make frameworkinstall" to do the installation. Note that this installs +the Python executable in a place that is not normally on your PATH, you may +want to set up a symlink in /usr/local/bin. On Windows, see PCbuild/readme.txt. -If you wish, you can create a subdirectory and invoke configure from there. For -example: +If you wish, you can create a subdirectory and invoke configure from there. +For example: mkdir debug cd debug @@ -42,21 +43,21 @@ make make test -(This will fail if you *also* built at the top-level directory. You should do a -"make clean" at the toplevel first.) +(This will fail if you *also* built at the top-level directory. +You should do a "make clean" at the toplevel first.) What's New ---------- -We try to have a comprehensive overview of the changes in the "What's New in +We have a comprehensive overview of the changes in the "What's New in Python 3.5" document, found at http://docs.python.org/3.5/whatsnew/3.5.html -For a more detailed change log, read Misc/NEWS (though this file, too, is -incomplete, and also doesn't list anything merged in from the 2.7 release under -development). +For a more detailed change log, read Misc/NEWS (though this file, too, +is incomplete, and also doesn't list anything merged in from the 2.7 +release under development). If you want to install multiple versions of Python see the section below entitled "Installing multiple versions". @@ -98,10 +99,11 @@ Testing ------- -To test the interpreter, type "make test" in the top-level directory. The test -set produces some output. You can generally ignore the messages about skipped -tests due to optional features which can't be imported. If a message is printed -about a failed test or a traceback or core dump is produced, something is wrong. +To test the interpreter, type "make test" in the top-level directory. +The test set produces some output. You can generally ignore the messages +about skipped tests due to optional features which can't be imported. +If a message is printed about a failed test or a traceback or core dump +is produced, something is wrong. By default, tests are prevented from overusing resources like disk space and memory. To enable these tests, run "make testall". @@ -121,14 +123,14 @@ On Unix and Mac systems if you intend to install multiple versions of Python using the same installation prefix (--prefix argument to the configure script) -you must take care that your primary python executable is not overwritten by the -installation of a different version. All files and directories installed using -"make altinstall" contain the major and minor version and can thus live -side-by-side. "make install" also creates ${prefix}/bin/python3 which refers to -${prefix}/bin/pythonX.Y. If you intend to install multiple versions using the -same prefix you must decide which version (if any) is your "primary" version. -Install that version using "make install". Install all other versions using -"make altinstall". +you must take care that your primary python executable is not overwritten by +the installation of a different version. All files and directories installed +using "make altinstall" contain the major and minor version and can thus live +side-by-side. "make install" also creates ${prefix}/bin/python3 which refers +to ${prefix}/bin/pythonX.Y. If you intend to install multiple versions using +the same prefix you must decide which version (if any) is your "primary" +version. Install that version using "make install". Install all other +versions using "make altinstall". For example, if you want to install Python 2.6, 2.7 and 3.5 with 2.7 being the primary version, you would execute "make install" in your 2.7 build directory @@ -139,7 +141,7 @@ ------------------------------ We're soliciting bug reports about all aspects of the language. Fixes are also -welcome, preferable in unified diff format. Please use the issue tracker: +welcome, preferably in unified diff format. Please use the issue tracker: http://bugs.python.org/ @@ -182,11 +184,11 @@ Copyright (c) 1991-1995 Stichting Mathematisch Centrum. All rights reserved. -See the file "LICENSE" for information on the history of this software, terms & -conditions for usage, and a DISCLAIMER OF ALL WARRANTIES. +See the file "LICENSE" for information on the history of this software, +terms & conditions for usage, and a DISCLAIMER OF ALL WARRANTIES. -This Python distribution contains *no* GNU General Public License (GPL) code, so -it may be used in proprietary projects. There are interfaces to some GNU code -but these are entirely optional. +This Python distribution contains *no* GNU General Public License (GPL) code, +so it may be used in proprietary projects. There are interfaces to some GNU +code but these are entirely optional. All trademarks referenced herein are property of their respective holders. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 13 16:46:04 2015 From: python-checkins at python.org (larry.hastings) Date: Sun, 13 Sep 2015 14:46:04 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_Backported_the?= =?utf-8?q?_What=27s_New_In_3=2E5_from_3=2E5=2E1_to_3=2E5=2E0_=28final!=29?= =?utf-8?q?=2E?= Message-ID: <20150913144603.14855.42749@psf.io> https://hg.python.org/cpython/rev/955911b49328 changeset: 97979:955911b49328 branch: 3.5 parent: 97923:da8f2767b6cc user: Larry Hastings date: Sat Sep 12 17:12:36 2015 +0100 summary: Backported the What's New In 3.5 from 3.5.1 to 3.5.0 (final!). files: Doc/whatsnew/3.5.rst | 1937 ++++++++++++++++++++--------- 1 files changed, 1345 insertions(+), 592 deletions(-) diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -4,6 +4,7 @@ :Release: |release| :Date: |today| +:Editors: Elvis Pranskevichus , Yury Selivanov .. Rules for maintenance: @@ -46,7 +47,6 @@ when researching a change. This article explains the new features in Python 3.5, compared to 3.4. - For full details, see the :source:`Misc/NEWS` file. .. note:: @@ -69,8 +69,8 @@ New syntax features: +* :pep:`492`, coroutines with async and await syntax. * :pep:`465`, a new matrix multiplication operator: ``a @ b``. -* :pep:`492`, coroutines with async and await syntax. * :pep:`448`, additional unpacking generalizations. New library modules: @@ -81,104 +81,286 @@ New built-in features: * ``bytes % args``, ``bytearray % args``: :pep:`461` - Adding ``%`` formatting - to bytes and bytearray + to bytes and bytearray. + * ``b'\xf0\x9f\x90\x8d'.hex()``, ``bytearray(b'\xf0\x9f\x90\x8d').hex()``, ``memoryview(b'\xf0\x9f\x90\x8d').hex()``: :issue:`9951` - A ``hex`` method has been added to bytes, bytearray, and memoryview. + +* :class:`memoryview` (including multi-dimensional) now supports tuple indexing. + (Contributed by Antoine Pitrou in :issue:`23632`.) + * Generators have new ``gi_yieldfrom`` attribute, which returns the object being iterated by ``yield from`` expressions. (Contributed by Benno Leslie and Yury Selivanov in :issue:`24450`.) -* New :exc:`RecursionError` exception. (Contributed by Georg Brandl + +* New :exc:`RecursionError` exception. (Contributed by Georg Brandl in :issue:`19235`.) -Implementation improvements: +* New :exc:`StopAsyncIteration` exception. (Contributed by + Yury Selivanov in :issue:`24017`. See also :pep:`492`.) + +CPython implementation improvements: * When the ``LC_TYPE`` locale is the POSIX locale (``C`` locale), :py:data:`sys.stdin` and :py:data:`sys.stdout` are now using the - ``surrogateescape`` error handler, instead of the ``strict`` error handler - (:issue:`19977`). - -* :pep:`488`, the elimination of ``.pyo`` files. -* :pep:`489`, multi-phase initialization of extension modules. + ``surrogateescape`` error handler, instead of the ``strict`` error handler. + (Contributed by Victor Stinner in :issue:`19977`.) + +* ``.pyo`` files are no longer used and have been replaced by a more flexible + scheme that inclides the optimization level explicitly in ``.pyc`` name. + (:pep:`488`) + +* Builtin and extension modules are now initialized in a multi-phase process, + which is similar to how Python modules are loaded. (:pep:`489`). Significantly Improved Library Modules: -* :class:`collections.OrderedDict` is now implemented in C, which improves - its performance between 4x to 100x times. Contributed by Eric Snow in - :issue:`16991`. +* :class:`collections.OrderedDict` is now implemented in C, which makes it + 4 to 100 times faster. (Contributed by Eric Snow in :issue:`16991`.) * You may now pass bytes to the :mod:`tempfile` module's APIs and it will - return the temporary pathname as bytes instead of str. It also accepts - a value of ``None`` on parameters where only str was accepted in the past to - do the right thing based on the types of the other inputs. Two functions, - :func:`gettempdirb` and :func:`gettempprefixb`, have been added to go along - with this. This behavior matches that of the :mod:`os` APIs. + return the temporary pathname as :class:`bytes` instead of :class:`str`. + It also accepts a value of ``None`` on parameters where only str was + accepted in the past to do the right thing based on the types of the + other inputs. Two functions, :func:`gettempdirb` and + :func:`gettempprefixb`, have been added to go along with this. + This behavior matches that of the :mod:`os` APIs. + (Contributed by Gregory P. Smith in :issue:`24230`.) * :mod:`ssl` module gained support for Memory BIO, which decouples SSL protocol handling from network IO. (Contributed by Geert Jansen in :issue:`21965`.) +* :mod:`traceback` has new lightweight and convenient to work with + classes :class:`~traceback.TracebackException`, + :class:`~traceback.StackSummary`, and :class:`~traceback.FrameSummary`. + (Contributed by Robert Collins in :issue:`17911`.) + +* Most of :func:`functools.lru_cache` machinery is now implemented in C. + (Contributed by Matt Joiner, Alexey Kachayev, and Serhiy Storchaka + in :issue:`14373`.) + Security improvements: -* None yet. +* SSLv3 is now disabled throughout the standard library. + It can still be enabled by instantiating a :class:`ssl.SSLContext` + manually. (See :issue:`22638` for more details; this change was + backported to CPython 3.4 and 2.7.) + +* HTTP cookie parsing is now stricter, in order to protect + against potential injection attacks. (Contributed by Antoine Pitrou + in :issue:`22796`.) Windows improvements: -* A new installer for Windows has replaced the old MSI. See :ref:`using-on-windows` - for more information. +* A new installer for Windows has replaced the old MSI. + See :ref:`using-on-windows` for more information. + * Windows builds now use Microsoft Visual C++ 14.0, and extension modules should use the same. - -Please read on for a comprehensive list of user-facing changes. - - -.. PEP-sized items next. - -.. _pep-4XX: - -.. PEP 4XX: Virtual Environments -.. ============================= - - -.. (Implemented by Foo Bar.) - -.. .. seealso:: - - :pep:`4XX` - Python Virtual Environments - PEP written by Carl Meyer - +Please read on for a comprehensive list of user-facing changes, including many +other smaller improvements, CPython optimizations, deprecations, and potential +porting issues. + + +New Features +============ + +.. _whatsnew-pep-492: PEP 492 - Coroutines with async and await syntax ------------------------------------------------ -The PEP added dedicated syntax for declaring :term:`coroutines `, -:keyword:`await` expressions, new asynchronous :keyword:`async for` -and :keyword:`async with` statements. - -Example:: - - async def read_data(db): - async with db.transaction(): - data = await db.fetch('SELECT ...') - -PEP written and implemented by Yury Selivanov. +:pep:`492` greatly improves support for asynchronous programming in Python +by adding :term:`awaitable objects `, +:term:`coroutine functions `, +:term:`asynchronous iteration `, +and :term:`asynchronous context managers `. + +Coroutine functions are declared using the new :keyword:`async def` syntax:: + + >>> async def coro(): + ... return 'spam' + +Inside a coroutine function, the new :keyword:`await` expression can be used +to suspend coroutine execution until the result is available. Any object +can be *awaited*, as long as it implements the :term:`awaitable` protocol by +defining the :meth:`__await__` method. + +PEP 492 also adds :keyword:`async for` statement for convenient iteration +over asynchronous iterables. + +An example of a simple HTTP client written using the new syntax:: + + import asyncio + + async def http_get(domain): + reader, writer = await asyncio.open_connection(domain, 80) + + writer.write(b'\r\n'.join([ + b'GET / HTTP/1.1', + b'Host: %b' % domain.encode('latin-1'), + b'Connection: close', + b'', b'' + ])) + + async for line in reader: + print('>>>', line) + + writer.close() + + loop = asyncio.get_event_loop() + try: + loop.run_until_complete(http_get('example.com')) + finally: + loop.close() + + +Similarly to asynchronous iteration, there is a new syntax for asynchronous +context managers. The following script:: + + import asyncio + + async def coro(name, lock): + print('coro {}: waiting for lock'.format(name)) + async with lock: + print('coro {}: holding the lock'.format(name)) + await asyncio.sleep(1) + print('coro {}: releasing the lock'.format(name)) + + loop = asyncio.get_event_loop() + lock = asyncio.Lock() + coros = asyncio.gather(coro(1, lock), coro(2, lock)) + try: + loop.run_until_complete(coros) + finally: + loop.close() + +will print:: + + coro 2: waiting for lock + coro 2: holding the lock + coro 1: waiting for lock + coro 2: releasing the lock + coro 1: holding the lock + coro 1: releasing the lock + +Note that both :keyword:`async for` and :keyword:`async with` can only +be used inside a coroutine function declared with :keyword:`async def`. + +Coroutine functions are intended to be run inside a compatible event loop, +such as :class:`asyncio.Loop`. .. seealso:: :pep:`492` -- Coroutines with async and await syntax - - -PEP 461 - Formatting support for bytes and bytearray ----------------------------------------------------- - -This PEP proposes adding % formatting operations similar to Python 2's ``str`` -type to :class:`bytes` and :class:`bytearray`. + PEP written and implemented by Yury Selivanov. + + +.. _whatsnew-pep-465: + +PEP 465 - A dedicated infix operator for matrix multiplication +-------------------------------------------------------------- + +:pep:`465` adds the ``@`` infix operator for matrix multiplication. +Currently, no builtin Python types implement the new operator, however, it +can be implemented by defining :meth:`__matmul__`, :meth:`__rmatmul__`, +and :meth:`__imatmul__` for regular, reflected, and in-place matrix +multiplication. The semantics of these methods is similar to that of +methods defining other infix arithmetic operators. + +Matrix multiplication is a notably common operation in many fields of +mathematics, science, engineering, and the addition of ``@`` allows writing +cleaner code:: + + S = (H @ beta - r).T @ inv(H @ V @ H.T) @ (H @ beta - r) + +instead of:: + + S = dot((dot(H, beta) - r).T, + dot(inv(dot(dot(H, V), H.T)), dot(H, beta) - r)) + +An upcoming release of NumPy 1.10 will add support for the new operator:: + + >>> import numpy + + >>> x = numpy.ones(3) + >>> x + array([ 1., 1., 1.]) + + >>> m = numpy.eye(3) + >>> m + array([[ 1., 0., 0.], + [ 0., 1., 0.], + [ 0., 0., 1.]]) + + >>> x @ m + array([ 1., 1., 1.]) + + +.. seealso:: + + :pep:`465` -- A dedicated infix operator for matrix multiplication + PEP written by Nathaniel J. Smith; implemented by Benjamin Peterson. + + +.. _whatsnew-pep-448: + +PEP 448 - Additional Unpacking Generalizations +---------------------------------------------- + +:pep:`448` extends the allowed uses of the ``*`` iterable unpacking +operator and ``**`` dictionary unpacking operator. It is now possible +to use an arbitrary number of unpackings in function calls:: + + >>> print(*[1], *[2], 3, *[4, 5]) + 1 2 3 4 5 + + >>> def fn(a, b, c, d): + ... print(a, b, c, d) + ... + + >>> fn(**{'a': 1, 'c': 3}, **{'b': 2, 'd': 4}) + 1 2 3 4 + +Similarly, tuple, list, set, and dictionary displays allow multiple +unpackings:: + + >>> *range(4), 4 + (0, 1, 2, 3, 4) + + >>> [*range(4), 4] + [0, 1, 2, 3, 4] + + >>> {*range(4), 4, *(5, 6, 7)} + {0, 1, 2, 3, 4, 5, 6, 7} + + >>> {'x': 1, **{'y': 2}} + {'x': 1, 'y': 2} + +.. seealso:: + + :pep:`448` -- Additional Unpacking Generalizations + PEP written by Joshua Landau; implemented by Neil Girdhar, + Thomas Wouters, and Joshua Landau. + + +.. _whatsnew-pep-461: + +PEP 461 - % formatting support for bytes and bytearray +------------------------------------------------------ + +PEP 461 adds % formatting to :class:`bytes` and :class:`bytearray`, aiding in +handling data that is a mixture of binary and ASCII compatible text. This +feature also eases porting such code from Python 2. Examples:: >>> b'Hello %s!' % b'World' b'Hello World!' + >>> b'x=%i y=%f' % (1, 2.5) b'x=1 y=2.500000' @@ -189,67 +371,18 @@ Traceback (most recent call last): File "", line 1, in TypeError: %b requires bytes, or an object that implements __bytes__, not 'str' + >>> b'price: %a' % '10?' b"price: '10\\u20ac'" .. seealso:: :pep:`461` -- Adding % formatting to bytes and bytearray - - -PEP 465 - A dedicated infix operator for matrix multiplication --------------------------------------------------------------- - -This PEP proposes a new binary operator to be used for matrix multiplication, -called ``@``. (Mnemonic: ``@`` is ``*`` for mATrices.) - -.. seealso:: - - :pep:`465` -- A dedicated infix operator for matrix multiplication - - -PEP 448 - Additional Unpacking Generalizations ----------------------------------------------- - -This PEP proposes extended usages of the ``*`` iterable unpacking -operator and ``**`` dictionary unpacking operators -to allow unpacking in more positions, an arbitrary number of -times, and in additional circumstances. Specifically, -in function calls, in comprehensions and generator expressions, and -in displays. - -Function calls are proposed to support an arbitrary number of -unpackings rather than just one:: - - >>> print(*[1], *[2], 3) - 1 2 3 - >>> dict(**{'x': 1}, y=2, **{'z': 3}) - {'x': 1, 'y': 2, 'z': 3} - -Unpacking is proposed to be allowed inside tuple, list, set, -and dictionary displays:: - - >>> *range(4), 4 - (0, 1, 2, 3, 4) - >>> [*range(4), 4] - [0, 1, 2, 3, 4] - >>> {*range(4), 4} - {0, 1, 2, 3, 4} - >>> {'x': 1, **{'y': 2}} - {'x': 1, 'y': 2} - -In dictionaries, later values will always override earlier ones:: - - >>> {'x': 1, **{'x': 2}} - {'x': 2} - - >>> {**{'x': 2}, 'x': 1} - {'x': 1} - -.. seealso:: - - :pep:`448` -- Additional Unpacking Generalizations - + PEP written by Ethan Furman; implemented by Neil Schemenauer and + Ethan Furman. + + +.. _whatsnew-pep-484: PEP 484 - Type Hints -------------------- @@ -261,8 +394,8 @@ For example, here is a simple function whose argument and return type are declared in the annotations:: - def greeting(name: str) -> str: - return 'Hello ' + name + def greeting(name: str) -> str: + return 'Hello ' + name The type system supports unions, generic types, and a special type named ``Any`` which is consistent with (i.e. assignable to and from) all @@ -270,25 +403,29 @@ .. seealso:: + * :mod:`typing` module documentation * :pep:`484` -- Type Hints - * :mod:`typing` module documentation - + PEP written by Guido van Rossum, Jukka Lehtosalo, and ?ukasz Langa; + implemented by Guido van Rossum. + + +.. _whatsnew-pep-471: PEP 471 - os.scandir() function -- a better and faster directory iterator ------------------------------------------------------------------------- :pep:`471` adds a new directory iteration function, :func:`os.scandir`, -to the standard library. Additionally, :func:`os.walk` is now +to the standard library. Additionally, :func:`os.walk` is now implemented using :func:`os.scandir`, which speeds it up by 3-5 times on POSIX systems and by 7-20 times on Windows systems. -PEP and implementation written by Ben Hoyt with the help of Victor Stinner. - .. seealso:: - :pep:`471` -- os.scandir() function -- a better and faster directory - iterator - + :pep:`471` -- os.scandir() function -- a better and faster directory iterator + PEP written and implemented by Ben Hoyt with the help of Victor Stinner. + + +.. _whatsnew-pep-475: PEP 475: Retry system calls failing with EINTR ---------------------------------------------- @@ -302,68 +439,47 @@ instead of raising :exc:`InterruptedError` if the Python signal handler does not raise an exception: -* :func:`open`, :func:`os.open`, :func:`io.open` -* functions of the :mod:`faulthandler` module -* :mod:`os` functions: - - - :func:`os.fchdir` - - :func:`os.fchmod` - - :func:`os.fchown` - - :func:`os.fdatasync` - - :func:`os.fstat` - - :func:`os.fstatvfs` - - :func:`os.fsync` - - :func:`os.ftruncate` - - :func:`os.mkfifo` - - :func:`os.mknod` - - :func:`os.posix_fadvise` - - :func:`os.posix_fallocate` - - :func:`os.pread` - - :func:`os.pwrite` - - :func:`os.read` - - :func:`os.readv` - - :func:`os.sendfile` - - :func:`os.wait3` - - :func:`os.wait4` - - :func:`os.wait` - - :func:`os.waitid` - - :func:`os.waitpid` - - :func:`os.write` - - :func:`os.writev` - - special cases: :func:`os.close` and :func:`os.dup2` now ignore - :py:data:`~errno.EINTR` error, the syscall is not retried (see the PEP - for the rationale) - -* :mod:`select` functions: - - - :func:`select.devpoll.poll` - - :func:`select.epoll.poll` - - :func:`select.kqueue.control` - - :func:`select.poll.poll` - - :func:`select.select` - -* :func:`socket.socket` methods: - - - :meth:`~socket.socket.accept` - - :meth:`~socket.socket.connect` (except for non-blocking sockets) - - :meth:`~socket.socket.recv` - - :meth:`~socket.socket.recvfrom` - - :meth:`~socket.socket.recvmsg` - - :meth:`~socket.socket.send` - - :meth:`~socket.socket.sendall` - - :meth:`~socket.socket.sendmsg` - - :meth:`~socket.socket.sendto` - -* :func:`signal.sigtimedwait`, :func:`signal.sigwaitinfo` -* :func:`time.sleep` - -PEP and implementation written by Charles-Fran?ois Natali and Victor Stinner, -with the help of Antoine Pitrou (the french connection). +* :func:`open`, :func:`os.open`, :func:`io.open`; + +* functions of the :mod:`faulthandler` module; + +* :mod:`os` functions: :func:`~os.fchdir`, :func:`~os.fchmod`, + :func:`~os.fchown`, :func:`~os.fdatasync`, :func:`~os.fstat`, + :func:`~os.fstatvfs`, :func:`~os.fsync`, :func:`~os.ftruncate`, + :func:`~os.mkfifo`, :func:`~os.mknod`, :func:`~os.posix_fadvise`, + :func:`~os.posix_fallocate`, :func:`~os.pread`, :func:`~os.pwrite`, + :func:`~os.read`, :func:`~os.readv`, :func:`~os.sendfile`, + :func:`~os.wait3`, :func:`~os.wait4`, :func:`~os.wait`, + :func:`~os.waitid`, :func:`~os.waitpid`, :func:`~os.write`, + :func:`~os.writev`; + +* special cases: :func:`os.close` and :func:`os.dup2` now ignore + :py:data:`~errno.EINTR` error, the syscall is not retried (see the PEP + for the rationale); + +* :mod:`select` functions: :func:`~select.devpoll.poll`, + :func:`~select.epoll.poll`, :func:`~select.kqueue.control`, + :func:`~select.poll.poll`, :func:`~select.select`; + +* :func:`socket.socket` methods: :meth:`~socket.socket.accept`, + :meth:`~socket.socket.connect` (except for non-blocking sockets), + :meth:`~socket.socket.recv`, :meth:`~socket.socket.recvfrom`, + :meth:`~socket.socket.recvmsg`, :meth:`~socket.socket.send`, + :meth:`~socket.socket.sendall`, :meth:`~socket.socket.sendmsg`, + :meth:`~socket.socket.sendto`; + +* :func:`signal.sigtimedwait`, :func:`signal.sigwaitinfo`; + +* :func:`time.sleep`. .. seealso:: :pep:`475` -- Retry system calls failing with EINTR - + PEP and implementation written by Charles-Fran?ois Natali and + Victor Stinner, with the help of Antoine Pitrou (the french connection). + + +.. _whatsnew-pep-479: PEP 479: Change StopIteration handling inside generators -------------------------------------------------------- @@ -378,13 +494,14 @@ Without a ``__future__`` import, a :exc:`PendingDeprecationWarning` will be raised. -PEP written by Chris Angelico and Guido van Rossum. Implemented by -Chris Angelico, Yury Selivanov and Nick Coghlan. - .. seealso:: :pep:`479` -- Change StopIteration handling inside generators - + PEP written by Chris Angelico and Guido van Rossum. Implemented by + Chris Angelico, Yury Selivanov and Nick Coghlan. + + +.. _whatsnew-pep-486: PEP 486: Make the Python Launcher aware of virtual environments --------------------------------------------------------------- @@ -397,7 +514,10 @@ .. seealso:: :pep:`486` -- Make the Python Launcher aware of virtual environments - + PEP written and implemented by Paul Moore. + + +.. _whatsnew-pep-488: PEP 488: Elimination of PYO files --------------------------------- @@ -407,14 +527,18 @@ need to constantly regenerate bytecode files, ``.pyc`` files now have an optional ``opt-`` tag in their name when the bytecode is optimized. This has the side-effect of no more bytecode file name clashes when running under either -``-O`` or ``-OO``. Consequently, bytecode files generated from ``-O``, and -``-OO`` may now exist simultaneously. :func:`importlib.util.cache_from_source` -has an updated API to help with this change. +:option:`-O` or :option:`-OO`. Consequently, bytecode files generated from +:option:`-O`, and :option:`-OO` may now exist simultaneously. +:func:`importlib.util.cache_from_source` has an updated API to help with +this change. .. seealso:: :pep:`488` -- Elimination of PYO files - + PEP written and implemented by Brett Cannon. + + +.. _whatsnew-pep-489: PEP 489: Multi-phase extension module initialization ---------------------------------------------------- @@ -429,7 +553,12 @@ .. seealso:: - :pep:`488` -- Multi-phase extension module initialization + :pep:`489` -- Multi-phase extension module initialization + PEP written by Petr Viktorin, Stefan Behnel, and Nick Coghlan; + implemented by Petr Viktorin. + + +.. _whatsnew-pep-485: PEP 485: A function for testing approximate equality ---------------------------------------------------- @@ -442,18 +571,21 @@ .. seealso:: :pep:`485` -- A function for testing approximate equality + PEP written by Christopher Barker; implemented by Chris Barker and + Tal Einat. + Other Language Changes ====================== Some smaller changes made to the core Python language are: -* Added the ``'namereplace'`` error handlers. The ``'backslashreplace'`` +* Added the ``"namereplace"`` error handlers. The ``"backslashreplace"`` error handlers now works with decoding and translating. (Contributed by Serhiy Storchaka in :issue:`19676` and :issue:`22286`.) * The :option:`-b` option now affects comparisons of :class:`bytes` with - :class:`int`. (Contributed by Serhiy Storchaka in :issue:`23681`) + :class:`int`. (Contributed by Serhiy Storchaka in :issue:`23681`.) * New Kazakh :ref:`codec ` ``kz1048``. (Contributed by Serhiy Storchaka in :issue:`22682`.) @@ -465,6 +597,9 @@ * New Tajik :ref:`codec ` ``koi8_t``. (Contributed by Serhiy Storchaka in :issue:`22681`.) +* Circular imports involving relative imports are now supported. + (Contributed by Brett Cannon and Antoine Pitrou in :issue:`17636`.) + New Modules =========== @@ -476,8 +611,8 @@ The new :mod:`zipapp` module (specified in :pep:`441`) provides an API and command line tool for creating executable Python Zip Applications, which -were introduced in Python 2.6 in :issue:`1739468` but which were not well -publicised, either at the time or since. +were introduced in Python 2.6 in :issue:`1739468`, but which were not well +publicized, either at the time or since. With the new module, bundling your application is as simple as putting all the files, including a ``__main__.py`` file, into a directory ``myapp`` @@ -486,6 +621,13 @@ $ python -m zipapp myapp $ python myapp.pyz +The module implementation has been contributed by Paul Moore in +:issue:`23491`. + +.. seealso:: + + :pep:`441` -- Improving Python ZIP Application Support + Improved Modules ================ @@ -493,529 +635,1135 @@ argparse -------- -* :class:`~argparse.ArgumentParser` now allows to disable - :ref:`abbreviated usage ` of long options by setting - :ref:`allow_abbrev` to ``False``. - (Contributed by Jonathan Paugh, Steven Bethard, paul j3 and Daniel Eriksson.) +The :class:`~argparse.ArgumentParser` class now allows to disable +:ref:`abbreviated usage ` of long options by setting +:ref:`allow_abbrev` to ``False``. (Contributed by Jonathan Paugh, +Steven Bethard, paul j3 and Daniel Eriksson in :issue:`14910`.) + + +bz2 +--- + +The :meth:`BZ2Decompressor.decompress ` +method now accepts an optional *max_length* argument to limit the maximum +size of decompressed data. (Contributed by Nikolaus Rath in :issue:`15955`.) + cgi --- -* :class:`~cgi.FieldStorage` now supports the context management protocol. - (Contributed by Berker Peksag in :issue:`20289`.) +The :class:`~cgi.FieldStorage` class now supports the context management +protocol. (Contributed by Berker Peksag in :issue:`20289`.) + cmath ----- -* :func:`cmath.isclose` function added. - (Contributed by Chris Barker and Tal Einat in :issue:`24270`.) +A new function :func:`~cmath.isclose` provides a way to test for approximate +equality. (Contributed by Chris Barker and Tal Einat in :issue:`24270`.) code ---- -* The :func:`code.InteractiveInterpreter.showtraceback` method now prints - the full chained traceback, just like the interactive interpreter. - (Contributed by Claudiu Popa in :issue:`17442`.) +The :func:`InteractiveInterpreter.showtraceback ` +method now prints the full chained traceback, just like the interactive +interpreter. (Contributed by Claudiu Popa in :issue:`17442`.) + collections ----------- -* You can now update docstrings produced by :func:`collections.namedtuple`:: +The :class:`~collections.OrderedDict` class is now implemented in C, which +makes it 4 to 100 times faster. (Contributed by Eric Snow in :issue:`16991`.) + +:meth:`OrderedDict.items `, +:meth:`OrderedDict.keys `, +:meth:`OrderedDict.values ` views now support +:func:`reversed` iteration. +(Contributed by Serhiy Storchaka in :issue:`19505`.) + +The :class:`~collections.deque` class now defines +:meth:`~collections.deque.index`, :meth:`~collections.deque.insert`, and +:meth:`~collections.deque.copy`, as well as supports ``+`` and ``*`` operators. +This allows deques to be recognized as a :class:`~collections.abc.MutableSequence` +and improves their substitutability for lists. +(Contributed by Raymond Hettinger :issue:`23704`.) + +Docstrings produced by :func:`~collections.namedtuple` can now be updated:: Point = namedtuple('Point', ['x', 'y']) Point.__doc__ = 'ordered pair' Point.x.__doc__ = 'abscissa' Point.y.__doc__ = 'ordinate' - (Contributed by Berker Peksag in :issue:`24064`.) +(Contributed by Berker Peksag in :issue:`24064`.) + +The :class:`~collections.UserString` class now implements +:meth:`__getnewargs__`, :meth:`__rmod__`, :meth:`~str.casefold`, +:meth:`~str.format_map`, :meth:`~str.isprintable`, and :meth:`~str.maketrans` +methods to match corresponding methods of :class:`str`. +(Contributed by Joe Jevnik in :issue:`22189`.) + + +collections.abc +--------------- + +A new :class:`~collections.abc.Generator` abstract base class. (Contributed +by Stefan Behnel in :issue:`24018`.) + +New :class:`~collections.abc.Coroutine`, +:class:`~collections.abc.AsyncIterator`, and +:class:`~collections.abc.AsyncIterable` abstract base classes. +(Contributed by Yury Selivanov in :issue:`24184`.) + compileall ---------- -* :func:`compileall.compile_dir` and :mod:`compileall`'s command-line interface - can now do parallel bytecode compilation. - (Contributed by Claudiu Popa in :issue:`16104`.) +A new :mod:`compileall` option, ``-j N``, allows to run ``N`` workers +sumultaneously to perform parallel bytecode compilation. +The :func:`~compileall.compile_dir` function has a corresponding ``workers`` +parameter. (Contributed by Claudiu Popa in :issue:`16104`.) + +The ``-q`` command line option can now be specified more than once, in +which case all output, including errors, will be suppressed. The corresponding +``quiet`` parameter in :func:`~compileall.compile_dir`, +:func:`~compileall.compile_file`, and :func:`~compileall.compile_path` can now +accept an integer value indicating the level of output suppression. +(Contributed by Thomas Kluyver in :issue:`21338`.) + + +concurrent.futures +------------------ + +The :meth:`Executor.map ` method now accepts a +*chunksize* argument to allow batching of tasks to improve performance when +:meth:`~concurrent.futures.ProcessPoolExecutor` is used. +(Contributed by Dan O'Reilly in :issue:`11271`.) + contextlib ---------- -* The new :func:`contextlib.redirect_stderr` context manager(similar to - :func:`contextlib.redirect_stdout`) makes it easier for utility scripts to - handle inflexible APIs that write their output to :data:`sys.stderr` and - don't provide any options to redirect it. - (Contributed by Berker Peksag in :issue:`22389`.) +The new :func:`~contextlib.redirect_stderr` context manager (similar to +:func:`~contextlib.redirect_stdout`) makes it easier for utility scripts to +handle inflexible APIs that write their output to :data:`sys.stderr` and +don't provide any options to redirect it. (Contributed by Berker Peksag in +:issue:`22389`.) + curses ------ -* The new :func:`curses.update_lines_cols` function updates the variables - :envvar:`curses.LINES` and :envvar:`curses.COLS`. + +The new :func:`~curses.update_lines_cols` function updates :envvar:`LINES` +and :envvar:`COLS` environment variables. This is useful for detecting +manual screen resize. (Contributed by Arnon Yaari in :issue:`4254`.) + difflib ------- -* The charset of the HTML document generated by :meth:`difflib.HtmlDiff.make_file` - can now be customized by using *charset* keyword-only parameter. The default - charset of HTML document changed from ``'ISO-8859-1'`` to ``'utf-8'``. - (Contributed by Berker Peksag in :issue:`2052`.) - -* It's now possible to compare lists of byte strings with - :func:`difflib.diff_bytes` (fixes a regression from Python 2). +The charset of HTML documents generated by +:meth:`HtmlDiff.make_file ` +can now be customized by using a new *charset* keyword-only argument. +The default charset of HTML document changed from ``"ISO-8859-1"`` +to ``"utf-8"``. +(Contributed by Berker Peksag in :issue:`2052`.) + +The :func:`~difflib.diff_bytes` function can now compare lists of byte +strings. This fixes a regression from Python 2. +(Contributed by Terry J. Reedy and Greg Ward in :issue:`17445`.) + distutils --------- -* The ``build`` and ``build_ext`` commands now accept a ``-j`` - option to enable parallel building of extension modules. - (Contributed by Antoine Pitrou in :issue:`5309`.) - -* Added support for the LZMA compression. - (Contributed by Serhiy Storchaka in :issue:`16314`.) +Both ``build`` and ``build_ext`` commands now accept a ``-j`` option to +enable parallel building of extension modules. +(Contributed by Antoine Pitrou in :issue:`5309`.) + +The :mod:`distutils` module now supports ``xz`` compression, and can be +enabled by passing ``xztar`` as an argument to ``bdist --format``. +(Contributed by Serhiy Storchaka in :issue:`16314`.) + doctest ------- -* :func:`doctest.DocTestSuite` returns an empty :class:`unittest.TestSuite` if - *module* contains no docstrings instead of raising :exc:`ValueError`. - (Contributed by Glenn Jones in :issue:`15916`.) +The :func:`~doctest.DocTestSuite` function returns an empty +:class:`unittest.TestSuite` if *module* contains no docstrings instead of +raising :exc:`ValueError`. (Contributed by Glenn Jones in :issue:`15916`.) + email ----- -* A new policy option :attr:`~email.policy.Policy.mangle_from_` controls - whether or not lines that start with "From " in email bodies are prefixed with - a '>' character by generators. The default is ``True`` for - :attr:`~email.policy.compat32` and ``False`` for all other policies. - (Contributed by Milan Oberkirch in :issue:`20098`.) - -* A new method :meth:`~email.message.Message.get_content_disposition` provides - easy access to a canonical value for the :mailheader:`Content-Disposition` - header (``None`` if there is no such header). (Contributed by Abhilash Raj - in :issue:`21083`.) - -* A new policy option :attr:`~email.policy.EmailPolicy.utf8` can be set - ``True`` to encode email headers using the utf8 charset instead of using - encoded words. This allows ``Messages`` to be formatted according to - :rfc:`6532` and used with an SMTP server that supports the :rfc:`6531` - ``SMTPUTF8`` extension. (Contributed by R. David Murray in :issue:`24211`.) +A new policy option :attr:`Policy.mangle_from_ ` +controls whether or not lines that start with ``"From "`` in email bodies are +prefixed with a ``">"`` character by generators. The default is ``True`` for +:attr:`~email.policy.compat32` and ``False`` for all other policies. +(Contributed by Milan Oberkirch in :issue:`20098`.) + +A new +:meth:`Message.get_content_disposition ` +method provides easy access to a canonical value for the +:mailheader:`Content-Disposition` header. +(Contributed by Abhilash Raj in :issue:`21083`.) + +A new policy option :attr:`EmailPolicy.utf8 ` +can be set to ``True`` to encode email headers using the UTF-8 charset instead +of using encoded words. This allows ``Messages`` to be formatted according to +:rfc:`6532` and used with an SMTP server that supports the :rfc:`6531` +``SMTPUTF8`` extension. (Contributed by R. David Murray in +:issue:`24211`.) + + +enum +---- + +The :class:`~enum.Enum` callable has a new parameter *start* to +specify the initial number of enum values if only *names* are provided:: + + >>> Animal = enum.Enum('Animal', 'cat dog', start=10) + >>> Animal.cat + + >>> Animal.dog + + +(Contributed by Ethan Furman in :issue:`21706`.) + + +faulthandler +------------ + +:func:`~faulthandler.enable`, :func:`~faulthandler.register`, +:func:`~faulthandler.dump_traceback` and +:func:`~faulthandler.dump_traceback_later` functions now accept file +descriptors in addition to file-like objects. +(Contributed by Wei Wu in :issue:`23566`.) + + +functools +--------- + +Most of :func:`~functools.lru_cache` machinery is now implemented in C, making +it significantly faster. (Contributed by Matt Joiner, Alexey Kachayev, and +Serhiy Storchaka in :issue:`14373`.) + glob ---- -* :func:`~glob.iglob` and :func:`~glob.glob` now support recursive search in - subdirectories using the "``**``" pattern. - (Contributed by Serhiy Storchaka in :issue:`13968`.) +:func:`~glob.iglob` and :func:`~glob.glob` functions now support recursive +search in subdirectories using the ``"**"`` pattern. +(Contributed by Serhiy Storchaka in :issue:`13968`.) + + +heapq +----- + +Element comparison in :func:`~heapq.merge` can now be customized by +passing a :term:`key function` in a new optional ``key`` keyword argument. +A new optional ``reverse`` keyword argument can be used to reverse element +comparison. (Contributed by Raymond Hettinger in :issue:`13742`.) + + +http +---- + +A new :class:`HTTPStatus ` enum that defines a set of +HTTP status codes, reason phrases and long descriptions written in English. +(Contributed by Demian Brecht in :issue:`21793`.) + idlelib and IDLE ---------------- Since idlelib implements the IDLE shell and editor and is not intended for -import by other programs, it gets improvements with every release. See +import by other programs, it gets improvements with every release. See :file:`Lib/idlelib/NEWS.txt` for a cumulative list of changes since 3.4.0, as well as changes made in future 3.5.x releases. This file is also available from the IDLE Help -> About Idle dialog. + imaplib ------- -* :class:`IMAP4` now supports the context management protocol. When used in a - :keyword:`with` statement, the IMAP4 ``LOGOUT`` command will be called - automatically at the end of the block. (Contributed by Tarek Ziad? and - Serhiy Storchaka in :issue:`4972`.) - -* :mod:`imaplib` now supports :rfc:`5161`: the :meth:`~imaplib.IMAP4.enable` - extension), and :rfc:`6855`: utf-8 support (internationalized email, via the - ``UTF8=ACCEPT`` argument to :meth:`~imaplib.IMAP4.enable`). A new attribute, - :attr:`~imaplib.IMAP4.utf8_enabled`, tracks whether or not :rfc:`6855` - support is enabled. Milan Oberkirch, R. David Murray, and Maciej Szulik in - :issue:`21800`.) - -* :mod:`imaplib` now automatically encodes non-ASCII string usernames and - passwords using ``UTF8``, as recommended by the RFCs. (Contributed by Milan - Oberkirch in :issue:`21800`.) +The :class:`~imaplib.IMAP4` class now supports context manager protocol. +When used in a :keyword:`with` statement, the IMAP4 ``LOGOUT`` +command will be called automatically at the end of the block. +(Contributed by Tarek Ziad? and Serhiy Storchaka in :issue:`4972`.) + +The :mod:`imaplib` module now supports :rfc:`5161` (ENABLE Extension) +and :rfc:`6855` (UTF-8 Support) via the :meth:`IMAP4.enable ` +method. A new :attr:`IMAP4.utf8_enabled ` +attribute, tracks whether or not :rfc:`6855` support is enabled. +(Contributed by Milan Oberkirch, R. David Murray, and Maciej Szulik in +:issue:`21800`.) + +The :mod:`imaplib` module now automatically encodes non-ASCII string usernames +and passwords using UTF-8, as recommended by the RFCs. (Contributed by Milan +Oberkirch in :issue:`21800`.) + imghdr ------ -* :func:`~imghdr.what` now recognizes the `OpenEXR `_ - format. (Contributed by Martin Vignali and Claudiu Popa in :issue:`20295`.) +The :func:`~imghdr.what` function now recognizes the +`OpenEXR `_ format +(contributed by Martin Vignali and Claudiu Popa in :issue:`20295`), +and the `WebP `_ format +(contributed by Fabrice Aneche and Claudiu Popa in :issue:`20197`.) + importlib --------- -* :class:`importlib.util.LazyLoader` allows for the lazy loading of modules in - applications where startup time is paramount. - (Contributed by Brett Cannon in :issue:`17621`.) - -* :func:`importlib.abc.InspectLoader.source_to_code` is now a - static method to make it easier to work with source code in a string. - With a module object that you want to initialize you can then use - ``exec(code, module.__dict__)`` to execute the code in the module. - -* :func:`importlib.util.module_from_spec` is now the preferred way to create a - new module. Compared to :class:`types.ModuleType`, this new function will set - the various import-controlled attributes based on the passed-in spec object. +The :class:`util.LazyLoader ` class allows for +lazy loading of modules in applications where startup time is important. +(Contributed by Brett Cannon in :issue:`17621`.) + +The :func:`abc.InspectLoader.source_to_code ` +method is now a static method. This makes it easier to initialize a module +object with code compiled from a string by running +``exec(code, module.__dict__)``. +(Contributed by Brett Cannon in :issue:`21156`.) + +The new :func:`util.module_from_spec ` +function is now the preferred way to create a new module. As opposed to +creating a :class:`types.ModuleType` instance directly, this new function +will set the various import-controlled attributes based on the passed-in +spec object. (Contributed by Brett Cannon in :issue:`20383`.) + inspect ------- -* :class:`inspect.Signature` and :class:`inspect.Parameter` are now - picklable and hashable. (Contributed by Yury Selivanov in :issue:`20726` - and :issue:`20334`.) - -* New method :meth:`inspect.BoundArguments.apply_defaults`. (Contributed - by Yury Selivanov in :issue:`24190`.) - -* New class method :meth:`inspect.Signature.from_callable`, which makes - subclassing of :class:`~inspect.Signature` easier. (Contributed - by Yury Selivanov and Eric Snow in :issue:`17373`.) - -* New argument ``follow_wrapped`` for :func:`inspect.signature`. - (Contributed by Yury Selivanov in :issue:`20691`.) - -* New :func:`~inspect.iscoroutine`, :func:`~inspect.iscoroutinefunction` - and :func:`~inspect.isawaitable` functions. (Contributed by - Yury Selivanov in :issue:`24017`.) - -* New :func:`~inspect.getcoroutinelocals` and :func:`~inspect.getcoroutinestate` - functions. (Contributed by Yury Selivanov in :issue:`24400`.) +Both :class:`~inspect.Signature` and :class:`~inspect.Parameter` classes are +now picklable and hashable. (Contributed by Yury Selivanov in :issue:`20726` +and :issue:`20334`.) + +A new +:meth:`BoundArguments.apply_defaults ` +method provides a way to set default values for missing arguments. +(Contributed by Yury Selivanov in :issue:`24190`.) + +A new class method +:meth:`Signature.from_callable ` makes +subclassing of :class:`~inspect.Signature` easier. (Contributed +by Yury Selivanov and Eric Snow in :issue:`17373`.) + +The :func:`~inspect.signature` function now accepts a ``follow_wrapped`` +optional keyword argument, which, when set to ``False``, disables automatic +following of ``__wrapped__`` links. +(Contributed by Yury Selivanov in :issue:`20691`.) + +A set of new functions to inspect +:term:`coroutine functions ` and +:term:`coroutine objects ` has been added: +:func:`~inspect.iscoroutine`, :func:`~inspect.iscoroutinefunction`, +:func:`~inspect.isawaitable`, :func:`~inspect.getcoroutinelocals`, +and :func:`~inspect.getcoroutinestate`. +(Contributed by Yury Selivanov in :issue:`24017` and :issue:`24400`.) + +:func:`~inspect.stack`, :func:`~inspect.trace`, +:func:`~inspect.getouterframes`, and :func:`~inspect.getinnerframes` +functions now return a list of named tuples. +(Contributed by Daniel Shahaf in :issue:`16808`.) + + +io +-- + +A new :meth:`BufferedIOBase.readinto1 ` +method, that uses at most one call to the underlying raw stream's +:meth:`RawIOBase.read ` (or +:meth:`RawIOBase.readinto `) method. +(Contributed by Nikolaus Rath in :issue:`20578`.) + ipaddress --------- -* :class:`ipaddress.IPv4Network` and :class:`ipaddress.IPv6Network` now - accept an ``(address, netmask)`` tuple argument, so as to easily construct - network objects from existing addresses. (Contributed by Peter Moody - and Antoine Pitrou in :issue:`16531`.) +Both :class:`~ipaddress.IPv4Network` and :class:`~ipaddress.IPv6Network` classes +now accept an ``(address, netmask)`` tuple argument, so as to easily construct +network objects from existing addresses. (Contributed by Peter Moody +and Antoine Pitrou in :issue:`16531`.) + +A new :attr:`~ipaddress.IPv4Network.reverse_pointer>` attribute for +:class:`~ipaddress.IPv4Network` and :class:`~ipaddress.IPv6Network` classes +returns the name of the reverse DNS PTR record. +(Contributed by Leon Weber in :issue:`20480`.) + json ---- -* The output of :mod:`json.tool` command line interface is now in the same - order as the input. Use the :option:`--sort-keys` option to sort the output - of dictionaries alphabetically by key. (Contributed by Berker Peksag in - :issue:`21650`.) - -* JSON decoder now raises :exc:`json.JSONDecodeError` instead of - :exc:`ValueError`. (Contributed by Serhiy Storchaka in :issue:`19361`.) +The :mod:`json.tool` command line interface now preserves the order of keys in +JSON objects passed in input. The new ``--sort-keys`` option can be used +to sort the keys alphabetically. (Contributed by Berker Peksag +in :issue:`21650`.) + +JSON decoder now raises :exc:`json.JSONDecodeError` instead of +:exc:`ValueError`. (Contributed by Serhiy Storchaka in :issue:`19361`.) + + +locale +------ + +A new :func:`~locale.delocalize` function can be used to convert a string into +a normalized number string, taking the ``LC_NUMERIC`` settings into account. +(Contributed by C?dric Krier in :issue:`13918`.) + + +logging +------- + +All logging methods (:class:`~logging.Logger` :meth:`~logging.Logger.log`, +:meth:`~logging.Logger.exception`, :meth:`~logging.Logger.critical`, +:meth:`~logging.Logger.debug`, etc.), now accept exception instances +as an ``exc_info`` argument, in addition to boolean values and exception +tuples. (Contributed by Yury Selivanov in :issue:`20537`.) + +The :class:`handlers.HTTPHandler ` class now +accepts an optional :class:`ssl.SSLContext` instance to configure SSL +settings used in an HTTP connection. +(Contributed by Alex Gaynor in :issue:`22788`.) + +The :class:`handlers.QueueListener ` class now +takes a *respect_handler_level* keyword argument which, if set to ``True``, +will pass messages to handlers taking handler levels into account. +(Contributed by Vinay Sajip.) + + +lzma +---- + +The :meth:`LZMADecompressor.decompress ` +method now accepts an optional *max_length* argument to limit the maximum +size of decompressed data. +(Contributed by Martin Panter in :issue:`15955`.) + math ---- -* :data:`math.inf` and :data:`math.nan` constants added. (Contributed by Mark - Dickinson in :issue:`23185`.) -* :func:`math.isclose` function added. - (Contributed by Chris Barker and Tal Einat in :issue:`24270`.) +Two new constants have been added to the :mod:`math` module: :data:`~math.inf` +and :data:`~math.nan`. (Contributed by Mark Dickinson in :issue:`23185`.) + +A new function :func:`~math.isclose` provides a way to test for approximate +equality. (Contributed by Chris Barker and Tal Einat in :issue:`24270`.) + +A new :func:`~math.gcd` function has been added. The :func:`fractions.gcd` +function is now deprecated. (Contributed by Mark Dickinson and Serhiy +Storchaka in :issue:`22486`.) + + +operator +-------- + +:func:`~operator.attrgetter`, :func:`~operator.itemgetter`, +and :func:`~operator.methodcaller` objects now support pickling. +(Contributed by Josh Rosenberg and Serhiy Storchaka in :issue:`22955`.) + +New :func:`~operator.matmul` and :func:`~operator.imatmul` functions +to perform matrix multiplication. +(Contributed by Benjamin Peterson in :issue:`21176`.) + os -- -* New :func:`os.scandir` function that exposes file information from - the operating system when listing a directory. :func:`os.scandir` - returns an iterator of :class:`os.DirEntry` objects corresponding to - the entries in the directory given by *path*. (Contributed by Ben - Hoyt with the help of Victor Stinner in :issue:`22524`.) - -* :class:`os.stat_result` now has a :attr:`~os.stat_result.st_file_attributes` - attribute on Windows. (Contributed by Ben Hoyt in :issue:`21719`.) - -* :func:`os.urandom`: On Linux 3.17 and newer, the ``getrandom()`` syscall is - now used when available. On OpenBSD 5.6 and newer, the C ``getentropy()`` - function is now used. These functions avoid the usage of an internal file - descriptor. - -os.path +The new :func:`~os.scandir` function returning an iterator of +:class:`~os.DirEntry` objects has been added. If possible, :func:`~os.scandir` +extracts file attributes while scanning a directory, removing the need to +perform subsequent system calls to determine file type or attributes, which may +significantly improve performance. (Contributed by Ben Hoyt with the help +of Victor Stinner in :issue:`22524`.) + +On Windows, a new +:attr:`stat_result.st_file_attributes ` +attribute is now available. It corresponds to ``dwFileAttributes`` member of +the ``BY_HANDLE_FILE_INFORMATION`` structure returned by +``GetFileInformationByHandle()``. (Contributed by Ben Hoyt in :issue:`21719`.) + +The :func:`~os.urandom` function now uses ``getrandom()`` syscall on Linux 3.17 +or newer, and ``getentropy()`` on OpenBSD 5.6 and newer, removing the need to +use ``/dev/urandom`` and avoiding failures due to potential file descriptor +exhaustion. (Contributed by Victor Stinner in :issue:`22181`.) + +New :func:`~os.get_blocking` and :func:`~os.set_blocking` functions allow to +get and set a file descriptor blocking mode (:data:`~os.O_NONBLOCK`.) +(Contributed by Victor Stinner in :issue:`22054`.) + +The :func:`~os.truncate` and :func:`~os.ftruncate` functions are now supported +on Windows. (Contributed by Steve Dower in :issue:`23668`.) + +There is a new :func:`os.path.commonpath` function returning the longest +common sub-path of each passed pathname. Unlike the +:func:`os.path.commonprefix` function, it always returns a valid +path. (Contributed by Rafik Draoui and Serhiy Storchaka in :issue:`10395`.) + + +pathlib ------- -* New :func:`~os.path.commonpath` function that extracts common path prefix. - Unlike the :func:`~os.path.commonprefix` function, it always returns a valid - path. (Contributed by Rafik Draoui and Serhiy Storchaka in :issue:`10395`.) +The new :meth:`Path.samefile ` method can be used +to check whether the path points to the same file as other path, which can be +either an another :class:`~pathlib.Path` object, or a string. +(Contributed by Vajrasky Kok and Antoine Pitrou in :issue:`19775`.) + +The :meth:`Path.mkdir ` method how accepts a new optional +``exist_ok`` argument to match ``mkdir -p`` and :func:`os.makrdirs` +functionality. (Contributed by Berker Peksag in :issue:`21539`.) + +There is a new :meth:`Path.expanduser ` method to +expand ``~`` and ``~user`` prefixes. (Contributed by Serhiy Storchaka and +Claudiu Popa in :issue:`19776`.) + +A new :meth:`Path.home ` class method can be used to get +an instance of :class:`~pathlib.Path` object representing the user?s home +directory. +(Contributed by Victor Salgado and Mayank Tripathi in :issue:`19777`.) + +New :meth:`Path.write_text `, +:meth:`Path.read_text `, +:meth:`Path.write_bytes `, +:meth:`Path.read_bytes ` methods to simplify +read/write operations on files. +(Contributed by Christopher Welborn in :issue:`20218`.) + pickle ------ -* Serializing more "lookupable" objects (such as unbound methods or nested - classes) now are supported with pickle protocols < 4. - (Contributed by Serhiy Storchaka in :issue:`23611`.) +Nested objects, such as unbound methods or nested classes, can now be pickled +using :ref:`pickle protocols ` older than protocol version 4. +Protocol version 4 already supports these cases. (Contributed by Serhiy +Storchaka in :issue:`23611`.) + poplib ------ -* A new command :meth:`~poplib.POP3.utf8` enables :rfc:`6856` - (internationalized email) support if the POP server supports it. (Contributed - by Milan OberKirch in :issue:`21804`.) +A new :meth:`POP3.utf8 ` command enables :rfc:`6856` +(Internationalized Email) support, if a POP server supports it. +(Contributed by Milan OberKirch in :issue:`21804`.) + re -- -* Number of capturing groups in regular expression is no longer limited by 100. - (Contributed by Serhiy Storchaka in :issue:`22437`.) - -* Now unmatched groups are replaced with empty strings in :func:`re.sub` - and :func:`re.subn`. (Contributed by Serhiy Storchaka in :issue:`1519638`.) +The number of capturing groups in regular expression is no longer limited by +100. (Contributed by Serhiy Storchaka in :issue:`22437`.) + +The :func:`~re.sub` and :func:`~re.subn` functions now replace unmatched +groups with empty strings instead of raising an exception. +(Contributed by Serhiy Storchaka in :issue:`1519638`.) + +The :class:`re.error` exceptions have new attributes: +:attr:`~re.error.msg`, :attr:`~re.error.pattern`, +:attr:`~re.error.pos`, :attr:`~re.error.lineno`, +and :attr:`~re.error.colno` that provide better context +information about the error. +(Contributed by Serhiy Storchaka in :issue:`22578`.) + + +readline +-------- + +A new :func:`~readline.append_history_file` function can be used to append +the specified number of trailing elements in history to the given file. +(Contributed by Bruno Cauet in :issue:`22940`.) + + +selectors +--------- + +The new :class:`~selectors.DevpollSelector` supports efficient +``/dev/poll`` polling on Solaris. +(Contributed by Giampaolo Rodola' in :issue:`18931`.) + shutil ------ -* :func:`~shutil.move` now accepts a *copy_function* argument, allowing, - for example, :func:`~shutil.copy` to be used instead of the default - :func:`~shutil.copy2` if there is a need to ignore metadata. (Contributed by - Claudiu Popa in :issue:`19840`.) +The :func:`~shutil.move` function now accepts a *copy_function* argument, +allowing, for example, the :func:`~shutil.copy` function to be used instead of +the default :func:`~shutil.copy2` if there is a need to ignore file metadata +when moving. +(Contributed by Claudiu Popa in :issue:`19840`.) + +The :func:`~shutil.make_archive` function now supports the *xztar* format. +(Contributed by Serhiy Storchaka in :issue:`5411`.) + signal ------ -* On Windows, :func:`signal.set_wakeup_fd` now also supports socket handles. - (Contributed by Victor Stinner in :issue:`22018`.) - -* Different constants of :mod:`signal` module are now enumeration values using - the :mod:`enum` module. This allows meaningful names to be printed during - debugging, instead of integer ?magic numbers?. (Contributed by Giampaolo - Rodola' in :issue:`21076`.) +On Windows, the :func:`~signal.set_wakeup_fd` function now also supports +socket handles. (Contributed by Victor Stinner in :issue:`22018`.) + +Various ``SIG*`` constants in the :mod:`signal` module have been converted into +:mod:`Enums `. This allows meaningful names to be printed +during debugging, instead of integer "magic numbers". +(Contributed by Giampaolo Rodola' in :issue:`21076`.) + smtpd ----- -* Both :class:`~smtpd.SMTPServer` and :class:`smtpd.SMTPChannel` now accept a - *decode_data* keyword to determine if the DATA portion of the SMTP - transaction is decoded using the ``utf-8`` codec or is instead provided to - :meth:`~smtpd.SMTPServer.process_message` as a byte string. The default - is ``True`` for backward compatibility reasons, but will change to ``False`` - in Python 3.6. If *decode_data* is set to ``False``, the - :meth:`~smtpd.SMTPServer.process_message` method must be prepared to accept - keyword arguments. (Contributed by Maciej Szulik in :issue:`19662`.) - -* :class:`~smtpd.SMTPServer` now advertises the ``8BITMIME`` extension - (:rfc:`6152`) if if *decode_data* has been set ``True``. If the client - specifies ``BODY=8BITMIME`` on the ``MAIL`` command, it is passed to - :meth:`~smtpd.SMTPServer.process_message` via the ``mail_options`` keyword. - (Contributed by Milan Oberkirch and R. David Murray in :issue:`21795`.) - -* :class:`~smtpd.SMTPServer` now supports the ``SMTPUTF8`` extension - (:rfc:`6531`: Internationalized Email). If the client specified ``SMTPUTF8 - BODY=8BITMIME`` on the ``MAIL`` command, they are passed to - :meth:`~smtpd.SMTPServer.process_message` via the ``mail_options`` keyword. - It is the responsibility of the :meth:`~smtpd.SMTPServer.process_message` - method to correctly handle the ``SMTPUTF8`` data. (Contributed by Milan - Oberkirch in :issue:`21725`.) - -* It is now possible to provide, directly or via name resolution, IPv6 - addresses in the :class:`~smtpd.SMTPServer` constructor, and have it - successfully connect. (Contributed by Milan Oberkirch in :issue:`14758`.) +Both :class:`~smtpd.SMTPServer` and :class:`~smtpd.SMTPChannel` classes now +accept a *decode_data* keyword argument to determine if the ``DATA`` portion of +the SMTP transaction is decoded using the ``"utf-8"`` codec or is instead +provided to the +:meth:`SMTPServer.process_message ` +method as a byte string. The default is ``True`` for backward compatibility +reasons, but will change to ``False`` in Python 3.6. If *decode_data* is set +to ``False``, the :meth:`~smtpd.SMTPServer.process_message` method must +be prepared to accept keyword arguments. +(Contributed by Maciej Szulik in :issue:`19662`.) + +The :class:`~smtpd.SMTPServer` class now advertises the ``8BITMIME`` extension +(:rfc:`6152`) if *decode_data* has been set ``True``. If the client +specifies ``BODY=8BITMIME`` on the ``MAIL`` command, it is passed to +:meth:`SMTPServer.process_message ` +via the ``mail_options`` keyword. +(Contributed by Milan Oberkirch and R. David Murray in :issue:`21795`.) + +The :class:`~smtpd.SMTPServer` class now also supports the ``SMTPUTF8`` +extension (:rfc:`6531`: Internationalized Email). If the client specified +``SMTPUTF8 BODY=8BITMIME`` on the ``MAIL`` command, they are passed to +:meth:`SMTPServer.process_message ` +via the ``mail_options`` keyword. It is the responsibility of the +:meth:`~smtpd.SMTPServer.process_message` method to correctly handle the +``SMTPUTF8`` data. (Contributed by Milan Oberkirch in :issue:`21725`.) + +It is now possible to provide, directly or via name resolution, IPv6 +addresses in the :class:`~smtpd.SMTPServer` constructor, and have it +successfully connect. (Contributed by Milan Oberkirch in :issue:`14758`.) + smtplib ------- -* A new :meth:`~smtplib.SMTP.auth` method provides a convenient way to - implement custom authentication mechanisms. - (Contributed by Milan Oberkirch in :issue:`15014`.) - -* Additional debuglevel (2) shows timestamps for debug messages in - :class:`smtplib.SMTP`. (Contributed by Gavin Chappell and Maciej Szulik in - :issue:`16914`.) - -* :mod:`smtplib` now supports :rfc:`6531` (SMTPUTF8) in both the - :meth:`~smtplib.SMTP.sendmail` and :meth:`~smtplib.SMTP.send_message` - commands. (Contributed by Milan Oberkirch and R. David Murray in - :issue:`22027`.) +A new :meth:`SMTP.auth ` method provides a convenient way to +implement custom authentication mechanisms. (Contributed by Milan +Oberkirch in :issue:`15014`.) + +The :meth:`SMTP.set_debuglevel ` method now +accepts an additional debuglevel (2), which enables timestamps in debug +messages. (Contributed by Gavin Chappell and Maciej Szulik in :issue:`16914`.) + +Both :meth:`SMTP.sendmail ` and +:meth:`SMTP.send_message ` methods now +support support :rfc:`6531` (SMTPUTF8). +(Contributed by Milan Oberkirch and R. David Murray in :issue:`22027`.) + sndhdr ------ -* :func:`~sndhdr.what` and :func:`~sndhdr.whathdr` now return - :func:`~collections.namedtuple`. - (Contributed by Claudiu Popa in :issue:`18615`.) +:func:`~sndhdr.what` and :func:`~sndhdr.whathdr` functions now return +a :func:`~collections.namedtuple`. (Contributed by Claudiu Popa in +:issue:`18615`.) + ssl --- -* The :meth:`~ssl.SSLSocket.do_handshake`, :meth:`~ssl.SSLSocket.read`, - :meth:`~ssl.SSLSocket.shutdown`, and :meth:`~ssl.SSLSocket.write` methods of - :class:`ssl.SSLSocket` don't reset the socket timeout anymore each time bytes - are received or sent. The socket timeout is now the maximum total duration of - the method. - -* Memory BIO Support: new classes :class:`~ssl.SSLObject`, - :class:`~ssl.MemoryBIO`, and new - :meth:`SSLContext.wrap_bio ` method. - (Contributed by Geert Jansen in :issue:`21965`.) +Memory BIO Support +~~~~~~~~~~~~~~~~~~ + +(Contributed by Geert Jansen in :issue:`21965`.) + +The new :class:`~ssl.SSLObject` class has been added to provide SSL protocol +support for cases when the network I/O capabilities of :class:`~ssl.SSLSocket` +are not necessary or suboptimal. :class:`~ssl.SSLObject` represents +an SSL protocol instance, but does not implement any network I/O methods, and +instead provides a memory buffer interface. The new :class:`~ssl.MemoryBIO` +class can be used to pass data between Python and an SSL protocol instance. + +The memory BIO SSL support is primarily intended to be used in frameworks +implementing asynchronous I/O for which :class:`~ssl.SSLSocket`'s readiness +model ("select/poll") is inefficient. + +A new :meth:`SSLContext.wrap_bio ` method can be used +to create a new :class:`~ssl.SSLObject` instance. + + +Application-Layer Protocol Negotiation Support +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +(Contributed by Benjamin Peterson in :issue:`20188`.) + +Where OpenSSL support is present, :mod:`ssl` module now implements +*Application-Layer Protocol Negotiation* TLS extension as described +in :rfc:`7301`. + +The new :meth:`SSLContext.set_alpn_protocols ` +can be used to specify which protocols a socket should advertise during +the TLS handshake. + +The new +:meth:`SSLSocket.selected_alpn_protocol ` +returns the protocol that was selected during the TLS handshake. +:data:`~ssl.HAS_ALPN` flag indicates whether APLN support is present. + + +Other Changes +~~~~~~~~~~~~~ + +There is a new :meth:`SSLSocket.version ` method to query +the actual protocol version in use. +(Contributed by Antoine Pitrou in :issue:`20421`.) + +The :class:`~ssl.SSLSocket` class now implements +a :meth:`SSLSocket.sendfile ` method. +(Contributed by Giampaolo Rodola' in :issue:`17552`.) + +The :meth:`SSLSocket.send ` method now raises either +:exc:`ssl.SSLWantReadError` or :exc:`ssl.SSLWantWriteError` exception on a +non-blocking socket if the operation would block. Previously, it would return +``0``. (Contributed by Nikolaus Rath in :issue:`20951`.) + +The :func:`~ssl.cert_time_to_seconds` function now interprets the input time +as UTC and not as local time, per :rfc:`5280`. Additionally, the return +value is always an :class:`int`. (Contributed by Akira Li in :issue:`19940`.) + +New :meth:`SSLObject.shared_ciphers ` and +:meth:`SSLSocket.shared_ciphers ` methods return +the list of ciphers sent by the client during the handshake. +(Contributed by Benjamin Peterson in :issue:`23186`.) + +The :meth:`SSLSocket.do_handshake `, +:meth:`SSLSocket.read `, +:meth:`SSLSocket.shutdown `, and +:meth:`SSLSocket.write ` methods of :class:`ssl.SSLSocket` +class no longer reset the socket timeout every time bytes are received or sent. +The socket timeout is now the maximum total duration of the method. +(Contributed by Victor Stinner in :issue:`23853`.) + +The :func:`~ssl.match_hostname` function now supports matching of IP addresses. +(Contributed by Antoine Pitrou in :issue:`23239`.) + socket ------ -* New :meth:`socket.socket.sendfile` method allows to send a file over a socket - by using high-performance :func:`os.sendfile` function on UNIX resulting in - uploads being from 2x to 3x faster than when using plain - :meth:`socket.socket.send`. - (Contributed by Giampaolo Rodola' in :issue:`17552`.) - -* The :meth:`socket.socket.sendall` method don't reset the socket timeout - anymore each time bytes are received or sent. The socket timeout is now the - maximum total duration to send all data. +Functions with timeouts now use a monotonic clock, instead of a system clock. +(Contributed by Victor Stinner in :issue:`22043`.) + +A new :meth:`socket.sendfile ` method allows to +send a file over a socket by using the high-performance :func:`os.sendfile` +function on UNIX resulting in uploads being from 2 to 3 times faster than when +using plain :meth:`socket.send `. +(Contributed by Giampaolo Rodola' in :issue:`17552`.) + +The :meth:`socket.sendall ` method no longer resets the +socket timeout every time bytes are received or sent. The socket timeout is +now the maximum total duration to send all data. +(Contributed by Victor Stinner in :issue:`23853`.) + +The *backlog* argument of the :meth:`socket.listen ` +method is now optional. By default it is set to +:data:`SOMAXCONN ` or to ``128`` whichever is less. +(Contributed by Charles-Fran?ois Natali in :issue:`21455`.) + + +sqlite3 +------- + +The :class:`~sqlite3.Row` class now fully supports sequence protocol, +in particular :func:`reversed` iteration and slice indexing. +(Contributed by Claudiu Popa in :issue:`10203`; by Lucas Sinclair, +Jessica McKellar, and Serhiy Storchaka in :issue:`13583`.) + subprocess ---------- -* The new :func:`subprocess.run` function runs subprocesses and returns a - :class:`subprocess.CompletedProcess` object. It Provides a more consistent - API than :func:`~subprocess.call`, :func:`~subprocess.check_call` and - :func:`~subprocess.check_output`. +The new :func:`~subprocess.run` function has been added. +It runs the specified command and and returns a +:class:`~subprocess.CompletedProcess` object, which describes a finished +process. The new API is more consistent and is the recommended approach +to invoking subprocesses in Python code that does not need to maintain +compatibility with earlier Python versions. +(Contributed by Thomas Kluyver in :issue:`23342`.) + sys --- -* New :func:`~sys.set_coroutine_wrapper` and :func:`~sys.get_coroutine_wrapper` - functions. (Contributed by Yury Selivanov in :issue:`24017`.) +A new :func:`~sys.set_coroutine_wrapper` function allows setting a global +hook that will be called whenever a :term:`coroutine object ` +is created by an :keyword:`async def` function. A corresponding +:func:`~sys.get_coroutine_wrapper` can be used to obtain a currently set +wrapper. Both functions are provisional, and are intended for debugging +purposes only. (Contributed by Yury Selivanov in :issue:`24017`.) + +A new :func:`~sys.is_finalizing` function can be used to check if the Python +interpreter is :term:`shutting down `. +(Contributed by Antoine Pitrou in :issue:`22696`.) + sysconfig --------- -* The user scripts directory on Windows is now versioned. - (Contributed by Paul Moore in :issue:`23437`.) +The name of the user scripts directory on Windows now includes the first +two components of Python version. (Contributed by Paul Moore +in :issue:`23437`.) + tarfile ------- -* The :func:`tarfile.open` function now supports ``'x'`` (exclusive creation) - mode. (Contributed by Berker Peksag in :issue:`21717`.) - -* The :meth:`~tarfile.TarFile.extractall` and :meth:`~tarfile.TarFile.extract` - methods now take a keyword parameter *numeric_only*. If set to ``True``, - the extracted files and directories will be owned by the numeric uid and gid - from the tarfile. If set to ``False`` (the default, and the behavior in - versions prior to 3.5), they will be owned bythe named user and group in the - tarfile. (Contributed by Michael Vogt and Eric Smith in :issue:`23193`.) +The *mode* argument of the :func:`~tarfile.open` function now accepts ``"x"`` +to request exclusive creation. (Contributed by Berker Peksag in :issue:`21717`.) + +:meth:`TarFile.extractall ` and +:meth:`TarFile.extract ` methods now take a keyword +argument *numeric_only*. If set to ``True``, the extracted files and +directories will be owned by the numeric ``uid`` and ``gid`` from the tarfile. +If set to ``False`` (the default, and the behavior in versions prior to 3.5), +they will be owned by the named user and group in the tarfile. +(Contributed by Michael Vogt and Eric Smith in :issue:`23193`.) + + +threading +--------- + +Both :meth:`Lock.acquire ` and +:meth:`RLock.acquire ` methods +now use a monotonic clock for timeout management. +(Contributed by Victor Stinner in :issue:`22043`.) + time ---- -* The :func:`time.monotonic` function is now always available. (Contributed by - Victor Stinner in :issue:`22043`.) +The :func:`~time.monotonic` function is now always available. +(Contributed by Victor Stinner in :issue:`22043`.) + + +timeit +------ + +A new command line option ``-u`` or ``--unit=U`` can be used to specify the time +unit for the timer output. Supported options are ``usec``, ``msec``, +or ``sec``. (Contributed by Julian Gindi in :issue:`18983`.) + +The :func:`~timeit.timeit` function has a new *globals* parameter for +specifying the namespace in which the code will be running. +(Contributed by Ben Roberts in :issue:`2527`.) + tkinter ------- -* The :mod:`tkinter._fix` module used for setting up the Tcl/Tk environment - on Windows has been replaced by a private function in the :mod:`_tkinter` - module which makes no permanent changes to environment variables. - (Contributed by Zachary Ware in :issue:`20035`.) +The :mod:`tkinter._fix` module used for setting up the Tcl/Tk environment +on Windows has been replaced by a private function in the :mod:`_tkinter` +module which makes no permanent changes to environment variables. +(Contributed by Zachary Ware in :issue:`20035`.) + + +traceback +--------- + +New :func:`~traceback.walk_stack` and :func:`~traceback.walk_tb` +functions to conveniently traverse frame and traceback objects. +(Contributed by Robert Collins in :issue:`17911`.) + +New lightweight classes: :class:`~traceback.TracebackException`, +:class:`~traceback.StackSummary`, and :class:`traceback.FrameSummary`. +(Contributed by Robert Collins in :issue:`17911`.) + +Both :func:`~traceback.print_tb` and :func:`~traceback.print_stack` functions +now support negative values for the *limit* argument. +(Contributed by Dmitry Kazakov in :issue:`22619`.) + types ----- -* New :func:`~types.coroutine` function. (Contributed by Yury Selivanov - in :issue:`24017`.) - -* New :class:`~types.CoroutineType`. (Contributed by Yury Selivanov - in :issue:`24400`.) +A new :func:`~types.coroutine` function to transform +:term:`generator ` and +:class:`generator-like ` objects into +:term:`awaitables `. +(Contributed by Yury Selivanov in :issue:`24017`.) + +A new :class:`~types.CoroutineType` is the type of :term:`coroutine` objects +created by :keyword:`async def` functions. +(Contributed by Yury Selivanov in :issue:`24400`.) + urllib ------ -* A new :class:`~urllib.request.HTTPPasswordMgrWithPriorAuth` allows HTTP Basic - Authentication credentials to be managed so as to eliminate unnecessary - ``401`` response handling, or to unconditionally send credentials - on the first request in order to communicate with servers that return a - ``404`` response instead of a ``401`` if the ``Authorization`` header is not - sent. (Contributed by Matej Cepl in :issue:`19494` and Akshit Khurana in - :issue:`7159`.) - -* A new :func:`~urllib.parse.urlencode` parameter *quote_via* provides a way to - control the encoding of query parts if needed. (Contributed by Samwyse and - Arnon Yaari in :issue:`13866`.) +A new +:class:`request.HTTPPasswordMgrWithPriorAuth ` +class allows HTTP Basic Authentication credentials to be managed so as to +eliminate unnecessary ``401`` response handling, or to unconditionally send +credentials on the first request in order to communicate with servers that +return a ``404`` response instead of a ``401`` if the ``Authorization`` header +is not sent. (Contributed by Matej Cepl in :issue:`19494` and Akshit Khurana in +:issue:`7159`.) + +A new *quote_via* argument for the +:func:`parse.urlencode ` +function provides a way to control the encoding of query parts if needed. +(Contributed by Samwyse and Arnon Yaari in :issue:`13866`.) + +The :func:`request.urlopen ` function accepts an +:class:`ssl.SSLContext` object as a *context* argument, which will be used for +the HTTPS connection. (Contributed by Alex Gaynor in :issue:`22366`.) + +The :func:`parse.urljoin ` was updated to use the +:rfc:`3986` semantics for the resolution of relative URLs, rather than +:rfc:`1808` and :rfc:`2396`. +(Contributed by Demian Brecht and Senthil Kumaran in :issue:`22118`.) + unicodedata ----------- -* The :mod:`unicodedata` module now uses data from `Unicode 8.0.0 - `_. +The :mod:`unicodedata` module now uses data from `Unicode 8.0.0 +`_. + + +unittest +-------- + +A new command line option ``--locals`` to show local variables in +tracebacks. (Contributed by Robert Collins in :issue:`22936`.) + + +unittest.mock +------------- + +The :class:`~unittest.mock.Mock` has the following improvements: + +* Class constructor has a new *unsafe* parameter, which causes mock + objects to raise :exc:`AttributeError` on attribute names starting + with ``"assert"``. + (Contributed by Kushal Das in :issue:`21238`.) + +* A new :meth:`Mock.assert_not_called ` + method to check if the mock object was called. + (Contributed by Kushal Das in :issue:`21262`.) + +The :class:`~unittest.mock.MagicMock` class now supports :meth:`__truediv__`, +:meth:`__divmod__` and :meth:`__matmul__` operators. +(Contributed by Johannes Baiter in :issue:`20968`, and H?kan L?vdahl +in :issue:`23581` and :issue:`23568`.) wsgiref ------- -* *headers* parameter of :class:`wsgiref.headers.Headers` is now optional. - (Contributed by Pablo Torres Navarrete and SilentGhost in :issue:`5800`.) +The *headers* argument of the :class:`headers.Headers ` +class constructor is now optional. +(Contributed by Pablo Torres Navarrete and SilentGhost in :issue:`5800`.) + xmlrpc ------ -* :class:`xmlrpc.client.ServerProxy` is now a :term:`context manager`. - (Contributed by Claudiu Popa in :issue:`20627`.) +The :class:`client.ServerProxy ` class is now a +:term:`context manager`. +(Contributed by Claudiu Popa in :issue:`20627`.) + +:class:`client.ServerProxy ` constructor now accepts +an optional :class:`ssl.SSLContext` instance. +(Contributed by Alex Gaynor in :issue:`22960`.) + xml.sax ------- -* SAX parsers now support a character stream of - :class:`~xml.sax.xmlreader.InputSource` object. - (Contributed by Serhiy Storchaka in :issue:`2175`.) - -faulthandler ------------- - -* :func:`~faulthandler.enable`, :func:`~faulthandler.register`, - :func:`~faulthandler.dump_traceback` and - :func:`~faulthandler.dump_traceback_later` functions now accept file - descriptors. (Contributed by Wei Wu in :issue:`23566`.) +SAX parsers now support a character stream of the +:class:`xmlreader.InputSource ` object. +(Contributed by Serhiy Storchaka in :issue:`2175`.) + zipfile ------- -* Added support for writing ZIP files to unseekable streams. - (Contributed by Serhiy Storchaka in :issue:`23252`.) - -* The :func:`zipfile.ZipFile.open` function now supports ``'x'`` (exclusive - creation) mode. (Contributed by Serhiy Storchaka in :issue:`21717`.) +ZIP output can now be written to unseekable streams. +(Contributed by Serhiy Storchaka in :issue:`23252`.) + +The *mode* argument of :meth:`ZipFile.open ` method now +accepts ``"x"`` to request exclusive creation. +(Contributed by Serhiy Storchaka in :issue:`21717`.) + + +Other module-level changes +========================== + +Many functions in :mod:`mmap`, :mod:`ossaudiodev`, :mod:`socket`, +:mod:`ssl`, and :mod:`codecs` modules now accept writable +:term:`bytes-like objects `. +(Contributed by Serhiy Storchaka in :issue:`23001`.) Optimizations ============= -The following performance enhancements have been added: - -* :func:`os.walk` has been sped up by 3-5x on POSIX systems and 7-20x - on Windows. This was done using the new :func:`os.scandir` function, - which exposes file information from the underlying ``readdir`` and - ``FindFirstFile``/``FindNextFile`` system calls. (Contributed by - Ben Hoyt with help from Victor Stinner in :issue:`23605`.) - -* Construction of ``bytes(int)`` (filled by zero bytes) is faster and uses less - memory for large objects. ``calloc()`` is used instead of ``malloc()`` to - allocate memory for these objects. - -* Some operations on :class:`~ipaddress.IPv4Network` and - :class:`~ipaddress.IPv6Network` have been massively sped up, such as - :meth:`~ipaddress.IPv4Network.subnets`, :meth:`~ipaddress.IPv4Network.supernet`, - :func:`~ipaddress.summarize_address_range`, :func:`~ipaddress.collapse_addresses`. - The speed up can range from 3x to 15x. - (:issue:`21486`, :issue:`21487`, :issue:`20826`) - -* Many operations on :class:`io.BytesIO` are now 50% to 100% faster. - (Contributed by Serhiy Storchaka in :issue:`15381` and David Wilson in - :issue:`22003`.) - -* :func:`marshal.dumps` is now faster (65%-85% with versions 3--4, 20-25% with - versions 0--2 on typical data, and up to 5x in best cases). - (Contributed by Serhiy Storchaka in :issue:`20416` and :issue:`23344`.) - -* The UTF-32 encoder is now 3x to 7x faster. (Contributed by Serhiy Storchaka - in :issue:`15027`.) +The :func:`os.walk` function has been sped up by 3 to 5 times on POSIX systems, +and by 7 to 20 times on Windows. This was done using the new :func:`os.scandir` +function, which exposes file information from the underlying ``readdir`` or +``FindFirstFile``/``FindNextFile`` system calls. (Contributed by +Ben Hoyt with help from Victor Stinner in :issue:`23605`.) + +Construction of ``bytes(int)`` (filled by zero bytes) is faster and uses less +memory for large objects. ``calloc()`` is used instead of ``malloc()`` to +allocate memory for these objects. +(Contributed by Victor Stinner in :issue:`21233`.) + +Some operations on :mod:`ipaddress` :class:`~ipaddress.IPv4Network` and +:class:`~ipaddress.IPv6Network` have been massively sped up, such as +:meth:`~ipaddress.IPv4Network.subnets`, :meth:`~ipaddress.IPv4Network.supernet`, +:func:`~ipaddress.summarize_address_range`, :func:`~ipaddress.collapse_addresses`. +The speed up can range from 3 to 15 times. +(Contributed by Antoine Pitrou, Michel Albert, and Markus in +:issue:`21486`, :issue:`21487`, :issue:`20826`, :issue:`23266`.) + +Pickling of :mod:`ipaddress` objects was optimized to produce significantly +smaller output. (Contributed by Serhiy Storchaka in :issue:`23133`.) + +Many operations on :class:`io.BytesIO` are now 50% to 100% faster. +(Contributed by Serhiy Storchaka in :issue:`15381` and David Wilson in +:issue:`22003`.) + +The :func:`marshal.dumps` function is now faster: 65-85% with versions 3 +and 4, 20-25% with versions 0 to 2 on typical data, and up to 5 times in +best cases. +(Contributed by Serhiy Storchaka in :issue:`20416` and :issue:`23344`.) + +The UTF-32 encoder is now 3 to 7 times faster. +(Contributed by Serhiy Storchaka in :issue:`15027`.) + +Regular expressions are now parsed up to 10% faster. +(Contributed by Serhiy Storchaka in :issue:`19380`.) + +The :func:`json.dumps` function was optimized to run with +``ensure_ascii=False`` as fast as with ``ensure_ascii=True``. +(Contributed by Naoki Inada in :issue:`23206`.) + +The :c:func:`PyObject_IsInstance` and :c:func:`PyObject_IsSubclass` +functions have been sped up in the common case that the second argument +has :class:`type` as its metaclass. +(Contributed Georg Brandl by in :issue:`22540`.) + +Method caching was slightly improved, yielding up to 5% performance +improvement in some benchmarks. +(Contributed by Antoine Pitrou in :issue:`22847`.) + +Objects from :mod:`random` module now use two times less memory on 64-bit +builds. (Contributed by Serhiy Storchaka in :issue:`23488`.) + +The :func:`property` getter calls are up to 25% faster. +(Contributed by Joe Jevnik in :issue:`23910`.) + +Instantiation of :class:`fractions.Fraction` is now up to 30% faster. +(Contributed by Stefan Behnel in :issue:`22464`.) + +String methods :meth:`~str.find`, :meth:`~str.rfind`, :meth:`~str.split`, +:meth:`~str.partition` and :keyword:`in` string operator are now significantly +faster for searching 1-character substrings. +(Contributed by Serhiy Storchaka in :issue:`23573`.) Build and C API Changes ======================= -Changes to Python's build process and to the C API include: - -* New ``calloc`` functions: - - * :c:func:`PyMem_RawCalloc` - * :c:func:`PyMem_Calloc` - * :c:func:`PyObject_Calloc` - * :c:func:`_PyObject_GC_Calloc` - -* Windows builds now require Microsoft Visual C++ 14.0, which - is available as part of `Visual Studio 2015 `_. +New ``calloc`` functions were added: + + * :c:func:`PyMem_RawCalloc`, + * :c:func:`PyMem_Calloc`, + * :c:func:`PyObject_Calloc`, + * :c:func:`_PyObject_GC_Calloc`. + +(Contributed by Victor Stinner in :issue:`21233`.) + +New encoding/decoding helper functions: + + * :c:func:`Py_DecodeLocale` (replaced ``_Py_char2wchar()``), + * :c:func:`Py_EncodeLocale` (replaced ``_Py_wchar2char()``). + +(Contributed by Victor Stinner in :issue:`18395`.) + +A new :c:func:`PyCodec_NameReplaceErrors` function to replace the unicode +encode error with ``\N{...}`` escapes. +(Contributed by Serhiy Storchaka in :issue:`19676`.) + +A new :c:func:`PyErr_FormatV` function similar to :c:func:`PyErr_Format`, +but accepts a ``va_list`` argument. +(Contributed by Antoine Pitrou in :issue:`18711`.) + +A new :c:data:`PyExc_RecursionError` exception. +(Contributed by Georg Brandl in :issue:`19235`.) + +New :c:func:`PyModule_FromDefAndSpec`, :c:func:`PyModule_FromDefAndSpec2`, +and :c:func:`PyModule_ExecDef` introduced by :pep:`489` -- multi-phase +extension module initialization. +(Contributed by Petr Viktorin in :issue:`24268`.) + +New :c:func:`PyNumber_MatrixMultiply` and +:c:func:`PyNumber_InPlaceMatrixMultiply` functions to perform matrix +multiplication. +(Contributed by Benjamin Peterson in :issue:`21176`. See also :pep:`465` +for details.) + +The :c:member:`PyTypeObject.tp_finalize` slot is now part of stable ABI. + +Windows builds now require Microsoft Visual C++ 14.0, which +is available as part of `Visual Studio 2015 `_. + +Extension modules now include platform information tag in their filename on +some platforms (the tag is optional, and CPython will import extensions without +it; although if the tag is present and mismatched, the extension won't be +loaded): + +* On Linux, extension module filenames end with + ``.cpython-m--.pyd``: + + * ```` is the major number of the Python version; + for Python 3.5 this is ``3``. + + * ```` is the minor number of the Python version; + for Python 3.5 this is ``5``. + + * ```` is the hardware architecture the extension module + was built to run on. It's most commonly either ``i386`` for 32-bit Intel + platforms or ``x86_64`` for 64-bit Intel (and AMD) platforms. + + * ```` is always ``linux-gnu``, except for extensions built to + talk to the 32-bit ABI on 64-bit platforms, in which case it is + ``linux-gnu32`` (and ```` will be ``x86_64``). + +* On Windows, extension module filenames end with + ``.cp-.pyd``: + + * ```` is the major number of the Python version; + for Python 3.5 this is ``3``. + + * ```` is the minor number of the Python version; + for Python 3.5 this is ``5``. + + * ```` is the platform the extension module was built for, + either ``win32`` for Win32, ``win_amd64`` for Win64, ``win_ia64`` for + Windows Itanium 64, and ``win_arm`` for Windows on ARM. + + * If built in debug mode, ```` will be ``_d``, + otherwise it will be blank. + +* On OS X platforms, extension module filenames now end with ``-darwin.so``. + +* On all other platforms, extension module filenames are the same as they were + with Python 3.4. + Deprecated ========== @@ -1031,63 +1779,59 @@ Unsupported Operating Systems ----------------------------- -* Windows XP - Per :PEP:`11`, Microsoft support of Windows XP has ended. +Windows XP is no longer supported by Microsoft, thus, per :PEP:`11`, CPython +3.5 is no longer officially supported on this OS. Deprecated Python modules, functions and methods ------------------------------------------------ -* The :mod:`formatter` module has now graduated to full deprecation and is still - slated for removal in Python 3.6. - -* :mod:`smtpd` has in the past always decoded the DATA portion of email - messages using the ``utf-8`` codec. This can now be controlled by the new - *decode_data* keyword to :class:`~smtpd.SMTPServer`. The default value is - ``True``, but this default is deprecated. Specify the *decode_data* keyword - with an appropriate value to avoid the deprecation warning. - -* Directly assigning values to the :attr:`~http.cookies.Morsel.key`, - :attr:`~http.cookies.Morsel.value` and - :attr:`~http.cookies.Morsel.coded_value` of :class:`~http.cookies.Morsel` - objects is deprecated. Use the :func:`~http.cookies.Morsel.set` method - instead. In addition, the undocumented *LegalChars* parameter of - :func:`~http.cookies.Morsel.set` is deprecated, and is now ignored. - -* Passing a format string as keyword argument *format_string* to the - :meth:`~string.Formatter.format` method of the :class:`string.Formatter` - class has been deprecated. - -* :func:`platform.dist` and :func:`platform.linux_distribution` functions are - now deprecated and will be removed in Python 3.7. Linux distributions use - too many different ways of describing themselves, so the functionality is - left to a package. - (Contributed by Vajrasky Kok and Berker Peksag in :issue:`1322`.) - -* The previously undocumented ``from_function`` and ``from_builtin`` methods of - :class:`inspect.Signature` are deprecated. Use new - :meth:`inspect.Signature.from_callable` instead. (Contributed by Yury - Selivanov in :issue:`24248`.) - -* :func:`inspect.getargspec` is deprecated and scheduled to be removed in - Python 3.6. (See :issue:`20438` for details.) - -* :func:`~inspect.getfullargspec`, :func:`~inspect.getargvalues`, - :func:`~inspect.getcallargs`, :func:`~inspect.getargvalues`, - :func:`~inspect.formatargspec`, and :func:`~inspect.formatargvalues` are - deprecated in favor of :func:`inspect.signature` API. (See :issue:`20438` - for details.) - - -Deprecated functions and types of the C API -------------------------------------------- - -* None yet. - - -Deprecated features -------------------- - -* None yet. +The :mod:`formatter` module has now graduated to full deprecation and is still +slated for removal in Python 3.6. + +The :func:`asyncio.async` function is deprecated in favor of +:func:`~asyncio.ensure_future`. + +The :mod:`smtpd` module has in the past always decoded the DATA portion of +email messages using the ``utf-8`` codec. This can now be controlled by the +new *decode_data* keyword to :class:`~smtpd.SMTPServer`. The default value is +``True``, but this default is deprecated. Specify the *decode_data* keyword +with an appropriate value to avoid the deprecation warning. + +Directly assigning values to the :attr:`~http.cookies.Morsel.key`, +:attr:`~http.cookies.Morsel.value` and +:attr:`~http.cookies.Morsel.coded_value` of :class:`~http.cookies.Morsel` +objects is deprecated. Use the :func:`~http.cookies.Morsel.set` method +instead. In addition, the undocumented *LegalChars* parameter of +:func:`~http.cookies.Morsel.set` is deprecated, and is now ignored. + +Passing a format string as keyword argument *format_string* to the +:meth:`~string.Formatter.format` method of the :class:`string.Formatter` +class has been deprecated. + +The :func:`platform.dist` and :func:`platform.linux_distribution` functions +are now deprecated and will be removed in Python 3.7. Linux distributions use +too many different ways of describing themselves, so the functionality is +left to a package. +(Contributed by Vajrasky Kok and Berker Peksag in :issue:`1322`.) + +The previously undocumented ``from_function`` and ``from_builtin`` methods of +:class:`inspect.Signature` are deprecated. Use new +:meth:`inspect.Signature.from_callable` instead. (Contributed by Yury +Selivanov in :issue:`24248`.) + +The :func:`inspect.getargspec` function is deprecated and scheduled to be +removed in Python 3.6. (See :issue:`20438` for details.) + +The :mod:`inspect` :func:`~inspect.getfullargspec`, +:func:`~inspect.getargvalues`, :func:`~inspect.getcallargs`, +:func:`~inspect.getargvalues`, :func:`~inspect.formatargspec`, and +:func:`~inspect.formatargvalues` functions are deprecated in favor of +:func:`inspect.signature` API. +(Contributed by Yury Selivanov in :issue:`20438`.) + +Use of ``re.LOCALE`` flag with str patterns or ``re.ASCII`` is now +deprecated. (Contributed by Serhiy Storchaka in :issue:`22407`.) Removed @@ -1110,7 +1854,8 @@ * The concept of ``.pyo`` files has been removed. * The JoinableQueue class in the provisional asyncio module was deprecated - in 3.4.4 and is now removed (:issue:`23464`). + in 3.4.4 and is now removed. + (Contributed by A. Jesse Jiryu Davis in :issue:`23464`.) Porting to Python 3.5 @@ -1131,15 +1876,17 @@ error-prone and has been removed in Python 3.5. See :issue:`13936` for full details. -* :meth:`ssl.SSLSocket.send()` now raises either :exc:`ssl.SSLWantReadError` - or :exc:`ssl.SSLWantWriteError` on a non-blocking socket if the operation - would block. Previously, it would return 0. See :issue:`20951`. +* The :meth:`ssl.SSLSocket.send()` method now raises either + :exc:`ssl.SSLWantReadError` or :exc:`ssl.SSLWantWriteError` + on a non-blocking socket if the operation would block. Previously, + it would return ``0``. (Contributed by Nikolaus Rath in :issue:`20951`.) * The ``__name__`` attribute of generator is now set from the function name, instead of being set from the code name. Use ``gen.gi_code.co_name`` to retrieve the code name. Generators also have a new ``__qualname__`` attribute, the qualified name, which is now used for the representation - of a generator (``repr(gen)``). See :issue:`21205`. + of a generator (``repr(gen)``). + (Contributed by Victor Stinner in :issue:`21205`.) * The deprecated "strict" mode and argument of :class:`~html.parser.HTMLParser`, :meth:`HTMLParser.error`, and the :exc:`HTMLParserError` exception have been @@ -1150,8 +1897,8 @@ * Although it is not formally part of the API, it is worth noting for porting purposes (ie: fixing tests) that error messages that were previously of the form "'sometype' does not support the buffer protocol" are now of the form "a - bytes-like object is required, not 'sometype'". (Contributed by Ezio Melotti - in :issue:`16518`.) + :term:`bytes-like object` is required, not 'sometype'". + (Contributed by Ezio Melotti in :issue:`16518`.) * If the current directory is set to a directory that no longer exists then :exc:`FileNotFoundError` will no longer be raised and instead @@ -1170,14 +1917,14 @@ :exc:`DeprecationWarning` now, will be an error in Python 3.6). If the loader inherits from :class:`importlib.abc.Loader` then there is nothing to do, else simply define :meth:`~importlib.machinery.Loader.create_module` to return - ``None`` (:issue:`23014`). - -* :func:`re.split` always ignored empty pattern matches, so the ``'x*'`` - pattern worked the same as ``'x+'``, and the ``'\b'`` pattern never worked. - Now :func:`re.split` raises a warning if the pattern could match + ``None``. (Contributed by Brett Cannon in :issue:`23014`.) + +* The :func:`re.split` function always ignored empty pattern matches, so the + ``"x*"`` pattern worked the same as ``"x+"``, and the ``"\b"`` pattern never + worked. Now :func:`re.split` raises a warning if the pattern could match an empty string. For compatibility use patterns that never match an empty - string (e.g. ``'x+'`` instead of ``'x*'``). Patterns that could only match - an empty string (such as ``'\b'``) now raise an error. + string (e.g. ``"x+"`` instead of ``"x*"``). Patterns that could only match + an empty string (such as ``"\b"``) now raise an error. * The :class:`~http.cookies.Morsel` dict-like interface has been made self consistent: morsel comparison now takes the :attr:`~http.cookies.Morsel.key` @@ -1187,7 +1934,7 @@ :meth:`~http.cookies.Morsel.update` will now raise an exception if any of the keys in the update dictionary are invalid. In addition, the undocumented *LegalChars* parameter of :func:`~http.cookies.Morsel.set` is deprecated and - is now ignored. (:issue:`2211`) + is now ignored. (Contributed by Demian Brecht in :issue:`2211`.) * :pep:`488` has removed ``.pyo`` files from Python and introduced the optional ``opt-`` tag in ``.pyc`` file names. The @@ -1200,8 +1947,12 @@ in Python 3.5, all old `.pyo` files from previous versions of Python are invalid regardless of this PEP. -* The :mod:`socket` module now exports the CAN_RAW_FD_FRAMES constant on linux - 3.6 and greater. +* The :mod:`socket` module now exports the :data:`~socket.CAN_RAW_FD_FRAMES` + constant on linux 3.6 and greater. + +* The :func:`~ssl.cert_time_to_seconds` function now interprets the input time + as UTC and not as local time, per :rfc:`5280`. Additionally, the return + value is always an :class:`int`. (Contributed by Akira Li in :issue:`19940`.) * The ``pygettext.py`` Tool now uses the standard +NNNN format for timezones in the POT-Creation-Date header. @@ -1213,21 +1964,21 @@ * The :meth:`str.startswith` and :meth:`str.endswith` methods no longer return ``True`` when finding the empty string and the indexes are completely out of - range. See :issue:`24284`. + range. (Contributed by Serhiy Storchaka in :issue:`24284`.) * The :func:`inspect.getdoc` function now returns documentation strings inherited from base classes. Documentation strings no longer need to be duplicated if the inherited documentation is appropriate. To suppress an inherited string, an empty string must be specified (or the documentation may be filled in). This change affects the output of the :mod:`pydoc` - module and the :func:`help` function. See :issue:`15582`. + module and the :func:`help` function. + (Contributed by Serhiy Storchaka in :issue:`15582`.) Changes in the C API -------------------- * The undocumented :c:member:`~PyMemoryViewObject.format` member of the (non-public) :c:type:`PyMemoryViewObject` structure has been removed. - All extensions relying on the relevant parts in ``memoryobject.h`` must be rebuilt. @@ -1237,13 +1988,15 @@ * Removed non-documented macro :c:macro:`PyObject_REPR` which leaked references. Use format character ``%R`` in :c:func:`PyUnicode_FromFormat`-like functions to format the :func:`repr` of the object. + (Contributed by Serhiy Storchaka in :issue:`22453`.) * Because the lack of the :attr:`__module__` attribute breaks pickling and introspection, a deprecation warning now is raised for builtin type without the :attr:`__module__` attribute. Would be an AttributeError in future. - (:issue:`20204`) + (Contributed by Serhiy Storchaka in :issue:`20204`.) * As part of :pep:`492` implementation, ``tp_reserved`` slot of :c:type:`PyTypeObject` was replaced with a :c:member:`tp_as_async` slot. Refer to :ref:`coro-objects` for new types, structures and functions. + -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 13 16:46:05 2015 From: python-checkins at python.org (larry.hastings) Date: Sun, 13 Sep 2015 14:46:05 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_Final_touch-up?= =?utf-8?q?s_for_the_What=27s_New_In_Python_3=2E5_document=2E?= Message-ID: <20150913144604.15724.84435@psf.io> https://hg.python.org/cpython/rev/374f501f4567 changeset: 97982:374f501f4567 branch: 3.5 tag: v3.5.0 user: Larry Hastings date: Sat Sep 12 17:36:44 2015 +0100 summary: Final touch-ups for the What's New In Python 3.5 document. files: Doc/whatsnew/3.5.rst | 12 ++++-------- 1 files changed, 4 insertions(+), 8 deletions(-) diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -46,15 +46,11 @@ This saves the maintainer the effort of going through the Mercurial log when researching a change. +Python 3.5 was released on September 13, 2015. + This article explains the new features in Python 3.5, compared to 3.4. -For full details, see the :source:`Misc/NEWS` file. - -.. note:: - - Prerelease users should be aware that this document is currently in draft - form. It will be updated substantially as Python 3.5 moves towards release, - so it's worth checking back even after reading earlier versions. - +For full details, see the +`changelog `_. .. seealso:: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 13 16:46:05 2015 From: python-checkins at python.org (larry.hastings) Date: Sun, 13 Sep 2015 14:46:05 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_Regenerate_pyd?= =?utf-8?q?oc_topics=2C_fix_minor_non-RST_formatting_in_Misc/NEWS=2E?= Message-ID: <20150913144604.15718.54301@psf.io> https://hg.python.org/cpython/rev/61e6f9dbbcfd changeset: 97980:61e6f9dbbcfd branch: 3.5 user: Larry Hastings date: Sat Sep 12 17:24:02 2015 +0100 summary: Regenerate pydoc topics, fix minor non-RST formatting in Misc/NEWS. files: Lib/pydoc_data/topics.py | 12958 ++++++++++++++++++++++++- Misc/NEWS | 2 +- 2 files changed, 12881 insertions(+), 79 deletions(-) diff --git a/Lib/pydoc_data/topics.py b/Lib/pydoc_data/topics.py --- a/Lib/pydoc_data/topics.py +++ b/Lib/pydoc_data/topics.py @@ -1,79 +1,12881 @@ # -*- coding: utf-8 -*- -# Autogenerated by Sphinx on Mon Sep 7 05:10:25 2015 -topics = {'assert': u'\nThe "assert" statement\n**********************\n\nAssert statements are a convenient way to insert debugging assertions\ninto a program:\n\n assert_stmt ::= "assert" expression ["," expression]\n\nThe simple form, "assert expression", is equivalent to\n\n if __debug__:\n if not expression: raise AssertionError\n\nThe extended form, "assert expression1, expression2", is equivalent to\n\n if __debug__:\n if not expression1: raise AssertionError(expression2)\n\nThese equivalences assume that "__debug__" and "AssertionError" refer\nto the built-in variables with those names. In the current\nimplementation, the built-in variable "__debug__" is "True" under\nnormal circumstances, "False" when optimization is requested (command\nline option -O). The current code generator emits no code for an\nassert statement when optimization is requested at compile time. Note\nthat it is unnecessary to include the source code for the expression\nthat failed in the error message; it will be displayed as part of the\nstack trace.\n\nAssignments to "__debug__" are illegal. The value for the built-in\nvariable is determined when the interpreter starts.\n', - 'assignment': u'\nAssignment statements\n*********************\n\nAssignment statements are used to (re)bind names to values and to\nmodify attributes or items of mutable objects:\n\n assignment_stmt ::= (target_list "=")+ (expression_list | yield_expression)\n target_list ::= target ("," target)* [","]\n target ::= identifier\n | "(" target_list ")"\n | "[" target_list "]"\n | attributeref\n | subscription\n | slicing\n | "*" target\n\n(See section *Primaries* for the syntax definitions for\n*attributeref*, *subscription*, and *slicing*.)\n\nAn assignment statement evaluates the expression list (remember that\nthis can be a single expression or a comma-separated list, the latter\nyielding a tuple) and assigns the single resulting object to each of\nthe target lists, from left to right.\n\nAssignment is defined recursively depending on the form of the target\n(list). When a target is part of a mutable object (an attribute\nreference, subscription or slicing), the mutable object must\nultimately perform the assignment and decide about its validity, and\nmay raise an exception if the assignment is unacceptable. The rules\nobserved by various types and the exceptions raised are given with the\ndefinition of the object types (see section *The standard type\nhierarchy*).\n\nAssignment of an object to a target list, optionally enclosed in\nparentheses or square brackets, is recursively defined as follows.\n\n* If the target list is a single target: The object is assigned to\n that target.\n\n* If the target list is a comma-separated list of targets: The\n object must be an iterable with the same number of items as there\n are targets in the target list, and the items are assigned, from\n left to right, to the corresponding targets.\n\n * If the target list contains one target prefixed with an\n asterisk, called a "starred" target: The object must be a sequence\n with at least as many items as there are targets in the target\n list, minus one. The first items of the sequence are assigned,\n from left to right, to the targets before the starred target. The\n final items of the sequence are assigned to the targets after the\n starred target. A list of the remaining items in the sequence is\n then assigned to the starred target (the list can be empty).\n\n * Else: The object must be a sequence with the same number of\n items as there are targets in the target list, and the items are\n assigned, from left to right, to the corresponding targets.\n\nAssignment of an object to a single target is recursively defined as\nfollows.\n\n* If the target is an identifier (name):\n\n * If the name does not occur in a "global" or "nonlocal" statement\n in the current code block: the name is bound to the object in the\n current local namespace.\n\n * Otherwise: the name is bound to the object in the global\n namespace or the outer namespace determined by "nonlocal",\n respectively.\n\n The name is rebound if it was already bound. This may cause the\n reference count for the object previously bound to the name to reach\n zero, causing the object to be deallocated and its destructor (if it\n has one) to be called.\n\n* If the target is a target list enclosed in parentheses or in\n square brackets: The object must be an iterable with the same number\n of items as there are targets in the target list, and its items are\n assigned, from left to right, to the corresponding targets.\n\n* If the target is an attribute reference: The primary expression in\n the reference is evaluated. It should yield an object with\n assignable attributes; if this is not the case, "TypeError" is\n raised. That object is then asked to assign the assigned object to\n the given attribute; if it cannot perform the assignment, it raises\n an exception (usually but not necessarily "AttributeError").\n\n Note: If the object is a class instance and the attribute reference\n occurs on both sides of the assignment operator, the RHS expression,\n "a.x" can access either an instance attribute or (if no instance\n attribute exists) a class attribute. The LHS target "a.x" is always\n set as an instance attribute, creating it if necessary. Thus, the\n two occurrences of "a.x" do not necessarily refer to the same\n attribute: if the RHS expression refers to a class attribute, the\n LHS creates a new instance attribute as the target of the\n assignment:\n\n class Cls:\n x = 3 # class variable\n inst = Cls()\n inst.x = inst.x + 1 # writes inst.x as 4 leaving Cls.x as 3\n\n This description does not necessarily apply to descriptor\n attributes, such as properties created with "property()".\n\n* If the target is a subscription: The primary expression in the\n reference is evaluated. It should yield either a mutable sequence\n object (such as a list) or a mapping object (such as a dictionary).\n Next, the subscript expression is evaluated.\n\n If the primary is a mutable sequence object (such as a list), the\n subscript must yield an integer. If it is negative, the sequence\'s\n length is added to it. The resulting value must be a nonnegative\n integer less than the sequence\'s length, and the sequence is asked\n to assign the assigned object to its item with that index. If the\n index is out of range, "IndexError" is raised (assignment to a\n subscripted sequence cannot add new items to a list).\n\n If the primary is a mapping object (such as a dictionary), the\n subscript must have a type compatible with the mapping\'s key type,\n and the mapping is then asked to create a key/datum pair which maps\n the subscript to the assigned object. This can either replace an\n existing key/value pair with the same key value, or insert a new\n key/value pair (if no key with the same value existed).\n\n For user-defined objects, the "__setitem__()" method is called with\n appropriate arguments.\n\n* If the target is a slicing: The primary expression in the\n reference is evaluated. It should yield a mutable sequence object\n (such as a list). The assigned object should be a sequence object\n of the same type. Next, the lower and upper bound expressions are\n evaluated, insofar they are present; defaults are zero and the\n sequence\'s length. The bounds should evaluate to integers. If\n either bound is negative, the sequence\'s length is added to it. The\n resulting bounds are clipped to lie between zero and the sequence\'s\n length, inclusive. Finally, the sequence object is asked to replace\n the slice with the items of the assigned sequence. The length of\n the slice may be different from the length of the assigned sequence,\n thus changing the length of the target sequence, if the target\n sequence allows it.\n\n**CPython implementation detail:** In the current implementation, the\nsyntax for targets is taken to be the same as for expressions, and\ninvalid syntax is rejected during the code generation phase, causing\nless detailed error messages.\n\nAlthough the definition of assignment implies that overlaps between\nthe left-hand side and the right-hand side are \'simultanenous\' (for\nexample "a, b = b, a" swaps two variables), overlaps *within* the\ncollection of assigned-to variables occur left-to-right, sometimes\nresulting in confusion. For instance, the following program prints\n"[0, 2]":\n\n x = [0, 1]\n i = 0\n i, x[i] = 1, 2 # i is updated, then x[i] is updated\n print(x)\n\nSee also: **PEP 3132** - Extended Iterable Unpacking\n\n The specification for the "*target" feature.\n\n\nAugmented assignment statements\n===============================\n\nAugmented assignment is the combination, in a single statement, of a\nbinary operation and an assignment statement:\n\n augmented_assignment_stmt ::= augtarget augop (expression_list | yield_expression)\n augtarget ::= identifier | attributeref | subscription | slicing\n augop ::= "+=" | "-=" | "*=" | "@=" | "/=" | "//=" | "%=" | "**="\n | ">>=" | "<<=" | "&=" | "^=" | "|="\n\n(See section *Primaries* for the syntax definitions of the last three\nsymbols.)\n\nAn augmented assignment evaluates the target (which, unlike normal\nassignment statements, cannot be an unpacking) and the expression\nlist, performs the binary operation specific to the type of assignment\non the two operands, and assigns the result to the original target.\nThe target is only evaluated once.\n\nAn augmented assignment expression like "x += 1" can be rewritten as\n"x = x + 1" to achieve a similar, but not exactly equal effect. In the\naugmented version, "x" is only evaluated once. Also, when possible,\nthe actual operation is performed *in-place*, meaning that rather than\ncreating a new object and assigning that to the target, the old object\nis modified instead.\n\nUnlike normal assignments, augmented assignments evaluate the left-\nhand side *before* evaluating the right-hand side. For example, "a[i]\n+= f(x)" first looks-up "a[i]", then it evaluates "f(x)" and performs\nthe addition, and lastly, it writes the result back to "a[i]".\n\nWith the exception of assigning to tuples and multiple targets in a\nsingle statement, the assignment done by augmented assignment\nstatements is handled the same way as normal assignments. Similarly,\nwith the exception of the possible *in-place* behavior, the binary\noperation performed by augmented assignment is the same as the normal\nbinary operations.\n\nFor targets which are attribute references, the same *caveat about\nclass and instance attributes* applies as for regular assignments.\n', - 'atom-identifiers': u'\nIdentifiers (Names)\n*******************\n\nAn identifier occurring as an atom is a name. See section\n*Identifiers and keywords* for lexical definition and section *Naming\nand binding* for documentation of naming and binding.\n\nWhen the name is bound to an object, evaluation of the atom yields\nthat object. When a name is not bound, an attempt to evaluate it\nraises a "NameError" exception.\n\n**Private name mangling:** When an identifier that textually occurs in\na class definition begins with two or more underscore characters and\ndoes not end in two or more underscores, it is considered a *private\nname* of that class. Private names are transformed to a longer form\nbefore code is generated for them. The transformation inserts the\nclass name, with leading underscores removed and a single underscore\ninserted, in front of the name. For example, the identifier "__spam"\noccurring in a class named "Ham" will be transformed to "_Ham__spam".\nThis transformation is independent of the syntactical context in which\nthe identifier is used. If the transformed name is extremely long\n(longer than 255 characters), implementation defined truncation may\nhappen. If the class name consists only of underscores, no\ntransformation is done.\n', - 'atom-literals': u"\nLiterals\n********\n\nPython supports string and bytes literals and various numeric\nliterals:\n\n literal ::= stringliteral | bytesliteral\n | integer | floatnumber | imagnumber\n\nEvaluation of a literal yields an object of the given type (string,\nbytes, integer, floating point number, complex number) with the given\nvalue. The value may be approximated in the case of floating point\nand imaginary (complex) literals. See section *Literals* for details.\n\nAll literals correspond to immutable data types, and hence the\nobject's identity is less important than its value. Multiple\nevaluations of literals with the same value (either the same\noccurrence in the program text or a different occurrence) may obtain\nthe same object or a different object with the same value.\n", - 'attribute-access': u'\nCustomizing attribute access\n****************************\n\nThe following methods can be defined to customize the meaning of\nattribute access (use of, assignment to, or deletion of "x.name") for\nclass instances.\n\nobject.__getattr__(self, name)\n\n Called when an attribute lookup has not found the attribute in the\n usual places (i.e. it is not an instance attribute nor is it found\n in the class tree for "self"). "name" is the attribute name. This\n method should return the (computed) attribute value or raise an\n "AttributeError" exception.\n\n Note that if the attribute is found through the normal mechanism,\n "__getattr__()" is not called. (This is an intentional asymmetry\n between "__getattr__()" and "__setattr__()".) This is done both for\n efficiency reasons and because otherwise "__getattr__()" would have\n no way to access other attributes of the instance. Note that at\n least for instance variables, you can fake total control by not\n inserting any values in the instance attribute dictionary (but\n instead inserting them in another object). See the\n "__getattribute__()" method below for a way to actually get total\n control over attribute access.\n\nobject.__getattribute__(self, name)\n\n Called unconditionally to implement attribute accesses for\n instances of the class. If the class also defines "__getattr__()",\n the latter will not be called unless "__getattribute__()" either\n calls it explicitly or raises an "AttributeError". This method\n should return the (computed) attribute value or raise an\n "AttributeError" exception. In order to avoid infinite recursion in\n this method, its implementation should always call the base class\n method with the same name to access any attributes it needs, for\n example, "object.__getattribute__(self, name)".\n\n Note: This method may still be bypassed when looking up special\n methods as the result of implicit invocation via language syntax\n or built-in functions. See *Special method lookup*.\n\nobject.__setattr__(self, name, value)\n\n Called when an attribute assignment is attempted. This is called\n instead of the normal mechanism (i.e. store the value in the\n instance dictionary). *name* is the attribute name, *value* is the\n value to be assigned to it.\n\n If "__setattr__()" wants to assign to an instance attribute, it\n should call the base class method with the same name, for example,\n "object.__setattr__(self, name, value)".\n\nobject.__delattr__(self, name)\n\n Like "__setattr__()" but for attribute deletion instead of\n assignment. This should only be implemented if "del obj.name" is\n meaningful for the object.\n\nobject.__dir__(self)\n\n Called when "dir()" is called on the object. A sequence must be\n returned. "dir()" converts the returned sequence to a list and\n sorts it.\n\n\nImplementing Descriptors\n========================\n\nThe following methods only apply when an instance of the class\ncontaining the method (a so-called *descriptor* class) appears in an\n*owner* class (the descriptor must be in either the owner\'s class\ndictionary or in the class dictionary for one of its parents). In the\nexamples below, "the attribute" refers to the attribute whose name is\nthe key of the property in the owner class\' "__dict__".\n\nobject.__get__(self, instance, owner)\n\n Called to get the attribute of the owner class (class attribute\n access) or of an instance of that class (instance attribute\n access). *owner* is always the owner class, while *instance* is the\n instance that the attribute was accessed through, or "None" when\n the attribute is accessed through the *owner*. This method should\n return the (computed) attribute value or raise an "AttributeError"\n exception.\n\nobject.__set__(self, instance, value)\n\n Called to set the attribute on an instance *instance* of the owner\n class to a new value, *value*.\n\nobject.__delete__(self, instance)\n\n Called to delete the attribute on an instance *instance* of the\n owner class.\n\nThe attribute "__objclass__" is interpreted by the "inspect" module as\nspecifying the class where this object was defined (setting this\nappropriately can assist in runtime introspection of dynamic class\nattributes). For callables, it may indicate that an instance of the\ngiven type (or a subclass) is expected or required as the first\npositional argument (for example, CPython sets this attribute for\nunbound methods that are implemented in C).\n\n\nInvoking Descriptors\n====================\n\nIn general, a descriptor is an object attribute with "binding\nbehavior", one whose attribute access has been overridden by methods\nin the descriptor protocol: "__get__()", "__set__()", and\n"__delete__()". If any of those methods are defined for an object, it\nis said to be a descriptor.\n\nThe default behavior for attribute access is to get, set, or delete\nthe attribute from an object\'s dictionary. For instance, "a.x" has a\nlookup chain starting with "a.__dict__[\'x\']", then\n"type(a).__dict__[\'x\']", and continuing through the base classes of\n"type(a)" excluding metaclasses.\n\nHowever, if the looked-up value is an object defining one of the\ndescriptor methods, then Python may override the default behavior and\ninvoke the descriptor method instead. Where this occurs in the\nprecedence chain depends on which descriptor methods were defined and\nhow they were called.\n\nThe starting point for descriptor invocation is a binding, "a.x". How\nthe arguments are assembled depends on "a":\n\nDirect Call\n The simplest and least common call is when user code directly\n invokes a descriptor method: "x.__get__(a)".\n\nInstance Binding\n If binding to an object instance, "a.x" is transformed into the\n call: "type(a).__dict__[\'x\'].__get__(a, type(a))".\n\nClass Binding\n If binding to a class, "A.x" is transformed into the call:\n "A.__dict__[\'x\'].__get__(None, A)".\n\nSuper Binding\n If "a" is an instance of "super", then the binding "super(B,\n obj).m()" searches "obj.__class__.__mro__" for the base class "A"\n immediately preceding "B" and then invokes the descriptor with the\n call: "A.__dict__[\'m\'].__get__(obj, obj.__class__)".\n\nFor instance bindings, the precedence of descriptor invocation depends\non the which descriptor methods are defined. A descriptor can define\nany combination of "__get__()", "__set__()" and "__delete__()". If it\ndoes not define "__get__()", then accessing the attribute will return\nthe descriptor object itself unless there is a value in the object\'s\ninstance dictionary. If the descriptor defines "__set__()" and/or\n"__delete__()", it is a data descriptor; if it defines neither, it is\na non-data descriptor. Normally, data descriptors define both\n"__get__()" and "__set__()", while non-data descriptors have just the\n"__get__()" method. Data descriptors with "__set__()" and "__get__()"\ndefined always override a redefinition in an instance dictionary. In\ncontrast, non-data descriptors can be overridden by instances.\n\nPython methods (including "staticmethod()" and "classmethod()") are\nimplemented as non-data descriptors. Accordingly, instances can\nredefine and override methods. This allows individual instances to\nacquire behaviors that differ from other instances of the same class.\n\nThe "property()" function is implemented as a data descriptor.\nAccordingly, instances cannot override the behavior of a property.\n\n\n__slots__\n=========\n\nBy default, instances of classes have a dictionary for attribute\nstorage. This wastes space for objects having very few instance\nvariables. The space consumption can become acute when creating large\nnumbers of instances.\n\nThe default can be overridden by defining *__slots__* in a class\ndefinition. The *__slots__* declaration takes a sequence of instance\nvariables and reserves just enough space in each instance to hold a\nvalue for each variable. Space is saved because *__dict__* is not\ncreated for each instance.\n\nobject.__slots__\n\n This class variable can be assigned a string, iterable, or sequence\n of strings with variable names used by instances. *__slots__*\n reserves space for the declared variables and prevents the\n automatic creation of *__dict__* and *__weakref__* for each\n instance.\n\n\nNotes on using *__slots__*\n--------------------------\n\n* When inheriting from a class without *__slots__*, the *__dict__*\n attribute of that class will always be accessible, so a *__slots__*\n definition in the subclass is meaningless.\n\n* Without a *__dict__* variable, instances cannot be assigned new\n variables not listed in the *__slots__* definition. Attempts to\n assign to an unlisted variable name raises "AttributeError". If\n dynamic assignment of new variables is desired, then add\n "\'__dict__\'" to the sequence of strings in the *__slots__*\n declaration.\n\n* Without a *__weakref__* variable for each instance, classes\n defining *__slots__* do not support weak references to its\n instances. If weak reference support is needed, then add\n "\'__weakref__\'" to the sequence of strings in the *__slots__*\n declaration.\n\n* *__slots__* are implemented at the class level by creating\n descriptors (*Implementing Descriptors*) for each variable name. As\n a result, class attributes cannot be used to set default values for\n instance variables defined by *__slots__*; otherwise, the class\n attribute would overwrite the descriptor assignment.\n\n* The action of a *__slots__* declaration is limited to the class\n where it is defined. As a result, subclasses will have a *__dict__*\n unless they also define *__slots__* (which must only contain names\n of any *additional* slots).\n\n* If a class defines a slot also defined in a base class, the\n instance variable defined by the base class slot is inaccessible\n (except by retrieving its descriptor directly from the base class).\n This renders the meaning of the program undefined. In the future, a\n check may be added to prevent this.\n\n* Nonempty *__slots__* does not work for classes derived from\n "variable-length" built-in types such as "int", "bytes" and "tuple".\n\n* Any non-string iterable may be assigned to *__slots__*. Mappings\n may also be used; however, in the future, special meaning may be\n assigned to the values corresponding to each key.\n\n* *__class__* assignment works only if both classes have the same\n *__slots__*.\n', - 'attribute-references': u'\nAttribute references\n********************\n\nAn attribute reference is a primary followed by a period and a name:\n\n attributeref ::= primary "." identifier\n\nThe primary must evaluate to an object of a type that supports\nattribute references, which most objects do. This object is then\nasked to produce the attribute whose name is the identifier. This\nproduction can be customized by overriding the "__getattr__()" method.\nIf this attribute is not available, the exception "AttributeError" is\nraised. Otherwise, the type and value of the object produced is\ndetermined by the object. Multiple evaluations of the same attribute\nreference may yield different objects.\n', - 'augassign': u'\nAugmented assignment statements\n*******************************\n\nAugmented assignment is the combination, in a single statement, of a\nbinary operation and an assignment statement:\n\n augmented_assignment_stmt ::= augtarget augop (expression_list | yield_expression)\n augtarget ::= identifier | attributeref | subscription | slicing\n augop ::= "+=" | "-=" | "*=" | "@=" | "/=" | "//=" | "%=" | "**="\n | ">>=" | "<<=" | "&=" | "^=" | "|="\n\n(See section *Primaries* for the syntax definitions of the last three\nsymbols.)\n\nAn augmented assignment evaluates the target (which, unlike normal\nassignment statements, cannot be an unpacking) and the expression\nlist, performs the binary operation specific to the type of assignment\non the two operands, and assigns the result to the original target.\nThe target is only evaluated once.\n\nAn augmented assignment expression like "x += 1" can be rewritten as\n"x = x + 1" to achieve a similar, but not exactly equal effect. In the\naugmented version, "x" is only evaluated once. Also, when possible,\nthe actual operation is performed *in-place*, meaning that rather than\ncreating a new object and assigning that to the target, the old object\nis modified instead.\n\nUnlike normal assignments, augmented assignments evaluate the left-\nhand side *before* evaluating the right-hand side. For example, "a[i]\n+= f(x)" first looks-up "a[i]", then it evaluates "f(x)" and performs\nthe addition, and lastly, it writes the result back to "a[i]".\n\nWith the exception of assigning to tuples and multiple targets in a\nsingle statement, the assignment done by augmented assignment\nstatements is handled the same way as normal assignments. Similarly,\nwith the exception of the possible *in-place* behavior, the binary\noperation performed by augmented assignment is the same as the normal\nbinary operations.\n\nFor targets which are attribute references, the same *caveat about\nclass and instance attributes* applies as for regular assignments.\n', - 'binary': u'\nBinary arithmetic operations\n****************************\n\nThe binary arithmetic operations have the conventional priority\nlevels. Note that some of these operations also apply to certain non-\nnumeric types. Apart from the power operator, there are only two\nlevels, one for multiplicative operators and one for additive\noperators:\n\n m_expr ::= u_expr | m_expr "*" u_expr | m_expr "@" m_expr |\n m_expr "//" u_expr| m_expr "/" u_expr |\n m_expr "%" u_expr\n a_expr ::= m_expr | a_expr "+" m_expr | a_expr "-" m_expr\n\nThe "*" (multiplication) operator yields the product of its arguments.\nThe arguments must either both be numbers, or one argument must be an\ninteger and the other must be a sequence. In the former case, the\nnumbers are converted to a common type and then multiplied together.\nIn the latter case, sequence repetition is performed; a negative\nrepetition factor yields an empty sequence.\n\nThe "@" (at) operator is intended to be used for matrix\nmultiplication. No builtin Python types implement this operator.\n\nNew in version 3.5.\n\nThe "/" (division) and "//" (floor division) operators yield the\nquotient of their arguments. The numeric arguments are first\nconverted to a common type. Division of integers yields a float, while\nfloor division of integers results in an integer; the result is that\nof mathematical division with the \'floor\' function applied to the\nresult. Division by zero raises the "ZeroDivisionError" exception.\n\nThe "%" (modulo) operator yields the remainder from the division of\nthe first argument by the second. The numeric arguments are first\nconverted to a common type. A zero right argument raises the\n"ZeroDivisionError" exception. The arguments may be floating point\nnumbers, e.g., "3.14%0.7" equals "0.34" (since "3.14" equals "4*0.7 +\n0.34".) The modulo operator always yields a result with the same sign\nas its second operand (or zero); the absolute value of the result is\nstrictly smaller than the absolute value of the second operand [1].\n\nThe floor division and modulo operators are connected by the following\nidentity: "x == (x//y)*y + (x%y)". Floor division and modulo are also\nconnected with the built-in function "divmod()": "divmod(x, y) ==\n(x//y, x%y)". [2].\n\nIn addition to performing the modulo operation on numbers, the "%"\noperator is also overloaded by string objects to perform old-style\nstring formatting (also known as interpolation). The syntax for\nstring formatting is described in the Python Library Reference,\nsection *printf-style String Formatting*.\n\nThe floor division operator, the modulo operator, and the "divmod()"\nfunction are not defined for complex numbers. Instead, convert to a\nfloating point number using the "abs()" function if appropriate.\n\nThe "+" (addition) operator yields the sum of its arguments. The\narguments must either both be numbers or both be sequences of the same\ntype. In the former case, the numbers are converted to a common type\nand then added together. In the latter case, the sequences are\nconcatenated.\n\nThe "-" (subtraction) operator yields the difference of its arguments.\nThe numeric arguments are first converted to a common type.\n', - 'bitwise': u'\nBinary bitwise operations\n*************************\n\nEach of the three bitwise operations has a different priority level:\n\n and_expr ::= shift_expr | and_expr "&" shift_expr\n xor_expr ::= and_expr | xor_expr "^" and_expr\n or_expr ::= xor_expr | or_expr "|" xor_expr\n\nThe "&" operator yields the bitwise AND of its arguments, which must\nbe integers.\n\nThe "^" operator yields the bitwise XOR (exclusive OR) of its\narguments, which must be integers.\n\nThe "|" operator yields the bitwise (inclusive) OR of its arguments,\nwhich must be integers.\n', - 'bltin-code-objects': u'\nCode Objects\n************\n\nCode objects are used by the implementation to represent "pseudo-\ncompiled" executable Python code such as a function body. They differ\nfrom function objects because they don\'t contain a reference to their\nglobal execution environment. Code objects are returned by the built-\nin "compile()" function and can be extracted from function objects\nthrough their "__code__" attribute. See also the "code" module.\n\nA code object can be executed or evaluated by passing it (instead of a\nsource string) to the "exec()" or "eval()" built-in functions.\n\nSee *The standard type hierarchy* for more information.\n', - 'bltin-ellipsis-object': u'\nThe Ellipsis Object\n*******************\n\nThis object is commonly used by slicing (see *Slicings*). It supports\nno special operations. There is exactly one ellipsis object, named\n"Ellipsis" (a built-in name). "type(Ellipsis)()" produces the\n"Ellipsis" singleton.\n\nIt is written as "Ellipsis" or "...".\n', - 'bltin-null-object': u'\nThe Null Object\n***************\n\nThis object is returned by functions that don\'t explicitly return a\nvalue. It supports no special operations. There is exactly one null\nobject, named "None" (a built-in name). "type(None)()" produces the\nsame singleton.\n\nIt is written as "None".\n', - 'bltin-type-objects': u'\nType Objects\n************\n\nType objects represent the various object types. An object\'s type is\naccessed by the built-in function "type()". There are no special\noperations on types. The standard module "types" defines names for\nall standard built-in types.\n\nTypes are written like this: "".\n', - 'booleans': u'\nBoolean operations\n******************\n\n or_test ::= and_test | or_test "or" and_test\n and_test ::= not_test | and_test "and" not_test\n not_test ::= comparison | "not" not_test\n\nIn the context of Boolean operations, and also when expressions are\nused by control flow statements, the following values are interpreted\nas false: "False", "None", numeric zero of all types, and empty\nstrings and containers (including strings, tuples, lists,\ndictionaries, sets and frozensets). All other values are interpreted\nas true. User-defined objects can customize their truth value by\nproviding a "__bool__()" method.\n\nThe operator "not" yields "True" if its argument is false, "False"\notherwise.\n\nThe expression "x and y" first evaluates *x*; if *x* is false, its\nvalue is returned; otherwise, *y* is evaluated and the resulting value\nis returned.\n\nThe expression "x or y" first evaluates *x*; if *x* is true, its value\nis returned; otherwise, *y* is evaluated and the resulting value is\nreturned.\n\n(Note that neither "and" nor "or" restrict the value and type they\nreturn to "False" and "True", but rather return the last evaluated\nargument. This is sometimes useful, e.g., if "s" is a string that\nshould be replaced by a default value if it is empty, the expression\n"s or \'foo\'" yields the desired value. Because "not" has to create a\nnew value, it returns a boolean value regardless of the type of its\nargument (for example, "not \'foo\'" produces "False" rather than "\'\'".)\n', - 'break': u'\nThe "break" statement\n*********************\n\n break_stmt ::= "break"\n\n"break" may only occur syntactically nested in a "for" or "while"\nloop, but not nested in a function or class definition within that\nloop.\n\nIt terminates the nearest enclosing loop, skipping the optional "else"\nclause if the loop has one.\n\nIf a "for" loop is terminated by "break", the loop control target\nkeeps its current value.\n\nWhen "break" passes control out of a "try" statement with a "finally"\nclause, that "finally" clause is executed before really leaving the\nloop.\n', - 'callable-types': u'\nEmulating callable objects\n**************************\n\nobject.__call__(self[, args...])\n\n Called when the instance is "called" as a function; if this method\n is defined, "x(arg1, arg2, ...)" is a shorthand for\n "x.__call__(arg1, arg2, ...)".\n', - 'calls': u'\nCalls\n*****\n\nA call calls a callable object (e.g., a *function*) with a possibly\nempty series of *arguments*:\n\n call ::= primary "(" [argument_list [","] | comprehension] ")"\n argument_list ::= positional_arguments ["," keyword_arguments]\n ["," "*" expression] ["," keyword_arguments]\n ["," "**" expression]\n | keyword_arguments ["," "*" expression]\n ["," keyword_arguments] ["," "**" expression]\n | "*" expression ["," keyword_arguments] ["," "**" expression]\n | "**" expression\n positional_arguments ::= expression ("," expression)*\n keyword_arguments ::= keyword_item ("," keyword_item)*\n keyword_item ::= identifier "=" expression\n\nAn optional trailing comma may be present after the positional and\nkeyword arguments but does not affect the semantics.\n\nThe primary must evaluate to a callable object (user-defined\nfunctions, built-in functions, methods of built-in objects, class\nobjects, methods of class instances, and all objects having a\n"__call__()" method are callable). All argument expressions are\nevaluated before the call is attempted. Please refer to section\n*Function definitions* for the syntax of formal *parameter* lists.\n\nIf keyword arguments are present, they are first converted to\npositional arguments, as follows. First, a list of unfilled slots is\ncreated for the formal parameters. If there are N positional\narguments, they are placed in the first N slots. Next, for each\nkeyword argument, the identifier is used to determine the\ncorresponding slot (if the identifier is the same as the first formal\nparameter name, the first slot is used, and so on). If the slot is\nalready filled, a "TypeError" exception is raised. Otherwise, the\nvalue of the argument is placed in the slot, filling it (even if the\nexpression is "None", it fills the slot). When all arguments have\nbeen processed, the slots that are still unfilled are filled with the\ncorresponding default value from the function definition. (Default\nvalues are calculated, once, when the function is defined; thus, a\nmutable object such as a list or dictionary used as default value will\nbe shared by all calls that don\'t specify an argument value for the\ncorresponding slot; this should usually be avoided.) If there are any\nunfilled slots for which no default value is specified, a "TypeError"\nexception is raised. Otherwise, the list of filled slots is used as\nthe argument list for the call.\n\n**CPython implementation detail:** An implementation may provide\nbuilt-in functions whose positional parameters do not have names, even\nif they are \'named\' for the purpose of documentation, and which\ntherefore cannot be supplied by keyword. In CPython, this is the case\nfor functions implemented in C that use "PyArg_ParseTuple()" to parse\ntheir arguments.\n\nIf there are more positional arguments than there are formal parameter\nslots, a "TypeError" exception is raised, unless a formal parameter\nusing the syntax "*identifier" is present; in this case, that formal\nparameter receives a tuple containing the excess positional arguments\n(or an empty tuple if there were no excess positional arguments).\n\nIf any keyword argument does not correspond to a formal parameter\nname, a "TypeError" exception is raised, unless a formal parameter\nusing the syntax "**identifier" is present; in this case, that formal\nparameter receives a dictionary containing the excess keyword\narguments (using the keywords as keys and the argument values as\ncorresponding values), or a (new) empty dictionary if there were no\nexcess keyword arguments.\n\nIf the syntax "*expression" appears in the function call, "expression"\nmust evaluate to an iterable. Elements from this iterable are treated\nas if they were additional positional arguments; if there are\npositional arguments *x1*, ..., *xN*, and "expression" evaluates to a\nsequence *y1*, ..., *yM*, this is equivalent to a call with M+N\npositional arguments *x1*, ..., *xN*, *y1*, ..., *yM*.\n\nA consequence of this is that although the "*expression" syntax may\nappear *after* some keyword arguments, it is processed *before* the\nkeyword arguments (and the "**expression" argument, if any -- see\nbelow). So:\n\n >>> def f(a, b):\n ... print(a, b)\n ...\n >>> f(b=1, *(2,))\n 2 1\n >>> f(a=1, *(2,))\n Traceback (most recent call last):\n File "", line 1, in ?\n TypeError: f() got multiple values for keyword argument \'a\'\n >>> f(1, *(2,))\n 1 2\n\nIt is unusual for both keyword arguments and the "*expression" syntax\nto be used in the same call, so in practice this confusion does not\narise.\n\nIf the syntax "**expression" appears in the function call,\n"expression" must evaluate to a mapping, the contents of which are\ntreated as additional keyword arguments. In the case of a keyword\nappearing in both "expression" and as an explicit keyword argument, a\n"TypeError" exception is raised.\n\nFormal parameters using the syntax "*identifier" or "**identifier"\ncannot be used as positional argument slots or as keyword argument\nnames.\n\nA call always returns some value, possibly "None", unless it raises an\nexception. How this value is computed depends on the type of the\ncallable object.\n\nIf it is---\n\na user-defined function:\n The code block for the function is executed, passing it the\n argument list. The first thing the code block will do is bind the\n formal parameters to the arguments; this is described in section\n *Function definitions*. When the code block executes a "return"\n statement, this specifies the return value of the function call.\n\na built-in function or method:\n The result is up to the interpreter; see *Built-in Functions* for\n the descriptions of built-in functions and methods.\n\na class object:\n A new instance of that class is returned.\n\na class instance method:\n The corresponding user-defined function is called, with an argument\n list that is one longer than the argument list of the call: the\n instance becomes the first argument.\n\na class instance:\n The class must define a "__call__()" method; the effect is then the\n same as if that method was called.\n', - 'class': u'\nClass definitions\n*****************\n\nA class definition defines a class object (see section *The standard\ntype hierarchy*):\n\n classdef ::= [decorators] "class" classname [inheritance] ":" suite\n inheritance ::= "(" [parameter_list] ")"\n classname ::= identifier\n\nA class definition is an executable statement. The inheritance list\nusually gives a list of base classes (see *Customizing class creation*\nfor more advanced uses), so each item in the list should evaluate to a\nclass object which allows subclassing. Classes without an inheritance\nlist inherit, by default, from the base class "object"; hence,\n\n class Foo:\n pass\n\nis equivalent to\n\n class Foo(object):\n pass\n\nThe class\'s suite is then executed in a new execution frame (see\n*Naming and binding*), using a newly created local namespace and the\noriginal global namespace. (Usually, the suite contains mostly\nfunction definitions.) When the class\'s suite finishes execution, its\nexecution frame is discarded but its local namespace is saved. [4] A\nclass object is then created using the inheritance list for the base\nclasses and the saved local namespace for the attribute dictionary.\nThe class name is bound to this class object in the original local\nnamespace.\n\nClass creation can be customized heavily using *metaclasses*.\n\nClasses can also be decorated: just like when decorating functions,\n\n @f1(arg)\n @f2\n class Foo: pass\n\nis equivalent to\n\n class Foo: pass\n Foo = f1(arg)(f2(Foo))\n\nThe evaluation rules for the decorator expressions are the same as for\nfunction decorators. The result must be a class object, which is then\nbound to the class name.\n\n**Programmer\'s note:** Variables defined in the class definition are\nclass attributes; they are shared by instances. Instance attributes\ncan be set in a method with "self.name = value". Both class and\ninstance attributes are accessible through the notation ""self.name"",\nand an instance attribute hides a class attribute with the same name\nwhen accessed in this way. Class attributes can be used as defaults\nfor instance attributes, but using mutable values there can lead to\nunexpected results. *Descriptors* can be used to create instance\nvariables with different implementation details.\n\nSee also: **PEP 3115** - Metaclasses in Python 3 **PEP 3129** -\n Class Decorators\n', - 'comparisons': u'\nComparisons\n***********\n\nUnlike C, all comparison operations in Python have the same priority,\nwhich is lower than that of any arithmetic, shifting or bitwise\noperation. Also unlike C, expressions like "a < b < c" have the\ninterpretation that is conventional in mathematics:\n\n comparison ::= or_expr ( comp_operator or_expr )*\n comp_operator ::= "<" | ">" | "==" | ">=" | "<=" | "!="\n | "is" ["not"] | ["not"] "in"\n\nComparisons yield boolean values: "True" or "False".\n\nComparisons can be chained arbitrarily, e.g., "x < y <= z" is\nequivalent to "x < y and y <= z", except that "y" is evaluated only\nonce (but in both cases "z" is not evaluated at all when "x < y" is\nfound to be false).\n\nFormally, if *a*, *b*, *c*, ..., *y*, *z* are expressions and *op1*,\n*op2*, ..., *opN* are comparison operators, then "a op1 b op2 c ... y\nopN z" is equivalent to "a op1 b and b op2 c and ... y opN z", except\nthat each expression is evaluated at most once.\n\nNote that "a op1 b op2 c" doesn\'t imply any kind of comparison between\n*a* and *c*, so that, e.g., "x < y > z" is perfectly legal (though\nperhaps not pretty).\n\nThe operators "<", ">", "==", ">=", "<=", and "!=" compare the values\nof two objects. The objects need not have the same type. If both are\nnumbers, they are converted to a common type. Otherwise, the "==" and\n"!=" operators *always* consider objects of different types to be\nunequal, while the "<", ">", ">=" and "<=" operators raise a\n"TypeError" when comparing objects of different types that do not\nimplement these operators for the given pair of types. You can\ncontrol comparison behavior of objects of non-built-in types by\ndefining rich comparison methods like "__gt__()", described in section\n*Basic customization*.\n\nComparison of objects of the same type depends on the type:\n\n* Numbers are compared arithmetically.\n\n* The values "float(\'NaN\')" and "Decimal(\'NaN\')" are special. They\n are identical to themselves, "x is x" but are not equal to\n themselves, "x != x". Additionally, comparing any value to a\n not-a-number value will return "False". For example, both "3 <\n float(\'NaN\')" and "float(\'NaN\') < 3" will return "False".\n\n* Bytes objects are compared lexicographically using the numeric\n values of their elements.\n\n* Strings are compared lexicographically using the numeric\n equivalents (the result of the built-in function "ord()") of their\n characters. [3] String and bytes object can\'t be compared!\n\n* Tuples and lists are compared lexicographically using comparison\n of corresponding elements. This means that to compare equal, each\n element must compare equal and the two sequences must be of the same\n type and have the same length.\n\n If not equal, the sequences are ordered the same as their first\n differing elements. For example, "[1,2,x] <= [1,2,y]" has the same\n value as "x <= y". If the corresponding element does not exist, the\n shorter sequence is ordered first (for example, "[1,2] < [1,2,3]").\n\n* Mappings (dictionaries) compare equal if and only if they have the\n same "(key, value)" pairs. Order comparisons "(\'<\', \'<=\', \'>=\',\n \'>\')" raise "TypeError".\n\n* Sets and frozensets define comparison operators to mean subset and\n superset tests. Those relations do not define total orderings (the\n two sets "{1,2}" and "{2,3}" are not equal, nor subsets of one\n another, nor supersets of one another). Accordingly, sets are not\n appropriate arguments for functions which depend on total ordering.\n For example, "min()", "max()", and "sorted()" produce undefined\n results given a list of sets as inputs.\n\n* Most other objects of built-in types compare unequal unless they\n are the same object; the choice whether one object is considered\n smaller or larger than another one is made arbitrarily but\n consistently within one execution of a program.\n\nComparison of objects of differing types depends on whether either of\nthe types provide explicit support for the comparison. Most numeric\ntypes can be compared with one another. When cross-type comparison is\nnot supported, the comparison method returns "NotImplemented".\n\nThe operators "in" and "not in" test for membership. "x in s"\nevaluates to true if *x* is a member of *s*, and false otherwise. "x\nnot in s" returns the negation of "x in s". All built-in sequences\nand set types support this as well as dictionary, for which "in" tests\nwhether the dictionary has a given key. For container types such as\nlist, tuple, set, frozenset, dict, or collections.deque, the\nexpression "x in y" is equivalent to "any(x is e or x == e for e in\ny)".\n\nFor the string and bytes types, "x in y" is true if and only if *x* is\na substring of *y*. An equivalent test is "y.find(x) != -1". Empty\nstrings are always considered to be a substring of any other string,\nso """ in "abc"" will return "True".\n\nFor user-defined classes which define the "__contains__()" method, "x\nin y" is true if and only if "y.__contains__(x)" is true.\n\nFor user-defined classes which do not define "__contains__()" but do\ndefine "__iter__()", "x in y" is true if some value "z" with "x == z"\nis produced while iterating over "y". If an exception is raised\nduring the iteration, it is as if "in" raised that exception.\n\nLastly, the old-style iteration protocol is tried: if a class defines\n"__getitem__()", "x in y" is true if and only if there is a non-\nnegative integer index *i* such that "x == y[i]", and all lower\ninteger indices do not raise "IndexError" exception. (If any other\nexception is raised, it is as if "in" raised that exception).\n\nThe operator "not in" is defined to have the inverse true value of\n"in".\n\nThe operators "is" and "is not" test for object identity: "x is y" is\ntrue if and only if *x* and *y* are the same object. "x is not y"\nyields the inverse truth value. [4]\n', - 'compound': u'\nCompound statements\n*******************\n\nCompound statements contain (groups of) other statements; they affect\nor control the execution of those other statements in some way. In\ngeneral, compound statements span multiple lines, although in simple\nincarnations a whole compound statement may be contained in one line.\n\nThe "if", "while" and "for" statements implement traditional control\nflow constructs. "try" specifies exception handlers and/or cleanup\ncode for a group of statements, while the "with" statement allows the\nexecution of initialization and finalization code around a block of\ncode. Function and class definitions are also syntactically compound\nstatements.\n\nA compound statement consists of one or more \'clauses.\' A clause\nconsists of a header and a \'suite.\' The clause headers of a\nparticular compound statement are all at the same indentation level.\nEach clause header begins with a uniquely identifying keyword and ends\nwith a colon. A suite is a group of statements controlled by a\nclause. A suite can be one or more semicolon-separated simple\nstatements on the same line as the header, following the header\'s\ncolon, or it can be one or more indented statements on subsequent\nlines. Only the latter form of a suite can contain nested compound\nstatements; the following is illegal, mostly because it wouldn\'t be\nclear to which "if" clause a following "else" clause would belong:\n\n if test1: if test2: print(x)\n\nAlso note that the semicolon binds tighter than the colon in this\ncontext, so that in the following example, either all or none of the\n"print()" calls are executed:\n\n if x < y < z: print(x); print(y); print(z)\n\nSummarizing:\n\n compound_stmt ::= if_stmt\n | while_stmt\n | for_stmt\n | try_stmt\n | with_stmt\n | funcdef\n | classdef\n | async_with_stmt\n | async_for_stmt\n | async_funcdef\n suite ::= stmt_list NEWLINE | NEWLINE INDENT statement+ DEDENT\n statement ::= stmt_list NEWLINE | compound_stmt\n stmt_list ::= simple_stmt (";" simple_stmt)* [";"]\n\nNote that statements always end in a "NEWLINE" possibly followed by a\n"DEDENT". Also note that optional continuation clauses always begin\nwith a keyword that cannot start a statement, thus there are no\nambiguities (the \'dangling "else"\' problem is solved in Python by\nrequiring nested "if" statements to be indented).\n\nThe formatting of the grammar rules in the following sections places\neach clause on a separate line for clarity.\n\n\nThe "if" statement\n==================\n\nThe "if" statement is used for conditional execution:\n\n if_stmt ::= "if" expression ":" suite\n ( "elif" expression ":" suite )*\n ["else" ":" suite]\n\nIt selects exactly one of the suites by evaluating the expressions one\nby one until one is found to be true (see section *Boolean operations*\nfor the definition of true and false); then that suite is executed\n(and no other part of the "if" statement is executed or evaluated).\nIf all expressions are false, the suite of the "else" clause, if\npresent, is executed.\n\n\nThe "while" statement\n=====================\n\nThe "while" statement is used for repeated execution as long as an\nexpression is true:\n\n while_stmt ::= "while" expression ":" suite\n ["else" ":" suite]\n\nThis repeatedly tests the expression and, if it is true, executes the\nfirst suite; if the expression is false (which may be the first time\nit is tested) the suite of the "else" clause, if present, is executed\nand the loop terminates.\n\nA "break" statement executed in the first suite terminates the loop\nwithout executing the "else" clause\'s suite. A "continue" statement\nexecuted in the first suite skips the rest of the suite and goes back\nto testing the expression.\n\n\nThe "for" statement\n===================\n\nThe "for" statement is used to iterate over the elements of a sequence\n(such as a string, tuple or list) or other iterable object:\n\n for_stmt ::= "for" target_list "in" expression_list ":" suite\n ["else" ":" suite]\n\nThe expression list is evaluated once; it should yield an iterable\nobject. An iterator is created for the result of the\n"expression_list". The suite is then executed once for each item\nprovided by the iterator, in the order returned by the iterator. Each\nitem in turn is assigned to the target list using the standard rules\nfor assignments (see *Assignment statements*), and then the suite is\nexecuted. When the items are exhausted (which is immediately when the\nsequence is empty or an iterator raises a "StopIteration" exception),\nthe suite in the "else" clause, if present, is executed, and the loop\nterminates.\n\nA "break" statement executed in the first suite terminates the loop\nwithout executing the "else" clause\'s suite. A "continue" statement\nexecuted in the first suite skips the rest of the suite and continues\nwith the next item, or with the "else" clause if there is no next\nitem.\n\nThe for-loop makes assignments to the variables(s) in the target list.\nThis overwrites all previous assignments to those variables including\nthose made in the suite of the for-loop:\n\n for i in range(10):\n print(i)\n i = 5 # this will not affect the for-loop\n # because i will be overwritten with the next\n # index in the range\n\nNames in the target list are not deleted when the loop is finished,\nbut if the sequence is empty, they will not have been assigned to at\nall by the loop. Hint: the built-in function "range()" returns an\niterator of integers suitable to emulate the effect of Pascal\'s "for i\n:= a to b do"; e.g., "list(range(3))" returns the list "[0, 1, 2]".\n\nNote: There is a subtlety when the sequence is being modified by the\n loop (this can only occur for mutable sequences, i.e. lists). An\n internal counter is used to keep track of which item is used next,\n and this is incremented on each iteration. When this counter has\n reached the length of the sequence the loop terminates. This means\n that if the suite deletes the current (or a previous) item from the\n sequence, the next item will be skipped (since it gets the index of\n the current item which has already been treated). Likewise, if the\n suite inserts an item in the sequence before the current item, the\n current item will be treated again the next time through the loop.\n This can lead to nasty bugs that can be avoided by making a\n temporary copy using a slice of the whole sequence, e.g.,\n\n for x in a[:]:\n if x < 0: a.remove(x)\n\n\nThe "try" statement\n===================\n\nThe "try" statement specifies exception handlers and/or cleanup code\nfor a group of statements:\n\n try_stmt ::= try1_stmt | try2_stmt\n try1_stmt ::= "try" ":" suite\n ("except" [expression ["as" identifier]] ":" suite)+\n ["else" ":" suite]\n ["finally" ":" suite]\n try2_stmt ::= "try" ":" suite\n "finally" ":" suite\n\nThe "except" clause(s) specify one or more exception handlers. When no\nexception occurs in the "try" clause, no exception handler is\nexecuted. When an exception occurs in the "try" suite, a search for an\nexception handler is started. This search inspects the except clauses\nin turn until one is found that matches the exception. An expression-\nless except clause, if present, must be last; it matches any\nexception. For an except clause with an expression, that expression\nis evaluated, and the clause matches the exception if the resulting\nobject is "compatible" with the exception. An object is compatible\nwith an exception if it is the class or a base class of the exception\nobject or a tuple containing an item compatible with the exception.\n\nIf no except clause matches the exception, the search for an exception\nhandler continues in the surrounding code and on the invocation stack.\n[1]\n\nIf the evaluation of an expression in the header of an except clause\nraises an exception, the original search for a handler is canceled and\na search starts for the new exception in the surrounding code and on\nthe call stack (it is treated as if the entire "try" statement raised\nthe exception).\n\nWhen a matching except clause is found, the exception is assigned to\nthe target specified after the "as" keyword in that except clause, if\npresent, and the except clause\'s suite is executed. All except\nclauses must have an executable block. When the end of this block is\nreached, execution continues normally after the entire try statement.\n(This means that if two nested handlers exist for the same exception,\nand the exception occurs in the try clause of the inner handler, the\nouter handler will not handle the exception.)\n\nWhen an exception has been assigned using "as target", it is cleared\nat the end of the except clause. This is as if\n\n except E as N:\n foo\n\nwas translated to\n\n except E as N:\n try:\n foo\n finally:\n del N\n\nThis means the exception must be assigned to a different name to be\nable to refer to it after the except clause. Exceptions are cleared\nbecause with the traceback attached to them, they form a reference\ncycle with the stack frame, keeping all locals in that frame alive\nuntil the next garbage collection occurs.\n\nBefore an except clause\'s suite is executed, details about the\nexception are stored in the "sys" module and can be accessed via\n"sys.exc_info()". "sys.exc_info()" returns a 3-tuple consisting of the\nexception class, the exception instance and a traceback object (see\nsection *The standard type hierarchy*) identifying the point in the\nprogram where the exception occurred. "sys.exc_info()" values are\nrestored to their previous values (before the call) when returning\nfrom a function that handled an exception.\n\nThe optional "else" clause is executed if and when control flows off\nthe end of the "try" clause. [2] Exceptions in the "else" clause are\nnot handled by the preceding "except" clauses.\n\nIf "finally" is present, it specifies a \'cleanup\' handler. The "try"\nclause is executed, including any "except" and "else" clauses. If an\nexception occurs in any of the clauses and is not handled, the\nexception is temporarily saved. The "finally" clause is executed. If\nthere is a saved exception it is re-raised at the end of the "finally"\nclause. If the "finally" clause raises another exception, the saved\nexception is set as the context of the new exception. If the "finally"\nclause executes a "return" or "break" statement, the saved exception\nis discarded:\n\n >>> def f():\n ... try:\n ... 1/0\n ... finally:\n ... return 42\n ...\n >>> f()\n 42\n\nThe exception information is not available to the program during\nexecution of the "finally" clause.\n\nWhen a "return", "break" or "continue" statement is executed in the\n"try" suite of a "try"..."finally" statement, the "finally" clause is\nalso executed \'on the way out.\' A "continue" statement is illegal in\nthe "finally" clause. (The reason is a problem with the current\nimplementation --- this restriction may be lifted in the future).\n\nThe return value of a function is determined by the last "return"\nstatement executed. Since the "finally" clause always executes, a\n"return" statement executed in the "finally" clause will always be the\nlast one executed:\n\n >>> def foo():\n ... try:\n ... return \'try\'\n ... finally:\n ... return \'finally\'\n ...\n >>> foo()\n \'finally\'\n\nAdditional information on exceptions can be found in section\n*Exceptions*, and information on using the "raise" statement to\ngenerate exceptions may be found in section *The raise statement*.\n\n\nThe "with" statement\n====================\n\nThe "with" statement is used to wrap the execution of a block with\nmethods defined by a context manager (see section *With Statement\nContext Managers*). This allows common "try"..."except"..."finally"\nusage patterns to be encapsulated for convenient reuse.\n\n with_stmt ::= "with" with_item ("," with_item)* ":" suite\n with_item ::= expression ["as" target]\n\nThe execution of the "with" statement with one "item" proceeds as\nfollows:\n\n1. The context expression (the expression given in the "with_item")\n is evaluated to obtain a context manager.\n\n2. The context manager\'s "__exit__()" is loaded for later use.\n\n3. The context manager\'s "__enter__()" method is invoked.\n\n4. If a target was included in the "with" statement, the return\n value from "__enter__()" is assigned to it.\n\n Note: The "with" statement guarantees that if the "__enter__()"\n method returns without an error, then "__exit__()" will always be\n called. Thus, if an error occurs during the assignment to the\n target list, it will be treated the same as an error occurring\n within the suite would be. See step 6 below.\n\n5. The suite is executed.\n\n6. The context manager\'s "__exit__()" method is invoked. If an\n exception caused the suite to be exited, its type, value, and\n traceback are passed as arguments to "__exit__()". Otherwise, three\n "None" arguments are supplied.\n\n If the suite was exited due to an exception, and the return value\n from the "__exit__()" method was false, the exception is reraised.\n If the return value was true, the exception is suppressed, and\n execution continues with the statement following the "with"\n statement.\n\n If the suite was exited for any reason other than an exception, the\n return value from "__exit__()" is ignored, and execution proceeds\n at the normal location for the kind of exit that was taken.\n\nWith more than one item, the context managers are processed as if\nmultiple "with" statements were nested:\n\n with A() as a, B() as b:\n suite\n\nis equivalent to\n\n with A() as a:\n with B() as b:\n suite\n\nChanged in version 3.1: Support for multiple context expressions.\n\nSee also: **PEP 0343** - The "with" statement\n\n The specification, background, and examples for the Python "with"\n statement.\n\n\nFunction definitions\n====================\n\nA function definition defines a user-defined function object (see\nsection *The standard type hierarchy*):\n\n funcdef ::= [decorators] "def" funcname "(" [parameter_list] ")" ["->" expression] ":" suite\n decorators ::= decorator+\n decorator ::= "@" dotted_name ["(" [parameter_list [","]] ")"] NEWLINE\n dotted_name ::= identifier ("." identifier)*\n parameter_list ::= (defparameter ",")*\n | "*" [parameter] ("," defparameter)* ["," "**" parameter]\n | "**" parameter\n | defparameter [","] )\n parameter ::= identifier [":" expression]\n defparameter ::= parameter ["=" expression]\n funcname ::= identifier\n\nA function definition is an executable statement. Its execution binds\nthe function name in the current local namespace to a function object\n(a wrapper around the executable code for the function). This\nfunction object contains a reference to the current global namespace\nas the global namespace to be used when the function is called.\n\nThe function definition does not execute the function body; this gets\nexecuted only when the function is called. [3]\n\nA function definition may be wrapped by one or more *decorator*\nexpressions. Decorator expressions are evaluated when the function is\ndefined, in the scope that contains the function definition. The\nresult must be a callable, which is invoked with the function object\nas the only argument. The returned value is bound to the function name\ninstead of the function object. Multiple decorators are applied in\nnested fashion. For example, the following code\n\n @f1(arg)\n @f2\n def func(): pass\n\nis equivalent to\n\n def func(): pass\n func = f1(arg)(f2(func))\n\nWhen one or more *parameters* have the form *parameter* "="\n*expression*, the function is said to have "default parameter values."\nFor a parameter with a default value, the corresponding *argument* may\nbe omitted from a call, in which case the parameter\'s default value is\nsubstituted. If a parameter has a default value, all following\nparameters up until the ""*"" must also have a default value --- this\nis a syntactic restriction that is not expressed by the grammar.\n\n**Default parameter values are evaluated from left to right when the\nfunction definition is executed.** This means that the expression is\nevaluated once, when the function is defined, and that the same "pre-\ncomputed" value is used for each call. This is especially important\nto understand when a default parameter is a mutable object, such as a\nlist or a dictionary: if the function modifies the object (e.g. by\nappending an item to a list), the default value is in effect modified.\nThis is generally not what was intended. A way around this is to use\n"None" as the default, and explicitly test for it in the body of the\nfunction, e.g.:\n\n def whats_on_the_telly(penguin=None):\n if penguin is None:\n penguin = []\n penguin.append("property of the zoo")\n return penguin\n\nFunction call semantics are described in more detail in section\n*Calls*. A function call always assigns values to all parameters\nmentioned in the parameter list, either from position arguments, from\nkeyword arguments, or from default values. If the form\n""*identifier"" is present, it is initialized to a tuple receiving any\nexcess positional parameters, defaulting to the empty tuple. If the\nform ""**identifier"" is present, it is initialized to a new\ndictionary receiving any excess keyword arguments, defaulting to a new\nempty dictionary. Parameters after ""*"" or ""*identifier"" are\nkeyword-only parameters and may only be passed used keyword arguments.\n\nParameters may have annotations of the form "": expression"" following\nthe parameter name. Any parameter may have an annotation even those\nof the form "*identifier" or "**identifier". Functions may have\n"return" annotation of the form ""-> expression"" after the parameter\nlist. These annotations can be any valid Python expression and are\nevaluated when the function definition is executed. Annotations may\nbe evaluated in a different order than they appear in the source code.\nThe presence of annotations does not change the semantics of a\nfunction. The annotation values are available as values of a\ndictionary keyed by the parameters\' names in the "__annotations__"\nattribute of the function object.\n\nIt is also possible to create anonymous functions (functions not bound\nto a name), for immediate use in expressions. This uses lambda\nexpressions, described in section *Lambdas*. Note that the lambda\nexpression is merely a shorthand for a simplified function definition;\na function defined in a ""def"" statement can be passed around or\nassigned to another name just like a function defined by a lambda\nexpression. The ""def"" form is actually more powerful since it\nallows the execution of multiple statements and annotations.\n\n**Programmer\'s note:** Functions are first-class objects. A ""def""\nstatement executed inside a function definition defines a local\nfunction that can be returned or passed around. Free variables used\nin the nested function can access the local variables of the function\ncontaining the def. See section *Naming and binding* for details.\n\nSee also: **PEP 3107** - Function Annotations\n\n The original specification for function annotations.\n\n\nClass definitions\n=================\n\nA class definition defines a class object (see section *The standard\ntype hierarchy*):\n\n classdef ::= [decorators] "class" classname [inheritance] ":" suite\n inheritance ::= "(" [parameter_list] ")"\n classname ::= identifier\n\nA class definition is an executable statement. The inheritance list\nusually gives a list of base classes (see *Customizing class creation*\nfor more advanced uses), so each item in the list should evaluate to a\nclass object which allows subclassing. Classes without an inheritance\nlist inherit, by default, from the base class "object"; hence,\n\n class Foo:\n pass\n\nis equivalent to\n\n class Foo(object):\n pass\n\nThe class\'s suite is then executed in a new execution frame (see\n*Naming and binding*), using a newly created local namespace and the\noriginal global namespace. (Usually, the suite contains mostly\nfunction definitions.) When the class\'s suite finishes execution, its\nexecution frame is discarded but its local namespace is saved. [4] A\nclass object is then created using the inheritance list for the base\nclasses and the saved local namespace for the attribute dictionary.\nThe class name is bound to this class object in the original local\nnamespace.\n\nClass creation can be customized heavily using *metaclasses*.\n\nClasses can also be decorated: just like when decorating functions,\n\n @f1(arg)\n @f2\n class Foo: pass\n\nis equivalent to\n\n class Foo: pass\n Foo = f1(arg)(f2(Foo))\n\nThe evaluation rules for the decorator expressions are the same as for\nfunction decorators. The result must be a class object, which is then\nbound to the class name.\n\n**Programmer\'s note:** Variables defined in the class definition are\nclass attributes; they are shared by instances. Instance attributes\ncan be set in a method with "self.name = value". Both class and\ninstance attributes are accessible through the notation ""self.name"",\nand an instance attribute hides a class attribute with the same name\nwhen accessed in this way. Class attributes can be used as defaults\nfor instance attributes, but using mutable values there can lead to\nunexpected results. *Descriptors* can be used to create instance\nvariables with different implementation details.\n\nSee also: **PEP 3115** - Metaclasses in Python 3 **PEP 3129** -\n Class Decorators\n\n\nCoroutines\n==========\n\nNew in version 3.5.\n\n\nCoroutine function definition\n-----------------------------\n\n async_funcdef ::= [decorators] "async" "def" funcname "(" [parameter_list] ")" ["->" expression] ":" suite\n\nExecution of Python coroutines can be suspended and resumed at many\npoints (see *coroutine*). In the body of a coroutine, any "await" and\n"async" identifiers become reserved keywords; "await" expressions,\n"async for" and "async with" can only be used in coroutine bodies.\n\nFunctions defined with "async def" syntax are always coroutine\nfunctions, even if they do not contain "await" or "async" keywords.\n\nIt is a "SyntaxError" to use "yield" expressions in "async def"\ncoroutines.\n\nAn example of a coroutine function:\n\n async def func(param1, param2):\n do_stuff()\n await some_coroutine()\n\n\nThe "async for" statement\n-------------------------\n\n async_for_stmt ::= "async" for_stmt\n\nAn *asynchronous iterable* is able to call asynchronous code in its\n*iter* implementation, and *asynchronous iterator* can call\nasynchronous code in its *next* method.\n\nThe "async for" statement allows convenient iteration over\nasynchronous iterators.\n\nThe following code:\n\n async for TARGET in ITER:\n BLOCK\n else:\n BLOCK2\n\nIs semantically equivalent to:\n\n iter = (ITER)\n iter = await type(iter).__aiter__(iter)\n running = True\n while running:\n try:\n TARGET = await type(iter).__anext__(iter)\n except StopAsyncIteration:\n running = False\n else:\n BLOCK\n else:\n BLOCK2\n\nSee also "__aiter__()" and "__anext__()" for details.\n\nIt is a "SyntaxError" to use "async for" statement outside of an\n"async def" function.\n\n\nThe "async with" statement\n--------------------------\n\n async_with_stmt ::= "async" with_stmt\n\nAn *asynchronous context manager* is a *context manager* that is able\nto suspend execution in its *enter* and *exit* methods.\n\nThe following code:\n\n async with EXPR as VAR:\n BLOCK\n\nIs semantically equivalent to:\n\n mgr = (EXPR)\n aexit = type(mgr).__aexit__\n aenter = type(mgr).__aenter__(mgr)\n exc = True\n\n VAR = await aenter\n try:\n BLOCK\n except:\n if not await aexit(mgr, *sys.exc_info()):\n raise\n else:\n await aexit(mgr, None, None, None)\n\nSee also "__aenter__()" and "__aexit__()" for details.\n\nIt is a "SyntaxError" to use "async with" statement outside of an\n"async def" function.\n\nSee also: **PEP 492** - Coroutines with async and await syntax\n\n-[ Footnotes ]-\n\n[1] The exception is propagated to the invocation stack unless\n there is a "finally" clause which happens to raise another\n exception. That new exception causes the old one to be lost.\n\n[2] Currently, control "flows off the end" except in the case of\n an exception or the execution of a "return", "continue", or\n "break" statement.\n\n[3] A string literal appearing as the first statement in the\n function body is transformed into the function\'s "__doc__"\n attribute and therefore the function\'s *docstring*.\n\n[4] A string literal appearing as the first statement in the class\n body is transformed into the namespace\'s "__doc__" item and\n therefore the class\'s *docstring*.\n', - 'context-managers': u'\nWith Statement Context Managers\n*******************************\n\nA *context manager* is an object that defines the runtime context to\nbe established when executing a "with" statement. The context manager\nhandles the entry into, and the exit from, the desired runtime context\nfor the execution of the block of code. Context managers are normally\ninvoked using the "with" statement (described in section *The with\nstatement*), but can also be used by directly invoking their methods.\n\nTypical uses of context managers include saving and restoring various\nkinds of global state, locking and unlocking resources, closing opened\nfiles, etc.\n\nFor more information on context managers, see *Context Manager Types*.\n\nobject.__enter__(self)\n\n Enter the runtime context related to this object. The "with"\n statement will bind this method\'s return value to the target(s)\n specified in the "as" clause of the statement, if any.\n\nobject.__exit__(self, exc_type, exc_value, traceback)\n\n Exit the runtime context related to this object. The parameters\n describe the exception that caused the context to be exited. If the\n context was exited without an exception, all three arguments will\n be "None".\n\n If an exception is supplied, and the method wishes to suppress the\n exception (i.e., prevent it from being propagated), it should\n return a true value. Otherwise, the exception will be processed\n normally upon exit from this method.\n\n Note that "__exit__()" methods should not reraise the passed-in\n exception; this is the caller\'s responsibility.\n\nSee also: **PEP 0343** - The "with" statement\n\n The specification, background, and examples for the Python "with"\n statement.\n', - 'continue': u'\nThe "continue" statement\n************************\n\n continue_stmt ::= "continue"\n\n"continue" may only occur syntactically nested in a "for" or "while"\nloop, but not nested in a function or class definition or "finally"\nclause within that loop. It continues with the next cycle of the\nnearest enclosing loop.\n\nWhen "continue" passes control out of a "try" statement with a\n"finally" clause, that "finally" clause is executed before really\nstarting the next loop cycle.\n', - 'conversions': u'\nArithmetic conversions\n**********************\n\nWhen a description of an arithmetic operator below uses the phrase\n"the numeric arguments are converted to a common type," this means\nthat the operator implementation for built-in types works as follows:\n\n* If either argument is a complex number, the other is converted to\n complex;\n\n* otherwise, if either argument is a floating point number, the\n other is converted to floating point;\n\n* otherwise, both must be integers and no conversion is necessary.\n\nSome additional rules apply for certain operators (e.g., a string as a\nleft argument to the \'%\' operator). Extensions must define their own\nconversion behavior.\n', - 'customization': u'\nBasic customization\n*******************\n\nobject.__new__(cls[, ...])\n\n Called to create a new instance of class *cls*. "__new__()" is a\n static method (special-cased so you need not declare it as such)\n that takes the class of which an instance was requested as its\n first argument. The remaining arguments are those passed to the\n object constructor expression (the call to the class). The return\n value of "__new__()" should be the new object instance (usually an\n instance of *cls*).\n\n Typical implementations create a new instance of the class by\n invoking the superclass\'s "__new__()" method using\n "super(currentclass, cls).__new__(cls[, ...])" with appropriate\n arguments and then modifying the newly-created instance as\n necessary before returning it.\n\n If "__new__()" returns an instance of *cls*, then the new\n instance\'s "__init__()" method will be invoked like\n "__init__(self[, ...])", where *self* is the new instance and the\n remaining arguments are the same as were passed to "__new__()".\n\n If "__new__()" does not return an instance of *cls*, then the new\n instance\'s "__init__()" method will not be invoked.\n\n "__new__()" is intended mainly to allow subclasses of immutable\n types (like int, str, or tuple) to customize instance creation. It\n is also commonly overridden in custom metaclasses in order to\n customize class creation.\n\nobject.__init__(self[, ...])\n\n Called after the instance has been created (by "__new__()"), but\n before it is returned to the caller. The arguments are those\n passed to the class constructor expression. If a base class has an\n "__init__()" method, the derived class\'s "__init__()" method, if\n any, must explicitly call it to ensure proper initialization of the\n base class part of the instance; for example:\n "BaseClass.__init__(self, [args...])".\n\n Because "__new__()" and "__init__()" work together in constructing\n objects ("__new__()" to create it, and "__init__()" to customise\n it), no non-"None" value may be returned by "__init__()"; doing so\n will cause a "TypeError" to be raised at runtime.\n\nobject.__del__(self)\n\n Called when the instance is about to be destroyed. This is also\n called a destructor. If a base class has a "__del__()" method, the\n derived class\'s "__del__()" method, if any, must explicitly call it\n to ensure proper deletion of the base class part of the instance.\n Note that it is possible (though not recommended!) for the\n "__del__()" method to postpone destruction of the instance by\n creating a new reference to it. It may then be called at a later\n time when this new reference is deleted. It is not guaranteed that\n "__del__()" methods are called for objects that still exist when\n the interpreter exits.\n\n Note: "del x" doesn\'t directly call "x.__del__()" --- the former\n decrements the reference count for "x" by one, and the latter is\n only called when "x"\'s reference count reaches zero. Some common\n situations that may prevent the reference count of an object from\n going to zero include: circular references between objects (e.g.,\n a doubly-linked list or a tree data structure with parent and\n child pointers); a reference to the object on the stack frame of\n a function that caught an exception (the traceback stored in\n "sys.exc_info()[2]" keeps the stack frame alive); or a reference\n to the object on the stack frame that raised an unhandled\n exception in interactive mode (the traceback stored in\n "sys.last_traceback" keeps the stack frame alive). The first\n situation can only be remedied by explicitly breaking the cycles;\n the second can be resolved by freeing the reference to the\n traceback object when it is no longer useful, and the third can\n be resolved by storing "None" in "sys.last_traceback". Circular\n references which are garbage are detected and cleaned up when the\n cyclic garbage collector is enabled (it\'s on by default). Refer\n to the documentation for the "gc" module for more information\n about this topic.\n\n Warning: Due to the precarious circumstances under which\n "__del__()" methods are invoked, exceptions that occur during\n their execution are ignored, and a warning is printed to\n "sys.stderr" instead. Also, when "__del__()" is invoked in\n response to a module being deleted (e.g., when execution of the\n program is done), other globals referenced by the "__del__()"\n method may already have been deleted or in the process of being\n torn down (e.g. the import machinery shutting down). For this\n reason, "__del__()" methods should do the absolute minimum needed\n to maintain external invariants. Starting with version 1.5,\n Python guarantees that globals whose name begins with a single\n underscore are deleted from their module before other globals are\n deleted; if no other references to such globals exist, this may\n help in assuring that imported modules are still available at the\n time when the "__del__()" method is called.\n\nobject.__repr__(self)\n\n Called by the "repr()" built-in function to compute the "official"\n string representation of an object. If at all possible, this\n should look like a valid Python expression that could be used to\n recreate an object with the same value (given an appropriate\n environment). If this is not possible, a string of the form\n "<...some useful description...>" should be returned. The return\n value must be a string object. If a class defines "__repr__()" but\n not "__str__()", then "__repr__()" is also used when an "informal"\n string representation of instances of that class is required.\n\n This is typically used for debugging, so it is important that the\n representation is information-rich and unambiguous.\n\nobject.__str__(self)\n\n Called by "str(object)" and the built-in functions "format()" and\n "print()" to compute the "informal" or nicely printable string\n representation of an object. The return value must be a *string*\n object.\n\n This method differs from "object.__repr__()" in that there is no\n expectation that "__str__()" return a valid Python expression: a\n more convenient or concise representation can be used.\n\n The default implementation defined by the built-in type "object"\n calls "object.__repr__()".\n\nobject.__bytes__(self)\n\n Called by "bytes()" to compute a byte-string representation of an\n object. This should return a "bytes" object.\n\nobject.__format__(self, format_spec)\n\n Called by the "format()" built-in function (and by extension, the\n "str.format()" method of class "str") to produce a "formatted"\n string representation of an object. The "format_spec" argument is a\n string that contains a description of the formatting options\n desired. The interpretation of the "format_spec" argument is up to\n the type implementing "__format__()", however most classes will\n either delegate formatting to one of the built-in types, or use a\n similar formatting option syntax.\n\n See *Format Specification Mini-Language* for a description of the\n standard formatting syntax.\n\n The return value must be a string object.\n\n Changed in version 3.4: The __format__ method of "object" itself\n raises a "TypeError" if passed any non-empty string.\n\nobject.__lt__(self, other)\nobject.__le__(self, other)\nobject.__eq__(self, other)\nobject.__ne__(self, other)\nobject.__gt__(self, other)\nobject.__ge__(self, other)\n\n These are the so-called "rich comparison" methods. The\n correspondence between operator symbols and method names is as\n follows: "xy" calls\n "x.__gt__(y)", and "x>=y" calls "x.__ge__(y)".\n\n A rich comparison method may return the singleton "NotImplemented"\n if it does not implement the operation for a given pair of\n arguments. By convention, "False" and "True" are returned for a\n successful comparison. However, these methods can return any value,\n so if the comparison operator is used in a Boolean context (e.g.,\n in the condition of an "if" statement), Python will call "bool()"\n on the value to determine if the result is true or false.\n\n By default, "__ne__()" delegates to "__eq__()" and inverts the\n result unless it is "NotImplemented". There are no other implied\n relationships among the comparison operators, for example, the\n truth of "(x.__hash__".\n\n If a class that does not override "__eq__()" wishes to suppress\n hash support, it should include "__hash__ = None" in the class\n definition. A class which defines its own "__hash__()" that\n explicitly raises a "TypeError" would be incorrectly identified as\n hashable by an "isinstance(obj, collections.Hashable)" call.\n\n Note: By default, the "__hash__()" values of str, bytes and\n datetime objects are "salted" with an unpredictable random value.\n Although they remain constant within an individual Python\n process, they are not predictable between repeated invocations of\n Python.This is intended to provide protection against a denial-\n of-service caused by carefully-chosen inputs that exploit the\n worst case performance of a dict insertion, O(n^2) complexity.\n See http://www.ocert.org/advisories/ocert-2011-003.html for\n details.Changing hash values affects the iteration order of\n dicts, sets and other mappings. Python has never made guarantees\n about this ordering (and it typically varies between 32-bit and\n 64-bit builds).See also "PYTHONHASHSEED".\n\n Changed in version 3.3: Hash randomization is enabled by default.\n\nobject.__bool__(self)\n\n Called to implement truth value testing and the built-in operation\n "bool()"; should return "False" or "True". When this method is not\n defined, "__len__()" is called, if it is defined, and the object is\n considered true if its result is nonzero. If a class defines\n neither "__len__()" nor "__bool__()", all its instances are\n considered true.\n', - 'debugger': u'\n"pdb" --- The Python Debugger\n*****************************\n\n**Source code:** Lib/pdb.py\n\n======================================================================\n\nThe module "pdb" defines an interactive source code debugger for\nPython programs. It supports setting (conditional) breakpoints and\nsingle stepping at the source line level, inspection of stack frames,\nsource code listing, and evaluation of arbitrary Python code in the\ncontext of any stack frame. It also supports post-mortem debugging\nand can be called under program control.\n\nThe debugger is extensible -- it is actually defined as the class\n"Pdb". This is currently undocumented but easily understood by reading\nthe source. The extension interface uses the modules "bdb" and "cmd".\n\nThe debugger\'s prompt is "(Pdb)". Typical usage to run a program under\ncontrol of the debugger is:\n\n >>> import pdb\n >>> import mymodule\n >>> pdb.run(\'mymodule.test()\')\n > (0)?()\n (Pdb) continue\n > (1)?()\n (Pdb) continue\n NameError: \'spam\'\n > (1)?()\n (Pdb)\n\nChanged in version 3.3: Tab-completion via the "readline" module is\navailable for commands and command arguments, e.g. the current global\nand local names are offered as arguments of the "p" command.\n\n"pdb.py" can also be invoked as a script to debug other scripts. For\nexample:\n\n python3 -m pdb myscript.py\n\nWhen invoked as a script, pdb will automatically enter post-mortem\ndebugging if the program being debugged exits abnormally. After post-\nmortem debugging (or after normal exit of the program), pdb will\nrestart the program. Automatic restarting preserves pdb\'s state (such\nas breakpoints) and in most cases is more useful than quitting the\ndebugger upon program\'s exit.\n\nNew in version 3.2: "pdb.py" now accepts a "-c" option that executes\ncommands as if given in a ".pdbrc" file, see *Debugger Commands*.\n\nThe typical usage to break into the debugger from a running program is\nto insert\n\n import pdb; pdb.set_trace()\n\nat the location you want to break into the debugger. You can then\nstep through the code following this statement, and continue running\nwithout the debugger using the "continue" command.\n\nThe typical usage to inspect a crashed program is:\n\n >>> import pdb\n >>> import mymodule\n >>> mymodule.test()\n Traceback (most recent call last):\n File "", line 1, in ?\n File "./mymodule.py", line 4, in test\n test2()\n File "./mymodule.py", line 3, in test2\n print(spam)\n NameError: spam\n >>> pdb.pm()\n > ./mymodule.py(3)test2()\n -> print(spam)\n (Pdb)\n\nThe module defines the following functions; each enters the debugger\nin a slightly different way:\n\npdb.run(statement, globals=None, locals=None)\n\n Execute the *statement* (given as a string or a code object) under\n debugger control. The debugger prompt appears before any code is\n executed; you can set breakpoints and type "continue", or you can\n step through the statement using "step" or "next" (all these\n commands are explained below). The optional *globals* and *locals*\n arguments specify the environment in which the code is executed; by\n default the dictionary of the module "__main__" is used. (See the\n explanation of the built-in "exec()" or "eval()" functions.)\n\npdb.runeval(expression, globals=None, locals=None)\n\n Evaluate the *expression* (given as a string or a code object)\n under debugger control. When "runeval()" returns, it returns the\n value of the expression. Otherwise this function is similar to\n "run()".\n\npdb.runcall(function, *args, **kwds)\n\n Call the *function* (a function or method object, not a string)\n with the given arguments. When "runcall()" returns, it returns\n whatever the function call returned. The debugger prompt appears\n as soon as the function is entered.\n\npdb.set_trace()\n\n Enter the debugger at the calling stack frame. This is useful to\n hard-code a breakpoint at a given point in a program, even if the\n code is not otherwise being debugged (e.g. when an assertion\n fails).\n\npdb.post_mortem(traceback=None)\n\n Enter post-mortem debugging of the given *traceback* object. If no\n *traceback* is given, it uses the one of the exception that is\n currently being handled (an exception must be being handled if the\n default is to be used).\n\npdb.pm()\n\n Enter post-mortem debugging of the traceback found in\n "sys.last_traceback".\n\nThe "run*" functions and "set_trace()" are aliases for instantiating\nthe "Pdb" class and calling the method of the same name. If you want\nto access further features, you have to do this yourself:\n\nclass class pdb.Pdb(completekey=\'tab\', stdin=None, stdout=None, skip=None, nosigint=False)\n\n "Pdb" is the debugger class.\n\n The *completekey*, *stdin* and *stdout* arguments are passed to the\n underlying "cmd.Cmd" class; see the description there.\n\n The *skip* argument, if given, must be an iterable of glob-style\n module name patterns. The debugger will not step into frames that\n originate in a module that matches one of these patterns. [1]\n\n By default, Pdb sets a handler for the SIGINT signal (which is sent\n when the user presses Ctrl-C on the console) when you give a\n "continue" command. This allows you to break into the debugger\n again by pressing Ctrl-C. If you want Pdb not to touch the SIGINT\n handler, set *nosigint* tot true.\n\n Example call to enable tracing with *skip*:\n\n import pdb; pdb.Pdb(skip=[\'django.*\']).set_trace()\n\n New in version 3.1: The *skip* argument.\n\n New in version 3.2: The *nosigint* argument. Previously, a SIGINT\n handler was never set by Pdb.\n\n run(statement, globals=None, locals=None)\n runeval(expression, globals=None, locals=None)\n runcall(function, *args, **kwds)\n set_trace()\n\n See the documentation for the functions explained above.\n\n\nDebugger Commands\n=================\n\nThe commands recognized by the debugger are listed below. Most\ncommands can be abbreviated to one or two letters as indicated; e.g.\n"h(elp)" means that either "h" or "help" can be used to enter the help\ncommand (but not "he" or "hel", nor "H" or "Help" or "HELP").\nArguments to commands must be separated by whitespace (spaces or\ntabs). Optional arguments are enclosed in square brackets ("[]") in\nthe command syntax; the square brackets must not be typed.\nAlternatives in the command syntax are separated by a vertical bar\n("|").\n\nEntering a blank line repeats the last command entered. Exception: if\nthe last command was a "list" command, the next 11 lines are listed.\n\nCommands that the debugger doesn\'t recognize are assumed to be Python\nstatements and are executed in the context of the program being\ndebugged. Python statements can also be prefixed with an exclamation\npoint ("!"). This is a powerful way to inspect the program being\ndebugged; it is even possible to change a variable or call a function.\nWhen an exception occurs in such a statement, the exception name is\nprinted but the debugger\'s state is not changed.\n\nThe debugger supports *aliases*. Aliases can have parameters which\nallows one a certain level of adaptability to the context under\nexamination.\n\nMultiple commands may be entered on a single line, separated by ";;".\n(A single ";" is not used as it is the separator for multiple commands\nin a line that is passed to the Python parser.) No intelligence is\napplied to separating the commands; the input is split at the first\n";;" pair, even if it is in the middle of a quoted string.\n\nIf a file ".pdbrc" exists in the user\'s home directory or in the\ncurrent directory, it is read in and executed as if it had been typed\nat the debugger prompt. This is particularly useful for aliases. If\nboth files exist, the one in the home directory is read first and\naliases defined there can be overridden by the local file.\n\nChanged in version 3.2: ".pdbrc" can now contain commands that\ncontinue debugging, such as "continue" or "next". Previously, these\ncommands had no effect.\n\nh(elp) [command]\n\n Without argument, print the list of available commands. With a\n *command* as argument, print help about that command. "help pdb"\n displays the full documentation (the docstring of the "pdb"\n module). Since the *command* argument must be an identifier, "help\n exec" must be entered to get help on the "!" command.\n\nw(here)\n\n Print a stack trace, with the most recent frame at the bottom. An\n arrow indicates the current frame, which determines the context of\n most commands.\n\nd(own) [count]\n\n Move the current frame *count* (default one) levels down in the\n stack trace (to a newer frame).\n\nu(p) [count]\n\n Move the current frame *count* (default one) levels up in the stack\n trace (to an older frame).\n\nb(reak) [([filename:]lineno | function) [, condition]]\n\n With a *lineno* argument, set a break there in the current file.\n With a *function* argument, set a break at the first executable\n statement within that function. The line number may be prefixed\n with a filename and a colon, to specify a breakpoint in another\n file (probably one that hasn\'t been loaded yet). The file is\n searched on "sys.path". Note that each breakpoint is assigned a\n number to which all the other breakpoint commands refer.\n\n If a second argument is present, it is an expression which must\n evaluate to true before the breakpoint is honored.\n\n Without argument, list all breaks, including for each breakpoint,\n the number of times that breakpoint has been hit, the current\n ignore count, and the associated condition if any.\n\ntbreak [([filename:]lineno | function) [, condition]]\n\n Temporary breakpoint, which is removed automatically when it is\n first hit. The arguments are the same as for "break".\n\ncl(ear) [filename:lineno | bpnumber [bpnumber ...]]\n\n With a *filename:lineno* argument, clear all the breakpoints at\n this line. With a space separated list of breakpoint numbers, clear\n those breakpoints. Without argument, clear all breaks (but first\n ask confirmation).\n\ndisable [bpnumber [bpnumber ...]]\n\n Disable the breakpoints given as a space separated list of\n breakpoint numbers. Disabling a breakpoint means it cannot cause\n the program to stop execution, but unlike clearing a breakpoint, it\n remains in the list of breakpoints and can be (re-)enabled.\n\nenable [bpnumber [bpnumber ...]]\n\n Enable the breakpoints specified.\n\nignore bpnumber [count]\n\n Set the ignore count for the given breakpoint number. If count is\n omitted, the ignore count is set to 0. A breakpoint becomes active\n when the ignore count is zero. When non-zero, the count is\n decremented each time the breakpoint is reached and the breakpoint\n is not disabled and any associated condition evaluates to true.\n\ncondition bpnumber [condition]\n\n Set a new *condition* for the breakpoint, an expression which must\n evaluate to true before the breakpoint is honored. If *condition*\n is absent, any existing condition is removed; i.e., the breakpoint\n is made unconditional.\n\ncommands [bpnumber]\n\n Specify a list of commands for breakpoint number *bpnumber*. The\n commands themselves appear on the following lines. Type a line\n containing just "end" to terminate the commands. An example:\n\n (Pdb) commands 1\n (com) p some_variable\n (com) end\n (Pdb)\n\n To remove all commands from a breakpoint, type commands and follow\n it immediately with "end"; that is, give no commands.\n\n With no *bpnumber* argument, commands refers to the last breakpoint\n set.\n\n You can use breakpoint commands to start your program up again.\n Simply use the continue command, or step, or any other command that\n resumes execution.\n\n Specifying any command resuming execution (currently continue,\n step, next, return, jump, quit and their abbreviations) terminates\n the command list (as if that command was immediately followed by\n end). This is because any time you resume execution (even with a\n simple next or step), you may encounter another breakpoint--which\n could have its own command list, leading to ambiguities about which\n list to execute.\n\n If you use the \'silent\' command in the command list, the usual\n message about stopping at a breakpoint is not printed. This may be\n desirable for breakpoints that are to print a specific message and\n then continue. If none of the other commands print anything, you\n see no sign that the breakpoint was reached.\n\ns(tep)\n\n Execute the current line, stop at the first possible occasion\n (either in a function that is called or on the next line in the\n current function).\n\nn(ext)\n\n Continue execution until the next line in the current function is\n reached or it returns. (The difference between "next" and "step"\n is that "step" stops inside a called function, while "next"\n executes called functions at (nearly) full speed, only stopping at\n the next line in the current function.)\n\nunt(il) [lineno]\n\n Without argument, continue execution until the line with a number\n greater than the current one is reached.\n\n With a line number, continue execution until a line with a number\n greater or equal to that is reached. In both cases, also stop when\n the current frame returns.\n\n Changed in version 3.2: Allow giving an explicit line number.\n\nr(eturn)\n\n Continue execution until the current function returns.\n\nc(ont(inue))\n\n Continue execution, only stop when a breakpoint is encountered.\n\nj(ump) lineno\n\n Set the next line that will be executed. Only available in the\n bottom-most frame. This lets you jump back and execute code again,\n or jump forward to skip code that you don\'t want to run.\n\n It should be noted that not all jumps are allowed -- for instance\n it is not possible to jump into the middle of a "for" loop or out\n of a "finally" clause.\n\nl(ist) [first[, last]]\n\n List source code for the current file. Without arguments, list 11\n lines around the current line or continue the previous listing.\n With "." as argument, list 11 lines around the current line. With\n one argument, list 11 lines around at that line. With two\n arguments, list the given range; if the second argument is less\n than the first, it is interpreted as a count.\n\n The current line in the current frame is indicated by "->". If an\n exception is being debugged, the line where the exception was\n originally raised or propagated is indicated by ">>", if it differs\n from the current line.\n\n New in version 3.2: The ">>" marker.\n\nll | longlist\n\n List all source code for the current function or frame.\n Interesting lines are marked as for "list".\n\n New in version 3.2.\n\na(rgs)\n\n Print the argument list of the current function.\n\np expression\n\n Evaluate the *expression* in the current context and print its\n value.\n\n Note: "print()" can also be used, but is not a debugger command\n --- this executes the Python "print()" function.\n\npp expression\n\n Like the "p" command, except the value of the expression is pretty-\n printed using the "pprint" module.\n\nwhatis expression\n\n Print the type of the *expression*.\n\nsource expression\n\n Try to get source code for the given object and display it.\n\n New in version 3.2.\n\ndisplay [expression]\n\n Display the value of the expression if it changed, each time\n execution stops in the current frame.\n\n Without expression, list all display expressions for the current\n frame.\n\n New in version 3.2.\n\nundisplay [expression]\n\n Do not display the expression any more in the current frame.\n Without expression, clear all display expressions for the current\n frame.\n\n New in version 3.2.\n\ninteract\n\n Start an interative interpreter (using the "code" module) whose\n global namespace contains all the (global and local) names found in\n the current scope.\n\n New in version 3.2.\n\nalias [name [command]]\n\n Create an alias called *name* that executes *command*. The command\n must *not* be enclosed in quotes. Replaceable parameters can be\n indicated by "%1", "%2", and so on, while "%*" is replaced by all\n the parameters. If no command is given, the current alias for\n *name* is shown. If no arguments are given, all aliases are listed.\n\n Aliases may be nested and can contain anything that can be legally\n typed at the pdb prompt. Note that internal pdb commands *can* be\n overridden by aliases. Such a command is then hidden until the\n alias is removed. Aliasing is recursively applied to the first\n word of the command line; all other words in the line are left\n alone.\n\n As an example, here are two useful aliases (especially when placed\n in the ".pdbrc" file):\n\n # Print instance variables (usage "pi classInst")\n alias pi for k in %1.__dict__.keys(): print("%1.",k,"=",%1.__dict__[k])\n # Print instance variables in self\n alias ps pi self\n\nunalias name\n\n Delete the specified alias.\n\n! statement\n\n Execute the (one-line) *statement* in the context of the current\n stack frame. The exclamation point can be omitted unless the first\n word of the statement resembles a debugger command. To set a\n global variable, you can prefix the assignment command with a\n "global" statement on the same line, e.g.:\n\n (Pdb) global list_options; list_options = [\'-l\']\n (Pdb)\n\nrun [args ...]\nrestart [args ...]\n\n Restart the debugged Python program. If an argument is supplied,\n it is split with "shlex" and the result is used as the new\n "sys.argv". History, breakpoints, actions and debugger options are\n preserved. "restart" is an alias for "run".\n\nq(uit)\n\n Quit from the debugger. The program being executed is aborted.\n\n-[ Footnotes ]-\n\n[1] Whether a frame is considered to originate in a certain module\n is determined by the "__name__" in the frame globals.\n', - 'del': u'\nThe "del" statement\n*******************\n\n del_stmt ::= "del" target_list\n\nDeletion is recursively defined very similar to the way assignment is\ndefined. Rather than spelling it out in full details, here are some\nhints.\n\nDeletion of a target list recursively deletes each target, from left\nto right.\n\nDeletion of a name removes the binding of that name from the local or\nglobal namespace, depending on whether the name occurs in a "global"\nstatement in the same code block. If the name is unbound, a\n"NameError" exception will be raised.\n\nDeletion of attribute references, subscriptions and slicings is passed\nto the primary object involved; deletion of a slicing is in general\nequivalent to assignment of an empty slice of the right type (but even\nthis is determined by the sliced object).\n\nChanged in version 3.2: Previously it was illegal to delete a name\nfrom the local namespace if it occurs as a free variable in a nested\nblock.\n', - 'dict': u'\nDictionary displays\n*******************\n\nA dictionary display is a possibly empty series of key/datum pairs\nenclosed in curly braces:\n\n dict_display ::= "{" [key_datum_list | dict_comprehension] "}"\n key_datum_list ::= key_datum ("," key_datum)* [","]\n key_datum ::= expression ":" expression\n dict_comprehension ::= expression ":" expression comp_for\n\nA dictionary display yields a new dictionary object.\n\nIf a comma-separated sequence of key/datum pairs is given, they are\nevaluated from left to right to define the entries of the dictionary:\neach key object is used as a key into the dictionary to store the\ncorresponding datum. This means that you can specify the same key\nmultiple times in the key/datum list, and the final dictionary\'s value\nfor that key will be the last one given.\n\nA dict comprehension, in contrast to list and set comprehensions,\nneeds two expressions separated with a colon followed by the usual\n"for" and "if" clauses. When the comprehension is run, the resulting\nkey and value elements are inserted in the new dictionary in the order\nthey are produced.\n\nRestrictions on the types of the key values are listed earlier in\nsection *The standard type hierarchy*. (To summarize, the key type\nshould be *hashable*, which excludes all mutable objects.) Clashes\nbetween duplicate keys are not detected; the last datum (textually\nrightmost in the display) stored for a given key value prevails.\n', - 'dynamic-features': u'\nInteraction with dynamic features\n*********************************\n\nName resolution of free variables occurs at runtime, not at compile\ntime. This means that the following code will print 42:\n\n i = 10\n def f():\n print(i)\n i = 42\n f()\n\nThere are several cases where Python statements are illegal when used\nin conjunction with nested scopes that contain free variables.\n\nIf a variable is referenced in an enclosing scope, it is illegal to\ndelete the name. An error will be reported at compile time.\n\nThe "eval()" and "exec()" functions do not have access to the full\nenvironment for resolving names. Names may be resolved in the local\nand global namespaces of the caller. Free variables are not resolved\nin the nearest enclosing namespace, but in the global namespace. [1]\nThe "exec()" and "eval()" functions have optional arguments to\noverride the global and local namespace. If only one namespace is\nspecified, it is used for both.\n', - 'else': u'\nThe "if" statement\n******************\n\nThe "if" statement is used for conditional execution:\n\n if_stmt ::= "if" expression ":" suite\n ( "elif" expression ":" suite )*\n ["else" ":" suite]\n\nIt selects exactly one of the suites by evaluating the expressions one\nby one until one is found to be true (see section *Boolean operations*\nfor the definition of true and false); then that suite is executed\n(and no other part of the "if" statement is executed or evaluated).\nIf all expressions are false, the suite of the "else" clause, if\npresent, is executed.\n', - 'exceptions': u'\nExceptions\n**********\n\nExceptions are a means of breaking out of the normal flow of control\nof a code block in order to handle errors or other exceptional\nconditions. An exception is *raised* at the point where the error is\ndetected; it may be *handled* by the surrounding code block or by any\ncode block that directly or indirectly invoked the code block where\nthe error occurred.\n\nThe Python interpreter raises an exception when it detects a run-time\nerror (such as division by zero). A Python program can also\nexplicitly raise an exception with the "raise" statement. Exception\nhandlers are specified with the "try" ... "except" statement. The\n"finally" clause of such a statement can be used to specify cleanup\ncode which does not handle the exception, but is executed whether an\nexception occurred or not in the preceding code.\n\nPython uses the "termination" model of error handling: an exception\nhandler can find out what happened and continue execution at an outer\nlevel, but it cannot repair the cause of the error and retry the\nfailing operation (except by re-entering the offending piece of code\nfrom the top).\n\nWhen an exception is not handled at all, the interpreter terminates\nexecution of the program, or returns to its interactive main loop. In\neither case, it prints a stack backtrace, except when the exception is\n"SystemExit".\n\nExceptions are identified by class instances. The "except" clause is\nselected depending on the class of the instance: it must reference the\nclass of the instance or a base class thereof. The instance can be\nreceived by the handler and can carry additional information about the\nexceptional condition.\n\nNote: Exception messages are not part of the Python API. Their\n contents may change from one version of Python to the next without\n warning and should not be relied on by code which will run under\n multiple versions of the interpreter.\n\nSee also the description of the "try" statement in section *The try\nstatement* and "raise" statement in section *The raise statement*.\n\n-[ Footnotes ]-\n\n[1] This limitation occurs because the code that is executed by\n these operations is not available at the time the module is\n compiled.\n', - 'execmodel': u'\nExecution model\n***************\n\n\nStructure of a programm\n=======================\n\nA Python program is constructed from code blocks. A *block* is a piece\nof Python program text that is executed as a unit. The following are\nblocks: a module, a function body, and a class definition. Each\ncommand typed interactively is a block. A script file (a file given\nas standard input to the interpreter or specified as a command line\nargument to the interpreter) is a code block. A script command (a\ncommand specified on the interpreter command line with the \'**-c**\'\noption) is a code block. The string argument passed to the built-in\nfunctions "eval()" and "exec()" is a code block.\n\nA code block is executed in an *execution frame*. A frame contains\nsome administrative information (used for debugging) and determines\nwhere and how execution continues after the code block\'s execution has\ncompleted.\n\n\nNaming and binding\n==================\n\n\nBinding of names\n----------------\n\n*Names* refer to objects. Names are introduced by name binding\noperations.\n\nThe following constructs bind names: formal parameters to functions,\n"import" statements, class and function definitions (these bind the\nclass or function name in the defining block), and targets that are\nidentifiers if occurring in an assignment, "for" loop header, or after\n"as" in a "with" statement or "except" clause. The "import" statement\nof the form "from ... import *" binds all names defined in the\nimported module, except those beginning with an underscore. This form\nmay only be used at the module level.\n\nA target occurring in a "del" statement is also considered bound for\nthis purpose (though the actual semantics are to unbind the name).\n\nEach assignment or import statement occurs within a block defined by a\nclass or function definition or at the module level (the top-level\ncode block).\n\nIf a name is bound in a block, it is a local variable of that block,\nunless declared as "nonlocal" or "global". If a name is bound at the\nmodule level, it is a global variable. (The variables of the module\ncode block are local and global.) If a variable is used in a code\nblock but not defined there, it is a *free variable*.\n\nEach occurrence of a name in the program text refers to the *binding*\nof that name established by the following name resolution rules.\n\n\nResolution of names\n-------------------\n\nA *scope* defines the visibility of a name within a block. If a local\nvariable is defined in a block, its scope includes that block. If the\ndefinition occurs in a function block, the scope extends to any blocks\ncontained within the defining one, unless a contained block introduces\na different binding for the name.\n\nWhen a name is used in a code block, it is resolved using the nearest\nenclosing scope. The set of all such scopes visible to a code block\nis called the block\'s *environment*.\n\nWhen a name is not found at all, a "NameError" exception is raised. If\nthe current scope is a function scope, and the name refers to a local\nvariable that has not yet been bound to a value at the point where the\nname is used, an "UnboundLocalError" exception is raised.\n"UnboundLocalError" is a subclass of "NameError".\n\nIf a name binding operation occurs anywhere within a code block, all\nuses of the name within the block are treated as references to the\ncurrent block. This can lead to errors when a name is used within a\nblock before it is bound. This rule is subtle. Python lacks\ndeclarations and allows name binding operations to occur anywhere\nwithin a code block. The local variables of a code block can be\ndetermined by scanning the entire text of the block for name binding\noperations.\n\nIf the "global" statement occurs within a block, all uses of the name\nspecified in the statement refer to the binding of that name in the\ntop-level namespace. Names are resolved in the top-level namespace by\nsearching the global namespace, i.e. the namespace of the module\ncontaining the code block, and the builtins namespace, the namespace\nof the module "builtins". The global namespace is searched first. If\nthe name is not found there, the builtins namespace is searched. The\n"global" statement must precede all uses of the name.\n\nThe "global" statement has the same scope as a name binding operation\nin the same block. If the nearest enclosing scope for a free variable\ncontains a global statement, the free variable is treated as a global.\n\nThe "nonlocal" statement causes corresponding names to refer to\npreviously bound variables in the nearest enclosing function scope.\n"SyntaxError" is raised at compile time if the given name does not\nexist in any enclosing function scope.\n\nThe namespace for a module is automatically created the first time a\nmodule is imported. The main module for a script is always called\n"__main__".\n\nClass definition blocks and arguments to "exec()" and "eval()" are\nspecial in the context of name resolution. A class definition is an\nexecutable statement that may use and define names. These references\nfollow the normal rules for name resolution with an exception that\nunbound local variables are looked up in the global namespace. The\nnamespace of the class definition becomes the attribute dictionary of\nthe class. The scope of names defined in a class block is limited to\nthe class block; it does not extend to the code blocks of methods --\nthis includes comprehensions and generator expressions since they are\nimplemented using a function scope. This means that the following\nwill fail:\n\n class A:\n a = 42\n b = list(a + i for i in range(10))\n\n\nBuiltins and restricted execution\n---------------------------------\n\nThe builtins namespace associated with the execution of a code block\nis actually found by looking up the name "__builtins__" in its global\nnamespace; this should be a dictionary or a module (in the latter case\nthe module\'s dictionary is used). By default, when in the "__main__"\nmodule, "__builtins__" is the built-in module "builtins"; when in any\nother module, "__builtins__" is an alias for the dictionary of the\n"builtins" module itself. "__builtins__" can be set to a user-created\ndictionary to create a weak form of restricted execution.\n\n**CPython implementation detail:** Users should not touch\n"__builtins__"; it is strictly an implementation detail. Users\nwanting to override values in the builtins namespace should "import"\nthe "builtins" module and modify its attributes appropriately.\n\n\nInteraction with dynamic features\n---------------------------------\n\nName resolution of free variables occurs at runtime, not at compile\ntime. This means that the following code will print 42:\n\n i = 10\n def f():\n print(i)\n i = 42\n f()\n\nThere are several cases where Python statements are illegal when used\nin conjunction with nested scopes that contain free variables.\n\nIf a variable is referenced in an enclosing scope, it is illegal to\ndelete the name. An error will be reported at compile time.\n\nThe "eval()" and "exec()" functions do not have access to the full\nenvironment for resolving names. Names may be resolved in the local\nand global namespaces of the caller. Free variables are not resolved\nin the nearest enclosing namespace, but in the global namespace. [1]\nThe "exec()" and "eval()" functions have optional arguments to\noverride the global and local namespace. If only one namespace is\nspecified, it is used for both.\n\n\nExceptions\n==========\n\nExceptions are a means of breaking out of the normal flow of control\nof a code block in order to handle errors or other exceptional\nconditions. An exception is *raised* at the point where the error is\ndetected; it may be *handled* by the surrounding code block or by any\ncode block that directly or indirectly invoked the code block where\nthe error occurred.\n\nThe Python interpreter raises an exception when it detects a run-time\nerror (such as division by zero). A Python program can also\nexplicitly raise an exception with the "raise" statement. Exception\nhandlers are specified with the "try" ... "except" statement. The\n"finally" clause of such a statement can be used to specify cleanup\ncode which does not handle the exception, but is executed whether an\nexception occurred or not in the preceding code.\n\nPython uses the "termination" model of error handling: an exception\nhandler can find out what happened and continue execution at an outer\nlevel, but it cannot repair the cause of the error and retry the\nfailing operation (except by re-entering the offending piece of code\nfrom the top).\n\nWhen an exception is not handled at all, the interpreter terminates\nexecution of the program, or returns to its interactive main loop. In\neither case, it prints a stack backtrace, except when the exception is\n"SystemExit".\n\nExceptions are identified by class instances. The "except" clause is\nselected depending on the class of the instance: it must reference the\nclass of the instance or a base class thereof. The instance can be\nreceived by the handler and can carry additional information about the\nexceptional condition.\n\nNote: Exception messages are not part of the Python API. Their\n contents may change from one version of Python to the next without\n warning and should not be relied on by code which will run under\n multiple versions of the interpreter.\n\nSee also the description of the "try" statement in section *The try\nstatement* and "raise" statement in section *The raise statement*.\n\n-[ Footnotes ]-\n\n[1] This limitation occurs because the code that is executed by\n these operations is not available at the time the module is\n compiled.\n', - 'exprlists': u'\nExpression lists\n****************\n\n expression_list ::= expression ( "," expression )* [","]\n\nAn expression list containing at least one comma yields a tuple. The\nlength of the tuple is the number of expressions in the list. The\nexpressions are evaluated from left to right.\n\nThe trailing comma is required only to create a single tuple (a.k.a. a\n*singleton*); it is optional in all other cases. A single expression\nwithout a trailing comma doesn\'t create a tuple, but rather yields the\nvalue of that expression. (To create an empty tuple, use an empty pair\nof parentheses: "()".)\n', - 'floating': u'\nFloating point literals\n***********************\n\nFloating point literals are described by the following lexical\ndefinitions:\n\n floatnumber ::= pointfloat | exponentfloat\n pointfloat ::= [intpart] fraction | intpart "."\n exponentfloat ::= (intpart | pointfloat) exponent\n intpart ::= digit+\n fraction ::= "." digit+\n exponent ::= ("e" | "E") ["+" | "-"] digit+\n\nNote that the integer and exponent parts are always interpreted using\nradix 10. For example, "077e010" is legal, and denotes the same number\nas "77e10". The allowed range of floating point literals is\nimplementation-dependent. Some examples of floating point literals:\n\n 3.14 10. .001 1e100 3.14e-10 0e0\n\nNote that numeric literals do not include a sign; a phrase like "-1"\nis actually an expression composed of the unary operator "-" and the\nliteral "1".\n', - 'for': u'\nThe "for" statement\n*******************\n\nThe "for" statement is used to iterate over the elements of a sequence\n(such as a string, tuple or list) or other iterable object:\n\n for_stmt ::= "for" target_list "in" expression_list ":" suite\n ["else" ":" suite]\n\nThe expression list is evaluated once; it should yield an iterable\nobject. An iterator is created for the result of the\n"expression_list". The suite is then executed once for each item\nprovided by the iterator, in the order returned by the iterator. Each\nitem in turn is assigned to the target list using the standard rules\nfor assignments (see *Assignment statements*), and then the suite is\nexecuted. When the items are exhausted (which is immediately when the\nsequence is empty or an iterator raises a "StopIteration" exception),\nthe suite in the "else" clause, if present, is executed, and the loop\nterminates.\n\nA "break" statement executed in the first suite terminates the loop\nwithout executing the "else" clause\'s suite. A "continue" statement\nexecuted in the first suite skips the rest of the suite and continues\nwith the next item, or with the "else" clause if there is no next\nitem.\n\nThe for-loop makes assignments to the variables(s) in the target list.\nThis overwrites all previous assignments to those variables including\nthose made in the suite of the for-loop:\n\n for i in range(10):\n print(i)\n i = 5 # this will not affect the for-loop\n # because i will be overwritten with the next\n # index in the range\n\nNames in the target list are not deleted when the loop is finished,\nbut if the sequence is empty, they will not have been assigned to at\nall by the loop. Hint: the built-in function "range()" returns an\niterator of integers suitable to emulate the effect of Pascal\'s "for i\n:= a to b do"; e.g., "list(range(3))" returns the list "[0, 1, 2]".\n\nNote: There is a subtlety when the sequence is being modified by the\n loop (this can only occur for mutable sequences, i.e. lists). An\n internal counter is used to keep track of which item is used next,\n and this is incremented on each iteration. When this counter has\n reached the length of the sequence the loop terminates. This means\n that if the suite deletes the current (or a previous) item from the\n sequence, the next item will be skipped (since it gets the index of\n the current item which has already been treated). Likewise, if the\n suite inserts an item in the sequence before the current item, the\n current item will be treated again the next time through the loop.\n This can lead to nasty bugs that can be avoided by making a\n temporary copy using a slice of the whole sequence, e.g.,\n\n for x in a[:]:\n if x < 0: a.remove(x)\n', - 'formatstrings': u'\nFormat String Syntax\n********************\n\nThe "str.format()" method and the "Formatter" class share the same\nsyntax for format strings (although in the case of "Formatter",\nsubclasses can define their own format string syntax).\n\nFormat strings contain "replacement fields" surrounded by curly braces\n"{}". Anything that is not contained in braces is considered literal\ntext, which is copied unchanged to the output. If you need to include\na brace character in the literal text, it can be escaped by doubling:\n"{{" and "}}".\n\nThe grammar for a replacement field is as follows:\n\n replacement_field ::= "{" [field_name] ["!" conversion] [":" format_spec] "}"\n field_name ::= arg_name ("." attribute_name | "[" element_index "]")*\n arg_name ::= [identifier | integer]\n attribute_name ::= identifier\n element_index ::= integer | index_string\n index_string ::= +\n conversion ::= "r" | "s" | "a"\n format_spec ::= \n\nIn less formal terms, the replacement field can start with a\n*field_name* that specifies the object whose value is to be formatted\nand inserted into the output instead of the replacement field. The\n*field_name* is optionally followed by a *conversion* field, which is\npreceded by an exclamation point "\'!\'", and a *format_spec*, which is\npreceded by a colon "\':\'". These specify a non-default format for the\nreplacement value.\n\nSee also the *Format Specification Mini-Language* section.\n\nThe *field_name* itself begins with an *arg_name* that is either a\nnumber or a keyword. If it\'s a number, it refers to a positional\nargument, and if it\'s a keyword, it refers to a named keyword\nargument. If the numerical arg_names in a format string are 0, 1, 2,\n... in sequence, they can all be omitted (not just some) and the\nnumbers 0, 1, 2, ... will be automatically inserted in that order.\nBecause *arg_name* is not quote-delimited, it is not possible to\nspecify arbitrary dictionary keys (e.g., the strings "\'10\'" or\n"\':-]\'") within a format string. The *arg_name* can be followed by any\nnumber of index or attribute expressions. An expression of the form\n"\'.name\'" selects the named attribute using "getattr()", while an\nexpression of the form "\'[index]\'" does an index lookup using\n"__getitem__()".\n\nChanged in version 3.1: The positional argument specifiers can be\nomitted, so "\'{} {}\'" is equivalent to "\'{0} {1}\'".\n\nSome simple format string examples:\n\n "First, thou shalt count to {0}" # References first positional argument\n "Bring me a {}" # Implicitly references the first positional argument\n "From {} to {}" # Same as "From {0} to {1}"\n "My quest is {name}" # References keyword argument \'name\'\n "Weight in tons {0.weight}" # \'weight\' attribute of first positional arg\n "Units destroyed: {players[0]}" # First element of keyword argument \'players\'.\n\nThe *conversion* field causes a type coercion before formatting.\nNormally, the job of formatting a value is done by the "__format__()"\nmethod of the value itself. However, in some cases it is desirable to\nforce a type to be formatted as a string, overriding its own\ndefinition of formatting. By converting the value to a string before\ncalling "__format__()", the normal formatting logic is bypassed.\n\nThree conversion flags are currently supported: "\'!s\'" which calls\n"str()" on the value, "\'!r\'" which calls "repr()" and "\'!a\'" which\ncalls "ascii()".\n\nSome examples:\n\n "Harold\'s a clever {0!s}" # Calls str() on the argument first\n "Bring out the holy {name!r}" # Calls repr() on the argument first\n "More {!a}" # Calls ascii() on the argument first\n\nThe *format_spec* field contains a specification of how the value\nshould be presented, including such details as field width, alignment,\npadding, decimal precision and so on. Each value type can define its\nown "formatting mini-language" or interpretation of the *format_spec*.\n\nMost built-in types support a common formatting mini-language, which\nis described in the next section.\n\nA *format_spec* field can also include nested replacement fields\nwithin it. These nested replacement fields can contain only a field\nname; conversion flags and format specifications are not allowed. The\nreplacement fields within the format_spec are substituted before the\n*format_spec* string is interpreted. This allows the formatting of a\nvalue to be dynamically specified.\n\nSee the *Format examples* section for some examples.\n\n\nFormat Specification Mini-Language\n==================================\n\n"Format specifications" are used within replacement fields contained\nwithin a format string to define how individual values are presented\n(see *Format String Syntax*). They can also be passed directly to the\nbuilt-in "format()" function. Each formattable type may define how\nthe format specification is to be interpreted.\n\nMost built-in types implement the following options for format\nspecifications, although some of the formatting options are only\nsupported by the numeric types.\n\nA general convention is that an empty format string ("""") produces\nthe same result as if you had called "str()" on the value. A non-empty\nformat string typically modifies the result.\n\nThe general form of a *standard format specifier* is:\n\n format_spec ::= [[fill]align][sign][#][0][width][,][.precision][type]\n fill ::= \n align ::= "<" | ">" | "=" | "^"\n sign ::= "+" | "-" | " "\n width ::= integer\n precision ::= integer\n type ::= "b" | "c" | "d" | "e" | "E" | "f" | "F" | "g" | "G" | "n" | "o" | "s" | "x" | "X" | "%"\n\nIf a valid *align* value is specified, it can be preceded by a *fill*\ncharacter that can be any character and defaults to a space if\nomitted. Note that it is not possible to use "{" and "}" as *fill*\nchar while using the "str.format()" method; this limitation however\ndoesn\'t affect the "format()" function.\n\nThe meaning of the various alignment options is as follows:\n\n +-----------+------------------------------------------------------------+\n | Option | Meaning |\n +===========+============================================================+\n | "\'<\'" | Forces the field to be left-aligned within the available |\n | | space (this is the default for most objects). |\n +-----------+------------------------------------------------------------+\n | "\'>\'" | Forces the field to be right-aligned within the available |\n | | space (this is the default for numbers). |\n +-----------+------------------------------------------------------------+\n | "\'=\'" | Forces the padding to be placed after the sign (if any) |\n | | but before the digits. This is used for printing fields |\n | | in the form \'+000000120\'. This alignment option is only |\n | | valid for numeric types. |\n +-----------+------------------------------------------------------------+\n | "\'^\'" | Forces the field to be centered within the available |\n | | space. |\n +-----------+------------------------------------------------------------+\n\nNote that unless a minimum field width is defined, the field width\nwill always be the same size as the data to fill it, so that the\nalignment option has no meaning in this case.\n\nThe *sign* option is only valid for number types, and can be one of\nthe following:\n\n +-----------+------------------------------------------------------------+\n | Option | Meaning |\n +===========+============================================================+\n | "\'+\'" | indicates that a sign should be used for both positive as |\n | | well as negative numbers. |\n +-----------+------------------------------------------------------------+\n | "\'-\'" | indicates that a sign should be used only for negative |\n | | numbers (this is the default behavior). |\n +-----------+------------------------------------------------------------+\n | space | indicates that a leading space should be used on positive |\n | | numbers, and a minus sign on negative numbers. |\n +-----------+------------------------------------------------------------+\n\nThe "\'#\'" option causes the "alternate form" to be used for the\nconversion. The alternate form is defined differently for different\ntypes. This option is only valid for integer, float, complex and\nDecimal types. For integers, when binary, octal, or hexadecimal output\nis used, this option adds the prefix respective "\'0b\'", "\'0o\'", or\n"\'0x\'" to the output value. For floats, complex and Decimal the\nalternate form causes the result of the conversion to always contain a\ndecimal-point character, even if no digits follow it. Normally, a\ndecimal-point character appears in the result of these conversions\nonly if a digit follows it. In addition, for "\'g\'" and "\'G\'"\nconversions, trailing zeros are not removed from the result.\n\nThe "\',\'" option signals the use of a comma for a thousands separator.\nFor a locale aware separator, use the "\'n\'" integer presentation type\ninstead.\n\nChanged in version 3.1: Added the "\',\'" option (see also **PEP 378**).\n\n*width* is a decimal integer defining the minimum field width. If not\nspecified, then the field width will be determined by the content.\n\nPreceding the *width* field by a zero ("\'0\'") character enables sign-\naware zero-padding for numeric types. This is equivalent to a *fill*\ncharacter of "\'0\'" with an *alignment* type of "\'=\'".\n\nThe *precision* is a decimal number indicating how many digits should\nbe displayed after the decimal point for a floating point value\nformatted with "\'f\'" and "\'F\'", or before and after the decimal point\nfor a floating point value formatted with "\'g\'" or "\'G\'". For non-\nnumber types the field indicates the maximum field size - in other\nwords, how many characters will be used from the field content. The\n*precision* is not allowed for integer values.\n\nFinally, the *type* determines how the data should be presented.\n\nThe available string presentation types are:\n\n +-----------+------------------------------------------------------------+\n | Type | Meaning |\n +===========+============================================================+\n | "\'s\'" | String format. This is the default type for strings and |\n | | may be omitted. |\n +-----------+------------------------------------------------------------+\n | None | The same as "\'s\'". |\n +-----------+------------------------------------------------------------+\n\nThe available integer presentation types are:\n\n +-----------+------------------------------------------------------------+\n | Type | Meaning |\n +===========+============================================================+\n | "\'b\'" | Binary format. Outputs the number in base 2. |\n +-----------+------------------------------------------------------------+\n | "\'c\'" | Character. Converts the integer to the corresponding |\n | | unicode character before printing. |\n +-----------+------------------------------------------------------------+\n | "\'d\'" | Decimal Integer. Outputs the number in base 10. |\n +-----------+------------------------------------------------------------+\n | "\'o\'" | Octal format. Outputs the number in base 8. |\n +-----------+------------------------------------------------------------+\n | "\'x\'" | Hex format. Outputs the number in base 16, using lower- |\n | | case letters for the digits above 9. |\n +-----------+------------------------------------------------------------+\n | "\'X\'" | Hex format. Outputs the number in base 16, using upper- |\n | | case letters for the digits above 9. |\n +-----------+------------------------------------------------------------+\n | "\'n\'" | Number. This is the same as "\'d\'", except that it uses the |\n | | current locale setting to insert the appropriate number |\n | | separator characters. |\n +-----------+------------------------------------------------------------+\n | None | The same as "\'d\'". |\n +-----------+------------------------------------------------------------+\n\nIn addition to the above presentation types, integers can be formatted\nwith the floating point presentation types listed below (except "\'n\'"\nand None). When doing so, "float()" is used to convert the integer to\na floating point number before formatting.\n\nThe available presentation types for floating point and decimal values\nare:\n\n +-----------+------------------------------------------------------------+\n | Type | Meaning |\n +===========+============================================================+\n | "\'e\'" | Exponent notation. Prints the number in scientific |\n | | notation using the letter \'e\' to indicate the exponent. |\n | | The default precision is "6". |\n +-----------+------------------------------------------------------------+\n | "\'E\'" | Exponent notation. Same as "\'e\'" except it uses an upper |\n | | case \'E\' as the separator character. |\n +-----------+------------------------------------------------------------+\n | "\'f\'" | Fixed point. Displays the number as a fixed-point number. |\n | | The default precision is "6". |\n +-----------+------------------------------------------------------------+\n | "\'F\'" | Fixed point. Same as "\'f\'", but converts "nan" to "NAN" |\n | | and "inf" to "INF". |\n +-----------+------------------------------------------------------------+\n | "\'g\'" | General format. For a given precision "p >= 1", this |\n | | rounds the number to "p" significant digits and then |\n | | formats the result in either fixed-point format or in |\n | | scientific notation, depending on its magnitude. The |\n | | precise rules are as follows: suppose that the result |\n | | formatted with presentation type "\'e\'" and precision "p-1" |\n | | would have exponent "exp". Then if "-4 <= exp < p", the |\n | | number is formatted with presentation type "\'f\'" and |\n | | precision "p-1-exp". Otherwise, the number is formatted |\n | | with presentation type "\'e\'" and precision "p-1". In both |\n | | cases insignificant trailing zeros are removed from the |\n | | significand, and the decimal point is also removed if |\n | | there are no remaining digits following it. Positive and |\n | | negative infinity, positive and negative zero, and nans, |\n | | are formatted as "inf", "-inf", "0", "-0" and "nan" |\n | | respectively, regardless of the precision. A precision of |\n | | "0" is treated as equivalent to a precision of "1". The |\n | | default precision is "6". |\n +-----------+------------------------------------------------------------+\n | "\'G\'" | General format. Same as "\'g\'" except switches to "\'E\'" if |\n | | the number gets too large. The representations of infinity |\n | | and NaN are uppercased, too. |\n +-----------+------------------------------------------------------------+\n | "\'n\'" | Number. This is the same as "\'g\'", except that it uses the |\n | | current locale setting to insert the appropriate number |\n | | separator characters. |\n +-----------+------------------------------------------------------------+\n | "\'%\'" | Percentage. Multiplies the number by 100 and displays in |\n | | fixed ("\'f\'") format, followed by a percent sign. |\n +-----------+------------------------------------------------------------+\n | None | Similar to "\'g\'", except that fixed-point notation, when |\n | | used, has at least one digit past the decimal point. The |\n | | default precision is as high as needed to represent the |\n | | particular value. The overall effect is to match the |\n | | output of "str()" as altered by the other format |\n | | modifiers. |\n +-----------+------------------------------------------------------------+\n\n\nFormat examples\n===============\n\nThis section contains examples of the new format syntax and comparison\nwith the old "%"-formatting.\n\nIn most of the cases the syntax is similar to the old "%"-formatting,\nwith the addition of the "{}" and with ":" used instead of "%". For\nexample, "\'%03.2f\'" can be translated to "\'{:03.2f}\'".\n\nThe new format syntax also supports new and different options, shown\nin the follow examples.\n\nAccessing arguments by position:\n\n >>> \'{0}, {1}, {2}\'.format(\'a\', \'b\', \'c\')\n \'a, b, c\'\n >>> \'{}, {}, {}\'.format(\'a\', \'b\', \'c\') # 3.1+ only\n \'a, b, c\'\n >>> \'{2}, {1}, {0}\'.format(\'a\', \'b\', \'c\')\n \'c, b, a\'\n >>> \'{2}, {1}, {0}\'.format(*\'abc\') # unpacking argument sequence\n \'c, b, a\'\n >>> \'{0}{1}{0}\'.format(\'abra\', \'cad\') # arguments\' indices can be repeated\n \'abracadabra\'\n\nAccessing arguments by name:\n\n >>> \'Coordinates: {latitude}, {longitude}\'.format(latitude=\'37.24N\', longitude=\'-115.81W\')\n \'Coordinates: 37.24N, -115.81W\'\n >>> coord = {\'latitude\': \'37.24N\', \'longitude\': \'-115.81W\'}\n >>> \'Coordinates: {latitude}, {longitude}\'.format(**coord)\n \'Coordinates: 37.24N, -115.81W\'\n\nAccessing arguments\' attributes:\n\n >>> c = 3-5j\n >>> (\'The complex number {0} is formed from the real part {0.real} \'\n ... \'and the imaginary part {0.imag}.\').format(c)\n \'The complex number (3-5j) is formed from the real part 3.0 and the imaginary part -5.0.\'\n >>> class Point:\n ... def __init__(self, x, y):\n ... self.x, self.y = x, y\n ... def __str__(self):\n ... return \'Point({self.x}, {self.y})\'.format(self=self)\n ...\n >>> str(Point(4, 2))\n \'Point(4, 2)\'\n\nAccessing arguments\' items:\n\n >>> coord = (3, 5)\n >>> \'X: {0[0]}; Y: {0[1]}\'.format(coord)\n \'X: 3; Y: 5\'\n\nReplacing "%s" and "%r":\n\n >>> "repr() shows quotes: {!r}; str() doesn\'t: {!s}".format(\'test1\', \'test2\')\n "repr() shows quotes: \'test1\'; str() doesn\'t: test2"\n\nAligning the text and specifying a width:\n\n >>> \'{:<30}\'.format(\'left aligned\')\n \'left aligned \'\n >>> \'{:>30}\'.format(\'right aligned\')\n \' right aligned\'\n >>> \'{:^30}\'.format(\'centered\')\n \' centered \'\n >>> \'{:*^30}\'.format(\'centered\') # use \'*\' as a fill char\n \'***********centered***********\'\n\nReplacing "%+f", "%-f", and "% f" and specifying a sign:\n\n >>> \'{:+f}; {:+f}\'.format(3.14, -3.14) # show it always\n \'+3.140000; -3.140000\'\n >>> \'{: f}; {: f}\'.format(3.14, -3.14) # show a space for positive numbers\n \' 3.140000; -3.140000\'\n >>> \'{:-f}; {:-f}\'.format(3.14, -3.14) # show only the minus -- same as \'{:f}; {:f}\'\n \'3.140000; -3.140000\'\n\nReplacing "%x" and "%o" and converting the value to different bases:\n\n >>> # format also supports binary numbers\n >>> "int: {0:d}; hex: {0:x}; oct: {0:o}; bin: {0:b}".format(42)\n \'int: 42; hex: 2a; oct: 52; bin: 101010\'\n >>> # with 0x, 0o, or 0b as prefix:\n >>> "int: {0:d}; hex: {0:#x}; oct: {0:#o}; bin: {0:#b}".format(42)\n \'int: 42; hex: 0x2a; oct: 0o52; bin: 0b101010\'\n\nUsing the comma as a thousands separator:\n\n >>> \'{:,}\'.format(1234567890)\n \'1,234,567,890\'\n\nExpressing a percentage:\n\n >>> points = 19\n >>> total = 22\n >>> \'Correct answers: {:.2%}\'.format(points/total)\n \'Correct answers: 86.36%\'\n\nUsing type-specific formatting:\n\n >>> import datetime\n >>> d = datetime.datetime(2010, 7, 4, 12, 15, 58)\n >>> \'{:%Y-%m-%d %H:%M:%S}\'.format(d)\n \'2010-07-04 12:15:58\'\n\nNesting arguments and more complex examples:\n\n >>> for align, text in zip(\'<^>\', [\'left\', \'center\', \'right\']):\n ... \'{0:{fill}{align}16}\'.format(text, fill=align, align=align)\n ...\n \'left<<<<<<<<<<<<\'\n \'^^^^^center^^^^^\'\n \'>>>>>>>>>>>right\'\n >>>\n >>> octets = [192, 168, 0, 1]\n >>> \'{:02X}{:02X}{:02X}{:02X}\'.format(*octets)\n \'C0A80001\'\n >>> int(_, 16)\n 3232235521\n >>>\n >>> width = 5\n >>> for num in range(5,12): #doctest: +NORMALIZE_WHITESPACE\n ... for base in \'dXob\':\n ... print(\'{0:{width}{base}}\'.format(num, base=base, width=width), end=\' \')\n ... print()\n ...\n 5 5 5 101\n 6 6 6 110\n 7 7 7 111\n 8 8 10 1000\n 9 9 11 1001\n 10 A 12 1010\n 11 B 13 1011\n', - 'function': u'\nFunction definitions\n********************\n\nA function definition defines a user-defined function object (see\nsection *The standard type hierarchy*):\n\n funcdef ::= [decorators] "def" funcname "(" [parameter_list] ")" ["->" expression] ":" suite\n decorators ::= decorator+\n decorator ::= "@" dotted_name ["(" [parameter_list [","]] ")"] NEWLINE\n dotted_name ::= identifier ("." identifier)*\n parameter_list ::= (defparameter ",")*\n | "*" [parameter] ("," defparameter)* ["," "**" parameter]\n | "**" parameter\n | defparameter [","] )\n parameter ::= identifier [":" expression]\n defparameter ::= parameter ["=" expression]\n funcname ::= identifier\n\nA function definition is an executable statement. Its execution binds\nthe function name in the current local namespace to a function object\n(a wrapper around the executable code for the function). This\nfunction object contains a reference to the current global namespace\nas the global namespace to be used when the function is called.\n\nThe function definition does not execute the function body; this gets\nexecuted only when the function is called. [3]\n\nA function definition may be wrapped by one or more *decorator*\nexpressions. Decorator expressions are evaluated when the function is\ndefined, in the scope that contains the function definition. The\nresult must be a callable, which is invoked with the function object\nas the only argument. The returned value is bound to the function name\ninstead of the function object. Multiple decorators are applied in\nnested fashion. For example, the following code\n\n @f1(arg)\n @f2\n def func(): pass\n\nis equivalent to\n\n def func(): pass\n func = f1(arg)(f2(func))\n\nWhen one or more *parameters* have the form *parameter* "="\n*expression*, the function is said to have "default parameter values."\nFor a parameter with a default value, the corresponding *argument* may\nbe omitted from a call, in which case the parameter\'s default value is\nsubstituted. If a parameter has a default value, all following\nparameters up until the ""*"" must also have a default value --- this\nis a syntactic restriction that is not expressed by the grammar.\n\n**Default parameter values are evaluated from left to right when the\nfunction definition is executed.** This means that the expression is\nevaluated once, when the function is defined, and that the same "pre-\ncomputed" value is used for each call. This is especially important\nto understand when a default parameter is a mutable object, such as a\nlist or a dictionary: if the function modifies the object (e.g. by\nappending an item to a list), the default value is in effect modified.\nThis is generally not what was intended. A way around this is to use\n"None" as the default, and explicitly test for it in the body of the\nfunction, e.g.:\n\n def whats_on_the_telly(penguin=None):\n if penguin is None:\n penguin = []\n penguin.append("property of the zoo")\n return penguin\n\nFunction call semantics are described in more detail in section\n*Calls*. A function call always assigns values to all parameters\nmentioned in the parameter list, either from position arguments, from\nkeyword arguments, or from default values. If the form\n""*identifier"" is present, it is initialized to a tuple receiving any\nexcess positional parameters, defaulting to the empty tuple. If the\nform ""**identifier"" is present, it is initialized to a new\ndictionary receiving any excess keyword arguments, defaulting to a new\nempty dictionary. Parameters after ""*"" or ""*identifier"" are\nkeyword-only parameters and may only be passed used keyword arguments.\n\nParameters may have annotations of the form "": expression"" following\nthe parameter name. Any parameter may have an annotation even those\nof the form "*identifier" or "**identifier". Functions may have\n"return" annotation of the form ""-> expression"" after the parameter\nlist. These annotations can be any valid Python expression and are\nevaluated when the function definition is executed. Annotations may\nbe evaluated in a different order than they appear in the source code.\nThe presence of annotations does not change the semantics of a\nfunction. The annotation values are available as values of a\ndictionary keyed by the parameters\' names in the "__annotations__"\nattribute of the function object.\n\nIt is also possible to create anonymous functions (functions not bound\nto a name), for immediate use in expressions. This uses lambda\nexpressions, described in section *Lambdas*. Note that the lambda\nexpression is merely a shorthand for a simplified function definition;\na function defined in a ""def"" statement can be passed around or\nassigned to another name just like a function defined by a lambda\nexpression. The ""def"" form is actually more powerful since it\nallows the execution of multiple statements and annotations.\n\n**Programmer\'s note:** Functions are first-class objects. A ""def""\nstatement executed inside a function definition defines a local\nfunction that can be returned or passed around. Free variables used\nin the nested function can access the local variables of the function\ncontaining the def. See section *Naming and binding* for details.\n\nSee also: **PEP 3107** - Function Annotations\n\n The original specification for function annotations.\n', - 'global': u'\nThe "global" statement\n**********************\n\n global_stmt ::= "global" identifier ("," identifier)*\n\nThe "global" statement is a declaration which holds for the entire\ncurrent code block. It means that the listed identifiers are to be\ninterpreted as globals. It would be impossible to assign to a global\nvariable without "global", although free variables may refer to\nglobals without being declared global.\n\nNames listed in a "global" statement must not be used in the same code\nblock textually preceding that "global" statement.\n\nNames listed in a "global" statement must not be defined as formal\nparameters or in a "for" loop control target, "class" definition,\nfunction definition, or "import" statement.\n\n**CPython implementation detail:** The current implementation does not\nenforce the two restrictions, but programs should not abuse this\nfreedom, as future implementations may enforce them or silently change\nthe meaning of the program.\n\n**Programmer\'s note:** the "global" is a directive to the parser. It\napplies only to code parsed at the same time as the "global"\nstatement. In particular, a "global" statement contained in a string\nor code object supplied to the built-in "exec()" function does not\naffect the code block *containing* the function call, and code\ncontained in such a string is unaffected by "global" statements in the\ncode containing the function call. The same applies to the "eval()"\nand "compile()" functions.\n', - 'id-classes': u'\nReserved classes of identifiers\n*******************************\n\nCertain classes of identifiers (besides keywords) have special\nmeanings. These classes are identified by the patterns of leading and\ntrailing underscore characters:\n\n"_*"\n Not imported by "from module import *". The special identifier "_"\n is used in the interactive interpreter to store the result of the\n last evaluation; it is stored in the "builtins" module. When not\n in interactive mode, "_" has no special meaning and is not defined.\n See section *The import statement*.\n\n Note: The name "_" is often used in conjunction with\n internationalization; refer to the documentation for the\n "gettext" module for more information on this convention.\n\n"__*__"\n System-defined names. These names are defined by the interpreter\n and its implementation (including the standard library). Current\n system names are discussed in the *Special method names* section\n and elsewhere. More will likely be defined in future versions of\n Python. *Any* use of "__*__" names, in any context, that does not\n follow explicitly documented use, is subject to breakage without\n warning.\n\n"__*"\n Class-private names. Names in this category, when used within the\n context of a class definition, are re-written to use a mangled form\n to help avoid name clashes between "private" attributes of base and\n derived classes. See section *Identifiers (Names)*.\n', - 'identifiers': u'\nIdentifiers and keywords\n************************\n\nIdentifiers (also referred to as *names*) are described by the\nfollowing lexical definitions.\n\nThe syntax of identifiers in Python is based on the Unicode standard\nannex UAX-31, with elaboration and changes as defined below; see also\n**PEP 3131** for further details.\n\nWithin the ASCII range (U+0001..U+007F), the valid characters for\nidentifiers are the same as in Python 2.x: the uppercase and lowercase\nletters "A" through "Z", the underscore "_" and, except for the first\ncharacter, the digits "0" through "9".\n\nPython 3.0 introduces additional characters from outside the ASCII\nrange (see **PEP 3131**). For these characters, the classification\nuses the version of the Unicode Character Database as included in the\n"unicodedata" module.\n\nIdentifiers are unlimited in length. Case is significant.\n\n identifier ::= xid_start xid_continue*\n id_start ::= \n id_continue ::= \n xid_start ::= \n xid_continue ::= \n\nThe Unicode category codes mentioned above stand for:\n\n* *Lu* - uppercase letters\n\n* *Ll* - lowercase letters\n\n* *Lt* - titlecase letters\n\n* *Lm* - modifier letters\n\n* *Lo* - other letters\n\n* *Nl* - letter numbers\n\n* *Mn* - nonspacing marks\n\n* *Mc* - spacing combining marks\n\n* *Nd* - decimal numbers\n\n* *Pc* - connector punctuations\n\n* *Other_ID_Start* - explicit list of characters in PropList.txt to\n support backwards compatibility\n\n* *Other_ID_Continue* - likewise\n\nAll identifiers are converted into the normal form NFKC while parsing;\ncomparison of identifiers is based on NFKC.\n\nA non-normative HTML file listing all valid identifier characters for\nUnicode 4.1 can be found at http://www.dcl.hpi.uni-\npotsdam.de/home/loewis/table-3131.html.\n\n\nKeywords\n========\n\nThe following identifiers are used as reserved words, or *keywords* of\nthe language, and cannot be used as ordinary identifiers. They must\nbe spelled exactly as written here:\n\n False class finally is return\n None continue for lambda try\n True def from nonlocal while\n and del global not with\n as elif if or yield\n assert else import pass\n break except in raise\n\n\nReserved classes of identifiers\n===============================\n\nCertain classes of identifiers (besides keywords) have special\nmeanings. These classes are identified by the patterns of leading and\ntrailing underscore characters:\n\n"_*"\n Not imported by "from module import *". The special identifier "_"\n is used in the interactive interpreter to store the result of the\n last evaluation; it is stored in the "builtins" module. When not\n in interactive mode, "_" has no special meaning and is not defined.\n See section *The import statement*.\n\n Note: The name "_" is often used in conjunction with\n internationalization; refer to the documentation for the\n "gettext" module for more information on this convention.\n\n"__*__"\n System-defined names. These names are defined by the interpreter\n and its implementation (including the standard library). Current\n system names are discussed in the *Special method names* section\n and elsewhere. More will likely be defined in future versions of\n Python. *Any* use of "__*__" names, in any context, that does not\n follow explicitly documented use, is subject to breakage without\n warning.\n\n"__*"\n Class-private names. Names in this category, when used within the\n context of a class definition, are re-written to use a mangled form\n to help avoid name clashes between "private" attributes of base and\n derived classes. See section *Identifiers (Names)*.\n', - 'if': u'\nThe "if" statement\n******************\n\nThe "if" statement is used for conditional execution:\n\n if_stmt ::= "if" expression ":" suite\n ( "elif" expression ":" suite )*\n ["else" ":" suite]\n\nIt selects exactly one of the suites by evaluating the expressions one\nby one until one is found to be true (see section *Boolean operations*\nfor the definition of true and false); then that suite is executed\n(and no other part of the "if" statement is executed or evaluated).\nIf all expressions are false, the suite of the "else" clause, if\npresent, is executed.\n', - 'imaginary': u'\nImaginary literals\n******************\n\nImaginary literals are described by the following lexical definitions:\n\n imagnumber ::= (floatnumber | intpart) ("j" | "J")\n\nAn imaginary literal yields a complex number with a real part of 0.0.\nComplex numbers are represented as a pair of floating point numbers\nand have the same restrictions on their range. To create a complex\nnumber with a nonzero real part, add a floating point number to it,\ne.g., "(3+4j)". Some examples of imaginary literals:\n\n 3.14j 10.j 10j .001j 1e100j 3.14e-10j\n', - 'import': u'\nThe "import" statement\n**********************\n\n import_stmt ::= "import" module ["as" name] ( "," module ["as" name] )*\n | "from" relative_module "import" identifier ["as" name]\n ( "," identifier ["as" name] )*\n | "from" relative_module "import" "(" identifier ["as" name]\n ( "," identifier ["as" name] )* [","] ")"\n | "from" module "import" "*"\n module ::= (identifier ".")* identifier\n relative_module ::= "."* module | "."+\n name ::= identifier\n\nThe basic import statement (no "from" clause) is executed in two\nsteps:\n\n1. find a module, loading and initializing it if necessary\n\n2. define a name or names in the local namespace for the scope\n where the "import" statement occurs.\n\nWhen the statement contains multiple clauses (separated by commas) the\ntwo steps are carried out separately for each clause, just as though\nthe clauses had been separated out into individiual import statements.\n\nThe details of the first step, finding and loading modules are\ndescribed in greater detail in the section on the *import system*,\nwhich also describes the various types of packages and modules that\ncan be imported, as well as all the hooks that can be used to\ncustomize the import system. Note that failures in this step may\nindicate either that the module could not be located, *or* that an\nerror occurred while initializing the module, which includes execution\nof the module\'s code.\n\nIf the requested module is retrieved successfully, it will be made\navailable in the local namespace in one of three ways:\n\n* If the module name is followed by "as", then the name following\n "as" is bound directly to the imported module.\n\n* If no other name is specified, and the module being imported is a\n top level module, the module\'s name is bound in the local namespace\n as a reference to the imported module\n\n* If the module being imported is *not* a top level module, then the\n name of the top level package that contains the module is bound in\n the local namespace as a reference to the top level package. The\n imported module must be accessed using its full qualified name\n rather than directly\n\nThe "from" form uses a slightly more complex process:\n\n1. find the module specified in the "from" clause, loading and\n initializing it if necessary;\n\n2. for each of the identifiers specified in the "import" clauses:\n\n 1. check if the imported module has an attribute by that name\n\n 2. if not, attempt to import a submodule with that name and then\n check the imported module again for that attribute\n\n 3. if the attribute is not found, "ImportError" is raised.\n\n 4. otherwise, a reference to that value is stored in the local\n namespace, using the name in the "as" clause if it is present,\n otherwise using the attribute name\n\nExamples:\n\n import foo # foo imported and bound locally\n import foo.bar.baz # foo.bar.baz imported, foo bound locally\n import foo.bar.baz as fbb # foo.bar.baz imported and bound as fbb\n from foo.bar import baz # foo.bar.baz imported and bound as baz\n from foo import attr # foo imported and foo.attr bound as attr\n\nIf the list of identifiers is replaced by a star ("\'*\'"), all public\nnames defined in the module are bound in the local namespace for the\nscope where the "import" statement occurs.\n\nThe *public names* defined by a module are determined by checking the\nmodule\'s namespace for a variable named "__all__"; if defined, it must\nbe a sequence of strings which are names defined or imported by that\nmodule. The names given in "__all__" are all considered public and\nare required to exist. If "__all__" is not defined, the set of public\nnames includes all names found in the module\'s namespace which do not\nbegin with an underscore character ("\'_\'"). "__all__" should contain\nthe entire public API. It is intended to avoid accidentally exporting\nitems that are not part of the API (such as library modules which were\nimported and used within the module).\n\nThe wild card form of import --- "from module import *" --- is only\nallowed at the module level. Attempting to use it in class or\nfunction definitions will raise a "SyntaxError".\n\nWhen specifying what module to import you do not have to specify the\nabsolute name of the module. When a module or package is contained\nwithin another package it is possible to make a relative import within\nthe same top package without having to mention the package name. By\nusing leading dots in the specified module or package after "from" you\ncan specify how high to traverse up the current package hierarchy\nwithout specifying exact names. One leading dot means the current\npackage where the module making the import exists. Two dots means up\none package level. Three dots is up two levels, etc. So if you execute\n"from . import mod" from a module in the "pkg" package then you will\nend up importing "pkg.mod". If you execute "from ..subpkg2 import mod"\nfrom within "pkg.subpkg1" you will import "pkg.subpkg2.mod". The\nspecification for relative imports is contained within **PEP 328**.\n\n"importlib.import_module()" is provided to support applications that\ndetermine dynamically the modules to be loaded.\n\n\nFuture statements\n=================\n\nA *future statement* is a directive to the compiler that a particular\nmodule should be compiled using syntax or semantics that will be\navailable in a specified future release of Python where the feature\nbecomes standard.\n\nThe future statement is intended to ease migration to future versions\nof Python that introduce incompatible changes to the language. It\nallows use of the new features on a per-module basis before the\nrelease in which the feature becomes standard.\n\n future_statement ::= "from" "__future__" "import" feature ["as" name]\n ("," feature ["as" name])*\n | "from" "__future__" "import" "(" feature ["as" name]\n ("," feature ["as" name])* [","] ")"\n feature ::= identifier\n name ::= identifier\n\nA future statement must appear near the top of the module. The only\nlines that can appear before a future statement are:\n\n* the module docstring (if any),\n\n* comments,\n\n* blank lines, and\n\n* other future statements.\n\nThe features recognized by Python 3.0 are "absolute_import",\n"division", "generators", "unicode_literals", "print_function",\n"nested_scopes" and "with_statement". They are all redundant because\nthey are always enabled, and only kept for backwards compatibility.\n\nA future statement is recognized and treated specially at compile\ntime: Changes to the semantics of core constructs are often\nimplemented by generating different code. It may even be the case\nthat a new feature introduces new incompatible syntax (such as a new\nreserved word), in which case the compiler may need to parse the\nmodule differently. Such decisions cannot be pushed off until\nruntime.\n\nFor any given release, the compiler knows which feature names have\nbeen defined, and raises a compile-time error if a future statement\ncontains a feature not known to it.\n\nThe direct runtime semantics are the same as for any import statement:\nthere is a standard module "__future__", described later, and it will\nbe imported in the usual way at the time the future statement is\nexecuted.\n\nThe interesting runtime semantics depend on the specific feature\nenabled by the future statement.\n\nNote that there is nothing special about the statement:\n\n import __future__ [as name]\n\nThat is not a future statement; it\'s an ordinary import statement with\nno special semantics or syntax restrictions.\n\nCode compiled by calls to the built-in functions "exec()" and\n"compile()" that occur in a module "M" containing a future statement\nwill, by default, use the new syntax or semantics associated with the\nfuture statement. This can be controlled by optional arguments to\n"compile()" --- see the documentation of that function for details.\n\nA future statement typed at an interactive interpreter prompt will\ntake effect for the rest of the interpreter session. If an\ninterpreter is started with the *-i* option, is passed a script name\nto execute, and the script includes a future statement, it will be in\neffect in the interactive session started after the script is\nexecuted.\n\nSee also: **PEP 236** - Back to the __future__\n\n The original proposal for the __future__ mechanism.\n', - 'in': u'\nComparisons\n***********\n\nUnlike C, all comparison operations in Python have the same priority,\nwhich is lower than that of any arithmetic, shifting or bitwise\noperation. Also unlike C, expressions like "a < b < c" have the\ninterpretation that is conventional in mathematics:\n\n comparison ::= or_expr ( comp_operator or_expr )*\n comp_operator ::= "<" | ">" | "==" | ">=" | "<=" | "!="\n | "is" ["not"] | ["not"] "in"\n\nComparisons yield boolean values: "True" or "False".\n\nComparisons can be chained arbitrarily, e.g., "x < y <= z" is\nequivalent to "x < y and y <= z", except that "y" is evaluated only\nonce (but in both cases "z" is not evaluated at all when "x < y" is\nfound to be false).\n\nFormally, if *a*, *b*, *c*, ..., *y*, *z* are expressions and *op1*,\n*op2*, ..., *opN* are comparison operators, then "a op1 b op2 c ... y\nopN z" is equivalent to "a op1 b and b op2 c and ... y opN z", except\nthat each expression is evaluated at most once.\n\nNote that "a op1 b op2 c" doesn\'t imply any kind of comparison between\n*a* and *c*, so that, e.g., "x < y > z" is perfectly legal (though\nperhaps not pretty).\n\nThe operators "<", ">", "==", ">=", "<=", and "!=" compare the values\nof two objects. The objects need not have the same type. If both are\nnumbers, they are converted to a common type. Otherwise, the "==" and\n"!=" operators *always* consider objects of different types to be\nunequal, while the "<", ">", ">=" and "<=" operators raise a\n"TypeError" when comparing objects of different types that do not\nimplement these operators for the given pair of types. You can\ncontrol comparison behavior of objects of non-built-in types by\ndefining rich comparison methods like "__gt__()", described in section\n*Basic customization*.\n\nComparison of objects of the same type depends on the type:\n\n* Numbers are compared arithmetically.\n\n* The values "float(\'NaN\')" and "Decimal(\'NaN\')" are special. They\n are identical to themselves, "x is x" but are not equal to\n themselves, "x != x". Additionally, comparing any value to a\n not-a-number value will return "False". For example, both "3 <\n float(\'NaN\')" and "float(\'NaN\') < 3" will return "False".\n\n* Bytes objects are compared lexicographically using the numeric\n values of their elements.\n\n* Strings are compared lexicographically using the numeric\n equivalents (the result of the built-in function "ord()") of their\n characters. [3] String and bytes object can\'t be compared!\n\n* Tuples and lists are compared lexicographically using comparison\n of corresponding elements. This means that to compare equal, each\n element must compare equal and the two sequences must be of the same\n type and have the same length.\n\n If not equal, the sequences are ordered the same as their first\n differing elements. For example, "[1,2,x] <= [1,2,y]" has the same\n value as "x <= y". If the corresponding element does not exist, the\n shorter sequence is ordered first (for example, "[1,2] < [1,2,3]").\n\n* Mappings (dictionaries) compare equal if and only if they have the\n same "(key, value)" pairs. Order comparisons "(\'<\', \'<=\', \'>=\',\n \'>\')" raise "TypeError".\n\n* Sets and frozensets define comparison operators to mean subset and\n superset tests. Those relations do not define total orderings (the\n two sets "{1,2}" and "{2,3}" are not equal, nor subsets of one\n another, nor supersets of one another). Accordingly, sets are not\n appropriate arguments for functions which depend on total ordering.\n For example, "min()", "max()", and "sorted()" produce undefined\n results given a list of sets as inputs.\n\n* Most other objects of built-in types compare unequal unless they\n are the same object; the choice whether one object is considered\n smaller or larger than another one is made arbitrarily but\n consistently within one execution of a program.\n\nComparison of objects of differing types depends on whether either of\nthe types provide explicit support for the comparison. Most numeric\ntypes can be compared with one another. When cross-type comparison is\nnot supported, the comparison method returns "NotImplemented".\n\nThe operators "in" and "not in" test for membership. "x in s"\nevaluates to true if *x* is a member of *s*, and false otherwise. "x\nnot in s" returns the negation of "x in s". All built-in sequences\nand set types support this as well as dictionary, for which "in" tests\nwhether the dictionary has a given key. For container types such as\nlist, tuple, set, frozenset, dict, or collections.deque, the\nexpression "x in y" is equivalent to "any(x is e or x == e for e in\ny)".\n\nFor the string and bytes types, "x in y" is true if and only if *x* is\na substring of *y*. An equivalent test is "y.find(x) != -1". Empty\nstrings are always considered to be a substring of any other string,\nso """ in "abc"" will return "True".\n\nFor user-defined classes which define the "__contains__()" method, "x\nin y" is true if and only if "y.__contains__(x)" is true.\n\nFor user-defined classes which do not define "__contains__()" but do\ndefine "__iter__()", "x in y" is true if some value "z" with "x == z"\nis produced while iterating over "y". If an exception is raised\nduring the iteration, it is as if "in" raised that exception.\n\nLastly, the old-style iteration protocol is tried: if a class defines\n"__getitem__()", "x in y" is true if and only if there is a non-\nnegative integer index *i* such that "x == y[i]", and all lower\ninteger indices do not raise "IndexError" exception. (If any other\nexception is raised, it is as if "in" raised that exception).\n\nThe operator "not in" is defined to have the inverse true value of\n"in".\n\nThe operators "is" and "is not" test for object identity: "x is y" is\ntrue if and only if *x* and *y* are the same object. "x is not y"\nyields the inverse truth value. [4]\n', - 'integers': u'\nInteger literals\n****************\n\nInteger literals are described by the following lexical definitions:\n\n integer ::= decimalinteger | octinteger | hexinteger | bininteger\n decimalinteger ::= nonzerodigit digit* | "0"+\n nonzerodigit ::= "1"..."9"\n digit ::= "0"..."9"\n octinteger ::= "0" ("o" | "O") octdigit+\n hexinteger ::= "0" ("x" | "X") hexdigit+\n bininteger ::= "0" ("b" | "B") bindigit+\n octdigit ::= "0"..."7"\n hexdigit ::= digit | "a"..."f" | "A"..."F"\n bindigit ::= "0" | "1"\n\nThere is no limit for the length of integer literals apart from what\ncan be stored in available memory.\n\nNote that leading zeros in a non-zero decimal number are not allowed.\nThis is for disambiguation with C-style octal literals, which Python\nused before version 3.0.\n\nSome examples of integer literals:\n\n 7 2147483647 0o177 0b100110111\n 3 79228162514264337593543950336 0o377 0xdeadbeef\n', - 'lambda': u'\nLambdas\n*******\n\n lambda_expr ::= "lambda" [parameter_list]: expression\n lambda_expr_nocond ::= "lambda" [parameter_list]: expression_nocond\n\nLambda expressions (sometimes called lambda forms) are used to create\nanonymous functions. The expression "lambda arguments: expression"\nyields a function object. The unnamed object behaves like a function\nobject defined with\n\n def (arguments):\n return expression\n\nSee section *Function definitions* for the syntax of parameter lists.\nNote that functions created with lambda expressions cannot contain\nstatements or annotations.\n', - 'lists': u'\nList displays\n*************\n\nA list display is a possibly empty series of expressions enclosed in\nsquare brackets:\n\n list_display ::= "[" [expression_list | comprehension] "]"\n\nA list display yields a new list object, the contents being specified\nby either a list of expressions or a comprehension. When a comma-\nseparated list of expressions is supplied, its elements are evaluated\nfrom left to right and placed into the list object in that order.\nWhen a comprehension is supplied, the list is constructed from the\nelements resulting from the comprehension.\n', - 'naming': u'\nNaming and binding\n******************\n\n\nBinding of names\n================\n\n*Names* refer to objects. Names are introduced by name binding\noperations.\n\nThe following constructs bind names: formal parameters to functions,\n"import" statements, class and function definitions (these bind the\nclass or function name in the defining block), and targets that are\nidentifiers if occurring in an assignment, "for" loop header, or after\n"as" in a "with" statement or "except" clause. The "import" statement\nof the form "from ... import *" binds all names defined in the\nimported module, except those beginning with an underscore. This form\nmay only be used at the module level.\n\nA target occurring in a "del" statement is also considered bound for\nthis purpose (though the actual semantics are to unbind the name).\n\nEach assignment or import statement occurs within a block defined by a\nclass or function definition or at the module level (the top-level\ncode block).\n\nIf a name is bound in a block, it is a local variable of that block,\nunless declared as "nonlocal" or "global". If a name is bound at the\nmodule level, it is a global variable. (The variables of the module\ncode block are local and global.) If a variable is used in a code\nblock but not defined there, it is a *free variable*.\n\nEach occurrence of a name in the program text refers to the *binding*\nof that name established by the following name resolution rules.\n\n\nResolution of names\n===================\n\nA *scope* defines the visibility of a name within a block. If a local\nvariable is defined in a block, its scope includes that block. If the\ndefinition occurs in a function block, the scope extends to any blocks\ncontained within the defining one, unless a contained block introduces\na different binding for the name.\n\nWhen a name is used in a code block, it is resolved using the nearest\nenclosing scope. The set of all such scopes visible to a code block\nis called the block\'s *environment*.\n\nWhen a name is not found at all, a "NameError" exception is raised. If\nthe current scope is a function scope, and the name refers to a local\nvariable that has not yet been bound to a value at the point where the\nname is used, an "UnboundLocalError" exception is raised.\n"UnboundLocalError" is a subclass of "NameError".\n\nIf a name binding operation occurs anywhere within a code block, all\nuses of the name within the block are treated as references to the\ncurrent block. This can lead to errors when a name is used within a\nblock before it is bound. This rule is subtle. Python lacks\ndeclarations and allows name binding operations to occur anywhere\nwithin a code block. The local variables of a code block can be\ndetermined by scanning the entire text of the block for name binding\noperations.\n\nIf the "global" statement occurs within a block, all uses of the name\nspecified in the statement refer to the binding of that name in the\ntop-level namespace. Names are resolved in the top-level namespace by\nsearching the global namespace, i.e. the namespace of the module\ncontaining the code block, and the builtins namespace, the namespace\nof the module "builtins". The global namespace is searched first. If\nthe name is not found there, the builtins namespace is searched. The\n"global" statement must precede all uses of the name.\n\nThe "global" statement has the same scope as a name binding operation\nin the same block. If the nearest enclosing scope for a free variable\ncontains a global statement, the free variable is treated as a global.\n\nThe "nonlocal" statement causes corresponding names to refer to\npreviously bound variables in the nearest enclosing function scope.\n"SyntaxError" is raised at compile time if the given name does not\nexist in any enclosing function scope.\n\nThe namespace for a module is automatically created the first time a\nmodule is imported. The main module for a script is always called\n"__main__".\n\nClass definition blocks and arguments to "exec()" and "eval()" are\nspecial in the context of name resolution. A class definition is an\nexecutable statement that may use and define names. These references\nfollow the normal rules for name resolution with an exception that\nunbound local variables are looked up in the global namespace. The\nnamespace of the class definition becomes the attribute dictionary of\nthe class. The scope of names defined in a class block is limited to\nthe class block; it does not extend to the code blocks of methods --\nthis includes comprehensions and generator expressions since they are\nimplemented using a function scope. This means that the following\nwill fail:\n\n class A:\n a = 42\n b = list(a + i for i in range(10))\n\n\nBuiltins and restricted execution\n=================================\n\nThe builtins namespace associated with the execution of a code block\nis actually found by looking up the name "__builtins__" in its global\nnamespace; this should be a dictionary or a module (in the latter case\nthe module\'s dictionary is used). By default, when in the "__main__"\nmodule, "__builtins__" is the built-in module "builtins"; when in any\nother module, "__builtins__" is an alias for the dictionary of the\n"builtins" module itself. "__builtins__" can be set to a user-created\ndictionary to create a weak form of restricted execution.\n\n**CPython implementation detail:** Users should not touch\n"__builtins__"; it is strictly an implementation detail. Users\nwanting to override values in the builtins namespace should "import"\nthe "builtins" module and modify its attributes appropriately.\n\n\nInteraction with dynamic features\n=================================\n\nName resolution of free variables occurs at runtime, not at compile\ntime. This means that the following code will print 42:\n\n i = 10\n def f():\n print(i)\n i = 42\n f()\n\nThere are several cases where Python statements are illegal when used\nin conjunction with nested scopes that contain free variables.\n\nIf a variable is referenced in an enclosing scope, it is illegal to\ndelete the name. An error will be reported at compile time.\n\nThe "eval()" and "exec()" functions do not have access to the full\nenvironment for resolving names. Names may be resolved in the local\nand global namespaces of the caller. Free variables are not resolved\nin the nearest enclosing namespace, but in the global namespace. [1]\nThe "exec()" and "eval()" functions have optional arguments to\noverride the global and local namespace. If only one namespace is\nspecified, it is used for both.\n', - 'nonlocal': u'\nThe "nonlocal" statement\n************************\n\n nonlocal_stmt ::= "nonlocal" identifier ("," identifier)*\n\nThe "nonlocal" statement causes the listed identifiers to refer to\npreviously bound variables in the nearest enclosing scope excluding\nglobals. This is important because the default behavior for binding is\nto search the local namespace first. The statement allows\nencapsulated code to rebind variables outside of the local scope\nbesides the global (module) scope.\n\nNames listed in a "nonlocal" statement, unlike those listed in a\n"global" statement, must refer to pre-existing bindings in an\nenclosing scope (the scope in which a new binding should be created\ncannot be determined unambiguously).\n\nNames listed in a "nonlocal" statement must not collide with pre-\nexisting bindings in the local scope.\n\nSee also: **PEP 3104** - Access to Names in Outer Scopes\n\n The specification for the "nonlocal" statement.\n', - 'numbers': u'\nNumeric literals\n****************\n\nThere are three types of numeric literals: integers, floating point\nnumbers, and imaginary numbers. There are no complex literals\n(complex numbers can be formed by adding a real number and an\nimaginary number).\n\nNote that numeric literals do not include a sign; a phrase like "-1"\nis actually an expression composed of the unary operator \'"-"\' and the\nliteral "1".\n', - 'numeric-types': u'\nEmulating numeric types\n***********************\n\nThe following methods can be defined to emulate numeric objects.\nMethods corresponding to operations that are not supported by the\nparticular kind of number implemented (e.g., bitwise operations for\nnon-integral numbers) should be left undefined.\n\nobject.__add__(self, other)\nobject.__sub__(self, other)\nobject.__mul__(self, other)\nobject.__matmul__(self, other)\nobject.__truediv__(self, other)\nobject.__floordiv__(self, other)\nobject.__mod__(self, other)\nobject.__divmod__(self, other)\nobject.__pow__(self, other[, modulo])\nobject.__lshift__(self, other)\nobject.__rshift__(self, other)\nobject.__and__(self, other)\nobject.__xor__(self, other)\nobject.__or__(self, other)\n\n These methods are called to implement the binary arithmetic\n operations ("+", "-", "*", "@", "/", "//", "%", "divmod()",\n "pow()", "**", "<<", ">>", "&", "^", "|"). For instance, to\n evaluate the expression "x + y", where *x* is an instance of a\n class that has an "__add__()" method, "x.__add__(y)" is called.\n The "__divmod__()" method should be the equivalent to using\n "__floordiv__()" and "__mod__()"; it should not be related to\n "__truediv__()". Note that "__pow__()" should be defined to accept\n an optional third argument if the ternary version of the built-in\n "pow()" function is to be supported.\n\n If one of those methods does not support the operation with the\n supplied arguments, it should return "NotImplemented".\n\nobject.__radd__(self, other)\nobject.__rsub__(self, other)\nobject.__rmul__(self, other)\nobject.__rmatmul__(self, other)\nobject.__rtruediv__(self, other)\nobject.__rfloordiv__(self, other)\nobject.__rmod__(self, other)\nobject.__rdivmod__(self, other)\nobject.__rpow__(self, other)\nobject.__rlshift__(self, other)\nobject.__rrshift__(self, other)\nobject.__rand__(self, other)\nobject.__rxor__(self, other)\nobject.__ror__(self, other)\n\n These methods are called to implement the binary arithmetic\n operations ("+", "-", "*", "@", "/", "//", "%", "divmod()",\n "pow()", "**", "<<", ">>", "&", "^", "|") with reflected (swapped)\n operands. These functions are only called if the left operand does\n not support the corresponding operation and the operands are of\n different types. [2] For instance, to evaluate the expression "x -\n y", where *y* is an instance of a class that has an "__rsub__()"\n method, "y.__rsub__(x)" is called if "x.__sub__(y)" returns\n *NotImplemented*.\n\n Note that ternary "pow()" will not try calling "__rpow__()" (the\n coercion rules would become too complicated).\n\n Note: If the right operand\'s type is a subclass of the left\n operand\'s type and that subclass provides the reflected method\n for the operation, this method will be called before the left\n operand\'s non-reflected method. This behavior allows subclasses\n to override their ancestors\' operations.\n\nobject.__iadd__(self, other)\nobject.__isub__(self, other)\nobject.__imul__(self, other)\nobject.__imatmul__(self, other)\nobject.__itruediv__(self, other)\nobject.__ifloordiv__(self, other)\nobject.__imod__(self, other)\nobject.__ipow__(self, other[, modulo])\nobject.__ilshift__(self, other)\nobject.__irshift__(self, other)\nobject.__iand__(self, other)\nobject.__ixor__(self, other)\nobject.__ior__(self, other)\n\n These methods are called to implement the augmented arithmetic\n assignments ("+=", "-=", "*=", "@=", "/=", "//=", "%=", "**=",\n "<<=", ">>=", "&=", "^=", "|="). These methods should attempt to\n do the operation in-place (modifying *self*) and return the result\n (which could be, but does not have to be, *self*). If a specific\n method is not defined, the augmented assignment falls back to the\n normal methods. For instance, if *x* is an instance of a class\n with an "__iadd__()" method, "x += y" is equivalent to "x =\n x.__iadd__(y)" . Otherwise, "x.__add__(y)" and "y.__radd__(x)" are\n considered, as with the evaluation of "x + y". In certain\n situations, augmented assignment can result in unexpected errors\n (see *Why does a_tuple[i] += [\'item\'] raise an exception when the\n addition works?*), but this behavior is in fact part of the data\n model.\n\nobject.__neg__(self)\nobject.__pos__(self)\nobject.__abs__(self)\nobject.__invert__(self)\n\n Called to implement the unary arithmetic operations ("-", "+",\n "abs()" and "~").\n\nobject.__complex__(self)\nobject.__int__(self)\nobject.__float__(self)\nobject.__round__(self[, n])\n\n Called to implement the built-in functions "complex()", "int()",\n "float()" and "round()". Should return a value of the appropriate\n type.\n\nobject.__index__(self)\n\n Called to implement "operator.index()", and whenever Python needs\n to losslessly convert the numeric object to an integer object (such\n as in slicing, or in the built-in "bin()", "hex()" and "oct()"\n functions). Presence of this method indicates that the numeric\n object is an integer type. Must return an integer.\n\n Note: In order to have a coherent integer type class, when\n "__index__()" is defined "__int__()" should also be defined, and\n both should return the same value.\n', - 'objects': u'\nObjects, values and types\n*************************\n\n*Objects* are Python\'s abstraction for data. All data in a Python\nprogram is represented by objects or by relations between objects. (In\na sense, and in conformance to Von Neumann\'s model of a "stored\nprogram computer," code is also represented by objects.)\n\nEvery object has an identity, a type and a value. An object\'s\n*identity* never changes once it has been created; you may think of it\nas the object\'s address in memory. The \'"is"\' operator compares the\nidentity of two objects; the "id()" function returns an integer\nrepresenting its identity.\n\n**CPython implementation detail:** For CPython, "id(x)" is the memory\naddress where "x" is stored.\n\nAn object\'s type determines the operations that the object supports\n(e.g., "does it have a length?") and also defines the possible values\nfor objects of that type. The "type()" function returns an object\'s\ntype (which is an object itself). Like its identity, an object\'s\n*type* is also unchangeable. [1]\n\nThe *value* of some objects can change. Objects whose value can\nchange are said to be *mutable*; objects whose value is unchangeable\nonce they are created are called *immutable*. (The value of an\nimmutable container object that contains a reference to a mutable\nobject can change when the latter\'s value is changed; however the\ncontainer is still considered immutable, because the collection of\nobjects it contains cannot be changed. So, immutability is not\nstrictly the same as having an unchangeable value, it is more subtle.)\nAn object\'s mutability is determined by its type; for instance,\nnumbers, strings and tuples are immutable, while dictionaries and\nlists are mutable.\n\nObjects are never explicitly destroyed; however, when they become\nunreachable they may be garbage-collected. An implementation is\nallowed to postpone garbage collection or omit it altogether --- it is\na matter of implementation quality how garbage collection is\nimplemented, as long as no objects are collected that are still\nreachable.\n\n**CPython implementation detail:** CPython currently uses a reference-\ncounting scheme with (optional) delayed detection of cyclically linked\ngarbage, which collects most objects as soon as they become\nunreachable, but is not guaranteed to collect garbage containing\ncircular references. See the documentation of the "gc" module for\ninformation on controlling the collection of cyclic garbage. Other\nimplementations act differently and CPython may change. Do not depend\non immediate finalization of objects when they become unreachable (so\nyou should always close files explicitly).\n\nNote that the use of the implementation\'s tracing or debugging\nfacilities may keep objects alive that would normally be collectable.\nAlso note that catching an exception with a \'"try"..."except"\'\nstatement may keep objects alive.\n\nSome objects contain references to "external" resources such as open\nfiles or windows. It is understood that these resources are freed\nwhen the object is garbage-collected, but since garbage collection is\nnot guaranteed to happen, such objects also provide an explicit way to\nrelease the external resource, usually a "close()" method. Programs\nare strongly recommended to explicitly close such objects. The\n\'"try"..."finally"\' statement and the \'"with"\' statement provide\nconvenient ways to do this.\n\nSome objects contain references to other objects; these are called\n*containers*. Examples of containers are tuples, lists and\ndictionaries. The references are part of a container\'s value. In\nmost cases, when we talk about the value of a container, we imply the\nvalues, not the identities of the contained objects; however, when we\ntalk about the mutability of a container, only the identities of the\nimmediately contained objects are implied. So, if an immutable\ncontainer (like a tuple) contains a reference to a mutable object, its\nvalue changes if that mutable object is changed.\n\nTypes affect almost all aspects of object behavior. Even the\nimportance of object identity is affected in some sense: for immutable\ntypes, operations that compute new values may actually return a\nreference to any existing object with the same type and value, while\nfor mutable objects this is not allowed. E.g., after "a = 1; b = 1",\n"a" and "b" may or may not refer to the same object with the value\none, depending on the implementation, but after "c = []; d = []", "c"\nand "d" are guaranteed to refer to two different, unique, newly\ncreated empty lists. (Note that "c = d = []" assigns the same object\nto both "c" and "d".)\n', - 'operator-summary': u'\nOperator precedence\n*******************\n\nThe following table summarizes the operator precedence in Python, from\nlowest precedence (least binding) to highest precedence (most\nbinding). Operators in the same box have the same precedence. Unless\nthe syntax is explicitly given, operators are binary. Operators in\nthe same box group left to right (except for exponentiation, which\ngroups from right to left).\n\nNote that comparisons, membership tests, and identity tests, all have\nthe same precedence and have a left-to-right chaining feature as\ndescribed in the *Comparisons* section.\n\n+-------------------------------------------------+---------------------------------------+\n| Operator | Description |\n+=================================================+=======================================+\n| "lambda" | Lambda expression |\n+-------------------------------------------------+---------------------------------------+\n| "if" -- "else" | Conditional expression |\n+-------------------------------------------------+---------------------------------------+\n| "or" | Boolean OR |\n+-------------------------------------------------+---------------------------------------+\n| "and" | Boolean AND |\n+-------------------------------------------------+---------------------------------------+\n| "not" "x" | Boolean NOT |\n+-------------------------------------------------+---------------------------------------+\n| "in", "not in", "is", "is not", "<", "<=", ">", | Comparisons, including membership |\n| ">=", "!=", "==" | tests and identity tests |\n+-------------------------------------------------+---------------------------------------+\n| "|" | Bitwise OR |\n+-------------------------------------------------+---------------------------------------+\n| "^" | Bitwise XOR |\n+-------------------------------------------------+---------------------------------------+\n| "&" | Bitwise AND |\n+-------------------------------------------------+---------------------------------------+\n| "<<", ">>" | Shifts |\n+-------------------------------------------------+---------------------------------------+\n| "+", "-" | Addition and subtraction |\n+-------------------------------------------------+---------------------------------------+\n| "*", "@", "/", "//", "%" | Multiplication, matrix multiplication |\n| | division, remainder [5] |\n+-------------------------------------------------+---------------------------------------+\n| "+x", "-x", "~x" | Positive, negative, bitwise NOT |\n+-------------------------------------------------+---------------------------------------+\n| "**" | Exponentiation [6] |\n+-------------------------------------------------+---------------------------------------+\n| "await" "x" | Await expression |\n+-------------------------------------------------+---------------------------------------+\n| "x[index]", "x[index:index]", | Subscription, slicing, call, |\n| "x(arguments...)", "x.attribute" | attribute reference |\n+-------------------------------------------------+---------------------------------------+\n| "(expressions...)", "[expressions...]", "{key: | Binding or tuple display, list |\n| value...}", "{expressions...}" | display, dictionary display, set |\n| | display |\n+-------------------------------------------------+---------------------------------------+\n\n-[ Footnotes ]-\n\n[1] While "abs(x%y) < abs(y)" is true mathematically, for floats\n it may not be true numerically due to roundoff. For example, and\n assuming a platform on which a Python float is an IEEE 754 double-\n precision number, in order that "-1e-100 % 1e100" have the same\n sign as "1e100", the computed result is "-1e-100 + 1e100", which\n is numerically exactly equal to "1e100". The function\n "math.fmod()" returns a result whose sign matches the sign of the\n first argument instead, and so returns "-1e-100" in this case.\n Which approach is more appropriate depends on the application.\n\n[2] If x is very close to an exact integer multiple of y, it\'s\n possible for "x//y" to be one larger than "(x-x%y)//y" due to\n rounding. In such cases, Python returns the latter result, in\n order to preserve that "divmod(x,y)[0] * y + x % y" be very close\n to "x".\n\n[3] While comparisons between strings make sense at the byte\n level, they may be counter-intuitive to users. For example, the\n strings ""\\u00C7"" and ""\\u0043\\u0327"" compare differently, even\n though they both represent the same unicode character (LATIN\n CAPITAL LETTER C WITH CEDILLA). To compare strings in a human\n recognizable way, compare using "unicodedata.normalize()".\n\n[4] Due to automatic garbage-collection, free lists, and the\n dynamic nature of descriptors, you may notice seemingly unusual\n behaviour in certain uses of the "is" operator, like those\n involving comparisons between instance methods, or constants.\n Check their documentation for more info.\n\n[5] The "%" operator is also used for string formatting; the same\n precedence applies.\n\n[6] The power operator "**" binds less tightly than an arithmetic\n or bitwise unary operator on its right, that is, "2**-1" is "0.5".\n', - 'pass': u'\nThe "pass" statement\n********************\n\n pass_stmt ::= "pass"\n\n"pass" is a null operation --- when it is executed, nothing happens.\nIt is useful as a placeholder when a statement is required\nsyntactically, but no code needs to be executed, for example:\n\n def f(arg): pass # a function that does nothing (yet)\n\n class C: pass # a class with no methods (yet)\n', - 'power': u'\nThe power operator\n******************\n\nThe power operator binds more tightly than unary operators on its\nleft; it binds less tightly than unary operators on its right. The\nsyntax is:\n\n power ::= await ["**" u_expr]\n\nThus, in an unparenthesized sequence of power and unary operators, the\noperators are evaluated from right to left (this does not constrain\nthe evaluation order for the operands): "-1**2" results in "-1".\n\nThe power operator has the same semantics as the built-in "pow()"\nfunction, when called with two arguments: it yields its left argument\nraised to the power of its right argument. The numeric arguments are\nfirst converted to a common type, and the result is of that type.\n\nFor int operands, the result has the same type as the operands unless\nthe second argument is negative; in that case, all arguments are\nconverted to float and a float result is delivered. For example,\n"10**2" returns "100", but "10**-2" returns "0.01".\n\nRaising "0.0" to a negative power results in a "ZeroDivisionError".\nRaising a negative number to a fractional power results in a "complex"\nnumber. (In earlier versions it raised a "ValueError".)\n', - 'raise': u'\nThe "raise" statement\n*********************\n\n raise_stmt ::= "raise" [expression ["from" expression]]\n\nIf no expressions are present, "raise" re-raises the last exception\nthat was active in the current scope. If no exception is active in\nthe current scope, a "RuntimeError" exception is raised indicating\nthat this is an error.\n\nOtherwise, "raise" evaluates the first expression as the exception\nobject. It must be either a subclass or an instance of\n"BaseException". If it is a class, the exception instance will be\nobtained when needed by instantiating the class with no arguments.\n\nThe *type* of the exception is the exception instance\'s class, the\n*value* is the instance itself.\n\nA traceback object is normally created automatically when an exception\nis raised and attached to it as the "__traceback__" attribute, which\nis writable. You can create an exception and set your own traceback in\none step using the "with_traceback()" exception method (which returns\nthe same exception instance, with its traceback set to its argument),\nlike so:\n\n raise Exception("foo occurred").with_traceback(tracebackobj)\n\nThe "from" clause is used for exception chaining: if given, the second\n*expression* must be another exception class or instance, which will\nthen be attached to the raised exception as the "__cause__" attribute\n(which is writable). If the raised exception is not handled, both\nexceptions will be printed:\n\n >>> try:\n ... print(1 / 0)\n ... except Exception as exc:\n ... raise RuntimeError("Something bad happened") from exc\n ...\n Traceback (most recent call last):\n File "", line 2, in \n ZeroDivisionError: int division or modulo by zero\n\n The above exception was the direct cause of the following exception:\n\n Traceback (most recent call last):\n File "", line 4, in \n RuntimeError: Something bad happened\n\nA similar mechanism works implicitly if an exception is raised inside\nan exception handler or a "finally" clause: the previous exception is\nthen attached as the new exception\'s "__context__" attribute:\n\n >>> try:\n ... print(1 / 0)\n ... except:\n ... raise RuntimeError("Something bad happened")\n ...\n Traceback (most recent call last):\n File "", line 2, in \n ZeroDivisionError: int division or modulo by zero\n\n During handling of the above exception, another exception occurred:\n\n Traceback (most recent call last):\n File "", line 4, in \n RuntimeError: Something bad happened\n\nAdditional information on exceptions can be found in section\n*Exceptions*, and information about handling exceptions is in section\n*The try statement*.\n', - 'return': u'\nThe "return" statement\n**********************\n\n return_stmt ::= "return" [expression_list]\n\n"return" may only occur syntactically nested in a function definition,\nnot within a nested class definition.\n\nIf an expression list is present, it is evaluated, else "None" is\nsubstituted.\n\n"return" leaves the current function call with the expression list (or\n"None") as return value.\n\nWhen "return" passes control out of a "try" statement with a "finally"\nclause, that "finally" clause is executed before really leaving the\nfunction.\n\nIn a generator function, the "return" statement indicates that the\ngenerator is done and will cause "StopIteration" to be raised. The\nreturned value (if any) is used as an argument to construct\n"StopIteration" and becomes the "StopIteration.value" attribute.\n', - 'sequence-types': u'\nEmulating container types\n*************************\n\nThe following methods can be defined to implement container objects.\nContainers usually are sequences (such as lists or tuples) or mappings\n(like dictionaries), but can represent other containers as well. The\nfirst set of methods is used either to emulate a sequence or to\nemulate a mapping; the difference is that for a sequence, the\nallowable keys should be the integers *k* for which "0 <= k < N" where\n*N* is the length of the sequence, or slice objects, which define a\nrange of items. It is also recommended that mappings provide the\nmethods "keys()", "values()", "items()", "get()", "clear()",\n"setdefault()", "pop()", "popitem()", "copy()", and "update()"\nbehaving similar to those for Python\'s standard dictionary objects.\nThe "collections" module provides a "MutableMapping" abstract base\nclass to help create those methods from a base set of "__getitem__()",\n"__setitem__()", "__delitem__()", and "keys()". Mutable sequences\nshould provide methods "append()", "count()", "index()", "extend()",\n"insert()", "pop()", "remove()", "reverse()" and "sort()", like Python\nstandard list objects. Finally, sequence types should implement\naddition (meaning concatenation) and multiplication (meaning\nrepetition) by defining the methods "__add__()", "__radd__()",\n"__iadd__()", "__mul__()", "__rmul__()" and "__imul__()" described\nbelow; they should not define other numerical operators. It is\nrecommended that both mappings and sequences implement the\n"__contains__()" method to allow efficient use of the "in" operator;\nfor mappings, "in" should search the mapping\'s keys; for sequences, it\nshould search through the values. It is further recommended that both\nmappings and sequences implement the "__iter__()" method to allow\nefficient iteration through the container; for mappings, "__iter__()"\nshould be the same as "keys()"; for sequences, it should iterate\nthrough the values.\n\nobject.__len__(self)\n\n Called to implement the built-in function "len()". Should return\n the length of the object, an integer ">=" 0. Also, an object that\n doesn\'t define a "__bool__()" method and whose "__len__()" method\n returns zero is considered to be false in a Boolean context.\n\nobject.__length_hint__(self)\n\n Called to implement "operator.length_hint()". Should return an\n estimated length for the object (which may be greater or less than\n the actual length). The length must be an integer ">=" 0. This\n method is purely an optimization and is never required for\n correctness.\n\n New in version 3.4.\n\nNote: Slicing is done exclusively with the following three methods.\n A call like\n\n a[1:2] = b\n\n is translated to\n\n a[slice(1, 2, None)] = b\n\n and so forth. Missing slice items are always filled in with "None".\n\nobject.__getitem__(self, key)\n\n Called to implement evaluation of "self[key]". For sequence types,\n the accepted keys should be integers and slice objects. Note that\n the special interpretation of negative indexes (if the class wishes\n to emulate a sequence type) is up to the "__getitem__()" method. If\n *key* is of an inappropriate type, "TypeError" may be raised; if of\n a value outside the set of indexes for the sequence (after any\n special interpretation of negative values), "IndexError" should be\n raised. For mapping types, if *key* is missing (not in the\n container), "KeyError" should be raised.\n\n Note: "for" loops expect that an "IndexError" will be raised for\n illegal indexes to allow proper detection of the end of the\n sequence.\n\nobject.__missing__(self, key)\n\n Called by "dict"."__getitem__()" to implement "self[key]" for dict\n subclasses when key is not in the dictionary.\n\nobject.__setitem__(self, key, value)\n\n Called to implement assignment to "self[key]". Same note as for\n "__getitem__()". This should only be implemented for mappings if\n the objects support changes to the values for keys, or if new keys\n can be added, or for sequences if elements can be replaced. The\n same exceptions should be raised for improper *key* values as for\n the "__getitem__()" method.\n\nobject.__delitem__(self, key)\n\n Called to implement deletion of "self[key]". Same note as for\n "__getitem__()". This should only be implemented for mappings if\n the objects support removal of keys, or for sequences if elements\n can be removed from the sequence. The same exceptions should be\n raised for improper *key* values as for the "__getitem__()" method.\n\nobject.__iter__(self)\n\n This method is called when an iterator is required for a container.\n This method should return a new iterator object that can iterate\n over all the objects in the container. For mappings, it should\n iterate over the keys of the container.\n\n Iterator objects also need to implement this method; they are\n required to return themselves. For more information on iterator\n objects, see *Iterator Types*.\n\nobject.__reversed__(self)\n\n Called (if present) by the "reversed()" built-in to implement\n reverse iteration. It should return a new iterator object that\n iterates over all the objects in the container in reverse order.\n\n If the "__reversed__()" method is not provided, the "reversed()"\n built-in will fall back to using the sequence protocol ("__len__()"\n and "__getitem__()"). Objects that support the sequence protocol\n should only provide "__reversed__()" if they can provide an\n implementation that is more efficient than the one provided by\n "reversed()".\n\nThe membership test operators ("in" and "not in") are normally\nimplemented as an iteration through a sequence. However, container\nobjects can supply the following special method with a more efficient\nimplementation, which also does not require the object be a sequence.\n\nobject.__contains__(self, item)\n\n Called to implement membership test operators. Should return true\n if *item* is in *self*, false otherwise. For mapping objects, this\n should consider the keys of the mapping rather than the values or\n the key-item pairs.\n\n For objects that don\'t define "__contains__()", the membership test\n first tries iteration via "__iter__()", then the old sequence\n iteration protocol via "__getitem__()", see *this section in the\n language reference*.\n', - 'shifting': u'\nShifting operations\n*******************\n\nThe shifting operations have lower priority than the arithmetic\noperations:\n\n shift_expr ::= a_expr | shift_expr ( "<<" | ">>" ) a_expr\n\nThese operators accept integers as arguments. They shift the first\nargument to the left or right by the number of bits given by the\nsecond argument.\n\nA right shift by *n* bits is defined as floor division by "pow(2,n)".\nA left shift by *n* bits is defined as multiplication with "pow(2,n)".\n\nNote: In the current implementation, the right-hand operand is\n required to be at most "sys.maxsize". If the right-hand operand is\n larger than "sys.maxsize" an "OverflowError" exception is raised.\n', - 'slicings': u'\nSlicings\n********\n\nA slicing selects a range of items in a sequence object (e.g., a\nstring, tuple or list). Slicings may be used as expressions or as\ntargets in assignment or "del" statements. The syntax for a slicing:\n\n slicing ::= primary "[" slice_list "]"\n slice_list ::= slice_item ("," slice_item)* [","]\n slice_item ::= expression | proper_slice\n proper_slice ::= [lower_bound] ":" [upper_bound] [ ":" [stride] ]\n lower_bound ::= expression\n upper_bound ::= expression\n stride ::= expression\n\nThere is ambiguity in the formal syntax here: anything that looks like\nan expression list also looks like a slice list, so any subscription\ncan be interpreted as a slicing. Rather than further complicating the\nsyntax, this is disambiguated by defining that in this case the\ninterpretation as a subscription takes priority over the\ninterpretation as a slicing (this is the case if the slice list\ncontains no proper slice).\n\nThe semantics for a slicing are as follows. The primary is indexed\n(using the same "__getitem__()" method as normal subscription) with a\nkey that is constructed from the slice list, as follows. If the slice\nlist contains at least one comma, the key is a tuple containing the\nconversion of the slice items; otherwise, the conversion of the lone\nslice item is the key. The conversion of a slice item that is an\nexpression is that expression. The conversion of a proper slice is a\nslice object (see section *The standard type hierarchy*) whose\n"start", "stop" and "step" attributes are the values of the\nexpressions given as lower bound, upper bound and stride,\nrespectively, substituting "None" for missing expressions.\n', - 'specialattrs': u'\nSpecial Attributes\n******************\n\nThe implementation adds a few special read-only attributes to several\nobject types, where they are relevant. Some of these are not reported\nby the "dir()" built-in function.\n\nobject.__dict__\n\n A dictionary or other mapping object used to store an object\'s\n (writable) attributes.\n\ninstance.__class__\n\n The class to which a class instance belongs.\n\nclass.__bases__\n\n The tuple of base classes of a class object.\n\nclass.__name__\n\n The name of the class or type.\n\nclass.__qualname__\n\n The *qualified name* of the class or type.\n\n New in version 3.3.\n\nclass.__mro__\n\n This attribute is a tuple of classes that are considered when\n looking for base classes during method resolution.\n\nclass.mro()\n\n This method can be overridden by a metaclass to customize the\n method resolution order for its instances. It is called at class\n instantiation, and its result is stored in "__mro__".\n\nclass.__subclasses__()\n\n Each class keeps a list of weak references to its immediate\n subclasses. This method returns a list of all those references\n still alive. Example:\n\n >>> int.__subclasses__()\n []\n\n-[ Footnotes ]-\n\n[1] Additional information on these special methods may be found\n in the Python Reference Manual (*Basic customization*).\n\n[2] As a consequence, the list "[1, 2]" is considered equal to\n "[1.0, 2.0]", and similarly for tuples.\n\n[3] They must have since the parser can\'t tell the type of the\n operands.\n\n[4] Cased characters are those with general category property\n being one of "Lu" (Letter, uppercase), "Ll" (Letter, lowercase),\n or "Lt" (Letter, titlecase).\n\n[5] To format only a tuple you should therefore provide a\n singleton tuple whose only element is the tuple to be formatted.\n', - 'specialnames': u'\nSpecial method names\n********************\n\nA class can implement certain operations that are invoked by special\nsyntax (such as arithmetic operations or subscripting and slicing) by\ndefining methods with special names. This is Python\'s approach to\n*operator overloading*, allowing classes to define their own behavior\nwith respect to language operators. For instance, if a class defines\na method named "__getitem__()", and "x" is an instance of this class,\nthen "x[i]" is roughly equivalent to "type(x).__getitem__(x, i)".\nExcept where mentioned, attempts to execute an operation raise an\nexception when no appropriate method is defined (typically\n"AttributeError" or "TypeError").\n\nWhen implementing a class that emulates any built-in type, it is\nimportant that the emulation only be implemented to the degree that it\nmakes sense for the object being modelled. For example, some\nsequences may work well with retrieval of individual elements, but\nextracting a slice may not make sense. (One example of this is the\n"NodeList" interface in the W3C\'s Document Object Model.)\n\n\nBasic customization\n===================\n\nobject.__new__(cls[, ...])\n\n Called to create a new instance of class *cls*. "__new__()" is a\n static method (special-cased so you need not declare it as such)\n that takes the class of which an instance was requested as its\n first argument. The remaining arguments are those passed to the\n object constructor expression (the call to the class). The return\n value of "__new__()" should be the new object instance (usually an\n instance of *cls*).\n\n Typical implementations create a new instance of the class by\n invoking the superclass\'s "__new__()" method using\n "super(currentclass, cls).__new__(cls[, ...])" with appropriate\n arguments and then modifying the newly-created instance as\n necessary before returning it.\n\n If "__new__()" returns an instance of *cls*, then the new\n instance\'s "__init__()" method will be invoked like\n "__init__(self[, ...])", where *self* is the new instance and the\n remaining arguments are the same as were passed to "__new__()".\n\n If "__new__()" does not return an instance of *cls*, then the new\n instance\'s "__init__()" method will not be invoked.\n\n "__new__()" is intended mainly to allow subclasses of immutable\n types (like int, str, or tuple) to customize instance creation. It\n is also commonly overridden in custom metaclasses in order to\n customize class creation.\n\nobject.__init__(self[, ...])\n\n Called after the instance has been created (by "__new__()"), but\n before it is returned to the caller. The arguments are those\n passed to the class constructor expression. If a base class has an\n "__init__()" method, the derived class\'s "__init__()" method, if\n any, must explicitly call it to ensure proper initialization of the\n base class part of the instance; for example:\n "BaseClass.__init__(self, [args...])".\n\n Because "__new__()" and "__init__()" work together in constructing\n objects ("__new__()" to create it, and "__init__()" to customise\n it), no non-"None" value may be returned by "__init__()"; doing so\n will cause a "TypeError" to be raised at runtime.\n\nobject.__del__(self)\n\n Called when the instance is about to be destroyed. This is also\n called a destructor. If a base class has a "__del__()" method, the\n derived class\'s "__del__()" method, if any, must explicitly call it\n to ensure proper deletion of the base class part of the instance.\n Note that it is possible (though not recommended!) for the\n "__del__()" method to postpone destruction of the instance by\n creating a new reference to it. It may then be called at a later\n time when this new reference is deleted. It is not guaranteed that\n "__del__()" methods are called for objects that still exist when\n the interpreter exits.\n\n Note: "del x" doesn\'t directly call "x.__del__()" --- the former\n decrements the reference count for "x" by one, and the latter is\n only called when "x"\'s reference count reaches zero. Some common\n situations that may prevent the reference count of an object from\n going to zero include: circular references between objects (e.g.,\n a doubly-linked list or a tree data structure with parent and\n child pointers); a reference to the object on the stack frame of\n a function that caught an exception (the traceback stored in\n "sys.exc_info()[2]" keeps the stack frame alive); or a reference\n to the object on the stack frame that raised an unhandled\n exception in interactive mode (the traceback stored in\n "sys.last_traceback" keeps the stack frame alive). The first\n situation can only be remedied by explicitly breaking the cycles;\n the second can be resolved by freeing the reference to the\n traceback object when it is no longer useful, and the third can\n be resolved by storing "None" in "sys.last_traceback". Circular\n references which are garbage are detected and cleaned up when the\n cyclic garbage collector is enabled (it\'s on by default). Refer\n to the documentation for the "gc" module for more information\n about this topic.\n\n Warning: Due to the precarious circumstances under which\n "__del__()" methods are invoked, exceptions that occur during\n their execution are ignored, and a warning is printed to\n "sys.stderr" instead. Also, when "__del__()" is invoked in\n response to a module being deleted (e.g., when execution of the\n program is done), other globals referenced by the "__del__()"\n method may already have been deleted or in the process of being\n torn down (e.g. the import machinery shutting down). For this\n reason, "__del__()" methods should do the absolute minimum needed\n to maintain external invariants. Starting with version 1.5,\n Python guarantees that globals whose name begins with a single\n underscore are deleted from their module before other globals are\n deleted; if no other references to such globals exist, this may\n help in assuring that imported modules are still available at the\n time when the "__del__()" method is called.\n\nobject.__repr__(self)\n\n Called by the "repr()" built-in function to compute the "official"\n string representation of an object. If at all possible, this\n should look like a valid Python expression that could be used to\n recreate an object with the same value (given an appropriate\n environment). If this is not possible, a string of the form\n "<...some useful description...>" should be returned. The return\n value must be a string object. If a class defines "__repr__()" but\n not "__str__()", then "__repr__()" is also used when an "informal"\n string representation of instances of that class is required.\n\n This is typically used for debugging, so it is important that the\n representation is information-rich and unambiguous.\n\nobject.__str__(self)\n\n Called by "str(object)" and the built-in functions "format()" and\n "print()" to compute the "informal" or nicely printable string\n representation of an object. The return value must be a *string*\n object.\n\n This method differs from "object.__repr__()" in that there is no\n expectation that "__str__()" return a valid Python expression: a\n more convenient or concise representation can be used.\n\n The default implementation defined by the built-in type "object"\n calls "object.__repr__()".\n\nobject.__bytes__(self)\n\n Called by "bytes()" to compute a byte-string representation of an\n object. This should return a "bytes" object.\n\nobject.__format__(self, format_spec)\n\n Called by the "format()" built-in function (and by extension, the\n "str.format()" method of class "str") to produce a "formatted"\n string representation of an object. The "format_spec" argument is a\n string that contains a description of the formatting options\n desired. The interpretation of the "format_spec" argument is up to\n the type implementing "__format__()", however most classes will\n either delegate formatting to one of the built-in types, or use a\n similar formatting option syntax.\n\n See *Format Specification Mini-Language* for a description of the\n standard formatting syntax.\n\n The return value must be a string object.\n\n Changed in version 3.4: The __format__ method of "object" itself\n raises a "TypeError" if passed any non-empty string.\n\nobject.__lt__(self, other)\nobject.__le__(self, other)\nobject.__eq__(self, other)\nobject.__ne__(self, other)\nobject.__gt__(self, other)\nobject.__ge__(self, other)\n\n These are the so-called "rich comparison" methods. The\n correspondence between operator symbols and method names is as\n follows: "xy" calls\n "x.__gt__(y)", and "x>=y" calls "x.__ge__(y)".\n\n A rich comparison method may return the singleton "NotImplemented"\n if it does not implement the operation for a given pair of\n arguments. By convention, "False" and "True" are returned for a\n successful comparison. However, these methods can return any value,\n so if the comparison operator is used in a Boolean context (e.g.,\n in the condition of an "if" statement), Python will call "bool()"\n on the value to determine if the result is true or false.\n\n By default, "__ne__()" delegates to "__eq__()" and inverts the\n result unless it is "NotImplemented". There are no other implied\n relationships among the comparison operators, for example, the\n truth of "(x.__hash__".\n\n If a class that does not override "__eq__()" wishes to suppress\n hash support, it should include "__hash__ = None" in the class\n definition. A class which defines its own "__hash__()" that\n explicitly raises a "TypeError" would be incorrectly identified as\n hashable by an "isinstance(obj, collections.Hashable)" call.\n\n Note: By default, the "__hash__()" values of str, bytes and\n datetime objects are "salted" with an unpredictable random value.\n Although they remain constant within an individual Python\n process, they are not predictable between repeated invocations of\n Python.This is intended to provide protection against a denial-\n of-service caused by carefully-chosen inputs that exploit the\n worst case performance of a dict insertion, O(n^2) complexity.\n See http://www.ocert.org/advisories/ocert-2011-003.html for\n details.Changing hash values affects the iteration order of\n dicts, sets and other mappings. Python has never made guarantees\n about this ordering (and it typically varies between 32-bit and\n 64-bit builds).See also "PYTHONHASHSEED".\n\n Changed in version 3.3: Hash randomization is enabled by default.\n\nobject.__bool__(self)\n\n Called to implement truth value testing and the built-in operation\n "bool()"; should return "False" or "True". When this method is not\n defined, "__len__()" is called, if it is defined, and the object is\n considered true if its result is nonzero. If a class defines\n neither "__len__()" nor "__bool__()", all its instances are\n considered true.\n\n\nCustomizing attribute access\n============================\n\nThe following methods can be defined to customize the meaning of\nattribute access (use of, assignment to, or deletion of "x.name") for\nclass instances.\n\nobject.__getattr__(self, name)\n\n Called when an attribute lookup has not found the attribute in the\n usual places (i.e. it is not an instance attribute nor is it found\n in the class tree for "self"). "name" is the attribute name. This\n method should return the (computed) attribute value or raise an\n "AttributeError" exception.\n\n Note that if the attribute is found through the normal mechanism,\n "__getattr__()" is not called. (This is an intentional asymmetry\n between "__getattr__()" and "__setattr__()".) This is done both for\n efficiency reasons and because otherwise "__getattr__()" would have\n no way to access other attributes of the instance. Note that at\n least for instance variables, you can fake total control by not\n inserting any values in the instance attribute dictionary (but\n instead inserting them in another object). See the\n "__getattribute__()" method below for a way to actually get total\n control over attribute access.\n\nobject.__getattribute__(self, name)\n\n Called unconditionally to implement attribute accesses for\n instances of the class. If the class also defines "__getattr__()",\n the latter will not be called unless "__getattribute__()" either\n calls it explicitly or raises an "AttributeError". This method\n should return the (computed) attribute value or raise an\n "AttributeError" exception. In order to avoid infinite recursion in\n this method, its implementation should always call the base class\n method with the same name to access any attributes it needs, for\n example, "object.__getattribute__(self, name)".\n\n Note: This method may still be bypassed when looking up special\n methods as the result of implicit invocation via language syntax\n or built-in functions. See *Special method lookup*.\n\nobject.__setattr__(self, name, value)\n\n Called when an attribute assignment is attempted. This is called\n instead of the normal mechanism (i.e. store the value in the\n instance dictionary). *name* is the attribute name, *value* is the\n value to be assigned to it.\n\n If "__setattr__()" wants to assign to an instance attribute, it\n should call the base class method with the same name, for example,\n "object.__setattr__(self, name, value)".\n\nobject.__delattr__(self, name)\n\n Like "__setattr__()" but for attribute deletion instead of\n assignment. This should only be implemented if "del obj.name" is\n meaningful for the object.\n\nobject.__dir__(self)\n\n Called when "dir()" is called on the object. A sequence must be\n returned. "dir()" converts the returned sequence to a list and\n sorts it.\n\n\nImplementing Descriptors\n------------------------\n\nThe following methods only apply when an instance of the class\ncontaining the method (a so-called *descriptor* class) appears in an\n*owner* class (the descriptor must be in either the owner\'s class\ndictionary or in the class dictionary for one of its parents). In the\nexamples below, "the attribute" refers to the attribute whose name is\nthe key of the property in the owner class\' "__dict__".\n\nobject.__get__(self, instance, owner)\n\n Called to get the attribute of the owner class (class attribute\n access) or of an instance of that class (instance attribute\n access). *owner* is always the owner class, while *instance* is the\n instance that the attribute was accessed through, or "None" when\n the attribute is accessed through the *owner*. This method should\n return the (computed) attribute value or raise an "AttributeError"\n exception.\n\nobject.__set__(self, instance, value)\n\n Called to set the attribute on an instance *instance* of the owner\n class to a new value, *value*.\n\nobject.__delete__(self, instance)\n\n Called to delete the attribute on an instance *instance* of the\n owner class.\n\nThe attribute "__objclass__" is interpreted by the "inspect" module as\nspecifying the class where this object was defined (setting this\nappropriately can assist in runtime introspection of dynamic class\nattributes). For callables, it may indicate that an instance of the\ngiven type (or a subclass) is expected or required as the first\npositional argument (for example, CPython sets this attribute for\nunbound methods that are implemented in C).\n\n\nInvoking Descriptors\n--------------------\n\nIn general, a descriptor is an object attribute with "binding\nbehavior", one whose attribute access has been overridden by methods\nin the descriptor protocol: "__get__()", "__set__()", and\n"__delete__()". If any of those methods are defined for an object, it\nis said to be a descriptor.\n\nThe default behavior for attribute access is to get, set, or delete\nthe attribute from an object\'s dictionary. For instance, "a.x" has a\nlookup chain starting with "a.__dict__[\'x\']", then\n"type(a).__dict__[\'x\']", and continuing through the base classes of\n"type(a)" excluding metaclasses.\n\nHowever, if the looked-up value is an object defining one of the\ndescriptor methods, then Python may override the default behavior and\ninvoke the descriptor method instead. Where this occurs in the\nprecedence chain depends on which descriptor methods were defined and\nhow they were called.\n\nThe starting point for descriptor invocation is a binding, "a.x". How\nthe arguments are assembled depends on "a":\n\nDirect Call\n The simplest and least common call is when user code directly\n invokes a descriptor method: "x.__get__(a)".\n\nInstance Binding\n If binding to an object instance, "a.x" is transformed into the\n call: "type(a).__dict__[\'x\'].__get__(a, type(a))".\n\nClass Binding\n If binding to a class, "A.x" is transformed into the call:\n "A.__dict__[\'x\'].__get__(None, A)".\n\nSuper Binding\n If "a" is an instance of "super", then the binding "super(B,\n obj).m()" searches "obj.__class__.__mro__" for the base class "A"\n immediately preceding "B" and then invokes the descriptor with the\n call: "A.__dict__[\'m\'].__get__(obj, obj.__class__)".\n\nFor instance bindings, the precedence of descriptor invocation depends\non the which descriptor methods are defined. A descriptor can define\nany combination of "__get__()", "__set__()" and "__delete__()". If it\ndoes not define "__get__()", then accessing the attribute will return\nthe descriptor object itself unless there is a value in the object\'s\ninstance dictionary. If the descriptor defines "__set__()" and/or\n"__delete__()", it is a data descriptor; if it defines neither, it is\na non-data descriptor. Normally, data descriptors define both\n"__get__()" and "__set__()", while non-data descriptors have just the\n"__get__()" method. Data descriptors with "__set__()" and "__get__()"\ndefined always override a redefinition in an instance dictionary. In\ncontrast, non-data descriptors can be overridden by instances.\n\nPython methods (including "staticmethod()" and "classmethod()") are\nimplemented as non-data descriptors. Accordingly, instances can\nredefine and override methods. This allows individual instances to\nacquire behaviors that differ from other instances of the same class.\n\nThe "property()" function is implemented as a data descriptor.\nAccordingly, instances cannot override the behavior of a property.\n\n\n__slots__\n---------\n\nBy default, instances of classes have a dictionary for attribute\nstorage. This wastes space for objects having very few instance\nvariables. The space consumption can become acute when creating large\nnumbers of instances.\n\nThe default can be overridden by defining *__slots__* in a class\ndefinition. The *__slots__* declaration takes a sequence of instance\nvariables and reserves just enough space in each instance to hold a\nvalue for each variable. Space is saved because *__dict__* is not\ncreated for each instance.\n\nobject.__slots__\n\n This class variable can be assigned a string, iterable, or sequence\n of strings with variable names used by instances. *__slots__*\n reserves space for the declared variables and prevents the\n automatic creation of *__dict__* and *__weakref__* for each\n instance.\n\n\nNotes on using *__slots__*\n~~~~~~~~~~~~~~~~~~~~~~~~~~\n\n* When inheriting from a class without *__slots__*, the *__dict__*\n attribute of that class will always be accessible, so a *__slots__*\n definition in the subclass is meaningless.\n\n* Without a *__dict__* variable, instances cannot be assigned new\n variables not listed in the *__slots__* definition. Attempts to\n assign to an unlisted variable name raises "AttributeError". If\n dynamic assignment of new variables is desired, then add\n "\'__dict__\'" to the sequence of strings in the *__slots__*\n declaration.\n\n* Without a *__weakref__* variable for each instance, classes\n defining *__slots__* do not support weak references to its\n instances. If weak reference support is needed, then add\n "\'__weakref__\'" to the sequence of strings in the *__slots__*\n declaration.\n\n* *__slots__* are implemented at the class level by creating\n descriptors (*Implementing Descriptors*) for each variable name. As\n a result, class attributes cannot be used to set default values for\n instance variables defined by *__slots__*; otherwise, the class\n attribute would overwrite the descriptor assignment.\n\n* The action of a *__slots__* declaration is limited to the class\n where it is defined. As a result, subclasses will have a *__dict__*\n unless they also define *__slots__* (which must only contain names\n of any *additional* slots).\n\n* If a class defines a slot also defined in a base class, the\n instance variable defined by the base class slot is inaccessible\n (except by retrieving its descriptor directly from the base class).\n This renders the meaning of the program undefined. In the future, a\n check may be added to prevent this.\n\n* Nonempty *__slots__* does not work for classes derived from\n "variable-length" built-in types such as "int", "bytes" and "tuple".\n\n* Any non-string iterable may be assigned to *__slots__*. Mappings\n may also be used; however, in the future, special meaning may be\n assigned to the values corresponding to each key.\n\n* *__class__* assignment works only if both classes have the same\n *__slots__*.\n\n\nCustomizing class creation\n==========================\n\nBy default, classes are constructed using "type()". The class body is\nexecuted in a new namespace and the class name is bound locally to the\nresult of "type(name, bases, namespace)".\n\nThe class creation process can be customised by passing the\n"metaclass" keyword argument in the class definition line, or by\ninheriting from an existing class that included such an argument. In\nthe following example, both "MyClass" and "MySubclass" are instances\nof "Meta":\n\n class Meta(type):\n pass\n\n class MyClass(metaclass=Meta):\n pass\n\n class MySubclass(MyClass):\n pass\n\nAny other keyword arguments that are specified in the class definition\nare passed through to all metaclass operations described below.\n\nWhen a class definition is executed, the following steps occur:\n\n* the appropriate metaclass is determined\n\n* the class namespace is prepared\n\n* the class body is executed\n\n* the class object is created\n\n\nDetermining the appropriate metaclass\n-------------------------------------\n\nThe appropriate metaclass for a class definition is determined as\nfollows:\n\n* if no bases and no explicit metaclass are given, then "type()" is\n used\n\n* if an explicit metaclass is given and it is *not* an instance of\n "type()", then it is used directly as the metaclass\n\n* if an instance of "type()" is given as the explicit metaclass, or\n bases are defined, then the most derived metaclass is used\n\nThe most derived metaclass is selected from the explicitly specified\nmetaclass (if any) and the metaclasses (i.e. "type(cls)") of all\nspecified base classes. The most derived metaclass is one which is a\nsubtype of *all* of these candidate metaclasses. If none of the\ncandidate metaclasses meets that criterion, then the class definition\nwill fail with "TypeError".\n\n\nPreparing the class namespace\n-----------------------------\n\nOnce the appropriate metaclass has been identified, then the class\nnamespace is prepared. If the metaclass has a "__prepare__" attribute,\nit is called as "namespace = metaclass.__prepare__(name, bases,\n**kwds)" (where the additional keyword arguments, if any, come from\nthe class definition).\n\nIf the metaclass has no "__prepare__" attribute, then the class\nnamespace is initialised as an empty "dict()" instance.\n\nSee also: **PEP 3115** - Metaclasses in Python 3000\n\n Introduced the "__prepare__" namespace hook\n\n\nExecuting the class body\n------------------------\n\nThe class body is executed (approximately) as "exec(body, globals(),\nnamespace)". The key difference from a normal call to "exec()" is that\nlexical scoping allows the class body (including any methods) to\nreference names from the current and outer scopes when the class\ndefinition occurs inside a function.\n\nHowever, even when the class definition occurs inside the function,\nmethods defined inside the class still cannot see names defined at the\nclass scope. Class variables must be accessed through the first\nparameter of instance or class methods, and cannot be accessed at all\nfrom static methods.\n\n\nCreating the class object\n-------------------------\n\nOnce the class namespace has been populated by executing the class\nbody, the class object is created by calling "metaclass(name, bases,\nnamespace, **kwds)" (the additional keywords passed here are the same\nas those passed to "__prepare__").\n\nThis class object is the one that will be referenced by the zero-\nargument form of "super()". "__class__" is an implicit closure\nreference created by the compiler if any methods in a class body refer\nto either "__class__" or "super". This allows the zero argument form\nof "super()" to correctly identify the class being defined based on\nlexical scoping, while the class or instance that was used to make the\ncurrent call is identified based on the first argument passed to the\nmethod.\n\nAfter the class object is created, it is passed to the class\ndecorators included in the class definition (if any) and the resulting\nobject is bound in the local namespace as the defined class.\n\nSee also: **PEP 3135** - New super\n\n Describes the implicit "__class__" closure reference\n\n\nMetaclass example\n-----------------\n\nThe potential uses for metaclasses are boundless. Some ideas that have\nbeen explored include logging, interface checking, automatic\ndelegation, automatic property creation, proxies, frameworks, and\nautomatic resource locking/synchronization.\n\nHere is an example of a metaclass that uses an\n"collections.OrderedDict" to remember the order that class variables\nare defined:\n\n class OrderedClass(type):\n\n @classmethod\n def __prepare__(metacls, name, bases, **kwds):\n return collections.OrderedDict()\n\n def __new__(cls, name, bases, namespace, **kwds):\n result = type.__new__(cls, name, bases, dict(namespace))\n result.members = tuple(namespace)\n return result\n\n class A(metaclass=OrderedClass):\n def one(self): pass\n def two(self): pass\n def three(self): pass\n def four(self): pass\n\n >>> A.members\n (\'__module__\', \'one\', \'two\', \'three\', \'four\')\n\nWhen the class definition for *A* gets executed, the process begins\nwith calling the metaclass\'s "__prepare__()" method which returns an\nempty "collections.OrderedDict". That mapping records the methods and\nattributes of *A* as they are defined within the body of the class\nstatement. Once those definitions are executed, the ordered dictionary\nis fully populated and the metaclass\'s "__new__()" method gets\ninvoked. That method builds the new type and it saves the ordered\ndictionary keys in an attribute called "members".\n\n\nCustomizing instance and subclass checks\n========================================\n\nThe following methods are used to override the default behavior of the\n"isinstance()" and "issubclass()" built-in functions.\n\nIn particular, the metaclass "abc.ABCMeta" implements these methods in\norder to allow the addition of Abstract Base Classes (ABCs) as\n"virtual base classes" to any class or type (including built-in\ntypes), including other ABCs.\n\nclass.__instancecheck__(self, instance)\n\n Return true if *instance* should be considered a (direct or\n indirect) instance of *class*. If defined, called to implement\n "isinstance(instance, class)".\n\nclass.__subclasscheck__(self, subclass)\n\n Return true if *subclass* should be considered a (direct or\n indirect) subclass of *class*. If defined, called to implement\n "issubclass(subclass, class)".\n\nNote that these methods are looked up on the type (metaclass) of a\nclass. They cannot be defined as class methods in the actual class.\nThis is consistent with the lookup of special methods that are called\non instances, only in this case the instance is itself a class.\n\nSee also: **PEP 3119** - Introducing Abstract Base Classes\n\n Includes the specification for customizing "isinstance()" and\n "issubclass()" behavior through "__instancecheck__()" and\n "__subclasscheck__()", with motivation for this functionality in\n the context of adding Abstract Base Classes (see the "abc"\n module) to the language.\n\n\nEmulating callable objects\n==========================\n\nobject.__call__(self[, args...])\n\n Called when the instance is "called" as a function; if this method\n is defined, "x(arg1, arg2, ...)" is a shorthand for\n "x.__call__(arg1, arg2, ...)".\n\n\nEmulating container types\n=========================\n\nThe following methods can be defined to implement container objects.\nContainers usually are sequences (such as lists or tuples) or mappings\n(like dictionaries), but can represent other containers as well. The\nfirst set of methods is used either to emulate a sequence or to\nemulate a mapping; the difference is that for a sequence, the\nallowable keys should be the integers *k* for which "0 <= k < N" where\n*N* is the length of the sequence, or slice objects, which define a\nrange of items. It is also recommended that mappings provide the\nmethods "keys()", "values()", "items()", "get()", "clear()",\n"setdefault()", "pop()", "popitem()", "copy()", and "update()"\nbehaving similar to those for Python\'s standard dictionary objects.\nThe "collections" module provides a "MutableMapping" abstract base\nclass to help create those methods from a base set of "__getitem__()",\n"__setitem__()", "__delitem__()", and "keys()". Mutable sequences\nshould provide methods "append()", "count()", "index()", "extend()",\n"insert()", "pop()", "remove()", "reverse()" and "sort()", like Python\nstandard list objects. Finally, sequence types should implement\naddition (meaning concatenation) and multiplication (meaning\nrepetition) by defining the methods "__add__()", "__radd__()",\n"__iadd__()", "__mul__()", "__rmul__()" and "__imul__()" described\nbelow; they should not define other numerical operators. It is\nrecommended that both mappings and sequences implement the\n"__contains__()" method to allow efficient use of the "in" operator;\nfor mappings, "in" should search the mapping\'s keys; for sequences, it\nshould search through the values. It is further recommended that both\nmappings and sequences implement the "__iter__()" method to allow\nefficient iteration through the container; for mappings, "__iter__()"\nshould be the same as "keys()"; for sequences, it should iterate\nthrough the values.\n\nobject.__len__(self)\n\n Called to implement the built-in function "len()". Should return\n the length of the object, an integer ">=" 0. Also, an object that\n doesn\'t define a "__bool__()" method and whose "__len__()" method\n returns zero is considered to be false in a Boolean context.\n\nobject.__length_hint__(self)\n\n Called to implement "operator.length_hint()". Should return an\n estimated length for the object (which may be greater or less than\n the actual length). The length must be an integer ">=" 0. This\n method is purely an optimization and is never required for\n correctness.\n\n New in version 3.4.\n\nNote: Slicing is done exclusively with the following three methods.\n A call like\n\n a[1:2] = b\n\n is translated to\n\n a[slice(1, 2, None)] = b\n\n and so forth. Missing slice items are always filled in with "None".\n\nobject.__getitem__(self, key)\n\n Called to implement evaluation of "self[key]". For sequence types,\n the accepted keys should be integers and slice objects. Note that\n the special interpretation of negative indexes (if the class wishes\n to emulate a sequence type) is up to the "__getitem__()" method. If\n *key* is of an inappropriate type, "TypeError" may be raised; if of\n a value outside the set of indexes for the sequence (after any\n special interpretation of negative values), "IndexError" should be\n raised. For mapping types, if *key* is missing (not in the\n container), "KeyError" should be raised.\n\n Note: "for" loops expect that an "IndexError" will be raised for\n illegal indexes to allow proper detection of the end of the\n sequence.\n\nobject.__missing__(self, key)\n\n Called by "dict"."__getitem__()" to implement "self[key]" for dict\n subclasses when key is not in the dictionary.\n\nobject.__setitem__(self, key, value)\n\n Called to implement assignment to "self[key]". Same note as for\n "__getitem__()". This should only be implemented for mappings if\n the objects support changes to the values for keys, or if new keys\n can be added, or for sequences if elements can be replaced. The\n same exceptions should be raised for improper *key* values as for\n the "__getitem__()" method.\n\nobject.__delitem__(self, key)\n\n Called to implement deletion of "self[key]". Same note as for\n "__getitem__()". This should only be implemented for mappings if\n the objects support removal of keys, or for sequences if elements\n can be removed from the sequence. The same exceptions should be\n raised for improper *key* values as for the "__getitem__()" method.\n\nobject.__iter__(self)\n\n This method is called when an iterator is required for a container.\n This method should return a new iterator object that can iterate\n over all the objects in the container. For mappings, it should\n iterate over the keys of the container.\n\n Iterator objects also need to implement this method; they are\n required to return themselves. For more information on iterator\n objects, see *Iterator Types*.\n\nobject.__reversed__(self)\n\n Called (if present) by the "reversed()" built-in to implement\n reverse iteration. It should return a new iterator object that\n iterates over all the objects in the container in reverse order.\n\n If the "__reversed__()" method is not provided, the "reversed()"\n built-in will fall back to using the sequence protocol ("__len__()"\n and "__getitem__()"). Objects that support the sequence protocol\n should only provide "__reversed__()" if they can provide an\n implementation that is more efficient than the one provided by\n "reversed()".\n\nThe membership test operators ("in" and "not in") are normally\nimplemented as an iteration through a sequence. However, container\nobjects can supply the following special method with a more efficient\nimplementation, which also does not require the object be a sequence.\n\nobject.__contains__(self, item)\n\n Called to implement membership test operators. Should return true\n if *item* is in *self*, false otherwise. For mapping objects, this\n should consider the keys of the mapping rather than the values or\n the key-item pairs.\n\n For objects that don\'t define "__contains__()", the membership test\n first tries iteration via "__iter__()", then the old sequence\n iteration protocol via "__getitem__()", see *this section in the\n language reference*.\n\n\nEmulating numeric types\n=======================\n\nThe following methods can be defined to emulate numeric objects.\nMethods corresponding to operations that are not supported by the\nparticular kind of number implemented (e.g., bitwise operations for\nnon-integral numbers) should be left undefined.\n\nobject.__add__(self, other)\nobject.__sub__(self, other)\nobject.__mul__(self, other)\nobject.__matmul__(self, other)\nobject.__truediv__(self, other)\nobject.__floordiv__(self, other)\nobject.__mod__(self, other)\nobject.__divmod__(self, other)\nobject.__pow__(self, other[, modulo])\nobject.__lshift__(self, other)\nobject.__rshift__(self, other)\nobject.__and__(self, other)\nobject.__xor__(self, other)\nobject.__or__(self, other)\n\n These methods are called to implement the binary arithmetic\n operations ("+", "-", "*", "@", "/", "//", "%", "divmod()",\n "pow()", "**", "<<", ">>", "&", "^", "|"). For instance, to\n evaluate the expression "x + y", where *x* is an instance of a\n class that has an "__add__()" method, "x.__add__(y)" is called.\n The "__divmod__()" method should be the equivalent to using\n "__floordiv__()" and "__mod__()"; it should not be related to\n "__truediv__()". Note that "__pow__()" should be defined to accept\n an optional third argument if the ternary version of the built-in\n "pow()" function is to be supported.\n\n If one of those methods does not support the operation with the\n supplied arguments, it should return "NotImplemented".\n\nobject.__radd__(self, other)\nobject.__rsub__(self, other)\nobject.__rmul__(self, other)\nobject.__rmatmul__(self, other)\nobject.__rtruediv__(self, other)\nobject.__rfloordiv__(self, other)\nobject.__rmod__(self, other)\nobject.__rdivmod__(self, other)\nobject.__rpow__(self, other)\nobject.__rlshift__(self, other)\nobject.__rrshift__(self, other)\nobject.__rand__(self, other)\nobject.__rxor__(self, other)\nobject.__ror__(self, other)\n\n These methods are called to implement the binary arithmetic\n operations ("+", "-", "*", "@", "/", "//", "%", "divmod()",\n "pow()", "**", "<<", ">>", "&", "^", "|") with reflected (swapped)\n operands. These functions are only called if the left operand does\n not support the corresponding operation and the operands are of\n different types. [2] For instance, to evaluate the expression "x -\n y", where *y* is an instance of a class that has an "__rsub__()"\n method, "y.__rsub__(x)" is called if "x.__sub__(y)" returns\n *NotImplemented*.\n\n Note that ternary "pow()" will not try calling "__rpow__()" (the\n coercion rules would become too complicated).\n\n Note: If the right operand\'s type is a subclass of the left\n operand\'s type and that subclass provides the reflected method\n for the operation, this method will be called before the left\n operand\'s non-reflected method. This behavior allows subclasses\n to override their ancestors\' operations.\n\nobject.__iadd__(self, other)\nobject.__isub__(self, other)\nobject.__imul__(self, other)\nobject.__imatmul__(self, other)\nobject.__itruediv__(self, other)\nobject.__ifloordiv__(self, other)\nobject.__imod__(self, other)\nobject.__ipow__(self, other[, modulo])\nobject.__ilshift__(self, other)\nobject.__irshift__(self, other)\nobject.__iand__(self, other)\nobject.__ixor__(self, other)\nobject.__ior__(self, other)\n\n These methods are called to implement the augmented arithmetic\n assignments ("+=", "-=", "*=", "@=", "/=", "//=", "%=", "**=",\n "<<=", ">>=", "&=", "^=", "|="). These methods should attempt to\n do the operation in-place (modifying *self*) and return the result\n (which could be, but does not have to be, *self*). If a specific\n method is not defined, the augmented assignment falls back to the\n normal methods. For instance, if *x* is an instance of a class\n with an "__iadd__()" method, "x += y" is equivalent to "x =\n x.__iadd__(y)" . Otherwise, "x.__add__(y)" and "y.__radd__(x)" are\n considered, as with the evaluation of "x + y". In certain\n situations, augmented assignment can result in unexpected errors\n (see *Why does a_tuple[i] += [\'item\'] raise an exception when the\n addition works?*), but this behavior is in fact part of the data\n model.\n\nobject.__neg__(self)\nobject.__pos__(self)\nobject.__abs__(self)\nobject.__invert__(self)\n\n Called to implement the unary arithmetic operations ("-", "+",\n "abs()" and "~").\n\nobject.__complex__(self)\nobject.__int__(self)\nobject.__float__(self)\nobject.__round__(self[, n])\n\n Called to implement the built-in functions "complex()", "int()",\n "float()" and "round()". Should return a value of the appropriate\n type.\n\nobject.__index__(self)\n\n Called to implement "operator.index()", and whenever Python needs\n to losslessly convert the numeric object to an integer object (such\n as in slicing, or in the built-in "bin()", "hex()" and "oct()"\n functions). Presence of this method indicates that the numeric\n object is an integer type. Must return an integer.\n\n Note: In order to have a coherent integer type class, when\n "__index__()" is defined "__int__()" should also be defined, and\n both should return the same value.\n\n\nWith Statement Context Managers\n===============================\n\nA *context manager* is an object that defines the runtime context to\nbe established when executing a "with" statement. The context manager\nhandles the entry into, and the exit from, the desired runtime context\nfor the execution of the block of code. Context managers are normally\ninvoked using the "with" statement (described in section *The with\nstatement*), but can also be used by directly invoking their methods.\n\nTypical uses of context managers include saving and restoring various\nkinds of global state, locking and unlocking resources, closing opened\nfiles, etc.\n\nFor more information on context managers, see *Context Manager Types*.\n\nobject.__enter__(self)\n\n Enter the runtime context related to this object. The "with"\n statement will bind this method\'s return value to the target(s)\n specified in the "as" clause of the statement, if any.\n\nobject.__exit__(self, exc_type, exc_value, traceback)\n\n Exit the runtime context related to this object. The parameters\n describe the exception that caused the context to be exited. If the\n context was exited without an exception, all three arguments will\n be "None".\n\n If an exception is supplied, and the method wishes to suppress the\n exception (i.e., prevent it from being propagated), it should\n return a true value. Otherwise, the exception will be processed\n normally upon exit from this method.\n\n Note that "__exit__()" methods should not reraise the passed-in\n exception; this is the caller\'s responsibility.\n\nSee also: **PEP 0343** - The "with" statement\n\n The specification, background, and examples for the Python "with"\n statement.\n\n\nSpecial method lookup\n=====================\n\nFor custom classes, implicit invocations of special methods are only\nguaranteed to work correctly if defined on an object\'s type, not in\nthe object\'s instance dictionary. That behaviour is the reason why\nthe following code raises an exception:\n\n >>> class C:\n ... pass\n ...\n >>> c = C()\n >>> c.__len__ = lambda: 5\n >>> len(c)\n Traceback (most recent call last):\n File "", line 1, in \n TypeError: object of type \'C\' has no len()\n\nThe rationale behind this behaviour lies with a number of special\nmethods such as "__hash__()" and "__repr__()" that are implemented by\nall objects, including type objects. If the implicit lookup of these\nmethods used the conventional lookup process, they would fail when\ninvoked on the type object itself:\n\n >>> 1 .__hash__() == hash(1)\n True\n >>> int.__hash__() == hash(int)\n Traceback (most recent call last):\n File "", line 1, in \n TypeError: descriptor \'__hash__\' of \'int\' object needs an argument\n\nIncorrectly attempting to invoke an unbound method of a class in this\nway is sometimes referred to as \'metaclass confusion\', and is avoided\nby bypassing the instance when looking up special methods:\n\n >>> type(1).__hash__(1) == hash(1)\n True\n >>> type(int).__hash__(int) == hash(int)\n True\n\nIn addition to bypassing any instance attributes in the interest of\ncorrectness, implicit special method lookup generally also bypasses\nthe "__getattribute__()" method even of the object\'s metaclass:\n\n >>> class Meta(type):\n ... def __getattribute__(*args):\n ... print("Metaclass getattribute invoked")\n ... return type.__getattribute__(*args)\n ...\n >>> class C(object, metaclass=Meta):\n ... def __len__(self):\n ... return 10\n ... def __getattribute__(*args):\n ... print("Class getattribute invoked")\n ... return object.__getattribute__(*args)\n ...\n >>> c = C()\n >>> c.__len__() # Explicit lookup via instance\n Class getattribute invoked\n 10\n >>> type(c).__len__(c) # Explicit lookup via type\n Metaclass getattribute invoked\n 10\n >>> len(c) # Implicit lookup\n 10\n\nBypassing the "__getattribute__()" machinery in this fashion provides\nsignificant scope for speed optimisations within the interpreter, at\nthe cost of some flexibility in the handling of special methods (the\nspecial method *must* be set on the class object itself in order to be\nconsistently invoked by the interpreter).\n', - 'string-methods': u'\nString Methods\n**************\n\nStrings implement all of the *common* sequence operations, along with\nthe additional methods described below.\n\nStrings also support two styles of string formatting, one providing a\nlarge degree of flexibility and customization (see "str.format()",\n*Format String Syntax* and *String Formatting*) and the other based on\nC "printf" style formatting that handles a narrower range of types and\nis slightly harder to use correctly, but is often faster for the cases\nit can handle (*printf-style String Formatting*).\n\nThe *Text Processing Services* section of the standard library covers\na number of other modules that provide various text related utilities\n(including regular expression support in the "re" module).\n\nstr.capitalize()\n\n Return a copy of the string with its first character capitalized\n and the rest lowercased.\n\nstr.casefold()\n\n Return a casefolded copy of the string. Casefolded strings may be\n used for caseless matching.\n\n Casefolding is similar to lowercasing but more aggressive because\n it is intended to remove all case distinctions in a string. For\n example, the German lowercase letter "\'\xdf\'" is equivalent to ""ss"".\n Since it is already lowercase, "lower()" would do nothing to "\'\xdf\'";\n "casefold()" converts it to ""ss"".\n\n The casefolding algorithm is described in section 3.13 of the\n Unicode Standard.\n\n New in version 3.3.\n\nstr.center(width[, fillchar])\n\n Return centered in a string of length *width*. Padding is done\n using the specified *fillchar* (default is an ASCII space). The\n original string is returned if *width* is less than or equal to\n "len(s)".\n\nstr.count(sub[, start[, end]])\n\n Return the number of non-overlapping occurrences of substring *sub*\n in the range [*start*, *end*]. Optional arguments *start* and\n *end* are interpreted as in slice notation.\n\nstr.encode(encoding="utf-8", errors="strict")\n\n Return an encoded version of the string as a bytes object. Default\n encoding is "\'utf-8\'". *errors* may be given to set a different\n error handling scheme. The default for *errors* is "\'strict\'",\n meaning that encoding errors raise a "UnicodeError". Other possible\n values are "\'ignore\'", "\'replace\'", "\'xmlcharrefreplace\'",\n "\'backslashreplace\'" and any other name registered via\n "codecs.register_error()", see section *Error Handlers*. For a list\n of possible encodings, see section *Standard Encodings*.\n\n Changed in version 3.1: Support for keyword arguments added.\n\nstr.endswith(suffix[, start[, end]])\n\n Return "True" if the string ends with the specified *suffix*,\n otherwise return "False". *suffix* can also be a tuple of suffixes\n to look for. With optional *start*, test beginning at that\n position. With optional *end*, stop comparing at that position.\n\nstr.expandtabs(tabsize=8)\n\n Return a copy of the string where all tab characters are replaced\n by one or more spaces, depending on the current column and the\n given tab size. Tab positions occur every *tabsize* characters\n (default is 8, giving tab positions at columns 0, 8, 16 and so on).\n To expand the string, the current column is set to zero and the\n string is examined character by character. If the character is a\n tab ("\\t"), one or more space characters are inserted in the result\n until the current column is equal to the next tab position. (The\n tab character itself is not copied.) If the character is a newline\n ("\\n") or return ("\\r"), it is copied and the current column is\n reset to zero. Any other character is copied unchanged and the\n current column is incremented by one regardless of how the\n character is represented when printed.\n\n >>> \'01\\t012\\t0123\\t01234\'.expandtabs()\n \'01 012 0123 01234\'\n >>> \'01\\t012\\t0123\\t01234\'.expandtabs(4)\n \'01 012 0123 01234\'\n\nstr.find(sub[, start[, end]])\n\n Return the lowest index in the string where substring *sub* is\n found, such that *sub* is contained in the slice "s[start:end]".\n Optional arguments *start* and *end* are interpreted as in slice\n notation. Return "-1" if *sub* is not found.\n\n Note: The "find()" method should be used only if you need to know\n the position of *sub*. To check if *sub* is a substring or not,\n use the "in" operator:\n\n >>> \'Py\' in \'Python\'\n True\n\nstr.format(*args, **kwargs)\n\n Perform a string formatting operation. The string on which this\n method is called can contain literal text or replacement fields\n delimited by braces "{}". Each replacement field contains either\n the numeric index of a positional argument, or the name of a\n keyword argument. Returns a copy of the string where each\n replacement field is replaced with the string value of the\n corresponding argument.\n\n >>> "The sum of 1 + 2 is {0}".format(1+2)\n \'The sum of 1 + 2 is 3\'\n\n See *Format String Syntax* for a description of the various\n formatting options that can be specified in format strings.\n\nstr.format_map(mapping)\n\n Similar to "str.format(**mapping)", except that "mapping" is used\n directly and not copied to a "dict". This is useful if for example\n "mapping" is a dict subclass:\n\n >>> class Default(dict):\n ... def __missing__(self, key):\n ... return key\n ...\n >>> \'{name} was born in {country}\'.format_map(Default(name=\'Guido\'))\n \'Guido was born in country\'\n\n New in version 3.2.\n\nstr.index(sub[, start[, end]])\n\n Like "find()", but raise "ValueError" when the substring is not\n found.\n\nstr.isalnum()\n\n Return true if all characters in the string are alphanumeric and\n there is at least one character, false otherwise. A character "c"\n is alphanumeric if one of the following returns "True":\n "c.isalpha()", "c.isdecimal()", "c.isdigit()", or "c.isnumeric()".\n\nstr.isalpha()\n\n Return true if all characters in the string are alphabetic and\n there is at least one character, false otherwise. Alphabetic\n characters are those characters defined in the Unicode character\n database as "Letter", i.e., those with general category property\n being one of "Lm", "Lt", "Lu", "Ll", or "Lo". Note that this is\n different from the "Alphabetic" property defined in the Unicode\n Standard.\n\nstr.isdecimal()\n\n Return true if all characters in the string are decimal characters\n and there is at least one character, false otherwise. Decimal\n characters are those from general category "Nd". This category\n includes digit characters, and all characters that can be used to\n form decimal-radix numbers, e.g. U+0660, ARABIC-INDIC DIGIT ZERO.\n\nstr.isdigit()\n\n Return true if all characters in the string are digits and there is\n at least one character, false otherwise. Digits include decimal\n characters and digits that need special handling, such as the\n compatibility superscript digits. Formally, a digit is a character\n that has the property value Numeric_Type=Digit or\n Numeric_Type=Decimal.\n\nstr.isidentifier()\n\n Return true if the string is a valid identifier according to the\n language definition, section *Identifiers and keywords*.\n\n Use "keyword.iskeyword()" to test for reserved identifiers such as\n "def" and "class".\n\nstr.islower()\n\n Return true if all cased characters [4] in the string are lowercase\n and there is at least one cased character, false otherwise.\n\nstr.isnumeric()\n\n Return true if all characters in the string are numeric characters,\n and there is at least one character, false otherwise. Numeric\n characters include digit characters, and all characters that have\n the Unicode numeric value property, e.g. U+2155, VULGAR FRACTION\n ONE FIFTH. Formally, numeric characters are those with the\n property value Numeric_Type=Digit, Numeric_Type=Decimal or\n Numeric_Type=Numeric.\n\nstr.isprintable()\n\n Return true if all characters in the string are printable or the\n string is empty, false otherwise. Nonprintable characters are\n those characters defined in the Unicode character database as\n "Other" or "Separator", excepting the ASCII space (0x20) which is\n considered printable. (Note that printable characters in this\n context are those which should not be escaped when "repr()" is\n invoked on a string. It has no bearing on the handling of strings\n written to "sys.stdout" or "sys.stderr".)\n\nstr.isspace()\n\n Return true if there are only whitespace characters in the string\n and there is at least one character, false otherwise. Whitespace\n characters are those characters defined in the Unicode character\n database as "Other" or "Separator" and those with bidirectional\n property being one of "WS", "B", or "S".\n\nstr.istitle()\n\n Return true if the string is a titlecased string and there is at\n least one character, for example uppercase characters may only\n follow uncased characters and lowercase characters only cased ones.\n Return false otherwise.\n\nstr.isupper()\n\n Return true if all cased characters [4] in the string are uppercase\n and there is at least one cased character, false otherwise.\n\nstr.join(iterable)\n\n Return a string which is the concatenation of the strings in the\n *iterable* *iterable*. A "TypeError" will be raised if there are\n any non-string values in *iterable*, including "bytes" objects.\n The separator between elements is the string providing this method.\n\nstr.ljust(width[, fillchar])\n\n Return the string left justified in a string of length *width*.\n Padding is done using the specified *fillchar* (default is an ASCII\n space). The original string is returned if *width* is less than or\n equal to "len(s)".\n\nstr.lower()\n\n Return a copy of the string with all the cased characters [4]\n converted to lowercase.\n\n The lowercasing algorithm used is described in section 3.13 of the\n Unicode Standard.\n\nstr.lstrip([chars])\n\n Return a copy of the string with leading characters removed. The\n *chars* argument is a string specifying the set of characters to be\n removed. If omitted or "None", the *chars* argument defaults to\n removing whitespace. The *chars* argument is not a prefix; rather,\n all combinations of its values are stripped:\n\n >>> \' spacious \'.lstrip()\n \'spacious \'\n >>> \'www.example.com\'.lstrip(\'cmowz.\')\n \'example.com\'\n\nstatic str.maketrans(x[, y[, z]])\n\n This static method returns a translation table usable for\n "str.translate()".\n\n If there is only one argument, it must be a dictionary mapping\n Unicode ordinals (integers) or characters (strings of length 1) to\n Unicode ordinals, strings (of arbitrary lengths) or None.\n Character keys will then be converted to ordinals.\n\n If there are two arguments, they must be strings of equal length,\n and in the resulting dictionary, each character in x will be mapped\n to the character at the same position in y. If there is a third\n argument, it must be a string, whose characters will be mapped to\n None in the result.\n\nstr.partition(sep)\n\n Split the string at the first occurrence of *sep*, and return a\n 3-tuple containing the part before the separator, the separator\n itself, and the part after the separator. If the separator is not\n found, return a 3-tuple containing the string itself, followed by\n two empty strings.\n\nstr.replace(old, new[, count])\n\n Return a copy of the string with all occurrences of substring *old*\n replaced by *new*. If the optional argument *count* is given, only\n the first *count* occurrences are replaced.\n\nstr.rfind(sub[, start[, end]])\n\n Return the highest index in the string where substring *sub* is\n found, such that *sub* is contained within "s[start:end]".\n Optional arguments *start* and *end* are interpreted as in slice\n notation. Return "-1" on failure.\n\nstr.rindex(sub[, start[, end]])\n\n Like "rfind()" but raises "ValueError" when the substring *sub* is\n not found.\n\nstr.rjust(width[, fillchar])\n\n Return the string right justified in a string of length *width*.\n Padding is done using the specified *fillchar* (default is an ASCII\n space). The original string is returned if *width* is less than or\n equal to "len(s)".\n\nstr.rpartition(sep)\n\n Split the string at the last occurrence of *sep*, and return a\n 3-tuple containing the part before the separator, the separator\n itself, and the part after the separator. If the separator is not\n found, return a 3-tuple containing two empty strings, followed by\n the string itself.\n\nstr.rsplit(sep=None, maxsplit=-1)\n\n Return a list of the words in the string, using *sep* as the\n delimiter string. If *maxsplit* is given, at most *maxsplit* splits\n are done, the *rightmost* ones. If *sep* is not specified or\n "None", any whitespace string is a separator. Except for splitting\n from the right, "rsplit()" behaves like "split()" which is\n described in detail below.\n\nstr.rstrip([chars])\n\n Return a copy of the string with trailing characters removed. The\n *chars* argument is a string specifying the set of characters to be\n removed. If omitted or "None", the *chars* argument defaults to\n removing whitespace. The *chars* argument is not a suffix; rather,\n all combinations of its values are stripped:\n\n >>> \' spacious \'.rstrip()\n \' spacious\'\n >>> \'mississippi\'.rstrip(\'ipz\')\n \'mississ\'\n\nstr.split(sep=None, maxsplit=-1)\n\n Return a list of the words in the string, using *sep* as the\n delimiter string. If *maxsplit* is given, at most *maxsplit*\n splits are done (thus, the list will have at most "maxsplit+1"\n elements). If *maxsplit* is not specified or "-1", then there is\n no limit on the number of splits (all possible splits are made).\n\n If *sep* is given, consecutive delimiters are not grouped together\n and are deemed to delimit empty strings (for example,\n "\'1,,2\'.split(\',\')" returns "[\'1\', \'\', \'2\']"). The *sep* argument\n may consist of multiple characters (for example,\n "\'1<>2<>3\'.split(\'<>\')" returns "[\'1\', \'2\', \'3\']"). Splitting an\n empty string with a specified separator returns "[\'\']".\n\n For example:\n\n >>> \'1,2,3\'.split(\',\')\n [\'1\', \'2\', \'3\']\n >>> \'1,2,3\'.split(\',\', maxsplit=1)\n [\'1\', \'2,3\']\n >>> \'1,2,,3,\'.split(\',\')\n [\'1\', \'2\', \'\', \'3\', \'\']\n\n If *sep* is not specified or is "None", a different splitting\n algorithm is applied: runs of consecutive whitespace are regarded\n as a single separator, and the result will contain no empty strings\n at the start or end if the string has leading or trailing\n whitespace. Consequently, splitting an empty string or a string\n consisting of just whitespace with a "None" separator returns "[]".\n\n For example:\n\n >>> \'1 2 3\'.split()\n [\'1\', \'2\', \'3\']\n >>> \'1 2 3\'.split(maxsplit=1)\n [\'1\', \'2 3\']\n >>> \' 1 2 3 \'.split()\n [\'1\', \'2\', \'3\']\n\nstr.splitlines([keepends])\n\n Return a list of the lines in the string, breaking at line\n boundaries. Line breaks are not included in the resulting list\n unless *keepends* is given and true.\n\n This method splits on the following line boundaries. In\n particular, the boundaries are a superset of *universal newlines*.\n\n +-------------------------+-------------------------------+\n | Representation | Description |\n +=========================+===============================+\n | "\\n" | Line Feed |\n +-------------------------+-------------------------------+\n | "\\r" | Carriage Return |\n +-------------------------+-------------------------------+\n | "\\r\\n" | Carriage Return + Line Feed |\n +-------------------------+-------------------------------+\n | "\\v" or "\\x0b" | Line Tabulation |\n +-------------------------+-------------------------------+\n | "\\f" or "\\x0c" | Form Feed |\n +-------------------------+-------------------------------+\n | "\\x1c" | File Separator |\n +-------------------------+-------------------------------+\n | "\\x1d" | Group Separator |\n +-------------------------+-------------------------------+\n | "\\x1e" | Record Separator |\n +-------------------------+-------------------------------+\n | "\\x85" | Next Line (C1 Control Code) |\n +-------------------------+-------------------------------+\n | "\\u2028" | Line Separator |\n +-------------------------+-------------------------------+\n | "\\u2029" | Paragraph Separator |\n +-------------------------+-------------------------------+\n\n Changed in version 3.2: "\\v" and "\\f" added to list of line\n boundaries.\n\n For example:\n\n >>> \'ab c\\n\\nde fg\\rkl\\r\\n\'.splitlines()\n [\'ab c\', \'\', \'de fg\', \'kl\']\n >>> \'ab c\\n\\nde fg\\rkl\\r\\n\'.splitlines(keepends=True)\n [\'ab c\\n\', \'\\n\', \'de fg\\r\', \'kl\\r\\n\']\n\n Unlike "split()" when a delimiter string *sep* is given, this\n method returns an empty list for the empty string, and a terminal\n line break does not result in an extra line:\n\n >>> "".splitlines()\n []\n >>> "One line\\n".splitlines()\n [\'One line\']\n\n For comparison, "split(\'\\n\')" gives:\n\n >>> \'\'.split(\'\\n\')\n [\'\']\n >>> \'Two lines\\n\'.split(\'\\n\')\n [\'Two lines\', \'\']\n\nstr.startswith(prefix[, start[, end]])\n\n Return "True" if string starts with the *prefix*, otherwise return\n "False". *prefix* can also be a tuple of prefixes to look for.\n With optional *start*, test string beginning at that position.\n With optional *end*, stop comparing string at that position.\n\nstr.strip([chars])\n\n Return a copy of the string with the leading and trailing\n characters removed. The *chars* argument is a string specifying the\n set of characters to be removed. If omitted or "None", the *chars*\n argument defaults to removing whitespace. The *chars* argument is\n not a prefix or suffix; rather, all combinations of its values are\n stripped:\n\n >>> \' spacious \'.strip()\n \'spacious\'\n >>> \'www.example.com\'.strip(\'cmowz.\')\n \'example\'\n\n The outermost leading and trailing *chars* argument values are\n stripped from the string. Characters are removed from the leading\n end until reaching a string character that is not contained in the\n set of characters in *chars*. A similar action takes place on the\n trailing end. For example:\n\n >>> comment_string = \'#....... Section 3.2.1 Issue #32 .......\'\n >>> comment_string.strip(\'.#! \')\n \'Section 3.2.1 Issue #32\'\n\nstr.swapcase()\n\n Return a copy of the string with uppercase characters converted to\n lowercase and vice versa. Note that it is not necessarily true that\n "s.swapcase().swapcase() == s".\n\nstr.title()\n\n Return a titlecased version of the string where words start with an\n uppercase character and the remaining characters are lowercase.\n\n For example:\n\n >>> \'Hello world\'.title()\n \'Hello World\'\n\n The algorithm uses a simple language-independent definition of a\n word as groups of consecutive letters. The definition works in\n many contexts but it means that apostrophes in contractions and\n possessives form word boundaries, which may not be the desired\n result:\n\n >>> "they\'re bill\'s friends from the UK".title()\n "They\'Re Bill\'S Friends From The Uk"\n\n A workaround for apostrophes can be constructed using regular\n expressions:\n\n >>> import re\n >>> def titlecase(s):\n ... return re.sub(r"[A-Za-z]+(\'[A-Za-z]+)?",\n ... lambda mo: mo.group(0)[0].upper() +\n ... mo.group(0)[1:].lower(),\n ... s)\n ...\n >>> titlecase("they\'re bill\'s friends.")\n "They\'re Bill\'s Friends."\n\nstr.translate(table)\n\n Return a copy of the string in which each character has been mapped\n through the given translation table. The table must be an object\n that implements indexing via "__getitem__()", typically a *mapping*\n or *sequence*. When indexed by a Unicode ordinal (an integer), the\n table object can do any of the following: return a Unicode ordinal\n or a string, to map the character to one or more other characters;\n return "None", to delete the character from the return string; or\n raise a "LookupError" exception, to map the character to itself.\n\n You can use "str.maketrans()" to create a translation map from\n character-to-character mappings in different formats.\n\n See also the "codecs" module for a more flexible approach to custom\n character mappings.\n\nstr.upper()\n\n Return a copy of the string with all the cased characters [4]\n converted to uppercase. Note that "str.upper().isupper()" might be\n "False" if "s" contains uncased characters or if the Unicode\n category of the resulting character(s) is not "Lu" (Letter,\n uppercase), but e.g. "Lt" (Letter, titlecase).\n\n The uppercasing algorithm used is described in section 3.13 of the\n Unicode Standard.\n\nstr.zfill(width)\n\n Return a copy of the string left filled with ASCII "\'0\'" digits to\n make a string of length *width*. A leading sign prefix\n ("\'+\'"/"\'-\'") is handled by inserting the padding *after* the sign\n character rather than before. The original string is returned if\n *width* is less than or equal to "len(s)".\n\n For example:\n\n >>> "42".zfill(5)\n \'00042\'\n >>> "-42".zfill(5)\n \'-0042\'\n', - 'strings': u'\nString and Bytes literals\n*************************\n\nString literals are described by the following lexical definitions:\n\n stringliteral ::= [stringprefix](shortstring | longstring)\n stringprefix ::= "r" | "u" | "R" | "U"\n shortstring ::= "\'" shortstringitem* "\'" | \'"\' shortstringitem* \'"\'\n longstring ::= "\'\'\'" longstringitem* "\'\'\'" | \'"""\' longstringitem* \'"""\'\n shortstringitem ::= shortstringchar | stringescapeseq\n longstringitem ::= longstringchar | stringescapeseq\n shortstringchar ::= \n longstringchar ::= \n stringescapeseq ::= "\\" \n\n bytesliteral ::= bytesprefix(shortbytes | longbytes)\n bytesprefix ::= "b" | "B" | "br" | "Br" | "bR" | "BR" | "rb" | "rB" | "Rb" | "RB"\n shortbytes ::= "\'" shortbytesitem* "\'" | \'"\' shortbytesitem* \'"\'\n longbytes ::= "\'\'\'" longbytesitem* "\'\'\'" | \'"""\' longbytesitem* \'"""\'\n shortbytesitem ::= shortbyteschar | bytesescapeseq\n longbytesitem ::= longbyteschar | bytesescapeseq\n shortbyteschar ::= \n longbyteschar ::= \n bytesescapeseq ::= "\\" \n\nOne syntactic restriction not indicated by these productions is that\nwhitespace is not allowed between the "stringprefix" or "bytesprefix"\nand the rest of the literal. The source character set is defined by\nthe encoding declaration; it is UTF-8 if no encoding declaration is\ngiven in the source file; see section *Encoding declarations*.\n\nIn plain English: Both types of literals can be enclosed in matching\nsingle quotes ("\'") or double quotes ("""). They can also be enclosed\nin matching groups of three single or double quotes (these are\ngenerally referred to as *triple-quoted strings*). The backslash\n("\\") character is used to escape characters that otherwise have a\nspecial meaning, such as newline, backslash itself, or the quote\ncharacter.\n\nBytes literals are always prefixed with "\'b\'" or "\'B\'"; they produce\nan instance of the "bytes" type instead of the "str" type. They may\nonly contain ASCII characters; bytes with a numeric value of 128 or\ngreater must be expressed with escapes.\n\nAs of Python 3.3 it is possible again to prefix string literals with a\n"u" prefix to simplify maintenance of dual 2.x and 3.x codebases.\n\nBoth string and bytes literals may optionally be prefixed with a\nletter "\'r\'" or "\'R\'"; such strings are called *raw strings* and treat\nbackslashes as literal characters. As a result, in string literals,\n"\'\\U\'" and "\'\\u\'" escapes in raw strings are not treated specially.\nGiven that Python 2.x\'s raw unicode literals behave differently than\nPython 3.x\'s the "\'ur\'" syntax is not supported.\n\nNew in version 3.3: The "\'rb\'" prefix of raw bytes literals has been\nadded as a synonym of "\'br\'".\n\nNew in version 3.3: Support for the unicode legacy literal\n("u\'value\'") was reintroduced to simplify the maintenance of dual\nPython 2.x and 3.x codebases. See **PEP 414** for more information.\n\nIn triple-quoted literals, unescaped newlines and quotes are allowed\n(and are retained), except that three unescaped quotes in a row\nterminate the literal. (A "quote" is the character used to open the\nliteral, i.e. either "\'" or """.)\n\nUnless an "\'r\'" or "\'R\'" prefix is present, escape sequences in string\nand bytes literals are interpreted according to rules similar to those\nused by Standard C. The recognized escape sequences are:\n\n+-------------------+-----------------------------------+---------+\n| Escape Sequence | Meaning | Notes |\n+===================+===================================+=========+\n| "\\newline" | Backslash and newline ignored | |\n+-------------------+-----------------------------------+---------+\n| "\\\\" | Backslash ("\\") | |\n+-------------------+-----------------------------------+---------+\n| "\\\'" | Single quote ("\'") | |\n+-------------------+-----------------------------------+---------+\n| "\\"" | Double quote (""") | |\n+-------------------+-----------------------------------+---------+\n| "\\a" | ASCII Bell (BEL) | |\n+-------------------+-----------------------------------+---------+\n| "\\b" | ASCII Backspace (BS) | |\n+-------------------+-----------------------------------+---------+\n| "\\f" | ASCII Formfeed (FF) | |\n+-------------------+-----------------------------------+---------+\n| "\\n" | ASCII Linefeed (LF) | |\n+-------------------+-----------------------------------+---------+\n| "\\r" | ASCII Carriage Return (CR) | |\n+-------------------+-----------------------------------+---------+\n| "\\t" | ASCII Horizontal Tab (TAB) | |\n+-------------------+-----------------------------------+---------+\n| "\\v" | ASCII Vertical Tab (VT) | |\n+-------------------+-----------------------------------+---------+\n| "\\ooo" | Character with octal value *ooo* | (1,3) |\n+-------------------+-----------------------------------+---------+\n| "\\xhh" | Character with hex value *hh* | (2,3) |\n+-------------------+-----------------------------------+---------+\n\nEscape sequences only recognized in string literals are:\n\n+-------------------+-----------------------------------+---------+\n| Escape Sequence | Meaning | Notes |\n+===================+===================================+=========+\n| "\\N{name}" | Character named *name* in the | (4) |\n| | Unicode database | |\n+-------------------+-----------------------------------+---------+\n| "\\uxxxx" | Character with 16-bit hex value | (5) |\n| | *xxxx* | |\n+-------------------+-----------------------------------+---------+\n| "\\Uxxxxxxxx" | Character with 32-bit hex value | (6) |\n| | *xxxxxxxx* | |\n+-------------------+-----------------------------------+---------+\n\nNotes:\n\n1. As in Standard C, up to three octal digits are accepted.\n\n2. Unlike in Standard C, exactly two hex digits are required.\n\n3. In a bytes literal, hexadecimal and octal escapes denote the\n byte with the given value. In a string literal, these escapes\n denote a Unicode character with the given value.\n\n4. Changed in version 3.3: Support for name aliases [1] has been\n added.\n\n5. Individual code units which form parts of a surrogate pair can\n be encoded using this escape sequence. Exactly four hex digits are\n required.\n\n6. Any Unicode character can be encoded this way. Exactly eight\n hex digits are required.\n\nUnlike Standard C, all unrecognized escape sequences are left in the\nstring unchanged, i.e., *the backslash is left in the result*. (This\nbehavior is useful when debugging: if an escape sequence is mistyped,\nthe resulting output is more easily recognized as broken.) It is also\nimportant to note that the escape sequences only recognized in string\nliterals fall into the category of unrecognized escapes for bytes\nliterals.\n\nEven in a raw literal, quotes can be escaped with a backslash, but the\nbackslash remains in the result; for example, "r"\\""" is a valid\nstring literal consisting of two characters: a backslash and a double\nquote; "r"\\"" is not a valid string literal (even a raw string cannot\nend in an odd number of backslashes). Specifically, *a raw literal\ncannot end in a single backslash* (since the backslash would escape\nthe following quote character). Note also that a single backslash\nfollowed by a newline is interpreted as those two characters as part\nof the literal, *not* as a line continuation.\n', - 'subscriptions': u'\nSubscriptions\n*************\n\nA subscription selects an item of a sequence (string, tuple or list)\nor mapping (dictionary) object:\n\n subscription ::= primary "[" expression_list "]"\n\nThe primary must evaluate to an object that supports subscription\n(lists or dictionaries for example). User-defined objects can support\nsubscription by defining a "__getitem__()" method.\n\nFor built-in objects, there are two types of objects that support\nsubscription:\n\nIf the primary is a mapping, the expression list must evaluate to an\nobject whose value is one of the keys of the mapping, and the\nsubscription selects the value in the mapping that corresponds to that\nkey. (The expression list is a tuple except if it has exactly one\nitem.)\n\nIf the primary is a sequence, the expression (list) must evaluate to\nan integer or a slice (as discussed in the following section).\n\nThe formal syntax makes no special provision for negative indices in\nsequences; however, built-in sequences all provide a "__getitem__()"\nmethod that interprets negative indices by adding the length of the\nsequence to the index (so that "x[-1]" selects the last item of "x").\nThe resulting value must be a nonnegative integer less than the number\nof items in the sequence, and the subscription selects the item whose\nindex is that value (counting from zero). Since the support for\nnegative indices and slicing occurs in the object\'s "__getitem__()"\nmethod, subclasses overriding this method will need to explicitly add\nthat support.\n\nA string\'s items are characters. A character is not a separate data\ntype but a string of exactly one character.\n', - 'truth': u'\nTruth Value Testing\n*******************\n\nAny object can be tested for truth value, for use in an "if" or\n"while" condition or as operand of the Boolean operations below. The\nfollowing values are considered false:\n\n* "None"\n\n* "False"\n\n* zero of any numeric type, for example, "0", "0.0", "0j".\n\n* any empty sequence, for example, "\'\'", "()", "[]".\n\n* any empty mapping, for example, "{}".\n\n* instances of user-defined classes, if the class defines a\n "__bool__()" or "__len__()" method, when that method returns the\n integer zero or "bool" value "False". [1]\n\nAll other values are considered true --- so objects of many types are\nalways true.\n\nOperations and built-in functions that have a Boolean result always\nreturn "0" or "False" for false and "1" or "True" for true, unless\notherwise stated. (Important exception: the Boolean operations "or"\nand "and" always return one of their operands.)\n', - 'try': u'\nThe "try" statement\n*******************\n\nThe "try" statement specifies exception handlers and/or cleanup code\nfor a group of statements:\n\n try_stmt ::= try1_stmt | try2_stmt\n try1_stmt ::= "try" ":" suite\n ("except" [expression ["as" identifier]] ":" suite)+\n ["else" ":" suite]\n ["finally" ":" suite]\n try2_stmt ::= "try" ":" suite\n "finally" ":" suite\n\nThe "except" clause(s) specify one or more exception handlers. When no\nexception occurs in the "try" clause, no exception handler is\nexecuted. When an exception occurs in the "try" suite, a search for an\nexception handler is started. This search inspects the except clauses\nin turn until one is found that matches the exception. An expression-\nless except clause, if present, must be last; it matches any\nexception. For an except clause with an expression, that expression\nis evaluated, and the clause matches the exception if the resulting\nobject is "compatible" with the exception. An object is compatible\nwith an exception if it is the class or a base class of the exception\nobject or a tuple containing an item compatible with the exception.\n\nIf no except clause matches the exception, the search for an exception\nhandler continues in the surrounding code and on the invocation stack.\n[1]\n\nIf the evaluation of an expression in the header of an except clause\nraises an exception, the original search for a handler is canceled and\na search starts for the new exception in the surrounding code and on\nthe call stack (it is treated as if the entire "try" statement raised\nthe exception).\n\nWhen a matching except clause is found, the exception is assigned to\nthe target specified after the "as" keyword in that except clause, if\npresent, and the except clause\'s suite is executed. All except\nclauses must have an executable block. When the end of this block is\nreached, execution continues normally after the entire try statement.\n(This means that if two nested handlers exist for the same exception,\nand the exception occurs in the try clause of the inner handler, the\nouter handler will not handle the exception.)\n\nWhen an exception has been assigned using "as target", it is cleared\nat the end of the except clause. This is as if\n\n except E as N:\n foo\n\nwas translated to\n\n except E as N:\n try:\n foo\n finally:\n del N\n\nThis means the exception must be assigned to a different name to be\nable to refer to it after the except clause. Exceptions are cleared\nbecause with the traceback attached to them, they form a reference\ncycle with the stack frame, keeping all locals in that frame alive\nuntil the next garbage collection occurs.\n\nBefore an except clause\'s suite is executed, details about the\nexception are stored in the "sys" module and can be accessed via\n"sys.exc_info()". "sys.exc_info()" returns a 3-tuple consisting of the\nexception class, the exception instance and a traceback object (see\nsection *The standard type hierarchy*) identifying the point in the\nprogram where the exception occurred. "sys.exc_info()" values are\nrestored to their previous values (before the call) when returning\nfrom a function that handled an exception.\n\nThe optional "else" clause is executed if and when control flows off\nthe end of the "try" clause. [2] Exceptions in the "else" clause are\nnot handled by the preceding "except" clauses.\n\nIf "finally" is present, it specifies a \'cleanup\' handler. The "try"\nclause is executed, including any "except" and "else" clauses. If an\nexception occurs in any of the clauses and is not handled, the\nexception is temporarily saved. The "finally" clause is executed. If\nthere is a saved exception it is re-raised at the end of the "finally"\nclause. If the "finally" clause raises another exception, the saved\nexception is set as the context of the new exception. If the "finally"\nclause executes a "return" or "break" statement, the saved exception\nis discarded:\n\n >>> def f():\n ... try:\n ... 1/0\n ... finally:\n ... return 42\n ...\n >>> f()\n 42\n\nThe exception information is not available to the program during\nexecution of the "finally" clause.\n\nWhen a "return", "break" or "continue" statement is executed in the\n"try" suite of a "try"..."finally" statement, the "finally" clause is\nalso executed \'on the way out.\' A "continue" statement is illegal in\nthe "finally" clause. (The reason is a problem with the current\nimplementation --- this restriction may be lifted in the future).\n\nThe return value of a function is determined by the last "return"\nstatement executed. Since the "finally" clause always executes, a\n"return" statement executed in the "finally" clause will always be the\nlast one executed:\n\n >>> def foo():\n ... try:\n ... return \'try\'\n ... finally:\n ... return \'finally\'\n ...\n >>> foo()\n \'finally\'\n\nAdditional information on exceptions can be found in section\n*Exceptions*, and information on using the "raise" statement to\ngenerate exceptions may be found in section *The raise statement*.\n', - 'types': u'\nThe standard type hierarchy\n***************************\n\nBelow is a list of the types that are built into Python. Extension\nmodules (written in C, Java, or other languages, depending on the\nimplementation) can define additional types. Future versions of\nPython may add types to the type hierarchy (e.g., rational numbers,\nefficiently stored arrays of integers, etc.), although such additions\nwill often be provided via the standard library instead.\n\nSome of the type descriptions below contain a paragraph listing\n\'special attributes.\' These are attributes that provide access to the\nimplementation and are not intended for general use. Their definition\nmay change in the future.\n\nNone\n This type has a single value. There is a single object with this\n value. This object is accessed through the built-in name "None". It\n is used to signify the absence of a value in many situations, e.g.,\n it is returned from functions that don\'t explicitly return\n anything. Its truth value is false.\n\nNotImplemented\n This type has a single value. There is a single object with this\n value. This object is accessed through the built-in name\n "NotImplemented". Numeric methods and rich comparison methods\n should return this value if they do not implement the operation for\n the operands provided. (The interpreter will then try the\n reflected operation, or some other fallback, depending on the\n operator.) Its truth value is true.\n\n See *Implementing the arithmetic operations* for more details.\n\nEllipsis\n This type has a single value. There is a single object with this\n value. This object is accessed through the literal "..." or the\n built-in name "Ellipsis". Its truth value is true.\n\n"numbers.Number"\n These are created by numeric literals and returned as results by\n arithmetic operators and arithmetic built-in functions. Numeric\n objects are immutable; once created their value never changes.\n Python numbers are of course strongly related to mathematical\n numbers, but subject to the limitations of numerical representation\n in computers.\n\n Python distinguishes between integers, floating point numbers, and\n complex numbers:\n\n "numbers.Integral"\n These represent elements from the mathematical set of integers\n (positive and negative).\n\n There are two types of integers:\n\n Integers ("int")\n\n These represent numbers in an unlimited range, subject to\n available (virtual) memory only. For the purpose of shift\n and mask operations, a binary representation is assumed, and\n negative numbers are represented in a variant of 2\'s\n complement which gives the illusion of an infinite string of\n sign bits extending to the left.\n\n Booleans ("bool")\n These represent the truth values False and True. The two\n objects representing the values "False" and "True" are the\n only Boolean objects. The Boolean type is a subtype of the\n integer type, and Boolean values behave like the values 0 and\n 1, respectively, in almost all contexts, the exception being\n that when converted to a string, the strings ""False"" or\n ""True"" are returned, respectively.\n\n The rules for integer representation are intended to give the\n most meaningful interpretation of shift and mask operations\n involving negative integers.\n\n "numbers.Real" ("float")\n These represent machine-level double precision floating point\n numbers. You are at the mercy of the underlying machine\n architecture (and C or Java implementation) for the accepted\n range and handling of overflow. Python does not support single-\n precision floating point numbers; the savings in processor and\n memory usage that are usually the reason for using these are\n dwarfed by the overhead of using objects in Python, so there is\n no reason to complicate the language with two kinds of floating\n point numbers.\n\n "numbers.Complex" ("complex")\n These represent complex numbers as a pair of machine-level\n double precision floating point numbers. The same caveats apply\n as for floating point numbers. The real and imaginary parts of a\n complex number "z" can be retrieved through the read-only\n attributes "z.real" and "z.imag".\n\nSequences\n These represent finite ordered sets indexed by non-negative\n numbers. The built-in function "len()" returns the number of items\n of a sequence. When the length of a sequence is *n*, the index set\n contains the numbers 0, 1, ..., *n*-1. Item *i* of sequence *a* is\n selected by "a[i]".\n\n Sequences also support slicing: "a[i:j]" selects all items with\n index *k* such that *i* "<=" *k* "<" *j*. When used as an\n expression, a slice is a sequence of the same type. This implies\n that the index set is renumbered so that it starts at 0.\n\n Some sequences also support "extended slicing" with a third "step"\n parameter: "a[i:j:k]" selects all items of *a* with index *x* where\n "x = i + n*k", *n* ">=" "0" and *i* "<=" *x* "<" *j*.\n\n Sequences are distinguished according to their mutability:\n\n Immutable sequences\n An object of an immutable sequence type cannot change once it is\n created. (If the object contains references to other objects,\n these other objects may be mutable and may be changed; however,\n the collection of objects directly referenced by an immutable\n object cannot change.)\n\n The following types are immutable sequences:\n\n Strings\n A string is a sequence of values that represent Unicode code\n points. All the code points in the range "U+0000 - U+10FFFF"\n can be represented in a string. Python doesn\'t have a "char"\n type; instead, every code point in the string is represented\n as a string object with length "1". The built-in function\n "ord()" converts a code point from its string form to an\n integer in the range "0 - 10FFFF"; "chr()" converts an\n integer in the range "0 - 10FFFF" to the corresponding length\n "1" string object. "str.encode()" can be used to convert a\n "str" to "bytes" using the given text encoding, and\n "bytes.decode()" can be used to achieve the opposite.\n\n Tuples\n The items of a tuple are arbitrary Python objects. Tuples of\n two or more items are formed by comma-separated lists of\n expressions. A tuple of one item (a \'singleton\') can be\n formed by affixing a comma to an expression (an expression by\n itself does not create a tuple, since parentheses must be\n usable for grouping of expressions). An empty tuple can be\n formed by an empty pair of parentheses.\n\n Bytes\n A bytes object is an immutable array. The items are 8-bit\n bytes, represented by integers in the range 0 <= x < 256.\n Bytes literals (like "b\'abc\'") and the built-in function\n "bytes()" can be used to construct bytes objects. Also,\n bytes objects can be decoded to strings via the "decode()"\n method.\n\n Mutable sequences\n Mutable sequences can be changed after they are created. The\n subscription and slicing notations can be used as the target of\n assignment and "del" (delete) statements.\n\n There are currently two intrinsic mutable sequence types:\n\n Lists\n The items of a list are arbitrary Python objects. Lists are\n formed by placing a comma-separated list of expressions in\n square brackets. (Note that there are no special cases needed\n to form lists of length 0 or 1.)\n\n Byte Arrays\n A bytearray object is a mutable array. They are created by\n the built-in "bytearray()" constructor. Aside from being\n mutable (and hence unhashable), byte arrays otherwise provide\n the same interface and functionality as immutable bytes\n objects.\n\n The extension module "array" provides an additional example of a\n mutable sequence type, as does the "collections" module.\n\nSet types\n These represent unordered, finite sets of unique, immutable\n objects. As such, they cannot be indexed by any subscript. However,\n they can be iterated over, and the built-in function "len()"\n returns the number of items in a set. Common uses for sets are fast\n membership testing, removing duplicates from a sequence, and\n computing mathematical operations such as intersection, union,\n difference, and symmetric difference.\n\n For set elements, the same immutability rules apply as for\n dictionary keys. Note that numeric types obey the normal rules for\n numeric comparison: if two numbers compare equal (e.g., "1" and\n "1.0"), only one of them can be contained in a set.\n\n There are currently two intrinsic set types:\n\n Sets\n These represent a mutable set. They are created by the built-in\n "set()" constructor and can be modified afterwards by several\n methods, such as "add()".\n\n Frozen sets\n These represent an immutable set. They are created by the\n built-in "frozenset()" constructor. As a frozenset is immutable\n and *hashable*, it can be used again as an element of another\n set, or as a dictionary key.\n\nMappings\n These represent finite sets of objects indexed by arbitrary index\n sets. The subscript notation "a[k]" selects the item indexed by "k"\n from the mapping "a"; this can be used in expressions and as the\n target of assignments or "del" statements. The built-in function\n "len()" returns the number of items in a mapping.\n\n There is currently a single intrinsic mapping type:\n\n Dictionaries\n These represent finite sets of objects indexed by nearly\n arbitrary values. The only types of values not acceptable as\n keys are values containing lists or dictionaries or other\n mutable types that are compared by value rather than by object\n identity, the reason being that the efficient implementation of\n dictionaries requires a key\'s hash value to remain constant.\n Numeric types used for keys obey the normal rules for numeric\n comparison: if two numbers compare equal (e.g., "1" and "1.0")\n then they can be used interchangeably to index the same\n dictionary entry.\n\n Dictionaries are mutable; they can be created by the "{...}"\n notation (see section *Dictionary displays*).\n\n The extension modules "dbm.ndbm" and "dbm.gnu" provide\n additional examples of mapping types, as does the "collections"\n module.\n\nCallable types\n These are the types to which the function call operation (see\n section *Calls*) can be applied:\n\n User-defined functions\n A user-defined function object is created by a function\n definition (see section *Function definitions*). It should be\n called with an argument list containing the same number of items\n as the function\'s formal parameter list.\n\n Special attributes:\n\n +---------------------------+---------------------------------+-------------+\n | Attribute | Meaning | |\n +===========================+=================================+=============+\n | "__doc__" | The function\'s documentation | Writable |\n | | string, or "None" if | |\n | | unavailable; not inherited by | |\n | | subclasses | |\n +---------------------------+---------------------------------+-------------+\n | "__name__" | The function\'s name | Writable |\n +---------------------------+---------------------------------+-------------+\n | "__qualname__" | The function\'s *qualified name* | Writable |\n | | New in version 3.3. | |\n +---------------------------+---------------------------------+-------------+\n | "__module__" | The name of the module the | Writable |\n | | function was defined in, or | |\n | | "None" if unavailable. | |\n +---------------------------+---------------------------------+-------------+\n | "__defaults__" | A tuple containing default | Writable |\n | | argument values for those | |\n | | arguments that have defaults, | |\n | | or "None" if no arguments have | |\n | | a default value | |\n +---------------------------+---------------------------------+-------------+\n | "__code__" | The code object representing | Writable |\n | | the compiled function body. | |\n +---------------------------+---------------------------------+-------------+\n | "__globals__" | A reference to the dictionary | Read-only |\n | | that holds the function\'s | |\n | | global variables --- the global | |\n | | namespace of the module in | |\n | | which the function was defined. | |\n +---------------------------+---------------------------------+-------------+\n | "__dict__" | The namespace supporting | Writable |\n | | arbitrary function attributes. | |\n +---------------------------+---------------------------------+-------------+\n | "__closure__" | "None" or a tuple of cells that | Read-only |\n | | contain bindings for the | |\n | | function\'s free variables. | |\n +---------------------------+---------------------------------+-------------+\n | "__annotations__" | A dict containing annotations | Writable |\n | | of parameters. The keys of the | |\n | | dict are the parameter names, | |\n | | and "\'return\'" for the return | |\n | | annotation, if provided. | |\n +---------------------------+---------------------------------+-------------+\n | "__kwdefaults__" | A dict containing defaults for | Writable |\n | | keyword-only parameters. | |\n +---------------------------+---------------------------------+-------------+\n\n Most of the attributes labelled "Writable" check the type of the\n assigned value.\n\n Function objects also support getting and setting arbitrary\n attributes, which can be used, for example, to attach metadata\n to functions. Regular attribute dot-notation is used to get and\n set such attributes. *Note that the current implementation only\n supports function attributes on user-defined functions. Function\n attributes on built-in functions may be supported in the\n future.*\n\n Additional information about a function\'s definition can be\n retrieved from its code object; see the description of internal\n types below.\n\n Instance methods\n An instance method object combines a class, a class instance and\n any callable object (normally a user-defined function).\n\n Special read-only attributes: "__self__" is the class instance\n object, "__func__" is the function object; "__doc__" is the\n method\'s documentation (same as "__func__.__doc__"); "__name__"\n is the method name (same as "__func__.__name__"); "__module__"\n is the name of the module the method was defined in, or "None"\n if unavailable.\n\n Methods also support accessing (but not setting) the arbitrary\n function attributes on the underlying function object.\n\n User-defined method objects may be created when getting an\n attribute of a class (perhaps via an instance of that class), if\n that attribute is a user-defined function object or a class\n method object.\n\n When an instance method object is created by retrieving a user-\n defined function object from a class via one of its instances,\n its "__self__" attribute is the instance, and the method object\n is said to be bound. The new method\'s "__func__" attribute is\n the original function object.\n\n When a user-defined method object is created by retrieving\n another method object from a class or instance, the behaviour is\n the same as for a function object, except that the "__func__"\n attribute of the new instance is not the original method object\n but its "__func__" attribute.\n\n When an instance method object is created by retrieving a class\n method object from a class or instance, its "__self__" attribute\n is the class itself, and its "__func__" attribute is the\n function object underlying the class method.\n\n When an instance method object is called, the underlying\n function ("__func__") is called, inserting the class instance\n ("__self__") in front of the argument list. For instance, when\n "C" is a class which contains a definition for a function "f()",\n and "x" is an instance of "C", calling "x.f(1)" is equivalent to\n calling "C.f(x, 1)".\n\n When an instance method object is derived from a class method\n object, the "class instance" stored in "__self__" will actually\n be the class itself, so that calling either "x.f(1)" or "C.f(1)"\n is equivalent to calling "f(C,1)" where "f" is the underlying\n function.\n\n Note that the transformation from function object to instance\n method object happens each time the attribute is retrieved from\n the instance. In some cases, a fruitful optimization is to\n assign the attribute to a local variable and call that local\n variable. Also notice that this transformation only happens for\n user-defined functions; other callable objects (and all non-\n callable objects) are retrieved without transformation. It is\n also important to note that user-defined functions which are\n attributes of a class instance are not converted to bound\n methods; this *only* happens when the function is an attribute\n of the class.\n\n Generator functions\n A function or method which uses the "yield" statement (see\n section *The yield statement*) is called a *generator function*.\n Such a function, when called, always returns an iterator object\n which can be used to execute the body of the function: calling\n the iterator\'s "iterator.__next__()" method will cause the\n function to execute until it provides a value using the "yield"\n statement. When the function executes a "return" statement or\n falls off the end, a "StopIteration" exception is raised and the\n iterator will have reached the end of the set of values to be\n returned.\n\n Coroutine functions\n A function or method which is defined using "async def" is\n called a *coroutine function*. Such a function, when called,\n returns a *coroutine* object. It may contain "await"\n expressions, as well as "async with" and "async for" statements.\n See also the *Coroutine Objects* section.\n\n Built-in functions\n A built-in function object is a wrapper around a C function.\n Examples of built-in functions are "len()" and "math.sin()"\n ("math" is a standard built-in module). The number and type of\n the arguments are determined by the C function. Special read-\n only attributes: "__doc__" is the function\'s documentation\n string, or "None" if unavailable; "__name__" is the function\'s\n name; "__self__" is set to "None" (but see the next item);\n "__module__" is the name of the module the function was defined\n in or "None" if unavailable.\n\n Built-in methods\n This is really a different disguise of a built-in function, this\n time containing an object passed to the C function as an\n implicit extra argument. An example of a built-in method is\n "alist.append()", assuming *alist* is a list object. In this\n case, the special read-only attribute "__self__" is set to the\n object denoted by *alist*.\n\n Classes\n Classes are callable. These objects normally act as factories\n for new instances of themselves, but variations are possible for\n class types that override "__new__()". The arguments of the\n call are passed to "__new__()" and, in the typical case, to\n "__init__()" to initialize the new instance.\n\n Class Instances\n Instances of arbitrary classes can be made callable by defining\n a "__call__()" method in their class.\n\nModules\n Modules are a basic organizational unit of Python code, and are\n created by the *import system* as invoked either by the "import"\n statement (see "import"), or by calling functions such as\n "importlib.import_module()" and built-in "__import__()". A module\n object has a namespace implemented by a dictionary object (this is\n the dictionary referenced by the "__globals__" attribute of\n functions defined in the module). Attribute references are\n translated to lookups in this dictionary, e.g., "m.x" is equivalent\n to "m.__dict__["x"]". A module object does not contain the code\n object used to initialize the module (since it isn\'t needed once\n the initialization is done).\n\n Attribute assignment updates the module\'s namespace dictionary,\n e.g., "m.x = 1" is equivalent to "m.__dict__["x"] = 1".\n\n Special read-only attribute: "__dict__" is the module\'s namespace\n as a dictionary object.\n\n **CPython implementation detail:** Because of the way CPython\n clears module dictionaries, the module dictionary will be cleared\n when the module falls out of scope even if the dictionary still has\n live references. To avoid this, copy the dictionary or keep the\n module around while using its dictionary directly.\n\n Predefined (writable) attributes: "__name__" is the module\'s name;\n "__doc__" is the module\'s documentation string, or "None" if\n unavailable; "__file__" is the pathname of the file from which the\n module was loaded, if it was loaded from a file. The "__file__"\n attribute may be missing for certain types of modules, such as C\n modules that are statically linked into the interpreter; for\n extension modules loaded dynamically from a shared library, it is\n the pathname of the shared library file.\n\nCustom classes\n Custom class types are typically created by class definitions (see\n section *Class definitions*). A class has a namespace implemented\n by a dictionary object. Class attribute references are translated\n to lookups in this dictionary, e.g., "C.x" is translated to\n "C.__dict__["x"]" (although there are a number of hooks which allow\n for other means of locating attributes). When the attribute name is\n not found there, the attribute search continues in the base\n classes. This search of the base classes uses the C3 method\n resolution order which behaves correctly even in the presence of\n \'diamond\' inheritance structures where there are multiple\n inheritance paths leading back to a common ancestor. Additional\n details on the C3 MRO used by Python can be found in the\n documentation accompanying the 2.3 release at\n https://www.python.org/download/releases/2.3/mro/.\n\n When a class attribute reference (for class "C", say) would yield a\n class method object, it is transformed into an instance method\n object whose "__self__" attributes is "C". When it would yield a\n static method object, it is transformed into the object wrapped by\n the static method object. See section *Implementing Descriptors*\n for another way in which attributes retrieved from a class may\n differ from those actually contained in its "__dict__".\n\n Class attribute assignments update the class\'s dictionary, never\n the dictionary of a base class.\n\n A class object can be called (see above) to yield a class instance\n (see below).\n\n Special attributes: "__name__" is the class name; "__module__" is\n the module name in which the class was defined; "__dict__" is the\n dictionary containing the class\'s namespace; "__bases__" is a tuple\n (possibly empty or a singleton) containing the base classes, in the\n order of their occurrence in the base class list; "__doc__" is the\n class\'s documentation string, or None if undefined.\n\nClass instances\n A class instance is created by calling a class object (see above).\n A class instance has a namespace implemented as a dictionary which\n is the first place in which attribute references are searched.\n When an attribute is not found there, and the instance\'s class has\n an attribute by that name, the search continues with the class\n attributes. If a class attribute is found that is a user-defined\n function object, it is transformed into an instance method object\n whose "__self__" attribute is the instance. Static method and\n class method objects are also transformed; see above under\n "Classes". See section *Implementing Descriptors* for another way\n in which attributes of a class retrieved via its instances may\n differ from the objects actually stored in the class\'s "__dict__".\n If no class attribute is found, and the object\'s class has a\n "__getattr__()" method, that is called to satisfy the lookup.\n\n Attribute assignments and deletions update the instance\'s\n dictionary, never a class\'s dictionary. If the class has a\n "__setattr__()" or "__delattr__()" method, this is called instead\n of updating the instance dictionary directly.\n\n Class instances can pretend to be numbers, sequences, or mappings\n if they have methods with certain special names. See section\n *Special method names*.\n\n Special attributes: "__dict__" is the attribute dictionary;\n "__class__" is the instance\'s class.\n\nI/O objects (also known as file objects)\n A *file object* represents an open file. Various shortcuts are\n available to create file objects: the "open()" built-in function,\n and also "os.popen()", "os.fdopen()", and the "makefile()" method\n of socket objects (and perhaps by other functions or methods\n provided by extension modules).\n\n The objects "sys.stdin", "sys.stdout" and "sys.stderr" are\n initialized to file objects corresponding to the interpreter\'s\n standard input, output and error streams; they are all open in text\n mode and therefore follow the interface defined by the\n "io.TextIOBase" abstract class.\n\nInternal types\n A few types used internally by the interpreter are exposed to the\n user. Their definitions may change with future versions of the\n interpreter, but they are mentioned here for completeness.\n\n Code objects\n Code objects represent *byte-compiled* executable Python code,\n or *bytecode*. The difference between a code object and a\n function object is that the function object contains an explicit\n reference to the function\'s globals (the module in which it was\n defined), while a code object contains no context; also the\n default argument values are stored in the function object, not\n in the code object (because they represent values calculated at\n run-time). Unlike function objects, code objects are immutable\n and contain no references (directly or indirectly) to mutable\n objects.\n\n Special read-only attributes: "co_name" gives the function name;\n "co_argcount" is the number of positional arguments (including\n arguments with default values); "co_nlocals" is the number of\n local variables used by the function (including arguments);\n "co_varnames" is a tuple containing the names of the local\n variables (starting with the argument names); "co_cellvars" is a\n tuple containing the names of local variables that are\n referenced by nested functions; "co_freevars" is a tuple\n containing the names of free variables; "co_code" is a string\n representing the sequence of bytecode instructions; "co_consts"\n is a tuple containing the literals used by the bytecode;\n "co_names" is a tuple containing the names used by the bytecode;\n "co_filename" is the filename from which the code was compiled;\n "co_firstlineno" is the first line number of the function;\n "co_lnotab" is a string encoding the mapping from bytecode\n offsets to line numbers (for details see the source code of the\n interpreter); "co_stacksize" is the required stack size\n (including local variables); "co_flags" is an integer encoding a\n number of flags for the interpreter.\n\n The following flag bits are defined for "co_flags": bit "0x04"\n is set if the function uses the "*arguments" syntax to accept an\n arbitrary number of positional arguments; bit "0x08" is set if\n the function uses the "**keywords" syntax to accept arbitrary\n keyword arguments; bit "0x20" is set if the function is a\n generator.\n\n Future feature declarations ("from __future__ import division")\n also use bits in "co_flags" to indicate whether a code object\n was compiled with a particular feature enabled: bit "0x2000" is\n set if the function was compiled with future division enabled;\n bits "0x10" and "0x1000" were used in earlier versions of\n Python.\n\n Other bits in "co_flags" are reserved for internal use.\n\n If a code object represents a function, the first item in\n "co_consts" is the documentation string of the function, or\n "None" if undefined.\n\n Frame objects\n Frame objects represent execution frames. They may occur in\n traceback objects (see below).\n\n Special read-only attributes: "f_back" is to the previous stack\n frame (towards the caller), or "None" if this is the bottom\n stack frame; "f_code" is the code object being executed in this\n frame; "f_locals" is the dictionary used to look up local\n variables; "f_globals" is used for global variables;\n "f_builtins" is used for built-in (intrinsic) names; "f_lasti"\n gives the precise instruction (this is an index into the\n bytecode string of the code object).\n\n Special writable attributes: "f_trace", if not "None", is a\n function called at the start of each source code line (this is\n used by the debugger); "f_lineno" is the current line number of\n the frame --- writing to this from within a trace function jumps\n to the given line (only for the bottom-most frame). A debugger\n can implement a Jump command (aka Set Next Statement) by writing\n to f_lineno.\n\n Frame objects support one method:\n\n frame.clear()\n\n This method clears all references to local variables held by\n the frame. Also, if the frame belonged to a generator, the\n generator is finalized. This helps break reference cycles\n involving frame objects (for example when catching an\n exception and storing its traceback for later use).\n\n "RuntimeError" is raised if the frame is currently executing.\n\n New in version 3.4.\n\n Traceback objects\n Traceback objects represent a stack trace of an exception. A\n traceback object is created when an exception occurs. When the\n search for an exception handler unwinds the execution stack, at\n each unwound level a traceback object is inserted in front of\n the current traceback. When an exception handler is entered,\n the stack trace is made available to the program. (See section\n *The try statement*.) It is accessible as the third item of the\n tuple returned by "sys.exc_info()". When the program contains no\n suitable handler, the stack trace is written (nicely formatted)\n to the standard error stream; if the interpreter is interactive,\n it is also made available to the user as "sys.last_traceback".\n\n Special read-only attributes: "tb_next" is the next level in the\n stack trace (towards the frame where the exception occurred), or\n "None" if there is no next level; "tb_frame" points to the\n execution frame of the current level; "tb_lineno" gives the line\n number where the exception occurred; "tb_lasti" indicates the\n precise instruction. The line number and last instruction in\n the traceback may differ from the line number of its frame\n object if the exception occurred in a "try" statement with no\n matching except clause or with a finally clause.\n\n Slice objects\n Slice objects are used to represent slices for "__getitem__()"\n methods. They are also created by the built-in "slice()"\n function.\n\n Special read-only attributes: "start" is the lower bound; "stop"\n is the upper bound; "step" is the step value; each is "None" if\n omitted. These attributes can have any type.\n\n Slice objects support one method:\n\n slice.indices(self, length)\n\n This method takes a single integer argument *length* and\n computes information about the slice that the slice object\n would describe if applied to a sequence of *length* items.\n It returns a tuple of three integers; respectively these are\n the *start* and *stop* indices and the *step* or stride\n length of the slice. Missing or out-of-bounds indices are\n handled in a manner consistent with regular slices.\n\n Static method objects\n Static method objects provide a way of defeating the\n transformation of function objects to method objects described\n above. A static method object is a wrapper around any other\n object, usually a user-defined method object. When a static\n method object is retrieved from a class or a class instance, the\n object actually returned is the wrapped object, which is not\n subject to any further transformation. Static method objects are\n not themselves callable, although the objects they wrap usually\n are. Static method objects are created by the built-in\n "staticmethod()" constructor.\n\n Class method objects\n A class method object, like a static method object, is a wrapper\n around another object that alters the way in which that object\n is retrieved from classes and class instances. The behaviour of\n class method objects upon such retrieval is described above,\n under "User-defined methods". Class method objects are created\n by the built-in "classmethod()" constructor.\n', - 'typesfunctions': u'\nFunctions\n*********\n\nFunction objects are created by function definitions. The only\noperation on a function object is to call it: "func(argument-list)".\n\nThere are really two flavors of function objects: built-in functions\nand user-defined functions. Both support the same operation (to call\nthe function), but the implementation is different, hence the\ndifferent object types.\n\nSee *Function definitions* for more information.\n', - 'typesmapping': u'\nMapping Types --- "dict"\n************************\n\nA *mapping* object maps *hashable* values to arbitrary objects.\nMappings are mutable objects. There is currently only one standard\nmapping type, the *dictionary*. (For other containers see the built-\nin "list", "set", and "tuple" classes, and the "collections" module.)\n\nA dictionary\'s keys are *almost* arbitrary values. Values that are\nnot *hashable*, that is, values containing lists, dictionaries or\nother mutable types (that are compared by value rather than by object\nidentity) may not be used as keys. Numeric types used for keys obey\nthe normal rules for numeric comparison: if two numbers compare equal\n(such as "1" and "1.0") then they can be used interchangeably to index\nthe same dictionary entry. (Note however, that since computers store\nfloating-point numbers as approximations it is usually unwise to use\nthem as dictionary keys.)\n\nDictionaries can be created by placing a comma-separated list of "key:\nvalue" pairs within braces, for example: "{\'jack\': 4098, \'sjoerd\':\n4127}" or "{4098: \'jack\', 4127: \'sjoerd\'}", or by the "dict"\nconstructor.\n\nclass class dict(**kwarg)\nclass class dict(mapping, **kwarg)\nclass class dict(iterable, **kwarg)\n\n Return a new dictionary initialized from an optional positional\n argument and a possibly empty set of keyword arguments.\n\n If no positional argument is given, an empty dictionary is created.\n If a positional argument is given and it is a mapping object, a\n dictionary is created with the same key-value pairs as the mapping\n object. Otherwise, the positional argument must be an *iterable*\n object. Each item in the iterable must itself be an iterable with\n exactly two objects. The first object of each item becomes a key\n in the new dictionary, and the second object the corresponding\n value. If a key occurs more than once, the last value for that key\n becomes the corresponding value in the new dictionary.\n\n If keyword arguments are given, the keyword arguments and their\n values are added to the dictionary created from the positional\n argument. If a key being added is already present, the value from\n the keyword argument replaces the value from the positional\n argument.\n\n To illustrate, the following examples all return a dictionary equal\n to "{"one": 1, "two": 2, "three": 3}":\n\n >>> a = dict(one=1, two=2, three=3)\n >>> b = {\'one\': 1, \'two\': 2, \'three\': 3}\n >>> c = dict(zip([\'one\', \'two\', \'three\'], [1, 2, 3]))\n >>> d = dict([(\'two\', 2), (\'one\', 1), (\'three\', 3)])\n >>> e = dict({\'three\': 3, \'one\': 1, \'two\': 2})\n >>> a == b == c == d == e\n True\n\n Providing keyword arguments as in the first example only works for\n keys that are valid Python identifiers. Otherwise, any valid keys\n can be used.\n\n These are the operations that dictionaries support (and therefore,\n custom mapping types should support too):\n\n len(d)\n\n Return the number of items in the dictionary *d*.\n\n d[key]\n\n Return the item of *d* with key *key*. Raises a "KeyError" if\n *key* is not in the map.\n\n If a subclass of dict defines a method "__missing__()" and *key*\n is not present, the "d[key]" operation calls that method with\n the key *key* as argument. The "d[key]" operation then returns\n or raises whatever is returned or raised by the\n "__missing__(key)" call. No other operations or methods invoke\n "__missing__()". If "__missing__()" is not defined, "KeyError"\n is raised. "__missing__()" must be a method; it cannot be an\n instance variable:\n\n >>> class Counter(dict):\n ... def __missing__(self, key):\n ... return 0\n >>> c = Counter()\n >>> c[\'red\']\n 0\n >>> c[\'red\'] += 1\n >>> c[\'red\']\n 1\n\n The example above shows part of the implementation of\n "collections.Counter". A different "__missing__" method is used\n by "collections.defaultdict".\n\n d[key] = value\n\n Set "d[key]" to *value*.\n\n del d[key]\n\n Remove "d[key]" from *d*. Raises a "KeyError" if *key* is not\n in the map.\n\n key in d\n\n Return "True" if *d* has a key *key*, else "False".\n\n key not in d\n\n Equivalent to "not key in d".\n\n iter(d)\n\n Return an iterator over the keys of the dictionary. This is a\n shortcut for "iter(d.keys())".\n\n clear()\n\n Remove all items from the dictionary.\n\n copy()\n\n Return a shallow copy of the dictionary.\n\n classmethod fromkeys(seq[, value])\n\n Create a new dictionary with keys from *seq* and values set to\n *value*.\n\n "fromkeys()" is a class method that returns a new dictionary.\n *value* defaults to "None".\n\n get(key[, default])\n\n Return the value for *key* if *key* is in the dictionary, else\n *default*. If *default* is not given, it defaults to "None", so\n that this method never raises a "KeyError".\n\n items()\n\n Return a new view of the dictionary\'s items ("(key, value)"\n pairs). See the *documentation of view objects*.\n\n keys()\n\n Return a new view of the dictionary\'s keys. See the\n *documentation of view objects*.\n\n pop(key[, default])\n\n If *key* is in the dictionary, remove it and return its value,\n else return *default*. If *default* is not given and *key* is\n not in the dictionary, a "KeyError" is raised.\n\n popitem()\n\n Remove and return an arbitrary "(key, value)" pair from the\n dictionary.\n\n "popitem()" is useful to destructively iterate over a\n dictionary, as often used in set algorithms. If the dictionary\n is empty, calling "popitem()" raises a "KeyError".\n\n setdefault(key[, default])\n\n If *key* is in the dictionary, return its value. If not, insert\n *key* with a value of *default* and return *default*. *default*\n defaults to "None".\n\n update([other])\n\n Update the dictionary with the key/value pairs from *other*,\n overwriting existing keys. Return "None".\n\n "update()" accepts either another dictionary object or an\n iterable of key/value pairs (as tuples or other iterables of\n length two). If keyword arguments are specified, the dictionary\n is then updated with those key/value pairs: "d.update(red=1,\n blue=2)".\n\n values()\n\n Return a new view of the dictionary\'s values. See the\n *documentation of view objects*.\n\n Dictionaries compare equal if and only if they have the same "(key,\n value)" pairs. Order comparisons (\'<\', \'<=\', \'>=\', \'>\') raise\n "TypeError".\n\nSee also: "types.MappingProxyType" can be used to create a read-only\n view of a "dict".\n\n\nDictionary view objects\n=======================\n\nThe objects returned by "dict.keys()", "dict.values()" and\n"dict.items()" are *view objects*. They provide a dynamic view on the\ndictionary\'s entries, which means that when the dictionary changes,\nthe view reflects these changes.\n\nDictionary views can be iterated over to yield their respective data,\nand support membership tests:\n\nlen(dictview)\n\n Return the number of entries in the dictionary.\n\niter(dictview)\n\n Return an iterator over the keys, values or items (represented as\n tuples of "(key, value)") in the dictionary.\n\n Keys and values are iterated over in an arbitrary order which is\n non-random, varies across Python implementations, and depends on\n the dictionary\'s history of insertions and deletions. If keys,\n values and items views are iterated over with no intervening\n modifications to the dictionary, the order of items will directly\n correspond. This allows the creation of "(value, key)" pairs using\n "zip()": "pairs = zip(d.values(), d.keys())". Another way to\n create the same list is "pairs = [(v, k) for (k, v) in d.items()]".\n\n Iterating views while adding or deleting entries in the dictionary\n may raise a "RuntimeError" or fail to iterate over all entries.\n\nx in dictview\n\n Return "True" if *x* is in the underlying dictionary\'s keys, values\n or items (in the latter case, *x* should be a "(key, value)"\n tuple).\n\nKeys views are set-like since their entries are unique and hashable.\nIf all values are hashable, so that "(key, value)" pairs are unique\nand hashable, then the items view is also set-like. (Values views are\nnot treated as set-like since the entries are generally not unique.)\nFor set-like views, all of the operations defined for the abstract\nbase class "collections.abc.Set" are available (for example, "==",\n"<", or "^").\n\nAn example of dictionary view usage:\n\n >>> dishes = {\'eggs\': 2, \'sausage\': 1, \'bacon\': 1, \'spam\': 500}\n >>> keys = dishes.keys()\n >>> values = dishes.values()\n\n >>> # iteration\n >>> n = 0\n >>> for val in values:\n ... n += val\n >>> print(n)\n 504\n\n >>> # keys and values are iterated over in the same order\n >>> list(keys)\n [\'eggs\', \'bacon\', \'sausage\', \'spam\']\n >>> list(values)\n [2, 1, 1, 500]\n\n >>> # view objects are dynamic and reflect dict changes\n >>> del dishes[\'eggs\']\n >>> del dishes[\'sausage\']\n >>> list(keys)\n [\'spam\', \'bacon\']\n\n >>> # set operations\n >>> keys & {\'eggs\', \'bacon\', \'salad\'}\n {\'bacon\'}\n >>> keys ^ {\'sausage\', \'juice\'}\n {\'juice\', \'sausage\', \'bacon\', \'spam\'}\n', - 'typesmethods': u'\nMethods\n*******\n\nMethods are functions that are called using the attribute notation.\nThere are two flavors: built-in methods (such as "append()" on lists)\nand class instance methods. Built-in methods are described with the\ntypes that support them.\n\nIf you access a method (a function defined in a class namespace)\nthrough an instance, you get a special object: a *bound method* (also\ncalled *instance method*) object. When called, it will add the "self"\nargument to the argument list. Bound methods have two special read-\nonly attributes: "m.__self__" is the object on which the method\noperates, and "m.__func__" is the function implementing the method.\nCalling "m(arg-1, arg-2, ..., arg-n)" is completely equivalent to\ncalling "m.__func__(m.__self__, arg-1, arg-2, ..., arg-n)".\n\nLike function objects, bound method objects support getting arbitrary\nattributes. However, since method attributes are actually stored on\nthe underlying function object ("meth.__func__"), setting method\nattributes on bound methods is disallowed. Attempting to set an\nattribute on a method results in an "AttributeError" being raised. In\norder to set a method attribute, you need to explicitly set it on the\nunderlying function object:\n\n >>> class C:\n ... def method(self):\n ... pass\n ...\n >>> c = C()\n >>> c.method.whoami = \'my name is method\' # can\'t set on the method\n Traceback (most recent call last):\n File "", line 1, in \n AttributeError: \'method\' object has no attribute \'whoami\'\n >>> c.method.__func__.whoami = \'my name is method\'\n >>> c.method.whoami\n \'my name is method\'\n\nSee *The standard type hierarchy* for more information.\n', - 'typesmodules': u'\nModules\n*******\n\nThe only special operation on a module is attribute access: "m.name",\nwhere *m* is a module and *name* accesses a name defined in *m*\'s\nsymbol table. Module attributes can be assigned to. (Note that the\n"import" statement is not, strictly speaking, an operation on a module\nobject; "import foo" does not require a module object named *foo* to\nexist, rather it requires an (external) *definition* for a module\nnamed *foo* somewhere.)\n\nA special attribute of every module is "__dict__". This is the\ndictionary containing the module\'s symbol table. Modifying this\ndictionary will actually change the module\'s symbol table, but direct\nassignment to the "__dict__" attribute is not possible (you can write\n"m.__dict__[\'a\'] = 1", which defines "m.a" to be "1", but you can\'t\nwrite "m.__dict__ = {}"). Modifying "__dict__" directly is not\nrecommended.\n\nModules built into the interpreter are written like this: "". If loaded from a file, they are written as\n"".\n', - 'typesseq': u'\nSequence Types --- "list", "tuple", "range"\n*******************************************\n\nThere are three basic sequence types: lists, tuples, and range\nobjects. Additional sequence types tailored for processing of *binary\ndata* and *text strings* are described in dedicated sections.\n\n\nCommon Sequence Operations\n==========================\n\nThe operations in the following table are supported by most sequence\ntypes, both mutable and immutable. The "collections.abc.Sequence" ABC\nis provided to make it easier to correctly implement these operations\non custom sequence types.\n\nThis table lists the sequence operations sorted in ascending priority.\nIn the table, *s* and *t* are sequences of the same type, *n*, *i*,\n*j* and *k* are integers and *x* is an arbitrary object that meets any\ntype and value restrictions imposed by *s*.\n\nThe "in" and "not in" operations have the same priorities as the\ncomparison operations. The "+" (concatenation) and "*" (repetition)\noperations have the same priority as the corresponding numeric\noperations.\n\n+----------------------------+----------------------------------+------------+\n| Operation | Result | Notes |\n+============================+==================================+============+\n| "x in s" | "True" if an item of *s* is | (1) |\n| | equal to *x*, else "False" | |\n+----------------------------+----------------------------------+------------+\n| "x not in s" | "False" if an item of *s* is | (1) |\n| | equal to *x*, else "True" | |\n+----------------------------+----------------------------------+------------+\n| "s + t" | the concatenation of *s* and *t* | (6)(7) |\n+----------------------------+----------------------------------+------------+\n| "s * n" or "n * s" | *n* shallow copies of *s* | (2)(7) |\n| | concatenated | |\n+----------------------------+----------------------------------+------------+\n| "s[i]" | *i*th item of *s*, origin 0 | (3) |\n+----------------------------+----------------------------------+------------+\n| "s[i:j]" | slice of *s* from *i* to *j* | (3)(4) |\n+----------------------------+----------------------------------+------------+\n| "s[i:j:k]" | slice of *s* from *i* to *j* | (3)(5) |\n| | with step *k* | |\n+----------------------------+----------------------------------+------------+\n| "len(s)" | length of *s* | |\n+----------------------------+----------------------------------+------------+\n| "min(s)" | smallest item of *s* | |\n+----------------------------+----------------------------------+------------+\n| "max(s)" | largest item of *s* | |\n+----------------------------+----------------------------------+------------+\n| "s.index(x[, i[, j]])" | index of the first occurrence of | (8) |\n| | *x* in *s* (at or after index | |\n| | *i* and before index *j*) | |\n+----------------------------+----------------------------------+------------+\n| "s.count(x)" | total number of occurrences of | |\n| | *x* in *s* | |\n+----------------------------+----------------------------------+------------+\n\nSequences of the same type also support comparisons. In particular,\ntuples and lists are compared lexicographically by comparing\ncorresponding elements. This means that to compare equal, every\nelement must compare equal and the two sequences must be of the same\ntype and have the same length. (For full details see *Comparisons* in\nthe language reference.)\n\nNotes:\n\n1. While the "in" and "not in" operations are used only for simple\n containment testing in the general case, some specialised sequences\n (such as "str", "bytes" and "bytearray") also use them for\n subsequence testing:\n\n >>> "gg" in "eggs"\n True\n\n2. Values of *n* less than "0" are treated as "0" (which yields an\n empty sequence of the same type as *s*). Note also that the copies\n are shallow; nested structures are not copied. This often haunts\n new Python programmers; consider:\n\n >>> lists = [[]] * 3\n >>> lists\n [[], [], []]\n >>> lists[0].append(3)\n >>> lists\n [[3], [3], [3]]\n\n What has happened is that "[[]]" is a one-element list containing\n an empty list, so all three elements of "[[]] * 3" are (pointers\n to) this single empty list. Modifying any of the elements of\n "lists" modifies this single list. You can create a list of\n different lists this way:\n\n >>> lists = [[] for i in range(3)]\n >>> lists[0].append(3)\n >>> lists[1].append(5)\n >>> lists[2].append(7)\n >>> lists\n [[3], [5], [7]]\n\n3. If *i* or *j* is negative, the index is relative to the end of\n the string: "len(s) + i" or "len(s) + j" is substituted. But note\n that "-0" is still "0".\n\n4. The slice of *s* from *i* to *j* is defined as the sequence of\n items with index *k* such that "i <= k < j". If *i* or *j* is\n greater than "len(s)", use "len(s)". If *i* is omitted or "None",\n use "0". If *j* is omitted or "None", use "len(s)". If *i* is\n greater than or equal to *j*, the slice is empty.\n\n5. The slice of *s* from *i* to *j* with step *k* is defined as the\n sequence of items with index "x = i + n*k" such that "0 <= n <\n (j-i)/k". In other words, the indices are "i", "i+k", "i+2*k",\n "i+3*k" and so on, stopping when *j* is reached (but never\n including *j*). If *i* or *j* is greater than "len(s)", use\n "len(s)". If *i* or *j* are omitted or "None", they become "end"\n values (which end depends on the sign of *k*). Note, *k* cannot be\n zero. If *k* is "None", it is treated like "1".\n\n6. Concatenating immutable sequences always results in a new\n object. This means that building up a sequence by repeated\n concatenation will have a quadratic runtime cost in the total\n sequence length. To get a linear runtime cost, you must switch to\n one of the alternatives below:\n\n * if concatenating "str" objects, you can build a list and use\n "str.join()" at the end or else write to a "io.StringIO" instance\n and retrieve its value when complete\n\n * if concatenating "bytes" objects, you can similarly use\n "bytes.join()" or "io.BytesIO", or you can do in-place\n concatenation with a "bytearray" object. "bytearray" objects are\n mutable and have an efficient overallocation mechanism\n\n * if concatenating "tuple" objects, extend a "list" instead\n\n * for other types, investigate the relevant class documentation\n\n7. Some sequence types (such as "range") only support item\n sequences that follow specific patterns, and hence don\'t support\n sequence concatenation or repetition.\n\n8. "index" raises "ValueError" when *x* is not found in *s*. When\n supported, the additional arguments to the index method allow\n efficient searching of subsections of the sequence. Passing the\n extra arguments is roughly equivalent to using "s[i:j].index(x)",\n only without copying any data and with the returned index being\n relative to the start of the sequence rather than the start of the\n slice.\n\n\nImmutable Sequence Types\n========================\n\nThe only operation that immutable sequence types generally implement\nthat is not also implemented by mutable sequence types is support for\nthe "hash()" built-in.\n\nThis support allows immutable sequences, such as "tuple" instances, to\nbe used as "dict" keys and stored in "set" and "frozenset" instances.\n\nAttempting to hash an immutable sequence that contains unhashable\nvalues will result in "TypeError".\n\n\nMutable Sequence Types\n======================\n\nThe operations in the following table are defined on mutable sequence\ntypes. The "collections.abc.MutableSequence" ABC is provided to make\nit easier to correctly implement these operations on custom sequence\ntypes.\n\nIn the table *s* is an instance of a mutable sequence type, *t* is any\niterable object and *x* is an arbitrary object that meets any type and\nvalue restrictions imposed by *s* (for example, "bytearray" only\naccepts integers that meet the value restriction "0 <= x <= 255").\n\n+--------------------------------+----------------------------------+-----------------------+\n| Operation | Result | Notes |\n+================================+==================================+=======================+\n| "s[i] = x" | item *i* of *s* is replaced by | |\n| | *x* | |\n+--------------------------------+----------------------------------+-----------------------+\n| "s[i:j] = t" | slice of *s* from *i* to *j* is | |\n| | replaced by the contents of the | |\n| | iterable *t* | |\n+--------------------------------+----------------------------------+-----------------------+\n| "del s[i:j]" | same as "s[i:j] = []" | |\n+--------------------------------+----------------------------------+-----------------------+\n| "s[i:j:k] = t" | the elements of "s[i:j:k]" are | (1) |\n| | replaced by those of *t* | |\n+--------------------------------+----------------------------------+-----------------------+\n| "del s[i:j:k]" | removes the elements of | |\n| | "s[i:j:k]" from the list | |\n+--------------------------------+----------------------------------+-----------------------+\n| "s.append(x)" | appends *x* to the end of the | |\n| | sequence (same as | |\n| | "s[len(s):len(s)] = [x]") | |\n+--------------------------------+----------------------------------+-----------------------+\n| "s.clear()" | removes all items from "s" (same | (5) |\n| | as "del s[:]") | |\n+--------------------------------+----------------------------------+-----------------------+\n| "s.copy()" | creates a shallow copy of "s" | (5) |\n| | (same as "s[:]") | |\n+--------------------------------+----------------------------------+-----------------------+\n| "s.extend(t)" | extends *s* with the contents of | |\n| | *t* (same as "s[len(s):len(s)] = | |\n| | t") | |\n+--------------------------------+----------------------------------+-----------------------+\n| "s.insert(i, x)" | inserts *x* into *s* at the | |\n| | index given by *i* (same as | |\n| | "s[i:i] = [x]") | |\n+--------------------------------+----------------------------------+-----------------------+\n| "s.pop([i])" | retrieves the item at *i* and | (2) |\n| | also removes it from *s* | |\n+--------------------------------+----------------------------------+-----------------------+\n| "s.remove(x)" | remove the first item from *s* | (3) |\n| | where "s[i] == x" | |\n+--------------------------------+----------------------------------+-----------------------+\n| "s.reverse()" | reverses the items of *s* in | (4) |\n| | place | |\n+--------------------------------+----------------------------------+-----------------------+\n\nNotes:\n\n1. *t* must have the same length as the slice it is replacing.\n\n2. The optional argument *i* defaults to "-1", so that by default\n the last item is removed and returned.\n\n3. "remove" raises "ValueError" when *x* is not found in *s*.\n\n4. The "reverse()" method modifies the sequence in place for\n economy of space when reversing a large sequence. To remind users\n that it operates by side effect, it does not return the reversed\n sequence.\n\n5. "clear()" and "copy()" are included for consistency with the\n interfaces of mutable containers that don\'t support slicing\n operations (such as "dict" and "set")\n\n New in version 3.3: "clear()" and "copy()" methods.\n\n\nLists\n=====\n\nLists are mutable sequences, typically used to store collections of\nhomogeneous items (where the precise degree of similarity will vary by\napplication).\n\nclass class list([iterable])\n\n Lists may be constructed in several ways:\n\n * Using a pair of square brackets to denote the empty list: "[]"\n\n * Using square brackets, separating items with commas: "[a]",\n "[a, b, c]"\n\n * Using a list comprehension: "[x for x in iterable]"\n\n * Using the type constructor: "list()" or "list(iterable)"\n\n The constructor builds a list whose items are the same and in the\n same order as *iterable*\'s items. *iterable* may be either a\n sequence, a container that supports iteration, or an iterator\n object. If *iterable* is already a list, a copy is made and\n returned, similar to "iterable[:]". For example, "list(\'abc\')"\n returns "[\'a\', \'b\', \'c\']" and "list( (1, 2, 3) )" returns "[1, 2,\n 3]". If no argument is given, the constructor creates a new empty\n list, "[]".\n\n Many other operations also produce lists, including the "sorted()"\n built-in.\n\n Lists implement all of the *common* and *mutable* sequence\n operations. Lists also provide the following additional method:\n\n sort(*, key=None, reverse=None)\n\n This method sorts the list in place, using only "<" comparisons\n between items. Exceptions are not suppressed - if any comparison\n operations fail, the entire sort operation will fail (and the\n list will likely be left in a partially modified state).\n\n "sort()" accepts two arguments that can only be passed by\n keyword (*keyword-only arguments*):\n\n *key* specifies a function of one argument that is used to\n extract a comparison key from each list element (for example,\n "key=str.lower"). The key corresponding to each item in the list\n is calculated once and then used for the entire sorting process.\n The default value of "None" means that list items are sorted\n directly without calculating a separate key value.\n\n The "functools.cmp_to_key()" utility is available to convert a\n 2.x style *cmp* function to a *key* function.\n\n *reverse* is a boolean value. If set to "True", then the list\n elements are sorted as if each comparison were reversed.\n\n This method modifies the sequence in place for economy of space\n when sorting a large sequence. To remind users that it operates\n by side effect, it does not return the sorted sequence (use\n "sorted()" to explicitly request a new sorted list instance).\n\n The "sort()" method is guaranteed to be stable. A sort is\n stable if it guarantees not to change the relative order of\n elements that compare equal --- this is helpful for sorting in\n multiple passes (for example, sort by department, then by salary\n grade).\n\n **CPython implementation detail:** While a list is being sorted,\n the effect of attempting to mutate, or even inspect, the list is\n undefined. The C implementation of Python makes the list appear\n empty for the duration, and raises "ValueError" if it can detect\n that the list has been mutated during a sort.\n\n\nTuples\n======\n\nTuples are immutable sequences, typically used to store collections of\nheterogeneous data (such as the 2-tuples produced by the "enumerate()"\nbuilt-in). Tuples are also used for cases where an immutable sequence\nof homogeneous data is needed (such as allowing storage in a "set" or\n"dict" instance).\n\nclass class tuple([iterable])\n\n Tuples may be constructed in a number of ways:\n\n * Using a pair of parentheses to denote the empty tuple: "()"\n\n * Using a trailing comma for a singleton tuple: "a," or "(a,)"\n\n * Separating items with commas: "a, b, c" or "(a, b, c)"\n\n * Using the "tuple()" built-in: "tuple()" or "tuple(iterable)"\n\n The constructor builds a tuple whose items are the same and in the\n same order as *iterable*\'s items. *iterable* may be either a\n sequence, a container that supports iteration, or an iterator\n object. If *iterable* is already a tuple, it is returned\n unchanged. For example, "tuple(\'abc\')" returns "(\'a\', \'b\', \'c\')"\n and "tuple( [1, 2, 3] )" returns "(1, 2, 3)". If no argument is\n given, the constructor creates a new empty tuple, "()".\n\n Note that it is actually the comma which makes a tuple, not the\n parentheses. The parentheses are optional, except in the empty\n tuple case, or when they are needed to avoid syntactic ambiguity.\n For example, "f(a, b, c)" is a function call with three arguments,\n while "f((a, b, c))" is a function call with a 3-tuple as the sole\n argument.\n\n Tuples implement all of the *common* sequence operations.\n\nFor heterogeneous collections of data where access by name is clearer\nthan access by index, "collections.namedtuple()" may be a more\nappropriate choice than a simple tuple object.\n\n\nRanges\n======\n\nThe "range" type represents an immutable sequence of numbers and is\ncommonly used for looping a specific number of times in "for" loops.\n\nclass class range(stop)\nclass class range(start, stop[, step])\n\n The arguments to the range constructor must be integers (either\n built-in "int" or any object that implements the "__index__"\n special method). If the *step* argument is omitted, it defaults to\n "1". If the *start* argument is omitted, it defaults to "0". If\n *step* is zero, "ValueError" is raised.\n\n For a positive *step*, the contents of a range "r" are determined\n by the formula "r[i] = start + step*i" where "i >= 0" and "r[i] <\n stop".\n\n For a negative *step*, the contents of the range are still\n determined by the formula "r[i] = start + step*i", but the\n constraints are "i >= 0" and "r[i] > stop".\n\n A range object will be empty if "r[0]" does not meet the value\n constraint. Ranges do support negative indices, but these are\n interpreted as indexing from the end of the sequence determined by\n the positive indices.\n\n Ranges containing absolute values larger than "sys.maxsize" are\n permitted but some features (such as "len()") may raise\n "OverflowError".\n\n Range examples:\n\n >>> list(range(10))\n [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]\n >>> list(range(1, 11))\n [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]\n >>> list(range(0, 30, 5))\n [0, 5, 10, 15, 20, 25]\n >>> list(range(0, 10, 3))\n [0, 3, 6, 9]\n >>> list(range(0, -10, -1))\n [0, -1, -2, -3, -4, -5, -6, -7, -8, -9]\n >>> list(range(0))\n []\n >>> list(range(1, 0))\n []\n\n Ranges implement all of the *common* sequence operations except\n concatenation and repetition (due to the fact that range objects\n can only represent sequences that follow a strict pattern and\n repetition and concatenation will usually violate that pattern).\n\nThe advantage of the "range" type over a regular "list" or "tuple" is\nthat a "range" object will always take the same (small) amount of\nmemory, no matter the size of the range it represents (as it only\nstores the "start", "stop" and "step" values, calculating individual\nitems and subranges as needed).\n\nRange objects implement the "collections.abc.Sequence" ABC, and\nprovide features such as containment tests, element index lookup,\nslicing and support for negative indices (see *Sequence Types ---\nlist, tuple, range*):\n\n>>> r = range(0, 20, 2)\n>>> r\nrange(0, 20, 2)\n>>> 11 in r\nFalse\n>>> 10 in r\nTrue\n>>> r.index(10)\n5\n>>> r[5]\n10\n>>> r[:5]\nrange(0, 10, 2)\n>>> r[-1]\n18\n\nTesting range objects for equality with "==" and "!=" compares them as\nsequences. That is, two range objects are considered equal if they\nrepresent the same sequence of values. (Note that two range objects\nthat compare equal might have different "start", "stop" and "step"\nattributes, for example "range(0) == range(2, 1, 3)" or "range(0, 3,\n2) == range(0, 4, 2)".)\n\nChanged in version 3.2: Implement the Sequence ABC. Support slicing\nand negative indices. Test "int" objects for membership in constant\ntime instead of iterating through all items.\n\nChanged in version 3.3: Define \'==\' and \'!=\' to compare range objects\nbased on the sequence of values they define (instead of comparing\nbased on object identity).\n\nNew in version 3.3: The "start", "stop" and "step" attributes.\n', - 'typesseq-mutable': u'\nMutable Sequence Types\n**********************\n\nThe operations in the following table are defined on mutable sequence\ntypes. The "collections.abc.MutableSequence" ABC is provided to make\nit easier to correctly implement these operations on custom sequence\ntypes.\n\nIn the table *s* is an instance of a mutable sequence type, *t* is any\niterable object and *x* is an arbitrary object that meets any type and\nvalue restrictions imposed by *s* (for example, "bytearray" only\naccepts integers that meet the value restriction "0 <= x <= 255").\n\n+--------------------------------+----------------------------------+-----------------------+\n| Operation | Result | Notes |\n+================================+==================================+=======================+\n| "s[i] = x" | item *i* of *s* is replaced by | |\n| | *x* | |\n+--------------------------------+----------------------------------+-----------------------+\n| "s[i:j] = t" | slice of *s* from *i* to *j* is | |\n| | replaced by the contents of the | |\n| | iterable *t* | |\n+--------------------------------+----------------------------------+-----------------------+\n| "del s[i:j]" | same as "s[i:j] = []" | |\n+--------------------------------+----------------------------------+-----------------------+\n| "s[i:j:k] = t" | the elements of "s[i:j:k]" are | (1) |\n| | replaced by those of *t* | |\n+--------------------------------+----------------------------------+-----------------------+\n| "del s[i:j:k]" | removes the elements of | |\n| | "s[i:j:k]" from the list | |\n+--------------------------------+----------------------------------+-----------------------+\n| "s.append(x)" | appends *x* to the end of the | |\n| | sequence (same as | |\n| | "s[len(s):len(s)] = [x]") | |\n+--------------------------------+----------------------------------+-----------------------+\n| "s.clear()" | removes all items from "s" (same | (5) |\n| | as "del s[:]") | |\n+--------------------------------+----------------------------------+-----------------------+\n| "s.copy()" | creates a shallow copy of "s" | (5) |\n| | (same as "s[:]") | |\n+--------------------------------+----------------------------------+-----------------------+\n| "s.extend(t)" | extends *s* with the contents of | |\n| | *t* (same as "s[len(s):len(s)] = | |\n| | t") | |\n+--------------------------------+----------------------------------+-----------------------+\n| "s.insert(i, x)" | inserts *x* into *s* at the | |\n| | index given by *i* (same as | |\n| | "s[i:i] = [x]") | |\n+--------------------------------+----------------------------------+-----------------------+\n| "s.pop([i])" | retrieves the item at *i* and | (2) |\n| | also removes it from *s* | |\n+--------------------------------+----------------------------------+-----------------------+\n| "s.remove(x)" | remove the first item from *s* | (3) |\n| | where "s[i] == x" | |\n+--------------------------------+----------------------------------+-----------------------+\n| "s.reverse()" | reverses the items of *s* in | (4) |\n| | place | |\n+--------------------------------+----------------------------------+-----------------------+\n\nNotes:\n\n1. *t* must have the same length as the slice it is replacing.\n\n2. The optional argument *i* defaults to "-1", so that by default\n the last item is removed and returned.\n\n3. "remove" raises "ValueError" when *x* is not found in *s*.\n\n4. The "reverse()" method modifies the sequence in place for\n economy of space when reversing a large sequence. To remind users\n that it operates by side effect, it does not return the reversed\n sequence.\n\n5. "clear()" and "copy()" are included for consistency with the\n interfaces of mutable containers that don\'t support slicing\n operations (such as "dict" and "set")\n\n New in version 3.3: "clear()" and "copy()" methods.\n', - 'unary': u'\nUnary arithmetic and bitwise operations\n***************************************\n\nAll unary arithmetic and bitwise operations have the same priority:\n\n u_expr ::= power | "-" u_expr | "+" u_expr | "~" u_expr\n\nThe unary "-" (minus) operator yields the negation of its numeric\nargument.\n\nThe unary "+" (plus) operator yields its numeric argument unchanged.\n\nThe unary "~" (invert) operator yields the bitwise inversion of its\ninteger argument. The bitwise inversion of "x" is defined as\n"-(x+1)". It only applies to integral numbers.\n\nIn all three cases, if the argument does not have the proper type, a\n"TypeError" exception is raised.\n', - 'while': u'\nThe "while" statement\n*********************\n\nThe "while" statement is used for repeated execution as long as an\nexpression is true:\n\n while_stmt ::= "while" expression ":" suite\n ["else" ":" suite]\n\nThis repeatedly tests the expression and, if it is true, executes the\nfirst suite; if the expression is false (which may be the first time\nit is tested) the suite of the "else" clause, if present, is executed\nand the loop terminates.\n\nA "break" statement executed in the first suite terminates the loop\nwithout executing the "else" clause\'s suite. A "continue" statement\nexecuted in the first suite skips the rest of the suite and goes back\nto testing the expression.\n', - 'with': u'\nThe "with" statement\n********************\n\nThe "with" statement is used to wrap the execution of a block with\nmethods defined by a context manager (see section *With Statement\nContext Managers*). This allows common "try"..."except"..."finally"\nusage patterns to be encapsulated for convenient reuse.\n\n with_stmt ::= "with" with_item ("," with_item)* ":" suite\n with_item ::= expression ["as" target]\n\nThe execution of the "with" statement with one "item" proceeds as\nfollows:\n\n1. The context expression (the expression given in the "with_item")\n is evaluated to obtain a context manager.\n\n2. The context manager\'s "__exit__()" is loaded for later use.\n\n3. The context manager\'s "__enter__()" method is invoked.\n\n4. If a target was included in the "with" statement, the return\n value from "__enter__()" is assigned to it.\n\n Note: The "with" statement guarantees that if the "__enter__()"\n method returns without an error, then "__exit__()" will always be\n called. Thus, if an error occurs during the assignment to the\n target list, it will be treated the same as an error occurring\n within the suite would be. See step 6 below.\n\n5. The suite is executed.\n\n6. The context manager\'s "__exit__()" method is invoked. If an\n exception caused the suite to be exited, its type, value, and\n traceback are passed as arguments to "__exit__()". Otherwise, three\n "None" arguments are supplied.\n\n If the suite was exited due to an exception, and the return value\n from the "__exit__()" method was false, the exception is reraised.\n If the return value was true, the exception is suppressed, and\n execution continues with the statement following the "with"\n statement.\n\n If the suite was exited for any reason other than an exception, the\n return value from "__exit__()" is ignored, and execution proceeds\n at the normal location for the kind of exit that was taken.\n\nWith more than one item, the context managers are processed as if\nmultiple "with" statements were nested:\n\n with A() as a, B() as b:\n suite\n\nis equivalent to\n\n with A() as a:\n with B() as b:\n suite\n\nChanged in version 3.1: Support for multiple context expressions.\n\nSee also: **PEP 0343** - The "with" statement\n\n The specification, background, and examples for the Python "with"\n statement.\n', - 'yield': u'\nThe "yield" statement\n*********************\n\n yield_stmt ::= yield_expression\n\nA "yield" statement is semantically equivalent to a *yield\nexpression*. The yield statement can be used to omit the parentheses\nthat would otherwise be required in the equivalent yield expression\nstatement. For example, the yield statements\n\n yield \n yield from \n\nare equivalent to the yield expression statements\n\n (yield )\n (yield from )\n\nYield expressions and statements are only used when defining a\n*generator* function, and are only used in the body of the generator\nfunction. Using yield in a function definition is sufficient to cause\nthat definition to create a generator function instead of a normal\nfunction.\n\nFor full details of "yield" semantics, refer to the *Yield\nexpressions* section.\n'} +# Autogenerated by Sphinx on Sat Sep 12 17:22:24 2015 +topics = {'assert': '\n' + 'The "assert" statement\n' + '**********************\n' + '\n' + 'Assert statements are a convenient way to insert debugging ' + 'assertions\n' + 'into a program:\n' + '\n' + ' assert_stmt ::= "assert" expression ["," expression]\n' + '\n' + 'The simple form, "assert expression", is equivalent to\n' + '\n' + ' if __debug__:\n' + ' if not expression: raise AssertionError\n' + '\n' + 'The extended form, "assert expression1, expression2", is ' + 'equivalent to\n' + '\n' + ' if __debug__:\n' + ' if not expression1: raise AssertionError(expression2)\n' + '\n' + 'These equivalences assume that "__debug__" and "AssertionError" ' + 'refer\n' + 'to the built-in variables with those names. In the current\n' + 'implementation, the built-in variable "__debug__" is "True" ' + 'under\n' + 'normal circumstances, "False" when optimization is requested ' + '(command\n' + 'line option -O). The current code generator emits no code for ' + 'an\n' + 'assert statement when optimization is requested at compile ' + 'time. Note\n' + 'that it is unnecessary to include the source code for the ' + 'expression\n' + 'that failed in the error message; it will be displayed as part ' + 'of the\n' + 'stack trace.\n' + '\n' + 'Assignments to "__debug__" are illegal. The value for the ' + 'built-in\n' + 'variable is determined when the interpreter starts.\n', + 'assignment': '\n' + 'Assignment statements\n' + '*********************\n' + '\n' + 'Assignment statements are used to (re)bind names to values ' + 'and to\n' + 'modify attributes or items of mutable objects:\n' + '\n' + ' assignment_stmt ::= (target_list "=")+ (expression_list | ' + 'yield_expression)\n' + ' target_list ::= target ("," target)* [","]\n' + ' target ::= identifier\n' + ' | "(" target_list ")"\n' + ' | "[" target_list "]"\n' + ' | attributeref\n' + ' | subscription\n' + ' | slicing\n' + ' | "*" target\n' + '\n' + '(See section *Primaries* for the syntax definitions for\n' + '*attributeref*, *subscription*, and *slicing*.)\n' + '\n' + 'An assignment statement evaluates the expression list ' + '(remember that\n' + 'this can be a single expression or a comma-separated list, ' + 'the latter\n' + 'yielding a tuple) and assigns the single resulting object to ' + 'each of\n' + 'the target lists, from left to right.\n' + '\n' + 'Assignment is defined recursively depending on the form of ' + 'the target\n' + '(list). When a target is part of a mutable object (an ' + 'attribute\n' + 'reference, subscription or slicing), the mutable object ' + 'must\n' + 'ultimately perform the assignment and decide about its ' + 'validity, and\n' + 'may raise an exception if the assignment is unacceptable. ' + 'The rules\n' + 'observed by various types and the exceptions raised are ' + 'given with the\n' + 'definition of the object types (see section *The standard ' + 'type\n' + 'hierarchy*).\n' + '\n' + 'Assignment of an object to a target list, optionally ' + 'enclosed in\n' + 'parentheses or square brackets, is recursively defined as ' + 'follows.\n' + '\n' + '* If the target list is a single target: The object is ' + 'assigned to\n' + ' that target.\n' + '\n' + '* If the target list is a comma-separated list of targets: ' + 'The\n' + ' object must be an iterable with the same number of items ' + 'as there\n' + ' are targets in the target list, and the items are ' + 'assigned, from\n' + ' left to right, to the corresponding targets.\n' + '\n' + ' * If the target list contains one target prefixed with an\n' + ' asterisk, called a "starred" target: The object must be ' + 'a sequence\n' + ' with at least as many items as there are targets in the ' + 'target\n' + ' list, minus one. The first items of the sequence are ' + 'assigned,\n' + ' from left to right, to the targets before the starred ' + 'target. The\n' + ' final items of the sequence are assigned to the targets ' + 'after the\n' + ' starred target. A list of the remaining items in the ' + 'sequence is\n' + ' then assigned to the starred target (the list can be ' + 'empty).\n' + '\n' + ' * Else: The object must be a sequence with the same number ' + 'of\n' + ' items as there are targets in the target list, and the ' + 'items are\n' + ' assigned, from left to right, to the corresponding ' + 'targets.\n' + '\n' + 'Assignment of an object to a single target is recursively ' + 'defined as\n' + 'follows.\n' + '\n' + '* If the target is an identifier (name):\n' + '\n' + ' * If the name does not occur in a "global" or "nonlocal" ' + 'statement\n' + ' in the current code block: the name is bound to the ' + 'object in the\n' + ' current local namespace.\n' + '\n' + ' * Otherwise: the name is bound to the object in the ' + 'global\n' + ' namespace or the outer namespace determined by ' + '"nonlocal",\n' + ' respectively.\n' + '\n' + ' The name is rebound if it was already bound. This may ' + 'cause the\n' + ' reference count for the object previously bound to the ' + 'name to reach\n' + ' zero, causing the object to be deallocated and its ' + 'destructor (if it\n' + ' has one) to be called.\n' + '\n' + '* If the target is a target list enclosed in parentheses or ' + 'in\n' + ' square brackets: The object must be an iterable with the ' + 'same number\n' + ' of items as there are targets in the target list, and its ' + 'items are\n' + ' assigned, from left to right, to the corresponding ' + 'targets.\n' + '\n' + '* If the target is an attribute reference: The primary ' + 'expression in\n' + ' the reference is evaluated. It should yield an object ' + 'with\n' + ' assignable attributes; if this is not the case, ' + '"TypeError" is\n' + ' raised. That object is then asked to assign the assigned ' + 'object to\n' + ' the given attribute; if it cannot perform the assignment, ' + 'it raises\n' + ' an exception (usually but not necessarily ' + '"AttributeError").\n' + '\n' + ' Note: If the object is a class instance and the attribute ' + 'reference\n' + ' occurs on both sides of the assignment operator, the RHS ' + 'expression,\n' + ' "a.x" can access either an instance attribute or (if no ' + 'instance\n' + ' attribute exists) a class attribute. The LHS target "a.x" ' + 'is always\n' + ' set as an instance attribute, creating it if necessary. ' + 'Thus, the\n' + ' two occurrences of "a.x" do not necessarily refer to the ' + 'same\n' + ' attribute: if the RHS expression refers to a class ' + 'attribute, the\n' + ' LHS creates a new instance attribute as the target of the\n' + ' assignment:\n' + '\n' + ' class Cls:\n' + ' x = 3 # class variable\n' + ' inst = Cls()\n' + ' inst.x = inst.x + 1 # writes inst.x as 4 leaving ' + 'Cls.x as 3\n' + '\n' + ' This description does not necessarily apply to descriptor\n' + ' attributes, such as properties created with "property()".\n' + '\n' + '* If the target is a subscription: The primary expression in ' + 'the\n' + ' reference is evaluated. It should yield either a mutable ' + 'sequence\n' + ' object (such as a list) or a mapping object (such as a ' + 'dictionary).\n' + ' Next, the subscript expression is evaluated.\n' + '\n' + ' If the primary is a mutable sequence object (such as a ' + 'list), the\n' + ' subscript must yield an integer. If it is negative, the ' + "sequence's\n" + ' length is added to it. The resulting value must be a ' + 'nonnegative\n' + " integer less than the sequence's length, and the sequence " + 'is asked\n' + ' to assign the assigned object to its item with that ' + 'index. If the\n' + ' index is out of range, "IndexError" is raised (assignment ' + 'to a\n' + ' subscripted sequence cannot add new items to a list).\n' + '\n' + ' If the primary is a mapping object (such as a dictionary), ' + 'the\n' + " subscript must have a type compatible with the mapping's " + 'key type,\n' + ' and the mapping is then asked to create a key/datum pair ' + 'which maps\n' + ' the subscript to the assigned object. This can either ' + 'replace an\n' + ' existing key/value pair with the same key value, or insert ' + 'a new\n' + ' key/value pair (if no key with the same value existed).\n' + '\n' + ' For user-defined objects, the "__setitem__()" method is ' + 'called with\n' + ' appropriate arguments.\n' + '\n' + '* If the target is a slicing: The primary expression in the\n' + ' reference is evaluated. It should yield a mutable ' + 'sequence object\n' + ' (such as a list). The assigned object should be a ' + 'sequence object\n' + ' of the same type. Next, the lower and upper bound ' + 'expressions are\n' + ' evaluated, insofar they are present; defaults are zero and ' + 'the\n' + " sequence's length. The bounds should evaluate to " + 'integers. If\n' + " either bound is negative, the sequence's length is added " + 'to it. The\n' + ' resulting bounds are clipped to lie between zero and the ' + "sequence's\n" + ' length, inclusive. Finally, the sequence object is asked ' + 'to replace\n' + ' the slice with the items of the assigned sequence. The ' + 'length of\n' + ' the slice may be different from the length of the assigned ' + 'sequence,\n' + ' thus changing the length of the target sequence, if the ' + 'target\n' + ' sequence allows it.\n' + '\n' + '**CPython implementation detail:** In the current ' + 'implementation, the\n' + 'syntax for targets is taken to be the same as for ' + 'expressions, and\n' + 'invalid syntax is rejected during the code generation phase, ' + 'causing\n' + 'less detailed error messages.\n' + '\n' + 'Although the definition of assignment implies that overlaps ' + 'between\n' + 'the left-hand side and the right-hand side are ' + "'simultanenous' (for\n" + 'example "a, b = b, a" swaps two variables), overlaps ' + '*within* the\n' + 'collection of assigned-to variables occur left-to-right, ' + 'sometimes\n' + 'resulting in confusion. For instance, the following program ' + 'prints\n' + '"[0, 2]":\n' + '\n' + ' x = [0, 1]\n' + ' i = 0\n' + ' i, x[i] = 1, 2 # i is updated, then x[i] is ' + 'updated\n' + ' print(x)\n' + '\n' + 'See also: **PEP 3132** - Extended Iterable Unpacking\n' + '\n' + ' The specification for the "*target" feature.\n' + '\n' + '\n' + 'Augmented assignment statements\n' + '===============================\n' + '\n' + 'Augmented assignment is the combination, in a single ' + 'statement, of a\n' + 'binary operation and an assignment statement:\n' + '\n' + ' augmented_assignment_stmt ::= augtarget augop ' + '(expression_list | yield_expression)\n' + ' augtarget ::= identifier | attributeref | ' + 'subscription | slicing\n' + ' augop ::= "+=" | "-=" | "*=" | "@=" | ' + '"/=" | "//=" | "%=" | "**="\n' + ' | ">>=" | "<<=" | "&=" | "^=" | "|="\n' + '\n' + '(See section *Primaries* for the syntax definitions of the ' + 'last three\n' + 'symbols.)\n' + '\n' + 'An augmented assignment evaluates the target (which, unlike ' + 'normal\n' + 'assignment statements, cannot be an unpacking) and the ' + 'expression\n' + 'list, performs the binary operation specific to the type of ' + 'assignment\n' + 'on the two operands, and assigns the result to the original ' + 'target.\n' + 'The target is only evaluated once.\n' + '\n' + 'An augmented assignment expression like "x += 1" can be ' + 'rewritten as\n' + '"x = x + 1" to achieve a similar, but not exactly equal ' + 'effect. In the\n' + 'augmented version, "x" is only evaluated once. Also, when ' + 'possible,\n' + 'the actual operation is performed *in-place*, meaning that ' + 'rather than\n' + 'creating a new object and assigning that to the target, the ' + 'old object\n' + 'is modified instead.\n' + '\n' + 'Unlike normal assignments, augmented assignments evaluate ' + 'the left-\n' + 'hand side *before* evaluating the right-hand side. For ' + 'example, "a[i]\n' + '+= f(x)" first looks-up "a[i]", then it evaluates "f(x)" and ' + 'performs\n' + 'the addition, and lastly, it writes the result back to ' + '"a[i]".\n' + '\n' + 'With the exception of assigning to tuples and multiple ' + 'targets in a\n' + 'single statement, the assignment done by augmented ' + 'assignment\n' + 'statements is handled the same way as normal assignments. ' + 'Similarly,\n' + 'with the exception of the possible *in-place* behavior, the ' + 'binary\n' + 'operation performed by augmented assignment is the same as ' + 'the normal\n' + 'binary operations.\n' + '\n' + 'For targets which are attribute references, the same *caveat ' + 'about\n' + 'class and instance attributes* applies as for regular ' + 'assignments.\n', + 'atom-identifiers': '\n' + 'Identifiers (Names)\n' + '*******************\n' + '\n' + 'An identifier occurring as an atom is a name. See ' + 'section\n' + '*Identifiers and keywords* for lexical definition and ' + 'section *Naming\n' + 'and binding* for documentation of naming and binding.\n' + '\n' + 'When the name is bound to an object, evaluation of the ' + 'atom yields\n' + 'that object. When a name is not bound, an attempt to ' + 'evaluate it\n' + 'raises a "NameError" exception.\n' + '\n' + '**Private name mangling:** When an identifier that ' + 'textually occurs in\n' + 'a class definition begins with two or more underscore ' + 'characters and\n' + 'does not end in two or more underscores, it is ' + 'considered a *private\n' + 'name* of that class. Private names are transformed to ' + 'a longer form\n' + 'before code is generated for them. The transformation ' + 'inserts the\n' + 'class name, with leading underscores removed and a ' + 'single underscore\n' + 'inserted, in front of the name. For example, the ' + 'identifier "__spam"\n' + 'occurring in a class named "Ham" will be transformed ' + 'to "_Ham__spam".\n' + 'This transformation is independent of the syntactical ' + 'context in which\n' + 'the identifier is used. If the transformed name is ' + 'extremely long\n' + '(longer than 255 characters), implementation defined ' + 'truncation may\n' + 'happen. If the class name consists only of ' + 'underscores, no\n' + 'transformation is done.\n', + 'atom-literals': '\n' + 'Literals\n' + '********\n' + '\n' + 'Python supports string and bytes literals and various ' + 'numeric\n' + 'literals:\n' + '\n' + ' literal ::= stringliteral | bytesliteral\n' + ' | integer | floatnumber | imagnumber\n' + '\n' + 'Evaluation of a literal yields an object of the given ' + 'type (string,\n' + 'bytes, integer, floating point number, complex number) ' + 'with the given\n' + 'value. The value may be approximated in the case of ' + 'floating point\n' + 'and imaginary (complex) literals. See section *Literals* ' + 'for details.\n' + '\n' + 'All literals correspond to immutable data types, and ' + 'hence the\n' + "object's identity is less important than its value. " + 'Multiple\n' + 'evaluations of literals with the same value (either the ' + 'same\n' + 'occurrence in the program text or a different occurrence) ' + 'may obtain\n' + 'the same object or a different object with the same ' + 'value.\n', + 'attribute-access': '\n' + 'Customizing attribute access\n' + '****************************\n' + '\n' + 'The following methods can be defined to customize the ' + 'meaning of\n' + 'attribute access (use of, assignment to, or deletion ' + 'of "x.name") for\n' + 'class instances.\n' + '\n' + 'object.__getattr__(self, name)\n' + '\n' + ' Called when an attribute lookup has not found the ' + 'attribute in the\n' + ' usual places (i.e. it is not an instance attribute ' + 'nor is it found\n' + ' in the class tree for "self"). "name" is the ' + 'attribute name. This\n' + ' method should return the (computed) attribute value ' + 'or raise an\n' + ' "AttributeError" exception.\n' + '\n' + ' Note that if the attribute is found through the ' + 'normal mechanism,\n' + ' "__getattr__()" is not called. (This is an ' + 'intentional asymmetry\n' + ' between "__getattr__()" and "__setattr__()".) This ' + 'is done both for\n' + ' efficiency reasons and because otherwise ' + '"__getattr__()" would have\n' + ' no way to access other attributes of the instance. ' + 'Note that at\n' + ' least for instance variables, you can fake total ' + 'control by not\n' + ' inserting any values in the instance attribute ' + 'dictionary (but\n' + ' instead inserting them in another object). See ' + 'the\n' + ' "__getattribute__()" method below for a way to ' + 'actually get total\n' + ' control over attribute access.\n' + '\n' + 'object.__getattribute__(self, name)\n' + '\n' + ' Called unconditionally to implement attribute ' + 'accesses for\n' + ' instances of the class. If the class also defines ' + '"__getattr__()",\n' + ' the latter will not be called unless ' + '"__getattribute__()" either\n' + ' calls it explicitly or raises an "AttributeError". ' + 'This method\n' + ' should return the (computed) attribute value or ' + 'raise an\n' + ' "AttributeError" exception. In order to avoid ' + 'infinite recursion in\n' + ' this method, its implementation should always call ' + 'the base class\n' + ' method with the same name to access any attributes ' + 'it needs, for\n' + ' example, "object.__getattribute__(self, name)".\n' + '\n' + ' Note: This method may still be bypassed when ' + 'looking up special\n' + ' methods as the result of implicit invocation via ' + 'language syntax\n' + ' or built-in functions. See *Special method ' + 'lookup*.\n' + '\n' + 'object.__setattr__(self, name, value)\n' + '\n' + ' Called when an attribute assignment is attempted. ' + 'This is called\n' + ' instead of the normal mechanism (i.e. store the ' + 'value in the\n' + ' instance dictionary). *name* is the attribute name, ' + '*value* is the\n' + ' value to be assigned to it.\n' + '\n' + ' If "__setattr__()" wants to assign to an instance ' + 'attribute, it\n' + ' should call the base class method with the same ' + 'name, for example,\n' + ' "object.__setattr__(self, name, value)".\n' + '\n' + 'object.__delattr__(self, name)\n' + '\n' + ' Like "__setattr__()" but for attribute deletion ' + 'instead of\n' + ' assignment. This should only be implemented if ' + '"del obj.name" is\n' + ' meaningful for the object.\n' + '\n' + 'object.__dir__(self)\n' + '\n' + ' Called when "dir()" is called on the object. A ' + 'sequence must be\n' + ' returned. "dir()" converts the returned sequence to ' + 'a list and\n' + ' sorts it.\n' + '\n' + '\n' + 'Implementing Descriptors\n' + '========================\n' + '\n' + 'The following methods only apply when an instance of ' + 'the class\n' + 'containing the method (a so-called *descriptor* class) ' + 'appears in an\n' + '*owner* class (the descriptor must be in either the ' + "owner's class\n" + 'dictionary or in the class dictionary for one of its ' + 'parents). In the\n' + 'examples below, "the attribute" refers to the ' + 'attribute whose name is\n' + "the key of the property in the owner class' " + '"__dict__".\n' + '\n' + 'object.__get__(self, instance, owner)\n' + '\n' + ' Called to get the attribute of the owner class ' + '(class attribute\n' + ' access) or of an instance of that class (instance ' + 'attribute\n' + ' access). *owner* is always the owner class, while ' + '*instance* is the\n' + ' instance that the attribute was accessed through, ' + 'or "None" when\n' + ' the attribute is accessed through the *owner*. ' + 'This method should\n' + ' return the (computed) attribute value or raise an ' + '"AttributeError"\n' + ' exception.\n' + '\n' + 'object.__set__(self, instance, value)\n' + '\n' + ' Called to set the attribute on an instance ' + '*instance* of the owner\n' + ' class to a new value, *value*.\n' + '\n' + 'object.__delete__(self, instance)\n' + '\n' + ' Called to delete the attribute on an instance ' + '*instance* of the\n' + ' owner class.\n' + '\n' + 'The attribute "__objclass__" is interpreted by the ' + '"inspect" module as\n' + 'specifying the class where this object was defined ' + '(setting this\n' + 'appropriately can assist in runtime introspection of ' + 'dynamic class\n' + 'attributes). For callables, it may indicate that an ' + 'instance of the\n' + 'given type (or a subclass) is expected or required as ' + 'the first\n' + 'positional argument (for example, CPython sets this ' + 'attribute for\n' + 'unbound methods that are implemented in C).\n' + '\n' + '\n' + 'Invoking Descriptors\n' + '====================\n' + '\n' + 'In general, a descriptor is an object attribute with ' + '"binding\n' + 'behavior", one whose attribute access has been ' + 'overridden by methods\n' + 'in the descriptor protocol: "__get__()", "__set__()", ' + 'and\n' + '"__delete__()". If any of those methods are defined ' + 'for an object, it\n' + 'is said to be a descriptor.\n' + '\n' + 'The default behavior for attribute access is to get, ' + 'set, or delete\n' + "the attribute from an object's dictionary. For " + 'instance, "a.x" has a\n' + 'lookup chain starting with "a.__dict__[\'x\']", then\n' + '"type(a).__dict__[\'x\']", and continuing through the ' + 'base classes of\n' + '"type(a)" excluding metaclasses.\n' + '\n' + 'However, if the looked-up value is an object defining ' + 'one of the\n' + 'descriptor methods, then Python may override the ' + 'default behavior and\n' + 'invoke the descriptor method instead. Where this ' + 'occurs in the\n' + 'precedence chain depends on which descriptor methods ' + 'were defined and\n' + 'how they were called.\n' + '\n' + 'The starting point for descriptor invocation is a ' + 'binding, "a.x". How\n' + 'the arguments are assembled depends on "a":\n' + '\n' + 'Direct Call\n' + ' The simplest and least common call is when user ' + 'code directly\n' + ' invokes a descriptor method: "x.__get__(a)".\n' + '\n' + 'Instance Binding\n' + ' If binding to an object instance, "a.x" is ' + 'transformed into the\n' + ' call: "type(a).__dict__[\'x\'].__get__(a, ' + 'type(a))".\n' + '\n' + 'Class Binding\n' + ' If binding to a class, "A.x" is transformed into ' + 'the call:\n' + ' "A.__dict__[\'x\'].__get__(None, A)".\n' + '\n' + 'Super Binding\n' + ' If "a" is an instance of "super", then the binding ' + '"super(B,\n' + ' obj).m()" searches "obj.__class__.__mro__" for the ' + 'base class "A"\n' + ' immediately preceding "B" and then invokes the ' + 'descriptor with the\n' + ' call: "A.__dict__[\'m\'].__get__(obj, ' + 'obj.__class__)".\n' + '\n' + 'For instance bindings, the precedence of descriptor ' + 'invocation depends\n' + 'on the which descriptor methods are defined. A ' + 'descriptor can define\n' + 'any combination of "__get__()", "__set__()" and ' + '"__delete__()". If it\n' + 'does not define "__get__()", then accessing the ' + 'attribute will return\n' + 'the descriptor object itself unless there is a value ' + "in the object's\n" + 'instance dictionary. If the descriptor defines ' + '"__set__()" and/or\n' + '"__delete__()", it is a data descriptor; if it defines ' + 'neither, it is\n' + 'a non-data descriptor. Normally, data descriptors ' + 'define both\n' + '"__get__()" and "__set__()", while non-data ' + 'descriptors have just the\n' + '"__get__()" method. Data descriptors with "__set__()" ' + 'and "__get__()"\n' + 'defined always override a redefinition in an instance ' + 'dictionary. In\n' + 'contrast, non-data descriptors can be overridden by ' + 'instances.\n' + '\n' + 'Python methods (including "staticmethod()" and ' + '"classmethod()") are\n' + 'implemented as non-data descriptors. Accordingly, ' + 'instances can\n' + 'redefine and override methods. This allows individual ' + 'instances to\n' + 'acquire behaviors that differ from other instances of ' + 'the same class.\n' + '\n' + 'The "property()" function is implemented as a data ' + 'descriptor.\n' + 'Accordingly, instances cannot override the behavior of ' + 'a property.\n' + '\n' + '\n' + '__slots__\n' + '=========\n' + '\n' + 'By default, instances of classes have a dictionary for ' + 'attribute\n' + 'storage. This wastes space for objects having very ' + 'few instance\n' + 'variables. The space consumption can become acute ' + 'when creating large\n' + 'numbers of instances.\n' + '\n' + 'The default can be overridden by defining *__slots__* ' + 'in a class\n' + 'definition. The *__slots__* declaration takes a ' + 'sequence of instance\n' + 'variables and reserves just enough space in each ' + 'instance to hold a\n' + 'value for each variable. Space is saved because ' + '*__dict__* is not\n' + 'created for each instance.\n' + '\n' + 'object.__slots__\n' + '\n' + ' This class variable can be assigned a string, ' + 'iterable, or sequence\n' + ' of strings with variable names used by instances. ' + '*__slots__*\n' + ' reserves space for the declared variables and ' + 'prevents the\n' + ' automatic creation of *__dict__* and *__weakref__* ' + 'for each\n' + ' instance.\n' + '\n' + '\n' + 'Notes on using *__slots__*\n' + '--------------------------\n' + '\n' + '* When inheriting from a class without *__slots__*, ' + 'the *__dict__*\n' + ' attribute of that class will always be accessible, ' + 'so a *__slots__*\n' + ' definition in the subclass is meaningless.\n' + '\n' + '* Without a *__dict__* variable, instances cannot be ' + 'assigned new\n' + ' variables not listed in the *__slots__* definition. ' + 'Attempts to\n' + ' assign to an unlisted variable name raises ' + '"AttributeError". If\n' + ' dynamic assignment of new variables is desired, then ' + 'add\n' + ' "\'__dict__\'" to the sequence of strings in the ' + '*__slots__*\n' + ' declaration.\n' + '\n' + '* Without a *__weakref__* variable for each instance, ' + 'classes\n' + ' defining *__slots__* do not support weak references ' + 'to its\n' + ' instances. If weak reference support is needed, then ' + 'add\n' + ' "\'__weakref__\'" to the sequence of strings in the ' + '*__slots__*\n' + ' declaration.\n' + '\n' + '* *__slots__* are implemented at the class level by ' + 'creating\n' + ' descriptors (*Implementing Descriptors*) for each ' + 'variable name. As\n' + ' a result, class attributes cannot be used to set ' + 'default values for\n' + ' instance variables defined by *__slots__*; ' + 'otherwise, the class\n' + ' attribute would overwrite the descriptor ' + 'assignment.\n' + '\n' + '* The action of a *__slots__* declaration is limited ' + 'to the class\n' + ' where it is defined. As a result, subclasses will ' + 'have a *__dict__*\n' + ' unless they also define *__slots__* (which must only ' + 'contain names\n' + ' of any *additional* slots).\n' + '\n' + '* If a class defines a slot also defined in a base ' + 'class, the\n' + ' instance variable defined by the base class slot is ' + 'inaccessible\n' + ' (except by retrieving its descriptor directly from ' + 'the base class).\n' + ' This renders the meaning of the program undefined. ' + 'In the future, a\n' + ' check may be added to prevent this.\n' + '\n' + '* Nonempty *__slots__* does not work for classes ' + 'derived from\n' + ' "variable-length" built-in types such as "int", ' + '"bytes" and "tuple".\n' + '\n' + '* Any non-string iterable may be assigned to ' + '*__slots__*. Mappings\n' + ' may also be used; however, in the future, special ' + 'meaning may be\n' + ' assigned to the values corresponding to each key.\n' + '\n' + '* *__class__* assignment works only if both classes ' + 'have the same\n' + ' *__slots__*.\n', + 'attribute-references': '\n' + 'Attribute references\n' + '********************\n' + '\n' + 'An attribute reference is a primary followed by a ' + 'period and a name:\n' + '\n' + ' attributeref ::= primary "." identifier\n' + '\n' + 'The primary must evaluate to an object of a type ' + 'that supports\n' + 'attribute references, which most objects do. This ' + 'object is then\n' + 'asked to produce the attribute whose name is the ' + 'identifier. This\n' + 'production can be customized by overriding the ' + '"__getattr__()" method.\n' + 'If this attribute is not available, the exception ' + '"AttributeError" is\n' + 'raised. Otherwise, the type and value of the ' + 'object produced is\n' + 'determined by the object. Multiple evaluations of ' + 'the same attribute\n' + 'reference may yield different objects.\n', + 'augassign': '\n' + 'Augmented assignment statements\n' + '*******************************\n' + '\n' + 'Augmented assignment is the combination, in a single ' + 'statement, of a\n' + 'binary operation and an assignment statement:\n' + '\n' + ' augmented_assignment_stmt ::= augtarget augop ' + '(expression_list | yield_expression)\n' + ' augtarget ::= identifier | attributeref | ' + 'subscription | slicing\n' + ' augop ::= "+=" | "-=" | "*=" | "@=" | ' + '"/=" | "//=" | "%=" | "**="\n' + ' | ">>=" | "<<=" | "&=" | "^=" | "|="\n' + '\n' + '(See section *Primaries* for the syntax definitions of the ' + 'last three\n' + 'symbols.)\n' + '\n' + 'An augmented assignment evaluates the target (which, unlike ' + 'normal\n' + 'assignment statements, cannot be an unpacking) and the ' + 'expression\n' + 'list, performs the binary operation specific to the type of ' + 'assignment\n' + 'on the two operands, and assigns the result to the original ' + 'target.\n' + 'The target is only evaluated once.\n' + '\n' + 'An augmented assignment expression like "x += 1" can be ' + 'rewritten as\n' + '"x = x + 1" to achieve a similar, but not exactly equal ' + 'effect. In the\n' + 'augmented version, "x" is only evaluated once. Also, when ' + 'possible,\n' + 'the actual operation is performed *in-place*, meaning that ' + 'rather than\n' + 'creating a new object and assigning that to the target, the ' + 'old object\n' + 'is modified instead.\n' + '\n' + 'Unlike normal assignments, augmented assignments evaluate the ' + 'left-\n' + 'hand side *before* evaluating the right-hand side. For ' + 'example, "a[i]\n' + '+= f(x)" first looks-up "a[i]", then it evaluates "f(x)" and ' + 'performs\n' + 'the addition, and lastly, it writes the result back to ' + '"a[i]".\n' + '\n' + 'With the exception of assigning to tuples and multiple ' + 'targets in a\n' + 'single statement, the assignment done by augmented ' + 'assignment\n' + 'statements is handled the same way as normal assignments. ' + 'Similarly,\n' + 'with the exception of the possible *in-place* behavior, the ' + 'binary\n' + 'operation performed by augmented assignment is the same as ' + 'the normal\n' + 'binary operations.\n' + '\n' + 'For targets which are attribute references, the same *caveat ' + 'about\n' + 'class and instance attributes* applies as for regular ' + 'assignments.\n', + 'binary': '\n' + 'Binary arithmetic operations\n' + '****************************\n' + '\n' + 'The binary arithmetic operations have the conventional priority\n' + 'levels. Note that some of these operations also apply to ' + 'certain non-\n' + 'numeric types. Apart from the power operator, there are only ' + 'two\n' + 'levels, one for multiplicative operators and one for additive\n' + 'operators:\n' + '\n' + ' m_expr ::= u_expr | m_expr "*" u_expr | m_expr "@" m_expr |\n' + ' m_expr "//" u_expr| m_expr "/" u_expr |\n' + ' m_expr "%" u_expr\n' + ' a_expr ::= m_expr | a_expr "+" m_expr | a_expr "-" m_expr\n' + '\n' + 'The "*" (multiplication) operator yields the product of its ' + 'arguments.\n' + 'The arguments must either both be numbers, or one argument must ' + 'be an\n' + 'integer and the other must be a sequence. In the former case, ' + 'the\n' + 'numbers are converted to a common type and then multiplied ' + 'together.\n' + 'In the latter case, sequence repetition is performed; a ' + 'negative\n' + 'repetition factor yields an empty sequence.\n' + '\n' + 'The "@" (at) operator is intended to be used for matrix\n' + 'multiplication. No builtin Python types implement this ' + 'operator.\n' + '\n' + 'New in version 3.5.\n' + '\n' + 'The "/" (division) and "//" (floor division) operators yield ' + 'the\n' + 'quotient of their arguments. The numeric arguments are first\n' + 'converted to a common type. Division of integers yields a float, ' + 'while\n' + 'floor division of integers results in an integer; the result is ' + 'that\n' + "of mathematical division with the 'floor' function applied to " + 'the\n' + 'result. Division by zero raises the "ZeroDivisionError" ' + 'exception.\n' + '\n' + 'The "%" (modulo) operator yields the remainder from the division ' + 'of\n' + 'the first argument by the second. The numeric arguments are ' + 'first\n' + 'converted to a common type. A zero right argument raises the\n' + '"ZeroDivisionError" exception. The arguments may be floating ' + 'point\n' + 'numbers, e.g., "3.14%0.7" equals "0.34" (since "3.14" equals ' + '"4*0.7 +\n' + '0.34".) The modulo operator always yields a result with the ' + 'same sign\n' + 'as its second operand (or zero); the absolute value of the ' + 'result is\n' + 'strictly smaller than the absolute value of the second operand ' + '[1].\n' + '\n' + 'The floor division and modulo operators are connected by the ' + 'following\n' + 'identity: "x == (x//y)*y + (x%y)". Floor division and modulo ' + 'are also\n' + 'connected with the built-in function "divmod()": "divmod(x, y) ' + '==\n' + '(x//y, x%y)". [2].\n' + '\n' + 'In addition to performing the modulo operation on numbers, the ' + '"%"\n' + 'operator is also overloaded by string objects to perform ' + 'old-style\n' + 'string formatting (also known as interpolation). The syntax ' + 'for\n' + 'string formatting is described in the Python Library Reference,\n' + 'section *printf-style String Formatting*.\n' + '\n' + 'The floor division operator, the modulo operator, and the ' + '"divmod()"\n' + 'function are not defined for complex numbers. Instead, convert ' + 'to a\n' + 'floating point number using the "abs()" function if ' + 'appropriate.\n' + '\n' + 'The "+" (addition) operator yields the sum of its arguments. ' + 'The\n' + 'arguments must either both be numbers or both be sequences of ' + 'the same\n' + 'type. In the former case, the numbers are converted to a common ' + 'type\n' + 'and then added together. In the latter case, the sequences are\n' + 'concatenated.\n' + '\n' + 'The "-" (subtraction) operator yields the difference of its ' + 'arguments.\n' + 'The numeric arguments are first converted to a common type.\n', + 'bitwise': '\n' + 'Binary bitwise operations\n' + '*************************\n' + '\n' + 'Each of the three bitwise operations has a different priority ' + 'level:\n' + '\n' + ' and_expr ::= shift_expr | and_expr "&" shift_expr\n' + ' xor_expr ::= and_expr | xor_expr "^" and_expr\n' + ' or_expr ::= xor_expr | or_expr "|" xor_expr\n' + '\n' + 'The "&" operator yields the bitwise AND of its arguments, which ' + 'must\n' + 'be integers.\n' + '\n' + 'The "^" operator yields the bitwise XOR (exclusive OR) of its\n' + 'arguments, which must be integers.\n' + '\n' + 'The "|" operator yields the bitwise (inclusive) OR of its ' + 'arguments,\n' + 'which must be integers.\n', + 'bltin-code-objects': '\n' + 'Code Objects\n' + '************\n' + '\n' + 'Code objects are used by the implementation to ' + 'represent "pseudo-\n' + 'compiled" executable Python code such as a function ' + 'body. They differ\n' + "from function objects because they don't contain a " + 'reference to their\n' + 'global execution environment. Code objects are ' + 'returned by the built-\n' + 'in "compile()" function and can be extracted from ' + 'function objects\n' + 'through their "__code__" attribute. See also the ' + '"code" module.\n' + '\n' + 'A code object can be executed or evaluated by ' + 'passing it (instead of a\n' + 'source string) to the "exec()" or "eval()" built-in ' + 'functions.\n' + '\n' + 'See *The standard type hierarchy* for more ' + 'information.\n', + 'bltin-ellipsis-object': '\n' + 'The Ellipsis Object\n' + '*******************\n' + '\n' + 'This object is commonly used by slicing (see ' + '*Slicings*). It supports\n' + 'no special operations. There is exactly one ' + 'ellipsis object, named\n' + '"Ellipsis" (a built-in name). "type(Ellipsis)()" ' + 'produces the\n' + '"Ellipsis" singleton.\n' + '\n' + 'It is written as "Ellipsis" or "...".\n', + 'bltin-null-object': '\n' + 'The Null Object\n' + '***************\n' + '\n' + "This object is returned by functions that don't " + 'explicitly return a\n' + 'value. It supports no special operations. There is ' + 'exactly one null\n' + 'object, named "None" (a built-in name). ' + '"type(None)()" produces the\n' + 'same singleton.\n' + '\n' + 'It is written as "None".\n', + 'bltin-type-objects': '\n' + 'Type Objects\n' + '************\n' + '\n' + 'Type objects represent the various object types. An ' + "object's type is\n" + 'accessed by the built-in function "type()". There ' + 'are no special\n' + 'operations on types. The standard module "types" ' + 'defines names for\n' + 'all standard built-in types.\n' + '\n' + 'Types are written like this: "".\n', + 'booleans': '\n' + 'Boolean operations\n' + '******************\n' + '\n' + ' or_test ::= and_test | or_test "or" and_test\n' + ' and_test ::= not_test | and_test "and" not_test\n' + ' not_test ::= comparison | "not" not_test\n' + '\n' + 'In the context of Boolean operations, and also when ' + 'expressions are\n' + 'used by control flow statements, the following values are ' + 'interpreted\n' + 'as false: "False", "None", numeric zero of all types, and ' + 'empty\n' + 'strings and containers (including strings, tuples, lists,\n' + 'dictionaries, sets and frozensets). All other values are ' + 'interpreted\n' + 'as true. User-defined objects can customize their truth value ' + 'by\n' + 'providing a "__bool__()" method.\n' + '\n' + 'The operator "not" yields "True" if its argument is false, ' + '"False"\n' + 'otherwise.\n' + '\n' + 'The expression "x and y" first evaluates *x*; if *x* is false, ' + 'its\n' + 'value is returned; otherwise, *y* is evaluated and the ' + 'resulting value\n' + 'is returned.\n' + '\n' + 'The expression "x or y" first evaluates *x*; if *x* is true, ' + 'its value\n' + 'is returned; otherwise, *y* is evaluated and the resulting ' + 'value is\n' + 'returned.\n' + '\n' + '(Note that neither "and" nor "or" restrict the value and type ' + 'they\n' + 'return to "False" and "True", but rather return the last ' + 'evaluated\n' + 'argument. This is sometimes useful, e.g., if "s" is a string ' + 'that\n' + 'should be replaced by a default value if it is empty, the ' + 'expression\n' + '"s or \'foo\'" yields the desired value. Because "not" has to ' + 'create a\n' + 'new value, it returns a boolean value regardless of the type ' + 'of its\n' + 'argument (for example, "not \'foo\'" produces "False" rather ' + 'than "\'\'".)\n', + 'break': '\n' + 'The "break" statement\n' + '*********************\n' + '\n' + ' break_stmt ::= "break"\n' + '\n' + '"break" may only occur syntactically nested in a "for" or ' + '"while"\n' + 'loop, but not nested in a function or class definition within ' + 'that\n' + 'loop.\n' + '\n' + 'It terminates the nearest enclosing loop, skipping the optional ' + '"else"\n' + 'clause if the loop has one.\n' + '\n' + 'If a "for" loop is terminated by "break", the loop control ' + 'target\n' + 'keeps its current value.\n' + '\n' + 'When "break" passes control out of a "try" statement with a ' + '"finally"\n' + 'clause, that "finally" clause is executed before really leaving ' + 'the\n' + 'loop.\n', + 'callable-types': '\n' + 'Emulating callable objects\n' + '**************************\n' + '\n' + 'object.__call__(self[, args...])\n' + '\n' + ' Called when the instance is "called" as a function; ' + 'if this method\n' + ' is defined, "x(arg1, arg2, ...)" is a shorthand for\n' + ' "x.__call__(arg1, arg2, ...)".\n', + 'calls': '\n' + 'Calls\n' + '*****\n' + '\n' + 'A call calls a callable object (e.g., a *function*) with a ' + 'possibly\n' + 'empty series of *arguments*:\n' + '\n' + ' call ::= primary "(" [argument_list [","] | ' + 'comprehension] ")"\n' + ' argument_list ::= positional_arguments ["," ' + 'keyword_arguments]\n' + ' ["," "*" expression] ["," ' + 'keyword_arguments]\n' + ' ["," "**" expression]\n' + ' | keyword_arguments ["," "*" expression]\n' + ' ["," keyword_arguments] ["," "**" ' + 'expression]\n' + ' | "*" expression ["," keyword_arguments] ' + '["," "**" expression]\n' + ' | "**" expression\n' + ' positional_arguments ::= expression ("," expression)*\n' + ' keyword_arguments ::= keyword_item ("," keyword_item)*\n' + ' keyword_item ::= identifier "=" expression\n' + '\n' + 'An optional trailing comma may be present after the positional ' + 'and\n' + 'keyword arguments but does not affect the semantics.\n' + '\n' + 'The primary must evaluate to a callable object (user-defined\n' + 'functions, built-in functions, methods of built-in objects, ' + 'class\n' + 'objects, methods of class instances, and all objects having a\n' + '"__call__()" method are callable). All argument expressions are\n' + 'evaluated before the call is attempted. Please refer to section\n' + '*Function definitions* for the syntax of formal *parameter* ' + 'lists.\n' + '\n' + 'If keyword arguments are present, they are first converted to\n' + 'positional arguments, as follows. First, a list of unfilled ' + 'slots is\n' + 'created for the formal parameters. If there are N positional\n' + 'arguments, they are placed in the first N slots. Next, for each\n' + 'keyword argument, the identifier is used to determine the\n' + 'corresponding slot (if the identifier is the same as the first ' + 'formal\n' + 'parameter name, the first slot is used, and so on). If the slot ' + 'is\n' + 'already filled, a "TypeError" exception is raised. Otherwise, ' + 'the\n' + 'value of the argument is placed in the slot, filling it (even if ' + 'the\n' + 'expression is "None", it fills the slot). When all arguments ' + 'have\n' + 'been processed, the slots that are still unfilled are filled with ' + 'the\n' + 'corresponding default value from the function definition. ' + '(Default\n' + 'values are calculated, once, when the function is defined; thus, ' + 'a\n' + 'mutable object such as a list or dictionary used as default value ' + 'will\n' + "be shared by all calls that don't specify an argument value for " + 'the\n' + 'corresponding slot; this should usually be avoided.) If there ' + 'are any\n' + 'unfilled slots for which no default value is specified, a ' + '"TypeError"\n' + 'exception is raised. Otherwise, the list of filled slots is used ' + 'as\n' + 'the argument list for the call.\n' + '\n' + '**CPython implementation detail:** An implementation may provide\n' + 'built-in functions whose positional parameters do not have names, ' + 'even\n' + "if they are 'named' for the purpose of documentation, and which\n" + 'therefore cannot be supplied by keyword. In CPython, this is the ' + 'case\n' + 'for functions implemented in C that use "PyArg_ParseTuple()" to ' + 'parse\n' + 'their arguments.\n' + '\n' + 'If there are more positional arguments than there are formal ' + 'parameter\n' + 'slots, a "TypeError" exception is raised, unless a formal ' + 'parameter\n' + 'using the syntax "*identifier" is present; in this case, that ' + 'formal\n' + 'parameter receives a tuple containing the excess positional ' + 'arguments\n' + '(or an empty tuple if there were no excess positional ' + 'arguments).\n' + '\n' + 'If any keyword argument does not correspond to a formal ' + 'parameter\n' + 'name, a "TypeError" exception is raised, unless a formal ' + 'parameter\n' + 'using the syntax "**identifier" is present; in this case, that ' + 'formal\n' + 'parameter receives a dictionary containing the excess keyword\n' + 'arguments (using the keywords as keys and the argument values as\n' + 'corresponding values), or a (new) empty dictionary if there were ' + 'no\n' + 'excess keyword arguments.\n' + '\n' + 'If the syntax "*expression" appears in the function call, ' + '"expression"\n' + 'must evaluate to an iterable. Elements from this iterable are ' + 'treated\n' + 'as if they were additional positional arguments; if there are\n' + 'positional arguments *x1*, ..., *xN*, and "expression" evaluates ' + 'to a\n' + 'sequence *y1*, ..., *yM*, this is equivalent to a call with M+N\n' + 'positional arguments *x1*, ..., *xN*, *y1*, ..., *yM*.\n' + '\n' + 'A consequence of this is that although the "*expression" syntax ' + 'may\n' + 'appear *after* some keyword arguments, it is processed *before* ' + 'the\n' + 'keyword arguments (and the "**expression" argument, if any -- ' + 'see\n' + 'below). So:\n' + '\n' + ' >>> def f(a, b):\n' + ' ... print(a, b)\n' + ' ...\n' + ' >>> f(b=1, *(2,))\n' + ' 2 1\n' + ' >>> f(a=1, *(2,))\n' + ' Traceback (most recent call last):\n' + ' File "", line 1, in ?\n' + " TypeError: f() got multiple values for keyword argument 'a'\n" + ' >>> f(1, *(2,))\n' + ' 1 2\n' + '\n' + 'It is unusual for both keyword arguments and the "*expression" ' + 'syntax\n' + 'to be used in the same call, so in practice this confusion does ' + 'not\n' + 'arise.\n' + '\n' + 'If the syntax "**expression" appears in the function call,\n' + '"expression" must evaluate to a mapping, the contents of which ' + 'are\n' + 'treated as additional keyword arguments. In the case of a ' + 'keyword\n' + 'appearing in both "expression" and as an explicit keyword ' + 'argument, a\n' + '"TypeError" exception is raised.\n' + '\n' + 'Formal parameters using the syntax "*identifier" or ' + '"**identifier"\n' + 'cannot be used as positional argument slots or as keyword ' + 'argument\n' + 'names.\n' + '\n' + 'A call always returns some value, possibly "None", unless it ' + 'raises an\n' + 'exception. How this value is computed depends on the type of ' + 'the\n' + 'callable object.\n' + '\n' + 'If it is---\n' + '\n' + 'a user-defined function:\n' + ' The code block for the function is executed, passing it the\n' + ' argument list. The first thing the code block will do is bind ' + 'the\n' + ' formal parameters to the arguments; this is described in ' + 'section\n' + ' *Function definitions*. When the code block executes a ' + '"return"\n' + ' statement, this specifies the return value of the function ' + 'call.\n' + '\n' + 'a built-in function or method:\n' + ' The result is up to the interpreter; see *Built-in Functions* ' + 'for\n' + ' the descriptions of built-in functions and methods.\n' + '\n' + 'a class object:\n' + ' A new instance of that class is returned.\n' + '\n' + 'a class instance method:\n' + ' The corresponding user-defined function is called, with an ' + 'argument\n' + ' list that is one longer than the argument list of the call: ' + 'the\n' + ' instance becomes the first argument.\n' + '\n' + 'a class instance:\n' + ' The class must define a "__call__()" method; the effect is ' + 'then the\n' + ' same as if that method was called.\n', + 'class': '\n' + 'Class definitions\n' + '*****************\n' + '\n' + 'A class definition defines a class object (see section *The ' + 'standard\n' + 'type hierarchy*):\n' + '\n' + ' classdef ::= [decorators] "class" classname [inheritance] ' + '":" suite\n' + ' inheritance ::= "(" [parameter_list] ")"\n' + ' classname ::= identifier\n' + '\n' + 'A class definition is an executable statement. The inheritance ' + 'list\n' + 'usually gives a list of base classes (see *Customizing class ' + 'creation*\n' + 'for more advanced uses), so each item in the list should evaluate ' + 'to a\n' + 'class object which allows subclassing. Classes without an ' + 'inheritance\n' + 'list inherit, by default, from the base class "object"; hence,\n' + '\n' + ' class Foo:\n' + ' pass\n' + '\n' + 'is equivalent to\n' + '\n' + ' class Foo(object):\n' + ' pass\n' + '\n' + "The class's suite is then executed in a new execution frame (see\n" + '*Naming and binding*), using a newly created local namespace and ' + 'the\n' + 'original global namespace. (Usually, the suite contains mostly\n' + "function definitions.) When the class's suite finishes " + 'execution, its\n' + 'execution frame is discarded but its local namespace is saved. ' + '[4] A\n' + 'class object is then created using the inheritance list for the ' + 'base\n' + 'classes and the saved local namespace for the attribute ' + 'dictionary.\n' + 'The class name is bound to this class object in the original ' + 'local\n' + 'namespace.\n' + '\n' + 'Class creation can be customized heavily using *metaclasses*.\n' + '\n' + 'Classes can also be decorated: just like when decorating ' + 'functions,\n' + '\n' + ' @f1(arg)\n' + ' @f2\n' + ' class Foo: pass\n' + '\n' + 'is equivalent to\n' + '\n' + ' class Foo: pass\n' + ' Foo = f1(arg)(f2(Foo))\n' + '\n' + 'The evaluation rules for the decorator expressions are the same ' + 'as for\n' + 'function decorators. The result must be a class object, which is ' + 'then\n' + 'bound to the class name.\n' + '\n' + "**Programmer's note:** Variables defined in the class definition " + 'are\n' + 'class attributes; they are shared by instances. Instance ' + 'attributes\n' + 'can be set in a method with "self.name = value". Both class and\n' + 'instance attributes are accessible through the notation ' + '""self.name"",\n' + 'and an instance attribute hides a class attribute with the same ' + 'name\n' + 'when accessed in this way. Class attributes can be used as ' + 'defaults\n' + 'for instance attributes, but using mutable values there can lead ' + 'to\n' + 'unexpected results. *Descriptors* can be used to create ' + 'instance\n' + 'variables with different implementation details.\n' + '\n' + 'See also: **PEP 3115** - Metaclasses in Python 3 **PEP 3129** -\n' + ' Class Decorators\n', + 'comparisons': '\n' + 'Comparisons\n' + '***********\n' + '\n' + 'Unlike C, all comparison operations in Python have the same ' + 'priority,\n' + 'which is lower than that of any arithmetic, shifting or ' + 'bitwise\n' + 'operation. Also unlike C, expressions like "a < b < c" ' + 'have the\n' + 'interpretation that is conventional in mathematics:\n' + '\n' + ' comparison ::= or_expr ( comp_operator or_expr )*\n' + ' comp_operator ::= "<" | ">" | "==" | ">=" | "<=" | "!="\n' + ' | "is" ["not"] | ["not"] "in"\n' + '\n' + 'Comparisons yield boolean values: "True" or "False".\n' + '\n' + 'Comparisons can be chained arbitrarily, e.g., "x < y <= z" ' + 'is\n' + 'equivalent to "x < y and y <= z", except that "y" is ' + 'evaluated only\n' + 'once (but in both cases "z" is not evaluated at all when "x ' + '< y" is\n' + 'found to be false).\n' + '\n' + 'Formally, if *a*, *b*, *c*, ..., *y*, *z* are expressions ' + 'and *op1*,\n' + '*op2*, ..., *opN* are comparison operators, then "a op1 b ' + 'op2 c ... y\n' + 'opN z" is equivalent to "a op1 b and b op2 c and ... y opN ' + 'z", except\n' + 'that each expression is evaluated at most once.\n' + '\n' + 'Note that "a op1 b op2 c" doesn\'t imply any kind of ' + 'comparison between\n' + '*a* and *c*, so that, e.g., "x < y > z" is perfectly legal ' + '(though\n' + 'perhaps not pretty).\n' + '\n' + 'The operators "<", ">", "==", ">=", "<=", and "!=" compare ' + 'the values\n' + 'of two objects. The objects need not have the same type. ' + 'If both are\n' + 'numbers, they are converted to a common type. Otherwise, ' + 'the "==" and\n' + '"!=" operators *always* consider objects of different types ' + 'to be\n' + 'unequal, while the "<", ">", ">=" and "<=" operators raise ' + 'a\n' + '"TypeError" when comparing objects of different types that ' + 'do not\n' + 'implement these operators for the given pair of types. You ' + 'can\n' + 'control comparison behavior of objects of non-built-in ' + 'types by\n' + 'defining rich comparison methods like "__gt__()", described ' + 'in section\n' + '*Basic customization*.\n' + '\n' + 'Comparison of objects of the same type depends on the ' + 'type:\n' + '\n' + '* Numbers are compared arithmetically.\n' + '\n' + '* The values "float(\'NaN\')" and "Decimal(\'NaN\')" are ' + 'special. They\n' + ' are identical to themselves, "x is x" but are not equal ' + 'to\n' + ' themselves, "x != x". Additionally, comparing any value ' + 'to a\n' + ' not-a-number value will return "False". For example, ' + 'both "3 <\n' + ' float(\'NaN\')" and "float(\'NaN\') < 3" will return ' + '"False".\n' + '\n' + '* Bytes objects are compared lexicographically using the ' + 'numeric\n' + ' values of their elements.\n' + '\n' + '* Strings are compared lexicographically using the numeric\n' + ' equivalents (the result of the built-in function "ord()") ' + 'of their\n' + " characters. [3] String and bytes object can't be " + 'compared!\n' + '\n' + '* Tuples and lists are compared lexicographically using ' + 'comparison\n' + ' of corresponding elements. This means that to compare ' + 'equal, each\n' + ' element must compare equal and the two sequences must be ' + 'of the same\n' + ' type and have the same length.\n' + '\n' + ' If not equal, the sequences are ordered the same as their ' + 'first\n' + ' differing elements. For example, "[1,2,x] <= [1,2,y]" ' + 'has the same\n' + ' value as "x <= y". If the corresponding element does not ' + 'exist, the\n' + ' shorter sequence is ordered first (for example, "[1,2] < ' + '[1,2,3]").\n' + '\n' + '* Mappings (dictionaries) compare equal if and only if they ' + 'have the\n' + ' same "(key, value)" pairs. Order comparisons "(\'<\', ' + "'<=', '>=',\n" + ' \'>\')" raise "TypeError".\n' + '\n' + '* Sets and frozensets define comparison operators to mean ' + 'subset and\n' + ' superset tests. Those relations do not define total ' + 'orderings (the\n' + ' two sets "{1,2}" and "{2,3}" are not equal, nor subsets ' + 'of one\n' + ' another, nor supersets of one another). Accordingly, ' + 'sets are not\n' + ' appropriate arguments for functions which depend on total ' + 'ordering.\n' + ' For example, "min()", "max()", and "sorted()" produce ' + 'undefined\n' + ' results given a list of sets as inputs.\n' + '\n' + '* Most other objects of built-in types compare unequal ' + 'unless they\n' + ' are the same object; the choice whether one object is ' + 'considered\n' + ' smaller or larger than another one is made arbitrarily ' + 'but\n' + ' consistently within one execution of a program.\n' + '\n' + 'Comparison of objects of differing types depends on whether ' + 'either of\n' + 'the types provide explicit support for the comparison. ' + 'Most numeric\n' + 'types can be compared with one another. When cross-type ' + 'comparison is\n' + 'not supported, the comparison method returns ' + '"NotImplemented".\n' + '\n' + 'The operators "in" and "not in" test for membership. "x in ' + 's"\n' + 'evaluates to true if *x* is a member of *s*, and false ' + 'otherwise. "x\n' + 'not in s" returns the negation of "x in s". All built-in ' + 'sequences\n' + 'and set types support this as well as dictionary, for which ' + '"in" tests\n' + 'whether the dictionary has a given key. For container types ' + 'such as\n' + 'list, tuple, set, frozenset, dict, or collections.deque, ' + 'the\n' + 'expression "x in y" is equivalent to "any(x is e or x == e ' + 'for e in\n' + 'y)".\n' + '\n' + 'For the string and bytes types, "x in y" is true if and ' + 'only if *x* is\n' + 'a substring of *y*. An equivalent test is "y.find(x) != ' + '-1". Empty\n' + 'strings are always considered to be a substring of any ' + 'other string,\n' + 'so """ in "abc"" will return "True".\n' + '\n' + 'For user-defined classes which define the "__contains__()" ' + 'method, "x\n' + 'in y" is true if and only if "y.__contains__(x)" is true.\n' + '\n' + 'For user-defined classes which do not define ' + '"__contains__()" but do\n' + 'define "__iter__()", "x in y" is true if some value "z" ' + 'with "x == z"\n' + 'is produced while iterating over "y". If an exception is ' + 'raised\n' + 'during the iteration, it is as if "in" raised that ' + 'exception.\n' + '\n' + 'Lastly, the old-style iteration protocol is tried: if a ' + 'class defines\n' + '"__getitem__()", "x in y" is true if and only if there is a ' + 'non-\n' + 'negative integer index *i* such that "x == y[i]", and all ' + 'lower\n' + 'integer indices do not raise "IndexError" exception. (If ' + 'any other\n' + 'exception is raised, it is as if "in" raised that ' + 'exception).\n' + '\n' + 'The operator "not in" is defined to have the inverse true ' + 'value of\n' + '"in".\n' + '\n' + 'The operators "is" and "is not" test for object identity: ' + '"x is y" is\n' + 'true if and only if *x* and *y* are the same object. "x is ' + 'not y"\n' + 'yields the inverse truth value. [4]\n', + 'compound': '\n' + 'Compound statements\n' + '*******************\n' + '\n' + 'Compound statements contain (groups of) other statements; they ' + 'affect\n' + 'or control the execution of those other statements in some ' + 'way. In\n' + 'general, compound statements span multiple lines, although in ' + 'simple\n' + 'incarnations a whole compound statement may be contained in ' + 'one line.\n' + '\n' + 'The "if", "while" and "for" statements implement traditional ' + 'control\n' + 'flow constructs. "try" specifies exception handlers and/or ' + 'cleanup\n' + 'code for a group of statements, while the "with" statement ' + 'allows the\n' + 'execution of initialization and finalization code around a ' + 'block of\n' + 'code. Function and class definitions are also syntactically ' + 'compound\n' + 'statements.\n' + '\n' + "A compound statement consists of one or more 'clauses.' A " + 'clause\n' + "consists of a header and a 'suite.' The clause headers of a\n" + 'particular compound statement are all at the same indentation ' + 'level.\n' + 'Each clause header begins with a uniquely identifying keyword ' + 'and ends\n' + 'with a colon. A suite is a group of statements controlled by ' + 'a\n' + 'clause. A suite can be one or more semicolon-separated ' + 'simple\n' + 'statements on the same line as the header, following the ' + "header's\n" + 'colon, or it can be one or more indented statements on ' + 'subsequent\n' + 'lines. Only the latter form of a suite can contain nested ' + 'compound\n' + 'statements; the following is illegal, mostly because it ' + "wouldn't be\n" + 'clear to which "if" clause a following "else" clause would ' + 'belong:\n' + '\n' + ' if test1: if test2: print(x)\n' + '\n' + 'Also note that the semicolon binds tighter than the colon in ' + 'this\n' + 'context, so that in the following example, either all or none ' + 'of the\n' + '"print()" calls are executed:\n' + '\n' + ' if x < y < z: print(x); print(y); print(z)\n' + '\n' + 'Summarizing:\n' + '\n' + ' compound_stmt ::= if_stmt\n' + ' | while_stmt\n' + ' | for_stmt\n' + ' | try_stmt\n' + ' | with_stmt\n' + ' | funcdef\n' + ' | classdef\n' + ' | async_with_stmt\n' + ' | async_for_stmt\n' + ' | async_funcdef\n' + ' suite ::= stmt_list NEWLINE | NEWLINE INDENT ' + 'statement+ DEDENT\n' + ' statement ::= stmt_list NEWLINE | compound_stmt\n' + ' stmt_list ::= simple_stmt (";" simple_stmt)* [";"]\n' + '\n' + 'Note that statements always end in a "NEWLINE" possibly ' + 'followed by a\n' + '"DEDENT". Also note that optional continuation clauses always ' + 'begin\n' + 'with a keyword that cannot start a statement, thus there are ' + 'no\n' + 'ambiguities (the \'dangling "else"\' problem is solved in ' + 'Python by\n' + 'requiring nested "if" statements to be indented).\n' + '\n' + 'The formatting of the grammar rules in the following sections ' + 'places\n' + 'each clause on a separate line for clarity.\n' + '\n' + '\n' + 'The "if" statement\n' + '==================\n' + '\n' + 'The "if" statement is used for conditional execution:\n' + '\n' + ' if_stmt ::= "if" expression ":" suite\n' + ' ( "elif" expression ":" suite )*\n' + ' ["else" ":" suite]\n' + '\n' + 'It selects exactly one of the suites by evaluating the ' + 'expressions one\n' + 'by one until one is found to be true (see section *Boolean ' + 'operations*\n' + 'for the definition of true and false); then that suite is ' + 'executed\n' + '(and no other part of the "if" statement is executed or ' + 'evaluated).\n' + 'If all expressions are false, the suite of the "else" clause, ' + 'if\n' + 'present, is executed.\n' + '\n' + '\n' + 'The "while" statement\n' + '=====================\n' + '\n' + 'The "while" statement is used for repeated execution as long ' + 'as an\n' + 'expression is true:\n' + '\n' + ' while_stmt ::= "while" expression ":" suite\n' + ' ["else" ":" suite]\n' + '\n' + 'This repeatedly tests the expression and, if it is true, ' + 'executes the\n' + 'first suite; if the expression is false (which may be the ' + 'first time\n' + 'it is tested) the suite of the "else" clause, if present, is ' + 'executed\n' + 'and the loop terminates.\n' + '\n' + 'A "break" statement executed in the first suite terminates the ' + 'loop\n' + 'without executing the "else" clause\'s suite. A "continue" ' + 'statement\n' + 'executed in the first suite skips the rest of the suite and ' + 'goes back\n' + 'to testing the expression.\n' + '\n' + '\n' + 'The "for" statement\n' + '===================\n' + '\n' + 'The "for" statement is used to iterate over the elements of a ' + 'sequence\n' + '(such as a string, tuple or list) or other iterable object:\n' + '\n' + ' for_stmt ::= "for" target_list "in" expression_list ":" ' + 'suite\n' + ' ["else" ":" suite]\n' + '\n' + 'The expression list is evaluated once; it should yield an ' + 'iterable\n' + 'object. An iterator is created for the result of the\n' + '"expression_list". The suite is then executed once for each ' + 'item\n' + 'provided by the iterator, in the order returned by the ' + 'iterator. Each\n' + 'item in turn is assigned to the target list using the standard ' + 'rules\n' + 'for assignments (see *Assignment statements*), and then the ' + 'suite is\n' + 'executed. When the items are exhausted (which is immediately ' + 'when the\n' + 'sequence is empty or an iterator raises a "StopIteration" ' + 'exception),\n' + 'the suite in the "else" clause, if present, is executed, and ' + 'the loop\n' + 'terminates.\n' + '\n' + 'A "break" statement executed in the first suite terminates the ' + 'loop\n' + 'without executing the "else" clause\'s suite. A "continue" ' + 'statement\n' + 'executed in the first suite skips the rest of the suite and ' + 'continues\n' + 'with the next item, or with the "else" clause if there is no ' + 'next\n' + 'item.\n' + '\n' + 'The for-loop makes assignments to the variables(s) in the ' + 'target list.\n' + 'This overwrites all previous assignments to those variables ' + 'including\n' + 'those made in the suite of the for-loop:\n' + '\n' + ' for i in range(10):\n' + ' print(i)\n' + ' i = 5 # this will not affect the for-loop\n' + ' # because i will be overwritten with ' + 'the next\n' + ' # index in the range\n' + '\n' + 'Names in the target list are not deleted when the loop is ' + 'finished,\n' + 'but if the sequence is empty, they will not have been assigned ' + 'to at\n' + 'all by the loop. Hint: the built-in function "range()" ' + 'returns an\n' + 'iterator of integers suitable to emulate the effect of ' + 'Pascal\'s "for i\n' + ':= a to b do"; e.g., "list(range(3))" returns the list "[0, 1, ' + '2]".\n' + '\n' + 'Note: There is a subtlety when the sequence is being modified ' + 'by the\n' + ' loop (this can only occur for mutable sequences, i.e. ' + 'lists). An\n' + ' internal counter is used to keep track of which item is used ' + 'next,\n' + ' and this is incremented on each iteration. When this ' + 'counter has\n' + ' reached the length of the sequence the loop terminates. ' + 'This means\n' + ' that if the suite deletes the current (or a previous) item ' + 'from the\n' + ' sequence, the next item will be skipped (since it gets the ' + 'index of\n' + ' the current item which has already been treated). Likewise, ' + 'if the\n' + ' suite inserts an item in the sequence before the current ' + 'item, the\n' + ' current item will be treated again the next time through the ' + 'loop.\n' + ' This can lead to nasty bugs that can be avoided by making a\n' + ' temporary copy using a slice of the whole sequence, e.g.,\n' + '\n' + ' for x in a[:]:\n' + ' if x < 0: a.remove(x)\n' + '\n' + '\n' + 'The "try" statement\n' + '===================\n' + '\n' + 'The "try" statement specifies exception handlers and/or ' + 'cleanup code\n' + 'for a group of statements:\n' + '\n' + ' try_stmt ::= try1_stmt | try2_stmt\n' + ' try1_stmt ::= "try" ":" suite\n' + ' ("except" [expression ["as" identifier]] ":" ' + 'suite)+\n' + ' ["else" ":" suite]\n' + ' ["finally" ":" suite]\n' + ' try2_stmt ::= "try" ":" suite\n' + ' "finally" ":" suite\n' + '\n' + 'The "except" clause(s) specify one or more exception handlers. ' + 'When no\n' + 'exception occurs in the "try" clause, no exception handler is\n' + 'executed. When an exception occurs in the "try" suite, a ' + 'search for an\n' + 'exception handler is started. This search inspects the except ' + 'clauses\n' + 'in turn until one is found that matches the exception. An ' + 'expression-\n' + 'less except clause, if present, must be last; it matches any\n' + 'exception. For an except clause with an expression, that ' + 'expression\n' + 'is evaluated, and the clause matches the exception if the ' + 'resulting\n' + 'object is "compatible" with the exception. An object is ' + 'compatible\n' + 'with an exception if it is the class or a base class of the ' + 'exception\n' + 'object or a tuple containing an item compatible with the ' + 'exception.\n' + '\n' + 'If no except clause matches the exception, the search for an ' + 'exception\n' + 'handler continues in the surrounding code and on the ' + 'invocation stack.\n' + '[1]\n' + '\n' + 'If the evaluation of an expression in the header of an except ' + 'clause\n' + 'raises an exception, the original search for a handler is ' + 'canceled and\n' + 'a search starts for the new exception in the surrounding code ' + 'and on\n' + 'the call stack (it is treated as if the entire "try" statement ' + 'raised\n' + 'the exception).\n' + '\n' + 'When a matching except clause is found, the exception is ' + 'assigned to\n' + 'the target specified after the "as" keyword in that except ' + 'clause, if\n' + "present, and the except clause's suite is executed. All " + 'except\n' + 'clauses must have an executable block. When the end of this ' + 'block is\n' + 'reached, execution continues normally after the entire try ' + 'statement.\n' + '(This means that if two nested handlers exist for the same ' + 'exception,\n' + 'and the exception occurs in the try clause of the inner ' + 'handler, the\n' + 'outer handler will not handle the exception.)\n' + '\n' + 'When an exception has been assigned using "as target", it is ' + 'cleared\n' + 'at the end of the except clause. This is as if\n' + '\n' + ' except E as N:\n' + ' foo\n' + '\n' + 'was translated to\n' + '\n' + ' except E as N:\n' + ' try:\n' + ' foo\n' + ' finally:\n' + ' del N\n' + '\n' + 'This means the exception must be assigned to a different name ' + 'to be\n' + 'able to refer to it after the except clause. Exceptions are ' + 'cleared\n' + 'because with the traceback attached to them, they form a ' + 'reference\n' + 'cycle with the stack frame, keeping all locals in that frame ' + 'alive\n' + 'until the next garbage collection occurs.\n' + '\n' + "Before an except clause's suite is executed, details about " + 'the\n' + 'exception are stored in the "sys" module and can be accessed ' + 'via\n' + '"sys.exc_info()". "sys.exc_info()" returns a 3-tuple ' + 'consisting of the\n' + 'exception class, the exception instance and a traceback object ' + '(see\n' + 'section *The standard type hierarchy*) identifying the point ' + 'in the\n' + 'program where the exception occurred. "sys.exc_info()" values ' + 'are\n' + 'restored to their previous values (before the call) when ' + 'returning\n' + 'from a function that handled an exception.\n' + '\n' + 'The optional "else" clause is executed if and when control ' + 'flows off\n' + 'the end of the "try" clause. [2] Exceptions in the "else" ' + 'clause are\n' + 'not handled by the preceding "except" clauses.\n' + '\n' + 'If "finally" is present, it specifies a \'cleanup\' handler. ' + 'The "try"\n' + 'clause is executed, including any "except" and "else" ' + 'clauses. If an\n' + 'exception occurs in any of the clauses and is not handled, ' + 'the\n' + 'exception is temporarily saved. The "finally" clause is ' + 'executed. If\n' + 'there is a saved exception it is re-raised at the end of the ' + '"finally"\n' + 'clause. If the "finally" clause raises another exception, the ' + 'saved\n' + 'exception is set as the context of the new exception. If the ' + '"finally"\n' + 'clause executes a "return" or "break" statement, the saved ' + 'exception\n' + 'is discarded:\n' + '\n' + ' >>> def f():\n' + ' ... try:\n' + ' ... 1/0\n' + ' ... finally:\n' + ' ... return 42\n' + ' ...\n' + ' >>> f()\n' + ' 42\n' + '\n' + 'The exception information is not available to the program ' + 'during\n' + 'execution of the "finally" clause.\n' + '\n' + 'When a "return", "break" or "continue" statement is executed ' + 'in the\n' + '"try" suite of a "try"..."finally" statement, the "finally" ' + 'clause is\n' + 'also executed \'on the way out.\' A "continue" statement is ' + 'illegal in\n' + 'the "finally" clause. (The reason is a problem with the ' + 'current\n' + 'implementation --- this restriction may be lifted in the ' + 'future).\n' + '\n' + 'The return value of a function is determined by the last ' + '"return"\n' + 'statement executed. Since the "finally" clause always ' + 'executes, a\n' + '"return" statement executed in the "finally" clause will ' + 'always be the\n' + 'last one executed:\n' + '\n' + ' >>> def foo():\n' + ' ... try:\n' + " ... return 'try'\n" + ' ... finally:\n' + " ... return 'finally'\n" + ' ...\n' + ' >>> foo()\n' + " 'finally'\n" + '\n' + 'Additional information on exceptions can be found in section\n' + '*Exceptions*, and information on using the "raise" statement ' + 'to\n' + 'generate exceptions may be found in section *The raise ' + 'statement*.\n' + '\n' + '\n' + 'The "with" statement\n' + '====================\n' + '\n' + 'The "with" statement is used to wrap the execution of a block ' + 'with\n' + 'methods defined by a context manager (see section *With ' + 'Statement\n' + 'Context Managers*). This allows common ' + '"try"..."except"..."finally"\n' + 'usage patterns to be encapsulated for convenient reuse.\n' + '\n' + ' with_stmt ::= "with" with_item ("," with_item)* ":" suite\n' + ' with_item ::= expression ["as" target]\n' + '\n' + 'The execution of the "with" statement with one "item" proceeds ' + 'as\n' + 'follows:\n' + '\n' + '1. The context expression (the expression given in the ' + '"with_item")\n' + ' is evaluated to obtain a context manager.\n' + '\n' + '2. The context manager\'s "__exit__()" is loaded for later ' + 'use.\n' + '\n' + '3. The context manager\'s "__enter__()" method is invoked.\n' + '\n' + '4. If a target was included in the "with" statement, the ' + 'return\n' + ' value from "__enter__()" is assigned to it.\n' + '\n' + ' Note: The "with" statement guarantees that if the ' + '"__enter__()"\n' + ' method returns without an error, then "__exit__()" will ' + 'always be\n' + ' called. Thus, if an error occurs during the assignment to ' + 'the\n' + ' target list, it will be treated the same as an error ' + 'occurring\n' + ' within the suite would be. See step 6 below.\n' + '\n' + '5. The suite is executed.\n' + '\n' + '6. The context manager\'s "__exit__()" method is invoked. If ' + 'an\n' + ' exception caused the suite to be exited, its type, value, ' + 'and\n' + ' traceback are passed as arguments to "__exit__()". ' + 'Otherwise, three\n' + ' "None" arguments are supplied.\n' + '\n' + ' If the suite was exited due to an exception, and the return ' + 'value\n' + ' from the "__exit__()" method was false, the exception is ' + 'reraised.\n' + ' If the return value was true, the exception is suppressed, ' + 'and\n' + ' execution continues with the statement following the ' + '"with"\n' + ' statement.\n' + '\n' + ' If the suite was exited for any reason other than an ' + 'exception, the\n' + ' return value from "__exit__()" is ignored, and execution ' + 'proceeds\n' + ' at the normal location for the kind of exit that was ' + 'taken.\n' + '\n' + 'With more than one item, the context managers are processed as ' + 'if\n' + 'multiple "with" statements were nested:\n' + '\n' + ' with A() as a, B() as b:\n' + ' suite\n' + '\n' + 'is equivalent to\n' + '\n' + ' with A() as a:\n' + ' with B() as b:\n' + ' suite\n' + '\n' + 'Changed in version 3.1: Support for multiple context ' + 'expressions.\n' + '\n' + 'See also: **PEP 0343** - The "with" statement\n' + '\n' + ' The specification, background, and examples for the ' + 'Python "with"\n' + ' statement.\n' + '\n' + '\n' + 'Function definitions\n' + '====================\n' + '\n' + 'A function definition defines a user-defined function object ' + '(see\n' + 'section *The standard type hierarchy*):\n' + '\n' + ' funcdef ::= [decorators] "def" funcname "(" ' + '[parameter_list] ")" ["->" expression] ":" suite\n' + ' decorators ::= decorator+\n' + ' decorator ::= "@" dotted_name ["(" [parameter_list ' + '[","]] ")"] NEWLINE\n' + ' dotted_name ::= identifier ("." identifier)*\n' + ' parameter_list ::= (defparameter ",")*\n' + ' | "*" [parameter] ("," defparameter)* ' + '["," "**" parameter]\n' + ' | "**" parameter\n' + ' | defparameter [","] )\n' + ' parameter ::= identifier [":" expression]\n' + ' defparameter ::= parameter ["=" expression]\n' + ' funcname ::= identifier\n' + '\n' + 'A function definition is an executable statement. Its ' + 'execution binds\n' + 'the function name in the current local namespace to a function ' + 'object\n' + '(a wrapper around the executable code for the function). ' + 'This\n' + 'function object contains a reference to the current global ' + 'namespace\n' + 'as the global namespace to be used when the function is ' + 'called.\n' + '\n' + 'The function definition does not execute the function body; ' + 'this gets\n' + 'executed only when the function is called. [3]\n' + '\n' + 'A function definition may be wrapped by one or more ' + '*decorator*\n' + 'expressions. Decorator expressions are evaluated when the ' + 'function is\n' + 'defined, in the scope that contains the function definition. ' + 'The\n' + 'result must be a callable, which is invoked with the function ' + 'object\n' + 'as the only argument. The returned value is bound to the ' + 'function name\n' + 'instead of the function object. Multiple decorators are ' + 'applied in\n' + 'nested fashion. For example, the following code\n' + '\n' + ' @f1(arg)\n' + ' @f2\n' + ' def func(): pass\n' + '\n' + 'is equivalent to\n' + '\n' + ' def func(): pass\n' + ' func = f1(arg)(f2(func))\n' + '\n' + 'When one or more *parameters* have the form *parameter* "="\n' + '*expression*, the function is said to have "default parameter ' + 'values."\n' + 'For a parameter with a default value, the corresponding ' + '*argument* may\n' + "be omitted from a call, in which case the parameter's default " + 'value is\n' + 'substituted. If a parameter has a default value, all ' + 'following\n' + 'parameters up until the ""*"" must also have a default value ' + '--- this\n' + 'is a syntactic restriction that is not expressed by the ' + 'grammar.\n' + '\n' + '**Default parameter values are evaluated from left to right ' + 'when the\n' + 'function definition is executed.** This means that the ' + 'expression is\n' + 'evaluated once, when the function is defined, and that the ' + 'same "pre-\n' + 'computed" value is used for each call. This is especially ' + 'important\n' + 'to understand when a default parameter is a mutable object, ' + 'such as a\n' + 'list or a dictionary: if the function modifies the object ' + '(e.g. by\n' + 'appending an item to a list), the default value is in effect ' + 'modified.\n' + 'This is generally not what was intended. A way around this is ' + 'to use\n' + '"None" as the default, and explicitly test for it in the body ' + 'of the\n' + 'function, e.g.:\n' + '\n' + ' def whats_on_the_telly(penguin=None):\n' + ' if penguin is None:\n' + ' penguin = []\n' + ' penguin.append("property of the zoo")\n' + ' return penguin\n' + '\n' + 'Function call semantics are described in more detail in ' + 'section\n' + '*Calls*. A function call always assigns values to all ' + 'parameters\n' + 'mentioned in the parameter list, either from position ' + 'arguments, from\n' + 'keyword arguments, or from default values. If the form\n' + '""*identifier"" is present, it is initialized to a tuple ' + 'receiving any\n' + 'excess positional parameters, defaulting to the empty tuple. ' + 'If the\n' + 'form ""**identifier"" is present, it is initialized to a new\n' + 'dictionary receiving any excess keyword arguments, defaulting ' + 'to a new\n' + 'empty dictionary. Parameters after ""*"" or ""*identifier"" ' + 'are\n' + 'keyword-only parameters and may only be passed used keyword ' + 'arguments.\n' + '\n' + 'Parameters may have annotations of the form "": expression"" ' + 'following\n' + 'the parameter name. Any parameter may have an annotation even ' + 'those\n' + 'of the form "*identifier" or "**identifier". Functions may ' + 'have\n' + '"return" annotation of the form ""-> expression"" after the ' + 'parameter\n' + 'list. These annotations can be any valid Python expression ' + 'and are\n' + 'evaluated when the function definition is executed. ' + 'Annotations may\n' + 'be evaluated in a different order than they appear in the ' + 'source code.\n' + 'The presence of annotations does not change the semantics of ' + 'a\n' + 'function. The annotation values are available as values of a\n' + "dictionary keyed by the parameters' names in the " + '"__annotations__"\n' + 'attribute of the function object.\n' + '\n' + 'It is also possible to create anonymous functions (functions ' + 'not bound\n' + 'to a name), for immediate use in expressions. This uses ' + 'lambda\n' + 'expressions, described in section *Lambdas*. Note that the ' + 'lambda\n' + 'expression is merely a shorthand for a simplified function ' + 'definition;\n' + 'a function defined in a ""def"" statement can be passed around ' + 'or\n' + 'assigned to another name just like a function defined by a ' + 'lambda\n' + 'expression. The ""def"" form is actually more powerful since ' + 'it\n' + 'allows the execution of multiple statements and annotations.\n' + '\n' + "**Programmer's note:** Functions are first-class objects. A " + '""def""\n' + 'statement executed inside a function definition defines a ' + 'local\n' + 'function that can be returned or passed around. Free ' + 'variables used\n' + 'in the nested function can access the local variables of the ' + 'function\n' + 'containing the def. See section *Naming and binding* for ' + 'details.\n' + '\n' + 'See also: **PEP 3107** - Function Annotations\n' + '\n' + ' The original specification for function annotations.\n' + '\n' + '\n' + 'Class definitions\n' + '=================\n' + '\n' + 'A class definition defines a class object (see section *The ' + 'standard\n' + 'type hierarchy*):\n' + '\n' + ' classdef ::= [decorators] "class" classname ' + '[inheritance] ":" suite\n' + ' inheritance ::= "(" [parameter_list] ")"\n' + ' classname ::= identifier\n' + '\n' + 'A class definition is an executable statement. The ' + 'inheritance list\n' + 'usually gives a list of base classes (see *Customizing class ' + 'creation*\n' + 'for more advanced uses), so each item in the list should ' + 'evaluate to a\n' + 'class object which allows subclassing. Classes without an ' + 'inheritance\n' + 'list inherit, by default, from the base class "object"; ' + 'hence,\n' + '\n' + ' class Foo:\n' + ' pass\n' + '\n' + 'is equivalent to\n' + '\n' + ' class Foo(object):\n' + ' pass\n' + '\n' + "The class's suite is then executed in a new execution frame " + '(see\n' + '*Naming and binding*), using a newly created local namespace ' + 'and the\n' + 'original global namespace. (Usually, the suite contains ' + 'mostly\n' + "function definitions.) When the class's suite finishes " + 'execution, its\n' + 'execution frame is discarded but its local namespace is saved. ' + '[4] A\n' + 'class object is then created using the inheritance list for ' + 'the base\n' + 'classes and the saved local namespace for the attribute ' + 'dictionary.\n' + 'The class name is bound to this class object in the original ' + 'local\n' + 'namespace.\n' + '\n' + 'Class creation can be customized heavily using *metaclasses*.\n' + '\n' + 'Classes can also be decorated: just like when decorating ' + 'functions,\n' + '\n' + ' @f1(arg)\n' + ' @f2\n' + ' class Foo: pass\n' + '\n' + 'is equivalent to\n' + '\n' + ' class Foo: pass\n' + ' Foo = f1(arg)(f2(Foo))\n' + '\n' + 'The evaluation rules for the decorator expressions are the ' + 'same as for\n' + 'function decorators. The result must be a class object, which ' + 'is then\n' + 'bound to the class name.\n' + '\n' + "**Programmer's note:** Variables defined in the class " + 'definition are\n' + 'class attributes; they are shared by instances. Instance ' + 'attributes\n' + 'can be set in a method with "self.name = value". Both class ' + 'and\n' + 'instance attributes are accessible through the notation ' + '""self.name"",\n' + 'and an instance attribute hides a class attribute with the ' + 'same name\n' + 'when accessed in this way. Class attributes can be used as ' + 'defaults\n' + 'for instance attributes, but using mutable values there can ' + 'lead to\n' + 'unexpected results. *Descriptors* can be used to create ' + 'instance\n' + 'variables with different implementation details.\n' + '\n' + 'See also: **PEP 3115** - Metaclasses in Python 3 **PEP 3129** ' + '-\n' + ' Class Decorators\n' + '\n' + '\n' + 'Coroutines\n' + '==========\n' + '\n' + 'New in version 3.5.\n' + '\n' + '\n' + 'Coroutine function definition\n' + '-----------------------------\n' + '\n' + ' async_funcdef ::= [decorators] "async" "def" funcname "(" ' + '[parameter_list] ")" ["->" expression] ":" suite\n' + '\n' + 'Execution of Python coroutines can be suspended and resumed at ' + 'many\n' + 'points (see *coroutine*). In the body of a coroutine, any ' + '"await" and\n' + '"async" identifiers become reserved keywords; "await" ' + 'expressions,\n' + '"async for" and "async with" can only be used in coroutine ' + 'bodies.\n' + '\n' + 'Functions defined with "async def" syntax are always ' + 'coroutine\n' + 'functions, even if they do not contain "await" or "async" ' + 'keywords.\n' + '\n' + 'It is a "SyntaxError" to use "yield" expressions in "async ' + 'def"\n' + 'coroutines.\n' + '\n' + 'An example of a coroutine function:\n' + '\n' + ' async def func(param1, param2):\n' + ' do_stuff()\n' + ' await some_coroutine()\n' + '\n' + '\n' + 'The "async for" statement\n' + '-------------------------\n' + '\n' + ' async_for_stmt ::= "async" for_stmt\n' + '\n' + 'An *asynchronous iterable* is able to call asynchronous code ' + 'in its\n' + '*iter* implementation, and *asynchronous iterator* can call\n' + 'asynchronous code in its *next* method.\n' + '\n' + 'The "async for" statement allows convenient iteration over\n' + 'asynchronous iterators.\n' + '\n' + 'The following code:\n' + '\n' + ' async for TARGET in ITER:\n' + ' BLOCK\n' + ' else:\n' + ' BLOCK2\n' + '\n' + 'Is semantically equivalent to:\n' + '\n' + ' iter = (ITER)\n' + ' iter = await type(iter).__aiter__(iter)\n' + ' running = True\n' + ' while running:\n' + ' try:\n' + ' TARGET = await type(iter).__anext__(iter)\n' + ' except StopAsyncIteration:\n' + ' running = False\n' + ' else:\n' + ' BLOCK\n' + ' else:\n' + ' BLOCK2\n' + '\n' + 'See also "__aiter__()" and "__anext__()" for details.\n' + '\n' + 'It is a "SyntaxError" to use "async for" statement outside of ' + 'an\n' + '"async def" function.\n' + '\n' + '\n' + 'The "async with" statement\n' + '--------------------------\n' + '\n' + ' async_with_stmt ::= "async" with_stmt\n' + '\n' + 'An *asynchronous context manager* is a *context manager* that ' + 'is able\n' + 'to suspend execution in its *enter* and *exit* methods.\n' + '\n' + 'The following code:\n' + '\n' + ' async with EXPR as VAR:\n' + ' BLOCK\n' + '\n' + 'Is semantically equivalent to:\n' + '\n' + ' mgr = (EXPR)\n' + ' aexit = type(mgr).__aexit__\n' + ' aenter = type(mgr).__aenter__(mgr)\n' + ' exc = True\n' + '\n' + ' VAR = await aenter\n' + ' try:\n' + ' BLOCK\n' + ' except:\n' + ' if not await aexit(mgr, *sys.exc_info()):\n' + ' raise\n' + ' else:\n' + ' await aexit(mgr, None, None, None)\n' + '\n' + 'See also "__aenter__()" and "__aexit__()" for details.\n' + '\n' + 'It is a "SyntaxError" to use "async with" statement outside of ' + 'an\n' + '"async def" function.\n' + '\n' + 'See also: **PEP 492** - Coroutines with async and await ' + 'syntax\n' + '\n' + '-[ Footnotes ]-\n' + '\n' + '[1] The exception is propagated to the invocation stack ' + 'unless\n' + ' there is a "finally" clause which happens to raise ' + 'another\n' + ' exception. That new exception causes the old one to be ' + 'lost.\n' + '\n' + '[2] Currently, control "flows off the end" except in the case ' + 'of\n' + ' an exception or the execution of a "return", "continue", ' + 'or\n' + ' "break" statement.\n' + '\n' + '[3] A string literal appearing as the first statement in the\n' + " function body is transformed into the function's " + '"__doc__"\n' + " attribute and therefore the function's *docstring*.\n" + '\n' + '[4] A string literal appearing as the first statement in the ' + 'class\n' + ' body is transformed into the namespace\'s "__doc__" item ' + 'and\n' + " therefore the class's *docstring*.\n", + 'context-managers': '\n' + 'With Statement Context Managers\n' + '*******************************\n' + '\n' + 'A *context manager* is an object that defines the ' + 'runtime context to\n' + 'be established when executing a "with" statement. The ' + 'context manager\n' + 'handles the entry into, and the exit from, the desired ' + 'runtime context\n' + 'for the execution of the block of code. Context ' + 'managers are normally\n' + 'invoked using the "with" statement (described in ' + 'section *The with\n' + 'statement*), but can also be used by directly invoking ' + 'their methods.\n' + '\n' + 'Typical uses of context managers include saving and ' + 'restoring various\n' + 'kinds of global state, locking and unlocking ' + 'resources, closing opened\n' + 'files, etc.\n' + '\n' + 'For more information on context managers, see *Context ' + 'Manager Types*.\n' + '\n' + 'object.__enter__(self)\n' + '\n' + ' Enter the runtime context related to this object. ' + 'The "with"\n' + " statement will bind this method's return value to " + 'the target(s)\n' + ' specified in the "as" clause of the statement, if ' + 'any.\n' + '\n' + 'object.__exit__(self, exc_type, exc_value, traceback)\n' + '\n' + ' Exit the runtime context related to this object. ' + 'The parameters\n' + ' describe the exception that caused the context to ' + 'be exited. If the\n' + ' context was exited without an exception, all three ' + 'arguments will\n' + ' be "None".\n' + '\n' + ' If an exception is supplied, and the method wishes ' + 'to suppress the\n' + ' exception (i.e., prevent it from being propagated), ' + 'it should\n' + ' return a true value. Otherwise, the exception will ' + 'be processed\n' + ' normally upon exit from this method.\n' + '\n' + ' Note that "__exit__()" methods should not reraise ' + 'the passed-in\n' + " exception; this is the caller's responsibility.\n" + '\n' + 'See also: **PEP 0343** - The "with" statement\n' + '\n' + ' The specification, background, and examples for ' + 'the Python "with"\n' + ' statement.\n', + 'continue': '\n' + 'The "continue" statement\n' + '************************\n' + '\n' + ' continue_stmt ::= "continue"\n' + '\n' + '"continue" may only occur syntactically nested in a "for" or ' + '"while"\n' + 'loop, but not nested in a function or class definition or ' + '"finally"\n' + 'clause within that loop. It continues with the next cycle of ' + 'the\n' + 'nearest enclosing loop.\n' + '\n' + 'When "continue" passes control out of a "try" statement with ' + 'a\n' + '"finally" clause, that "finally" clause is executed before ' + 'really\n' + 'starting the next loop cycle.\n', + 'conversions': '\n' + 'Arithmetic conversions\n' + '**********************\n' + '\n' + 'When a description of an arithmetic operator below uses the ' + 'phrase\n' + '"the numeric arguments are converted to a common type," ' + 'this means\n' + 'that the operator implementation for built-in types works ' + 'as follows:\n' + '\n' + '* If either argument is a complex number, the other is ' + 'converted to\n' + ' complex;\n' + '\n' + '* otherwise, if either argument is a floating point number, ' + 'the\n' + ' other is converted to floating point;\n' + '\n' + '* otherwise, both must be integers and no conversion is ' + 'necessary.\n' + '\n' + 'Some additional rules apply for certain operators (e.g., a ' + 'string as a\n' + "left argument to the '%' operator). Extensions must define " + 'their own\n' + 'conversion behavior.\n', + 'customization': '\n' + 'Basic customization\n' + '*******************\n' + '\n' + 'object.__new__(cls[, ...])\n' + '\n' + ' Called to create a new instance of class *cls*. ' + '"__new__()" is a\n' + ' static method (special-cased so you need not declare ' + 'it as such)\n' + ' that takes the class of which an instance was ' + 'requested as its\n' + ' first argument. The remaining arguments are those ' + 'passed to the\n' + ' object constructor expression (the call to the ' + 'class). The return\n' + ' value of "__new__()" should be the new object instance ' + '(usually an\n' + ' instance of *cls*).\n' + '\n' + ' Typical implementations create a new instance of the ' + 'class by\n' + ' invoking the superclass\'s "__new__()" method using\n' + ' "super(currentclass, cls).__new__(cls[, ...])" with ' + 'appropriate\n' + ' arguments and then modifying the newly-created ' + 'instance as\n' + ' necessary before returning it.\n' + '\n' + ' If "__new__()" returns an instance of *cls*, then the ' + 'new\n' + ' instance\'s "__init__()" method will be invoked like\n' + ' "__init__(self[, ...])", where *self* is the new ' + 'instance and the\n' + ' remaining arguments are the same as were passed to ' + '"__new__()".\n' + '\n' + ' If "__new__()" does not return an instance of *cls*, ' + 'then the new\n' + ' instance\'s "__init__()" method will not be invoked.\n' + '\n' + ' "__new__()" is intended mainly to allow subclasses of ' + 'immutable\n' + ' types (like int, str, or tuple) to customize instance ' + 'creation. It\n' + ' is also commonly overridden in custom metaclasses in ' + 'order to\n' + ' customize class creation.\n' + '\n' + 'object.__init__(self[, ...])\n' + '\n' + ' Called after the instance has been created (by ' + '"__new__()"), but\n' + ' before it is returned to the caller. The arguments ' + 'are those\n' + ' passed to the class constructor expression. If a base ' + 'class has an\n' + ' "__init__()" method, the derived class\'s "__init__()" ' + 'method, if\n' + ' any, must explicitly call it to ensure proper ' + 'initialization of the\n' + ' base class part of the instance; for example:\n' + ' "BaseClass.__init__(self, [args...])".\n' + '\n' + ' Because "__new__()" and "__init__()" work together in ' + 'constructing\n' + ' objects ("__new__()" to create it, and "__init__()" to ' + 'customise\n' + ' it), no non-"None" value may be returned by ' + '"__init__()"; doing so\n' + ' will cause a "TypeError" to be raised at runtime.\n' + '\n' + 'object.__del__(self)\n' + '\n' + ' Called when the instance is about to be destroyed. ' + 'This is also\n' + ' called a destructor. If a base class has a ' + '"__del__()" method, the\n' + ' derived class\'s "__del__()" method, if any, must ' + 'explicitly call it\n' + ' to ensure proper deletion of the base class part of ' + 'the instance.\n' + ' Note that it is possible (though not recommended!) for ' + 'the\n' + ' "__del__()" method to postpone destruction of the ' + 'instance by\n' + ' creating a new reference to it. It may then be called ' + 'at a later\n' + ' time when this new reference is deleted. It is not ' + 'guaranteed that\n' + ' "__del__()" methods are called for objects that still ' + 'exist when\n' + ' the interpreter exits.\n' + '\n' + ' Note: "del x" doesn\'t directly call "x.__del__()" --- ' + 'the former\n' + ' decrements the reference count for "x" by one, and ' + 'the latter is\n' + ' only called when "x"\'s reference count reaches ' + 'zero. Some common\n' + ' situations that may prevent the reference count of ' + 'an object from\n' + ' going to zero include: circular references between ' + 'objects (e.g.,\n' + ' a doubly-linked list or a tree data structure with ' + 'parent and\n' + ' child pointers); a reference to the object on the ' + 'stack frame of\n' + ' a function that caught an exception (the traceback ' + 'stored in\n' + ' "sys.exc_info()[2]" keeps the stack frame alive); or ' + 'a reference\n' + ' to the object on the stack frame that raised an ' + 'unhandled\n' + ' exception in interactive mode (the traceback stored ' + 'in\n' + ' "sys.last_traceback" keeps the stack frame alive). ' + 'The first\n' + ' situation can only be remedied by explicitly ' + 'breaking the cycles;\n' + ' the second can be resolved by freeing the reference ' + 'to the\n' + ' traceback object when it is no longer useful, and ' + 'the third can\n' + ' be resolved by storing "None" in ' + '"sys.last_traceback". Circular\n' + ' references which are garbage are detected and ' + 'cleaned up when the\n' + " cyclic garbage collector is enabled (it's on by " + 'default). Refer\n' + ' to the documentation for the "gc" module for more ' + 'information\n' + ' about this topic.\n' + '\n' + ' Warning: Due to the precarious circumstances under ' + 'which\n' + ' "__del__()" methods are invoked, exceptions that ' + 'occur during\n' + ' their execution are ignored, and a warning is ' + 'printed to\n' + ' "sys.stderr" instead. Also, when "__del__()" is ' + 'invoked in\n' + ' response to a module being deleted (e.g., when ' + 'execution of the\n' + ' program is done), other globals referenced by the ' + '"__del__()"\n' + ' method may already have been deleted or in the ' + 'process of being\n' + ' torn down (e.g. the import machinery shutting ' + 'down). For this\n' + ' reason, "__del__()" methods should do the absolute ' + 'minimum needed\n' + ' to maintain external invariants. Starting with ' + 'version 1.5,\n' + ' Python guarantees that globals whose name begins ' + 'with a single\n' + ' underscore are deleted from their module before ' + 'other globals are\n' + ' deleted; if no other references to such globals ' + 'exist, this may\n' + ' help in assuring that imported modules are still ' + 'available at the\n' + ' time when the "__del__()" method is called.\n' + '\n' + 'object.__repr__(self)\n' + '\n' + ' Called by the "repr()" built-in function to compute ' + 'the "official"\n' + ' string representation of an object. If at all ' + 'possible, this\n' + ' should look like a valid Python expression that could ' + 'be used to\n' + ' recreate an object with the same value (given an ' + 'appropriate\n' + ' environment). If this is not possible, a string of ' + 'the form\n' + ' "<...some useful description...>" should be returned. ' + 'The return\n' + ' value must be a string object. If a class defines ' + '"__repr__()" but\n' + ' not "__str__()", then "__repr__()" is also used when ' + 'an "informal"\n' + ' string representation of instances of that class is ' + 'required.\n' + '\n' + ' This is typically used for debugging, so it is ' + 'important that the\n' + ' representation is information-rich and unambiguous.\n' + '\n' + 'object.__str__(self)\n' + '\n' + ' Called by "str(object)" and the built-in functions ' + '"format()" and\n' + ' "print()" to compute the "informal" or nicely ' + 'printable string\n' + ' representation of an object. The return value must be ' + 'a *string*\n' + ' object.\n' + '\n' + ' This method differs from "object.__repr__()" in that ' + 'there is no\n' + ' expectation that "__str__()" return a valid Python ' + 'expression: a\n' + ' more convenient or concise representation can be ' + 'used.\n' + '\n' + ' The default implementation defined by the built-in ' + 'type "object"\n' + ' calls "object.__repr__()".\n' + '\n' + 'object.__bytes__(self)\n' + '\n' + ' Called by "bytes()" to compute a byte-string ' + 'representation of an\n' + ' object. This should return a "bytes" object.\n' + '\n' + 'object.__format__(self, format_spec)\n' + '\n' + ' Called by the "format()" built-in function (and by ' + 'extension, the\n' + ' "str.format()" method of class "str") to produce a ' + '"formatted"\n' + ' string representation of an object. The "format_spec" ' + 'argument is a\n' + ' string that contains a description of the formatting ' + 'options\n' + ' desired. The interpretation of the "format_spec" ' + 'argument is up to\n' + ' the type implementing "__format__()", however most ' + 'classes will\n' + ' either delegate formatting to one of the built-in ' + 'types, or use a\n' + ' similar formatting option syntax.\n' + '\n' + ' See *Format Specification Mini-Language* for a ' + 'description of the\n' + ' standard formatting syntax.\n' + '\n' + ' The return value must be a string object.\n' + '\n' + ' Changed in version 3.4: The __format__ method of ' + '"object" itself\n' + ' raises a "TypeError" if passed any non-empty string.\n' + '\n' + 'object.__lt__(self, other)\n' + 'object.__le__(self, other)\n' + 'object.__eq__(self, other)\n' + 'object.__ne__(self, other)\n' + 'object.__gt__(self, other)\n' + 'object.__ge__(self, other)\n' + '\n' + ' These are the so-called "rich comparison" methods. ' + 'The\n' + ' correspondence between operator symbols and method ' + 'names is as\n' + ' follows: "xy" calls\n' + ' "x.__gt__(y)", and "x>=y" calls "x.__ge__(y)".\n' + '\n' + ' A rich comparison method may return the singleton ' + '"NotImplemented"\n' + ' if it does not implement the operation for a given ' + 'pair of\n' + ' arguments. By convention, "False" and "True" are ' + 'returned for a\n' + ' successful comparison. However, these methods can ' + 'return any value,\n' + ' so if the comparison operator is used in a Boolean ' + 'context (e.g.,\n' + ' in the condition of an "if" statement), Python will ' + 'call "bool()"\n' + ' on the value to determine if the result is true or ' + 'false.\n' + '\n' + ' By default, "__ne__()" delegates to "__eq__()" and ' + 'inverts the\n' + ' result unless it is "NotImplemented". There are no ' + 'other implied\n' + ' relationships among the comparison operators, for ' + 'example, the\n' + ' truth of "(x.__hash__".\n' + '\n' + ' If a class that does not override "__eq__()" wishes to ' + 'suppress\n' + ' hash support, it should include "__hash__ = None" in ' + 'the class\n' + ' definition. A class which defines its own "__hash__()" ' + 'that\n' + ' explicitly raises a "TypeError" would be incorrectly ' + 'identified as\n' + ' hashable by an "isinstance(obj, collections.Hashable)" ' + 'call.\n' + '\n' + ' Note: By default, the "__hash__()" values of str, ' + 'bytes and\n' + ' datetime objects are "salted" with an unpredictable ' + 'random value.\n' + ' Although they remain constant within an individual ' + 'Python\n' + ' process, they are not predictable between repeated ' + 'invocations of\n' + ' Python.This is intended to provide protection ' + 'against a denial-\n' + ' of-service caused by carefully-chosen inputs that ' + 'exploit the\n' + ' worst case performance of a dict insertion, O(n^2) ' + 'complexity.\n' + ' See ' + 'http://www.ocert.org/advisories/ocert-2011-003.html for\n' + ' details.Changing hash values affects the iteration ' + 'order of\n' + ' dicts, sets and other mappings. Python has never ' + 'made guarantees\n' + ' about this ordering (and it typically varies between ' + '32-bit and\n' + ' 64-bit builds).See also "PYTHONHASHSEED".\n' + '\n' + ' Changed in version 3.3: Hash randomization is enabled ' + 'by default.\n' + '\n' + 'object.__bool__(self)\n' + '\n' + ' Called to implement truth value testing and the ' + 'built-in operation\n' + ' "bool()"; should return "False" or "True". When this ' + 'method is not\n' + ' defined, "__len__()" is called, if it is defined, and ' + 'the object is\n' + ' considered true if its result is nonzero. If a class ' + 'defines\n' + ' neither "__len__()" nor "__bool__()", all its ' + 'instances are\n' + ' considered true.\n', + 'debugger': '\n' + '"pdb" --- The Python Debugger\n' + '*****************************\n' + '\n' + '**Source code:** Lib/pdb.py\n' + '\n' + '======================================================================\n' + '\n' + 'The module "pdb" defines an interactive source code debugger ' + 'for\n' + 'Python programs. It supports setting (conditional) ' + 'breakpoints and\n' + 'single stepping at the source line level, inspection of stack ' + 'frames,\n' + 'source code listing, and evaluation of arbitrary Python code ' + 'in the\n' + 'context of any stack frame. It also supports post-mortem ' + 'debugging\n' + 'and can be called under program control.\n' + '\n' + 'The debugger is extensible -- it is actually defined as the ' + 'class\n' + '"Pdb". This is currently undocumented but easily understood by ' + 'reading\n' + 'the source. The extension interface uses the modules "bdb" ' + 'and "cmd".\n' + '\n' + 'The debugger\'s prompt is "(Pdb)". Typical usage to run a ' + 'program under\n' + 'control of the debugger is:\n' + '\n' + ' >>> import pdb\n' + ' >>> import mymodule\n' + " >>> pdb.run('mymodule.test()')\n" + ' > (0)?()\n' + ' (Pdb) continue\n' + ' > (1)?()\n' + ' (Pdb) continue\n' + " NameError: 'spam'\n" + ' > (1)?()\n' + ' (Pdb)\n' + '\n' + 'Changed in version 3.3: Tab-completion via the "readline" ' + 'module is\n' + 'available for commands and command arguments, e.g. the current ' + 'global\n' + 'and local names are offered as arguments of the "p" command.\n' + '\n' + '"pdb.py" can also be invoked as a script to debug other ' + 'scripts. For\n' + 'example:\n' + '\n' + ' python3 -m pdb myscript.py\n' + '\n' + 'When invoked as a script, pdb will automatically enter ' + 'post-mortem\n' + 'debugging if the program being debugged exits abnormally. ' + 'After post-\n' + 'mortem debugging (or after normal exit of the program), pdb ' + 'will\n' + "restart the program. Automatic restarting preserves pdb's " + 'state (such\n' + 'as breakpoints) and in most cases is more useful than quitting ' + 'the\n' + "debugger upon program's exit.\n" + '\n' + 'New in version 3.2: "pdb.py" now accepts a "-c" option that ' + 'executes\n' + 'commands as if given in a ".pdbrc" file, see *Debugger ' + 'Commands*.\n' + '\n' + 'The typical usage to break into the debugger from a running ' + 'program is\n' + 'to insert\n' + '\n' + ' import pdb; pdb.set_trace()\n' + '\n' + 'at the location you want to break into the debugger. You can ' + 'then\n' + 'step through the code following this statement, and continue ' + 'running\n' + 'without the debugger using the "continue" command.\n' + '\n' + 'The typical usage to inspect a crashed program is:\n' + '\n' + ' >>> import pdb\n' + ' >>> import mymodule\n' + ' >>> mymodule.test()\n' + ' Traceback (most recent call last):\n' + ' File "", line 1, in ?\n' + ' File "./mymodule.py", line 4, in test\n' + ' test2()\n' + ' File "./mymodule.py", line 3, in test2\n' + ' print(spam)\n' + ' NameError: spam\n' + ' >>> pdb.pm()\n' + ' > ./mymodule.py(3)test2()\n' + ' -> print(spam)\n' + ' (Pdb)\n' + '\n' + 'The module defines the following functions; each enters the ' + 'debugger\n' + 'in a slightly different way:\n' + '\n' + 'pdb.run(statement, globals=None, locals=None)\n' + '\n' + ' Execute the *statement* (given as a string or a code ' + 'object) under\n' + ' debugger control. The debugger prompt appears before any ' + 'code is\n' + ' executed; you can set breakpoints and type "continue", or ' + 'you can\n' + ' step through the statement using "step" or "next" (all ' + 'these\n' + ' commands are explained below). The optional *globals* and ' + '*locals*\n' + ' arguments specify the environment in which the code is ' + 'executed; by\n' + ' default the dictionary of the module "__main__" is used. ' + '(See the\n' + ' explanation of the built-in "exec()" or "eval()" ' + 'functions.)\n' + '\n' + 'pdb.runeval(expression, globals=None, locals=None)\n' + '\n' + ' Evaluate the *expression* (given as a string or a code ' + 'object)\n' + ' under debugger control. When "runeval()" returns, it ' + 'returns the\n' + ' value of the expression. Otherwise this function is ' + 'similar to\n' + ' "run()".\n' + '\n' + 'pdb.runcall(function, *args, **kwds)\n' + '\n' + ' Call the *function* (a function or method object, not a ' + 'string)\n' + ' with the given arguments. When "runcall()" returns, it ' + 'returns\n' + ' whatever the function call returned. The debugger prompt ' + 'appears\n' + ' as soon as the function is entered.\n' + '\n' + 'pdb.set_trace()\n' + '\n' + ' Enter the debugger at the calling stack frame. This is ' + 'useful to\n' + ' hard-code a breakpoint at a given point in a program, even ' + 'if the\n' + ' code is not otherwise being debugged (e.g. when an ' + 'assertion\n' + ' fails).\n' + '\n' + 'pdb.post_mortem(traceback=None)\n' + '\n' + ' Enter post-mortem debugging of the given *traceback* ' + 'object. If no\n' + ' *traceback* is given, it uses the one of the exception that ' + 'is\n' + ' currently being handled (an exception must be being handled ' + 'if the\n' + ' default is to be used).\n' + '\n' + 'pdb.pm()\n' + '\n' + ' Enter post-mortem debugging of the traceback found in\n' + ' "sys.last_traceback".\n' + '\n' + 'The "run*" functions and "set_trace()" are aliases for ' + 'instantiating\n' + 'the "Pdb" class and calling the method of the same name. If ' + 'you want\n' + 'to access further features, you have to do this yourself:\n' + '\n' + "class class pdb.Pdb(completekey='tab', stdin=None, " + 'stdout=None, skip=None, nosigint=False)\n' + '\n' + ' "Pdb" is the debugger class.\n' + '\n' + ' The *completekey*, *stdin* and *stdout* arguments are ' + 'passed to the\n' + ' underlying "cmd.Cmd" class; see the description there.\n' + '\n' + ' The *skip* argument, if given, must be an iterable of ' + 'glob-style\n' + ' module name patterns. The debugger will not step into ' + 'frames that\n' + ' originate in a module that matches one of these patterns. ' + '[1]\n' + '\n' + ' By default, Pdb sets a handler for the SIGINT signal (which ' + 'is sent\n' + ' when the user presses Ctrl-C on the console) when you give ' + 'a\n' + ' "continue" command. This allows you to break into the ' + 'debugger\n' + ' again by pressing Ctrl-C. If you want Pdb not to touch the ' + 'SIGINT\n' + ' handler, set *nosigint* tot true.\n' + '\n' + ' Example call to enable tracing with *skip*:\n' + '\n' + " import pdb; pdb.Pdb(skip=['django.*']).set_trace()\n" + '\n' + ' New in version 3.1: The *skip* argument.\n' + '\n' + ' New in version 3.2: The *nosigint* argument. Previously, a ' + 'SIGINT\n' + ' handler was never set by Pdb.\n' + '\n' + ' run(statement, globals=None, locals=None)\n' + ' runeval(expression, globals=None, locals=None)\n' + ' runcall(function, *args, **kwds)\n' + ' set_trace()\n' + '\n' + ' See the documentation for the functions explained ' + 'above.\n' + '\n' + '\n' + 'Debugger Commands\n' + '=================\n' + '\n' + 'The commands recognized by the debugger are listed below. ' + 'Most\n' + 'commands can be abbreviated to one or two letters as ' + 'indicated; e.g.\n' + '"h(elp)" means that either "h" or "help" can be used to enter ' + 'the help\n' + 'command (but not "he" or "hel", nor "H" or "Help" or "HELP").\n' + 'Arguments to commands must be separated by whitespace (spaces ' + 'or\n' + 'tabs). Optional arguments are enclosed in square brackets ' + '("[]") in\n' + 'the command syntax; the square brackets must not be typed.\n' + 'Alternatives in the command syntax are separated by a vertical ' + 'bar\n' + '("|").\n' + '\n' + 'Entering a blank line repeats the last command entered. ' + 'Exception: if\n' + 'the last command was a "list" command, the next 11 lines are ' + 'listed.\n' + '\n' + "Commands that the debugger doesn't recognize are assumed to be " + 'Python\n' + 'statements and are executed in the context of the program ' + 'being\n' + 'debugged. Python statements can also be prefixed with an ' + 'exclamation\n' + 'point ("!"). This is a powerful way to inspect the program ' + 'being\n' + 'debugged; it is even possible to change a variable or call a ' + 'function.\n' + 'When an exception occurs in such a statement, the exception ' + 'name is\n' + "printed but the debugger's state is not changed.\n" + '\n' + 'The debugger supports *aliases*. Aliases can have parameters ' + 'which\n' + 'allows one a certain level of adaptability to the context ' + 'under\n' + 'examination.\n' + '\n' + 'Multiple commands may be entered on a single line, separated ' + 'by ";;".\n' + '(A single ";" is not used as it is the separator for multiple ' + 'commands\n' + 'in a line that is passed to the Python parser.) No ' + 'intelligence is\n' + 'applied to separating the commands; the input is split at the ' + 'first\n' + '";;" pair, even if it is in the middle of a quoted string.\n' + '\n' + 'If a file ".pdbrc" exists in the user\'s home directory or in ' + 'the\n' + 'current directory, it is read in and executed as if it had ' + 'been typed\n' + 'at the debugger prompt. This is particularly useful for ' + 'aliases. If\n' + 'both files exist, the one in the home directory is read first ' + 'and\n' + 'aliases defined there can be overridden by the local file.\n' + '\n' + 'Changed in version 3.2: ".pdbrc" can now contain commands ' + 'that\n' + 'continue debugging, such as "continue" or "next". Previously, ' + 'these\n' + 'commands had no effect.\n' + '\n' + 'h(elp) [command]\n' + '\n' + ' Without argument, print the list of available commands. ' + 'With a\n' + ' *command* as argument, print help about that command. ' + '"help pdb"\n' + ' displays the full documentation (the docstring of the ' + '"pdb"\n' + ' module). Since the *command* argument must be an ' + 'identifier, "help\n' + ' exec" must be entered to get help on the "!" command.\n' + '\n' + 'w(here)\n' + '\n' + ' Print a stack trace, with the most recent frame at the ' + 'bottom. An\n' + ' arrow indicates the current frame, which determines the ' + 'context of\n' + ' most commands.\n' + '\n' + 'd(own) [count]\n' + '\n' + ' Move the current frame *count* (default one) levels down in ' + 'the\n' + ' stack trace (to a newer frame).\n' + '\n' + 'u(p) [count]\n' + '\n' + ' Move the current frame *count* (default one) levels up in ' + 'the stack\n' + ' trace (to an older frame).\n' + '\n' + 'b(reak) [([filename:]lineno | function) [, condition]]\n' + '\n' + ' With a *lineno* argument, set a break there in the current ' + 'file.\n' + ' With a *function* argument, set a break at the first ' + 'executable\n' + ' statement within that function. The line number may be ' + 'prefixed\n' + ' with a filename and a colon, to specify a breakpoint in ' + 'another\n' + " file (probably one that hasn't been loaded yet). The file " + 'is\n' + ' searched on "sys.path". Note that each breakpoint is ' + 'assigned a\n' + ' number to which all the other breakpoint commands refer.\n' + '\n' + ' If a second argument is present, it is an expression which ' + 'must\n' + ' evaluate to true before the breakpoint is honored.\n' + '\n' + ' Without argument, list all breaks, including for each ' + 'breakpoint,\n' + ' the number of times that breakpoint has been hit, the ' + 'current\n' + ' ignore count, and the associated condition if any.\n' + '\n' + 'tbreak [([filename:]lineno | function) [, condition]]\n' + '\n' + ' Temporary breakpoint, which is removed automatically when ' + 'it is\n' + ' first hit. The arguments are the same as for "break".\n' + '\n' + 'cl(ear) [filename:lineno | bpnumber [bpnumber ...]]\n' + '\n' + ' With a *filename:lineno* argument, clear all the ' + 'breakpoints at\n' + ' this line. With a space separated list of breakpoint ' + 'numbers, clear\n' + ' those breakpoints. Without argument, clear all breaks (but ' + 'first\n' + ' ask confirmation).\n' + '\n' + 'disable [bpnumber [bpnumber ...]]\n' + '\n' + ' Disable the breakpoints given as a space separated list of\n' + ' breakpoint numbers. Disabling a breakpoint means it cannot ' + 'cause\n' + ' the program to stop execution, but unlike clearing a ' + 'breakpoint, it\n' + ' remains in the list of breakpoints and can be ' + '(re-)enabled.\n' + '\n' + 'enable [bpnumber [bpnumber ...]]\n' + '\n' + ' Enable the breakpoints specified.\n' + '\n' + 'ignore bpnumber [count]\n' + '\n' + ' Set the ignore count for the given breakpoint number. If ' + 'count is\n' + ' omitted, the ignore count is set to 0. A breakpoint ' + 'becomes active\n' + ' when the ignore count is zero. When non-zero, the count ' + 'is\n' + ' decremented each time the breakpoint is reached and the ' + 'breakpoint\n' + ' is not disabled and any associated condition evaluates to ' + 'true.\n' + '\n' + 'condition bpnumber [condition]\n' + '\n' + ' Set a new *condition* for the breakpoint, an expression ' + 'which must\n' + ' evaluate to true before the breakpoint is honored. If ' + '*condition*\n' + ' is absent, any existing condition is removed; i.e., the ' + 'breakpoint\n' + ' is made unconditional.\n' + '\n' + 'commands [bpnumber]\n' + '\n' + ' Specify a list of commands for breakpoint number ' + '*bpnumber*. The\n' + ' commands themselves appear on the following lines. Type a ' + 'line\n' + ' containing just "end" to terminate the commands. An ' + 'example:\n' + '\n' + ' (Pdb) commands 1\n' + ' (com) p some_variable\n' + ' (com) end\n' + ' (Pdb)\n' + '\n' + ' To remove all commands from a breakpoint, type commands and ' + 'follow\n' + ' it immediately with "end"; that is, give no commands.\n' + '\n' + ' With no *bpnumber* argument, commands refers to the last ' + 'breakpoint\n' + ' set.\n' + '\n' + ' You can use breakpoint commands to start your program up ' + 'again.\n' + ' Simply use the continue command, or step, or any other ' + 'command that\n' + ' resumes execution.\n' + '\n' + ' Specifying any command resuming execution (currently ' + 'continue,\n' + ' step, next, return, jump, quit and their abbreviations) ' + 'terminates\n' + ' the command list (as if that command was immediately ' + 'followed by\n' + ' end). This is because any time you resume execution (even ' + 'with a\n' + ' simple next or step), you may encounter another ' + 'breakpoint--which\n' + ' could have its own command list, leading to ambiguities ' + 'about which\n' + ' list to execute.\n' + '\n' + " If you use the 'silent' command in the command list, the " + 'usual\n' + ' message about stopping at a breakpoint is not printed. ' + 'This may be\n' + ' desirable for breakpoints that are to print a specific ' + 'message and\n' + ' then continue. If none of the other commands print ' + 'anything, you\n' + ' see no sign that the breakpoint was reached.\n' + '\n' + 's(tep)\n' + '\n' + ' Execute the current line, stop at the first possible ' + 'occasion\n' + ' (either in a function that is called or on the next line in ' + 'the\n' + ' current function).\n' + '\n' + 'n(ext)\n' + '\n' + ' Continue execution until the next line in the current ' + 'function is\n' + ' reached or it returns. (The difference between "next" and ' + '"step"\n' + ' is that "step" stops inside a called function, while ' + '"next"\n' + ' executes called functions at (nearly) full speed, only ' + 'stopping at\n' + ' the next line in the current function.)\n' + '\n' + 'unt(il) [lineno]\n' + '\n' + ' Without argument, continue execution until the line with a ' + 'number\n' + ' greater than the current one is reached.\n' + '\n' + ' With a line number, continue execution until a line with a ' + 'number\n' + ' greater or equal to that is reached. In both cases, also ' + 'stop when\n' + ' the current frame returns.\n' + '\n' + ' Changed in version 3.2: Allow giving an explicit line ' + 'number.\n' + '\n' + 'r(eturn)\n' + '\n' + ' Continue execution until the current function returns.\n' + '\n' + 'c(ont(inue))\n' + '\n' + ' Continue execution, only stop when a breakpoint is ' + 'encountered.\n' + '\n' + 'j(ump) lineno\n' + '\n' + ' Set the next line that will be executed. Only available in ' + 'the\n' + ' bottom-most frame. This lets you jump back and execute ' + 'code again,\n' + " or jump forward to skip code that you don't want to run.\n" + '\n' + ' It should be noted that not all jumps are allowed -- for ' + 'instance\n' + ' it is not possible to jump into the middle of a "for" loop ' + 'or out\n' + ' of a "finally" clause.\n' + '\n' + 'l(ist) [first[, last]]\n' + '\n' + ' List source code for the current file. Without arguments, ' + 'list 11\n' + ' lines around the current line or continue the previous ' + 'listing.\n' + ' With "." as argument, list 11 lines around the current ' + 'line. With\n' + ' one argument, list 11 lines around at that line. With two\n' + ' arguments, list the given range; if the second argument is ' + 'less\n' + ' than the first, it is interpreted as a count.\n' + '\n' + ' The current line in the current frame is indicated by ' + '"->". If an\n' + ' exception is being debugged, the line where the exception ' + 'was\n' + ' originally raised or propagated is indicated by ">>", if it ' + 'differs\n' + ' from the current line.\n' + '\n' + ' New in version 3.2: The ">>" marker.\n' + '\n' + 'll | longlist\n' + '\n' + ' List all source code for the current function or frame.\n' + ' Interesting lines are marked as for "list".\n' + '\n' + ' New in version 3.2.\n' + '\n' + 'a(rgs)\n' + '\n' + ' Print the argument list of the current function.\n' + '\n' + 'p expression\n' + '\n' + ' Evaluate the *expression* in the current context and print ' + 'its\n' + ' value.\n' + '\n' + ' Note: "print()" can also be used, but is not a debugger ' + 'command\n' + ' --- this executes the Python "print()" function.\n' + '\n' + 'pp expression\n' + '\n' + ' Like the "p" command, except the value of the expression is ' + 'pretty-\n' + ' printed using the "pprint" module.\n' + '\n' + 'whatis expression\n' + '\n' + ' Print the type of the *expression*.\n' + '\n' + 'source expression\n' + '\n' + ' Try to get source code for the given object and display ' + 'it.\n' + '\n' + ' New in version 3.2.\n' + '\n' + 'display [expression]\n' + '\n' + ' Display the value of the expression if it changed, each ' + 'time\n' + ' execution stops in the current frame.\n' + '\n' + ' Without expression, list all display expressions for the ' + 'current\n' + ' frame.\n' + '\n' + ' New in version 3.2.\n' + '\n' + 'undisplay [expression]\n' + '\n' + ' Do not display the expression any more in the current ' + 'frame.\n' + ' Without expression, clear all display expressions for the ' + 'current\n' + ' frame.\n' + '\n' + ' New in version 3.2.\n' + '\n' + 'interact\n' + '\n' + ' Start an interative interpreter (using the "code" module) ' + 'whose\n' + ' global namespace contains all the (global and local) names ' + 'found in\n' + ' the current scope.\n' + '\n' + ' New in version 3.2.\n' + '\n' + 'alias [name [command]]\n' + '\n' + ' Create an alias called *name* that executes *command*. The ' + 'command\n' + ' must *not* be enclosed in quotes. Replaceable parameters ' + 'can be\n' + ' indicated by "%1", "%2", and so on, while "%*" is replaced ' + 'by all\n' + ' the parameters. If no command is given, the current alias ' + 'for\n' + ' *name* is shown. If no arguments are given, all aliases are ' + 'listed.\n' + '\n' + ' Aliases may be nested and can contain anything that can be ' + 'legally\n' + ' typed at the pdb prompt. Note that internal pdb commands ' + '*can* be\n' + ' overridden by aliases. Such a command is then hidden until ' + 'the\n' + ' alias is removed. Aliasing is recursively applied to the ' + 'first\n' + ' word of the command line; all other words in the line are ' + 'left\n' + ' alone.\n' + '\n' + ' As an example, here are two useful aliases (especially when ' + 'placed\n' + ' in the ".pdbrc" file):\n' + '\n' + ' # Print instance variables (usage "pi classInst")\n' + ' alias pi for k in %1.__dict__.keys(): ' + 'print("%1.",k,"=",%1.__dict__[k])\n' + ' # Print instance variables in self\n' + ' alias ps pi self\n' + '\n' + 'unalias name\n' + '\n' + ' Delete the specified alias.\n' + '\n' + '! statement\n' + '\n' + ' Execute the (one-line) *statement* in the context of the ' + 'current\n' + ' stack frame. The exclamation point can be omitted unless ' + 'the first\n' + ' word of the statement resembles a debugger command. To set ' + 'a\n' + ' global variable, you can prefix the assignment command with ' + 'a\n' + ' "global" statement on the same line, e.g.:\n' + '\n' + " (Pdb) global list_options; list_options = ['-l']\n" + ' (Pdb)\n' + '\n' + 'run [args ...]\n' + 'restart [args ...]\n' + '\n' + ' Restart the debugged Python program. If an argument is ' + 'supplied,\n' + ' it is split with "shlex" and the result is used as the new\n' + ' "sys.argv". History, breakpoints, actions and debugger ' + 'options are\n' + ' preserved. "restart" is an alias for "run".\n' + '\n' + 'q(uit)\n' + '\n' + ' Quit from the debugger. The program being executed is ' + 'aborted.\n' + '\n' + '-[ Footnotes ]-\n' + '\n' + '[1] Whether a frame is considered to originate in a certain ' + 'module\n' + ' is determined by the "__name__" in the frame globals.\n', + 'del': '\n' + 'The "del" statement\n' + '*******************\n' + '\n' + ' del_stmt ::= "del" target_list\n' + '\n' + 'Deletion is recursively defined very similar to the way assignment ' + 'is\n' + 'defined. Rather than spelling it out in full details, here are ' + 'some\n' + 'hints.\n' + '\n' + 'Deletion of a target list recursively deletes each target, from ' + 'left\n' + 'to right.\n' + '\n' + 'Deletion of a name removes the binding of that name from the local ' + 'or\n' + 'global namespace, depending on whether the name occurs in a ' + '"global"\n' + 'statement in the same code block. If the name is unbound, a\n' + '"NameError" exception will be raised.\n' + '\n' + 'Deletion of attribute references, subscriptions and slicings is ' + 'passed\n' + 'to the primary object involved; deletion of a slicing is in ' + 'general\n' + 'equivalent to assignment of an empty slice of the right type (but ' + 'even\n' + 'this is determined by the sliced object).\n' + '\n' + 'Changed in version 3.2: Previously it was illegal to delete a name\n' + 'from the local namespace if it occurs as a free variable in a ' + 'nested\n' + 'block.\n', + 'dict': '\n' + 'Dictionary displays\n' + '*******************\n' + '\n' + 'A dictionary display is a possibly empty series of key/datum ' + 'pairs\n' + 'enclosed in curly braces:\n' + '\n' + ' dict_display ::= "{" [key_datum_list | ' + 'dict_comprehension] "}"\n' + ' key_datum_list ::= key_datum ("," key_datum)* [","]\n' + ' key_datum ::= expression ":" expression\n' + ' dict_comprehension ::= expression ":" expression comp_for\n' + '\n' + 'A dictionary display yields a new dictionary object.\n' + '\n' + 'If a comma-separated sequence of key/datum pairs is given, they ' + 'are\n' + 'evaluated from left to right to define the entries of the ' + 'dictionary:\n' + 'each key object is used as a key into the dictionary to store the\n' + 'corresponding datum. This means that you can specify the same ' + 'key\n' + "multiple times in the key/datum list, and the final dictionary's " + 'value\n' + 'for that key will be the last one given.\n' + '\n' + 'A dict comprehension, in contrast to list and set comprehensions,\n' + 'needs two expressions separated with a colon followed by the ' + 'usual\n' + '"for" and "if" clauses. When the comprehension is run, the ' + 'resulting\n' + 'key and value elements are inserted in the new dictionary in the ' + 'order\n' + 'they are produced.\n' + '\n' + 'Restrictions on the types of the key values are listed earlier in\n' + 'section *The standard type hierarchy*. (To summarize, the key ' + 'type\n' + 'should be *hashable*, which excludes all mutable objects.) ' + 'Clashes\n' + 'between duplicate keys are not detected; the last datum ' + '(textually\n' + 'rightmost in the display) stored for a given key value prevails.\n', + 'dynamic-features': '\n' + 'Interaction with dynamic features\n' + '*********************************\n' + '\n' + 'Name resolution of free variables occurs at runtime, ' + 'not at compile\n' + 'time. This means that the following code will print ' + '42:\n' + '\n' + ' i = 10\n' + ' def f():\n' + ' print(i)\n' + ' i = 42\n' + ' f()\n' + '\n' + 'There are several cases where Python statements are ' + 'illegal when used\n' + 'in conjunction with nested scopes that contain free ' + 'variables.\n' + '\n' + 'If a variable is referenced in an enclosing scope, it ' + 'is illegal to\n' + 'delete the name. An error will be reported at compile ' + 'time.\n' + '\n' + 'The "eval()" and "exec()" functions do not have access ' + 'to the full\n' + 'environment for resolving names. Names may be ' + 'resolved in the local\n' + 'and global namespaces of the caller. Free variables ' + 'are not resolved\n' + 'in the nearest enclosing namespace, but in the global ' + 'namespace. [1]\n' + 'The "exec()" and "eval()" functions have optional ' + 'arguments to\n' + 'override the global and local namespace. If only one ' + 'namespace is\n' + 'specified, it is used for both.\n', + 'else': '\n' + 'The "if" statement\n' + '******************\n' + '\n' + 'The "if" statement is used for conditional execution:\n' + '\n' + ' if_stmt ::= "if" expression ":" suite\n' + ' ( "elif" expression ":" suite )*\n' + ' ["else" ":" suite]\n' + '\n' + 'It selects exactly one of the suites by evaluating the expressions ' + 'one\n' + 'by one until one is found to be true (see section *Boolean ' + 'operations*\n' + 'for the definition of true and false); then that suite is ' + 'executed\n' + '(and no other part of the "if" statement is executed or ' + 'evaluated).\n' + 'If all expressions are false, the suite of the "else" clause, if\n' + 'present, is executed.\n', + 'exceptions': '\n' + 'Exceptions\n' + '**********\n' + '\n' + 'Exceptions are a means of breaking out of the normal flow of ' + 'control\n' + 'of a code block in order to handle errors or other ' + 'exceptional\n' + 'conditions. An exception is *raised* at the point where the ' + 'error is\n' + 'detected; it may be *handled* by the surrounding code block ' + 'or by any\n' + 'code block that directly or indirectly invoked the code ' + 'block where\n' + 'the error occurred.\n' + '\n' + 'The Python interpreter raises an exception when it detects a ' + 'run-time\n' + 'error (such as division by zero). A Python program can ' + 'also\n' + 'explicitly raise an exception with the "raise" statement. ' + 'Exception\n' + 'handlers are specified with the "try" ... "except" ' + 'statement. The\n' + '"finally" clause of such a statement can be used to specify ' + 'cleanup\n' + 'code which does not handle the exception, but is executed ' + 'whether an\n' + 'exception occurred or not in the preceding code.\n' + '\n' + 'Python uses the "termination" model of error handling: an ' + 'exception\n' + 'handler can find out what happened and continue execution at ' + 'an outer\n' + 'level, but it cannot repair the cause of the error and retry ' + 'the\n' + 'failing operation (except by re-entering the offending piece ' + 'of code\n' + 'from the top).\n' + '\n' + 'When an exception is not handled at all, the interpreter ' + 'terminates\n' + 'execution of the program, or returns to its interactive main ' + 'loop. In\n' + 'either case, it prints a stack backtrace, except when the ' + 'exception is\n' + '"SystemExit".\n' + '\n' + 'Exceptions are identified by class instances. The "except" ' + 'clause is\n' + 'selected depending on the class of the instance: it must ' + 'reference the\n' + 'class of the instance or a base class thereof. The instance ' + 'can be\n' + 'received by the handler and can carry additional information ' + 'about the\n' + 'exceptional condition.\n' + '\n' + 'Note: Exception messages are not part of the Python API. ' + 'Their\n' + ' contents may change from one version of Python to the next ' + 'without\n' + ' warning and should not be relied on by code which will run ' + 'under\n' + ' multiple versions of the interpreter.\n' + '\n' + 'See also the description of the "try" statement in section ' + '*The try\n' + 'statement* and "raise" statement in section *The raise ' + 'statement*.\n' + '\n' + '-[ Footnotes ]-\n' + '\n' + '[1] This limitation occurs because the code that is executed ' + 'by\n' + ' these operations is not available at the time the module ' + 'is\n' + ' compiled.\n', + 'execmodel': '\n' + 'Execution model\n' + '***************\n' + '\n' + '\n' + 'Structure of a programm\n' + '=======================\n' + '\n' + 'A Python program is constructed from code blocks. A *block* ' + 'is a piece\n' + 'of Python program text that is executed as a unit. The ' + 'following are\n' + 'blocks: a module, a function body, and a class definition. ' + 'Each\n' + 'command typed interactively is a block. A script file (a ' + 'file given\n' + 'as standard input to the interpreter or specified as a ' + 'command line\n' + 'argument to the interpreter) is a code block. A script ' + 'command (a\n' + 'command specified on the interpreter command line with the ' + "'**-c**'\n" + 'option) is a code block. The string argument passed to the ' + 'built-in\n' + 'functions "eval()" and "exec()" is a code block.\n' + '\n' + 'A code block is executed in an *execution frame*. A frame ' + 'contains\n' + 'some administrative information (used for debugging) and ' + 'determines\n' + "where and how execution continues after the code block's " + 'execution has\n' + 'completed.\n' + '\n' + '\n' + 'Naming and binding\n' + '==================\n' + '\n' + '\n' + 'Binding of names\n' + '----------------\n' + '\n' + '*Names* refer to objects. Names are introduced by name ' + 'binding\n' + 'operations.\n' + '\n' + 'The following constructs bind names: formal parameters to ' + 'functions,\n' + '"import" statements, class and function definitions (these ' + 'bind the\n' + 'class or function name in the defining block), and targets ' + 'that are\n' + 'identifiers if occurring in an assignment, "for" loop header, ' + 'or after\n' + '"as" in a "with" statement or "except" clause. The "import" ' + 'statement\n' + 'of the form "from ... import *" binds all names defined in ' + 'the\n' + 'imported module, except those beginning with an underscore. ' + 'This form\n' + 'may only be used at the module level.\n' + '\n' + 'A target occurring in a "del" statement is also considered ' + 'bound for\n' + 'this purpose (though the actual semantics are to unbind the ' + 'name).\n' + '\n' + 'Each assignment or import statement occurs within a block ' + 'defined by a\n' + 'class or function definition or at the module level (the ' + 'top-level\n' + 'code block).\n' + '\n' + 'If a name is bound in a block, it is a local variable of that ' + 'block,\n' + 'unless declared as "nonlocal" or "global". If a name is ' + 'bound at the\n' + 'module level, it is a global variable. (The variables of the ' + 'module\n' + 'code block are local and global.) If a variable is used in a ' + 'code\n' + 'block but not defined there, it is a *free variable*.\n' + '\n' + 'Each occurrence of a name in the program text refers to the ' + '*binding*\n' + 'of that name established by the following name resolution ' + 'rules.\n' + '\n' + '\n' + 'Resolution of names\n' + '-------------------\n' + '\n' + 'A *scope* defines the visibility of a name within a block. ' + 'If a local\n' + 'variable is defined in a block, its scope includes that ' + 'block. If the\n' + 'definition occurs in a function block, the scope extends to ' + 'any blocks\n' + 'contained within the defining one, unless a contained block ' + 'introduces\n' + 'a different binding for the name.\n' + '\n' + 'When a name is used in a code block, it is resolved using the ' + 'nearest\n' + 'enclosing scope. The set of all such scopes visible to a ' + 'code block\n' + "is called the block's *environment*.\n" + '\n' + 'When a name is not found at all, a "NameError" exception is ' + 'raised. If\n' + 'the current scope is a function scope, and the name refers to ' + 'a local\n' + 'variable that has not yet been bound to a value at the point ' + 'where the\n' + 'name is used, an "UnboundLocalError" exception is raised.\n' + '"UnboundLocalError" is a subclass of "NameError".\n' + '\n' + 'If a name binding operation occurs anywhere within a code ' + 'block, all\n' + 'uses of the name within the block are treated as references ' + 'to the\n' + 'current block. This can lead to errors when a name is used ' + 'within a\n' + 'block before it is bound. This rule is subtle. Python ' + 'lacks\n' + 'declarations and allows name binding operations to occur ' + 'anywhere\n' + 'within a code block. The local variables of a code block can ' + 'be\n' + 'determined by scanning the entire text of the block for name ' + 'binding\n' + 'operations.\n' + '\n' + 'If the "global" statement occurs within a block, all uses of ' + 'the name\n' + 'specified in the statement refer to the binding of that name ' + 'in the\n' + 'top-level namespace. Names are resolved in the top-level ' + 'namespace by\n' + 'searching the global namespace, i.e. the namespace of the ' + 'module\n' + 'containing the code block, and the builtins namespace, the ' + 'namespace\n' + 'of the module "builtins". The global namespace is searched ' + 'first. If\n' + 'the name is not found there, the builtins namespace is ' + 'searched. The\n' + '"global" statement must precede all uses of the name.\n' + '\n' + 'The "global" statement has the same scope as a name binding ' + 'operation\n' + 'in the same block. If the nearest enclosing scope for a free ' + 'variable\n' + 'contains a global statement, the free variable is treated as ' + 'a global.\n' + '\n' + 'The "nonlocal" statement causes corresponding names to refer ' + 'to\n' + 'previously bound variables in the nearest enclosing function ' + 'scope.\n' + '"SyntaxError" is raised at compile time if the given name ' + 'does not\n' + 'exist in any enclosing function scope.\n' + '\n' + 'The namespace for a module is automatically created the first ' + 'time a\n' + 'module is imported. The main module for a script is always ' + 'called\n' + '"__main__".\n' + '\n' + 'Class definition blocks and arguments to "exec()" and ' + '"eval()" are\n' + 'special in the context of name resolution. A class definition ' + 'is an\n' + 'executable statement that may use and define names. These ' + 'references\n' + 'follow the normal rules for name resolution with an exception ' + 'that\n' + 'unbound local variables are looked up in the global ' + 'namespace. The\n' + 'namespace of the class definition becomes the attribute ' + 'dictionary of\n' + 'the class. The scope of names defined in a class block is ' + 'limited to\n' + 'the class block; it does not extend to the code blocks of ' + 'methods --\n' + 'this includes comprehensions and generator expressions since ' + 'they are\n' + 'implemented using a function scope. This means that the ' + 'following\n' + 'will fail:\n' + '\n' + ' class A:\n' + ' a = 42\n' + ' b = list(a + i for i in range(10))\n' + '\n' + '\n' + 'Builtins and restricted execution\n' + '---------------------------------\n' + '\n' + 'The builtins namespace associated with the execution of a ' + 'code block\n' + 'is actually found by looking up the name "__builtins__" in ' + 'its global\n' + 'namespace; this should be a dictionary or a module (in the ' + 'latter case\n' + "the module's dictionary is used). By default, when in the " + '"__main__"\n' + 'module, "__builtins__" is the built-in module "builtins"; ' + 'when in any\n' + 'other module, "__builtins__" is an alias for the dictionary ' + 'of the\n' + '"builtins" module itself. "__builtins__" can be set to a ' + 'user-created\n' + 'dictionary to create a weak form of restricted execution.\n' + '\n' + '**CPython implementation detail:** Users should not touch\n' + '"__builtins__"; it is strictly an implementation detail. ' + 'Users\n' + 'wanting to override values in the builtins namespace should ' + '"import"\n' + 'the "builtins" module and modify its attributes ' + 'appropriately.\n' + '\n' + '\n' + 'Interaction with dynamic features\n' + '---------------------------------\n' + '\n' + 'Name resolution of free variables occurs at runtime, not at ' + 'compile\n' + 'time. This means that the following code will print 42:\n' + '\n' + ' i = 10\n' + ' def f():\n' + ' print(i)\n' + ' i = 42\n' + ' f()\n' + '\n' + 'There are several cases where Python statements are illegal ' + 'when used\n' + 'in conjunction with nested scopes that contain free ' + 'variables.\n' + '\n' + 'If a variable is referenced in an enclosing scope, it is ' + 'illegal to\n' + 'delete the name. An error will be reported at compile time.\n' + '\n' + 'The "eval()" and "exec()" functions do not have access to the ' + 'full\n' + 'environment for resolving names. Names may be resolved in ' + 'the local\n' + 'and global namespaces of the caller. Free variables are not ' + 'resolved\n' + 'in the nearest enclosing namespace, but in the global ' + 'namespace. [1]\n' + 'The "exec()" and "eval()" functions have optional arguments ' + 'to\n' + 'override the global and local namespace. If only one ' + 'namespace is\n' + 'specified, it is used for both.\n' + '\n' + '\n' + 'Exceptions\n' + '==========\n' + '\n' + 'Exceptions are a means of breaking out of the normal flow of ' + 'control\n' + 'of a code block in order to handle errors or other ' + 'exceptional\n' + 'conditions. An exception is *raised* at the point where the ' + 'error is\n' + 'detected; it may be *handled* by the surrounding code block ' + 'or by any\n' + 'code block that directly or indirectly invoked the code block ' + 'where\n' + 'the error occurred.\n' + '\n' + 'The Python interpreter raises an exception when it detects a ' + 'run-time\n' + 'error (such as division by zero). A Python program can also\n' + 'explicitly raise an exception with the "raise" statement. ' + 'Exception\n' + 'handlers are specified with the "try" ... "except" ' + 'statement. The\n' + '"finally" clause of such a statement can be used to specify ' + 'cleanup\n' + 'code which does not handle the exception, but is executed ' + 'whether an\n' + 'exception occurred or not in the preceding code.\n' + '\n' + 'Python uses the "termination" model of error handling: an ' + 'exception\n' + 'handler can find out what happened and continue execution at ' + 'an outer\n' + 'level, but it cannot repair the cause of the error and retry ' + 'the\n' + 'failing operation (except by re-entering the offending piece ' + 'of code\n' + 'from the top).\n' + '\n' + 'When an exception is not handled at all, the interpreter ' + 'terminates\n' + 'execution of the program, or returns to its interactive main ' + 'loop. In\n' + 'either case, it prints a stack backtrace, except when the ' + 'exception is\n' + '"SystemExit".\n' + '\n' + 'Exceptions are identified by class instances. The "except" ' + 'clause is\n' + 'selected depending on the class of the instance: it must ' + 'reference the\n' + 'class of the instance or a base class thereof. The instance ' + 'can be\n' + 'received by the handler and can carry additional information ' + 'about the\n' + 'exceptional condition.\n' + '\n' + 'Note: Exception messages are not part of the Python API. ' + 'Their\n' + ' contents may change from one version of Python to the next ' + 'without\n' + ' warning and should not be relied on by code which will run ' + 'under\n' + ' multiple versions of the interpreter.\n' + '\n' + 'See also the description of the "try" statement in section ' + '*The try\n' + 'statement* and "raise" statement in section *The raise ' + 'statement*.\n' + '\n' + '-[ Footnotes ]-\n' + '\n' + '[1] This limitation occurs because the code that is executed ' + 'by\n' + ' these operations is not available at the time the module ' + 'is\n' + ' compiled.\n', + 'exprlists': '\n' + 'Expression lists\n' + '****************\n' + '\n' + ' expression_list ::= expression ( "," expression )* [","]\n' + '\n' + 'An expression list containing at least one comma yields a ' + 'tuple. The\n' + 'length of the tuple is the number of expressions in the ' + 'list. The\n' + 'expressions are evaluated from left to right.\n' + '\n' + 'The trailing comma is required only to create a single tuple ' + '(a.k.a. a\n' + '*singleton*); it is optional in all other cases. A single ' + 'expression\n' + "without a trailing comma doesn't create a tuple, but rather " + 'yields the\n' + 'value of that expression. (To create an empty tuple, use an ' + 'empty pair\n' + 'of parentheses: "()".)\n', + 'floating': '\n' + 'Floating point literals\n' + '***********************\n' + '\n' + 'Floating point literals are described by the following ' + 'lexical\n' + 'definitions:\n' + '\n' + ' floatnumber ::= pointfloat | exponentfloat\n' + ' pointfloat ::= [intpart] fraction | intpart "."\n' + ' exponentfloat ::= (intpart | pointfloat) exponent\n' + ' intpart ::= digit+\n' + ' fraction ::= "." digit+\n' + ' exponent ::= ("e" | "E") ["+" | "-"] digit+\n' + '\n' + 'Note that the integer and exponent parts are always ' + 'interpreted using\n' + 'radix 10. For example, "077e010" is legal, and denotes the ' + 'same number\n' + 'as "77e10". The allowed range of floating point literals is\n' + 'implementation-dependent. Some examples of floating point ' + 'literals:\n' + '\n' + ' 3.14 10. .001 1e100 3.14e-10 0e0\n' + '\n' + 'Note that numeric literals do not include a sign; a phrase ' + 'like "-1"\n' + 'is actually an expression composed of the unary operator "-" ' + 'and the\n' + 'literal "1".\n', + 'for': '\n' + 'The "for" statement\n' + '*******************\n' + '\n' + 'The "for" statement is used to iterate over the elements of a ' + 'sequence\n' + '(such as a string, tuple or list) or other iterable object:\n' + '\n' + ' for_stmt ::= "for" target_list "in" expression_list ":" suite\n' + ' ["else" ":" suite]\n' + '\n' + 'The expression list is evaluated once; it should yield an iterable\n' + 'object. An iterator is created for the result of the\n' + '"expression_list". The suite is then executed once for each item\n' + 'provided by the iterator, in the order returned by the iterator. ' + 'Each\n' + 'item in turn is assigned to the target list using the standard ' + 'rules\n' + 'for assignments (see *Assignment statements*), and then the suite ' + 'is\n' + 'executed. When the items are exhausted (which is immediately when ' + 'the\n' + 'sequence is empty or an iterator raises a "StopIteration" ' + 'exception),\n' + 'the suite in the "else" clause, if present, is executed, and the ' + 'loop\n' + 'terminates.\n' + '\n' + 'A "break" statement executed in the first suite terminates the ' + 'loop\n' + 'without executing the "else" clause\'s suite. A "continue" ' + 'statement\n' + 'executed in the first suite skips the rest of the suite and ' + 'continues\n' + 'with the next item, or with the "else" clause if there is no next\n' + 'item.\n' + '\n' + 'The for-loop makes assignments to the variables(s) in the target ' + 'list.\n' + 'This overwrites all previous assignments to those variables ' + 'including\n' + 'those made in the suite of the for-loop:\n' + '\n' + ' for i in range(10):\n' + ' print(i)\n' + ' i = 5 # this will not affect the for-loop\n' + ' # because i will be overwritten with the ' + 'next\n' + ' # index in the range\n' + '\n' + 'Names in the target list are not deleted when the loop is ' + 'finished,\n' + 'but if the sequence is empty, they will not have been assigned to ' + 'at\n' + 'all by the loop. Hint: the built-in function "range()" returns an\n' + "iterator of integers suitable to emulate the effect of Pascal's " + '"for i\n' + ':= a to b do"; e.g., "list(range(3))" returns the list "[0, 1, ' + '2]".\n' + '\n' + 'Note: There is a subtlety when the sequence is being modified by ' + 'the\n' + ' loop (this can only occur for mutable sequences, i.e. lists). ' + 'An\n' + ' internal counter is used to keep track of which item is used ' + 'next,\n' + ' and this is incremented on each iteration. When this counter ' + 'has\n' + ' reached the length of the sequence the loop terminates. This ' + 'means\n' + ' that if the suite deletes the current (or a previous) item from ' + 'the\n' + ' sequence, the next item will be skipped (since it gets the index ' + 'of\n' + ' the current item which has already been treated). Likewise, if ' + 'the\n' + ' suite inserts an item in the sequence before the current item, ' + 'the\n' + ' current item will be treated again the next time through the ' + 'loop.\n' + ' This can lead to nasty bugs that can be avoided by making a\n' + ' temporary copy using a slice of the whole sequence, e.g.,\n' + '\n' + ' for x in a[:]:\n' + ' if x < 0: a.remove(x)\n', + 'formatstrings': '\n' + 'Format String Syntax\n' + '********************\n' + '\n' + 'The "str.format()" method and the "Formatter" class share ' + 'the same\n' + 'syntax for format strings (although in the case of ' + '"Formatter",\n' + 'subclasses can define their own format string syntax).\n' + '\n' + 'Format strings contain "replacement fields" surrounded by ' + 'curly braces\n' + '"{}". Anything that is not contained in braces is ' + 'considered literal\n' + 'text, which is copied unchanged to the output. If you ' + 'need to include\n' + 'a brace character in the literal text, it can be escaped ' + 'by doubling:\n' + '"{{" and "}}".\n' + '\n' + 'The grammar for a replacement field is as follows:\n' + '\n' + ' replacement_field ::= "{" [field_name] ["!" ' + 'conversion] [":" format_spec] "}"\n' + ' field_name ::= arg_name ("." attribute_name ' + '| "[" element_index "]")*\n' + ' arg_name ::= [identifier | integer]\n' + ' attribute_name ::= identifier\n' + ' element_index ::= integer | index_string\n' + ' index_string ::= +\n' + ' conversion ::= "r" | "s" | "a"\n' + ' format_spec ::= \n' + '\n' + 'In less formal terms, the replacement field can start ' + 'with a\n' + '*field_name* that specifies the object whose value is to ' + 'be formatted\n' + 'and inserted into the output instead of the replacement ' + 'field. The\n' + '*field_name* is optionally followed by a *conversion* ' + 'field, which is\n' + 'preceded by an exclamation point "\'!\'", and a ' + '*format_spec*, which is\n' + 'preceded by a colon "\':\'". These specify a non-default ' + 'format for the\n' + 'replacement value.\n' + '\n' + 'See also the *Format Specification Mini-Language* ' + 'section.\n' + '\n' + 'The *field_name* itself begins with an *arg_name* that is ' + 'either a\n' + "number or a keyword. If it's a number, it refers to a " + 'positional\n' + "argument, and if it's a keyword, it refers to a named " + 'keyword\n' + 'argument. If the numerical arg_names in a format string ' + 'are 0, 1, 2,\n' + '... in sequence, they can all be omitted (not just some) ' + 'and the\n' + 'numbers 0, 1, 2, ... will be automatically inserted in ' + 'that order.\n' + 'Because *arg_name* is not quote-delimited, it is not ' + 'possible to\n' + 'specify arbitrary dictionary keys (e.g., the strings ' + '"\'10\'" or\n' + '"\':-]\'") within a format string. The *arg_name* can be ' + 'followed by any\n' + 'number of index or attribute expressions. An expression ' + 'of the form\n' + '"\'.name\'" selects the named attribute using ' + '"getattr()", while an\n' + 'expression of the form "\'[index]\'" does an index lookup ' + 'using\n' + '"__getitem__()".\n' + '\n' + 'Changed in version 3.1: The positional argument ' + 'specifiers can be\n' + 'omitted, so "\'{} {}\'" is equivalent to "\'{0} {1}\'".\n' + '\n' + 'Some simple format string examples:\n' + '\n' + ' "First, thou shalt count to {0}" # References first ' + 'positional argument\n' + ' "Bring me a {}" # Implicitly ' + 'references the first positional argument\n' + ' "From {} to {}" # Same as "From {0} ' + 'to {1}"\n' + ' "My quest is {name}" # References keyword ' + "argument 'name'\n" + ' "Weight in tons {0.weight}" # \'weight\' ' + 'attribute of first positional arg\n' + ' "Units destroyed: {players[0]}" # First element of ' + "keyword argument 'players'.\n" + '\n' + 'The *conversion* field causes a type coercion before ' + 'formatting.\n' + 'Normally, the job of formatting a value is done by the ' + '"__format__()"\n' + 'method of the value itself. However, in some cases it is ' + 'desirable to\n' + 'force a type to be formatted as a string, overriding its ' + 'own\n' + 'definition of formatting. By converting the value to a ' + 'string before\n' + 'calling "__format__()", the normal formatting logic is ' + 'bypassed.\n' + '\n' + 'Three conversion flags are currently supported: "\'!s\'" ' + 'which calls\n' + '"str()" on the value, "\'!r\'" which calls "repr()" and ' + '"\'!a\'" which\n' + 'calls "ascii()".\n' + '\n' + 'Some examples:\n' + '\n' + ' "Harold\'s a clever {0!s}" # Calls str() on the ' + 'argument first\n' + ' "Bring out the holy {name!r}" # Calls repr() on the ' + 'argument first\n' + ' "More {!a}" # Calls ascii() on ' + 'the argument first\n' + '\n' + 'The *format_spec* field contains a specification of how ' + 'the value\n' + 'should be presented, including such details as field ' + 'width, alignment,\n' + 'padding, decimal precision and so on. Each value type ' + 'can define its\n' + 'own "formatting mini-language" or interpretation of the ' + '*format_spec*.\n' + '\n' + 'Most built-in types support a common formatting ' + 'mini-language, which\n' + 'is described in the next section.\n' + '\n' + 'A *format_spec* field can also include nested replacement ' + 'fields\n' + 'within it. These nested replacement fields can contain ' + 'only a field\n' + 'name; conversion flags and format specifications are not ' + 'allowed. The\n' + 'replacement fields within the format_spec are substituted ' + 'before the\n' + '*format_spec* string is interpreted. This allows the ' + 'formatting of a\n' + 'value to be dynamically specified.\n' + '\n' + 'See the *Format examples* section for some examples.\n' + '\n' + '\n' + 'Format Specification Mini-Language\n' + '==================================\n' + '\n' + '"Format specifications" are used within replacement ' + 'fields contained\n' + 'within a format string to define how individual values ' + 'are presented\n' + '(see *Format String Syntax*). They can also be passed ' + 'directly to the\n' + 'built-in "format()" function. Each formattable type may ' + 'define how\n' + 'the format specification is to be interpreted.\n' + '\n' + 'Most built-in types implement the following options for ' + 'format\n' + 'specifications, although some of the formatting options ' + 'are only\n' + 'supported by the numeric types.\n' + '\n' + 'A general convention is that an empty format string ' + '("""") produces\n' + 'the same result as if you had called "str()" on the ' + 'value. A non-empty\n' + 'format string typically modifies the result.\n' + '\n' + 'The general form of a *standard format specifier* is:\n' + '\n' + ' format_spec ::= ' + '[[fill]align][sign][#][0][width][,][.precision][type]\n' + ' fill ::= \n' + ' align ::= "<" | ">" | "=" | "^"\n' + ' sign ::= "+" | "-" | " "\n' + ' width ::= integer\n' + ' precision ::= integer\n' + ' type ::= "b" | "c" | "d" | "e" | "E" | "f" | ' + '"F" | "g" | "G" | "n" | "o" | "s" | "x" | "X" | "%"\n' + '\n' + 'If a valid *align* value is specified, it can be preceded ' + 'by a *fill*\n' + 'character that can be any character and defaults to a ' + 'space if\n' + 'omitted. Note that it is not possible to use "{" and "}" ' + 'as *fill*\n' + 'char while using the "str.format()" method; this ' + 'limitation however\n' + 'doesn\'t affect the "format()" function.\n' + '\n' + 'The meaning of the various alignment options is as ' + 'follows:\n' + '\n' + ' ' + '+-----------+------------------------------------------------------------+\n' + ' | Option | ' + 'Meaning ' + '|\n' + ' ' + '+===========+============================================================+\n' + ' | "\'<\'" | Forces the field to be left-aligned ' + 'within the available |\n' + ' | | space (this is the default for most ' + 'objects). |\n' + ' ' + '+-----------+------------------------------------------------------------+\n' + ' | "\'>\'" | Forces the field to be right-aligned ' + 'within the available |\n' + ' | | space (this is the default for ' + 'numbers). |\n' + ' ' + '+-----------+------------------------------------------------------------+\n' + ' | "\'=\'" | Forces the padding to be placed after ' + 'the sign (if any) |\n' + ' | | but before the digits. This is used for ' + 'printing fields |\n' + " | | in the form '+000000120'. This alignment " + 'option is only |\n' + ' | | valid for numeric ' + 'types. |\n' + ' ' + '+-----------+------------------------------------------------------------+\n' + ' | "\'^\'" | Forces the field to be centered within ' + 'the available |\n' + ' | | ' + 'space. ' + '|\n' + ' ' + '+-----------+------------------------------------------------------------+\n' + '\n' + 'Note that unless a minimum field width is defined, the ' + 'field width\n' + 'will always be the same size as the data to fill it, so ' + 'that the\n' + 'alignment option has no meaning in this case.\n' + '\n' + 'The *sign* option is only valid for number types, and can ' + 'be one of\n' + 'the following:\n' + '\n' + ' ' + '+-----------+------------------------------------------------------------+\n' + ' | Option | ' + 'Meaning ' + '|\n' + ' ' + '+===========+============================================================+\n' + ' | "\'+\'" | indicates that a sign should be used ' + 'for both positive as |\n' + ' | | well as negative ' + 'numbers. |\n' + ' ' + '+-----------+------------------------------------------------------------+\n' + ' | "\'-\'" | indicates that a sign should be used ' + 'only for negative |\n' + ' | | numbers (this is the default ' + 'behavior). |\n' + ' ' + '+-----------+------------------------------------------------------------+\n' + ' | space | indicates that a leading space should be ' + 'used on positive |\n' + ' | | numbers, and a minus sign on negative ' + 'numbers. |\n' + ' ' + '+-----------+------------------------------------------------------------+\n' + '\n' + 'The "\'#\'" option causes the "alternate form" to be used ' + 'for the\n' + 'conversion. The alternate form is defined differently ' + 'for different\n' + 'types. This option is only valid for integer, float, ' + 'complex and\n' + 'Decimal types. For integers, when binary, octal, or ' + 'hexadecimal output\n' + 'is used, this option adds the prefix respective "\'0b\'", ' + '"\'0o\'", or\n' + '"\'0x\'" to the output value. For floats, complex and ' + 'Decimal the\n' + 'alternate form causes the result of the conversion to ' + 'always contain a\n' + 'decimal-point character, even if no digits follow it. ' + 'Normally, a\n' + 'decimal-point character appears in the result of these ' + 'conversions\n' + 'only if a digit follows it. In addition, for "\'g\'" and ' + '"\'G\'"\n' + 'conversions, trailing zeros are not removed from the ' + 'result.\n' + '\n' + 'The "\',\'" option signals the use of a comma for a ' + 'thousands separator.\n' + 'For a locale aware separator, use the "\'n\'" integer ' + 'presentation type\n' + 'instead.\n' + '\n' + 'Changed in version 3.1: Added the "\',\'" option (see ' + 'also **PEP 378**).\n' + '\n' + '*width* is a decimal integer defining the minimum field ' + 'width. If not\n' + 'specified, then the field width will be determined by the ' + 'content.\n' + '\n' + 'Preceding the *width* field by a zero ("\'0\'") character ' + 'enables sign-\n' + 'aware zero-padding for numeric types. This is equivalent ' + 'to a *fill*\n' + 'character of "\'0\'" with an *alignment* type of ' + '"\'=\'".\n' + '\n' + 'The *precision* is a decimal number indicating how many ' + 'digits should\n' + 'be displayed after the decimal point for a floating point ' + 'value\n' + 'formatted with "\'f\'" and "\'F\'", or before and after ' + 'the decimal point\n' + 'for a floating point value formatted with "\'g\'" or ' + '"\'G\'". For non-\n' + 'number types the field indicates the maximum field size - ' + 'in other\n' + 'words, how many characters will be used from the field ' + 'content. The\n' + '*precision* is not allowed for integer values.\n' + '\n' + 'Finally, the *type* determines how the data should be ' + 'presented.\n' + '\n' + 'The available string presentation types are:\n' + '\n' + ' ' + '+-----------+------------------------------------------------------------+\n' + ' | Type | ' + 'Meaning ' + '|\n' + ' ' + '+===========+============================================================+\n' + ' | "\'s\'" | String format. This is the default ' + 'type for strings and |\n' + ' | | may be ' + 'omitted. |\n' + ' ' + '+-----------+------------------------------------------------------------+\n' + ' | None | The same as ' + '"\'s\'". |\n' + ' ' + '+-----------+------------------------------------------------------------+\n' + '\n' + 'The available integer presentation types are:\n' + '\n' + ' ' + '+-----------+------------------------------------------------------------+\n' + ' | Type | ' + 'Meaning ' + '|\n' + ' ' + '+===========+============================================================+\n' + ' | "\'b\'" | Binary format. Outputs the number in ' + 'base 2. |\n' + ' ' + '+-----------+------------------------------------------------------------+\n' + ' | "\'c\'" | Character. Converts the integer to the ' + 'corresponding |\n' + ' | | unicode character before ' + 'printing. |\n' + ' ' + '+-----------+------------------------------------------------------------+\n' + ' | "\'d\'" | Decimal Integer. Outputs the number in ' + 'base 10. |\n' + ' ' + '+-----------+------------------------------------------------------------+\n' + ' | "\'o\'" | Octal format. Outputs the number in ' + 'base 8. |\n' + ' ' + '+-----------+------------------------------------------------------------+\n' + ' | "\'x\'" | Hex format. Outputs the number in base ' + '16, using lower- |\n' + ' | | case letters for the digits above ' + '9. |\n' + ' ' + '+-----------+------------------------------------------------------------+\n' + ' | "\'X\'" | Hex format. Outputs the number in base ' + '16, using upper- |\n' + ' | | case letters for the digits above ' + '9. |\n' + ' ' + '+-----------+------------------------------------------------------------+\n' + ' | "\'n\'" | Number. This is the same as "\'d\'", ' + 'except that it uses the |\n' + ' | | current locale setting to insert the ' + 'appropriate number |\n' + ' | | separator ' + 'characters. |\n' + ' ' + '+-----------+------------------------------------------------------------+\n' + ' | None | The same as ' + '"\'d\'". |\n' + ' ' + '+-----------+------------------------------------------------------------+\n' + '\n' + 'In addition to the above presentation types, integers can ' + 'be formatted\n' + 'with the floating point presentation types listed below ' + '(except "\'n\'"\n' + 'and None). When doing so, "float()" is used to convert ' + 'the integer to\n' + 'a floating point number before formatting.\n' + '\n' + 'The available presentation types for floating point and ' + 'decimal values\n' + 'are:\n' + '\n' + ' ' + '+-----------+------------------------------------------------------------+\n' + ' | Type | ' + 'Meaning ' + '|\n' + ' ' + '+===========+============================================================+\n' + ' | "\'e\'" | Exponent notation. Prints the number ' + 'in scientific |\n' + " | | notation using the letter 'e' to " + 'indicate the exponent. |\n' + ' | | The default precision is ' + '"6". |\n' + ' ' + '+-----------+------------------------------------------------------------+\n' + ' | "\'E\'" | Exponent notation. Same as "\'e\'" ' + 'except it uses an upper |\n' + " | | case 'E' as the separator " + 'character. |\n' + ' ' + '+-----------+------------------------------------------------------------+\n' + ' | "\'f\'" | Fixed point. Displays the number as a ' + 'fixed-point number. |\n' + ' | | The default precision is ' + '"6". |\n' + ' ' + '+-----------+------------------------------------------------------------+\n' + ' | "\'F\'" | Fixed point. Same as "\'f\'", but ' + 'converts "nan" to "NAN" |\n' + ' | | and "inf" to ' + '"INF". |\n' + ' ' + '+-----------+------------------------------------------------------------+\n' + ' | "\'g\'" | General format. For a given precision ' + '"p >= 1", this |\n' + ' | | rounds the number to "p" significant ' + 'digits and then |\n' + ' | | formats the result in either fixed-point ' + 'format or in |\n' + ' | | scientific notation, depending on its ' + 'magnitude. The |\n' + ' | | precise rules are as follows: suppose ' + 'that the result |\n' + ' | | formatted with presentation type "\'e\'" ' + 'and precision "p-1" |\n' + ' | | would have exponent "exp". Then if "-4 ' + '<= exp < p", the |\n' + ' | | number is formatted with presentation ' + 'type "\'f\'" and |\n' + ' | | precision "p-1-exp". Otherwise, the ' + 'number is formatted |\n' + ' | | with presentation type "\'e\'" and ' + 'precision "p-1". In both |\n' + ' | | cases insignificant trailing zeros are ' + 'removed from the |\n' + ' | | significand, and the decimal point is ' + 'also removed if |\n' + ' | | there are no remaining digits following ' + 'it. Positive and |\n' + ' | | negative infinity, positive and negative ' + 'zero, and nans, |\n' + ' | | are formatted as "inf", "-inf", "0", ' + '"-0" and "nan" |\n' + ' | | respectively, regardless of the ' + 'precision. A precision of |\n' + ' | | "0" is treated as equivalent to a ' + 'precision of "1". The |\n' + ' | | default precision is ' + '"6". |\n' + ' ' + '+-----------+------------------------------------------------------------+\n' + ' | "\'G\'" | General format. Same as "\'g\'" except ' + 'switches to "\'E\'" if |\n' + ' | | the number gets too large. The ' + 'representations of infinity |\n' + ' | | and NaN are uppercased, ' + 'too. |\n' + ' ' + '+-----------+------------------------------------------------------------+\n' + ' | "\'n\'" | Number. This is the same as "\'g\'", ' + 'except that it uses the |\n' + ' | | current locale setting to insert the ' + 'appropriate number |\n' + ' | | separator ' + 'characters. |\n' + ' ' + '+-----------+------------------------------------------------------------+\n' + ' | "\'%\'" | Percentage. Multiplies the number by ' + '100 and displays in |\n' + ' | | fixed ("\'f\'") format, followed by a ' + 'percent sign. |\n' + ' ' + '+-----------+------------------------------------------------------------+\n' + ' | None | Similar to "\'g\'", except that ' + 'fixed-point notation, when |\n' + ' | | used, has at least one digit past the ' + 'decimal point. The |\n' + ' | | default precision is as high as needed ' + 'to represent the |\n' + ' | | particular value. The overall effect is ' + 'to match the |\n' + ' | | output of "str()" as altered by the ' + 'other format |\n' + ' | | ' + 'modifiers. ' + '|\n' + ' ' + '+-----------+------------------------------------------------------------+\n' + '\n' + '\n' + 'Format examples\n' + '===============\n' + '\n' + 'This section contains examples of the new format syntax ' + 'and comparison\n' + 'with the old "%"-formatting.\n' + '\n' + 'In most of the cases the syntax is similar to the old ' + '"%"-formatting,\n' + 'with the addition of the "{}" and with ":" used instead ' + 'of "%". For\n' + 'example, "\'%03.2f\'" can be translated to ' + '"\'{:03.2f}\'".\n' + '\n' + 'The new format syntax also supports new and different ' + 'options, shown\n' + 'in the follow examples.\n' + '\n' + 'Accessing arguments by position:\n' + '\n' + " >>> '{0}, {1}, {2}'.format('a', 'b', 'c')\n" + " 'a, b, c'\n" + " >>> '{}, {}, {}'.format('a', 'b', 'c') # 3.1+ only\n" + " 'a, b, c'\n" + " >>> '{2}, {1}, {0}'.format('a', 'b', 'c')\n" + " 'c, b, a'\n" + " >>> '{2}, {1}, {0}'.format(*'abc') # unpacking " + 'argument sequence\n' + " 'c, b, a'\n" + " >>> '{0}{1}{0}'.format('abra', 'cad') # arguments' " + 'indices can be repeated\n' + " 'abracadabra'\n" + '\n' + 'Accessing arguments by name:\n' + '\n' + " >>> 'Coordinates: {latitude}, " + "{longitude}'.format(latitude='37.24N', " + "longitude='-115.81W')\n" + " 'Coordinates: 37.24N, -115.81W'\n" + " >>> coord = {'latitude': '37.24N', 'longitude': " + "'-115.81W'}\n" + " >>> 'Coordinates: {latitude}, " + "{longitude}'.format(**coord)\n" + " 'Coordinates: 37.24N, -115.81W'\n" + '\n' + "Accessing arguments' attributes:\n" + '\n' + ' >>> c = 3-5j\n' + " >>> ('The complex number {0} is formed from the real " + "part {0.real} '\n" + " ... 'and the imaginary part {0.imag}.').format(c)\n" + " 'The complex number (3-5j) is formed from the real " + "part 3.0 and the imaginary part -5.0.'\n" + ' >>> class Point:\n' + ' ... def __init__(self, x, y):\n' + ' ... self.x, self.y = x, y\n' + ' ... def __str__(self):\n' + " ... return 'Point({self.x}, " + "{self.y})'.format(self=self)\n" + ' ...\n' + ' >>> str(Point(4, 2))\n' + " 'Point(4, 2)'\n" + '\n' + "Accessing arguments' items:\n" + '\n' + ' >>> coord = (3, 5)\n' + " >>> 'X: {0[0]}; Y: {0[1]}'.format(coord)\n" + " 'X: 3; Y: 5'\n" + '\n' + 'Replacing "%s" and "%r":\n' + '\n' + ' >>> "repr() shows quotes: {!r}; str() doesn\'t: ' + '{!s}".format(\'test1\', \'test2\')\n' + ' "repr() shows quotes: \'test1\'; str() doesn\'t: ' + 'test2"\n' + '\n' + 'Aligning the text and specifying a width:\n' + '\n' + " >>> '{:<30}'.format('left aligned')\n" + " 'left aligned '\n" + " >>> '{:>30}'.format('right aligned')\n" + " ' right aligned'\n" + " >>> '{:^30}'.format('centered')\n" + " ' centered '\n" + " >>> '{:*^30}'.format('centered') # use '*' as a fill " + 'char\n' + " '***********centered***********'\n" + '\n' + 'Replacing "%+f", "%-f", and "% f" and specifying a sign:\n' + '\n' + " >>> '{:+f}; {:+f}'.format(3.14, -3.14) # show it " + 'always\n' + " '+3.140000; -3.140000'\n" + " >>> '{: f}; {: f}'.format(3.14, -3.14) # show a space " + 'for positive numbers\n' + " ' 3.140000; -3.140000'\n" + " >>> '{:-f}; {:-f}'.format(3.14, -3.14) # show only " + "the minus -- same as '{:f}; {:f}'\n" + " '3.140000; -3.140000'\n" + '\n' + 'Replacing "%x" and "%o" and converting the value to ' + 'different bases:\n' + '\n' + ' >>> # format also supports binary numbers\n' + ' >>> "int: {0:d}; hex: {0:x}; oct: {0:o}; bin: ' + '{0:b}".format(42)\n' + " 'int: 42; hex: 2a; oct: 52; bin: 101010'\n" + ' >>> # with 0x, 0o, or 0b as prefix:\n' + ' >>> "int: {0:d}; hex: {0:#x}; oct: {0:#o}; bin: ' + '{0:#b}".format(42)\n' + " 'int: 42; hex: 0x2a; oct: 0o52; bin: 0b101010'\n" + '\n' + 'Using the comma as a thousands separator:\n' + '\n' + " >>> '{:,}'.format(1234567890)\n" + " '1,234,567,890'\n" + '\n' + 'Expressing a percentage:\n' + '\n' + ' >>> points = 19\n' + ' >>> total = 22\n' + " >>> 'Correct answers: {:.2%}'.format(points/total)\n" + " 'Correct answers: 86.36%'\n" + '\n' + 'Using type-specific formatting:\n' + '\n' + ' >>> import datetime\n' + ' >>> d = datetime.datetime(2010, 7, 4, 12, 15, 58)\n' + " >>> '{:%Y-%m-%d %H:%M:%S}'.format(d)\n" + " '2010-07-04 12:15:58'\n" + '\n' + 'Nesting arguments and more complex examples:\n' + '\n' + " >>> for align, text in zip('<^>', ['left', 'center', " + "'right']):\n" + " ... '{0:{fill}{align}16}'.format(text, fill=align, " + 'align=align)\n' + ' ...\n' + " 'left<<<<<<<<<<<<'\n" + " '^^^^^center^^^^^'\n" + " '>>>>>>>>>>>right'\n" + ' >>>\n' + ' >>> octets = [192, 168, 0, 1]\n' + " >>> '{:02X}{:02X}{:02X}{:02X}'.format(*octets)\n" + " 'C0A80001'\n" + ' >>> int(_, 16)\n' + ' 3232235521\n' + ' >>>\n' + ' >>> width = 5\n' + ' >>> for num in range(5,12): #doctest: ' + '+NORMALIZE_WHITESPACE\n' + " ... for base in 'dXob':\n" + " ... print('{0:{width}{base}}'.format(num, " + "base=base, width=width), end=' ')\n" + ' ... print()\n' + ' ...\n' + ' 5 5 5 101\n' + ' 6 6 6 110\n' + ' 7 7 7 111\n' + ' 8 8 10 1000\n' + ' 9 9 11 1001\n' + ' 10 A 12 1010\n' + ' 11 B 13 1011\n', + 'function': '\n' + 'Function definitions\n' + '********************\n' + '\n' + 'A function definition defines a user-defined function object ' + '(see\n' + 'section *The standard type hierarchy*):\n' + '\n' + ' funcdef ::= [decorators] "def" funcname "(" ' + '[parameter_list] ")" ["->" expression] ":" suite\n' + ' decorators ::= decorator+\n' + ' decorator ::= "@" dotted_name ["(" [parameter_list ' + '[","]] ")"] NEWLINE\n' + ' dotted_name ::= identifier ("." identifier)*\n' + ' parameter_list ::= (defparameter ",")*\n' + ' | "*" [parameter] ("," defparameter)* ' + '["," "**" parameter]\n' + ' | "**" parameter\n' + ' | defparameter [","] )\n' + ' parameter ::= identifier [":" expression]\n' + ' defparameter ::= parameter ["=" expression]\n' + ' funcname ::= identifier\n' + '\n' + 'A function definition is an executable statement. Its ' + 'execution binds\n' + 'the function name in the current local namespace to a function ' + 'object\n' + '(a wrapper around the executable code for the function). ' + 'This\n' + 'function object contains a reference to the current global ' + 'namespace\n' + 'as the global namespace to be used when the function is ' + 'called.\n' + '\n' + 'The function definition does not execute the function body; ' + 'this gets\n' + 'executed only when the function is called. [3]\n' + '\n' + 'A function definition may be wrapped by one or more ' + '*decorator*\n' + 'expressions. Decorator expressions are evaluated when the ' + 'function is\n' + 'defined, in the scope that contains the function definition. ' + 'The\n' + 'result must be a callable, which is invoked with the function ' + 'object\n' + 'as the only argument. The returned value is bound to the ' + 'function name\n' + 'instead of the function object. Multiple decorators are ' + 'applied in\n' + 'nested fashion. For example, the following code\n' + '\n' + ' @f1(arg)\n' + ' @f2\n' + ' def func(): pass\n' + '\n' + 'is equivalent to\n' + '\n' + ' def func(): pass\n' + ' func = f1(arg)(f2(func))\n' + '\n' + 'When one or more *parameters* have the form *parameter* "="\n' + '*expression*, the function is said to have "default parameter ' + 'values."\n' + 'For a parameter with a default value, the corresponding ' + '*argument* may\n' + "be omitted from a call, in which case the parameter's default " + 'value is\n' + 'substituted. If a parameter has a default value, all ' + 'following\n' + 'parameters up until the ""*"" must also have a default value ' + '--- this\n' + 'is a syntactic restriction that is not expressed by the ' + 'grammar.\n' + '\n' + '**Default parameter values are evaluated from left to right ' + 'when the\n' + 'function definition is executed.** This means that the ' + 'expression is\n' + 'evaluated once, when the function is defined, and that the ' + 'same "pre-\n' + 'computed" value is used for each call. This is especially ' + 'important\n' + 'to understand when a default parameter is a mutable object, ' + 'such as a\n' + 'list or a dictionary: if the function modifies the object ' + '(e.g. by\n' + 'appending an item to a list), the default value is in effect ' + 'modified.\n' + 'This is generally not what was intended. A way around this is ' + 'to use\n' + '"None" as the default, and explicitly test for it in the body ' + 'of the\n' + 'function, e.g.:\n' + '\n' + ' def whats_on_the_telly(penguin=None):\n' + ' if penguin is None:\n' + ' penguin = []\n' + ' penguin.append("property of the zoo")\n' + ' return penguin\n' + '\n' + 'Function call semantics are described in more detail in ' + 'section\n' + '*Calls*. A function call always assigns values to all ' + 'parameters\n' + 'mentioned in the parameter list, either from position ' + 'arguments, from\n' + 'keyword arguments, or from default values. If the form\n' + '""*identifier"" is present, it is initialized to a tuple ' + 'receiving any\n' + 'excess positional parameters, defaulting to the empty tuple. ' + 'If the\n' + 'form ""**identifier"" is present, it is initialized to a new\n' + 'dictionary receiving any excess keyword arguments, defaulting ' + 'to a new\n' + 'empty dictionary. Parameters after ""*"" or ""*identifier"" ' + 'are\n' + 'keyword-only parameters and may only be passed used keyword ' + 'arguments.\n' + '\n' + 'Parameters may have annotations of the form "": expression"" ' + 'following\n' + 'the parameter name. Any parameter may have an annotation even ' + 'those\n' + 'of the form "*identifier" or "**identifier". Functions may ' + 'have\n' + '"return" annotation of the form ""-> expression"" after the ' + 'parameter\n' + 'list. These annotations can be any valid Python expression ' + 'and are\n' + 'evaluated when the function definition is executed. ' + 'Annotations may\n' + 'be evaluated in a different order than they appear in the ' + 'source code.\n' + 'The presence of annotations does not change the semantics of ' + 'a\n' + 'function. The annotation values are available as values of a\n' + "dictionary keyed by the parameters' names in the " + '"__annotations__"\n' + 'attribute of the function object.\n' + '\n' + 'It is also possible to create anonymous functions (functions ' + 'not bound\n' + 'to a name), for immediate use in expressions. This uses ' + 'lambda\n' + 'expressions, described in section *Lambdas*. Note that the ' + 'lambda\n' + 'expression is merely a shorthand for a simplified function ' + 'definition;\n' + 'a function defined in a ""def"" statement can be passed around ' + 'or\n' + 'assigned to another name just like a function defined by a ' + 'lambda\n' + 'expression. The ""def"" form is actually more powerful since ' + 'it\n' + 'allows the execution of multiple statements and annotations.\n' + '\n' + "**Programmer's note:** Functions are first-class objects. A " + '""def""\n' + 'statement executed inside a function definition defines a ' + 'local\n' + 'function that can be returned or passed around. Free ' + 'variables used\n' + 'in the nested function can access the local variables of the ' + 'function\n' + 'containing the def. See section *Naming and binding* for ' + 'details.\n' + '\n' + 'See also: **PEP 3107** - Function Annotations\n' + '\n' + ' The original specification for function annotations.\n', + 'global': '\n' + 'The "global" statement\n' + '**********************\n' + '\n' + ' global_stmt ::= "global" identifier ("," identifier)*\n' + '\n' + 'The "global" statement is a declaration which holds for the ' + 'entire\n' + 'current code block. It means that the listed identifiers are to ' + 'be\n' + 'interpreted as globals. It would be impossible to assign to a ' + 'global\n' + 'variable without "global", although free variables may refer to\n' + 'globals without being declared global.\n' + '\n' + 'Names listed in a "global" statement must not be used in the ' + 'same code\n' + 'block textually preceding that "global" statement.\n' + '\n' + 'Names listed in a "global" statement must not be defined as ' + 'formal\n' + 'parameters or in a "for" loop control target, "class" ' + 'definition,\n' + 'function definition, or "import" statement.\n' + '\n' + '**CPython implementation detail:** The current implementation ' + 'does not\n' + 'enforce the two restrictions, but programs should not abuse ' + 'this\n' + 'freedom, as future implementations may enforce them or silently ' + 'change\n' + 'the meaning of the program.\n' + '\n' + '**Programmer\'s note:** the "global" is a directive to the ' + 'parser. It\n' + 'applies only to code parsed at the same time as the "global"\n' + 'statement. In particular, a "global" statement contained in a ' + 'string\n' + 'or code object supplied to the built-in "exec()" function does ' + 'not\n' + 'affect the code block *containing* the function call, and code\n' + 'contained in such a string is unaffected by "global" statements ' + 'in the\n' + 'code containing the function call. The same applies to the ' + '"eval()"\n' + 'and "compile()" functions.\n', + 'id-classes': '\n' + 'Reserved classes of identifiers\n' + '*******************************\n' + '\n' + 'Certain classes of identifiers (besides keywords) have ' + 'special\n' + 'meanings. These classes are identified by the patterns of ' + 'leading and\n' + 'trailing underscore characters:\n' + '\n' + '"_*"\n' + ' Not imported by "from module import *". The special ' + 'identifier "_"\n' + ' is used in the interactive interpreter to store the ' + 'result of the\n' + ' last evaluation; it is stored in the "builtins" module. ' + 'When not\n' + ' in interactive mode, "_" has no special meaning and is ' + 'not defined.\n' + ' See section *The import statement*.\n' + '\n' + ' Note: The name "_" is often used in conjunction with\n' + ' internationalization; refer to the documentation for ' + 'the\n' + ' "gettext" module for more information on this ' + 'convention.\n' + '\n' + '"__*__"\n' + ' System-defined names. These names are defined by the ' + 'interpreter\n' + ' and its implementation (including the standard library). ' + 'Current\n' + ' system names are discussed in the *Special method names* ' + 'section\n' + ' and elsewhere. More will likely be defined in future ' + 'versions of\n' + ' Python. *Any* use of "__*__" names, in any context, that ' + 'does not\n' + ' follow explicitly documented use, is subject to breakage ' + 'without\n' + ' warning.\n' + '\n' + '"__*"\n' + ' Class-private names. Names in this category, when used ' + 'within the\n' + ' context of a class definition, are re-written to use a ' + 'mangled form\n' + ' to help avoid name clashes between "private" attributes ' + 'of base and\n' + ' derived classes. See section *Identifiers (Names)*.\n', + 'identifiers': '\n' + 'Identifiers and keywords\n' + '************************\n' + '\n' + 'Identifiers (also referred to as *names*) are described by ' + 'the\n' + 'following lexical definitions.\n' + '\n' + 'The syntax of identifiers in Python is based on the Unicode ' + 'standard\n' + 'annex UAX-31, with elaboration and changes as defined ' + 'below; see also\n' + '**PEP 3131** for further details.\n' + '\n' + 'Within the ASCII range (U+0001..U+007F), the valid ' + 'characters for\n' + 'identifiers are the same as in Python 2.x: the uppercase ' + 'and lowercase\n' + 'letters "A" through "Z", the underscore "_" and, except for ' + 'the first\n' + 'character, the digits "0" through "9".\n' + '\n' + 'Python 3.0 introduces additional characters from outside ' + 'the ASCII\n' + 'range (see **PEP 3131**). For these characters, the ' + 'classification\n' + 'uses the version of the Unicode Character Database as ' + 'included in the\n' + '"unicodedata" module.\n' + '\n' + 'Identifiers are unlimited in length. Case is significant.\n' + '\n' + ' identifier ::= xid_start xid_continue*\n' + ' id_start ::= \n' + ' id_continue ::= \n' + ' xid_start ::= \n' + ' xid_continue ::= \n' + '\n' + 'The Unicode category codes mentioned above stand for:\n' + '\n' + '* *Lu* - uppercase letters\n' + '\n' + '* *Ll* - lowercase letters\n' + '\n' + '* *Lt* - titlecase letters\n' + '\n' + '* *Lm* - modifier letters\n' + '\n' + '* *Lo* - other letters\n' + '\n' + '* *Nl* - letter numbers\n' + '\n' + '* *Mn* - nonspacing marks\n' + '\n' + '* *Mc* - spacing combining marks\n' + '\n' + '* *Nd* - decimal numbers\n' + '\n' + '* *Pc* - connector punctuations\n' + '\n' + '* *Other_ID_Start* - explicit list of characters in ' + 'PropList.txt to\n' + ' support backwards compatibility\n' + '\n' + '* *Other_ID_Continue* - likewise\n' + '\n' + 'All identifiers are converted into the normal form NFKC ' + 'while parsing;\n' + 'comparison of identifiers is based on NFKC.\n' + '\n' + 'A non-normative HTML file listing all valid identifier ' + 'characters for\n' + 'Unicode 4.1 can be found at http://www.dcl.hpi.uni-\n' + 'potsdam.de/home/loewis/table-3131.html.\n' + '\n' + '\n' + 'Keywords\n' + '========\n' + '\n' + 'The following identifiers are used as reserved words, or ' + '*keywords* of\n' + 'the language, and cannot be used as ordinary identifiers. ' + 'They must\n' + 'be spelled exactly as written here:\n' + '\n' + ' False class finally is return\n' + ' None continue for lambda try\n' + ' True def from nonlocal while\n' + ' and del global not with\n' + ' as elif if or yield\n' + ' assert else import pass\n' + ' break except in raise\n' + '\n' + '\n' + 'Reserved classes of identifiers\n' + '===============================\n' + '\n' + 'Certain classes of identifiers (besides keywords) have ' + 'special\n' + 'meanings. These classes are identified by the patterns of ' + 'leading and\n' + 'trailing underscore characters:\n' + '\n' + '"_*"\n' + ' Not imported by "from module import *". The special ' + 'identifier "_"\n' + ' is used in the interactive interpreter to store the ' + 'result of the\n' + ' last evaluation; it is stored in the "builtins" module. ' + 'When not\n' + ' in interactive mode, "_" has no special meaning and is ' + 'not defined.\n' + ' See section *The import statement*.\n' + '\n' + ' Note: The name "_" is often used in conjunction with\n' + ' internationalization; refer to the documentation for ' + 'the\n' + ' "gettext" module for more information on this ' + 'convention.\n' + '\n' + '"__*__"\n' + ' System-defined names. These names are defined by the ' + 'interpreter\n' + ' and its implementation (including the standard ' + 'library). Current\n' + ' system names are discussed in the *Special method names* ' + 'section\n' + ' and elsewhere. More will likely be defined in future ' + 'versions of\n' + ' Python. *Any* use of "__*__" names, in any context, ' + 'that does not\n' + ' follow explicitly documented use, is subject to breakage ' + 'without\n' + ' warning.\n' + '\n' + '"__*"\n' + ' Class-private names. Names in this category, when used ' + 'within the\n' + ' context of a class definition, are re-written to use a ' + 'mangled form\n' + ' to help avoid name clashes between "private" attributes ' + 'of base and\n' + ' derived classes. See section *Identifiers (Names)*.\n', + 'if': '\n' + 'The "if" statement\n' + '******************\n' + '\n' + 'The "if" statement is used for conditional execution:\n' + '\n' + ' if_stmt ::= "if" expression ":" suite\n' + ' ( "elif" expression ":" suite )*\n' + ' ["else" ":" suite]\n' + '\n' + 'It selects exactly one of the suites by evaluating the expressions ' + 'one\n' + 'by one until one is found to be true (see section *Boolean ' + 'operations*\n' + 'for the definition of true and false); then that suite is executed\n' + '(and no other part of the "if" statement is executed or evaluated).\n' + 'If all expressions are false, the suite of the "else" clause, if\n' + 'present, is executed.\n', + 'imaginary': '\n' + 'Imaginary literals\n' + '******************\n' + '\n' + 'Imaginary literals are described by the following lexical ' + 'definitions:\n' + '\n' + ' imagnumber ::= (floatnumber | intpart) ("j" | "J")\n' + '\n' + 'An imaginary literal yields a complex number with a real part ' + 'of 0.0.\n' + 'Complex numbers are represented as a pair of floating point ' + 'numbers\n' + 'and have the same restrictions on their range. To create a ' + 'complex\n' + 'number with a nonzero real part, add a floating point number ' + 'to it,\n' + 'e.g., "(3+4j)". Some examples of imaginary literals:\n' + '\n' + ' 3.14j 10.j 10j .001j 1e100j 3.14e-10j\n', + 'import': '\n' + 'The "import" statement\n' + '**********************\n' + '\n' + ' import_stmt ::= "import" module ["as" name] ( "," module ' + '["as" name] )*\n' + ' | "from" relative_module "import" identifier ' + '["as" name]\n' + ' ( "," identifier ["as" name] )*\n' + ' | "from" relative_module "import" "(" ' + 'identifier ["as" name]\n' + ' ( "," identifier ["as" name] )* [","] ")"\n' + ' | "from" module "import" "*"\n' + ' module ::= (identifier ".")* identifier\n' + ' relative_module ::= "."* module | "."+\n' + ' name ::= identifier\n' + '\n' + 'The basic import statement (no "from" clause) is executed in ' + 'two\n' + 'steps:\n' + '\n' + '1. find a module, loading and initializing it if necessary\n' + '\n' + '2. define a name or names in the local namespace for the scope\n' + ' where the "import" statement occurs.\n' + '\n' + 'When the statement contains multiple clauses (separated by ' + 'commas) the\n' + 'two steps are carried out separately for each clause, just as ' + 'though\n' + 'the clauses had been separated out into individiual import ' + 'statements.\n' + '\n' + 'The details of the first step, finding and loading modules are\n' + 'described in greater detail in the section on the *import ' + 'system*,\n' + 'which also describes the various types of packages and modules ' + 'that\n' + 'can be imported, as well as all the hooks that can be used to\n' + 'customize the import system. Note that failures in this step ' + 'may\n' + 'indicate either that the module could not be located, *or* that ' + 'an\n' + 'error occurred while initializing the module, which includes ' + 'execution\n' + "of the module's code.\n" + '\n' + 'If the requested module is retrieved successfully, it will be ' + 'made\n' + 'available in the local namespace in one of three ways:\n' + '\n' + '* If the module name is followed by "as", then the name ' + 'following\n' + ' "as" is bound directly to the imported module.\n' + '\n' + '* If no other name is specified, and the module being imported ' + 'is a\n' + " top level module, the module's name is bound in the local " + 'namespace\n' + ' as a reference to the imported module\n' + '\n' + '* If the module being imported is *not* a top level module, then ' + 'the\n' + ' name of the top level package that contains the module is ' + 'bound in\n' + ' the local namespace as a reference to the top level package. ' + 'The\n' + ' imported module must be accessed using its full qualified ' + 'name\n' + ' rather than directly\n' + '\n' + 'The "from" form uses a slightly more complex process:\n' + '\n' + '1. find the module specified in the "from" clause, loading and\n' + ' initializing it if necessary;\n' + '\n' + '2. for each of the identifiers specified in the "import" ' + 'clauses:\n' + '\n' + ' 1. check if the imported module has an attribute by that ' + 'name\n' + '\n' + ' 2. if not, attempt to import a submodule with that name and ' + 'then\n' + ' check the imported module again for that attribute\n' + '\n' + ' 3. if the attribute is not found, "ImportError" is raised.\n' + '\n' + ' 4. otherwise, a reference to that value is stored in the ' + 'local\n' + ' namespace, using the name in the "as" clause if it is ' + 'present,\n' + ' otherwise using the attribute name\n' + '\n' + 'Examples:\n' + '\n' + ' import foo # foo imported and bound locally\n' + ' import foo.bar.baz # foo.bar.baz imported, foo bound ' + 'locally\n' + ' import foo.bar.baz as fbb # foo.bar.baz imported and bound ' + 'as fbb\n' + ' from foo.bar import baz # foo.bar.baz imported and bound ' + 'as baz\n' + ' from foo import attr # foo imported and foo.attr bound ' + 'as attr\n' + '\n' + 'If the list of identifiers is replaced by a star ("\'*\'"), all ' + 'public\n' + 'names defined in the module are bound in the local namespace for ' + 'the\n' + 'scope where the "import" statement occurs.\n' + '\n' + 'The *public names* defined by a module are determined by ' + 'checking the\n' + 'module\'s namespace for a variable named "__all__"; if defined, ' + 'it must\n' + 'be a sequence of strings which are names defined or imported by ' + 'that\n' + 'module. The names given in "__all__" are all considered public ' + 'and\n' + 'are required to exist. If "__all__" is not defined, the set of ' + 'public\n' + "names includes all names found in the module's namespace which " + 'do not\n' + 'begin with an underscore character ("\'_\'"). "__all__" should ' + 'contain\n' + 'the entire public API. It is intended to avoid accidentally ' + 'exporting\n' + 'items that are not part of the API (such as library modules ' + 'which were\n' + 'imported and used within the module).\n' + '\n' + 'The wild card form of import --- "from module import *" --- is ' + 'only\n' + 'allowed at the module level. Attempting to use it in class or\n' + 'function definitions will raise a "SyntaxError".\n' + '\n' + 'When specifying what module to import you do not have to specify ' + 'the\n' + 'absolute name of the module. When a module or package is ' + 'contained\n' + 'within another package it is possible to make a relative import ' + 'within\n' + 'the same top package without having to mention the package name. ' + 'By\n' + 'using leading dots in the specified module or package after ' + '"from" you\n' + 'can specify how high to traverse up the current package ' + 'hierarchy\n' + 'without specifying exact names. One leading dot means the ' + 'current\n' + 'package where the module making the import exists. Two dots ' + 'means up\n' + 'one package level. Three dots is up two levels, etc. So if you ' + 'execute\n' + '"from . import mod" from a module in the "pkg" package then you ' + 'will\n' + 'end up importing "pkg.mod". If you execute "from ..subpkg2 ' + 'import mod"\n' + 'from within "pkg.subpkg1" you will import "pkg.subpkg2.mod". ' + 'The\n' + 'specification for relative imports is contained within **PEP ' + '328**.\n' + '\n' + '"importlib.import_module()" is provided to support applications ' + 'that\n' + 'determine dynamically the modules to be loaded.\n' + '\n' + '\n' + 'Future statements\n' + '=================\n' + '\n' + 'A *future statement* is a directive to the compiler that a ' + 'particular\n' + 'module should be compiled using syntax or semantics that will ' + 'be\n' + 'available in a specified future release of Python where the ' + 'feature\n' + 'becomes standard.\n' + '\n' + 'The future statement is intended to ease migration to future ' + 'versions\n' + 'of Python that introduce incompatible changes to the language. ' + 'It\n' + 'allows use of the new features on a per-module basis before the\n' + 'release in which the feature becomes standard.\n' + '\n' + ' future_statement ::= "from" "__future__" "import" feature ' + '["as" name]\n' + ' ("," feature ["as" name])*\n' + ' | "from" "__future__" "import" "(" ' + 'feature ["as" name]\n' + ' ("," feature ["as" name])* [","] ")"\n' + ' feature ::= identifier\n' + ' name ::= identifier\n' + '\n' + 'A future statement must appear near the top of the module. The ' + 'only\n' + 'lines that can appear before a future statement are:\n' + '\n' + '* the module docstring (if any),\n' + '\n' + '* comments,\n' + '\n' + '* blank lines, and\n' + '\n' + '* other future statements.\n' + '\n' + 'The features recognized by Python 3.0 are "absolute_import",\n' + '"division", "generators", "unicode_literals", "print_function",\n' + '"nested_scopes" and "with_statement". They are all redundant ' + 'because\n' + 'they are always enabled, and only kept for backwards ' + 'compatibility.\n' + '\n' + 'A future statement is recognized and treated specially at ' + 'compile\n' + 'time: Changes to the semantics of core constructs are often\n' + 'implemented by generating different code. It may even be the ' + 'case\n' + 'that a new feature introduces new incompatible syntax (such as a ' + 'new\n' + 'reserved word), in which case the compiler may need to parse ' + 'the\n' + 'module differently. Such decisions cannot be pushed off until\n' + 'runtime.\n' + '\n' + 'For any given release, the compiler knows which feature names ' + 'have\n' + 'been defined, and raises a compile-time error if a future ' + 'statement\n' + 'contains a feature not known to it.\n' + '\n' + 'The direct runtime semantics are the same as for any import ' + 'statement:\n' + 'there is a standard module "__future__", described later, and it ' + 'will\n' + 'be imported in the usual way at the time the future statement ' + 'is\n' + 'executed.\n' + '\n' + 'The interesting runtime semantics depend on the specific ' + 'feature\n' + 'enabled by the future statement.\n' + '\n' + 'Note that there is nothing special about the statement:\n' + '\n' + ' import __future__ [as name]\n' + '\n' + "That is not a future statement; it's an ordinary import " + 'statement with\n' + 'no special semantics or syntax restrictions.\n' + '\n' + 'Code compiled by calls to the built-in functions "exec()" and\n' + '"compile()" that occur in a module "M" containing a future ' + 'statement\n' + 'will, by default, use the new syntax or semantics associated ' + 'with the\n' + 'future statement. This can be controlled by optional arguments ' + 'to\n' + '"compile()" --- see the documentation of that function for ' + 'details.\n' + '\n' + 'A future statement typed at an interactive interpreter prompt ' + 'will\n' + 'take effect for the rest of the interpreter session. If an\n' + 'interpreter is started with the *-i* option, is passed a script ' + 'name\n' + 'to execute, and the script includes a future statement, it will ' + 'be in\n' + 'effect in the interactive session started after the script is\n' + 'executed.\n' + '\n' + 'See also: **PEP 236** - Back to the __future__\n' + '\n' + ' The original proposal for the __future__ mechanism.\n', + 'in': '\n' + 'Comparisons\n' + '***********\n' + '\n' + 'Unlike C, all comparison operations in Python have the same ' + 'priority,\n' + 'which is lower than that of any arithmetic, shifting or bitwise\n' + 'operation. Also unlike C, expressions like "a < b < c" have the\n' + 'interpretation that is conventional in mathematics:\n' + '\n' + ' comparison ::= or_expr ( comp_operator or_expr )*\n' + ' comp_operator ::= "<" | ">" | "==" | ">=" | "<=" | "!="\n' + ' | "is" ["not"] | ["not"] "in"\n' + '\n' + 'Comparisons yield boolean values: "True" or "False".\n' + '\n' + 'Comparisons can be chained arbitrarily, e.g., "x < y <= z" is\n' + 'equivalent to "x < y and y <= z", except that "y" is evaluated only\n' + 'once (but in both cases "z" is not evaluated at all when "x < y" is\n' + 'found to be false).\n' + '\n' + 'Formally, if *a*, *b*, *c*, ..., *y*, *z* are expressions and ' + '*op1*,\n' + '*op2*, ..., *opN* are comparison operators, then "a op1 b op2 c ... ' + 'y\n' + 'opN z" is equivalent to "a op1 b and b op2 c and ... y opN z", ' + 'except\n' + 'that each expression is evaluated at most once.\n' + '\n' + 'Note that "a op1 b op2 c" doesn\'t imply any kind of comparison ' + 'between\n' + '*a* and *c*, so that, e.g., "x < y > z" is perfectly legal (though\n' + 'perhaps not pretty).\n' + '\n' + 'The operators "<", ">", "==", ">=", "<=", and "!=" compare the ' + 'values\n' + 'of two objects. The objects need not have the same type. If both ' + 'are\n' + 'numbers, they are converted to a common type. Otherwise, the "==" ' + 'and\n' + '"!=" operators *always* consider objects of different types to be\n' + 'unequal, while the "<", ">", ">=" and "<=" operators raise a\n' + '"TypeError" when comparing objects of different types that do not\n' + 'implement these operators for the given pair of types. You can\n' + 'control comparison behavior of objects of non-built-in types by\n' + 'defining rich comparison methods like "__gt__()", described in ' + 'section\n' + '*Basic customization*.\n' + '\n' + 'Comparison of objects of the same type depends on the type:\n' + '\n' + '* Numbers are compared arithmetically.\n' + '\n' + '* The values "float(\'NaN\')" and "Decimal(\'NaN\')" are special. ' + 'They\n' + ' are identical to themselves, "x is x" but are not equal to\n' + ' themselves, "x != x". Additionally, comparing any value to a\n' + ' not-a-number value will return "False". For example, both "3 <\n' + ' float(\'NaN\')" and "float(\'NaN\') < 3" will return "False".\n' + '\n' + '* Bytes objects are compared lexicographically using the numeric\n' + ' values of their elements.\n' + '\n' + '* Strings are compared lexicographically using the numeric\n' + ' equivalents (the result of the built-in function "ord()") of ' + 'their\n' + " characters. [3] String and bytes object can't be compared!\n" + '\n' + '* Tuples and lists are compared lexicographically using comparison\n' + ' of corresponding elements. This means that to compare equal, ' + 'each\n' + ' element must compare equal and the two sequences must be of the ' + 'same\n' + ' type and have the same length.\n' + '\n' + ' If not equal, the sequences are ordered the same as their first\n' + ' differing elements. For example, "[1,2,x] <= [1,2,y]" has the ' + 'same\n' + ' value as "x <= y". If the corresponding element does not exist, ' + 'the\n' + ' shorter sequence is ordered first (for example, "[1,2] < ' + '[1,2,3]").\n' + '\n' + '* Mappings (dictionaries) compare equal if and only if they have ' + 'the\n' + ' same "(key, value)" pairs. Order comparisons "(\'<\', \'<=\', ' + "'>=',\n" + ' \'>\')" raise "TypeError".\n' + '\n' + '* Sets and frozensets define comparison operators to mean subset ' + 'and\n' + ' superset tests. Those relations do not define total orderings ' + '(the\n' + ' two sets "{1,2}" and "{2,3}" are not equal, nor subsets of one\n' + ' another, nor supersets of one another). Accordingly, sets are ' + 'not\n' + ' appropriate arguments for functions which depend on total ' + 'ordering.\n' + ' For example, "min()", "max()", and "sorted()" produce undefined\n' + ' results given a list of sets as inputs.\n' + '\n' + '* Most other objects of built-in types compare unequal unless they\n' + ' are the same object; the choice whether one object is considered\n' + ' smaller or larger than another one is made arbitrarily but\n' + ' consistently within one execution of a program.\n' + '\n' + 'Comparison of objects of differing types depends on whether either ' + 'of\n' + 'the types provide explicit support for the comparison. Most ' + 'numeric\n' + 'types can be compared with one another. When cross-type comparison ' + 'is\n' + 'not supported, the comparison method returns "NotImplemented".\n' + '\n' + 'The operators "in" and "not in" test for membership. "x in s"\n' + 'evaluates to true if *x* is a member of *s*, and false otherwise. ' + '"x\n' + 'not in s" returns the negation of "x in s". All built-in sequences\n' + 'and set types support this as well as dictionary, for which "in" ' + 'tests\n' + 'whether the dictionary has a given key. For container types such as\n' + 'list, tuple, set, frozenset, dict, or collections.deque, the\n' + 'expression "x in y" is equivalent to "any(x is e or x == e for e in\n' + 'y)".\n' + '\n' + 'For the string and bytes types, "x in y" is true if and only if *x* ' + 'is\n' + 'a substring of *y*. An equivalent test is "y.find(x) != -1". ' + 'Empty\n' + 'strings are always considered to be a substring of any other ' + 'string,\n' + 'so """ in "abc"" will return "True".\n' + '\n' + 'For user-defined classes which define the "__contains__()" method, ' + '"x\n' + 'in y" is true if and only if "y.__contains__(x)" is true.\n' + '\n' + 'For user-defined classes which do not define "__contains__()" but ' + 'do\n' + 'define "__iter__()", "x in y" is true if some value "z" with "x == ' + 'z"\n' + 'is produced while iterating over "y". If an exception is raised\n' + 'during the iteration, it is as if "in" raised that exception.\n' + '\n' + 'Lastly, the old-style iteration protocol is tried: if a class ' + 'defines\n' + '"__getitem__()", "x in y" is true if and only if there is a non-\n' + 'negative integer index *i* such that "x == y[i]", and all lower\n' + 'integer indices do not raise "IndexError" exception. (If any other\n' + 'exception is raised, it is as if "in" raised that exception).\n' + '\n' + 'The operator "not in" is defined to have the inverse true value of\n' + '"in".\n' + '\n' + 'The operators "is" and "is not" test for object identity: "x is y" ' + 'is\n' + 'true if and only if *x* and *y* are the same object. "x is not y"\n' + 'yields the inverse truth value. [4]\n', + 'integers': '\n' + 'Integer literals\n' + '****************\n' + '\n' + 'Integer literals are described by the following lexical ' + 'definitions:\n' + '\n' + ' integer ::= decimalinteger | octinteger | hexinteger ' + '| bininteger\n' + ' decimalinteger ::= nonzerodigit digit* | "0"+\n' + ' nonzerodigit ::= "1"..."9"\n' + ' digit ::= "0"..."9"\n' + ' octinteger ::= "0" ("o" | "O") octdigit+\n' + ' hexinteger ::= "0" ("x" | "X") hexdigit+\n' + ' bininteger ::= "0" ("b" | "B") bindigit+\n' + ' octdigit ::= "0"..."7"\n' + ' hexdigit ::= digit | "a"..."f" | "A"..."F"\n' + ' bindigit ::= "0" | "1"\n' + '\n' + 'There is no limit for the length of integer literals apart ' + 'from what\n' + 'can be stored in available memory.\n' + '\n' + 'Note that leading zeros in a non-zero decimal number are not ' + 'allowed.\n' + 'This is for disambiguation with C-style octal literals, which ' + 'Python\n' + 'used before version 3.0.\n' + '\n' + 'Some examples of integer literals:\n' + '\n' + ' 7 2147483647 0o177 ' + '0b100110111\n' + ' 3 79228162514264337593543950336 0o377 ' + '0xdeadbeef\n', + 'lambda': '\n' + 'Lambdas\n' + '*******\n' + '\n' + ' lambda_expr ::= "lambda" [parameter_list]: expression\n' + ' lambda_expr_nocond ::= "lambda" [parameter_list]: ' + 'expression_nocond\n' + '\n' + 'Lambda expressions (sometimes called lambda forms) are used to ' + 'create\n' + 'anonymous functions. The expression "lambda arguments: ' + 'expression"\n' + 'yields a function object. The unnamed object behaves like a ' + 'function\n' + 'object defined with\n' + '\n' + ' def (arguments):\n' + ' return expression\n' + '\n' + 'See section *Function definitions* for the syntax of parameter ' + 'lists.\n' + 'Note that functions created with lambda expressions cannot ' + 'contain\n' + 'statements or annotations.\n', + 'lists': '\n' + 'List displays\n' + '*************\n' + '\n' + 'A list display is a possibly empty series of expressions enclosed ' + 'in\n' + 'square brackets:\n' + '\n' + ' list_display ::= "[" [expression_list | comprehension] "]"\n' + '\n' + 'A list display yields a new list object, the contents being ' + 'specified\n' + 'by either a list of expressions or a comprehension. When a ' + 'comma-\n' + 'separated list of expressions is supplied, its elements are ' + 'evaluated\n' + 'from left to right and placed into the list object in that ' + 'order.\n' + 'When a comprehension is supplied, the list is constructed from ' + 'the\n' + 'elements resulting from the comprehension.\n', + 'naming': '\n' + 'Naming and binding\n' + '******************\n' + '\n' + '\n' + 'Binding of names\n' + '================\n' + '\n' + '*Names* refer to objects. Names are introduced by name binding\n' + 'operations.\n' + '\n' + 'The following constructs bind names: formal parameters to ' + 'functions,\n' + '"import" statements, class and function definitions (these bind ' + 'the\n' + 'class or function name in the defining block), and targets that ' + 'are\n' + 'identifiers if occurring in an assignment, "for" loop header, or ' + 'after\n' + '"as" in a "with" statement or "except" clause. The "import" ' + 'statement\n' + 'of the form "from ... import *" binds all names defined in the\n' + 'imported module, except those beginning with an underscore. ' + 'This form\n' + 'may only be used at the module level.\n' + '\n' + 'A target occurring in a "del" statement is also considered bound ' + 'for\n' + 'this purpose (though the actual semantics are to unbind the ' + 'name).\n' + '\n' + 'Each assignment or import statement occurs within a block ' + 'defined by a\n' + 'class or function definition or at the module level (the ' + 'top-level\n' + 'code block).\n' + '\n' + 'If a name is bound in a block, it is a local variable of that ' + 'block,\n' + 'unless declared as "nonlocal" or "global". If a name is bound ' + 'at the\n' + 'module level, it is a global variable. (The variables of the ' + 'module\n' + 'code block are local and global.) If a variable is used in a ' + 'code\n' + 'block but not defined there, it is a *free variable*.\n' + '\n' + 'Each occurrence of a name in the program text refers to the ' + '*binding*\n' + 'of that name established by the following name resolution ' + 'rules.\n' + '\n' + '\n' + 'Resolution of names\n' + '===================\n' + '\n' + 'A *scope* defines the visibility of a name within a block. If a ' + 'local\n' + 'variable is defined in a block, its scope includes that block. ' + 'If the\n' + 'definition occurs in a function block, the scope extends to any ' + 'blocks\n' + 'contained within the defining one, unless a contained block ' + 'introduces\n' + 'a different binding for the name.\n' + '\n' + 'When a name is used in a code block, it is resolved using the ' + 'nearest\n' + 'enclosing scope. The set of all such scopes visible to a code ' + 'block\n' + "is called the block's *environment*.\n" + '\n' + 'When a name is not found at all, a "NameError" exception is ' + 'raised. If\n' + 'the current scope is a function scope, and the name refers to a ' + 'local\n' + 'variable that has not yet been bound to a value at the point ' + 'where the\n' + 'name is used, an "UnboundLocalError" exception is raised.\n' + '"UnboundLocalError" is a subclass of "NameError".\n' + '\n' + 'If a name binding operation occurs anywhere within a code block, ' + 'all\n' + 'uses of the name within the block are treated as references to ' + 'the\n' + 'current block. This can lead to errors when a name is used ' + 'within a\n' + 'block before it is bound. This rule is subtle. Python lacks\n' + 'declarations and allows name binding operations to occur ' + 'anywhere\n' + 'within a code block. The local variables of a code block can ' + 'be\n' + 'determined by scanning the entire text of the block for name ' + 'binding\n' + 'operations.\n' + '\n' + 'If the "global" statement occurs within a block, all uses of the ' + 'name\n' + 'specified in the statement refer to the binding of that name in ' + 'the\n' + 'top-level namespace. Names are resolved in the top-level ' + 'namespace by\n' + 'searching the global namespace, i.e. the namespace of the ' + 'module\n' + 'containing the code block, and the builtins namespace, the ' + 'namespace\n' + 'of the module "builtins". The global namespace is searched ' + 'first. If\n' + 'the name is not found there, the builtins namespace is ' + 'searched. The\n' + '"global" statement must precede all uses of the name.\n' + '\n' + 'The "global" statement has the same scope as a name binding ' + 'operation\n' + 'in the same block. If the nearest enclosing scope for a free ' + 'variable\n' + 'contains a global statement, the free variable is treated as a ' + 'global.\n' + '\n' + 'The "nonlocal" statement causes corresponding names to refer to\n' + 'previously bound variables in the nearest enclosing function ' + 'scope.\n' + '"SyntaxError" is raised at compile time if the given name does ' + 'not\n' + 'exist in any enclosing function scope.\n' + '\n' + 'The namespace for a module is automatically created the first ' + 'time a\n' + 'module is imported. The main module for a script is always ' + 'called\n' + '"__main__".\n' + '\n' + 'Class definition blocks and arguments to "exec()" and "eval()" ' + 'are\n' + 'special in the context of name resolution. A class definition is ' + 'an\n' + 'executable statement that may use and define names. These ' + 'references\n' + 'follow the normal rules for name resolution with an exception ' + 'that\n' + 'unbound local variables are looked up in the global namespace. ' + 'The\n' + 'namespace of the class definition becomes the attribute ' + 'dictionary of\n' + 'the class. The scope of names defined in a class block is ' + 'limited to\n' + 'the class block; it does not extend to the code blocks of ' + 'methods --\n' + 'this includes comprehensions and generator expressions since ' + 'they are\n' + 'implemented using a function scope. This means that the ' + 'following\n' + 'will fail:\n' + '\n' + ' class A:\n' + ' a = 42\n' + ' b = list(a + i for i in range(10))\n' + '\n' + '\n' + 'Builtins and restricted execution\n' + '=================================\n' + '\n' + 'The builtins namespace associated with the execution of a code ' + 'block\n' + 'is actually found by looking up the name "__builtins__" in its ' + 'global\n' + 'namespace; this should be a dictionary or a module (in the ' + 'latter case\n' + "the module's dictionary is used). By default, when in the " + '"__main__"\n' + 'module, "__builtins__" is the built-in module "builtins"; when ' + 'in any\n' + 'other module, "__builtins__" is an alias for the dictionary of ' + 'the\n' + '"builtins" module itself. "__builtins__" can be set to a ' + 'user-created\n' + 'dictionary to create a weak form of restricted execution.\n' + '\n' + '**CPython implementation detail:** Users should not touch\n' + '"__builtins__"; it is strictly an implementation detail. Users\n' + 'wanting to override values in the builtins namespace should ' + '"import"\n' + 'the "builtins" module and modify its attributes appropriately.\n' + '\n' + '\n' + 'Interaction with dynamic features\n' + '=================================\n' + '\n' + 'Name resolution of free variables occurs at runtime, not at ' + 'compile\n' + 'time. This means that the following code will print 42:\n' + '\n' + ' i = 10\n' + ' def f():\n' + ' print(i)\n' + ' i = 42\n' + ' f()\n' + '\n' + 'There are several cases where Python statements are illegal when ' + 'used\n' + 'in conjunction with nested scopes that contain free variables.\n' + '\n' + 'If a variable is referenced in an enclosing scope, it is illegal ' + 'to\n' + 'delete the name. An error will be reported at compile time.\n' + '\n' + 'The "eval()" and "exec()" functions do not have access to the ' + 'full\n' + 'environment for resolving names. Names may be resolved in the ' + 'local\n' + 'and global namespaces of the caller. Free variables are not ' + 'resolved\n' + 'in the nearest enclosing namespace, but in the global ' + 'namespace. [1]\n' + 'The "exec()" and "eval()" functions have optional arguments to\n' + 'override the global and local namespace. If only one namespace ' + 'is\n' + 'specified, it is used for both.\n', + 'nonlocal': '\n' + 'The "nonlocal" statement\n' + '************************\n' + '\n' + ' nonlocal_stmt ::= "nonlocal" identifier ("," identifier)*\n' + '\n' + 'The "nonlocal" statement causes the listed identifiers to ' + 'refer to\n' + 'previously bound variables in the nearest enclosing scope ' + 'excluding\n' + 'globals. This is important because the default behavior for ' + 'binding is\n' + 'to search the local namespace first. The statement allows\n' + 'encapsulated code to rebind variables outside of the local ' + 'scope\n' + 'besides the global (module) scope.\n' + '\n' + 'Names listed in a "nonlocal" statement, unlike those listed in ' + 'a\n' + '"global" statement, must refer to pre-existing bindings in an\n' + 'enclosing scope (the scope in which a new binding should be ' + 'created\n' + 'cannot be determined unambiguously).\n' + '\n' + 'Names listed in a "nonlocal" statement must not collide with ' + 'pre-\n' + 'existing bindings in the local scope.\n' + '\n' + 'See also: **PEP 3104** - Access to Names in Outer Scopes\n' + '\n' + ' The specification for the "nonlocal" statement.\n', + 'numbers': '\n' + 'Numeric literals\n' + '****************\n' + '\n' + 'There are three types of numeric literals: integers, floating ' + 'point\n' + 'numbers, and imaginary numbers. There are no complex literals\n' + '(complex numbers can be formed by adding a real number and an\n' + 'imaginary number).\n' + '\n' + 'Note that numeric literals do not include a sign; a phrase like ' + '"-1"\n' + 'is actually an expression composed of the unary operator ' + '\'"-"\' and the\n' + 'literal "1".\n', + 'numeric-types': '\n' + 'Emulating numeric types\n' + '***********************\n' + '\n' + 'The following methods can be defined to emulate numeric ' + 'objects.\n' + 'Methods corresponding to operations that are not ' + 'supported by the\n' + 'particular kind of number implemented (e.g., bitwise ' + 'operations for\n' + 'non-integral numbers) should be left undefined.\n' + '\n' + 'object.__add__(self, other)\n' + 'object.__sub__(self, other)\n' + 'object.__mul__(self, other)\n' + 'object.__matmul__(self, other)\n' + 'object.__truediv__(self, other)\n' + 'object.__floordiv__(self, other)\n' + 'object.__mod__(self, other)\n' + 'object.__divmod__(self, other)\n' + 'object.__pow__(self, other[, modulo])\n' + 'object.__lshift__(self, other)\n' + 'object.__rshift__(self, other)\n' + 'object.__and__(self, other)\n' + 'object.__xor__(self, other)\n' + 'object.__or__(self, other)\n' + '\n' + ' These methods are called to implement the binary ' + 'arithmetic\n' + ' operations ("+", "-", "*", "@", "/", "//", "%", ' + '"divmod()",\n' + ' "pow()", "**", "<<", ">>", "&", "^", "|"). For ' + 'instance, to\n' + ' evaluate the expression "x + y", where *x* is an ' + 'instance of a\n' + ' class that has an "__add__()" method, "x.__add__(y)" ' + 'is called.\n' + ' The "__divmod__()" method should be the equivalent to ' + 'using\n' + ' "__floordiv__()" and "__mod__()"; it should not be ' + 'related to\n' + ' "__truediv__()". Note that "__pow__()" should be ' + 'defined to accept\n' + ' an optional third argument if the ternary version of ' + 'the built-in\n' + ' "pow()" function is to be supported.\n' + '\n' + ' If one of those methods does not support the operation ' + 'with the\n' + ' supplied arguments, it should return ' + '"NotImplemented".\n' + '\n' + 'object.__radd__(self, other)\n' + 'object.__rsub__(self, other)\n' + 'object.__rmul__(self, other)\n' + 'object.__rmatmul__(self, other)\n' + 'object.__rtruediv__(self, other)\n' + 'object.__rfloordiv__(self, other)\n' + 'object.__rmod__(self, other)\n' + 'object.__rdivmod__(self, other)\n' + 'object.__rpow__(self, other)\n' + 'object.__rlshift__(self, other)\n' + 'object.__rrshift__(self, other)\n' + 'object.__rand__(self, other)\n' + 'object.__rxor__(self, other)\n' + 'object.__ror__(self, other)\n' + '\n' + ' These methods are called to implement the binary ' + 'arithmetic\n' + ' operations ("+", "-", "*", "@", "/", "//", "%", ' + '"divmod()",\n' + ' "pow()", "**", "<<", ">>", "&", "^", "|") with ' + 'reflected (swapped)\n' + ' operands. These functions are only called if the left ' + 'operand does\n' + ' not support the corresponding operation and the ' + 'operands are of\n' + ' different types. [2] For instance, to evaluate the ' + 'expression "x -\n' + ' y", where *y* is an instance of a class that has an ' + '"__rsub__()"\n' + ' method, "y.__rsub__(x)" is called if "x.__sub__(y)" ' + 'returns\n' + ' *NotImplemented*.\n' + '\n' + ' Note that ternary "pow()" will not try calling ' + '"__rpow__()" (the\n' + ' coercion rules would become too complicated).\n' + '\n' + " Note: If the right operand's type is a subclass of the " + 'left\n' + " operand's type and that subclass provides the " + 'reflected method\n' + ' for the operation, this method will be called before ' + 'the left\n' + " operand's non-reflected method. This behavior " + 'allows subclasses\n' + " to override their ancestors' operations.\n" + '\n' + 'object.__iadd__(self, other)\n' + 'object.__isub__(self, other)\n' + 'object.__imul__(self, other)\n' + 'object.__imatmul__(self, other)\n' + 'object.__itruediv__(self, other)\n' + 'object.__ifloordiv__(self, other)\n' + 'object.__imod__(self, other)\n' + 'object.__ipow__(self, other[, modulo])\n' + 'object.__ilshift__(self, other)\n' + 'object.__irshift__(self, other)\n' + 'object.__iand__(self, other)\n' + 'object.__ixor__(self, other)\n' + 'object.__ior__(self, other)\n' + '\n' + ' These methods are called to implement the augmented ' + 'arithmetic\n' + ' assignments ("+=", "-=", "*=", "@=", "/=", "//=", ' + '"%=", "**=",\n' + ' "<<=", ">>=", "&=", "^=", "|="). These methods should ' + 'attempt to\n' + ' do the operation in-place (modifying *self*) and ' + 'return the result\n' + ' (which could be, but does not have to be, *self*). If ' + 'a specific\n' + ' method is not defined, the augmented assignment falls ' + 'back to the\n' + ' normal methods. For instance, if *x* is an instance ' + 'of a class\n' + ' with an "__iadd__()" method, "x += y" is equivalent to ' + '"x =\n' + ' x.__iadd__(y)" . Otherwise, "x.__add__(y)" and ' + '"y.__radd__(x)" are\n' + ' considered, as with the evaluation of "x + y". In ' + 'certain\n' + ' situations, augmented assignment can result in ' + 'unexpected errors\n' + " (see *Why does a_tuple[i] += ['item'] raise an " + 'exception when the\n' + ' addition works?*), but this behavior is in fact part ' + 'of the data\n' + ' model.\n' + '\n' + 'object.__neg__(self)\n' + 'object.__pos__(self)\n' + 'object.__abs__(self)\n' + 'object.__invert__(self)\n' + '\n' + ' Called to implement the unary arithmetic operations ' + '("-", "+",\n' + ' "abs()" and "~").\n' + '\n' + 'object.__complex__(self)\n' + 'object.__int__(self)\n' + 'object.__float__(self)\n' + 'object.__round__(self[, n])\n' + '\n' + ' Called to implement the built-in functions ' + '"complex()", "int()",\n' + ' "float()" and "round()". Should return a value of the ' + 'appropriate\n' + ' type.\n' + '\n' + 'object.__index__(self)\n' + '\n' + ' Called to implement "operator.index()", and whenever ' + 'Python needs\n' + ' to losslessly convert the numeric object to an integer ' + 'object (such\n' + ' as in slicing, or in the built-in "bin()", "hex()" and ' + '"oct()"\n' + ' functions). Presence of this method indicates that the ' + 'numeric\n' + ' object is an integer type. Must return an integer.\n' + '\n' + ' Note: In order to have a coherent integer type class, ' + 'when\n' + ' "__index__()" is defined "__int__()" should also be ' + 'defined, and\n' + ' both should return the same value.\n', + 'objects': '\n' + 'Objects, values and types\n' + '*************************\n' + '\n' + "*Objects* are Python's abstraction for data. All data in a " + 'Python\n' + 'program is represented by objects or by relations between ' + 'objects. (In\n' + "a sense, and in conformance to Von Neumann's model of a " + '"stored\n' + 'program computer," code is also represented by objects.)\n' + '\n' + "Every object has an identity, a type and a value. An object's\n" + '*identity* never changes once it has been created; you may ' + 'think of it\n' + 'as the object\'s address in memory. The \'"is"\' operator ' + 'compares the\n' + 'identity of two objects; the "id()" function returns an ' + 'integer\n' + 'representing its identity.\n' + '\n' + '**CPython implementation detail:** For CPython, "id(x)" is the ' + 'memory\n' + 'address where "x" is stored.\n' + '\n' + "An object's type determines the operations that the object " + 'supports\n' + '(e.g., "does it have a length?") and also defines the possible ' + 'values\n' + 'for objects of that type. The "type()" function returns an ' + "object's\n" + 'type (which is an object itself). Like its identity, an ' + "object's\n" + '*type* is also unchangeable. [1]\n' + '\n' + 'The *value* of some objects can change. Objects whose value ' + 'can\n' + 'change are said to be *mutable*; objects whose value is ' + 'unchangeable\n' + 'once they are created are called *immutable*. (The value of an\n' + 'immutable container object that contains a reference to a ' + 'mutable\n' + "object can change when the latter's value is changed; however " + 'the\n' + 'container is still considered immutable, because the collection ' + 'of\n' + 'objects it contains cannot be changed. So, immutability is ' + 'not\n' + 'strictly the same as having an unchangeable value, it is more ' + 'subtle.)\n' + "An object's mutability is determined by its type; for " + 'instance,\n' + 'numbers, strings and tuples are immutable, while dictionaries ' + 'and\n' + 'lists are mutable.\n' + '\n' + 'Objects are never explicitly destroyed; however, when they ' + 'become\n' + 'unreachable they may be garbage-collected. An implementation ' + 'is\n' + 'allowed to postpone garbage collection or omit it altogether ' + '--- it is\n' + 'a matter of implementation quality how garbage collection is\n' + 'implemented, as long as no objects are collected that are ' + 'still\n' + 'reachable.\n' + '\n' + '**CPython implementation detail:** CPython currently uses a ' + 'reference-\n' + 'counting scheme with (optional) delayed detection of cyclically ' + 'linked\n' + 'garbage, which collects most objects as soon as they become\n' + 'unreachable, but is not guaranteed to collect garbage ' + 'containing\n' + 'circular references. See the documentation of the "gc" module ' + 'for\n' + 'information on controlling the collection of cyclic garbage. ' + 'Other\n' + 'implementations act differently and CPython may change. Do not ' + 'depend\n' + 'on immediate finalization of objects when they become ' + 'unreachable (so\n' + 'you should always close files explicitly).\n' + '\n' + "Note that the use of the implementation's tracing or debugging\n" + 'facilities may keep objects alive that would normally be ' + 'collectable.\n' + 'Also note that catching an exception with a ' + '\'"try"..."except"\'\n' + 'statement may keep objects alive.\n' + '\n' + 'Some objects contain references to "external" resources such as ' + 'open\n' + 'files or windows. It is understood that these resources are ' + 'freed\n' + 'when the object is garbage-collected, but since garbage ' + 'collection is\n' + 'not guaranteed to happen, such objects also provide an explicit ' + 'way to\n' + 'release the external resource, usually a "close()" method. ' + 'Programs\n' + 'are strongly recommended to explicitly close such objects. ' + 'The\n' + '\'"try"..."finally"\' statement and the \'"with"\' statement ' + 'provide\n' + 'convenient ways to do this.\n' + '\n' + 'Some objects contain references to other objects; these are ' + 'called\n' + '*containers*. Examples of containers are tuples, lists and\n' + "dictionaries. The references are part of a container's value. " + 'In\n' + 'most cases, when we talk about the value of a container, we ' + 'imply the\n' + 'values, not the identities of the contained objects; however, ' + 'when we\n' + 'talk about the mutability of a container, only the identities ' + 'of the\n' + 'immediately contained objects are implied. So, if an ' + 'immutable\n' + 'container (like a tuple) contains a reference to a mutable ' + 'object, its\n' + 'value changes if that mutable object is changed.\n' + '\n' + 'Types affect almost all aspects of object behavior. Even the\n' + 'importance of object identity is affected in some sense: for ' + 'immutable\n' + 'types, operations that compute new values may actually return ' + 'a\n' + 'reference to any existing object with the same type and value, ' + 'while\n' + 'for mutable objects this is not allowed. E.g., after "a = 1; b ' + '= 1",\n' + '"a" and "b" may or may not refer to the same object with the ' + 'value\n' + 'one, depending on the implementation, but after "c = []; d = ' + '[]", "c"\n' + 'and "d" are guaranteed to refer to two different, unique, ' + 'newly\n' + 'created empty lists. (Note that "c = d = []" assigns the same ' + 'object\n' + 'to both "c" and "d".)\n', + 'operator-summary': '\n' + 'Operator precedence\n' + '*******************\n' + '\n' + 'The following table summarizes the operator precedence ' + 'in Python, from\n' + 'lowest precedence (least binding) to highest ' + 'precedence (most\n' + 'binding). Operators in the same box have the same ' + 'precedence. Unless\n' + 'the syntax is explicitly given, operators are binary. ' + 'Operators in\n' + 'the same box group left to right (except for ' + 'exponentiation, which\n' + 'groups from right to left).\n' + '\n' + 'Note that comparisons, membership tests, and identity ' + 'tests, all have\n' + 'the same precedence and have a left-to-right chaining ' + 'feature as\n' + 'described in the *Comparisons* section.\n' + '\n' + '+-------------------------------------------------+---------------------------------------+\n' + '| Operator | ' + 'Description |\n' + '+=================================================+=======================================+\n' + '| "lambda" | ' + 'Lambda expression |\n' + '+-------------------------------------------------+---------------------------------------+\n' + '| "if" -- "else" | ' + 'Conditional expression |\n' + '+-------------------------------------------------+---------------------------------------+\n' + '| "or" | ' + 'Boolean OR |\n' + '+-------------------------------------------------+---------------------------------------+\n' + '| "and" | ' + 'Boolean AND |\n' + '+-------------------------------------------------+---------------------------------------+\n' + '| "not" "x" | ' + 'Boolean NOT |\n' + '+-------------------------------------------------+---------------------------------------+\n' + '| "in", "not in", "is", "is not", "<", "<=", ">", | ' + 'Comparisons, including membership |\n' + '| ">=", "!=", "==" | ' + 'tests and identity tests |\n' + '+-------------------------------------------------+---------------------------------------+\n' + '| "|" | ' + 'Bitwise OR |\n' + '+-------------------------------------------------+---------------------------------------+\n' + '| "^" | ' + 'Bitwise XOR |\n' + '+-------------------------------------------------+---------------------------------------+\n' + '| "&" | ' + 'Bitwise AND |\n' + '+-------------------------------------------------+---------------------------------------+\n' + '| "<<", ">>" | ' + 'Shifts |\n' + '+-------------------------------------------------+---------------------------------------+\n' + '| "+", "-" | ' + 'Addition and subtraction |\n' + '+-------------------------------------------------+---------------------------------------+\n' + '| "*", "@", "/", "//", "%" | ' + 'Multiplication, matrix multiplication |\n' + '| | ' + 'division, remainder [5] |\n' + '+-------------------------------------------------+---------------------------------------+\n' + '| "+x", "-x", "~x" | ' + 'Positive, negative, bitwise NOT |\n' + '+-------------------------------------------------+---------------------------------------+\n' + '| "**" | ' + 'Exponentiation [6] |\n' + '+-------------------------------------------------+---------------------------------------+\n' + '| "await" "x" | ' + 'Await expression |\n' + '+-------------------------------------------------+---------------------------------------+\n' + '| "x[index]", "x[index:index]", | ' + 'Subscription, slicing, call, |\n' + '| "x(arguments...)", "x.attribute" | ' + 'attribute reference |\n' + '+-------------------------------------------------+---------------------------------------+\n' + '| "(expressions...)", "[expressions...]", "{key: | ' + 'Binding or tuple display, list |\n' + '| value...}", "{expressions...}" | ' + 'display, dictionary display, set |\n' + '| | ' + 'display |\n' + '+-------------------------------------------------+---------------------------------------+\n' + '\n' + '-[ Footnotes ]-\n' + '\n' + '[1] While "abs(x%y) < abs(y)" is true mathematically, ' + 'for floats\n' + ' it may not be true numerically due to roundoff. ' + 'For example, and\n' + ' assuming a platform on which a Python float is an ' + 'IEEE 754 double-\n' + ' precision number, in order that "-1e-100 % 1e100" ' + 'have the same\n' + ' sign as "1e100", the computed result is "-1e-100 + ' + '1e100", which\n' + ' is numerically exactly equal to "1e100". The ' + 'function\n' + ' "math.fmod()" returns a result whose sign matches ' + 'the sign of the\n' + ' first argument instead, and so returns "-1e-100" ' + 'in this case.\n' + ' Which approach is more appropriate depends on the ' + 'application.\n' + '\n' + '[2] If x is very close to an exact integer multiple of ' + "y, it's\n" + ' possible for "x//y" to be one larger than ' + '"(x-x%y)//y" due to\n' + ' rounding. In such cases, Python returns the ' + 'latter result, in\n' + ' order to preserve that "divmod(x,y)[0] * y + x % ' + 'y" be very close\n' + ' to "x".\n' + '\n' + '[3] While comparisons between strings make sense at ' + 'the byte\n' + ' level, they may be counter-intuitive to users. ' + 'For example, the\n' + ' strings ""\\u00C7"" and ""\\u0043\\u0327"" compare ' + 'differently, even\n' + ' though they both represent the same unicode ' + 'character (LATIN\n' + ' CAPITAL LETTER C WITH CEDILLA). To compare ' + 'strings in a human\n' + ' recognizable way, compare using ' + '"unicodedata.normalize()".\n' + '\n' + '[4] Due to automatic garbage-collection, free lists, ' + 'and the\n' + ' dynamic nature of descriptors, you may notice ' + 'seemingly unusual\n' + ' behaviour in certain uses of the "is" operator, ' + 'like those\n' + ' involving comparisons between instance methods, or ' + 'constants.\n' + ' Check their documentation for more info.\n' + '\n' + '[5] The "%" operator is also used for string ' + 'formatting; the same\n' + ' precedence applies.\n' + '\n' + '[6] The power operator "**" binds less tightly than an ' + 'arithmetic\n' + ' or bitwise unary operator on its right, that is, ' + '"2**-1" is "0.5".\n', + 'pass': '\n' + 'The "pass" statement\n' + '********************\n' + '\n' + ' pass_stmt ::= "pass"\n' + '\n' + '"pass" is a null operation --- when it is executed, nothing ' + 'happens.\n' + 'It is useful as a placeholder when a statement is required\n' + 'syntactically, but no code needs to be executed, for example:\n' + '\n' + ' def f(arg): pass # a function that does nothing (yet)\n' + '\n' + ' class C: pass # a class with no methods (yet)\n', + 'power': '\n' + 'The power operator\n' + '******************\n' + '\n' + 'The power operator binds more tightly than unary operators on ' + 'its\n' + 'left; it binds less tightly than unary operators on its right. ' + 'The\n' + 'syntax is:\n' + '\n' + ' power ::= await ["**" u_expr]\n' + '\n' + 'Thus, in an unparenthesized sequence of power and unary ' + 'operators, the\n' + 'operators are evaluated from right to left (this does not ' + 'constrain\n' + 'the evaluation order for the operands): "-1**2" results in "-1".\n' + '\n' + 'The power operator has the same semantics as the built-in ' + '"pow()"\n' + 'function, when called with two arguments: it yields its left ' + 'argument\n' + 'raised to the power of its right argument. The numeric arguments ' + 'are\n' + 'first converted to a common type, and the result is of that ' + 'type.\n' + '\n' + 'For int operands, the result has the same type as the operands ' + 'unless\n' + 'the second argument is negative; in that case, all arguments are\n' + 'converted to float and a float result is delivered. For example,\n' + '"10**2" returns "100", but "10**-2" returns "0.01".\n' + '\n' + 'Raising "0.0" to a negative power results in a ' + '"ZeroDivisionError".\n' + 'Raising a negative number to a fractional power results in a ' + '"complex"\n' + 'number. (In earlier versions it raised a "ValueError".)\n', + 'raise': '\n' + 'The "raise" statement\n' + '*********************\n' + '\n' + ' raise_stmt ::= "raise" [expression ["from" expression]]\n' + '\n' + 'If no expressions are present, "raise" re-raises the last ' + 'exception\n' + 'that was active in the current scope. If no exception is active ' + 'in\n' + 'the current scope, a "RuntimeError" exception is raised ' + 'indicating\n' + 'that this is an error.\n' + '\n' + 'Otherwise, "raise" evaluates the first expression as the ' + 'exception\n' + 'object. It must be either a subclass or an instance of\n' + '"BaseException". If it is a class, the exception instance will ' + 'be\n' + 'obtained when needed by instantiating the class with no ' + 'arguments.\n' + '\n' + "The *type* of the exception is the exception instance's class, " + 'the\n' + '*value* is the instance itself.\n' + '\n' + 'A traceback object is normally created automatically when an ' + 'exception\n' + 'is raised and attached to it as the "__traceback__" attribute, ' + 'which\n' + 'is writable. You can create an exception and set your own ' + 'traceback in\n' + 'one step using the "with_traceback()" exception method (which ' + 'returns\n' + 'the same exception instance, with its traceback set to its ' + 'argument),\n' + 'like so:\n' + '\n' + ' raise Exception("foo occurred").with_traceback(tracebackobj)\n' + '\n' + 'The "from" clause is used for exception chaining: if given, the ' + 'second\n' + '*expression* must be another exception class or instance, which ' + 'will\n' + 'then be attached to the raised exception as the "__cause__" ' + 'attribute\n' + '(which is writable). If the raised exception is not handled, ' + 'both\n' + 'exceptions will be printed:\n' + '\n' + ' >>> try:\n' + ' ... print(1 / 0)\n' + ' ... except Exception as exc:\n' + ' ... raise RuntimeError("Something bad happened") from exc\n' + ' ...\n' + ' Traceback (most recent call last):\n' + ' File "", line 2, in \n' + ' ZeroDivisionError: int division or modulo by zero\n' + '\n' + ' The above exception was the direct cause of the following ' + 'exception:\n' + '\n' + ' Traceback (most recent call last):\n' + ' File "", line 4, in \n' + ' RuntimeError: Something bad happened\n' + '\n' + 'A similar mechanism works implicitly if an exception is raised ' + 'inside\n' + 'an exception handler or a "finally" clause: the previous ' + 'exception is\n' + 'then attached as the new exception\'s "__context__" attribute:\n' + '\n' + ' >>> try:\n' + ' ... print(1 / 0)\n' + ' ... except:\n' + ' ... raise RuntimeError("Something bad happened")\n' + ' ...\n' + ' Traceback (most recent call last):\n' + ' File "", line 2, in \n' + ' ZeroDivisionError: int division or modulo by zero\n' + '\n' + ' During handling of the above exception, another exception ' + 'occurred:\n' + '\n' + ' Traceback (most recent call last):\n' + ' File "", line 4, in \n' + ' RuntimeError: Something bad happened\n' + '\n' + 'Additional information on exceptions can be found in section\n' + '*Exceptions*, and information about handling exceptions is in ' + 'section\n' + '*The try statement*.\n', + 'return': '\n' + 'The "return" statement\n' + '**********************\n' + '\n' + ' return_stmt ::= "return" [expression_list]\n' + '\n' + '"return" may only occur syntactically nested in a function ' + 'definition,\n' + 'not within a nested class definition.\n' + '\n' + 'If an expression list is present, it is evaluated, else "None" ' + 'is\n' + 'substituted.\n' + '\n' + '"return" leaves the current function call with the expression ' + 'list (or\n' + '"None") as return value.\n' + '\n' + 'When "return" passes control out of a "try" statement with a ' + '"finally"\n' + 'clause, that "finally" clause is executed before really leaving ' + 'the\n' + 'function.\n' + '\n' + 'In a generator function, the "return" statement indicates that ' + 'the\n' + 'generator is done and will cause "StopIteration" to be raised. ' + 'The\n' + 'returned value (if any) is used as an argument to construct\n' + '"StopIteration" and becomes the "StopIteration.value" ' + 'attribute.\n', + 'sequence-types': '\n' + 'Emulating container types\n' + '*************************\n' + '\n' + 'The following methods can be defined to implement ' + 'container objects.\n' + 'Containers usually are sequences (such as lists or ' + 'tuples) or mappings\n' + '(like dictionaries), but can represent other containers ' + 'as well. The\n' + 'first set of methods is used either to emulate a ' + 'sequence or to\n' + 'emulate a mapping; the difference is that for a ' + 'sequence, the\n' + 'allowable keys should be the integers *k* for which "0 ' + '<= k < N" where\n' + '*N* is the length of the sequence, or slice objects, ' + 'which define a\n' + 'range of items. It is also recommended that mappings ' + 'provide the\n' + 'methods "keys()", "values()", "items()", "get()", ' + '"clear()",\n' + '"setdefault()", "pop()", "popitem()", "copy()", and ' + '"update()"\n' + "behaving similar to those for Python's standard " + 'dictionary objects.\n' + 'The "collections" module provides a "MutableMapping" ' + 'abstract base\n' + 'class to help create those methods from a base set of ' + '"__getitem__()",\n' + '"__setitem__()", "__delitem__()", and "keys()". Mutable ' + 'sequences\n' + 'should provide methods "append()", "count()", "index()", ' + '"extend()",\n' + '"insert()", "pop()", "remove()", "reverse()" and ' + '"sort()", like Python\n' + 'standard list objects. Finally, sequence types should ' + 'implement\n' + 'addition (meaning concatenation) and multiplication ' + '(meaning\n' + 'repetition) by defining the methods "__add__()", ' + '"__radd__()",\n' + '"__iadd__()", "__mul__()", "__rmul__()" and "__imul__()" ' + 'described\n' + 'below; they should not define other numerical ' + 'operators. It is\n' + 'recommended that both mappings and sequences implement ' + 'the\n' + '"__contains__()" method to allow efficient use of the ' + '"in" operator;\n' + 'for mappings, "in" should search the mapping\'s keys; ' + 'for sequences, it\n' + 'should search through the values. It is further ' + 'recommended that both\n' + 'mappings and sequences implement the "__iter__()" method ' + 'to allow\n' + 'efficient iteration through the container; for mappings, ' + '"__iter__()"\n' + 'should be the same as "keys()"; for sequences, it should ' + 'iterate\n' + 'through the values.\n' + '\n' + 'object.__len__(self)\n' + '\n' + ' Called to implement the built-in function "len()". ' + 'Should return\n' + ' the length of the object, an integer ">=" 0. Also, ' + 'an object that\n' + ' doesn\'t define a "__bool__()" method and whose ' + '"__len__()" method\n' + ' returns zero is considered to be false in a Boolean ' + 'context.\n' + '\n' + 'object.__length_hint__(self)\n' + '\n' + ' Called to implement "operator.length_hint()". Should ' + 'return an\n' + ' estimated length for the object (which may be greater ' + 'or less than\n' + ' the actual length). The length must be an integer ' + '">=" 0. This\n' + ' method is purely an optimization and is never ' + 'required for\n' + ' correctness.\n' + '\n' + ' New in version 3.4.\n' + '\n' + 'Note: Slicing is done exclusively with the following ' + 'three methods.\n' + ' A call like\n' + '\n' + ' a[1:2] = b\n' + '\n' + ' is translated to\n' + '\n' + ' a[slice(1, 2, None)] = b\n' + '\n' + ' and so forth. Missing slice items are always filled ' + 'in with "None".\n' + '\n' + 'object.__getitem__(self, key)\n' + '\n' + ' Called to implement evaluation of "self[key]". For ' + 'sequence types,\n' + ' the accepted keys should be integers and slice ' + 'objects. Note that\n' + ' the special interpretation of negative indexes (if ' + 'the class wishes\n' + ' to emulate a sequence type) is up to the ' + '"__getitem__()" method. If\n' + ' *key* is of an inappropriate type, "TypeError" may be ' + 'raised; if of\n' + ' a value outside the set of indexes for the sequence ' + '(after any\n' + ' special interpretation of negative values), ' + '"IndexError" should be\n' + ' raised. For mapping types, if *key* is missing (not ' + 'in the\n' + ' container), "KeyError" should be raised.\n' + '\n' + ' Note: "for" loops expect that an "IndexError" will be ' + 'raised for\n' + ' illegal indexes to allow proper detection of the ' + 'end of the\n' + ' sequence.\n' + '\n' + 'object.__missing__(self, key)\n' + '\n' + ' Called by "dict"."__getitem__()" to implement ' + '"self[key]" for dict\n' + ' subclasses when key is not in the dictionary.\n' + '\n' + 'object.__setitem__(self, key, value)\n' + '\n' + ' Called to implement assignment to "self[key]". Same ' + 'note as for\n' + ' "__getitem__()". This should only be implemented for ' + 'mappings if\n' + ' the objects support changes to the values for keys, ' + 'or if new keys\n' + ' can be added, or for sequences if elements can be ' + 'replaced. The\n' + ' same exceptions should be raised for improper *key* ' + 'values as for\n' + ' the "__getitem__()" method.\n' + '\n' + 'object.__delitem__(self, key)\n' + '\n' + ' Called to implement deletion of "self[key]". Same ' + 'note as for\n' + ' "__getitem__()". This should only be implemented for ' + 'mappings if\n' + ' the objects support removal of keys, or for sequences ' + 'if elements\n' + ' can be removed from the sequence. The same ' + 'exceptions should be\n' + ' raised for improper *key* values as for the ' + '"__getitem__()" method.\n' + '\n' + 'object.__iter__(self)\n' + '\n' + ' This method is called when an iterator is required ' + 'for a container.\n' + ' This method should return a new iterator object that ' + 'can iterate\n' + ' over all the objects in the container. For mappings, ' + 'it should\n' + ' iterate over the keys of the container.\n' + '\n' + ' Iterator objects also need to implement this method; ' + 'they are\n' + ' required to return themselves. For more information ' + 'on iterator\n' + ' objects, see *Iterator Types*.\n' + '\n' + 'object.__reversed__(self)\n' + '\n' + ' Called (if present) by the "reversed()" built-in to ' + 'implement\n' + ' reverse iteration. It should return a new iterator ' + 'object that\n' + ' iterates over all the objects in the container in ' + 'reverse order.\n' + '\n' + ' If the "__reversed__()" method is not provided, the ' + '"reversed()"\n' + ' built-in will fall back to using the sequence ' + 'protocol ("__len__()"\n' + ' and "__getitem__()"). Objects that support the ' + 'sequence protocol\n' + ' should only provide "__reversed__()" if they can ' + 'provide an\n' + ' implementation that is more efficient than the one ' + 'provided by\n' + ' "reversed()".\n' + '\n' + 'The membership test operators ("in" and "not in") are ' + 'normally\n' + 'implemented as an iteration through a sequence. ' + 'However, container\n' + 'objects can supply the following special method with a ' + 'more efficient\n' + 'implementation, which also does not require the object ' + 'be a sequence.\n' + '\n' + 'object.__contains__(self, item)\n' + '\n' + ' Called to implement membership test operators. ' + 'Should return true\n' + ' if *item* is in *self*, false otherwise. For mapping ' + 'objects, this\n' + ' should consider the keys of the mapping rather than ' + 'the values or\n' + ' the key-item pairs.\n' + '\n' + ' For objects that don\'t define "__contains__()", the ' + 'membership test\n' + ' first tries iteration via "__iter__()", then the old ' + 'sequence\n' + ' iteration protocol via "__getitem__()", see *this ' + 'section in the\n' + ' language reference*.\n', + 'shifting': '\n' + 'Shifting operations\n' + '*******************\n' + '\n' + 'The shifting operations have lower priority than the ' + 'arithmetic\n' + 'operations:\n' + '\n' + ' shift_expr ::= a_expr | shift_expr ( "<<" | ">>" ) a_expr\n' + '\n' + 'These operators accept integers as arguments. They shift the ' + 'first\n' + 'argument to the left or right by the number of bits given by ' + 'the\n' + 'second argument.\n' + '\n' + 'A right shift by *n* bits is defined as floor division by ' + '"pow(2,n)".\n' + 'A left shift by *n* bits is defined as multiplication with ' + '"pow(2,n)".\n' + '\n' + 'Note: In the current implementation, the right-hand operand ' + 'is\n' + ' required to be at most "sys.maxsize". If the right-hand ' + 'operand is\n' + ' larger than "sys.maxsize" an "OverflowError" exception is ' + 'raised.\n', + 'slicings': '\n' + 'Slicings\n' + '********\n' + '\n' + 'A slicing selects a range of items in a sequence object (e.g., ' + 'a\n' + 'string, tuple or list). Slicings may be used as expressions ' + 'or as\n' + 'targets in assignment or "del" statements. The syntax for a ' + 'slicing:\n' + '\n' + ' slicing ::= primary "[" slice_list "]"\n' + ' slice_list ::= slice_item ("," slice_item)* [","]\n' + ' slice_item ::= expression | proper_slice\n' + ' proper_slice ::= [lower_bound] ":" [upper_bound] [ ":" ' + '[stride] ]\n' + ' lower_bound ::= expression\n' + ' upper_bound ::= expression\n' + ' stride ::= expression\n' + '\n' + 'There is ambiguity in the formal syntax here: anything that ' + 'looks like\n' + 'an expression list also looks like a slice list, so any ' + 'subscription\n' + 'can be interpreted as a slicing. Rather than further ' + 'complicating the\n' + 'syntax, this is disambiguated by defining that in this case ' + 'the\n' + 'interpretation as a subscription takes priority over the\n' + 'interpretation as a slicing (this is the case if the slice ' + 'list\n' + 'contains no proper slice).\n' + '\n' + 'The semantics for a slicing are as follows. The primary is ' + 'indexed\n' + '(using the same "__getitem__()" method as normal subscription) ' + 'with a\n' + 'key that is constructed from the slice list, as follows. If ' + 'the slice\n' + 'list contains at least one comma, the key is a tuple ' + 'containing the\n' + 'conversion of the slice items; otherwise, the conversion of ' + 'the lone\n' + 'slice item is the key. The conversion of a slice item that is ' + 'an\n' + 'expression is that expression. The conversion of a proper ' + 'slice is a\n' + 'slice object (see section *The standard type hierarchy*) ' + 'whose\n' + '"start", "stop" and "step" attributes are the values of the\n' + 'expressions given as lower bound, upper bound and stride,\n' + 'respectively, substituting "None" for missing expressions.\n', + 'specialattrs': '\n' + 'Special Attributes\n' + '******************\n' + '\n' + 'The implementation adds a few special read-only attributes ' + 'to several\n' + 'object types, where they are relevant. Some of these are ' + 'not reported\n' + 'by the "dir()" built-in function.\n' + '\n' + 'object.__dict__\n' + '\n' + ' A dictionary or other mapping object used to store an ' + "object's\n" + ' (writable) attributes.\n' + '\n' + 'instance.__class__\n' + '\n' + ' The class to which a class instance belongs.\n' + '\n' + 'class.__bases__\n' + '\n' + ' The tuple of base classes of a class object.\n' + '\n' + 'class.__name__\n' + '\n' + ' The name of the class or type.\n' + '\n' + 'class.__qualname__\n' + '\n' + ' The *qualified name* of the class or type.\n' + '\n' + ' New in version 3.3.\n' + '\n' + 'class.__mro__\n' + '\n' + ' This attribute is a tuple of classes that are ' + 'considered when\n' + ' looking for base classes during method resolution.\n' + '\n' + 'class.mro()\n' + '\n' + ' This method can be overridden by a metaclass to ' + 'customize the\n' + ' method resolution order for its instances. It is ' + 'called at class\n' + ' instantiation, and its result is stored in "__mro__".\n' + '\n' + 'class.__subclasses__()\n' + '\n' + ' Each class keeps a list of weak references to its ' + 'immediate\n' + ' subclasses. This method returns a list of all those ' + 'references\n' + ' still alive. Example:\n' + '\n' + ' >>> int.__subclasses__()\n' + " []\n" + '\n' + '-[ Footnotes ]-\n' + '\n' + '[1] Additional information on these special methods may be ' + 'found\n' + ' in the Python Reference Manual (*Basic ' + 'customization*).\n' + '\n' + '[2] As a consequence, the list "[1, 2]" is considered ' + 'equal to\n' + ' "[1.0, 2.0]", and similarly for tuples.\n' + '\n' + "[3] They must have since the parser can't tell the type of " + 'the\n' + ' operands.\n' + '\n' + '[4] Cased characters are those with general category ' + 'property\n' + ' being one of "Lu" (Letter, uppercase), "Ll" (Letter, ' + 'lowercase),\n' + ' or "Lt" (Letter, titlecase).\n' + '\n' + '[5] To format only a tuple you should therefore provide a\n' + ' singleton tuple whose only element is the tuple to be ' + 'formatted.\n', + 'specialnames': '\n' + 'Special method names\n' + '********************\n' + '\n' + 'A class can implement certain operations that are invoked ' + 'by special\n' + 'syntax (such as arithmetic operations or subscripting and ' + 'slicing) by\n' + "defining methods with special names. This is Python's " + 'approach to\n' + '*operator overloading*, allowing classes to define their ' + 'own behavior\n' + 'with respect to language operators. For instance, if a ' + 'class defines\n' + 'a method named "__getitem__()", and "x" is an instance of ' + 'this class,\n' + 'then "x[i]" is roughly equivalent to ' + '"type(x).__getitem__(x, i)".\n' + 'Except where mentioned, attempts to execute an operation ' + 'raise an\n' + 'exception when no appropriate method is defined ' + '(typically\n' + '"AttributeError" or "TypeError").\n' + '\n' + 'When implementing a class that emulates any built-in type, ' + 'it is\n' + 'important that the emulation only be implemented to the ' + 'degree that it\n' + 'makes sense for the object being modelled. For example, ' + 'some\n' + 'sequences may work well with retrieval of individual ' + 'elements, but\n' + 'extracting a slice may not make sense. (One example of ' + 'this is the\n' + '"NodeList" interface in the W3C\'s Document Object ' + 'Model.)\n' + '\n' + '\n' + 'Basic customization\n' + '===================\n' + '\n' + 'object.__new__(cls[, ...])\n' + '\n' + ' Called to create a new instance of class *cls*. ' + '"__new__()" is a\n' + ' static method (special-cased so you need not declare it ' + 'as such)\n' + ' that takes the class of which an instance was requested ' + 'as its\n' + ' first argument. The remaining arguments are those ' + 'passed to the\n' + ' object constructor expression (the call to the class). ' + 'The return\n' + ' value of "__new__()" should be the new object instance ' + '(usually an\n' + ' instance of *cls*).\n' + '\n' + ' Typical implementations create a new instance of the ' + 'class by\n' + ' invoking the superclass\'s "__new__()" method using\n' + ' "super(currentclass, cls).__new__(cls[, ...])" with ' + 'appropriate\n' + ' arguments and then modifying the newly-created instance ' + 'as\n' + ' necessary before returning it.\n' + '\n' + ' If "__new__()" returns an instance of *cls*, then the ' + 'new\n' + ' instance\'s "__init__()" method will be invoked like\n' + ' "__init__(self[, ...])", where *self* is the new ' + 'instance and the\n' + ' remaining arguments are the same as were passed to ' + '"__new__()".\n' + '\n' + ' If "__new__()" does not return an instance of *cls*, ' + 'then the new\n' + ' instance\'s "__init__()" method will not be invoked.\n' + '\n' + ' "__new__()" is intended mainly to allow subclasses of ' + 'immutable\n' + ' types (like int, str, or tuple) to customize instance ' + 'creation. It\n' + ' is also commonly overridden in custom metaclasses in ' + 'order to\n' + ' customize class creation.\n' + '\n' + 'object.__init__(self[, ...])\n' + '\n' + ' Called after the instance has been created (by ' + '"__new__()"), but\n' + ' before it is returned to the caller. The arguments are ' + 'those\n' + ' passed to the class constructor expression. If a base ' + 'class has an\n' + ' "__init__()" method, the derived class\'s "__init__()" ' + 'method, if\n' + ' any, must explicitly call it to ensure proper ' + 'initialization of the\n' + ' base class part of the instance; for example:\n' + ' "BaseClass.__init__(self, [args...])".\n' + '\n' + ' Because "__new__()" and "__init__()" work together in ' + 'constructing\n' + ' objects ("__new__()" to create it, and "__init__()" to ' + 'customise\n' + ' it), no non-"None" value may be returned by ' + '"__init__()"; doing so\n' + ' will cause a "TypeError" to be raised at runtime.\n' + '\n' + 'object.__del__(self)\n' + '\n' + ' Called when the instance is about to be destroyed. ' + 'This is also\n' + ' called a destructor. If a base class has a "__del__()" ' + 'method, the\n' + ' derived class\'s "__del__()" method, if any, must ' + 'explicitly call it\n' + ' to ensure proper deletion of the base class part of the ' + 'instance.\n' + ' Note that it is possible (though not recommended!) for ' + 'the\n' + ' "__del__()" method to postpone destruction of the ' + 'instance by\n' + ' creating a new reference to it. It may then be called ' + 'at a later\n' + ' time when this new reference is deleted. It is not ' + 'guaranteed that\n' + ' "__del__()" methods are called for objects that still ' + 'exist when\n' + ' the interpreter exits.\n' + '\n' + ' Note: "del x" doesn\'t directly call "x.__del__()" --- ' + 'the former\n' + ' decrements the reference count for "x" by one, and ' + 'the latter is\n' + ' only called when "x"\'s reference count reaches ' + 'zero. Some common\n' + ' situations that may prevent the reference count of an ' + 'object from\n' + ' going to zero include: circular references between ' + 'objects (e.g.,\n' + ' a doubly-linked list or a tree data structure with ' + 'parent and\n' + ' child pointers); a reference to the object on the ' + 'stack frame of\n' + ' a function that caught an exception (the traceback ' + 'stored in\n' + ' "sys.exc_info()[2]" keeps the stack frame alive); or ' + 'a reference\n' + ' to the object on the stack frame that raised an ' + 'unhandled\n' + ' exception in interactive mode (the traceback stored ' + 'in\n' + ' "sys.last_traceback" keeps the stack frame alive). ' + 'The first\n' + ' situation can only be remedied by explicitly breaking ' + 'the cycles;\n' + ' the second can be resolved by freeing the reference ' + 'to the\n' + ' traceback object when it is no longer useful, and the ' + 'third can\n' + ' be resolved by storing "None" in ' + '"sys.last_traceback". Circular\n' + ' references which are garbage are detected and cleaned ' + 'up when the\n' + " cyclic garbage collector is enabled (it's on by " + 'default). Refer\n' + ' to the documentation for the "gc" module for more ' + 'information\n' + ' about this topic.\n' + '\n' + ' Warning: Due to the precarious circumstances under ' + 'which\n' + ' "__del__()" methods are invoked, exceptions that ' + 'occur during\n' + ' their execution are ignored, and a warning is printed ' + 'to\n' + ' "sys.stderr" instead. Also, when "__del__()" is ' + 'invoked in\n' + ' response to a module being deleted (e.g., when ' + 'execution of the\n' + ' program is done), other globals referenced by the ' + '"__del__()"\n' + ' method may already have been deleted or in the ' + 'process of being\n' + ' torn down (e.g. the import machinery shutting down). ' + 'For this\n' + ' reason, "__del__()" methods should do the absolute ' + 'minimum needed\n' + ' to maintain external invariants. Starting with ' + 'version 1.5,\n' + ' Python guarantees that globals whose name begins with ' + 'a single\n' + ' underscore are deleted from their module before other ' + 'globals are\n' + ' deleted; if no other references to such globals ' + 'exist, this may\n' + ' help in assuring that imported modules are still ' + 'available at the\n' + ' time when the "__del__()" method is called.\n' + '\n' + 'object.__repr__(self)\n' + '\n' + ' Called by the "repr()" built-in function to compute the ' + '"official"\n' + ' string representation of an object. If at all ' + 'possible, this\n' + ' should look like a valid Python expression that could ' + 'be used to\n' + ' recreate an object with the same value (given an ' + 'appropriate\n' + ' environment). If this is not possible, a string of the ' + 'form\n' + ' "<...some useful description...>" should be returned. ' + 'The return\n' + ' value must be a string object. If a class defines ' + '"__repr__()" but\n' + ' not "__str__()", then "__repr__()" is also used when an ' + '"informal"\n' + ' string representation of instances of that class is ' + 'required.\n' + '\n' + ' This is typically used for debugging, so it is ' + 'important that the\n' + ' representation is information-rich and unambiguous.\n' + '\n' + 'object.__str__(self)\n' + '\n' + ' Called by "str(object)" and the built-in functions ' + '"format()" and\n' + ' "print()" to compute the "informal" or nicely printable ' + 'string\n' + ' representation of an object. The return value must be ' + 'a *string*\n' + ' object.\n' + '\n' + ' This method differs from "object.__repr__()" in that ' + 'there is no\n' + ' expectation that "__str__()" return a valid Python ' + 'expression: a\n' + ' more convenient or concise representation can be used.\n' + '\n' + ' The default implementation defined by the built-in type ' + '"object"\n' + ' calls "object.__repr__()".\n' + '\n' + 'object.__bytes__(self)\n' + '\n' + ' Called by "bytes()" to compute a byte-string ' + 'representation of an\n' + ' object. This should return a "bytes" object.\n' + '\n' + 'object.__format__(self, format_spec)\n' + '\n' + ' Called by the "format()" built-in function (and by ' + 'extension, the\n' + ' "str.format()" method of class "str") to produce a ' + '"formatted"\n' + ' string representation of an object. The "format_spec" ' + 'argument is a\n' + ' string that contains a description of the formatting ' + 'options\n' + ' desired. The interpretation of the "format_spec" ' + 'argument is up to\n' + ' the type implementing "__format__()", however most ' + 'classes will\n' + ' either delegate formatting to one of the built-in ' + 'types, or use a\n' + ' similar formatting option syntax.\n' + '\n' + ' See *Format Specification Mini-Language* for a ' + 'description of the\n' + ' standard formatting syntax.\n' + '\n' + ' The return value must be a string object.\n' + '\n' + ' Changed in version 3.4: The __format__ method of ' + '"object" itself\n' + ' raises a "TypeError" if passed any non-empty string.\n' + '\n' + 'object.__lt__(self, other)\n' + 'object.__le__(self, other)\n' + 'object.__eq__(self, other)\n' + 'object.__ne__(self, other)\n' + 'object.__gt__(self, other)\n' + 'object.__ge__(self, other)\n' + '\n' + ' These are the so-called "rich comparison" methods. The\n' + ' correspondence between operator symbols and method ' + 'names is as\n' + ' follows: "xy" calls\n' + ' "x.__gt__(y)", and "x>=y" calls "x.__ge__(y)".\n' + '\n' + ' A rich comparison method may return the singleton ' + '"NotImplemented"\n' + ' if it does not implement the operation for a given pair ' + 'of\n' + ' arguments. By convention, "False" and "True" are ' + 'returned for a\n' + ' successful comparison. However, these methods can ' + 'return any value,\n' + ' so if the comparison operator is used in a Boolean ' + 'context (e.g.,\n' + ' in the condition of an "if" statement), Python will ' + 'call "bool()"\n' + ' on the value to determine if the result is true or ' + 'false.\n' + '\n' + ' By default, "__ne__()" delegates to "__eq__()" and ' + 'inverts the\n' + ' result unless it is "NotImplemented". There are no ' + 'other implied\n' + ' relationships among the comparison operators, for ' + 'example, the\n' + ' truth of "(x.__hash__".\n' + '\n' + ' If a class that does not override "__eq__()" wishes to ' + 'suppress\n' + ' hash support, it should include "__hash__ = None" in ' + 'the class\n' + ' definition. A class which defines its own "__hash__()" ' + 'that\n' + ' explicitly raises a "TypeError" would be incorrectly ' + 'identified as\n' + ' hashable by an "isinstance(obj, collections.Hashable)" ' + 'call.\n' + '\n' + ' Note: By default, the "__hash__()" values of str, bytes ' + 'and\n' + ' datetime objects are "salted" with an unpredictable ' + 'random value.\n' + ' Although they remain constant within an individual ' + 'Python\n' + ' process, they are not predictable between repeated ' + 'invocations of\n' + ' Python.This is intended to provide protection against ' + 'a denial-\n' + ' of-service caused by carefully-chosen inputs that ' + 'exploit the\n' + ' worst case performance of a dict insertion, O(n^2) ' + 'complexity.\n' + ' See ' + 'http://www.ocert.org/advisories/ocert-2011-003.html for\n' + ' details.Changing hash values affects the iteration ' + 'order of\n' + ' dicts, sets and other mappings. Python has never ' + 'made guarantees\n' + ' about this ordering (and it typically varies between ' + '32-bit and\n' + ' 64-bit builds).See also "PYTHONHASHSEED".\n' + '\n' + ' Changed in version 3.3: Hash randomization is enabled ' + 'by default.\n' + '\n' + 'object.__bool__(self)\n' + '\n' + ' Called to implement truth value testing and the ' + 'built-in operation\n' + ' "bool()"; should return "False" or "True". When this ' + 'method is not\n' + ' defined, "__len__()" is called, if it is defined, and ' + 'the object is\n' + ' considered true if its result is nonzero. If a class ' + 'defines\n' + ' neither "__len__()" nor "__bool__()", all its instances ' + 'are\n' + ' considered true.\n' + '\n' + '\n' + 'Customizing attribute access\n' + '============================\n' + '\n' + 'The following methods can be defined to customize the ' + 'meaning of\n' + 'attribute access (use of, assignment to, or deletion of ' + '"x.name") for\n' + 'class instances.\n' + '\n' + 'object.__getattr__(self, name)\n' + '\n' + ' Called when an attribute lookup has not found the ' + 'attribute in the\n' + ' usual places (i.e. it is not an instance attribute nor ' + 'is it found\n' + ' in the class tree for "self"). "name" is the attribute ' + 'name. This\n' + ' method should return the (computed) attribute value or ' + 'raise an\n' + ' "AttributeError" exception.\n' + '\n' + ' Note that if the attribute is found through the normal ' + 'mechanism,\n' + ' "__getattr__()" is not called. (This is an intentional ' + 'asymmetry\n' + ' between "__getattr__()" and "__setattr__()".) This is ' + 'done both for\n' + ' efficiency reasons and because otherwise ' + '"__getattr__()" would have\n' + ' no way to access other attributes of the instance. ' + 'Note that at\n' + ' least for instance variables, you can fake total ' + 'control by not\n' + ' inserting any values in the instance attribute ' + 'dictionary (but\n' + ' instead inserting them in another object). See the\n' + ' "__getattribute__()" method below for a way to actually ' + 'get total\n' + ' control over attribute access.\n' + '\n' + 'object.__getattribute__(self, name)\n' + '\n' + ' Called unconditionally to implement attribute accesses ' + 'for\n' + ' instances of the class. If the class also defines ' + '"__getattr__()",\n' + ' the latter will not be called unless ' + '"__getattribute__()" either\n' + ' calls it explicitly or raises an "AttributeError". This ' + 'method\n' + ' should return the (computed) attribute value or raise ' + 'an\n' + ' "AttributeError" exception. In order to avoid infinite ' + 'recursion in\n' + ' this method, its implementation should always call the ' + 'base class\n' + ' method with the same name to access any attributes it ' + 'needs, for\n' + ' example, "object.__getattribute__(self, name)".\n' + '\n' + ' Note: This method may still be bypassed when looking up ' + 'special\n' + ' methods as the result of implicit invocation via ' + 'language syntax\n' + ' or built-in functions. See *Special method lookup*.\n' + '\n' + 'object.__setattr__(self, name, value)\n' + '\n' + ' Called when an attribute assignment is attempted. This ' + 'is called\n' + ' instead of the normal mechanism (i.e. store the value ' + 'in the\n' + ' instance dictionary). *name* is the attribute name, ' + '*value* is the\n' + ' value to be assigned to it.\n' + '\n' + ' If "__setattr__()" wants to assign to an instance ' + 'attribute, it\n' + ' should call the base class method with the same name, ' + 'for example,\n' + ' "object.__setattr__(self, name, value)".\n' + '\n' + 'object.__delattr__(self, name)\n' + '\n' + ' Like "__setattr__()" but for attribute deletion instead ' + 'of\n' + ' assignment. This should only be implemented if "del ' + 'obj.name" is\n' + ' meaningful for the object.\n' + '\n' + 'object.__dir__(self)\n' + '\n' + ' Called when "dir()" is called on the object. A sequence ' + 'must be\n' + ' returned. "dir()" converts the returned sequence to a ' + 'list and\n' + ' sorts it.\n' + '\n' + '\n' + 'Implementing Descriptors\n' + '------------------------\n' + '\n' + 'The following methods only apply when an instance of the ' + 'class\n' + 'containing the method (a so-called *descriptor* class) ' + 'appears in an\n' + '*owner* class (the descriptor must be in either the ' + "owner's class\n" + 'dictionary or in the class dictionary for one of its ' + 'parents). In the\n' + 'examples below, "the attribute" refers to the attribute ' + 'whose name is\n' + 'the key of the property in the owner class\' "__dict__".\n' + '\n' + 'object.__get__(self, instance, owner)\n' + '\n' + ' Called to get the attribute of the owner class (class ' + 'attribute\n' + ' access) or of an instance of that class (instance ' + 'attribute\n' + ' access). *owner* is always the owner class, while ' + '*instance* is the\n' + ' instance that the attribute was accessed through, or ' + '"None" when\n' + ' the attribute is accessed through the *owner*. This ' + 'method should\n' + ' return the (computed) attribute value or raise an ' + '"AttributeError"\n' + ' exception.\n' + '\n' + 'object.__set__(self, instance, value)\n' + '\n' + ' Called to set the attribute on an instance *instance* ' + 'of the owner\n' + ' class to a new value, *value*.\n' + '\n' + 'object.__delete__(self, instance)\n' + '\n' + ' Called to delete the attribute on an instance ' + '*instance* of the\n' + ' owner class.\n' + '\n' + 'The attribute "__objclass__" is interpreted by the ' + '"inspect" module as\n' + 'specifying the class where this object was defined ' + '(setting this\n' + 'appropriately can assist in runtime introspection of ' + 'dynamic class\n' + 'attributes). For callables, it may indicate that an ' + 'instance of the\n' + 'given type (or a subclass) is expected or required as the ' + 'first\n' + 'positional argument (for example, CPython sets this ' + 'attribute for\n' + 'unbound methods that are implemented in C).\n' + '\n' + '\n' + 'Invoking Descriptors\n' + '--------------------\n' + '\n' + 'In general, a descriptor is an object attribute with ' + '"binding\n' + 'behavior", one whose attribute access has been overridden ' + 'by methods\n' + 'in the descriptor protocol: "__get__()", "__set__()", ' + 'and\n' + '"__delete__()". If any of those methods are defined for an ' + 'object, it\n' + 'is said to be a descriptor.\n' + '\n' + 'The default behavior for attribute access is to get, set, ' + 'or delete\n' + "the attribute from an object's dictionary. For instance, " + '"a.x" has a\n' + 'lookup chain starting with "a.__dict__[\'x\']", then\n' + '"type(a).__dict__[\'x\']", and continuing through the base ' + 'classes of\n' + '"type(a)" excluding metaclasses.\n' + '\n' + 'However, if the looked-up value is an object defining one ' + 'of the\n' + 'descriptor methods, then Python may override the default ' + 'behavior and\n' + 'invoke the descriptor method instead. Where this occurs ' + 'in the\n' + 'precedence chain depends on which descriptor methods were ' + 'defined and\n' + 'how they were called.\n' + '\n' + 'The starting point for descriptor invocation is a binding, ' + '"a.x". How\n' + 'the arguments are assembled depends on "a":\n' + '\n' + 'Direct Call\n' + ' The simplest and least common call is when user code ' + 'directly\n' + ' invokes a descriptor method: "x.__get__(a)".\n' + '\n' + 'Instance Binding\n' + ' If binding to an object instance, "a.x" is transformed ' + 'into the\n' + ' call: "type(a).__dict__[\'x\'].__get__(a, type(a))".\n' + '\n' + 'Class Binding\n' + ' If binding to a class, "A.x" is transformed into the ' + 'call:\n' + ' "A.__dict__[\'x\'].__get__(None, A)".\n' + '\n' + 'Super Binding\n' + ' If "a" is an instance of "super", then the binding ' + '"super(B,\n' + ' obj).m()" searches "obj.__class__.__mro__" for the base ' + 'class "A"\n' + ' immediately preceding "B" and then invokes the ' + 'descriptor with the\n' + ' call: "A.__dict__[\'m\'].__get__(obj, obj.__class__)".\n' + '\n' + 'For instance bindings, the precedence of descriptor ' + 'invocation depends\n' + 'on the which descriptor methods are defined. A descriptor ' + 'can define\n' + 'any combination of "__get__()", "__set__()" and ' + '"__delete__()". If it\n' + 'does not define "__get__()", then accessing the attribute ' + 'will return\n' + 'the descriptor object itself unless there is a value in ' + "the object's\n" + 'instance dictionary. If the descriptor defines ' + '"__set__()" and/or\n' + '"__delete__()", it is a data descriptor; if it defines ' + 'neither, it is\n' + 'a non-data descriptor. Normally, data descriptors define ' + 'both\n' + '"__get__()" and "__set__()", while non-data descriptors ' + 'have just the\n' + '"__get__()" method. Data descriptors with "__set__()" and ' + '"__get__()"\n' + 'defined always override a redefinition in an instance ' + 'dictionary. In\n' + 'contrast, non-data descriptors can be overridden by ' + 'instances.\n' + '\n' + 'Python methods (including "staticmethod()" and ' + '"classmethod()") are\n' + 'implemented as non-data descriptors. Accordingly, ' + 'instances can\n' + 'redefine and override methods. This allows individual ' + 'instances to\n' + 'acquire behaviors that differ from other instances of the ' + 'same class.\n' + '\n' + 'The "property()" function is implemented as a data ' + 'descriptor.\n' + 'Accordingly, instances cannot override the behavior of a ' + 'property.\n' + '\n' + '\n' + '__slots__\n' + '---------\n' + '\n' + 'By default, instances of classes have a dictionary for ' + 'attribute\n' + 'storage. This wastes space for objects having very few ' + 'instance\n' + 'variables. The space consumption can become acute when ' + 'creating large\n' + 'numbers of instances.\n' + '\n' + 'The default can be overridden by defining *__slots__* in a ' + 'class\n' + 'definition. The *__slots__* declaration takes a sequence ' + 'of instance\n' + 'variables and reserves just enough space in each instance ' + 'to hold a\n' + 'value for each variable. Space is saved because ' + '*__dict__* is not\n' + 'created for each instance.\n' + '\n' + 'object.__slots__\n' + '\n' + ' This class variable can be assigned a string, iterable, ' + 'or sequence\n' + ' of strings with variable names used by instances. ' + '*__slots__*\n' + ' reserves space for the declared variables and prevents ' + 'the\n' + ' automatic creation of *__dict__* and *__weakref__* for ' + 'each\n' + ' instance.\n' + '\n' + '\n' + 'Notes on using *__slots__*\n' + '~~~~~~~~~~~~~~~~~~~~~~~~~~\n' + '\n' + '* When inheriting from a class without *__slots__*, the ' + '*__dict__*\n' + ' attribute of that class will always be accessible, so a ' + '*__slots__*\n' + ' definition in the subclass is meaningless.\n' + '\n' + '* Without a *__dict__* variable, instances cannot be ' + 'assigned new\n' + ' variables not listed in the *__slots__* definition. ' + 'Attempts to\n' + ' assign to an unlisted variable name raises ' + '"AttributeError". If\n' + ' dynamic assignment of new variables is desired, then ' + 'add\n' + ' "\'__dict__\'" to the sequence of strings in the ' + '*__slots__*\n' + ' declaration.\n' + '\n' + '* Without a *__weakref__* variable for each instance, ' + 'classes\n' + ' defining *__slots__* do not support weak references to ' + 'its\n' + ' instances. If weak reference support is needed, then ' + 'add\n' + ' "\'__weakref__\'" to the sequence of strings in the ' + '*__slots__*\n' + ' declaration.\n' + '\n' + '* *__slots__* are implemented at the class level by ' + 'creating\n' + ' descriptors (*Implementing Descriptors*) for each ' + 'variable name. As\n' + ' a result, class attributes cannot be used to set default ' + 'values for\n' + ' instance variables defined by *__slots__*; otherwise, ' + 'the class\n' + ' attribute would overwrite the descriptor assignment.\n' + '\n' + '* The action of a *__slots__* declaration is limited to ' + 'the class\n' + ' where it is defined. As a result, subclasses will have ' + 'a *__dict__*\n' + ' unless they also define *__slots__* (which must only ' + 'contain names\n' + ' of any *additional* slots).\n' + '\n' + '* If a class defines a slot also defined in a base class, ' + 'the\n' + ' instance variable defined by the base class slot is ' + 'inaccessible\n' + ' (except by retrieving its descriptor directly from the ' + 'base class).\n' + ' This renders the meaning of the program undefined. In ' + 'the future, a\n' + ' check may be added to prevent this.\n' + '\n' + '* Nonempty *__slots__* does not work for classes derived ' + 'from\n' + ' "variable-length" built-in types such as "int", "bytes" ' + 'and "tuple".\n' + '\n' + '* Any non-string iterable may be assigned to *__slots__*. ' + 'Mappings\n' + ' may also be used; however, in the future, special ' + 'meaning may be\n' + ' assigned to the values corresponding to each key.\n' + '\n' + '* *__class__* assignment works only if both classes have ' + 'the same\n' + ' *__slots__*.\n' + '\n' + '\n' + 'Customizing class creation\n' + '==========================\n' + '\n' + 'By default, classes are constructed using "type()". The ' + 'class body is\n' + 'executed in a new namespace and the class name is bound ' + 'locally to the\n' + 'result of "type(name, bases, namespace)".\n' + '\n' + 'The class creation process can be customised by passing ' + 'the\n' + '"metaclass" keyword argument in the class definition line, ' + 'or by\n' + 'inheriting from an existing class that included such an ' + 'argument. In\n' + 'the following example, both "MyClass" and "MySubclass" are ' + 'instances\n' + 'of "Meta":\n' + '\n' + ' class Meta(type):\n' + ' pass\n' + '\n' + ' class MyClass(metaclass=Meta):\n' + ' pass\n' + '\n' + ' class MySubclass(MyClass):\n' + ' pass\n' + '\n' + 'Any other keyword arguments that are specified in the ' + 'class definition\n' + 'are passed through to all metaclass operations described ' + 'below.\n' + '\n' + 'When a class definition is executed, the following steps ' + 'occur:\n' + '\n' + '* the appropriate metaclass is determined\n' + '\n' + '* the class namespace is prepared\n' + '\n' + '* the class body is executed\n' + '\n' + '* the class object is created\n' + '\n' + '\n' + 'Determining the appropriate metaclass\n' + '-------------------------------------\n' + '\n' + 'The appropriate metaclass for a class definition is ' + 'determined as\n' + 'follows:\n' + '\n' + '* if no bases and no explicit metaclass are given, then ' + '"type()" is\n' + ' used\n' + '\n' + '* if an explicit metaclass is given and it is *not* an ' + 'instance of\n' + ' "type()", then it is used directly as the metaclass\n' + '\n' + '* if an instance of "type()" is given as the explicit ' + 'metaclass, or\n' + ' bases are defined, then the most derived metaclass is ' + 'used\n' + '\n' + 'The most derived metaclass is selected from the explicitly ' + 'specified\n' + 'metaclass (if any) and the metaclasses (i.e. "type(cls)") ' + 'of all\n' + 'specified base classes. The most derived metaclass is one ' + 'which is a\n' + 'subtype of *all* of these candidate metaclasses. If none ' + 'of the\n' + 'candidate metaclasses meets that criterion, then the class ' + 'definition\n' + 'will fail with "TypeError".\n' + '\n' + '\n' + 'Preparing the class namespace\n' + '-----------------------------\n' + '\n' + 'Once the appropriate metaclass has been identified, then ' + 'the class\n' + 'namespace is prepared. If the metaclass has a ' + '"__prepare__" attribute,\n' + 'it is called as "namespace = metaclass.__prepare__(name, ' + 'bases,\n' + '**kwds)" (where the additional keyword arguments, if any, ' + 'come from\n' + 'the class definition).\n' + '\n' + 'If the metaclass has no "__prepare__" attribute, then the ' + 'class\n' + 'namespace is initialised as an empty "dict()" instance.\n' + '\n' + 'See also: **PEP 3115** - Metaclasses in Python 3000\n' + '\n' + ' Introduced the "__prepare__" namespace hook\n' + '\n' + '\n' + 'Executing the class body\n' + '------------------------\n' + '\n' + 'The class body is executed (approximately) as "exec(body, ' + 'globals(),\n' + 'namespace)". The key difference from a normal call to ' + '"exec()" is that\n' + 'lexical scoping allows the class body (including any ' + 'methods) to\n' + 'reference names from the current and outer scopes when the ' + 'class\n' + 'definition occurs inside a function.\n' + '\n' + 'However, even when the class definition occurs inside the ' + 'function,\n' + 'methods defined inside the class still cannot see names ' + 'defined at the\n' + 'class scope. Class variables must be accessed through the ' + 'first\n' + 'parameter of instance or class methods, and cannot be ' + 'accessed at all\n' + 'from static methods.\n' + '\n' + '\n' + 'Creating the class object\n' + '-------------------------\n' + '\n' + 'Once the class namespace has been populated by executing ' + 'the class\n' + 'body, the class object is created by calling ' + '"metaclass(name, bases,\n' + 'namespace, **kwds)" (the additional keywords passed here ' + 'are the same\n' + 'as those passed to "__prepare__").\n' + '\n' + 'This class object is the one that will be referenced by ' + 'the zero-\n' + 'argument form of "super()". "__class__" is an implicit ' + 'closure\n' + 'reference created by the compiler if any methods in a ' + 'class body refer\n' + 'to either "__class__" or "super". This allows the zero ' + 'argument form\n' + 'of "super()" to correctly identify the class being defined ' + 'based on\n' + 'lexical scoping, while the class or instance that was used ' + 'to make the\n' + 'current call is identified based on the first argument ' + 'passed to the\n' + 'method.\n' + '\n' + 'After the class object is created, it is passed to the ' + 'class\n' + 'decorators included in the class definition (if any) and ' + 'the resulting\n' + 'object is bound in the local namespace as the defined ' + 'class.\n' + '\n' + 'See also: **PEP 3135** - New super\n' + '\n' + ' Describes the implicit "__class__" closure reference\n' + '\n' + '\n' + 'Metaclass example\n' + '-----------------\n' + '\n' + 'The potential uses for metaclasses are boundless. Some ' + 'ideas that have\n' + 'been explored include logging, interface checking, ' + 'automatic\n' + 'delegation, automatic property creation, proxies, ' + 'frameworks, and\n' + 'automatic resource locking/synchronization.\n' + '\n' + 'Here is an example of a metaclass that uses an\n' + '"collections.OrderedDict" to remember the order that class ' + 'variables\n' + 'are defined:\n' + '\n' + ' class OrderedClass(type):\n' + '\n' + ' @classmethod\n' + ' def __prepare__(metacls, name, bases, **kwds):\n' + ' return collections.OrderedDict()\n' + '\n' + ' def __new__(cls, name, bases, namespace, **kwds):\n' + ' result = type.__new__(cls, name, bases, ' + 'dict(namespace))\n' + ' result.members = tuple(namespace)\n' + ' return result\n' + '\n' + ' class A(metaclass=OrderedClass):\n' + ' def one(self): pass\n' + ' def two(self): pass\n' + ' def three(self): pass\n' + ' def four(self): pass\n' + '\n' + ' >>> A.members\n' + " ('__module__', 'one', 'two', 'three', 'four')\n" + '\n' + 'When the class definition for *A* gets executed, the ' + 'process begins\n' + 'with calling the metaclass\'s "__prepare__()" method which ' + 'returns an\n' + 'empty "collections.OrderedDict". That mapping records the ' + 'methods and\n' + 'attributes of *A* as they are defined within the body of ' + 'the class\n' + 'statement. Once those definitions are executed, the ' + 'ordered dictionary\n' + 'is fully populated and the metaclass\'s "__new__()" method ' + 'gets\n' + 'invoked. That method builds the new type and it saves the ' + 'ordered\n' + 'dictionary keys in an attribute called "members".\n' + '\n' + '\n' + 'Customizing instance and subclass checks\n' + '========================================\n' + '\n' + 'The following methods are used to override the default ' + 'behavior of the\n' + '"isinstance()" and "issubclass()" built-in functions.\n' + '\n' + 'In particular, the metaclass "abc.ABCMeta" implements ' + 'these methods in\n' + 'order to allow the addition of Abstract Base Classes ' + '(ABCs) as\n' + '"virtual base classes" to any class or type (including ' + 'built-in\n' + 'types), including other ABCs.\n' + '\n' + 'class.__instancecheck__(self, instance)\n' + '\n' + ' Return true if *instance* should be considered a ' + '(direct or\n' + ' indirect) instance of *class*. If defined, called to ' + 'implement\n' + ' "isinstance(instance, class)".\n' + '\n' + 'class.__subclasscheck__(self, subclass)\n' + '\n' + ' Return true if *subclass* should be considered a ' + '(direct or\n' + ' indirect) subclass of *class*. If defined, called to ' + 'implement\n' + ' "issubclass(subclass, class)".\n' + '\n' + 'Note that these methods are looked up on the type ' + '(metaclass) of a\n' + 'class. They cannot be defined as class methods in the ' + 'actual class.\n' + 'This is consistent with the lookup of special methods that ' + 'are called\n' + 'on instances, only in this case the instance is itself a ' + 'class.\n' + '\n' + 'See also: **PEP 3119** - Introducing Abstract Base ' + 'Classes\n' + '\n' + ' Includes the specification for customizing ' + '"isinstance()" and\n' + ' "issubclass()" behavior through "__instancecheck__()" ' + 'and\n' + ' "__subclasscheck__()", with motivation for this ' + 'functionality in\n' + ' the context of adding Abstract Base Classes (see the ' + '"abc"\n' + ' module) to the language.\n' + '\n' + '\n' + 'Emulating callable objects\n' + '==========================\n' + '\n' + 'object.__call__(self[, args...])\n' + '\n' + ' Called when the instance is "called" as a function; if ' + 'this method\n' + ' is defined, "x(arg1, arg2, ...)" is a shorthand for\n' + ' "x.__call__(arg1, arg2, ...)".\n' + '\n' + '\n' + 'Emulating container types\n' + '=========================\n' + '\n' + 'The following methods can be defined to implement ' + 'container objects.\n' + 'Containers usually are sequences (such as lists or tuples) ' + 'or mappings\n' + '(like dictionaries), but can represent other containers as ' + 'well. The\n' + 'first set of methods is used either to emulate a sequence ' + 'or to\n' + 'emulate a mapping; the difference is that for a sequence, ' + 'the\n' + 'allowable keys should be the integers *k* for which "0 <= ' + 'k < N" where\n' + '*N* is the length of the sequence, or slice objects, which ' + 'define a\n' + 'range of items. It is also recommended that mappings ' + 'provide the\n' + 'methods "keys()", "values()", "items()", "get()", ' + '"clear()",\n' + '"setdefault()", "pop()", "popitem()", "copy()", and ' + '"update()"\n' + "behaving similar to those for Python's standard dictionary " + 'objects.\n' + 'The "collections" module provides a "MutableMapping" ' + 'abstract base\n' + 'class to help create those methods from a base set of ' + '"__getitem__()",\n' + '"__setitem__()", "__delitem__()", and "keys()". Mutable ' + 'sequences\n' + 'should provide methods "append()", "count()", "index()", ' + '"extend()",\n' + '"insert()", "pop()", "remove()", "reverse()" and "sort()", ' + 'like Python\n' + 'standard list objects. Finally, sequence types should ' + 'implement\n' + 'addition (meaning concatenation) and multiplication ' + '(meaning\n' + 'repetition) by defining the methods "__add__()", ' + '"__radd__()",\n' + '"__iadd__()", "__mul__()", "__rmul__()" and "__imul__()" ' + 'described\n' + 'below; they should not define other numerical operators. ' + 'It is\n' + 'recommended that both mappings and sequences implement ' + 'the\n' + '"__contains__()" method to allow efficient use of the "in" ' + 'operator;\n' + 'for mappings, "in" should search the mapping\'s keys; for ' + 'sequences, it\n' + 'should search through the values. It is further ' + 'recommended that both\n' + 'mappings and sequences implement the "__iter__()" method ' + 'to allow\n' + 'efficient iteration through the container; for mappings, ' + '"__iter__()"\n' + 'should be the same as "keys()"; for sequences, it should ' + 'iterate\n' + 'through the values.\n' + '\n' + 'object.__len__(self)\n' + '\n' + ' Called to implement the built-in function "len()". ' + 'Should return\n' + ' the length of the object, an integer ">=" 0. Also, an ' + 'object that\n' + ' doesn\'t define a "__bool__()" method and whose ' + '"__len__()" method\n' + ' returns zero is considered to be false in a Boolean ' + 'context.\n' + '\n' + 'object.__length_hint__(self)\n' + '\n' + ' Called to implement "operator.length_hint()". Should ' + 'return an\n' + ' estimated length for the object (which may be greater ' + 'or less than\n' + ' the actual length). The length must be an integer ">=" ' + '0. This\n' + ' method is purely an optimization and is never required ' + 'for\n' + ' correctness.\n' + '\n' + ' New in version 3.4.\n' + '\n' + 'Note: Slicing is done exclusively with the following three ' + 'methods.\n' + ' A call like\n' + '\n' + ' a[1:2] = b\n' + '\n' + ' is translated to\n' + '\n' + ' a[slice(1, 2, None)] = b\n' + '\n' + ' and so forth. Missing slice items are always filled in ' + 'with "None".\n' + '\n' + 'object.__getitem__(self, key)\n' + '\n' + ' Called to implement evaluation of "self[key]". For ' + 'sequence types,\n' + ' the accepted keys should be integers and slice ' + 'objects. Note that\n' + ' the special interpretation of negative indexes (if the ' + 'class wishes\n' + ' to emulate a sequence type) is up to the ' + '"__getitem__()" method. If\n' + ' *key* is of an inappropriate type, "TypeError" may be ' + 'raised; if of\n' + ' a value outside the set of indexes for the sequence ' + '(after any\n' + ' special interpretation of negative values), ' + '"IndexError" should be\n' + ' raised. For mapping types, if *key* is missing (not in ' + 'the\n' + ' container), "KeyError" should be raised.\n' + '\n' + ' Note: "for" loops expect that an "IndexError" will be ' + 'raised for\n' + ' illegal indexes to allow proper detection of the end ' + 'of the\n' + ' sequence.\n' + '\n' + 'object.__missing__(self, key)\n' + '\n' + ' Called by "dict"."__getitem__()" to implement ' + '"self[key]" for dict\n' + ' subclasses when key is not in the dictionary.\n' + '\n' + 'object.__setitem__(self, key, value)\n' + '\n' + ' Called to implement assignment to "self[key]". Same ' + 'note as for\n' + ' "__getitem__()". This should only be implemented for ' + 'mappings if\n' + ' the objects support changes to the values for keys, or ' + 'if new keys\n' + ' can be added, or for sequences if elements can be ' + 'replaced. The\n' + ' same exceptions should be raised for improper *key* ' + 'values as for\n' + ' the "__getitem__()" method.\n' + '\n' + 'object.__delitem__(self, key)\n' + '\n' + ' Called to implement deletion of "self[key]". Same note ' + 'as for\n' + ' "__getitem__()". This should only be implemented for ' + 'mappings if\n' + ' the objects support removal of keys, or for sequences ' + 'if elements\n' + ' can be removed from the sequence. The same exceptions ' + 'should be\n' + ' raised for improper *key* values as for the ' + '"__getitem__()" method.\n' + '\n' + 'object.__iter__(self)\n' + '\n' + ' This method is called when an iterator is required for ' + 'a container.\n' + ' This method should return a new iterator object that ' + 'can iterate\n' + ' over all the objects in the container. For mappings, ' + 'it should\n' + ' iterate over the keys of the container.\n' + '\n' + ' Iterator objects also need to implement this method; ' + 'they are\n' + ' required to return themselves. For more information on ' + 'iterator\n' + ' objects, see *Iterator Types*.\n' + '\n' + 'object.__reversed__(self)\n' + '\n' + ' Called (if present) by the "reversed()" built-in to ' + 'implement\n' + ' reverse iteration. It should return a new iterator ' + 'object that\n' + ' iterates over all the objects in the container in ' + 'reverse order.\n' + '\n' + ' If the "__reversed__()" method is not provided, the ' + '"reversed()"\n' + ' built-in will fall back to using the sequence protocol ' + '("__len__()"\n' + ' and "__getitem__()"). Objects that support the ' + 'sequence protocol\n' + ' should only provide "__reversed__()" if they can ' + 'provide an\n' + ' implementation that is more efficient than the one ' + 'provided by\n' + ' "reversed()".\n' + '\n' + 'The membership test operators ("in" and "not in") are ' + 'normally\n' + 'implemented as an iteration through a sequence. However, ' + 'container\n' + 'objects can supply the following special method with a ' + 'more efficient\n' + 'implementation, which also does not require the object be ' + 'a sequence.\n' + '\n' + 'object.__contains__(self, item)\n' + '\n' + ' Called to implement membership test operators. Should ' + 'return true\n' + ' if *item* is in *self*, false otherwise. For mapping ' + 'objects, this\n' + ' should consider the keys of the mapping rather than the ' + 'values or\n' + ' the key-item pairs.\n' + '\n' + ' For objects that don\'t define "__contains__()", the ' + 'membership test\n' + ' first tries iteration via "__iter__()", then the old ' + 'sequence\n' + ' iteration protocol via "__getitem__()", see *this ' + 'section in the\n' + ' language reference*.\n' + '\n' + '\n' + 'Emulating numeric types\n' + '=======================\n' + '\n' + 'The following methods can be defined to emulate numeric ' + 'objects.\n' + 'Methods corresponding to operations that are not supported ' + 'by the\n' + 'particular kind of number implemented (e.g., bitwise ' + 'operations for\n' + 'non-integral numbers) should be left undefined.\n' + '\n' + 'object.__add__(self, other)\n' + 'object.__sub__(self, other)\n' + 'object.__mul__(self, other)\n' + 'object.__matmul__(self, other)\n' + 'object.__truediv__(self, other)\n' + 'object.__floordiv__(self, other)\n' + 'object.__mod__(self, other)\n' + 'object.__divmod__(self, other)\n' + 'object.__pow__(self, other[, modulo])\n' + 'object.__lshift__(self, other)\n' + 'object.__rshift__(self, other)\n' + 'object.__and__(self, other)\n' + 'object.__xor__(self, other)\n' + 'object.__or__(self, other)\n' + '\n' + ' These methods are called to implement the binary ' + 'arithmetic\n' + ' operations ("+", "-", "*", "@", "/", "//", "%", ' + '"divmod()",\n' + ' "pow()", "**", "<<", ">>", "&", "^", "|"). For ' + 'instance, to\n' + ' evaluate the expression "x + y", where *x* is an ' + 'instance of a\n' + ' class that has an "__add__()" method, "x.__add__(y)" is ' + 'called.\n' + ' The "__divmod__()" method should be the equivalent to ' + 'using\n' + ' "__floordiv__()" and "__mod__()"; it should not be ' + 'related to\n' + ' "__truediv__()". Note that "__pow__()" should be ' + 'defined to accept\n' + ' an optional third argument if the ternary version of ' + 'the built-in\n' + ' "pow()" function is to be supported.\n' + '\n' + ' If one of those methods does not support the operation ' + 'with the\n' + ' supplied arguments, it should return "NotImplemented".\n' + '\n' + 'object.__radd__(self, other)\n' + 'object.__rsub__(self, other)\n' + 'object.__rmul__(self, other)\n' + 'object.__rmatmul__(self, other)\n' + 'object.__rtruediv__(self, other)\n' + 'object.__rfloordiv__(self, other)\n' + 'object.__rmod__(self, other)\n' + 'object.__rdivmod__(self, other)\n' + 'object.__rpow__(self, other)\n' + 'object.__rlshift__(self, other)\n' + 'object.__rrshift__(self, other)\n' + 'object.__rand__(self, other)\n' + 'object.__rxor__(self, other)\n' + 'object.__ror__(self, other)\n' + '\n' + ' These methods are called to implement the binary ' + 'arithmetic\n' + ' operations ("+", "-", "*", "@", "/", "//", "%", ' + '"divmod()",\n' + ' "pow()", "**", "<<", ">>", "&", "^", "|") with ' + 'reflected (swapped)\n' + ' operands. These functions are only called if the left ' + 'operand does\n' + ' not support the corresponding operation and the ' + 'operands are of\n' + ' different types. [2] For instance, to evaluate the ' + 'expression "x -\n' + ' y", where *y* is an instance of a class that has an ' + '"__rsub__()"\n' + ' method, "y.__rsub__(x)" is called if "x.__sub__(y)" ' + 'returns\n' + ' *NotImplemented*.\n' + '\n' + ' Note that ternary "pow()" will not try calling ' + '"__rpow__()" (the\n' + ' coercion rules would become too complicated).\n' + '\n' + " Note: If the right operand's type is a subclass of the " + 'left\n' + " operand's type and that subclass provides the " + 'reflected method\n' + ' for the operation, this method will be called before ' + 'the left\n' + " operand's non-reflected method. This behavior allows " + 'subclasses\n' + " to override their ancestors' operations.\n" + '\n' + 'object.__iadd__(self, other)\n' + 'object.__isub__(self, other)\n' + 'object.__imul__(self, other)\n' + 'object.__imatmul__(self, other)\n' + 'object.__itruediv__(self, other)\n' + 'object.__ifloordiv__(self, other)\n' + 'object.__imod__(self, other)\n' + 'object.__ipow__(self, other[, modulo])\n' + 'object.__ilshift__(self, other)\n' + 'object.__irshift__(self, other)\n' + 'object.__iand__(self, other)\n' + 'object.__ixor__(self, other)\n' + 'object.__ior__(self, other)\n' + '\n' + ' These methods are called to implement the augmented ' + 'arithmetic\n' + ' assignments ("+=", "-=", "*=", "@=", "/=", "//=", "%=", ' + '"**=",\n' + ' "<<=", ">>=", "&=", "^=", "|="). These methods should ' + 'attempt to\n' + ' do the operation in-place (modifying *self*) and return ' + 'the result\n' + ' (which could be, but does not have to be, *self*). If ' + 'a specific\n' + ' method is not defined, the augmented assignment falls ' + 'back to the\n' + ' normal methods. For instance, if *x* is an instance of ' + 'a class\n' + ' with an "__iadd__()" method, "x += y" is equivalent to ' + '"x =\n' + ' x.__iadd__(y)" . Otherwise, "x.__add__(y)" and ' + '"y.__radd__(x)" are\n' + ' considered, as with the evaluation of "x + y". In ' + 'certain\n' + ' situations, augmented assignment can result in ' + 'unexpected errors\n' + " (see *Why does a_tuple[i] += ['item'] raise an " + 'exception when the\n' + ' addition works?*), but this behavior is in fact part of ' + 'the data\n' + ' model.\n' + '\n' + 'object.__neg__(self)\n' + 'object.__pos__(self)\n' + 'object.__abs__(self)\n' + 'object.__invert__(self)\n' + '\n' + ' Called to implement the unary arithmetic operations ' + '("-", "+",\n' + ' "abs()" and "~").\n' + '\n' + 'object.__complex__(self)\n' + 'object.__int__(self)\n' + 'object.__float__(self)\n' + 'object.__round__(self[, n])\n' + '\n' + ' Called to implement the built-in functions "complex()", ' + '"int()",\n' + ' "float()" and "round()". Should return a value of the ' + 'appropriate\n' + ' type.\n' + '\n' + 'object.__index__(self)\n' + '\n' + ' Called to implement "operator.index()", and whenever ' + 'Python needs\n' + ' to losslessly convert the numeric object to an integer ' + 'object (such\n' + ' as in slicing, or in the built-in "bin()", "hex()" and ' + '"oct()"\n' + ' functions). Presence of this method indicates that the ' + 'numeric\n' + ' object is an integer type. Must return an integer.\n' + '\n' + ' Note: In order to have a coherent integer type class, ' + 'when\n' + ' "__index__()" is defined "__int__()" should also be ' + 'defined, and\n' + ' both should return the same value.\n' + '\n' + '\n' + 'With Statement Context Managers\n' + '===============================\n' + '\n' + 'A *context manager* is an object that defines the runtime ' + 'context to\n' + 'be established when executing a "with" statement. The ' + 'context manager\n' + 'handles the entry into, and the exit from, the desired ' + 'runtime context\n' + 'for the execution of the block of code. Context managers ' + 'are normally\n' + 'invoked using the "with" statement (described in section ' + '*The with\n' + 'statement*), but can also be used by directly invoking ' + 'their methods.\n' + '\n' + 'Typical uses of context managers include saving and ' + 'restoring various\n' + 'kinds of global state, locking and unlocking resources, ' + 'closing opened\n' + 'files, etc.\n' + '\n' + 'For more information on context managers, see *Context ' + 'Manager Types*.\n' + '\n' + 'object.__enter__(self)\n' + '\n' + ' Enter the runtime context related to this object. The ' + '"with"\n' + " statement will bind this method's return value to the " + 'target(s)\n' + ' specified in the "as" clause of the statement, if any.\n' + '\n' + 'object.__exit__(self, exc_type, exc_value, traceback)\n' + '\n' + ' Exit the runtime context related to this object. The ' + 'parameters\n' + ' describe the exception that caused the context to be ' + 'exited. If the\n' + ' context was exited without an exception, all three ' + 'arguments will\n' + ' be "None".\n' + '\n' + ' If an exception is supplied, and the method wishes to ' + 'suppress the\n' + ' exception (i.e., prevent it from being propagated), it ' + 'should\n' + ' return a true value. Otherwise, the exception will be ' + 'processed\n' + ' normally upon exit from this method.\n' + '\n' + ' Note that "__exit__()" methods should not reraise the ' + 'passed-in\n' + " exception; this is the caller's responsibility.\n" + '\n' + 'See also: **PEP 0343** - The "with" statement\n' + '\n' + ' The specification, background, and examples for the ' + 'Python "with"\n' + ' statement.\n' + '\n' + '\n' + 'Special method lookup\n' + '=====================\n' + '\n' + 'For custom classes, implicit invocations of special ' + 'methods are only\n' + "guaranteed to work correctly if defined on an object's " + 'type, not in\n' + "the object's instance dictionary. That behaviour is the " + 'reason why\n' + 'the following code raises an exception:\n' + '\n' + ' >>> class C:\n' + ' ... pass\n' + ' ...\n' + ' >>> c = C()\n' + ' >>> c.__len__ = lambda: 5\n' + ' >>> len(c)\n' + ' Traceback (most recent call last):\n' + ' File "", line 1, in \n' + " TypeError: object of type 'C' has no len()\n" + '\n' + 'The rationale behind this behaviour lies with a number of ' + 'special\n' + 'methods such as "__hash__()" and "__repr__()" that are ' + 'implemented by\n' + 'all objects, including type objects. If the implicit ' + 'lookup of these\n' + 'methods used the conventional lookup process, they would ' + 'fail when\n' + 'invoked on the type object itself:\n' + '\n' + ' >>> 1 .__hash__() == hash(1)\n' + ' True\n' + ' >>> int.__hash__() == hash(int)\n' + ' Traceback (most recent call last):\n' + ' File "", line 1, in \n' + " TypeError: descriptor '__hash__' of 'int' object needs " + 'an argument\n' + '\n' + 'Incorrectly attempting to invoke an unbound method of a ' + 'class in this\n' + "way is sometimes referred to as 'metaclass confusion', and " + 'is avoided\n' + 'by bypassing the instance when looking up special ' + 'methods:\n' + '\n' + ' >>> type(1).__hash__(1) == hash(1)\n' + ' True\n' + ' >>> type(int).__hash__(int) == hash(int)\n' + ' True\n' + '\n' + 'In addition to bypassing any instance attributes in the ' + 'interest of\n' + 'correctness, implicit special method lookup generally also ' + 'bypasses\n' + 'the "__getattribute__()" method even of the object\'s ' + 'metaclass:\n' + '\n' + ' >>> class Meta(type):\n' + ' ... def __getattribute__(*args):\n' + ' ... print("Metaclass getattribute invoked")\n' + ' ... return type.__getattribute__(*args)\n' + ' ...\n' + ' >>> class C(object, metaclass=Meta):\n' + ' ... def __len__(self):\n' + ' ... return 10\n' + ' ... def __getattribute__(*args):\n' + ' ... print("Class getattribute invoked")\n' + ' ... return object.__getattribute__(*args)\n' + ' ...\n' + ' >>> c = C()\n' + ' >>> c.__len__() # Explicit lookup via ' + 'instance\n' + ' Class getattribute invoked\n' + ' 10\n' + ' >>> type(c).__len__(c) # Explicit lookup via ' + 'type\n' + ' Metaclass getattribute invoked\n' + ' 10\n' + ' >>> len(c) # Implicit lookup\n' + ' 10\n' + '\n' + 'Bypassing the "__getattribute__()" machinery in this ' + 'fashion provides\n' + 'significant scope for speed optimisations within the ' + 'interpreter, at\n' + 'the cost of some flexibility in the handling of special ' + 'methods (the\n' + 'special method *must* be set on the class object itself in ' + 'order to be\n' + 'consistently invoked by the interpreter).\n', + 'string-methods': '\n' + 'String Methods\n' + '**************\n' + '\n' + 'Strings implement all of the *common* sequence ' + 'operations, along with\n' + 'the additional methods described below.\n' + '\n' + 'Strings also support two styles of string formatting, ' + 'one providing a\n' + 'large degree of flexibility and customization (see ' + '"str.format()",\n' + '*Format String Syntax* and *String Formatting*) and the ' + 'other based on\n' + 'C "printf" style formatting that handles a narrower ' + 'range of types and\n' + 'is slightly harder to use correctly, but is often faster ' + 'for the cases\n' + 'it can handle (*printf-style String Formatting*).\n' + '\n' + 'The *Text Processing Services* section of the standard ' + 'library covers\n' + 'a number of other modules that provide various text ' + 'related utilities\n' + '(including regular expression support in the "re" ' + 'module).\n' + '\n' + 'str.capitalize()\n' + '\n' + ' Return a copy of the string with its first character ' + 'capitalized\n' + ' and the rest lowercased.\n' + '\n' + 'str.casefold()\n' + '\n' + ' Return a casefolded copy of the string. Casefolded ' + 'strings may be\n' + ' used for caseless matching.\n' + '\n' + ' Casefolding is similar to lowercasing but more ' + 'aggressive because\n' + ' it is intended to remove all case distinctions in a ' + 'string. For\n' + ' example, the German lowercase letter "\'?\'" is ' + 'equivalent to ""ss"".\n' + ' Since it is already lowercase, "lower()" would do ' + 'nothing to "\'?\'";\n' + ' "casefold()" converts it to ""ss"".\n' + '\n' + ' The casefolding algorithm is described in section ' + '3.13 of the\n' + ' Unicode Standard.\n' + '\n' + ' New in version 3.3.\n' + '\n' + 'str.center(width[, fillchar])\n' + '\n' + ' Return centered in a string of length *width*. ' + 'Padding is done\n' + ' using the specified *fillchar* (default is an ASCII ' + 'space). The\n' + ' original string is returned if *width* is less than ' + 'or equal to\n' + ' "len(s)".\n' + '\n' + 'str.count(sub[, start[, end]])\n' + '\n' + ' Return the number of non-overlapping occurrences of ' + 'substring *sub*\n' + ' in the range [*start*, *end*]. Optional arguments ' + '*start* and\n' + ' *end* are interpreted as in slice notation.\n' + '\n' + 'str.encode(encoding="utf-8", errors="strict")\n' + '\n' + ' Return an encoded version of the string as a bytes ' + 'object. Default\n' + ' encoding is "\'utf-8\'". *errors* may be given to set ' + 'a different\n' + ' error handling scheme. The default for *errors* is ' + '"\'strict\'",\n' + ' meaning that encoding errors raise a "UnicodeError". ' + 'Other possible\n' + ' values are "\'ignore\'", "\'replace\'", ' + '"\'xmlcharrefreplace\'",\n' + ' "\'backslashreplace\'" and any other name registered ' + 'via\n' + ' "codecs.register_error()", see section *Error ' + 'Handlers*. For a list\n' + ' of possible encodings, see section *Standard ' + 'Encodings*.\n' + '\n' + ' Changed in version 3.1: Support for keyword arguments ' + 'added.\n' + '\n' + 'str.endswith(suffix[, start[, end]])\n' + '\n' + ' Return "True" if the string ends with the specified ' + '*suffix*,\n' + ' otherwise return "False". *suffix* can also be a ' + 'tuple of suffixes\n' + ' to look for. With optional *start*, test beginning ' + 'at that\n' + ' position. With optional *end*, stop comparing at ' + 'that position.\n' + '\n' + 'str.expandtabs(tabsize=8)\n' + '\n' + ' Return a copy of the string where all tab characters ' + 'are replaced\n' + ' by one or more spaces, depending on the current ' + 'column and the\n' + ' given tab size. Tab positions occur every *tabsize* ' + 'characters\n' + ' (default is 8, giving tab positions at columns 0, 8, ' + '16 and so on).\n' + ' To expand the string, the current column is set to ' + 'zero and the\n' + ' string is examined character by character. If the ' + 'character is a\n' + ' tab ("\\t"), one or more space characters are ' + 'inserted in the result\n' + ' until the current column is equal to the next tab ' + 'position. (The\n' + ' tab character itself is not copied.) If the ' + 'character is a newline\n' + ' ("\\n") or return ("\\r"), it is copied and the ' + 'current column is\n' + ' reset to zero. Any other character is copied ' + 'unchanged and the\n' + ' current column is incremented by one regardless of ' + 'how the\n' + ' character is represented when printed.\n' + '\n' + " >>> '01\\t012\\t0123\\t01234'.expandtabs()\n" + " '01 012 0123 01234'\n" + " >>> '01\\t012\\t0123\\t01234'.expandtabs(4)\n" + " '01 012 0123 01234'\n" + '\n' + 'str.find(sub[, start[, end]])\n' + '\n' + ' Return the lowest index in the string where substring ' + '*sub* is\n' + ' found, such that *sub* is contained in the slice ' + '"s[start:end]".\n' + ' Optional arguments *start* and *end* are interpreted ' + 'as in slice\n' + ' notation. Return "-1" if *sub* is not found.\n' + '\n' + ' Note: The "find()" method should be used only if you ' + 'need to know\n' + ' the position of *sub*. To check if *sub* is a ' + 'substring or not,\n' + ' use the "in" operator:\n' + '\n' + " >>> 'Py' in 'Python'\n" + ' True\n' + '\n' + 'str.format(*args, **kwargs)\n' + '\n' + ' Perform a string formatting operation. The string on ' + 'which this\n' + ' method is called can contain literal text or ' + 'replacement fields\n' + ' delimited by braces "{}". Each replacement field ' + 'contains either\n' + ' the numeric index of a positional argument, or the ' + 'name of a\n' + ' keyword argument. Returns a copy of the string where ' + 'each\n' + ' replacement field is replaced with the string value ' + 'of the\n' + ' corresponding argument.\n' + '\n' + ' >>> "The sum of 1 + 2 is {0}".format(1+2)\n' + " 'The sum of 1 + 2 is 3'\n" + '\n' + ' See *Format String Syntax* for a description of the ' + 'various\n' + ' formatting options that can be specified in format ' + 'strings.\n' + '\n' + 'str.format_map(mapping)\n' + '\n' + ' Similar to "str.format(**mapping)", except that ' + '"mapping" is used\n' + ' directly and not copied to a "dict". This is useful ' + 'if for example\n' + ' "mapping" is a dict subclass:\n' + '\n' + ' >>> class Default(dict):\n' + ' ... def __missing__(self, key):\n' + ' ... return key\n' + ' ...\n' + " >>> '{name} was born in " + "{country}'.format_map(Default(name='Guido'))\n" + " 'Guido was born in country'\n" + '\n' + ' New in version 3.2.\n' + '\n' + 'str.index(sub[, start[, end]])\n' + '\n' + ' Like "find()", but raise "ValueError" when the ' + 'substring is not\n' + ' found.\n' + '\n' + 'str.isalnum()\n' + '\n' + ' Return true if all characters in the string are ' + 'alphanumeric and\n' + ' there is at least one character, false otherwise. A ' + 'character "c"\n' + ' is alphanumeric if one of the following returns ' + '"True":\n' + ' "c.isalpha()", "c.isdecimal()", "c.isdigit()", or ' + '"c.isnumeric()".\n' + '\n' + 'str.isalpha()\n' + '\n' + ' Return true if all characters in the string are ' + 'alphabetic and\n' + ' there is at least one character, false otherwise. ' + 'Alphabetic\n' + ' characters are those characters defined in the ' + 'Unicode character\n' + ' database as "Letter", i.e., those with general ' + 'category property\n' + ' being one of "Lm", "Lt", "Lu", "Ll", or "Lo". Note ' + 'that this is\n' + ' different from the "Alphabetic" property defined in ' + 'the Unicode\n' + ' Standard.\n' + '\n' + 'str.isdecimal()\n' + '\n' + ' Return true if all characters in the string are ' + 'decimal characters\n' + ' and there is at least one character, false otherwise. ' + 'Decimal\n' + ' characters are those from general category "Nd". This ' + 'category\n' + ' includes digit characters, and all characters that ' + 'can be used to\n' + ' form decimal-radix numbers, e.g. U+0660, ARABIC-INDIC ' + 'DIGIT ZERO.\n' + '\n' + 'str.isdigit()\n' + '\n' + ' Return true if all characters in the string are ' + 'digits and there is\n' + ' at least one character, false otherwise. Digits ' + 'include decimal\n' + ' characters and digits that need special handling, ' + 'such as the\n' + ' compatibility superscript digits. Formally, a digit ' + 'is a character\n' + ' that has the property value Numeric_Type=Digit or\n' + ' Numeric_Type=Decimal.\n' + '\n' + 'str.isidentifier()\n' + '\n' + ' Return true if the string is a valid identifier ' + 'according to the\n' + ' language definition, section *Identifiers and ' + 'keywords*.\n' + '\n' + ' Use "keyword.iskeyword()" to test for reserved ' + 'identifiers such as\n' + ' "def" and "class".\n' + '\n' + 'str.islower()\n' + '\n' + ' Return true if all cased characters [4] in the string ' + 'are lowercase\n' + ' and there is at least one cased character, false ' + 'otherwise.\n' + '\n' + 'str.isnumeric()\n' + '\n' + ' Return true if all characters in the string are ' + 'numeric characters,\n' + ' and there is at least one character, false otherwise. ' + 'Numeric\n' + ' characters include digit characters, and all ' + 'characters that have\n' + ' the Unicode numeric value property, e.g. U+2155, ' + 'VULGAR FRACTION\n' + ' ONE FIFTH. Formally, numeric characters are those ' + 'with the\n' + ' property value Numeric_Type=Digit, ' + 'Numeric_Type=Decimal or\n' + ' Numeric_Type=Numeric.\n' + '\n' + 'str.isprintable()\n' + '\n' + ' Return true if all characters in the string are ' + 'printable or the\n' + ' string is empty, false otherwise. Nonprintable ' + 'characters are\n' + ' those characters defined in the Unicode character ' + 'database as\n' + ' "Other" or "Separator", excepting the ASCII space ' + '(0x20) which is\n' + ' considered printable. (Note that printable ' + 'characters in this\n' + ' context are those which should not be escaped when ' + '"repr()" is\n' + ' invoked on a string. It has no bearing on the ' + 'handling of strings\n' + ' written to "sys.stdout" or "sys.stderr".)\n' + '\n' + 'str.isspace()\n' + '\n' + ' Return true if there are only whitespace characters ' + 'in the string\n' + ' and there is at least one character, false ' + 'otherwise. Whitespace\n' + ' characters are those characters defined in the ' + 'Unicode character\n' + ' database as "Other" or "Separator" and those with ' + 'bidirectional\n' + ' property being one of "WS", "B", or "S".\n' + '\n' + 'str.istitle()\n' + '\n' + ' Return true if the string is a titlecased string and ' + 'there is at\n' + ' least one character, for example uppercase characters ' + 'may only\n' + ' follow uncased characters and lowercase characters ' + 'only cased ones.\n' + ' Return false otherwise.\n' + '\n' + 'str.isupper()\n' + '\n' + ' Return true if all cased characters [4] in the string ' + 'are uppercase\n' + ' and there is at least one cased character, false ' + 'otherwise.\n' + '\n' + 'str.join(iterable)\n' + '\n' + ' Return a string which is the concatenation of the ' + 'strings in the\n' + ' *iterable* *iterable*. A "TypeError" will be raised ' + 'if there are\n' + ' any non-string values in *iterable*, including ' + '"bytes" objects.\n' + ' The separator between elements is the string ' + 'providing this method.\n' + '\n' + 'str.ljust(width[, fillchar])\n' + '\n' + ' Return the string left justified in a string of ' + 'length *width*.\n' + ' Padding is done using the specified *fillchar* ' + '(default is an ASCII\n' + ' space). The original string is returned if *width* is ' + 'less than or\n' + ' equal to "len(s)".\n' + '\n' + 'str.lower()\n' + '\n' + ' Return a copy of the string with all the cased ' + 'characters [4]\n' + ' converted to lowercase.\n' + '\n' + ' The lowercasing algorithm used is described in ' + 'section 3.13 of the\n' + ' Unicode Standard.\n' + '\n' + 'str.lstrip([chars])\n' + '\n' + ' Return a copy of the string with leading characters ' + 'removed. The\n' + ' *chars* argument is a string specifying the set of ' + 'characters to be\n' + ' removed. If omitted or "None", the *chars* argument ' + 'defaults to\n' + ' removing whitespace. The *chars* argument is not a ' + 'prefix; rather,\n' + ' all combinations of its values are stripped:\n' + '\n' + " >>> ' spacious '.lstrip()\n" + " 'spacious '\n" + " >>> 'www.example.com'.lstrip('cmowz.')\n" + " 'example.com'\n" + '\n' + 'static str.maketrans(x[, y[, z]])\n' + '\n' + ' This static method returns a translation table usable ' + 'for\n' + ' "str.translate()".\n' + '\n' + ' If there is only one argument, it must be a ' + 'dictionary mapping\n' + ' Unicode ordinals (integers) or characters (strings of ' + 'length 1) to\n' + ' Unicode ordinals, strings (of arbitrary lengths) or ' + 'None.\n' + ' Character keys will then be converted to ordinals.\n' + '\n' + ' If there are two arguments, they must be strings of ' + 'equal length,\n' + ' and in the resulting dictionary, each character in x ' + 'will be mapped\n' + ' to the character at the same position in y. If there ' + 'is a third\n' + ' argument, it must be a string, whose characters will ' + 'be mapped to\n' + ' None in the result.\n' + '\n' + 'str.partition(sep)\n' + '\n' + ' Split the string at the first occurrence of *sep*, ' + 'and return a\n' + ' 3-tuple containing the part before the separator, the ' + 'separator\n' + ' itself, and the part after the separator. If the ' + 'separator is not\n' + ' found, return a 3-tuple containing the string itself, ' + 'followed by\n' + ' two empty strings.\n' + '\n' + 'str.replace(old, new[, count])\n' + '\n' + ' Return a copy of the string with all occurrences of ' + 'substring *old*\n' + ' replaced by *new*. If the optional argument *count* ' + 'is given, only\n' + ' the first *count* occurrences are replaced.\n' + '\n' + 'str.rfind(sub[, start[, end]])\n' + '\n' + ' Return the highest index in the string where ' + 'substring *sub* is\n' + ' found, such that *sub* is contained within ' + '"s[start:end]".\n' + ' Optional arguments *start* and *end* are interpreted ' + 'as in slice\n' + ' notation. Return "-1" on failure.\n' + '\n' + 'str.rindex(sub[, start[, end]])\n' + '\n' + ' Like "rfind()" but raises "ValueError" when the ' + 'substring *sub* is\n' + ' not found.\n' + '\n' + 'str.rjust(width[, fillchar])\n' + '\n' + ' Return the string right justified in a string of ' + 'length *width*.\n' + ' Padding is done using the specified *fillchar* ' + '(default is an ASCII\n' + ' space). The original string is returned if *width* is ' + 'less than or\n' + ' equal to "len(s)".\n' + '\n' + 'str.rpartition(sep)\n' + '\n' + ' Split the string at the last occurrence of *sep*, and ' + 'return a\n' + ' 3-tuple containing the part before the separator, the ' + 'separator\n' + ' itself, and the part after the separator. If the ' + 'separator is not\n' + ' found, return a 3-tuple containing two empty strings, ' + 'followed by\n' + ' the string itself.\n' + '\n' + 'str.rsplit(sep=None, maxsplit=-1)\n' + '\n' + ' Return a list of the words in the string, using *sep* ' + 'as the\n' + ' delimiter string. If *maxsplit* is given, at most ' + '*maxsplit* splits\n' + ' are done, the *rightmost* ones. If *sep* is not ' + 'specified or\n' + ' "None", any whitespace string is a separator. Except ' + 'for splitting\n' + ' from the right, "rsplit()" behaves like "split()" ' + 'which is\n' + ' described in detail below.\n' + '\n' + 'str.rstrip([chars])\n' + '\n' + ' Return a copy of the string with trailing characters ' + 'removed. The\n' + ' *chars* argument is a string specifying the set of ' + 'characters to be\n' + ' removed. If omitted or "None", the *chars* argument ' + 'defaults to\n' + ' removing whitespace. The *chars* argument is not a ' + 'suffix; rather,\n' + ' all combinations of its values are stripped:\n' + '\n' + " >>> ' spacious '.rstrip()\n" + " ' spacious'\n" + " >>> 'mississippi'.rstrip('ipz')\n" + " 'mississ'\n" + '\n' + 'str.split(sep=None, maxsplit=-1)\n' + '\n' + ' Return a list of the words in the string, using *sep* ' + 'as the\n' + ' delimiter string. If *maxsplit* is given, at most ' + '*maxsplit*\n' + ' splits are done (thus, the list will have at most ' + '"maxsplit+1"\n' + ' elements). If *maxsplit* is not specified or "-1", ' + 'then there is\n' + ' no limit on the number of splits (all possible splits ' + 'are made).\n' + '\n' + ' If *sep* is given, consecutive delimiters are not ' + 'grouped together\n' + ' and are deemed to delimit empty strings (for ' + 'example,\n' + ' "\'1,,2\'.split(\',\')" returns "[\'1\', \'\', ' + '\'2\']"). The *sep* argument\n' + ' may consist of multiple characters (for example,\n' + ' "\'1<>2<>3\'.split(\'<>\')" returns "[\'1\', \'2\', ' + '\'3\']"). Splitting an\n' + ' empty string with a specified separator returns ' + '"[\'\']".\n' + '\n' + ' For example:\n' + '\n' + " >>> '1,2,3'.split(',')\n" + " ['1', '2', '3']\n" + " >>> '1,2,3'.split(',', maxsplit=1)\n" + " ['1', '2,3']\n" + " >>> '1,2,,3,'.split(',')\n" + " ['1', '2', '', '3', '']\n" + '\n' + ' If *sep* is not specified or is "None", a different ' + 'splitting\n' + ' algorithm is applied: runs of consecutive whitespace ' + 'are regarded\n' + ' as a single separator, and the result will contain no ' + 'empty strings\n' + ' at the start or end if the string has leading or ' + 'trailing\n' + ' whitespace. Consequently, splitting an empty string ' + 'or a string\n' + ' consisting of just whitespace with a "None" separator ' + 'returns "[]".\n' + '\n' + ' For example:\n' + '\n' + " >>> '1 2 3'.split()\n" + " ['1', '2', '3']\n" + " >>> '1 2 3'.split(maxsplit=1)\n" + " ['1', '2 3']\n" + " >>> ' 1 2 3 '.split()\n" + " ['1', '2', '3']\n" + '\n' + 'str.splitlines([keepends])\n' + '\n' + ' Return a list of the lines in the string, breaking at ' + 'line\n' + ' boundaries. Line breaks are not included in the ' + 'resulting list\n' + ' unless *keepends* is given and true.\n' + '\n' + ' This method splits on the following line boundaries. ' + 'In\n' + ' particular, the boundaries are a superset of ' + '*universal newlines*.\n' + '\n' + ' ' + '+-------------------------+-------------------------------+\n' + ' | Representation | ' + 'Description |\n' + ' ' + '+=========================+===============================+\n' + ' | "\\n" | Line ' + 'Feed |\n' + ' ' + '+-------------------------+-------------------------------+\n' + ' | "\\r" | Carriage ' + 'Return |\n' + ' ' + '+-------------------------+-------------------------------+\n' + ' | "\\r\\n" | Carriage Return + Line ' + 'Feed |\n' + ' ' + '+-------------------------+-------------------------------+\n' + ' | "\\v" or "\\x0b" | Line ' + 'Tabulation |\n' + ' ' + '+-------------------------+-------------------------------+\n' + ' | "\\f" or "\\x0c" | Form ' + 'Feed |\n' + ' ' + '+-------------------------+-------------------------------+\n' + ' | "\\x1c" | File ' + 'Separator |\n' + ' ' + '+-------------------------+-------------------------------+\n' + ' | "\\x1d" | Group ' + 'Separator |\n' + ' ' + '+-------------------------+-------------------------------+\n' + ' | "\\x1e" | Record ' + 'Separator |\n' + ' ' + '+-------------------------+-------------------------------+\n' + ' | "\\x85" | Next Line (C1 Control ' + 'Code) |\n' + ' ' + '+-------------------------+-------------------------------+\n' + ' | "\\u2028" | Line ' + 'Separator |\n' + ' ' + '+-------------------------+-------------------------------+\n' + ' | "\\u2029" | Paragraph ' + 'Separator |\n' + ' ' + '+-------------------------+-------------------------------+\n' + '\n' + ' Changed in version 3.2: "\\v" and "\\f" added to list ' + 'of line\n' + ' boundaries.\n' + '\n' + ' For example:\n' + '\n' + " >>> 'ab c\\n\\nde fg\\rkl\\r\\n'.splitlines()\n" + " ['ab c', '', 'de fg', 'kl']\n" + " >>> 'ab c\\n\\nde " + "fg\\rkl\\r\\n'.splitlines(keepends=True)\n" + " ['ab c\\n', '\\n', 'de fg\\r', 'kl\\r\\n']\n" + '\n' + ' Unlike "split()" when a delimiter string *sep* is ' + 'given, this\n' + ' method returns an empty list for the empty string, ' + 'and a terminal\n' + ' line break does not result in an extra line:\n' + '\n' + ' >>> "".splitlines()\n' + ' []\n' + ' >>> "One line\\n".splitlines()\n' + " ['One line']\n" + '\n' + ' For comparison, "split(\'\\n\')" gives:\n' + '\n' + " >>> ''.split('\\n')\n" + " ['']\n" + " >>> 'Two lines\\n'.split('\\n')\n" + " ['Two lines', '']\n" + '\n' + 'str.startswith(prefix[, start[, end]])\n' + '\n' + ' Return "True" if string starts with the *prefix*, ' + 'otherwise return\n' + ' "False". *prefix* can also be a tuple of prefixes to ' + 'look for.\n' + ' With optional *start*, test string beginning at that ' + 'position.\n' + ' With optional *end*, stop comparing string at that ' + 'position.\n' + '\n' + 'str.strip([chars])\n' + '\n' + ' Return a copy of the string with the leading and ' + 'trailing\n' + ' characters removed. The *chars* argument is a string ' + 'specifying the\n' + ' set of characters to be removed. If omitted or ' + '"None", the *chars*\n' + ' argument defaults to removing whitespace. The *chars* ' + 'argument is\n' + ' not a prefix or suffix; rather, all combinations of ' + 'its values are\n' + ' stripped:\n' + '\n' + " >>> ' spacious '.strip()\n" + " 'spacious'\n" + " >>> 'www.example.com'.strip('cmowz.')\n" + " 'example'\n" + '\n' + ' The outermost leading and trailing *chars* argument ' + 'values are\n' + ' stripped from the string. Characters are removed from ' + 'the leading\n' + ' end until reaching a string character that is not ' + 'contained in the\n' + ' set of characters in *chars*. A similar action takes ' + 'place on the\n' + ' trailing end. For example:\n' + '\n' + " >>> comment_string = '#....... Section 3.2.1 Issue " + "#32 .......'\n" + " >>> comment_string.strip('.#! ')\n" + " 'Section 3.2.1 Issue #32'\n" + '\n' + 'str.swapcase()\n' + '\n' + ' Return a copy of the string with uppercase characters ' + 'converted to\n' + ' lowercase and vice versa. Note that it is not ' + 'necessarily true that\n' + ' "s.swapcase().swapcase() == s".\n' + '\n' + 'str.title()\n' + '\n' + ' Return a titlecased version of the string where words ' + 'start with an\n' + ' uppercase character and the remaining characters are ' + 'lowercase.\n' + '\n' + ' For example:\n' + '\n' + " >>> 'Hello world'.title()\n" + " 'Hello World'\n" + '\n' + ' The algorithm uses a simple language-independent ' + 'definition of a\n' + ' word as groups of consecutive letters. The ' + 'definition works in\n' + ' many contexts but it means that apostrophes in ' + 'contractions and\n' + ' possessives form word boundaries, which may not be ' + 'the desired\n' + ' result:\n' + '\n' + ' >>> "they\'re bill\'s friends from the ' + 'UK".title()\n' + ' "They\'Re Bill\'S Friends From The Uk"\n' + '\n' + ' A workaround for apostrophes can be constructed using ' + 'regular\n' + ' expressions:\n' + '\n' + ' >>> import re\n' + ' >>> def titlecase(s):\n' + ' ... return re.sub(r"[A-Za-z]+(\'[A-Za-z]+)?",\n' + ' ... lambda mo: ' + 'mo.group(0)[0].upper() +\n' + ' ... ' + 'mo.group(0)[1:].lower(),\n' + ' ... s)\n' + ' ...\n' + ' >>> titlecase("they\'re bill\'s friends.")\n' + ' "They\'re Bill\'s Friends."\n' + '\n' + 'str.translate(table)\n' + '\n' + ' Return a copy of the string in which each character ' + 'has been mapped\n' + ' through the given translation table. The table must ' + 'be an object\n' + ' that implements indexing via "__getitem__()", ' + 'typically a *mapping*\n' + ' or *sequence*. When indexed by a Unicode ordinal (an ' + 'integer), the\n' + ' table object can do any of the following: return a ' + 'Unicode ordinal\n' + ' or a string, to map the character to one or more ' + 'other characters;\n' + ' return "None", to delete the character from the ' + 'return string; or\n' + ' raise a "LookupError" exception, to map the character ' + 'to itself.\n' + '\n' + ' You can use "str.maketrans()" to create a translation ' + 'map from\n' + ' character-to-character mappings in different ' + 'formats.\n' + '\n' + ' See also the "codecs" module for a more flexible ' + 'approach to custom\n' + ' character mappings.\n' + '\n' + 'str.upper()\n' + '\n' + ' Return a copy of the string with all the cased ' + 'characters [4]\n' + ' converted to uppercase. Note that ' + '"str.upper().isupper()" might be\n' + ' "False" if "s" contains uncased characters or if the ' + 'Unicode\n' + ' category of the resulting character(s) is not "Lu" ' + '(Letter,\n' + ' uppercase), but e.g. "Lt" (Letter, titlecase).\n' + '\n' + ' The uppercasing algorithm used is described in ' + 'section 3.13 of the\n' + ' Unicode Standard.\n' + '\n' + 'str.zfill(width)\n' + '\n' + ' Return a copy of the string left filled with ASCII ' + '"\'0\'" digits to\n' + ' make a string of length *width*. A leading sign ' + 'prefix\n' + ' ("\'+\'"/"\'-\'") is handled by inserting the padding ' + '*after* the sign\n' + ' character rather than before. The original string is ' + 'returned if\n' + ' *width* is less than or equal to "len(s)".\n' + '\n' + ' For example:\n' + '\n' + ' >>> "42".zfill(5)\n' + " '00042'\n" + ' >>> "-42".zfill(5)\n' + " '-0042'\n", + 'strings': '\n' + 'String and Bytes literals\n' + '*************************\n' + '\n' + 'String literals are described by the following lexical ' + 'definitions:\n' + '\n' + ' stringliteral ::= [stringprefix](shortstring | ' + 'longstring)\n' + ' stringprefix ::= "r" | "u" | "R" | "U"\n' + ' shortstring ::= "\'" shortstringitem* "\'" | \'"\' ' + 'shortstringitem* \'"\'\n' + ' longstring ::= "\'\'\'" longstringitem* "\'\'\'" | ' + '\'"""\' longstringitem* \'"""\'\n' + ' shortstringitem ::= shortstringchar | stringescapeseq\n' + ' longstringitem ::= longstringchar | stringescapeseq\n' + ' shortstringchar ::= \n' + ' longstringchar ::= \n' + ' stringescapeseq ::= "\\" \n' + '\n' + ' bytesliteral ::= bytesprefix(shortbytes | longbytes)\n' + ' bytesprefix ::= "b" | "B" | "br" | "Br" | "bR" | "BR" | ' + '"rb" | "rB" | "Rb" | "RB"\n' + ' shortbytes ::= "\'" shortbytesitem* "\'" | \'"\' ' + 'shortbytesitem* \'"\'\n' + ' longbytes ::= "\'\'\'" longbytesitem* "\'\'\'" | ' + '\'"""\' longbytesitem* \'"""\'\n' + ' shortbytesitem ::= shortbyteschar | bytesescapeseq\n' + ' longbytesitem ::= longbyteschar | bytesescapeseq\n' + ' shortbyteschar ::= \n' + ' longbyteschar ::= \n' + ' bytesescapeseq ::= "\\" \n' + '\n' + 'One syntactic restriction not indicated by these productions is ' + 'that\n' + 'whitespace is not allowed between the "stringprefix" or ' + '"bytesprefix"\n' + 'and the rest of the literal. The source character set is ' + 'defined by\n' + 'the encoding declaration; it is UTF-8 if no encoding ' + 'declaration is\n' + 'given in the source file; see section *Encoding declarations*.\n' + '\n' + 'In plain English: Both types of literals can be enclosed in ' + 'matching\n' + 'single quotes ("\'") or double quotes ("""). They can also be ' + 'enclosed\n' + 'in matching groups of three single or double quotes (these are\n' + 'generally referred to as *triple-quoted strings*). The ' + 'backslash\n' + '("\\") character is used to escape characters that otherwise ' + 'have a\n' + 'special meaning, such as newline, backslash itself, or the ' + 'quote\n' + 'character.\n' + '\n' + 'Bytes literals are always prefixed with "\'b\'" or "\'B\'"; ' + 'they produce\n' + 'an instance of the "bytes" type instead of the "str" type. ' + 'They may\n' + 'only contain ASCII characters; bytes with a numeric value of ' + '128 or\n' + 'greater must be expressed with escapes.\n' + '\n' + 'As of Python 3.3 it is possible again to prefix string literals ' + 'with a\n' + '"u" prefix to simplify maintenance of dual 2.x and 3.x ' + 'codebases.\n' + '\n' + 'Both string and bytes literals may optionally be prefixed with ' + 'a\n' + 'letter "\'r\'" or "\'R\'"; such strings are called *raw ' + 'strings* and treat\n' + 'backslashes as literal characters. As a result, in string ' + 'literals,\n' + '"\'\\U\'" and "\'\\u\'" escapes in raw strings are not treated ' + 'specially.\n' + "Given that Python 2.x's raw unicode literals behave differently " + 'than\n' + 'Python 3.x\'s the "\'ur\'" syntax is not supported.\n' + '\n' + 'New in version 3.3: The "\'rb\'" prefix of raw bytes literals ' + 'has been\n' + 'added as a synonym of "\'br\'".\n' + '\n' + 'New in version 3.3: Support for the unicode legacy literal\n' + '("u\'value\'") was reintroduced to simplify the maintenance of ' + 'dual\n' + 'Python 2.x and 3.x codebases. See **PEP 414** for more ' + 'information.\n' + '\n' + 'In triple-quoted literals, unescaped newlines and quotes are ' + 'allowed\n' + '(and are retained), except that three unescaped quotes in a ' + 'row\n' + 'terminate the literal. (A "quote" is the character used to ' + 'open the\n' + 'literal, i.e. either "\'" or """.)\n' + '\n' + 'Unless an "\'r\'" or "\'R\'" prefix is present, escape ' + 'sequences in string\n' + 'and bytes literals are interpreted according to rules similar ' + 'to those\n' + 'used by Standard C. The recognized escape sequences are:\n' + '\n' + '+-------------------+-----------------------------------+---------+\n' + '| Escape Sequence | Meaning | ' + 'Notes |\n' + '+===================+===================================+=========+\n' + '| "\\newline" | Backslash and newline ignored ' + '| |\n' + '+-------------------+-----------------------------------+---------+\n' + '| "\\\\" | Backslash ("\\") ' + '| |\n' + '+-------------------+-----------------------------------+---------+\n' + '| "\\\'" | Single quote ("\'") ' + '| |\n' + '+-------------------+-----------------------------------+---------+\n' + '| "\\"" | Double quote (""") ' + '| |\n' + '+-------------------+-----------------------------------+---------+\n' + '| "\\a" | ASCII Bell (BEL) ' + '| |\n' + '+-------------------+-----------------------------------+---------+\n' + '| "\\b" | ASCII Backspace (BS) ' + '| |\n' + '+-------------------+-----------------------------------+---------+\n' + '| "\\f" | ASCII Formfeed (FF) ' + '| |\n' + '+-------------------+-----------------------------------+---------+\n' + '| "\\n" | ASCII Linefeed (LF) ' + '| |\n' + '+-------------------+-----------------------------------+---------+\n' + '| "\\r" | ASCII Carriage Return (CR) ' + '| |\n' + '+-------------------+-----------------------------------+---------+\n' + '| "\\t" | ASCII Horizontal Tab (TAB) ' + '| |\n' + '+-------------------+-----------------------------------+---------+\n' + '| "\\v" | ASCII Vertical Tab (VT) ' + '| |\n' + '+-------------------+-----------------------------------+---------+\n' + '| "\\ooo" | Character with octal value *ooo* | ' + '(1,3) |\n' + '+-------------------+-----------------------------------+---------+\n' + '| "\\xhh" | Character with hex value *hh* | ' + '(2,3) |\n' + '+-------------------+-----------------------------------+---------+\n' + '\n' + 'Escape sequences only recognized in string literals are:\n' + '\n' + '+-------------------+-----------------------------------+---------+\n' + '| Escape Sequence | Meaning | ' + 'Notes |\n' + '+===================+===================================+=========+\n' + '| "\\N{name}" | Character named *name* in the | ' + '(4) |\n' + '| | Unicode database ' + '| |\n' + '+-------------------+-----------------------------------+---------+\n' + '| "\\uxxxx" | Character with 16-bit hex value | ' + '(5) |\n' + '| | *xxxx* ' + '| |\n' + '+-------------------+-----------------------------------+---------+\n' + '| "\\Uxxxxxxxx" | Character with 32-bit hex value | ' + '(6) |\n' + '| | *xxxxxxxx* ' + '| |\n' + '+-------------------+-----------------------------------+---------+\n' + '\n' + 'Notes:\n' + '\n' + '1. As in Standard C, up to three octal digits are accepted.\n' + '\n' + '2. Unlike in Standard C, exactly two hex digits are required.\n' + '\n' + '3. In a bytes literal, hexadecimal and octal escapes denote ' + 'the\n' + ' byte with the given value. In a string literal, these ' + 'escapes\n' + ' denote a Unicode character with the given value.\n' + '\n' + '4. Changed in version 3.3: Support for name aliases [1] has ' + 'been\n' + ' added.\n' + '\n' + '5. Individual code units which form parts of a surrogate pair ' + 'can\n' + ' be encoded using this escape sequence. Exactly four hex ' + 'digits are\n' + ' required.\n' + '\n' + '6. Any Unicode character can be encoded this way. Exactly ' + 'eight\n' + ' hex digits are required.\n' + '\n' + 'Unlike Standard C, all unrecognized escape sequences are left ' + 'in the\n' + 'string unchanged, i.e., *the backslash is left in the result*. ' + '(This\n' + 'behavior is useful when debugging: if an escape sequence is ' + 'mistyped,\n' + 'the resulting output is more easily recognized as broken.) It ' + 'is also\n' + 'important to note that the escape sequences only recognized in ' + 'string\n' + 'literals fall into the category of unrecognized escapes for ' + 'bytes\n' + 'literals.\n' + '\n' + 'Even in a raw literal, quotes can be escaped with a backslash, ' + 'but the\n' + 'backslash remains in the result; for example, "r"\\""" is a ' + 'valid\n' + 'string literal consisting of two characters: a backslash and a ' + 'double\n' + 'quote; "r"\\"" is not a valid string literal (even a raw string ' + 'cannot\n' + 'end in an odd number of backslashes). Specifically, *a raw ' + 'literal\n' + 'cannot end in a single backslash* (since the backslash would ' + 'escape\n' + 'the following quote character). Note also that a single ' + 'backslash\n' + 'followed by a newline is interpreted as those two characters as ' + 'part\n' + 'of the literal, *not* as a line continuation.\n', + 'subscriptions': '\n' + 'Subscriptions\n' + '*************\n' + '\n' + 'A subscription selects an item of a sequence (string, ' + 'tuple or list)\n' + 'or mapping (dictionary) object:\n' + '\n' + ' subscription ::= primary "[" expression_list "]"\n' + '\n' + 'The primary must evaluate to an object that supports ' + 'subscription\n' + '(lists or dictionaries for example). User-defined ' + 'objects can support\n' + 'subscription by defining a "__getitem__()" method.\n' + '\n' + 'For built-in objects, there are two types of objects that ' + 'support\n' + 'subscription:\n' + '\n' + 'If the primary is a mapping, the expression list must ' + 'evaluate to an\n' + 'object whose value is one of the keys of the mapping, and ' + 'the\n' + 'subscription selects the value in the mapping that ' + 'corresponds to that\n' + 'key. (The expression list is a tuple except if it has ' + 'exactly one\n' + 'item.)\n' + '\n' + 'If the primary is a sequence, the expression (list) must ' + 'evaluate to\n' + 'an integer or a slice (as discussed in the following ' + 'section).\n' + '\n' + 'The formal syntax makes no special provision for negative ' + 'indices in\n' + 'sequences; however, built-in sequences all provide a ' + '"__getitem__()"\n' + 'method that interprets negative indices by adding the ' + 'length of the\n' + 'sequence to the index (so that "x[-1]" selects the last ' + 'item of "x").\n' + 'The resulting value must be a nonnegative integer less ' + 'than the number\n' + 'of items in the sequence, and the subscription selects ' + 'the item whose\n' + 'index is that value (counting from zero). Since the ' + 'support for\n' + "negative indices and slicing occurs in the object's " + '"__getitem__()"\n' + 'method, subclasses overriding this method will need to ' + 'explicitly add\n' + 'that support.\n' + '\n' + "A string's items are characters. A character is not a " + 'separate data\n' + 'type but a string of exactly one character.\n', + 'truth': '\n' + 'Truth Value Testing\n' + '*******************\n' + '\n' + 'Any object can be tested for truth value, for use in an "if" or\n' + '"while" condition or as operand of the Boolean operations below. ' + 'The\n' + 'following values are considered false:\n' + '\n' + '* "None"\n' + '\n' + '* "False"\n' + '\n' + '* zero of any numeric type, for example, "0", "0.0", "0j".\n' + '\n' + '* any empty sequence, for example, "\'\'", "()", "[]".\n' + '\n' + '* any empty mapping, for example, "{}".\n' + '\n' + '* instances of user-defined classes, if the class defines a\n' + ' "__bool__()" or "__len__()" method, when that method returns ' + 'the\n' + ' integer zero or "bool" value "False". [1]\n' + '\n' + 'All other values are considered true --- so objects of many types ' + 'are\n' + 'always true.\n' + '\n' + 'Operations and built-in functions that have a Boolean result ' + 'always\n' + 'return "0" or "False" for false and "1" or "True" for true, ' + 'unless\n' + 'otherwise stated. (Important exception: the Boolean operations ' + '"or"\n' + 'and "and" always return one of their operands.)\n', + 'try': '\n' + 'The "try" statement\n' + '*******************\n' + '\n' + 'The "try" statement specifies exception handlers and/or cleanup ' + 'code\n' + 'for a group of statements:\n' + '\n' + ' try_stmt ::= try1_stmt | try2_stmt\n' + ' try1_stmt ::= "try" ":" suite\n' + ' ("except" [expression ["as" identifier]] ":" ' + 'suite)+\n' + ' ["else" ":" suite]\n' + ' ["finally" ":" suite]\n' + ' try2_stmt ::= "try" ":" suite\n' + ' "finally" ":" suite\n' + '\n' + 'The "except" clause(s) specify one or more exception handlers. When ' + 'no\n' + 'exception occurs in the "try" clause, no exception handler is\n' + 'executed. When an exception occurs in the "try" suite, a search for ' + 'an\n' + 'exception handler is started. This search inspects the except ' + 'clauses\n' + 'in turn until one is found that matches the exception. An ' + 'expression-\n' + 'less except clause, if present, must be last; it matches any\n' + 'exception. For an except clause with an expression, that ' + 'expression\n' + 'is evaluated, and the clause matches the exception if the ' + 'resulting\n' + 'object is "compatible" with the exception. An object is ' + 'compatible\n' + 'with an exception if it is the class or a base class of the ' + 'exception\n' + 'object or a tuple containing an item compatible with the ' + 'exception.\n' + '\n' + 'If no except clause matches the exception, the search for an ' + 'exception\n' + 'handler continues in the surrounding code and on the invocation ' + 'stack.\n' + '[1]\n' + '\n' + 'If the evaluation of an expression in the header of an except ' + 'clause\n' + 'raises an exception, the original search for a handler is canceled ' + 'and\n' + 'a search starts for the new exception in the surrounding code and ' + 'on\n' + 'the call stack (it is treated as if the entire "try" statement ' + 'raised\n' + 'the exception).\n' + '\n' + 'When a matching except clause is found, the exception is assigned ' + 'to\n' + 'the target specified after the "as" keyword in that except clause, ' + 'if\n' + "present, and the except clause's suite is executed. All except\n" + 'clauses must have an executable block. When the end of this block ' + 'is\n' + 'reached, execution continues normally after the entire try ' + 'statement.\n' + '(This means that if two nested handlers exist for the same ' + 'exception,\n' + 'and the exception occurs in the try clause of the inner handler, ' + 'the\n' + 'outer handler will not handle the exception.)\n' + '\n' + 'When an exception has been assigned using "as target", it is ' + 'cleared\n' + 'at the end of the except clause. This is as if\n' + '\n' + ' except E as N:\n' + ' foo\n' + '\n' + 'was translated to\n' + '\n' + ' except E as N:\n' + ' try:\n' + ' foo\n' + ' finally:\n' + ' del N\n' + '\n' + 'This means the exception must be assigned to a different name to ' + 'be\n' + 'able to refer to it after the except clause. Exceptions are ' + 'cleared\n' + 'because with the traceback attached to them, they form a reference\n' + 'cycle with the stack frame, keeping all locals in that frame alive\n' + 'until the next garbage collection occurs.\n' + '\n' + "Before an except clause's suite is executed, details about the\n" + 'exception are stored in the "sys" module and can be accessed via\n' + '"sys.exc_info()". "sys.exc_info()" returns a 3-tuple consisting of ' + 'the\n' + 'exception class, the exception instance and a traceback object ' + '(see\n' + 'section *The standard type hierarchy*) identifying the point in ' + 'the\n' + 'program where the exception occurred. "sys.exc_info()" values are\n' + 'restored to their previous values (before the call) when returning\n' + 'from a function that handled an exception.\n' + '\n' + 'The optional "else" clause is executed if and when control flows ' + 'off\n' + 'the end of the "try" clause. [2] Exceptions in the "else" clause ' + 'are\n' + 'not handled by the preceding "except" clauses.\n' + '\n' + 'If "finally" is present, it specifies a \'cleanup\' handler. The ' + '"try"\n' + 'clause is executed, including any "except" and "else" clauses. If ' + 'an\n' + 'exception occurs in any of the clauses and is not handled, the\n' + 'exception is temporarily saved. The "finally" clause is executed. ' + 'If\n' + 'there is a saved exception it is re-raised at the end of the ' + '"finally"\n' + 'clause. If the "finally" clause raises another exception, the ' + 'saved\n' + 'exception is set as the context of the new exception. If the ' + '"finally"\n' + 'clause executes a "return" or "break" statement, the saved ' + 'exception\n' + 'is discarded:\n' + '\n' + ' >>> def f():\n' + ' ... try:\n' + ' ... 1/0\n' + ' ... finally:\n' + ' ... return 42\n' + ' ...\n' + ' >>> f()\n' + ' 42\n' + '\n' + 'The exception information is not available to the program during\n' + 'execution of the "finally" clause.\n' + '\n' + 'When a "return", "break" or "continue" statement is executed in ' + 'the\n' + '"try" suite of a "try"..."finally" statement, the "finally" clause ' + 'is\n' + 'also executed \'on the way out.\' A "continue" statement is illegal ' + 'in\n' + 'the "finally" clause. (The reason is a problem with the current\n' + 'implementation --- this restriction may be lifted in the future).\n' + '\n' + 'The return value of a function is determined by the last "return"\n' + 'statement executed. Since the "finally" clause always executes, a\n' + '"return" statement executed in the "finally" clause will always be ' + 'the\n' + 'last one executed:\n' + '\n' + ' >>> def foo():\n' + ' ... try:\n' + " ... return 'try'\n" + ' ... finally:\n' + " ... return 'finally'\n" + ' ...\n' + ' >>> foo()\n' + " 'finally'\n" + '\n' + 'Additional information on exceptions can be found in section\n' + '*Exceptions*, and information on using the "raise" statement to\n' + 'generate exceptions may be found in section *The raise statement*.\n', + 'types': '\n' + 'The standard type hierarchy\n' + '***************************\n' + '\n' + 'Below is a list of the types that are built into Python. ' + 'Extension\n' + 'modules (written in C, Java, or other languages, depending on ' + 'the\n' + 'implementation) can define additional types. Future versions of\n' + 'Python may add types to the type hierarchy (e.g., rational ' + 'numbers,\n' + 'efficiently stored arrays of integers, etc.), although such ' + 'additions\n' + 'will often be provided via the standard library instead.\n' + '\n' + 'Some of the type descriptions below contain a paragraph listing\n' + "'special attributes.' These are attributes that provide access " + 'to the\n' + 'implementation and are not intended for general use. Their ' + 'definition\n' + 'may change in the future.\n' + '\n' + 'None\n' + ' This type has a single value. There is a single object with ' + 'this\n' + ' value. This object is accessed through the built-in name ' + '"None". It\n' + ' is used to signify the absence of a value in many situations, ' + 'e.g.,\n' + " it is returned from functions that don't explicitly return\n" + ' anything. Its truth value is false.\n' + '\n' + 'NotImplemented\n' + ' This type has a single value. There is a single object with ' + 'this\n' + ' value. This object is accessed through the built-in name\n' + ' "NotImplemented". Numeric methods and rich comparison methods\n' + ' should return this value if they do not implement the ' + 'operation for\n' + ' the operands provided. (The interpreter will then try the\n' + ' reflected operation, or some other fallback, depending on the\n' + ' operator.) Its truth value is true.\n' + '\n' + ' See *Implementing the arithmetic operations* for more ' + 'details.\n' + '\n' + 'Ellipsis\n' + ' This type has a single value. There is a single object with ' + 'this\n' + ' value. This object is accessed through the literal "..." or ' + 'the\n' + ' built-in name "Ellipsis". Its truth value is true.\n' + '\n' + '"numbers.Number"\n' + ' These are created by numeric literals and returned as results ' + 'by\n' + ' arithmetic operators and arithmetic built-in functions. ' + 'Numeric\n' + ' objects are immutable; once created their value never ' + 'changes.\n' + ' Python numbers are of course strongly related to mathematical\n' + ' numbers, but subject to the limitations of numerical ' + 'representation\n' + ' in computers.\n' + '\n' + ' Python distinguishes between integers, floating point numbers, ' + 'and\n' + ' complex numbers:\n' + '\n' + ' "numbers.Integral"\n' + ' These represent elements from the mathematical set of ' + 'integers\n' + ' (positive and negative).\n' + '\n' + ' There are two types of integers:\n' + '\n' + ' Integers ("int")\n' + '\n' + ' These represent numbers in an unlimited range, subject ' + 'to\n' + ' available (virtual) memory only. For the purpose of ' + 'shift\n' + ' and mask operations, a binary representation is assumed, ' + 'and\n' + " negative numbers are represented in a variant of 2's\n" + ' complement which gives the illusion of an infinite ' + 'string of\n' + ' sign bits extending to the left.\n' + '\n' + ' Booleans ("bool")\n' + ' These represent the truth values False and True. The ' + 'two\n' + ' objects representing the values "False" and "True" are ' + 'the\n' + ' only Boolean objects. The Boolean type is a subtype of ' + 'the\n' + ' integer type, and Boolean values behave like the values ' + '0 and\n' + ' 1, respectively, in almost all contexts, the exception ' + 'being\n' + ' that when converted to a string, the strings ""False"" ' + 'or\n' + ' ""True"" are returned, respectively.\n' + '\n' + ' The rules for integer representation are intended to give ' + 'the\n' + ' most meaningful interpretation of shift and mask ' + 'operations\n' + ' involving negative integers.\n' + '\n' + ' "numbers.Real" ("float")\n' + ' These represent machine-level double precision floating ' + 'point\n' + ' numbers. You are at the mercy of the underlying machine\n' + ' architecture (and C or Java implementation) for the ' + 'accepted\n' + ' range and handling of overflow. Python does not support ' + 'single-\n' + ' precision floating point numbers; the savings in processor ' + 'and\n' + ' memory usage that are usually the reason for using these ' + 'are\n' + ' dwarfed by the overhead of using objects in Python, so ' + 'there is\n' + ' no reason to complicate the language with two kinds of ' + 'floating\n' + ' point numbers.\n' + '\n' + ' "numbers.Complex" ("complex")\n' + ' These represent complex numbers as a pair of machine-level\n' + ' double precision floating point numbers. The same caveats ' + 'apply\n' + ' as for floating point numbers. The real and imaginary parts ' + 'of a\n' + ' complex number "z" can be retrieved through the read-only\n' + ' attributes "z.real" and "z.imag".\n' + '\n' + 'Sequences\n' + ' These represent finite ordered sets indexed by non-negative\n' + ' numbers. The built-in function "len()" returns the number of ' + 'items\n' + ' of a sequence. When the length of a sequence is *n*, the index ' + 'set\n' + ' contains the numbers 0, 1, ..., *n*-1. Item *i* of sequence ' + '*a* is\n' + ' selected by "a[i]".\n' + '\n' + ' Sequences also support slicing: "a[i:j]" selects all items ' + 'with\n' + ' index *k* such that *i* "<=" *k* "<" *j*. When used as an\n' + ' expression, a slice is a sequence of the same type. This ' + 'implies\n' + ' that the index set is renumbered so that it starts at 0.\n' + '\n' + ' Some sequences also support "extended slicing" with a third ' + '"step"\n' + ' parameter: "a[i:j:k]" selects all items of *a* with index *x* ' + 'where\n' + ' "x = i + n*k", *n* ">=" "0" and *i* "<=" *x* "<" *j*.\n' + '\n' + ' Sequences are distinguished according to their mutability:\n' + '\n' + ' Immutable sequences\n' + ' An object of an immutable sequence type cannot change once ' + 'it is\n' + ' created. (If the object contains references to other ' + 'objects,\n' + ' these other objects may be mutable and may be changed; ' + 'however,\n' + ' the collection of objects directly referenced by an ' + 'immutable\n' + ' object cannot change.)\n' + '\n' + ' The following types are immutable sequences:\n' + '\n' + ' Strings\n' + ' A string is a sequence of values that represent Unicode ' + 'code\n' + ' points. All the code points in the range "U+0000 - ' + 'U+10FFFF"\n' + " can be represented in a string. Python doesn't have a " + '"char"\n' + ' type; instead, every code point in the string is ' + 'represented\n' + ' as a string object with length "1". The built-in ' + 'function\n' + ' "ord()" converts a code point from its string form to ' + 'an\n' + ' integer in the range "0 - 10FFFF"; "chr()" converts an\n' + ' integer in the range "0 - 10FFFF" to the corresponding ' + 'length\n' + ' "1" string object. "str.encode()" can be used to convert ' + 'a\n' + ' "str" to "bytes" using the given text encoding, and\n' + ' "bytes.decode()" can be used to achieve the opposite.\n' + '\n' + ' Tuples\n' + ' The items of a tuple are arbitrary Python objects. ' + 'Tuples of\n' + ' two or more items are formed by comma-separated lists ' + 'of\n' + " expressions. A tuple of one item (a 'singleton') can " + 'be\n' + ' formed by affixing a comma to an expression (an ' + 'expression by\n' + ' itself does not create a tuple, since parentheses must ' + 'be\n' + ' usable for grouping of expressions). An empty tuple can ' + 'be\n' + ' formed by an empty pair of parentheses.\n' + '\n' + ' Bytes\n' + ' A bytes object is an immutable array. The items are ' + '8-bit\n' + ' bytes, represented by integers in the range 0 <= x < ' + '256.\n' + ' Bytes literals (like "b\'abc\'") and the built-in ' + 'function\n' + ' "bytes()" can be used to construct bytes objects. ' + 'Also,\n' + ' bytes objects can be decoded to strings via the ' + '"decode()"\n' + ' method.\n' + '\n' + ' Mutable sequences\n' + ' Mutable sequences can be changed after they are created. ' + 'The\n' + ' subscription and slicing notations can be used as the ' + 'target of\n' + ' assignment and "del" (delete) statements.\n' + '\n' + ' There are currently two intrinsic mutable sequence types:\n' + '\n' + ' Lists\n' + ' The items of a list are arbitrary Python objects. Lists ' + 'are\n' + ' formed by placing a comma-separated list of expressions ' + 'in\n' + ' square brackets. (Note that there are no special cases ' + 'needed\n' + ' to form lists of length 0 or 1.)\n' + '\n' + ' Byte Arrays\n' + ' A bytearray object is a mutable array. They are created ' + 'by\n' + ' the built-in "bytearray()" constructor. Aside from ' + 'being\n' + ' mutable (and hence unhashable), byte arrays otherwise ' + 'provide\n' + ' the same interface and functionality as immutable bytes\n' + ' objects.\n' + '\n' + ' The extension module "array" provides an additional example ' + 'of a\n' + ' mutable sequence type, as does the "collections" module.\n' + '\n' + 'Set types\n' + ' These represent unordered, finite sets of unique, immutable\n' + ' objects. As such, they cannot be indexed by any subscript. ' + 'However,\n' + ' they can be iterated over, and the built-in function "len()"\n' + ' returns the number of items in a set. Common uses for sets are ' + 'fast\n' + ' membership testing, removing duplicates from a sequence, and\n' + ' computing mathematical operations such as intersection, ' + 'union,\n' + ' difference, and symmetric difference.\n' + '\n' + ' For set elements, the same immutability rules apply as for\n' + ' dictionary keys. Note that numeric types obey the normal rules ' + 'for\n' + ' numeric comparison: if two numbers compare equal (e.g., "1" ' + 'and\n' + ' "1.0"), only one of them can be contained in a set.\n' + '\n' + ' There are currently two intrinsic set types:\n' + '\n' + ' Sets\n' + ' These represent a mutable set. They are created by the ' + 'built-in\n' + ' "set()" constructor and can be modified afterwards by ' + 'several\n' + ' methods, such as "add()".\n' + '\n' + ' Frozen sets\n' + ' These represent an immutable set. They are created by the\n' + ' built-in "frozenset()" constructor. As a frozenset is ' + 'immutable\n' + ' and *hashable*, it can be used again as an element of ' + 'another\n' + ' set, or as a dictionary key.\n' + '\n' + 'Mappings\n' + ' These represent finite sets of objects indexed by arbitrary ' + 'index\n' + ' sets. The subscript notation "a[k]" selects the item indexed ' + 'by "k"\n' + ' from the mapping "a"; this can be used in expressions and as ' + 'the\n' + ' target of assignments or "del" statements. The built-in ' + 'function\n' + ' "len()" returns the number of items in a mapping.\n' + '\n' + ' There is currently a single intrinsic mapping type:\n' + '\n' + ' Dictionaries\n' + ' These represent finite sets of objects indexed by nearly\n' + ' arbitrary values. The only types of values not acceptable ' + 'as\n' + ' keys are values containing lists or dictionaries or other\n' + ' mutable types that are compared by value rather than by ' + 'object\n' + ' identity, the reason being that the efficient ' + 'implementation of\n' + " dictionaries requires a key's hash value to remain " + 'constant.\n' + ' Numeric types used for keys obey the normal rules for ' + 'numeric\n' + ' comparison: if two numbers compare equal (e.g., "1" and ' + '"1.0")\n' + ' then they can be used interchangeably to index the same\n' + ' dictionary entry.\n' + '\n' + ' Dictionaries are mutable; they can be created by the ' + '"{...}"\n' + ' notation (see section *Dictionary displays*).\n' + '\n' + ' The extension modules "dbm.ndbm" and "dbm.gnu" provide\n' + ' additional examples of mapping types, as does the ' + '"collections"\n' + ' module.\n' + '\n' + 'Callable types\n' + ' These are the types to which the function call operation (see\n' + ' section *Calls*) can be applied:\n' + '\n' + ' User-defined functions\n' + ' A user-defined function object is created by a function\n' + ' definition (see section *Function definitions*). It should ' + 'be\n' + ' called with an argument list containing the same number of ' + 'items\n' + " as the function's formal parameter list.\n" + '\n' + ' Special attributes:\n' + '\n' + ' ' + '+---------------------------+---------------------------------+-------------+\n' + ' | Attribute | ' + 'Meaning | |\n' + ' ' + '+===========================+=================================+=============+\n' + ' | "__doc__" | The function\'s ' + 'documentation | Writable |\n' + ' | | string, or "None" ' + 'if | |\n' + ' | | unavailable; not inherited ' + 'by | |\n' + ' | | ' + 'subclasses | |\n' + ' ' + '+---------------------------+---------------------------------+-------------+\n' + ' | "__name__" | The function\'s ' + 'name | Writable |\n' + ' ' + '+---------------------------+---------------------------------+-------------+\n' + ' | "__qualname__" | The function\'s *qualified ' + 'name* | Writable |\n' + ' | | New in version ' + '3.3. | |\n' + ' ' + '+---------------------------+---------------------------------+-------------+\n' + ' | "__module__" | The name of the module ' + 'the | Writable |\n' + ' | | function was defined in, ' + 'or | |\n' + ' | | "None" if ' + 'unavailable. | |\n' + ' ' + '+---------------------------+---------------------------------+-------------+\n' + ' | "__defaults__" | A tuple containing ' + 'default | Writable |\n' + ' | | argument values for ' + 'those | |\n' + ' | | arguments that have ' + 'defaults, | |\n' + ' | | or "None" if no arguments ' + 'have | |\n' + ' | | a default ' + 'value | |\n' + ' ' + '+---------------------------+---------------------------------+-------------+\n' + ' | "__code__" | The code object ' + 'representing | Writable |\n' + ' | | the compiled function ' + 'body. | |\n' + ' ' + '+---------------------------+---------------------------------+-------------+\n' + ' | "__globals__" | A reference to the ' + 'dictionary | Read-only |\n' + ' | | that holds the ' + "function's | |\n" + ' | | global variables --- the ' + 'global | |\n' + ' | | namespace of the module ' + 'in | |\n' + ' | | which the function was ' + 'defined. | |\n' + ' ' + '+---------------------------+---------------------------------+-------------+\n' + ' | "__dict__" | The namespace ' + 'supporting | Writable |\n' + ' | | arbitrary function ' + 'attributes. | |\n' + ' ' + '+---------------------------+---------------------------------+-------------+\n' + ' | "__closure__" | "None" or a tuple of cells ' + 'that | Read-only |\n' + ' | | contain bindings for ' + 'the | |\n' + " | | function's free " + 'variables. | |\n' + ' ' + '+---------------------------+---------------------------------+-------------+\n' + ' | "__annotations__" | A dict containing ' + 'annotations | Writable |\n' + ' | | of parameters. The keys of ' + 'the | |\n' + ' | | dict are the parameter ' + 'names, | |\n' + ' | | and "\'return\'" for the ' + 'return | |\n' + ' | | annotation, if ' + 'provided. | |\n' + ' ' + '+---------------------------+---------------------------------+-------------+\n' + ' | "__kwdefaults__" | A dict containing defaults ' + 'for | Writable |\n' + ' | | keyword-only ' + 'parameters. | |\n' + ' ' + '+---------------------------+---------------------------------+-------------+\n' + '\n' + ' Most of the attributes labelled "Writable" check the type ' + 'of the\n' + ' assigned value.\n' + '\n' + ' Function objects also support getting and setting ' + 'arbitrary\n' + ' attributes, which can be used, for example, to attach ' + 'metadata\n' + ' to functions. Regular attribute dot-notation is used to ' + 'get and\n' + ' set such attributes. *Note that the current implementation ' + 'only\n' + ' supports function attributes on user-defined functions. ' + 'Function\n' + ' attributes on built-in functions may be supported in the\n' + ' future.*\n' + '\n' + " Additional information about a function's definition can " + 'be\n' + ' retrieved from its code object; see the description of ' + 'internal\n' + ' types below.\n' + '\n' + ' Instance methods\n' + ' An instance method object combines a class, a class ' + 'instance and\n' + ' any callable object (normally a user-defined function).\n' + '\n' + ' Special read-only attributes: "__self__" is the class ' + 'instance\n' + ' object, "__func__" is the function object; "__doc__" is ' + 'the\n' + ' method\'s documentation (same as "__func__.__doc__"); ' + '"__name__"\n' + ' is the method name (same as "__func__.__name__"); ' + '"__module__"\n' + ' is the name of the module the method was defined in, or ' + '"None"\n' + ' if unavailable.\n' + '\n' + ' Methods also support accessing (but not setting) the ' + 'arbitrary\n' + ' function attributes on the underlying function object.\n' + '\n' + ' User-defined method objects may be created when getting an\n' + ' attribute of a class (perhaps via an instance of that ' + 'class), if\n' + ' that attribute is a user-defined function object or a ' + 'class\n' + ' method object.\n' + '\n' + ' When an instance method object is created by retrieving a ' + 'user-\n' + ' defined function object from a class via one of its ' + 'instances,\n' + ' its "__self__" attribute is the instance, and the method ' + 'object\n' + ' is said to be bound. The new method\'s "__func__" ' + 'attribute is\n' + ' the original function object.\n' + '\n' + ' When a user-defined method object is created by retrieving\n' + ' another method object from a class or instance, the ' + 'behaviour is\n' + ' the same as for a function object, except that the ' + '"__func__"\n' + ' attribute of the new instance is not the original method ' + 'object\n' + ' but its "__func__" attribute.\n' + '\n' + ' When an instance method object is created by retrieving a ' + 'class\n' + ' method object from a class or instance, its "__self__" ' + 'attribute\n' + ' is the class itself, and its "__func__" attribute is the\n' + ' function object underlying the class method.\n' + '\n' + ' When an instance method object is called, the underlying\n' + ' function ("__func__") is called, inserting the class ' + 'instance\n' + ' ("__self__") in front of the argument list. For instance, ' + 'when\n' + ' "C" is a class which contains a definition for a function ' + '"f()",\n' + ' and "x" is an instance of "C", calling "x.f(1)" is ' + 'equivalent to\n' + ' calling "C.f(x, 1)".\n' + '\n' + ' When an instance method object is derived from a class ' + 'method\n' + ' object, the "class instance" stored in "__self__" will ' + 'actually\n' + ' be the class itself, so that calling either "x.f(1)" or ' + '"C.f(1)"\n' + ' is equivalent to calling "f(C,1)" where "f" is the ' + 'underlying\n' + ' function.\n' + '\n' + ' Note that the transformation from function object to ' + 'instance\n' + ' method object happens each time the attribute is retrieved ' + 'from\n' + ' the instance. In some cases, a fruitful optimization is ' + 'to\n' + ' assign the attribute to a local variable and call that ' + 'local\n' + ' variable. Also notice that this transformation only happens ' + 'for\n' + ' user-defined functions; other callable objects (and all ' + 'non-\n' + ' callable objects) are retrieved without transformation. It ' + 'is\n' + ' also important to note that user-defined functions which ' + 'are\n' + ' attributes of a class instance are not converted to bound\n' + ' methods; this *only* happens when the function is an ' + 'attribute\n' + ' of the class.\n' + '\n' + ' Generator functions\n' + ' A function or method which uses the "yield" statement (see\n' + ' section *The yield statement*) is called a *generator ' + 'function*.\n' + ' Such a function, when called, always returns an iterator ' + 'object\n' + ' which can be used to execute the body of the function: ' + 'calling\n' + ' the iterator\'s "iterator.__next__()" method will cause ' + 'the\n' + ' function to execute until it provides a value using the ' + '"yield"\n' + ' statement. When the function executes a "return" statement ' + 'or\n' + ' falls off the end, a "StopIteration" exception is raised ' + 'and the\n' + ' iterator will have reached the end of the set of values to ' + 'be\n' + ' returned.\n' + '\n' + ' Coroutine functions\n' + ' A function or method which is defined using "async def" is\n' + ' called a *coroutine function*. Such a function, when ' + 'called,\n' + ' returns a *coroutine* object. It may contain "await"\n' + ' expressions, as well as "async with" and "async for" ' + 'statements.\n' + ' See also the *Coroutine Objects* section.\n' + '\n' + ' Built-in functions\n' + ' A built-in function object is a wrapper around a C ' + 'function.\n' + ' Examples of built-in functions are "len()" and ' + '"math.sin()"\n' + ' ("math" is a standard built-in module). The number and type ' + 'of\n' + ' the arguments are determined by the C function. Special ' + 'read-\n' + ' only attributes: "__doc__" is the function\'s ' + 'documentation\n' + ' string, or "None" if unavailable; "__name__" is the ' + "function's\n" + ' name; "__self__" is set to "None" (but see the next item);\n' + ' "__module__" is the name of the module the function was ' + 'defined\n' + ' in or "None" if unavailable.\n' + '\n' + ' Built-in methods\n' + ' This is really a different disguise of a built-in function, ' + 'this\n' + ' time containing an object passed to the C function as an\n' + ' implicit extra argument. An example of a built-in method ' + 'is\n' + ' "alist.append()", assuming *alist* is a list object. In ' + 'this\n' + ' case, the special read-only attribute "__self__" is set to ' + 'the\n' + ' object denoted by *alist*.\n' + '\n' + ' Classes\n' + ' Classes are callable. These objects normally act as ' + 'factories\n' + ' for new instances of themselves, but variations are ' + 'possible for\n' + ' class types that override "__new__()". The arguments of ' + 'the\n' + ' call are passed to "__new__()" and, in the typical case, ' + 'to\n' + ' "__init__()" to initialize the new instance.\n' + '\n' + ' Class Instances\n' + ' Instances of arbitrary classes can be made callable by ' + 'defining\n' + ' a "__call__()" method in their class.\n' + '\n' + 'Modules\n' + ' Modules are a basic organizational unit of Python code, and ' + 'are\n' + ' created by the *import system* as invoked either by the ' + '"import"\n' + ' statement (see "import"), or by calling functions such as\n' + ' "importlib.import_module()" and built-in "__import__()". A ' + 'module\n' + ' object has a namespace implemented by a dictionary object ' + '(this is\n' + ' the dictionary referenced by the "__globals__" attribute of\n' + ' functions defined in the module). Attribute references are\n' + ' translated to lookups in this dictionary, e.g., "m.x" is ' + 'equivalent\n' + ' to "m.__dict__["x"]". A module object does not contain the ' + 'code\n' + " object used to initialize the module (since it isn't needed " + 'once\n' + ' the initialization is done).\n' + '\n' + " Attribute assignment updates the module's namespace " + 'dictionary,\n' + ' e.g., "m.x = 1" is equivalent to "m.__dict__["x"] = 1".\n' + '\n' + ' Special read-only attribute: "__dict__" is the module\'s ' + 'namespace\n' + ' as a dictionary object.\n' + '\n' + ' **CPython implementation detail:** Because of the way CPython\n' + ' clears module dictionaries, the module dictionary will be ' + 'cleared\n' + ' when the module falls out of scope even if the dictionary ' + 'still has\n' + ' live references. To avoid this, copy the dictionary or keep ' + 'the\n' + ' module around while using its dictionary directly.\n' + '\n' + ' Predefined (writable) attributes: "__name__" is the module\'s ' + 'name;\n' + ' "__doc__" is the module\'s documentation string, or "None" if\n' + ' unavailable; "__file__" is the pathname of the file from which ' + 'the\n' + ' module was loaded, if it was loaded from a file. The ' + '"__file__"\n' + ' attribute may be missing for certain types of modules, such as ' + 'C\n' + ' modules that are statically linked into the interpreter; for\n' + ' extension modules loaded dynamically from a shared library, it ' + 'is\n' + ' the pathname of the shared library file.\n' + '\n' + 'Custom classes\n' + ' Custom class types are typically created by class definitions ' + '(see\n' + ' section *Class definitions*). A class has a namespace ' + 'implemented\n' + ' by a dictionary object. Class attribute references are ' + 'translated\n' + ' to lookups in this dictionary, e.g., "C.x" is translated to\n' + ' "C.__dict__["x"]" (although there are a number of hooks which ' + 'allow\n' + ' for other means of locating attributes). When the attribute ' + 'name is\n' + ' not found there, the attribute search continues in the base\n' + ' classes. This search of the base classes uses the C3 method\n' + ' resolution order which behaves correctly even in the presence ' + 'of\n' + " 'diamond' inheritance structures where there are multiple\n" + ' inheritance paths leading back to a common ancestor. ' + 'Additional\n' + ' details on the C3 MRO used by Python can be found in the\n' + ' documentation accompanying the 2.3 release at\n' + ' https://www.python.org/download/releases/2.3/mro/.\n' + '\n' + ' When a class attribute reference (for class "C", say) would ' + 'yield a\n' + ' class method object, it is transformed into an instance ' + 'method\n' + ' object whose "__self__" attributes is "C". When it would ' + 'yield a\n' + ' static method object, it is transformed into the object ' + 'wrapped by\n' + ' the static method object. See section *Implementing ' + 'Descriptors*\n' + ' for another way in which attributes retrieved from a class ' + 'may\n' + ' differ from those actually contained in its "__dict__".\n' + '\n' + " Class attribute assignments update the class's dictionary, " + 'never\n' + ' the dictionary of a base class.\n' + '\n' + ' A class object can be called (see above) to yield a class ' + 'instance\n' + ' (see below).\n' + '\n' + ' Special attributes: "__name__" is the class name; "__module__" ' + 'is\n' + ' the module name in which the class was defined; "__dict__" is ' + 'the\n' + ' dictionary containing the class\'s namespace; "__bases__" is a ' + 'tuple\n' + ' (possibly empty or a singleton) containing the base classes, ' + 'in the\n' + ' order of their occurrence in the base class list; "__doc__" is ' + 'the\n' + " class's documentation string, or None if undefined.\n" + '\n' + 'Class instances\n' + ' A class instance is created by calling a class object (see ' + 'above).\n' + ' A class instance has a namespace implemented as a dictionary ' + 'which\n' + ' is the first place in which attribute references are ' + 'searched.\n' + " When an attribute is not found there, and the instance's class " + 'has\n' + ' an attribute by that name, the search continues with the ' + 'class\n' + ' attributes. If a class attribute is found that is a ' + 'user-defined\n' + ' function object, it is transformed into an instance method ' + 'object\n' + ' whose "__self__" attribute is the instance. Static method ' + 'and\n' + ' class method objects are also transformed; see above under\n' + ' "Classes". See section *Implementing Descriptors* for another ' + 'way\n' + ' in which attributes of a class retrieved via its instances ' + 'may\n' + " differ from the objects actually stored in the class's " + '"__dict__".\n' + " If no class attribute is found, and the object's class has a\n" + ' "__getattr__()" method, that is called to satisfy the lookup.\n' + '\n' + " Attribute assignments and deletions update the instance's\n" + " dictionary, never a class's dictionary. If the class has a\n" + ' "__setattr__()" or "__delattr__()" method, this is called ' + 'instead\n' + ' of updating the instance dictionary directly.\n' + '\n' + ' Class instances can pretend to be numbers, sequences, or ' + 'mappings\n' + ' if they have methods with certain special names. See section\n' + ' *Special method names*.\n' + '\n' + ' Special attributes: "__dict__" is the attribute dictionary;\n' + ' "__class__" is the instance\'s class.\n' + '\n' + 'I/O objects (also known as file objects)\n' + ' A *file object* represents an open file. Various shortcuts ' + 'are\n' + ' available to create file objects: the "open()" built-in ' + 'function,\n' + ' and also "os.popen()", "os.fdopen()", and the "makefile()" ' + 'method\n' + ' of socket objects (and perhaps by other functions or methods\n' + ' provided by extension modules).\n' + '\n' + ' The objects "sys.stdin", "sys.stdout" and "sys.stderr" are\n' + ' initialized to file objects corresponding to the ' + "interpreter's\n" + ' standard input, output and error streams; they are all open in ' + 'text\n' + ' mode and therefore follow the interface defined by the\n' + ' "io.TextIOBase" abstract class.\n' + '\n' + 'Internal types\n' + ' A few types used internally by the interpreter are exposed to ' + 'the\n' + ' user. Their definitions may change with future versions of ' + 'the\n' + ' interpreter, but they are mentioned here for completeness.\n' + '\n' + ' Code objects\n' + ' Code objects represent *byte-compiled* executable Python ' + 'code,\n' + ' or *bytecode*. The difference between a code object and a\n' + ' function object is that the function object contains an ' + 'explicit\n' + " reference to the function's globals (the module in which it " + 'was\n' + ' defined), while a code object contains no context; also ' + 'the\n' + ' default argument values are stored in the function object, ' + 'not\n' + ' in the code object (because they represent values ' + 'calculated at\n' + ' run-time). Unlike function objects, code objects are ' + 'immutable\n' + ' and contain no references (directly or indirectly) to ' + 'mutable\n' + ' objects.\n' + '\n' + ' Special read-only attributes: "co_name" gives the function ' + 'name;\n' + ' "co_argcount" is the number of positional arguments ' + '(including\n' + ' arguments with default values); "co_nlocals" is the number ' + 'of\n' + ' local variables used by the function (including ' + 'arguments);\n' + ' "co_varnames" is a tuple containing the names of the local\n' + ' variables (starting with the argument names); "co_cellvars" ' + 'is a\n' + ' tuple containing the names of local variables that are\n' + ' referenced by nested functions; "co_freevars" is a tuple\n' + ' containing the names of free variables; "co_code" is a ' + 'string\n' + ' representing the sequence of bytecode instructions; ' + '"co_consts"\n' + ' is a tuple containing the literals used by the bytecode;\n' + ' "co_names" is a tuple containing the names used by the ' + 'bytecode;\n' + ' "co_filename" is the filename from which the code was ' + 'compiled;\n' + ' "co_firstlineno" is the first line number of the function;\n' + ' "co_lnotab" is a string encoding the mapping from bytecode\n' + ' offsets to line numbers (for details see the source code of ' + 'the\n' + ' interpreter); "co_stacksize" is the required stack size\n' + ' (including local variables); "co_flags" is an integer ' + 'encoding a\n' + ' number of flags for the interpreter.\n' + '\n' + ' The following flag bits are defined for "co_flags": bit ' + '"0x04"\n' + ' is set if the function uses the "*arguments" syntax to ' + 'accept an\n' + ' arbitrary number of positional arguments; bit "0x08" is set ' + 'if\n' + ' the function uses the "**keywords" syntax to accept ' + 'arbitrary\n' + ' keyword arguments; bit "0x20" is set if the function is a\n' + ' generator.\n' + '\n' + ' Future feature declarations ("from __future__ import ' + 'division")\n' + ' also use bits in "co_flags" to indicate whether a code ' + 'object\n' + ' was compiled with a particular feature enabled: bit ' + '"0x2000" is\n' + ' set if the function was compiled with future division ' + 'enabled;\n' + ' bits "0x10" and "0x1000" were used in earlier versions of\n' + ' Python.\n' + '\n' + ' Other bits in "co_flags" are reserved for internal use.\n' + '\n' + ' If a code object represents a function, the first item in\n' + ' "co_consts" is the documentation string of the function, ' + 'or\n' + ' "None" if undefined.\n' + '\n' + ' Frame objects\n' + ' Frame objects represent execution frames. They may occur ' + 'in\n' + ' traceback objects (see below).\n' + '\n' + ' Special read-only attributes: "f_back" is to the previous ' + 'stack\n' + ' frame (towards the caller), or "None" if this is the ' + 'bottom\n' + ' stack frame; "f_code" is the code object being executed in ' + 'this\n' + ' frame; "f_locals" is the dictionary used to look up local\n' + ' variables; "f_globals" is used for global variables;\n' + ' "f_builtins" is used for built-in (intrinsic) names; ' + '"f_lasti"\n' + ' gives the precise instruction (this is an index into the\n' + ' bytecode string of the code object).\n' + '\n' + ' Special writable attributes: "f_trace", if not "None", is ' + 'a\n' + ' function called at the start of each source code line (this ' + 'is\n' + ' used by the debugger); "f_lineno" is the current line ' + 'number of\n' + ' the frame --- writing to this from within a trace function ' + 'jumps\n' + ' to the given line (only for the bottom-most frame). A ' + 'debugger\n' + ' can implement a Jump command (aka Set Next Statement) by ' + 'writing\n' + ' to f_lineno.\n' + '\n' + ' Frame objects support one method:\n' + '\n' + ' frame.clear()\n' + '\n' + ' This method clears all references to local variables ' + 'held by\n' + ' the frame. Also, if the frame belonged to a generator, ' + 'the\n' + ' generator is finalized. This helps break reference ' + 'cycles\n' + ' involving frame objects (for example when catching an\n' + ' exception and storing its traceback for later use).\n' + '\n' + ' "RuntimeError" is raised if the frame is currently ' + 'executing.\n' + '\n' + ' New in version 3.4.\n' + '\n' + ' Traceback objects\n' + ' Traceback objects represent a stack trace of an exception. ' + 'A\n' + ' traceback object is created when an exception occurs. When ' + 'the\n' + ' search for an exception handler unwinds the execution ' + 'stack, at\n' + ' each unwound level a traceback object is inserted in front ' + 'of\n' + ' the current traceback. When an exception handler is ' + 'entered,\n' + ' the stack trace is made available to the program. (See ' + 'section\n' + ' *The try statement*.) It is accessible as the third item of ' + 'the\n' + ' tuple returned by "sys.exc_info()". When the program ' + 'contains no\n' + ' suitable handler, the stack trace is written (nicely ' + 'formatted)\n' + ' to the standard error stream; if the interpreter is ' + 'interactive,\n' + ' it is also made available to the user as ' + '"sys.last_traceback".\n' + '\n' + ' Special read-only attributes: "tb_next" is the next level ' + 'in the\n' + ' stack trace (towards the frame where the exception ' + 'occurred), or\n' + ' "None" if there is no next level; "tb_frame" points to the\n' + ' execution frame of the current level; "tb_lineno" gives the ' + 'line\n' + ' number where the exception occurred; "tb_lasti" indicates ' + 'the\n' + ' precise instruction. The line number and last instruction ' + 'in\n' + ' the traceback may differ from the line number of its frame\n' + ' object if the exception occurred in a "try" statement with ' + 'no\n' + ' matching except clause or with a finally clause.\n' + '\n' + ' Slice objects\n' + ' Slice objects are used to represent slices for ' + '"__getitem__()"\n' + ' methods. They are also created by the built-in "slice()"\n' + ' function.\n' + '\n' + ' Special read-only attributes: "start" is the lower bound; ' + '"stop"\n' + ' is the upper bound; "step" is the step value; each is ' + '"None" if\n' + ' omitted. These attributes can have any type.\n' + '\n' + ' Slice objects support one method:\n' + '\n' + ' slice.indices(self, length)\n' + '\n' + ' This method takes a single integer argument *length* ' + 'and\n' + ' computes information about the slice that the slice ' + 'object\n' + ' would describe if applied to a sequence of *length* ' + 'items.\n' + ' It returns a tuple of three integers; respectively these ' + 'are\n' + ' the *start* and *stop* indices and the *step* or stride\n' + ' length of the slice. Missing or out-of-bounds indices ' + 'are\n' + ' handled in a manner consistent with regular slices.\n' + '\n' + ' Static method objects\n' + ' Static method objects provide a way of defeating the\n' + ' transformation of function objects to method objects ' + 'described\n' + ' above. A static method object is a wrapper around any ' + 'other\n' + ' object, usually a user-defined method object. When a ' + 'static\n' + ' method object is retrieved from a class or a class ' + 'instance, the\n' + ' object actually returned is the wrapped object, which is ' + 'not\n' + ' subject to any further transformation. Static method ' + 'objects are\n' + ' not themselves callable, although the objects they wrap ' + 'usually\n' + ' are. Static method objects are created by the built-in\n' + ' "staticmethod()" constructor.\n' + '\n' + ' Class method objects\n' + ' A class method object, like a static method object, is a ' + 'wrapper\n' + ' around another object that alters the way in which that ' + 'object\n' + ' is retrieved from classes and class instances. The ' + 'behaviour of\n' + ' class method objects upon such retrieval is described ' + 'above,\n' + ' under "User-defined methods". Class method objects are ' + 'created\n' + ' by the built-in "classmethod()" constructor.\n', + 'typesfunctions': '\n' + 'Functions\n' + '*********\n' + '\n' + 'Function objects are created by function definitions. ' + 'The only\n' + 'operation on a function object is to call it: ' + '"func(argument-list)".\n' + '\n' + 'There are really two flavors of function objects: ' + 'built-in functions\n' + 'and user-defined functions. Both support the same ' + 'operation (to call\n' + 'the function), but the implementation is different, ' + 'hence the\n' + 'different object types.\n' + '\n' + 'See *Function definitions* for more information.\n', + 'typesmapping': '\n' + 'Mapping Types --- "dict"\n' + '************************\n' + '\n' + 'A *mapping* object maps *hashable* values to arbitrary ' + 'objects.\n' + 'Mappings are mutable objects. There is currently only one ' + 'standard\n' + 'mapping type, the *dictionary*. (For other containers see ' + 'the built-\n' + 'in "list", "set", and "tuple" classes, and the ' + '"collections" module.)\n' + '\n' + "A dictionary's keys are *almost* arbitrary values. Values " + 'that are\n' + 'not *hashable*, that is, values containing lists, ' + 'dictionaries or\n' + 'other mutable types (that are compared by value rather ' + 'than by object\n' + 'identity) may not be used as keys. Numeric types used for ' + 'keys obey\n' + 'the normal rules for numeric comparison: if two numbers ' + 'compare equal\n' + '(such as "1" and "1.0") then they can be used ' + 'interchangeably to index\n' + 'the same dictionary entry. (Note however, that since ' + 'computers store\n' + 'floating-point numbers as approximations it is usually ' + 'unwise to use\n' + 'them as dictionary keys.)\n' + '\n' + 'Dictionaries can be created by placing a comma-separated ' + 'list of "key:\n' + 'value" pairs within braces, for example: "{\'jack\': 4098, ' + "'sjoerd':\n" + '4127}" or "{4098: \'jack\', 4127: \'sjoerd\'}", or by the ' + '"dict"\n' + 'constructor.\n' + '\n' + 'class class dict(**kwarg)\n' + 'class class dict(mapping, **kwarg)\n' + 'class class dict(iterable, **kwarg)\n' + '\n' + ' Return a new dictionary initialized from an optional ' + 'positional\n' + ' argument and a possibly empty set of keyword ' + 'arguments.\n' + '\n' + ' If no positional argument is given, an empty dictionary ' + 'is created.\n' + ' If a positional argument is given and it is a mapping ' + 'object, a\n' + ' dictionary is created with the same key-value pairs as ' + 'the mapping\n' + ' object. Otherwise, the positional argument must be an ' + '*iterable*\n' + ' object. Each item in the iterable must itself be an ' + 'iterable with\n' + ' exactly two objects. The first object of each item ' + 'becomes a key\n' + ' in the new dictionary, and the second object the ' + 'corresponding\n' + ' value. If a key occurs more than once, the last value ' + 'for that key\n' + ' becomes the corresponding value in the new dictionary.\n' + '\n' + ' If keyword arguments are given, the keyword arguments ' + 'and their\n' + ' values are added to the dictionary created from the ' + 'positional\n' + ' argument. If a key being added is already present, the ' + 'value from\n' + ' the keyword argument replaces the value from the ' + 'positional\n' + ' argument.\n' + '\n' + ' To illustrate, the following examples all return a ' + 'dictionary equal\n' + ' to "{"one": 1, "two": 2, "three": 3}":\n' + '\n' + ' >>> a = dict(one=1, two=2, three=3)\n' + " >>> b = {'one': 1, 'two': 2, 'three': 3}\n" + " >>> c = dict(zip(['one', 'two', 'three'], [1, 2, " + '3]))\n' + " >>> d = dict([('two', 2), ('one', 1), ('three', " + '3)])\n' + " >>> e = dict({'three': 3, 'one': 1, 'two': 2})\n" + ' >>> a == b == c == d == e\n' + ' True\n' + '\n' + ' Providing keyword arguments as in the first example ' + 'only works for\n' + ' keys that are valid Python identifiers. Otherwise, any ' + 'valid keys\n' + ' can be used.\n' + '\n' + ' These are the operations that dictionaries support (and ' + 'therefore,\n' + ' custom mapping types should support too):\n' + '\n' + ' len(d)\n' + '\n' + ' Return the number of items in the dictionary *d*.\n' + '\n' + ' d[key]\n' + '\n' + ' Return the item of *d* with key *key*. Raises a ' + '"KeyError" if\n' + ' *key* is not in the map.\n' + '\n' + ' If a subclass of dict defines a method ' + '"__missing__()" and *key*\n' + ' is not present, the "d[key]" operation calls that ' + 'method with\n' + ' the key *key* as argument. The "d[key]" operation ' + 'then returns\n' + ' or raises whatever is returned or raised by the\n' + ' "__missing__(key)" call. No other operations or ' + 'methods invoke\n' + ' "__missing__()". If "__missing__()" is not defined, ' + '"KeyError"\n' + ' is raised. "__missing__()" must be a method; it ' + 'cannot be an\n' + ' instance variable:\n' + '\n' + ' >>> class Counter(dict):\n' + ' ... def __missing__(self, key):\n' + ' ... return 0\n' + ' >>> c = Counter()\n' + " >>> c['red']\n" + ' 0\n' + " >>> c['red'] += 1\n" + " >>> c['red']\n" + ' 1\n' + '\n' + ' The example above shows part of the implementation ' + 'of\n' + ' "collections.Counter". A different "__missing__" ' + 'method is used\n' + ' by "collections.defaultdict".\n' + '\n' + ' d[key] = value\n' + '\n' + ' Set "d[key]" to *value*.\n' + '\n' + ' del d[key]\n' + '\n' + ' Remove "d[key]" from *d*. Raises a "KeyError" if ' + '*key* is not\n' + ' in the map.\n' + '\n' + ' key in d\n' + '\n' + ' Return "True" if *d* has a key *key*, else "False".\n' + '\n' + ' key not in d\n' + '\n' + ' Equivalent to "not key in d".\n' + '\n' + ' iter(d)\n' + '\n' + ' Return an iterator over the keys of the dictionary. ' + 'This is a\n' + ' shortcut for "iter(d.keys())".\n' + '\n' + ' clear()\n' + '\n' + ' Remove all items from the dictionary.\n' + '\n' + ' copy()\n' + '\n' + ' Return a shallow copy of the dictionary.\n' + '\n' + ' classmethod fromkeys(seq[, value])\n' + '\n' + ' Create a new dictionary with keys from *seq* and ' + 'values set to\n' + ' *value*.\n' + '\n' + ' "fromkeys()" is a class method that returns a new ' + 'dictionary.\n' + ' *value* defaults to "None".\n' + '\n' + ' get(key[, default])\n' + '\n' + ' Return the value for *key* if *key* is in the ' + 'dictionary, else\n' + ' *default*. If *default* is not given, it defaults to ' + '"None", so\n' + ' that this method never raises a "KeyError".\n' + '\n' + ' items()\n' + '\n' + ' Return a new view of the dictionary\'s items ("(key, ' + 'value)"\n' + ' pairs). See the *documentation of view objects*.\n' + '\n' + ' keys()\n' + '\n' + " Return a new view of the dictionary's keys. See " + 'the\n' + ' *documentation of view objects*.\n' + '\n' + ' pop(key[, default])\n' + '\n' + ' If *key* is in the dictionary, remove it and return ' + 'its value,\n' + ' else return *default*. If *default* is not given ' + 'and *key* is\n' + ' not in the dictionary, a "KeyError" is raised.\n' + '\n' + ' popitem()\n' + '\n' + ' Remove and return an arbitrary "(key, value)" pair ' + 'from the\n' + ' dictionary.\n' + '\n' + ' "popitem()" is useful to destructively iterate over ' + 'a\n' + ' dictionary, as often used in set algorithms. If the ' + 'dictionary\n' + ' is empty, calling "popitem()" raises a "KeyError".\n' + '\n' + ' setdefault(key[, default])\n' + '\n' + ' If *key* is in the dictionary, return its value. If ' + 'not, insert\n' + ' *key* with a value of *default* and return ' + '*default*. *default*\n' + ' defaults to "None".\n' + '\n' + ' update([other])\n' + '\n' + ' Update the dictionary with the key/value pairs from ' + '*other*,\n' + ' overwriting existing keys. Return "None".\n' + '\n' + ' "update()" accepts either another dictionary object ' + 'or an\n' + ' iterable of key/value pairs (as tuples or other ' + 'iterables of\n' + ' length two). If keyword arguments are specified, ' + 'the dictionary\n' + ' is then updated with those key/value pairs: ' + '"d.update(red=1,\n' + ' blue=2)".\n' + '\n' + ' values()\n' + '\n' + " Return a new view of the dictionary's values. See " + 'the\n' + ' *documentation of view objects*.\n' + '\n' + ' Dictionaries compare equal if and only if they have the ' + 'same "(key,\n' + ' value)" pairs. Order comparisons (\'<\', \'<=\', ' + "'>=', '>') raise\n" + ' "TypeError".\n' + '\n' + 'See also: "types.MappingProxyType" can be used to create a ' + 'read-only\n' + ' view of a "dict".\n' + '\n' + '\n' + 'Dictionary view objects\n' + '=======================\n' + '\n' + 'The objects returned by "dict.keys()", "dict.values()" ' + 'and\n' + '"dict.items()" are *view objects*. They provide a dynamic ' + 'view on the\n' + "dictionary's entries, which means that when the dictionary " + 'changes,\n' + 'the view reflects these changes.\n' + '\n' + 'Dictionary views can be iterated over to yield their ' + 'respective data,\n' + 'and support membership tests:\n' + '\n' + 'len(dictview)\n' + '\n' + ' Return the number of entries in the dictionary.\n' + '\n' + 'iter(dictview)\n' + '\n' + ' Return an iterator over the keys, values or items ' + '(represented as\n' + ' tuples of "(key, value)") in the dictionary.\n' + '\n' + ' Keys and values are iterated over in an arbitrary order ' + 'which is\n' + ' non-random, varies across Python implementations, and ' + 'depends on\n' + " the dictionary's history of insertions and deletions. " + 'If keys,\n' + ' values and items views are iterated over with no ' + 'intervening\n' + ' modifications to the dictionary, the order of items ' + 'will directly\n' + ' correspond. This allows the creation of "(value, key)" ' + 'pairs using\n' + ' "zip()": "pairs = zip(d.values(), d.keys())". Another ' + 'way to\n' + ' create the same list is "pairs = [(v, k) for (k, v) in ' + 'd.items()]".\n' + '\n' + ' Iterating views while adding or deleting entries in the ' + 'dictionary\n' + ' may raise a "RuntimeError" or fail to iterate over all ' + 'entries.\n' + '\n' + 'x in dictview\n' + '\n' + ' Return "True" if *x* is in the underlying dictionary\'s ' + 'keys, values\n' + ' or items (in the latter case, *x* should be a "(key, ' + 'value)"\n' + ' tuple).\n' + '\n' + 'Keys views are set-like since their entries are unique and ' + 'hashable.\n' + 'If all values are hashable, so that "(key, value)" pairs ' + 'are unique\n' + 'and hashable, then the items view is also set-like. ' + '(Values views are\n' + 'not treated as set-like since the entries are generally ' + 'not unique.)\n' + 'For set-like views, all of the operations defined for the ' + 'abstract\n' + 'base class "collections.abc.Set" are available (for ' + 'example, "==",\n' + '"<", or "^").\n' + '\n' + 'An example of dictionary view usage:\n' + '\n' + " >>> dishes = {'eggs': 2, 'sausage': 1, 'bacon': 1, " + "'spam': 500}\n" + ' >>> keys = dishes.keys()\n' + ' >>> values = dishes.values()\n' + '\n' + ' >>> # iteration\n' + ' >>> n = 0\n' + ' >>> for val in values:\n' + ' ... n += val\n' + ' >>> print(n)\n' + ' 504\n' + '\n' + ' >>> # keys and values are iterated over in the same ' + 'order\n' + ' >>> list(keys)\n' + " ['eggs', 'bacon', 'sausage', 'spam']\n" + ' >>> list(values)\n' + ' [2, 1, 1, 500]\n' + '\n' + ' >>> # view objects are dynamic and reflect dict ' + 'changes\n' + " >>> del dishes['eggs']\n" + " >>> del dishes['sausage']\n" + ' >>> list(keys)\n' + " ['spam', 'bacon']\n" + '\n' + ' >>> # set operations\n' + " >>> keys & {'eggs', 'bacon', 'salad'}\n" + " {'bacon'}\n" + " >>> keys ^ {'sausage', 'juice'}\n" + " {'juice', 'sausage', 'bacon', 'spam'}\n", + 'typesmethods': '\n' + 'Methods\n' + '*******\n' + '\n' + 'Methods are functions that are called using the attribute ' + 'notation.\n' + 'There are two flavors: built-in methods (such as ' + '"append()" on lists)\n' + 'and class instance methods. Built-in methods are ' + 'described with the\n' + 'types that support them.\n' + '\n' + 'If you access a method (a function defined in a class ' + 'namespace)\n' + 'through an instance, you get a special object: a *bound ' + 'method* (also\n' + 'called *instance method*) object. When called, it will add ' + 'the "self"\n' + 'argument to the argument list. Bound methods have two ' + 'special read-\n' + 'only attributes: "m.__self__" is the object on which the ' + 'method\n' + 'operates, and "m.__func__" is the function implementing ' + 'the method.\n' + 'Calling "m(arg-1, arg-2, ..., arg-n)" is completely ' + 'equivalent to\n' + 'calling "m.__func__(m.__self__, arg-1, arg-2, ..., ' + 'arg-n)".\n' + '\n' + 'Like function objects, bound method objects support ' + 'getting arbitrary\n' + 'attributes. However, since method attributes are actually ' + 'stored on\n' + 'the underlying function object ("meth.__func__"), setting ' + 'method\n' + 'attributes on bound methods is disallowed. Attempting to ' + 'set an\n' + 'attribute on a method results in an "AttributeError" being ' + 'raised. In\n' + 'order to set a method attribute, you need to explicitly ' + 'set it on the\n' + 'underlying function object:\n' + '\n' + ' >>> class C:\n' + ' ... def method(self):\n' + ' ... pass\n' + ' ...\n' + ' >>> c = C()\n' + " >>> c.method.whoami = 'my name is method' # can't set " + 'on the method\n' + ' Traceback (most recent call last):\n' + ' File "", line 1, in \n' + " AttributeError: 'method' object has no attribute " + "'whoami'\n" + " >>> c.method.__func__.whoami = 'my name is method'\n" + ' >>> c.method.whoami\n' + " 'my name is method'\n" + '\n' + 'See *The standard type hierarchy* for more information.\n', + 'typesmodules': '\n' + 'Modules\n' + '*******\n' + '\n' + 'The only special operation on a module is attribute ' + 'access: "m.name",\n' + 'where *m* is a module and *name* accesses a name defined ' + "in *m*'s\n" + 'symbol table. Module attributes can be assigned to. (Note ' + 'that the\n' + '"import" statement is not, strictly speaking, an operation ' + 'on a module\n' + 'object; "import foo" does not require a module object ' + 'named *foo* to\n' + 'exist, rather it requires an (external) *definition* for a ' + 'module\n' + 'named *foo* somewhere.)\n' + '\n' + 'A special attribute of every module is "__dict__". This is ' + 'the\n' + "dictionary containing the module's symbol table. Modifying " + 'this\n' + "dictionary will actually change the module's symbol table, " + 'but direct\n' + 'assignment to the "__dict__" attribute is not possible ' + '(you can write\n' + '"m.__dict__[\'a\'] = 1", which defines "m.a" to be "1", ' + "but you can't\n" + 'write "m.__dict__ = {}"). Modifying "__dict__" directly ' + 'is not\n' + 'recommended.\n' + '\n' + 'Modules built into the interpreter are written like this: ' + '"". If loaded from a file, they are ' + 'written as\n' + '"".\n', + 'typesseq': '\n' + 'Sequence Types --- "list", "tuple", "range"\n' + '*******************************************\n' + '\n' + 'There are three basic sequence types: lists, tuples, and ' + 'range\n' + 'objects. Additional sequence types tailored for processing of ' + '*binary\n' + 'data* and *text strings* are described in dedicated sections.\n' + '\n' + '\n' + 'Common Sequence Operations\n' + '==========================\n' + '\n' + 'The operations in the following table are supported by most ' + 'sequence\n' + 'types, both mutable and immutable. The ' + '"collections.abc.Sequence" ABC\n' + 'is provided to make it easier to correctly implement these ' + 'operations\n' + 'on custom sequence types.\n' + '\n' + 'This table lists the sequence operations sorted in ascending ' + 'priority.\n' + 'In the table, *s* and *t* are sequences of the same type, *n*, ' + '*i*,\n' + '*j* and *k* are integers and *x* is an arbitrary object that ' + 'meets any\n' + 'type and value restrictions imposed by *s*.\n' + '\n' + 'The "in" and "not in" operations have the same priorities as ' + 'the\n' + 'comparison operations. The "+" (concatenation) and "*" ' + '(repetition)\n' + 'operations have the same priority as the corresponding ' + 'numeric\n' + 'operations.\n' + '\n' + '+----------------------------+----------------------------------+------------+\n' + '| Operation | ' + 'Result | Notes |\n' + '+============================+==================================+============+\n' + '| "x in s" | "True" if an item of *s* ' + 'is | (1) |\n' + '| | equal to *x*, else ' + '"False" | |\n' + '+----------------------------+----------------------------------+------------+\n' + '| "x not in s" | "False" if an item of *s* ' + 'is | (1) |\n' + '| | equal to *x*, else ' + '"True" | |\n' + '+----------------------------+----------------------------------+------------+\n' + '| "s + t" | the concatenation of *s* and ' + '*t* | (6)(7) |\n' + '+----------------------------+----------------------------------+------------+\n' + '| "s * n" or "n * s" | *n* shallow copies of ' + '*s* | (2)(7) |\n' + '| | ' + 'concatenated | |\n' + '+----------------------------+----------------------------------+------------+\n' + '| "s[i]" | *i*th item of *s*, origin ' + '0 | (3) |\n' + '+----------------------------+----------------------------------+------------+\n' + '| "s[i:j]" | slice of *s* from *i* to ' + '*j* | (3)(4) |\n' + '+----------------------------+----------------------------------+------------+\n' + '| "s[i:j:k]" | slice of *s* from *i* to ' + '*j* | (3)(5) |\n' + '| | with step ' + '*k* | |\n' + '+----------------------------+----------------------------------+------------+\n' + '| "len(s)" | length of ' + '*s* | |\n' + '+----------------------------+----------------------------------+------------+\n' + '| "min(s)" | smallest item of ' + '*s* | |\n' + '+----------------------------+----------------------------------+------------+\n' + '| "max(s)" | largest item of ' + '*s* | |\n' + '+----------------------------+----------------------------------+------------+\n' + '| "s.index(x[, i[, j]])" | index of the first occurrence ' + 'of | (8) |\n' + '| | *x* in *s* (at or after ' + 'index | |\n' + '| | *i* and before index ' + '*j*) | |\n' + '+----------------------------+----------------------------------+------------+\n' + '| "s.count(x)" | total number of occurrences ' + 'of | |\n' + '| | *x* in ' + '*s* | |\n' + '+----------------------------+----------------------------------+------------+\n' + '\n' + 'Sequences of the same type also support comparisons. In ' + 'particular,\n' + 'tuples and lists are compared lexicographically by comparing\n' + 'corresponding elements. This means that to compare equal, ' + 'every\n' + 'element must compare equal and the two sequences must be of ' + 'the same\n' + 'type and have the same length. (For full details see ' + '*Comparisons* in\n' + 'the language reference.)\n' + '\n' + 'Notes:\n' + '\n' + '1. While the "in" and "not in" operations are used only for ' + 'simple\n' + ' containment testing in the general case, some specialised ' + 'sequences\n' + ' (such as "str", "bytes" and "bytearray") also use them for\n' + ' subsequence testing:\n' + '\n' + ' >>> "gg" in "eggs"\n' + ' True\n' + '\n' + '2. Values of *n* less than "0" are treated as "0" (which ' + 'yields an\n' + ' empty sequence of the same type as *s*). Note also that ' + 'the copies\n' + ' are shallow; nested structures are not copied. This often ' + 'haunts\n' + ' new Python programmers; consider:\n' + '\n' + ' >>> lists = [[]] * 3\n' + ' >>> lists\n' + ' [[], [], []]\n' + ' >>> lists[0].append(3)\n' + ' >>> lists\n' + ' [[3], [3], [3]]\n' + '\n' + ' What has happened is that "[[]]" is a one-element list ' + 'containing\n' + ' an empty list, so all three elements of "[[]] * 3" are ' + '(pointers\n' + ' to) this single empty list. Modifying any of the elements ' + 'of\n' + ' "lists" modifies this single list. You can create a list ' + 'of\n' + ' different lists this way:\n' + '\n' + ' >>> lists = [[] for i in range(3)]\n' + ' >>> lists[0].append(3)\n' + ' >>> lists[1].append(5)\n' + ' >>> lists[2].append(7)\n' + ' >>> lists\n' + ' [[3], [5], [7]]\n' + '\n' + '3. If *i* or *j* is negative, the index is relative to the end ' + 'of\n' + ' the string: "len(s) + i" or "len(s) + j" is substituted. ' + 'But note\n' + ' that "-0" is still "0".\n' + '\n' + '4. The slice of *s* from *i* to *j* is defined as the sequence ' + 'of\n' + ' items with index *k* such that "i <= k < j". If *i* or *j* ' + 'is\n' + ' greater than "len(s)", use "len(s)". If *i* is omitted or ' + '"None",\n' + ' use "0". If *j* is omitted or "None", use "len(s)". If ' + '*i* is\n' + ' greater than or equal to *j*, the slice is empty.\n' + '\n' + '5. The slice of *s* from *i* to *j* with step *k* is defined ' + 'as the\n' + ' sequence of items with index "x = i + n*k" such that "0 <= ' + 'n <\n' + ' (j-i)/k". In other words, the indices are "i", "i+k", ' + '"i+2*k",\n' + ' "i+3*k" and so on, stopping when *j* is reached (but never\n' + ' including *j*). If *i* or *j* is greater than "len(s)", ' + 'use\n' + ' "len(s)". If *i* or *j* are omitted or "None", they become ' + '"end"\n' + ' values (which end depends on the sign of *k*). Note, *k* ' + 'cannot be\n' + ' zero. If *k* is "None", it is treated like "1".\n' + '\n' + '6. Concatenating immutable sequences always results in a new\n' + ' object. This means that building up a sequence by repeated\n' + ' concatenation will have a quadratic runtime cost in the ' + 'total\n' + ' sequence length. To get a linear runtime cost, you must ' + 'switch to\n' + ' one of the alternatives below:\n' + '\n' + ' * if concatenating "str" objects, you can build a list and ' + 'use\n' + ' "str.join()" at the end or else write to a "io.StringIO" ' + 'instance\n' + ' and retrieve its value when complete\n' + '\n' + ' * if concatenating "bytes" objects, you can similarly use\n' + ' "bytes.join()" or "io.BytesIO", or you can do in-place\n' + ' concatenation with a "bytearray" object. "bytearray" ' + 'objects are\n' + ' mutable and have an efficient overallocation mechanism\n' + '\n' + ' * if concatenating "tuple" objects, extend a "list" ' + 'instead\n' + '\n' + ' * for other types, investigate the relevant class ' + 'documentation\n' + '\n' + '7. Some sequence types (such as "range") only support item\n' + " sequences that follow specific patterns, and hence don't " + 'support\n' + ' sequence concatenation or repetition.\n' + '\n' + '8. "index" raises "ValueError" when *x* is not found in *s*. ' + 'When\n' + ' supported, the additional arguments to the index method ' + 'allow\n' + ' efficient searching of subsections of the sequence. Passing ' + 'the\n' + ' extra arguments is roughly equivalent to using ' + '"s[i:j].index(x)",\n' + ' only without copying any data and with the returned index ' + 'being\n' + ' relative to the start of the sequence rather than the start ' + 'of the\n' + ' slice.\n' + '\n' + '\n' + 'Immutable Sequence Types\n' + '========================\n' + '\n' + 'The only operation that immutable sequence types generally ' + 'implement\n' + 'that is not also implemented by mutable sequence types is ' + 'support for\n' + 'the "hash()" built-in.\n' + '\n' + 'This support allows immutable sequences, such as "tuple" ' + 'instances, to\n' + 'be used as "dict" keys and stored in "set" and "frozenset" ' + 'instances.\n' + '\n' + 'Attempting to hash an immutable sequence that contains ' + 'unhashable\n' + 'values will result in "TypeError".\n' + '\n' + '\n' + 'Mutable Sequence Types\n' + '======================\n' + '\n' + 'The operations in the following table are defined on mutable ' + 'sequence\n' + 'types. The "collections.abc.MutableSequence" ABC is provided ' + 'to make\n' + 'it easier to correctly implement these operations on custom ' + 'sequence\n' + 'types.\n' + '\n' + 'In the table *s* is an instance of a mutable sequence type, ' + '*t* is any\n' + 'iterable object and *x* is an arbitrary object that meets any ' + 'type and\n' + 'value restrictions imposed by *s* (for example, "bytearray" ' + 'only\n' + 'accepts integers that meet the value restriction "0 <= x <= ' + '255").\n' + '\n' + '+--------------------------------+----------------------------------+-----------------------+\n' + '| Operation | ' + 'Result | Notes |\n' + '+================================+==================================+=======================+\n' + '| "s[i] = x" | item *i* of *s* is replaced ' + 'by | |\n' + '| | ' + '*x* | |\n' + '+--------------------------------+----------------------------------+-----------------------+\n' + '| "s[i:j] = t" | slice of *s* from *i* to ' + '*j* is | |\n' + '| | replaced by the contents of ' + 'the | |\n' + '| | iterable ' + '*t* | |\n' + '+--------------------------------+----------------------------------+-----------------------+\n' + '| "del s[i:j]" | same as "s[i:j] = ' + '[]" | |\n' + '+--------------------------------+----------------------------------+-----------------------+\n' + '| "s[i:j:k] = t" | the elements of "s[i:j:k]" ' + 'are | (1) |\n' + '| | replaced by those of ' + '*t* | |\n' + '+--------------------------------+----------------------------------+-----------------------+\n' + '| "del s[i:j:k]" | removes the elements ' + 'of | |\n' + '| | "s[i:j:k]" from the ' + 'list | |\n' + '+--------------------------------+----------------------------------+-----------------------+\n' + '| "s.append(x)" | appends *x* to the end of ' + 'the | |\n' + '| | sequence (same ' + 'as | |\n' + '| | "s[len(s):len(s)] = ' + '[x]") | |\n' + '+--------------------------------+----------------------------------+-----------------------+\n' + '| "s.clear()" | removes all items from "s" ' + '(same | (5) |\n' + '| | as "del ' + 's[:]") | |\n' + '+--------------------------------+----------------------------------+-----------------------+\n' + '| "s.copy()" | creates a shallow copy of ' + '"s" | (5) |\n' + '| | (same as ' + '"s[:]") | |\n' + '+--------------------------------+----------------------------------+-----------------------+\n' + '| "s.extend(t)" | extends *s* with the ' + 'contents of | |\n' + '| | *t* (same as ' + '"s[len(s):len(s)] = | |\n' + '| | ' + 't") | |\n' + '+--------------------------------+----------------------------------+-----------------------+\n' + '| "s.insert(i, x)" | inserts *x* into *s* at ' + 'the | |\n' + '| | index given by *i* (same ' + 'as | |\n' + '| | "s[i:i] = ' + '[x]") | |\n' + '+--------------------------------+----------------------------------+-----------------------+\n' + '| "s.pop([i])" | retrieves the item at *i* ' + 'and | (2) |\n' + '| | also removes it from ' + '*s* | |\n' + '+--------------------------------+----------------------------------+-----------------------+\n' + '| "s.remove(x)" | remove the first item from ' + '*s* | (3) |\n' + '| | where "s[i] == ' + 'x" | |\n' + '+--------------------------------+----------------------------------+-----------------------+\n' + '| "s.reverse()" | reverses the items of *s* ' + 'in | (4) |\n' + '| | ' + 'place | |\n' + '+--------------------------------+----------------------------------+-----------------------+\n' + '\n' + 'Notes:\n' + '\n' + '1. *t* must have the same length as the slice it is ' + 'replacing.\n' + '\n' + '2. The optional argument *i* defaults to "-1", so that by ' + 'default\n' + ' the last item is removed and returned.\n' + '\n' + '3. "remove" raises "ValueError" when *x* is not found in *s*.\n' + '\n' + '4. The "reverse()" method modifies the sequence in place for\n' + ' economy of space when reversing a large sequence. To ' + 'remind users\n' + ' that it operates by side effect, it does not return the ' + 'reversed\n' + ' sequence.\n' + '\n' + '5. "clear()" and "copy()" are included for consistency with ' + 'the\n' + " interfaces of mutable containers that don't support " + 'slicing\n' + ' operations (such as "dict" and "set")\n' + '\n' + ' New in version 3.3: "clear()" and "copy()" methods.\n' + '\n' + '\n' + 'Lists\n' + '=====\n' + '\n' + 'Lists are mutable sequences, typically used to store ' + 'collections of\n' + 'homogeneous items (where the precise degree of similarity will ' + 'vary by\n' + 'application).\n' + '\n' + 'class class list([iterable])\n' + '\n' + ' Lists may be constructed in several ways:\n' + '\n' + ' * Using a pair of square brackets to denote the empty list: ' + '"[]"\n' + '\n' + ' * Using square brackets, separating items with commas: ' + '"[a]",\n' + ' "[a, b, c]"\n' + '\n' + ' * Using a list comprehension: "[x for x in iterable]"\n' + '\n' + ' * Using the type constructor: "list()" or "list(iterable)"\n' + '\n' + ' The constructor builds a list whose items are the same and ' + 'in the\n' + " same order as *iterable*'s items. *iterable* may be either " + 'a\n' + ' sequence, a container that supports iteration, or an ' + 'iterator\n' + ' object. If *iterable* is already a list, a copy is made ' + 'and\n' + ' returned, similar to "iterable[:]". For example, ' + '"list(\'abc\')"\n' + ' returns "[\'a\', \'b\', \'c\']" and "list( (1, 2, 3) )" ' + 'returns "[1, 2,\n' + ' 3]". If no argument is given, the constructor creates a new ' + 'empty\n' + ' list, "[]".\n' + '\n' + ' Many other operations also produce lists, including the ' + '"sorted()"\n' + ' built-in.\n' + '\n' + ' Lists implement all of the *common* and *mutable* sequence\n' + ' operations. Lists also provide the following additional ' + 'method:\n' + '\n' + ' sort(*, key=None, reverse=None)\n' + '\n' + ' This method sorts the list in place, using only "<" ' + 'comparisons\n' + ' between items. Exceptions are not suppressed - if any ' + 'comparison\n' + ' operations fail, the entire sort operation will fail ' + '(and the\n' + ' list will likely be left in a partially modified ' + 'state).\n' + '\n' + ' "sort()" accepts two arguments that can only be passed ' + 'by\n' + ' keyword (*keyword-only arguments*):\n' + '\n' + ' *key* specifies a function of one argument that is used ' + 'to\n' + ' extract a comparison key from each list element (for ' + 'example,\n' + ' "key=str.lower"). The key corresponding to each item in ' + 'the list\n' + ' is calculated once and then used for the entire sorting ' + 'process.\n' + ' The default value of "None" means that list items are ' + 'sorted\n' + ' directly without calculating a separate key value.\n' + '\n' + ' The "functools.cmp_to_key()" utility is available to ' + 'convert a\n' + ' 2.x style *cmp* function to a *key* function.\n' + '\n' + ' *reverse* is a boolean value. If set to "True", then ' + 'the list\n' + ' elements are sorted as if each comparison were ' + 'reversed.\n' + '\n' + ' This method modifies the sequence in place for economy ' + 'of space\n' + ' when sorting a large sequence. To remind users that it ' + 'operates\n' + ' by side effect, it does not return the sorted sequence ' + '(use\n' + ' "sorted()" to explicitly request a new sorted list ' + 'instance).\n' + '\n' + ' The "sort()" method is guaranteed to be stable. A sort ' + 'is\n' + ' stable if it guarantees not to change the relative order ' + 'of\n' + ' elements that compare equal --- this is helpful for ' + 'sorting in\n' + ' multiple passes (for example, sort by department, then ' + 'by salary\n' + ' grade).\n' + '\n' + ' **CPython implementation detail:** While a list is being ' + 'sorted,\n' + ' the effect of attempting to mutate, or even inspect, the ' + 'list is\n' + ' undefined. The C implementation of Python makes the ' + 'list appear\n' + ' empty for the duration, and raises "ValueError" if it ' + 'can detect\n' + ' that the list has been mutated during a sort.\n' + '\n' + '\n' + 'Tuples\n' + '======\n' + '\n' + 'Tuples are immutable sequences, typically used to store ' + 'collections of\n' + 'heterogeneous data (such as the 2-tuples produced by the ' + '"enumerate()"\n' + 'built-in). Tuples are also used for cases where an immutable ' + 'sequence\n' + 'of homogeneous data is needed (such as allowing storage in a ' + '"set" or\n' + '"dict" instance).\n' + '\n' + 'class class tuple([iterable])\n' + '\n' + ' Tuples may be constructed in a number of ways:\n' + '\n' + ' * Using a pair of parentheses to denote the empty tuple: ' + '"()"\n' + '\n' + ' * Using a trailing comma for a singleton tuple: "a," or ' + '"(a,)"\n' + '\n' + ' * Separating items with commas: "a, b, c" or "(a, b, c)"\n' + '\n' + ' * Using the "tuple()" built-in: "tuple()" or ' + '"tuple(iterable)"\n' + '\n' + ' The constructor builds a tuple whose items are the same and ' + 'in the\n' + " same order as *iterable*'s items. *iterable* may be either " + 'a\n' + ' sequence, a container that supports iteration, or an ' + 'iterator\n' + ' object. If *iterable* is already a tuple, it is returned\n' + ' unchanged. For example, "tuple(\'abc\')" returns "(\'a\', ' + '\'b\', \'c\')"\n' + ' and "tuple( [1, 2, 3] )" returns "(1, 2, 3)". If no ' + 'argument is\n' + ' given, the constructor creates a new empty tuple, "()".\n' + '\n' + ' Note that it is actually the comma which makes a tuple, not ' + 'the\n' + ' parentheses. The parentheses are optional, except in the ' + 'empty\n' + ' tuple case, or when they are needed to avoid syntactic ' + 'ambiguity.\n' + ' For example, "f(a, b, c)" is a function call with three ' + 'arguments,\n' + ' while "f((a, b, c))" is a function call with a 3-tuple as ' + 'the sole\n' + ' argument.\n' + '\n' + ' Tuples implement all of the *common* sequence operations.\n' + '\n' + 'For heterogeneous collections of data where access by name is ' + 'clearer\n' + 'than access by index, "collections.namedtuple()" may be a ' + 'more\n' + 'appropriate choice than a simple tuple object.\n' + '\n' + '\n' + 'Ranges\n' + '======\n' + '\n' + 'The "range" type represents an immutable sequence of numbers ' + 'and is\n' + 'commonly used for looping a specific number of times in "for" ' + 'loops.\n' + '\n' + 'class class range(stop)\n' + 'class class range(start, stop[, step])\n' + '\n' + ' The arguments to the range constructor must be integers ' + '(either\n' + ' built-in "int" or any object that implements the ' + '"__index__"\n' + ' special method). If the *step* argument is omitted, it ' + 'defaults to\n' + ' "1". If the *start* argument is omitted, it defaults to ' + '"0". If\n' + ' *step* is zero, "ValueError" is raised.\n' + '\n' + ' For a positive *step*, the contents of a range "r" are ' + 'determined\n' + ' by the formula "r[i] = start + step*i" where "i >= 0" and ' + '"r[i] <\n' + ' stop".\n' + '\n' + ' For a negative *step*, the contents of the range are still\n' + ' determined by the formula "r[i] = start + step*i", but the\n' + ' constraints are "i >= 0" and "r[i] > stop".\n' + '\n' + ' A range object will be empty if "r[0]" does not meet the ' + 'value\n' + ' constraint. Ranges do support negative indices, but these ' + 'are\n' + ' interpreted as indexing from the end of the sequence ' + 'determined by\n' + ' the positive indices.\n' + '\n' + ' Ranges containing absolute values larger than "sys.maxsize" ' + 'are\n' + ' permitted but some features (such as "len()") may raise\n' + ' "OverflowError".\n' + '\n' + ' Range examples:\n' + '\n' + ' >>> list(range(10))\n' + ' [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]\n' + ' >>> list(range(1, 11))\n' + ' [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]\n' + ' >>> list(range(0, 30, 5))\n' + ' [0, 5, 10, 15, 20, 25]\n' + ' >>> list(range(0, 10, 3))\n' + ' [0, 3, 6, 9]\n' + ' >>> list(range(0, -10, -1))\n' + ' [0, -1, -2, -3, -4, -5, -6, -7, -8, -9]\n' + ' >>> list(range(0))\n' + ' []\n' + ' >>> list(range(1, 0))\n' + ' []\n' + '\n' + ' Ranges implement all of the *common* sequence operations ' + 'except\n' + ' concatenation and repetition (due to the fact that range ' + 'objects\n' + ' can only represent sequences that follow a strict pattern ' + 'and\n' + ' repetition and concatenation will usually violate that ' + 'pattern).\n' + '\n' + 'The advantage of the "range" type over a regular "list" or ' + '"tuple" is\n' + 'that a "range" object will always take the same (small) amount ' + 'of\n' + 'memory, no matter the size of the range it represents (as it ' + 'only\n' + 'stores the "start", "stop" and "step" values, calculating ' + 'individual\n' + 'items and subranges as needed).\n' + '\n' + 'Range objects implement the "collections.abc.Sequence" ABC, ' + 'and\n' + 'provide features such as containment tests, element index ' + 'lookup,\n' + 'slicing and support for negative indices (see *Sequence Types ' + '---\n' + 'list, tuple, range*):\n' + '\n' + '>>> r = range(0, 20, 2)\n' + '>>> r\n' + 'range(0, 20, 2)\n' + '>>> 11 in r\n' + 'False\n' + '>>> 10 in r\n' + 'True\n' + '>>> r.index(10)\n' + '5\n' + '>>> r[5]\n' + '10\n' + '>>> r[:5]\n' + 'range(0, 10, 2)\n' + '>>> r[-1]\n' + '18\n' + '\n' + 'Testing range objects for equality with "==" and "!=" compares ' + 'them as\n' + 'sequences. That is, two range objects are considered equal if ' + 'they\n' + 'represent the same sequence of values. (Note that two range ' + 'objects\n' + 'that compare equal might have different "start", "stop" and ' + '"step"\n' + 'attributes, for example "range(0) == range(2, 1, 3)" or ' + '"range(0, 3,\n' + '2) == range(0, 4, 2)".)\n' + '\n' + 'Changed in version 3.2: Implement the Sequence ABC. Support ' + 'slicing\n' + 'and negative indices. Test "int" objects for membership in ' + 'constant\n' + 'time instead of iterating through all items.\n' + '\n' + "Changed in version 3.3: Define '==' and '!=' to compare range " + 'objects\n' + 'based on the sequence of values they define (instead of ' + 'comparing\n' + 'based on object identity).\n' + '\n' + 'New in version 3.3: The "start", "stop" and "step" ' + 'attributes.\n', + 'typesseq-mutable': '\n' + 'Mutable Sequence Types\n' + '**********************\n' + '\n' + 'The operations in the following table are defined on ' + 'mutable sequence\n' + 'types. The "collections.abc.MutableSequence" ABC is ' + 'provided to make\n' + 'it easier to correctly implement these operations on ' + 'custom sequence\n' + 'types.\n' + '\n' + 'In the table *s* is an instance of a mutable sequence ' + 'type, *t* is any\n' + 'iterable object and *x* is an arbitrary object that ' + 'meets any type and\n' + 'value restrictions imposed by *s* (for example, ' + '"bytearray" only\n' + 'accepts integers that meet the value restriction "0 <= ' + 'x <= 255").\n' + '\n' + '+--------------------------------+----------------------------------+-----------------------+\n' + '| Operation | ' + 'Result | ' + 'Notes |\n' + '+================================+==================================+=======================+\n' + '| "s[i] = x" | item *i* of *s* is ' + 'replaced by | |\n' + '| | ' + '*x* ' + '| |\n' + '+--------------------------------+----------------------------------+-----------------------+\n' + '| "s[i:j] = t" | slice of *s* from ' + '*i* to *j* is | |\n' + '| | replaced by the ' + 'contents of the | |\n' + '| | iterable ' + '*t* | |\n' + '+--------------------------------+----------------------------------+-----------------------+\n' + '| "del s[i:j]" | same as "s[i:j] = ' + '[]" | |\n' + '+--------------------------------+----------------------------------+-----------------------+\n' + '| "s[i:j:k] = t" | the elements of ' + '"s[i:j:k]" are | (1) |\n' + '| | replaced by those ' + 'of *t* | |\n' + '+--------------------------------+----------------------------------+-----------------------+\n' + '| "del s[i:j:k]" | removes the ' + 'elements of | |\n' + '| | "s[i:j:k]" from the ' + 'list | |\n' + '+--------------------------------+----------------------------------+-----------------------+\n' + '| "s.append(x)" | appends *x* to the ' + 'end of the | |\n' + '| | sequence (same ' + 'as | |\n' + '| | "s[len(s):len(s)] = ' + '[x]") | |\n' + '+--------------------------------+----------------------------------+-----------------------+\n' + '| "s.clear()" | removes all items ' + 'from "s" (same | (5) |\n' + '| | as "del ' + 's[:]") | |\n' + '+--------------------------------+----------------------------------+-----------------------+\n' + '| "s.copy()" | creates a shallow ' + 'copy of "s" | (5) |\n' + '| | (same as ' + '"s[:]") | |\n' + '+--------------------------------+----------------------------------+-----------------------+\n' + '| "s.extend(t)" | extends *s* with ' + 'the contents of | |\n' + '| | *t* (same as ' + '"s[len(s):len(s)] = | |\n' + '| | ' + 't") ' + '| |\n' + '+--------------------------------+----------------------------------+-----------------------+\n' + '| "s.insert(i, x)" | inserts *x* into ' + '*s* at the | |\n' + '| | index given by *i* ' + '(same as | |\n' + '| | "s[i:i] = ' + '[x]") | |\n' + '+--------------------------------+----------------------------------+-----------------------+\n' + '| "s.pop([i])" | retrieves the item ' + 'at *i* and | (2) |\n' + '| | also removes it ' + 'from *s* | |\n' + '+--------------------------------+----------------------------------+-----------------------+\n' + '| "s.remove(x)" | remove the first ' + 'item from *s* | (3) |\n' + '| | where "s[i] == ' + 'x" | |\n' + '+--------------------------------+----------------------------------+-----------------------+\n' + '| "s.reverse()" | reverses the items ' + 'of *s* in | (4) |\n' + '| | ' + 'place ' + '| |\n' + '+--------------------------------+----------------------------------+-----------------------+\n' + '\n' + 'Notes:\n' + '\n' + '1. *t* must have the same length as the slice it is ' + 'replacing.\n' + '\n' + '2. The optional argument *i* defaults to "-1", so that ' + 'by default\n' + ' the last item is removed and returned.\n' + '\n' + '3. "remove" raises "ValueError" when *x* is not found ' + 'in *s*.\n' + '\n' + '4. The "reverse()" method modifies the sequence in ' + 'place for\n' + ' economy of space when reversing a large sequence. ' + 'To remind users\n' + ' that it operates by side effect, it does not return ' + 'the reversed\n' + ' sequence.\n' + '\n' + '5. "clear()" and "copy()" are included for consistency ' + 'with the\n' + " interfaces of mutable containers that don't support " + 'slicing\n' + ' operations (such as "dict" and "set")\n' + '\n' + ' New in version 3.3: "clear()" and "copy()" ' + 'methods.\n', + 'unary': '\n' + 'Unary arithmetic and bitwise operations\n' + '***************************************\n' + '\n' + 'All unary arithmetic and bitwise operations have the same ' + 'priority:\n' + '\n' + ' u_expr ::= power | "-" u_expr | "+" u_expr | "~" u_expr\n' + '\n' + 'The unary "-" (minus) operator yields the negation of its ' + 'numeric\n' + 'argument.\n' + '\n' + 'The unary "+" (plus) operator yields its numeric argument ' + 'unchanged.\n' + '\n' + 'The unary "~" (invert) operator yields the bitwise inversion of ' + 'its\n' + 'integer argument. The bitwise inversion of "x" is defined as\n' + '"-(x+1)". It only applies to integral numbers.\n' + '\n' + 'In all three cases, if the argument does not have the proper ' + 'type, a\n' + '"TypeError" exception is raised.\n', + 'while': '\n' + 'The "while" statement\n' + '*********************\n' + '\n' + 'The "while" statement is used for repeated execution as long as ' + 'an\n' + 'expression is true:\n' + '\n' + ' while_stmt ::= "while" expression ":" suite\n' + ' ["else" ":" suite]\n' + '\n' + 'This repeatedly tests the expression and, if it is true, executes ' + 'the\n' + 'first suite; if the expression is false (which may be the first ' + 'time\n' + 'it is tested) the suite of the "else" clause, if present, is ' + 'executed\n' + 'and the loop terminates.\n' + '\n' + 'A "break" statement executed in the first suite terminates the ' + 'loop\n' + 'without executing the "else" clause\'s suite. A "continue" ' + 'statement\n' + 'executed in the first suite skips the rest of the suite and goes ' + 'back\n' + 'to testing the expression.\n', + 'with': '\n' + 'The "with" statement\n' + '********************\n' + '\n' + 'The "with" statement is used to wrap the execution of a block ' + 'with\n' + 'methods defined by a context manager (see section *With Statement\n' + 'Context Managers*). This allows common ' + '"try"..."except"..."finally"\n' + 'usage patterns to be encapsulated for convenient reuse.\n' + '\n' + ' with_stmt ::= "with" with_item ("," with_item)* ":" suite\n' + ' with_item ::= expression ["as" target]\n' + '\n' + 'The execution of the "with" statement with one "item" proceeds as\n' + 'follows:\n' + '\n' + '1. The context expression (the expression given in the ' + '"with_item")\n' + ' is evaluated to obtain a context manager.\n' + '\n' + '2. The context manager\'s "__exit__()" is loaded for later use.\n' + '\n' + '3. The context manager\'s "__enter__()" method is invoked.\n' + '\n' + '4. If a target was included in the "with" statement, the return\n' + ' value from "__enter__()" is assigned to it.\n' + '\n' + ' Note: The "with" statement guarantees that if the ' + '"__enter__()"\n' + ' method returns without an error, then "__exit__()" will ' + 'always be\n' + ' called. Thus, if an error occurs during the assignment to ' + 'the\n' + ' target list, it will be treated the same as an error ' + 'occurring\n' + ' within the suite would be. See step 6 below.\n' + '\n' + '5. The suite is executed.\n' + '\n' + '6. The context manager\'s "__exit__()" method is invoked. If an\n' + ' exception caused the suite to be exited, its type, value, and\n' + ' traceback are passed as arguments to "__exit__()". Otherwise, ' + 'three\n' + ' "None" arguments are supplied.\n' + '\n' + ' If the suite was exited due to an exception, and the return ' + 'value\n' + ' from the "__exit__()" method was false, the exception is ' + 'reraised.\n' + ' If the return value was true, the exception is suppressed, and\n' + ' execution continues with the statement following the "with"\n' + ' statement.\n' + '\n' + ' If the suite was exited for any reason other than an exception, ' + 'the\n' + ' return value from "__exit__()" is ignored, and execution ' + 'proceeds\n' + ' at the normal location for the kind of exit that was taken.\n' + '\n' + 'With more than one item, the context managers are processed as if\n' + 'multiple "with" statements were nested:\n' + '\n' + ' with A() as a, B() as b:\n' + ' suite\n' + '\n' + 'is equivalent to\n' + '\n' + ' with A() as a:\n' + ' with B() as b:\n' + ' suite\n' + '\n' + 'Changed in version 3.1: Support for multiple context expressions.\n' + '\n' + 'See also: **PEP 0343** - The "with" statement\n' + '\n' + ' The specification, background, and examples for the Python ' + '"with"\n' + ' statement.\n', + 'yield': '\n' + 'The "yield" statement\n' + '*********************\n' + '\n' + ' yield_stmt ::= yield_expression\n' + '\n' + 'A "yield" statement is semantically equivalent to a *yield\n' + 'expression*. The yield statement can be used to omit the ' + 'parentheses\n' + 'that would otherwise be required in the equivalent yield ' + 'expression\n' + 'statement. For example, the yield statements\n' + '\n' + ' yield \n' + ' yield from \n' + '\n' + 'are equivalent to the yield expression statements\n' + '\n' + ' (yield )\n' + ' (yield from )\n' + '\n' + 'Yield expressions and statements are only used when defining a\n' + '*generator* function, and are only used in the body of the ' + 'generator\n' + 'function. Using yield in a function definition is sufficient to ' + 'cause\n' + 'that definition to create a generator function instead of a ' + 'normal\n' + 'function.\n' + '\n' + 'For full details of "yield" semantics, refer to the *Yield\n' + 'expressions* section.\n'} diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -146,7 +146,7 @@ - Issue #23812: Fix asyncio.Queue.get() to avoid loosing items on cancellation. Patch by Gustavo J. A. M. Carneiro. -- Issue #24791: Fix grammar regression for call syntax: 'g(*a or b)'. +- Issue #24791: Fix grammar regression for call syntax: ``g(*a or b)``. IDLE ---- -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 13 16:46:05 2015 From: python-checkins at python.org (larry.hastings) Date: Sun, 13 Sep 2015 14:46:05 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy41IC0+IDMuNSk6?= =?utf-8?q?_Merge_release_engineering_work_from_Python_3=2E5=2E0=2E?= Message-ID: <20150913144605.15712.59625@psf.io> https://hg.python.org/cpython/rev/f06f376fdd9c changeset: 97985:f06f376fdd9c branch: 3.5 parent: 97977:aa288ad94089 parent: 97984:5a30d334fffc user: Larry Hastings date: Sun Sep 13 15:43:21 2015 +0100 summary: Merge release engineering work from Python 3.5.0. files: .hgtags | 1 + Doc/whatsnew/3.5.rst | 5 +- Include/patchlevel.h | 6 +- Lib/pydoc_data/topics.py | 12958 ++++++++++++++++++++++++- Misc/NEWS | 7 +- README | 72 +- 6 files changed, 12929 insertions(+), 120 deletions(-) diff --git a/.hgtags b/.hgtags --- a/.hgtags +++ b/.hgtags @@ -156,3 +156,4 @@ cc15d736d860303b9da90d43cd32db39bab048df v3.5.0rc2 66ed52375df802f9d0a34480daaa8ce79fc41313 v3.5.0rc3 2d033fedfa7f1e325fd14ccdaa9cb42155da206f v3.5.0rc4 +374f501f4567b7595f2ad7798aa09afa2456bb28 v3.5.0 diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -44,8 +44,11 @@ This saves the maintainer the effort of going through the Mercurial log when researching a change. +Python 3.5 was released on September 13, 2015. + This article explains the new features in Python 3.5, compared to 3.4. -For full details, see the :source:`Misc/NEWS` file. +For full details, see the +`changelog `_. .. seealso:: diff --git a/Include/patchlevel.h b/Include/patchlevel.h --- a/Include/patchlevel.h +++ b/Include/patchlevel.h @@ -19,11 +19,11 @@ #define PY_MAJOR_VERSION 3 #define PY_MINOR_VERSION 5 #define PY_MICRO_VERSION 0 -#define PY_RELEASE_LEVEL PY_RELEASE_LEVEL_GAMMA -#define PY_RELEASE_SERIAL 4 +#define PY_RELEASE_LEVEL PY_RELEASE_LEVEL_FINAL +#define PY_RELEASE_SERIAL 0 /* Version as a string */ -#define PY_VERSION "3.5.0rc4+" +#define PY_VERSION "3.5.0+" /*--end constants--*/ /* Version as a single 4-byte hex number, e.g. 0x010502B2 == 1.5.2b2. diff --git a/Lib/pydoc_data/topics.py b/Lib/pydoc_data/topics.py --- a/Lib/pydoc_data/topics.py +++ b/Lib/pydoc_data/topics.py @@ -1,79 +1,12881 @@ # -*- coding: utf-8 -*- -# Autogenerated by Sphinx on Mon Sep 7 05:10:25 2015 -topics = {'assert': u'\nThe "assert" statement\n**********************\n\nAssert statements are a convenient way to insert debugging assertions\ninto a program:\n\n assert_stmt ::= "assert" expression ["," expression]\n\nThe simple form, "assert expression", is equivalent to\n\n if __debug__:\n if not expression: raise AssertionError\n\nThe extended form, "assert expression1, expression2", is equivalent to\n\n if __debug__:\n if not expression1: raise AssertionError(expression2)\n\nThese equivalences assume that "__debug__" and "AssertionError" refer\nto the built-in variables with those names. In the current\nimplementation, the built-in variable "__debug__" is "True" under\nnormal circumstances, "False" when optimization is requested (command\nline option -O). The current code generator emits no code for an\nassert statement when optimization is requested at compile time. Note\nthat it is unnecessary to include the source code for the expression\nthat failed in the error message; it will be displayed as part of the\nstack trace.\n\nAssignments to "__debug__" are illegal. The value for the built-in\nvariable is determined when the interpreter starts.\n', - 'assignment': u'\nAssignment statements\n*********************\n\nAssignment statements are used to (re)bind names to values and to\nmodify attributes or items of mutable objects:\n\n assignment_stmt ::= (target_list "=")+ (expression_list | yield_expression)\n target_list ::= target ("," target)* [","]\n target ::= identifier\n | "(" target_list ")"\n | "[" target_list "]"\n | attributeref\n | subscription\n | slicing\n | "*" target\n\n(See section *Primaries* for the syntax definitions for\n*attributeref*, *subscription*, and *slicing*.)\n\nAn assignment statement evaluates the expression list (remember that\nthis can be a single expression or a comma-separated list, the latter\nyielding a tuple) and assigns the single resulting object to each of\nthe target lists, from left to right.\n\nAssignment is defined recursively depending on the form of the target\n(list). When a target is part of a mutable object (an attribute\nreference, subscription or slicing), the mutable object must\nultimately perform the assignment and decide about its validity, and\nmay raise an exception if the assignment is unacceptable. The rules\nobserved by various types and the exceptions raised are given with the\ndefinition of the object types (see section *The standard type\nhierarchy*).\n\nAssignment of an object to a target list, optionally enclosed in\nparentheses or square brackets, is recursively defined as follows.\n\n* If the target list is a single target: The object is assigned to\n that target.\n\n* If the target list is a comma-separated list of targets: The\n object must be an iterable with the same number of items as there\n are targets in the target list, and the items are assigned, from\n left to right, to the corresponding targets.\n\n * If the target list contains one target prefixed with an\n asterisk, called a "starred" target: The object must be a sequence\n with at least as many items as there are targets in the target\n list, minus one. The first items of the sequence are assigned,\n from left to right, to the targets before the starred target. The\n final items of the sequence are assigned to the targets after the\n starred target. A list of the remaining items in the sequence is\n then assigned to the starred target (the list can be empty).\n\n * Else: The object must be a sequence with the same number of\n items as there are targets in the target list, and the items are\n assigned, from left to right, to the corresponding targets.\n\nAssignment of an object to a single target is recursively defined as\nfollows.\n\n* If the target is an identifier (name):\n\n * If the name does not occur in a "global" or "nonlocal" statement\n in the current code block: the name is bound to the object in the\n current local namespace.\n\n * Otherwise: the name is bound to the object in the global\n namespace or the outer namespace determined by "nonlocal",\n respectively.\n\n The name is rebound if it was already bound. This may cause the\n reference count for the object previously bound to the name to reach\n zero, causing the object to be deallocated and its destructor (if it\n has one) to be called.\n\n* If the target is a target list enclosed in parentheses or in\n square brackets: The object must be an iterable with the same number\n of items as there are targets in the target list, and its items are\n assigned, from left to right, to the corresponding targets.\n\n* If the target is an attribute reference: The primary expression in\n the reference is evaluated. It should yield an object with\n assignable attributes; if this is not the case, "TypeError" is\n raised. That object is then asked to assign the assigned object to\n the given attribute; if it cannot perform the assignment, it raises\n an exception (usually but not necessarily "AttributeError").\n\n Note: If the object is a class instance and the attribute reference\n occurs on both sides of the assignment operator, the RHS expression,\n "a.x" can access either an instance attribute or (if no instance\n attribute exists) a class attribute. The LHS target "a.x" is always\n set as an instance attribute, creating it if necessary. Thus, the\n two occurrences of "a.x" do not necessarily refer to the same\n attribute: if the RHS expression refers to a class attribute, the\n LHS creates a new instance attribute as the target of the\n assignment:\n\n class Cls:\n x = 3 # class variable\n inst = Cls()\n inst.x = inst.x + 1 # writes inst.x as 4 leaving Cls.x as 3\n\n This description does not necessarily apply to descriptor\n attributes, such as properties created with "property()".\n\n* If the target is a subscription: The primary expression in the\n reference is evaluated. It should yield either a mutable sequence\n object (such as a list) or a mapping object (such as a dictionary).\n Next, the subscript expression is evaluated.\n\n If the primary is a mutable sequence object (such as a list), the\n subscript must yield an integer. If it is negative, the sequence\'s\n length is added to it. The resulting value must be a nonnegative\n integer less than the sequence\'s length, and the sequence is asked\n to assign the assigned object to its item with that index. If the\n index is out of range, "IndexError" is raised (assignment to a\n subscripted sequence cannot add new items to a list).\n\n If the primary is a mapping object (such as a dictionary), the\n subscript must have a type compatible with the mapping\'s key type,\n and the mapping is then asked to create a key/datum pair which maps\n the subscript to the assigned object. This can either replace an\n existing key/value pair with the same key value, or insert a new\n key/value pair (if no key with the same value existed).\n\n For user-defined objects, the "__setitem__()" method is called with\n appropriate arguments.\n\n* If the target is a slicing: The primary expression in the\n reference is evaluated. It should yield a mutable sequence object\n (such as a list). The assigned object should be a sequence object\n of the same type. Next, the lower and upper bound expressions are\n evaluated, insofar they are present; defaults are zero and the\n sequence\'s length. The bounds should evaluate to integers. If\n either bound is negative, the sequence\'s length is added to it. The\n resulting bounds are clipped to lie between zero and the sequence\'s\n length, inclusive. Finally, the sequence object is asked to replace\n the slice with the items of the assigned sequence. The length of\n the slice may be different from the length of the assigned sequence,\n thus changing the length of the target sequence, if the target\n sequence allows it.\n\n**CPython implementation detail:** In the current implementation, the\nsyntax for targets is taken to be the same as for expressions, and\ninvalid syntax is rejected during the code generation phase, causing\nless detailed error messages.\n\nAlthough the definition of assignment implies that overlaps between\nthe left-hand side and the right-hand side are \'simultanenous\' (for\nexample "a, b = b, a" swaps two variables), overlaps *within* the\ncollection of assigned-to variables occur left-to-right, sometimes\nresulting in confusion. For instance, the following program prints\n"[0, 2]":\n\n x = [0, 1]\n i = 0\n i, x[i] = 1, 2 # i is updated, then x[i] is updated\n print(x)\n\nSee also: **PEP 3132** - Extended Iterable Unpacking\n\n The specification for the "*target" feature.\n\n\nAugmented assignment statements\n===============================\n\nAugmented assignment is the combination, in a single statement, of a\nbinary operation and an assignment statement:\n\n augmented_assignment_stmt ::= augtarget augop (expression_list | yield_expression)\n augtarget ::= identifier | attributeref | subscription | slicing\n augop ::= "+=" | "-=" | "*=" | "@=" | "/=" | "//=" | "%=" | "**="\n | ">>=" | "<<=" | "&=" | "^=" | "|="\n\n(See section *Primaries* for the syntax definitions of the last three\nsymbols.)\n\nAn augmented assignment evaluates the target (which, unlike normal\nassignment statements, cannot be an unpacking) and the expression\nlist, performs the binary operation specific to the type of assignment\non the two operands, and assigns the result to the original target.\nThe target is only evaluated once.\n\nAn augmented assignment expression like "x += 1" can be rewritten as\n"x = x + 1" to achieve a similar, but not exactly equal effect. In the\naugmented version, "x" is only evaluated once. Also, when possible,\nthe actual operation is performed *in-place*, meaning that rather than\ncreating a new object and assigning that to the target, the old object\nis modified instead.\n\nUnlike normal assignments, augmented assignments evaluate the left-\nhand side *before* evaluating the right-hand side. For example, "a[i]\n+= f(x)" first looks-up "a[i]", then it evaluates "f(x)" and performs\nthe addition, and lastly, it writes the result back to "a[i]".\n\nWith the exception of assigning to tuples and multiple targets in a\nsingle statement, the assignment done by augmented assignment\nstatements is handled the same way as normal assignments. Similarly,\nwith the exception of the possible *in-place* behavior, the binary\noperation performed by augmented assignment is the same as the normal\nbinary operations.\n\nFor targets which are attribute references, the same *caveat about\nclass and instance attributes* applies as for regular assignments.\n', - 'atom-identifiers': u'\nIdentifiers (Names)\n*******************\n\nAn identifier occurring as an atom is a name. See section\n*Identifiers and keywords* for lexical definition and section *Naming\nand binding* for documentation of naming and binding.\n\nWhen the name is bound to an object, evaluation of the atom yields\nthat object. When a name is not bound, an attempt to evaluate it\nraises a "NameError" exception.\n\n**Private name mangling:** When an identifier that textually occurs in\na class definition begins with two or more underscore characters and\ndoes not end in two or more underscores, it is considered a *private\nname* of that class. Private names are transformed to a longer form\nbefore code is generated for them. The transformation inserts the\nclass name, with leading underscores removed and a single underscore\ninserted, in front of the name. For example, the identifier "__spam"\noccurring in a class named "Ham" will be transformed to "_Ham__spam".\nThis transformation is independent of the syntactical context in which\nthe identifier is used. If the transformed name is extremely long\n(longer than 255 characters), implementation defined truncation may\nhappen. If the class name consists only of underscores, no\ntransformation is done.\n', - 'atom-literals': u"\nLiterals\n********\n\nPython supports string and bytes literals and various numeric\nliterals:\n\n literal ::= stringliteral | bytesliteral\n | integer | floatnumber | imagnumber\n\nEvaluation of a literal yields an object of the given type (string,\nbytes, integer, floating point number, complex number) with the given\nvalue. The value may be approximated in the case of floating point\nand imaginary (complex) literals. See section *Literals* for details.\n\nAll literals correspond to immutable data types, and hence the\nobject's identity is less important than its value. Multiple\nevaluations of literals with the same value (either the same\noccurrence in the program text or a different occurrence) may obtain\nthe same object or a different object with the same value.\n", - 'attribute-access': u'\nCustomizing attribute access\n****************************\n\nThe following methods can be defined to customize the meaning of\nattribute access (use of, assignment to, or deletion of "x.name") for\nclass instances.\n\nobject.__getattr__(self, name)\n\n Called when an attribute lookup has not found the attribute in the\n usual places (i.e. it is not an instance attribute nor is it found\n in the class tree for "self"). "name" is the attribute name. This\n method should return the (computed) attribute value or raise an\n "AttributeError" exception.\n\n Note that if the attribute is found through the normal mechanism,\n "__getattr__()" is not called. (This is an intentional asymmetry\n between "__getattr__()" and "__setattr__()".) This is done both for\n efficiency reasons and because otherwise "__getattr__()" would have\n no way to access other attributes of the instance. Note that at\n least for instance variables, you can fake total control by not\n inserting any values in the instance attribute dictionary (but\n instead inserting them in another object). See the\n "__getattribute__()" method below for a way to actually get total\n control over attribute access.\n\nobject.__getattribute__(self, name)\n\n Called unconditionally to implement attribute accesses for\n instances of the class. If the class also defines "__getattr__()",\n the latter will not be called unless "__getattribute__()" either\n calls it explicitly or raises an "AttributeError". This method\n should return the (computed) attribute value or raise an\n "AttributeError" exception. In order to avoid infinite recursion in\n this method, its implementation should always call the base class\n method with the same name to access any attributes it needs, for\n example, "object.__getattribute__(self, name)".\n\n Note: This method may still be bypassed when looking up special\n methods as the result of implicit invocation via language syntax\n or built-in functions. See *Special method lookup*.\n\nobject.__setattr__(self, name, value)\n\n Called when an attribute assignment is attempted. This is called\n instead of the normal mechanism (i.e. store the value in the\n instance dictionary). *name* is the attribute name, *value* is the\n value to be assigned to it.\n\n If "__setattr__()" wants to assign to an instance attribute, it\n should call the base class method with the same name, for example,\n "object.__setattr__(self, name, value)".\n\nobject.__delattr__(self, name)\n\n Like "__setattr__()" but for attribute deletion instead of\n assignment. This should only be implemented if "del obj.name" is\n meaningful for the object.\n\nobject.__dir__(self)\n\n Called when "dir()" is called on the object. A sequence must be\n returned. "dir()" converts the returned sequence to a list and\n sorts it.\n\n\nImplementing Descriptors\n========================\n\nThe following methods only apply when an instance of the class\ncontaining the method (a so-called *descriptor* class) appears in an\n*owner* class (the descriptor must be in either the owner\'s class\ndictionary or in the class dictionary for one of its parents). In the\nexamples below, "the attribute" refers to the attribute whose name is\nthe key of the property in the owner class\' "__dict__".\n\nobject.__get__(self, instance, owner)\n\n Called to get the attribute of the owner class (class attribute\n access) or of an instance of that class (instance attribute\n access). *owner* is always the owner class, while *instance* is the\n instance that the attribute was accessed through, or "None" when\n the attribute is accessed through the *owner*. This method should\n return the (computed) attribute value or raise an "AttributeError"\n exception.\n\nobject.__set__(self, instance, value)\n\n Called to set the attribute on an instance *instance* of the owner\n class to a new value, *value*.\n\nobject.__delete__(self, instance)\n\n Called to delete the attribute on an instance *instance* of the\n owner class.\n\nThe attribute "__objclass__" is interpreted by the "inspect" module as\nspecifying the class where this object was defined (setting this\nappropriately can assist in runtime introspection of dynamic class\nattributes). For callables, it may indicate that an instance of the\ngiven type (or a subclass) is expected or required as the first\npositional argument (for example, CPython sets this attribute for\nunbound methods that are implemented in C).\n\n\nInvoking Descriptors\n====================\n\nIn general, a descriptor is an object attribute with "binding\nbehavior", one whose attribute access has been overridden by methods\nin the descriptor protocol: "__get__()", "__set__()", and\n"__delete__()". If any of those methods are defined for an object, it\nis said to be a descriptor.\n\nThe default behavior for attribute access is to get, set, or delete\nthe attribute from an object\'s dictionary. For instance, "a.x" has a\nlookup chain starting with "a.__dict__[\'x\']", then\n"type(a).__dict__[\'x\']", and continuing through the base classes of\n"type(a)" excluding metaclasses.\n\nHowever, if the looked-up value is an object defining one of the\ndescriptor methods, then Python may override the default behavior and\ninvoke the descriptor method instead. Where this occurs in the\nprecedence chain depends on which descriptor methods were defined and\nhow they were called.\n\nThe starting point for descriptor invocation is a binding, "a.x". How\nthe arguments are assembled depends on "a":\n\nDirect Call\n The simplest and least common call is when user code directly\n invokes a descriptor method: "x.__get__(a)".\n\nInstance Binding\n If binding to an object instance, "a.x" is transformed into the\n call: "type(a).__dict__[\'x\'].__get__(a, type(a))".\n\nClass Binding\n If binding to a class, "A.x" is transformed into the call:\n "A.__dict__[\'x\'].__get__(None, A)".\n\nSuper Binding\n If "a" is an instance of "super", then the binding "super(B,\n obj).m()" searches "obj.__class__.__mro__" for the base class "A"\n immediately preceding "B" and then invokes the descriptor with the\n call: "A.__dict__[\'m\'].__get__(obj, obj.__class__)".\n\nFor instance bindings, the precedence of descriptor invocation depends\non the which descriptor methods are defined. A descriptor can define\nany combination of "__get__()", "__set__()" and "__delete__()". If it\ndoes not define "__get__()", then accessing the attribute will return\nthe descriptor object itself unless there is a value in the object\'s\ninstance dictionary. If the descriptor defines "__set__()" and/or\n"__delete__()", it is a data descriptor; if it defines neither, it is\na non-data descriptor. Normally, data descriptors define both\n"__get__()" and "__set__()", while non-data descriptors have just the\n"__get__()" method. Data descriptors with "__set__()" and "__get__()"\ndefined always override a redefinition in an instance dictionary. In\ncontrast, non-data descriptors can be overridden by instances.\n\nPython methods (including "staticmethod()" and "classmethod()") are\nimplemented as non-data descriptors. Accordingly, instances can\nredefine and override methods. This allows individual instances to\nacquire behaviors that differ from other instances of the same class.\n\nThe "property()" function is implemented as a data descriptor.\nAccordingly, instances cannot override the behavior of a property.\n\n\n__slots__\n=========\n\nBy default, instances of classes have a dictionary for attribute\nstorage. This wastes space for objects having very few instance\nvariables. The space consumption can become acute when creating large\nnumbers of instances.\n\nThe default can be overridden by defining *__slots__* in a class\ndefinition. The *__slots__* declaration takes a sequence of instance\nvariables and reserves just enough space in each instance to hold a\nvalue for each variable. Space is saved because *__dict__* is not\ncreated for each instance.\n\nobject.__slots__\n\n This class variable can be assigned a string, iterable, or sequence\n of strings with variable names used by instances. *__slots__*\n reserves space for the declared variables and prevents the\n automatic creation of *__dict__* and *__weakref__* for each\n instance.\n\n\nNotes on using *__slots__*\n--------------------------\n\n* When inheriting from a class without *__slots__*, the *__dict__*\n attribute of that class will always be accessible, so a *__slots__*\n definition in the subclass is meaningless.\n\n* Without a *__dict__* variable, instances cannot be assigned new\n variables not listed in the *__slots__* definition. Attempts to\n assign to an unlisted variable name raises "AttributeError". If\n dynamic assignment of new variables is desired, then add\n "\'__dict__\'" to the sequence of strings in the *__slots__*\n declaration.\n\n* Without a *__weakref__* variable for each instance, classes\n defining *__slots__* do not support weak references to its\n instances. If weak reference support is needed, then add\n "\'__weakref__\'" to the sequence of strings in the *__slots__*\n declaration.\n\n* *__slots__* are implemented at the class level by creating\n descriptors (*Implementing Descriptors*) for each variable name. As\n a result, class attributes cannot be used to set default values for\n instance variables defined by *__slots__*; otherwise, the class\n attribute would overwrite the descriptor assignment.\n\n* The action of a *__slots__* declaration is limited to the class\n where it is defined. As a result, subclasses will have a *__dict__*\n unless they also define *__slots__* (which must only contain names\n of any *additional* slots).\n\n* If a class defines a slot also defined in a base class, the\n instance variable defined by the base class slot is inaccessible\n (except by retrieving its descriptor directly from the base class).\n This renders the meaning of the program undefined. In the future, a\n check may be added to prevent this.\n\n* Nonempty *__slots__* does not work for classes derived from\n "variable-length" built-in types such as "int", "bytes" and "tuple".\n\n* Any non-string iterable may be assigned to *__slots__*. Mappings\n may also be used; however, in the future, special meaning may be\n assigned to the values corresponding to each key.\n\n* *__class__* assignment works only if both classes have the same\n *__slots__*.\n', - 'attribute-references': u'\nAttribute references\n********************\n\nAn attribute reference is a primary followed by a period and a name:\n\n attributeref ::= primary "." identifier\n\nThe primary must evaluate to an object of a type that supports\nattribute references, which most objects do. This object is then\nasked to produce the attribute whose name is the identifier. This\nproduction can be customized by overriding the "__getattr__()" method.\nIf this attribute is not available, the exception "AttributeError" is\nraised. Otherwise, the type and value of the object produced is\ndetermined by the object. Multiple evaluations of the same attribute\nreference may yield different objects.\n', - 'augassign': u'\nAugmented assignment statements\n*******************************\n\nAugmented assignment is the combination, in a single statement, of a\nbinary operation and an assignment statement:\n\n augmented_assignment_stmt ::= augtarget augop (expression_list | yield_expression)\n augtarget ::= identifier | attributeref | subscription | slicing\n augop ::= "+=" | "-=" | "*=" | "@=" | "/=" | "//=" | "%=" | "**="\n | ">>=" | "<<=" | "&=" | "^=" | "|="\n\n(See section *Primaries* for the syntax definitions of the last three\nsymbols.)\n\nAn augmented assignment evaluates the target (which, unlike normal\nassignment statements, cannot be an unpacking) and the expression\nlist, performs the binary operation specific to the type of assignment\non the two operands, and assigns the result to the original target.\nThe target is only evaluated once.\n\nAn augmented assignment expression like "x += 1" can be rewritten as\n"x = x + 1" to achieve a similar, but not exactly equal effect. In the\naugmented version, "x" is only evaluated once. Also, when possible,\nthe actual operation is performed *in-place*, meaning that rather than\ncreating a new object and assigning that to the target, the old object\nis modified instead.\n\nUnlike normal assignments, augmented assignments evaluate the left-\nhand side *before* evaluating the right-hand side. For example, "a[i]\n+= f(x)" first looks-up "a[i]", then it evaluates "f(x)" and performs\nthe addition, and lastly, it writes the result back to "a[i]".\n\nWith the exception of assigning to tuples and multiple targets in a\nsingle statement, the assignment done by augmented assignment\nstatements is handled the same way as normal assignments. Similarly,\nwith the exception of the possible *in-place* behavior, the binary\noperation performed by augmented assignment is the same as the normal\nbinary operations.\n\nFor targets which are attribute references, the same *caveat about\nclass and instance attributes* applies as for regular assignments.\n', - 'binary': u'\nBinary arithmetic operations\n****************************\n\nThe binary arithmetic operations have the conventional priority\nlevels. Note that some of these operations also apply to certain non-\nnumeric types. Apart from the power operator, there are only two\nlevels, one for multiplicative operators and one for additive\noperators:\n\n m_expr ::= u_expr | m_expr "*" u_expr | m_expr "@" m_expr |\n m_expr "//" u_expr| m_expr "/" u_expr |\n m_expr "%" u_expr\n a_expr ::= m_expr | a_expr "+" m_expr | a_expr "-" m_expr\n\nThe "*" (multiplication) operator yields the product of its arguments.\nThe arguments must either both be numbers, or one argument must be an\ninteger and the other must be a sequence. In the former case, the\nnumbers are converted to a common type and then multiplied together.\nIn the latter case, sequence repetition is performed; a negative\nrepetition factor yields an empty sequence.\n\nThe "@" (at) operator is intended to be used for matrix\nmultiplication. No builtin Python types implement this operator.\n\nNew in version 3.5.\n\nThe "/" (division) and "//" (floor division) operators yield the\nquotient of their arguments. The numeric arguments are first\nconverted to a common type. Division of integers yields a float, while\nfloor division of integers results in an integer; the result is that\nof mathematical division with the \'floor\' function applied to the\nresult. Division by zero raises the "ZeroDivisionError" exception.\n\nThe "%" (modulo) operator yields the remainder from the division of\nthe first argument by the second. The numeric arguments are first\nconverted to a common type. A zero right argument raises the\n"ZeroDivisionError" exception. The arguments may be floating point\nnumbers, e.g., "3.14%0.7" equals "0.34" (since "3.14" equals "4*0.7 +\n0.34".) The modulo operator always yields a result with the same sign\nas its second operand (or zero); the absolute value of the result is\nstrictly smaller than the absolute value of the second operand [1].\n\nThe floor division and modulo operators are connected by the following\nidentity: "x == (x//y)*y + (x%y)". Floor division and modulo are also\nconnected with the built-in function "divmod()": "divmod(x, y) ==\n(x//y, x%y)". [2].\n\nIn addition to performing the modulo operation on numbers, the "%"\noperator is also overloaded by string objects to perform old-style\nstring formatting (also known as interpolation). The syntax for\nstring formatting is described in the Python Library Reference,\nsection *printf-style String Formatting*.\n\nThe floor division operator, the modulo operator, and the "divmod()"\nfunction are not defined for complex numbers. Instead, convert to a\nfloating point number using the "abs()" function if appropriate.\n\nThe "+" (addition) operator yields the sum of its arguments. The\narguments must either both be numbers or both be sequences of the same\ntype. In the former case, the numbers are converted to a common type\nand then added together. In the latter case, the sequences are\nconcatenated.\n\nThe "-" (subtraction) operator yields the difference of its arguments.\nThe numeric arguments are first converted to a common type.\n', - 'bitwise': u'\nBinary bitwise operations\n*************************\n\nEach of the three bitwise operations has a different priority level:\n\n and_expr ::= shift_expr | and_expr "&" shift_expr\n xor_expr ::= and_expr | xor_expr "^" and_expr\n or_expr ::= xor_expr | or_expr "|" xor_expr\n\nThe "&" operator yields the bitwise AND of its arguments, which must\nbe integers.\n\nThe "^" operator yields the bitwise XOR (exclusive OR) of its\narguments, which must be integers.\n\nThe "|" operator yields the bitwise (inclusive) OR of its arguments,\nwhich must be integers.\n', - 'bltin-code-objects': u'\nCode Objects\n************\n\nCode objects are used by the implementation to represent "pseudo-\ncompiled" executable Python code such as a function body. They differ\nfrom function objects because they don\'t contain a reference to their\nglobal execution environment. Code objects are returned by the built-\nin "compile()" function and can be extracted from function objects\nthrough their "__code__" attribute. See also the "code" module.\n\nA code object can be executed or evaluated by passing it (instead of a\nsource string) to the "exec()" or "eval()" built-in functions.\n\nSee *The standard type hierarchy* for more information.\n', - 'bltin-ellipsis-object': u'\nThe Ellipsis Object\n*******************\n\nThis object is commonly used by slicing (see *Slicings*). It supports\nno special operations. There is exactly one ellipsis object, named\n"Ellipsis" (a built-in name). "type(Ellipsis)()" produces the\n"Ellipsis" singleton.\n\nIt is written as "Ellipsis" or "...".\n', - 'bltin-null-object': u'\nThe Null Object\n***************\n\nThis object is returned by functions that don\'t explicitly return a\nvalue. It supports no special operations. There is exactly one null\nobject, named "None" (a built-in name). "type(None)()" produces the\nsame singleton.\n\nIt is written as "None".\n', - 'bltin-type-objects': u'\nType Objects\n************\n\nType objects represent the various object types. An object\'s type is\naccessed by the built-in function "type()". There are no special\noperations on types. The standard module "types" defines names for\nall standard built-in types.\n\nTypes are written like this: "".\n', - 'booleans': u'\nBoolean operations\n******************\n\n or_test ::= and_test | or_test "or" and_test\n and_test ::= not_test | and_test "and" not_test\n not_test ::= comparison | "not" not_test\n\nIn the context of Boolean operations, and also when expressions are\nused by control flow statements, the following values are interpreted\nas false: "False", "None", numeric zero of all types, and empty\nstrings and containers (including strings, tuples, lists,\ndictionaries, sets and frozensets). All other values are interpreted\nas true. User-defined objects can customize their truth value by\nproviding a "__bool__()" method.\n\nThe operator "not" yields "True" if its argument is false, "False"\notherwise.\n\nThe expression "x and y" first evaluates *x*; if *x* is false, its\nvalue is returned; otherwise, *y* is evaluated and the resulting value\nis returned.\n\nThe expression "x or y" first evaluates *x*; if *x* is true, its value\nis returned; otherwise, *y* is evaluated and the resulting value is\nreturned.\n\n(Note that neither "and" nor "or" restrict the value and type they\nreturn to "False" and "True", but rather return the last evaluated\nargument. This is sometimes useful, e.g., if "s" is a string that\nshould be replaced by a default value if it is empty, the expression\n"s or \'foo\'" yields the desired value. Because "not" has to create a\nnew value, it returns a boolean value regardless of the type of its\nargument (for example, "not \'foo\'" produces "False" rather than "\'\'".)\n', - 'break': u'\nThe "break" statement\n*********************\n\n break_stmt ::= "break"\n\n"break" may only occur syntactically nested in a "for" or "while"\nloop, but not nested in a function or class definition within that\nloop.\n\nIt terminates the nearest enclosing loop, skipping the optional "else"\nclause if the loop has one.\n\nIf a "for" loop is terminated by "break", the loop control target\nkeeps its current value.\n\nWhen "break" passes control out of a "try" statement with a "finally"\nclause, that "finally" clause is executed before really leaving the\nloop.\n', - 'callable-types': u'\nEmulating callable objects\n**************************\n\nobject.__call__(self[, args...])\n\n Called when the instance is "called" as a function; if this method\n is defined, "x(arg1, arg2, ...)" is a shorthand for\n "x.__call__(arg1, arg2, ...)".\n', - 'calls': u'\nCalls\n*****\n\nA call calls a callable object (e.g., a *function*) with a possibly\nempty series of *arguments*:\n\n call ::= primary "(" [argument_list [","] | comprehension] ")"\n argument_list ::= positional_arguments ["," keyword_arguments]\n ["," "*" expression] ["," keyword_arguments]\n ["," "**" expression]\n | keyword_arguments ["," "*" expression]\n ["," keyword_arguments] ["," "**" expression]\n | "*" expression ["," keyword_arguments] ["," "**" expression]\n | "**" expression\n positional_arguments ::= expression ("," expression)*\n keyword_arguments ::= keyword_item ("," keyword_item)*\n keyword_item ::= identifier "=" expression\n\nAn optional trailing comma may be present after the positional and\nkeyword arguments but does not affect the semantics.\n\nThe primary must evaluate to a callable object (user-defined\nfunctions, built-in functions, methods of built-in objects, class\nobjects, methods of class instances, and all objects having a\n"__call__()" method are callable). All argument expressions are\nevaluated before the call is attempted. Please refer to section\n*Function definitions* for the syntax of formal *parameter* lists.\n\nIf keyword arguments are present, they are first converted to\npositional arguments, as follows. First, a list of unfilled slots is\ncreated for the formal parameters. If there are N positional\narguments, they are placed in the first N slots. Next, for each\nkeyword argument, the identifier is used to determine the\ncorresponding slot (if the identifier is the same as the first formal\nparameter name, the first slot is used, and so on). If the slot is\nalready filled, a "TypeError" exception is raised. Otherwise, the\nvalue of the argument is placed in the slot, filling it (even if the\nexpression is "None", it fills the slot). When all arguments have\nbeen processed, the slots that are still unfilled are filled with the\ncorresponding default value from the function definition. (Default\nvalues are calculated, once, when the function is defined; thus, a\nmutable object such as a list or dictionary used as default value will\nbe shared by all calls that don\'t specify an argument value for the\ncorresponding slot; this should usually be avoided.) If there are any\nunfilled slots for which no default value is specified, a "TypeError"\nexception is raised. Otherwise, the list of filled slots is used as\nthe argument list for the call.\n\n**CPython implementation detail:** An implementation may provide\nbuilt-in functions whose positional parameters do not have names, even\nif they are \'named\' for the purpose of documentation, and which\ntherefore cannot be supplied by keyword. In CPython, this is the case\nfor functions implemented in C that use "PyArg_ParseTuple()" to parse\ntheir arguments.\n\nIf there are more positional arguments than there are formal parameter\nslots, a "TypeError" exception is raised, unless a formal parameter\nusing the syntax "*identifier" is present; in this case, that formal\nparameter receives a tuple containing the excess positional arguments\n(or an empty tuple if there were no excess positional arguments).\n\nIf any keyword argument does not correspond to a formal parameter\nname, a "TypeError" exception is raised, unless a formal parameter\nusing the syntax "**identifier" is present; in this case, that formal\nparameter receives a dictionary containing the excess keyword\narguments (using the keywords as keys and the argument values as\ncorresponding values), or a (new) empty dictionary if there were no\nexcess keyword arguments.\n\nIf the syntax "*expression" appears in the function call, "expression"\nmust evaluate to an iterable. Elements from this iterable are treated\nas if they were additional positional arguments; if there are\npositional arguments *x1*, ..., *xN*, and "expression" evaluates to a\nsequence *y1*, ..., *yM*, this is equivalent to a call with M+N\npositional arguments *x1*, ..., *xN*, *y1*, ..., *yM*.\n\nA consequence of this is that although the "*expression" syntax may\nappear *after* some keyword arguments, it is processed *before* the\nkeyword arguments (and the "**expression" argument, if any -- see\nbelow). So:\n\n >>> def f(a, b):\n ... print(a, b)\n ...\n >>> f(b=1, *(2,))\n 2 1\n >>> f(a=1, *(2,))\n Traceback (most recent call last):\n File "", line 1, in ?\n TypeError: f() got multiple values for keyword argument \'a\'\n >>> f(1, *(2,))\n 1 2\n\nIt is unusual for both keyword arguments and the "*expression" syntax\nto be used in the same call, so in practice this confusion does not\narise.\n\nIf the syntax "**expression" appears in the function call,\n"expression" must evaluate to a mapping, the contents of which are\ntreated as additional keyword arguments. In the case of a keyword\nappearing in both "expression" and as an explicit keyword argument, a\n"TypeError" exception is raised.\n\nFormal parameters using the syntax "*identifier" or "**identifier"\ncannot be used as positional argument slots or as keyword argument\nnames.\n\nA call always returns some value, possibly "None", unless it raises an\nexception. How this value is computed depends on the type of the\ncallable object.\n\nIf it is---\n\na user-defined function:\n The code block for the function is executed, passing it the\n argument list. The first thing the code block will do is bind the\n formal parameters to the arguments; this is described in section\n *Function definitions*. When the code block executes a "return"\n statement, this specifies the return value of the function call.\n\na built-in function or method:\n The result is up to the interpreter; see *Built-in Functions* for\n the descriptions of built-in functions and methods.\n\na class object:\n A new instance of that class is returned.\n\na class instance method:\n The corresponding user-defined function is called, with an argument\n list that is one longer than the argument list of the call: the\n instance becomes the first argument.\n\na class instance:\n The class must define a "__call__()" method; the effect is then the\n same as if that method was called.\n', - 'class': u'\nClass definitions\n*****************\n\nA class definition defines a class object (see section *The standard\ntype hierarchy*):\n\n classdef ::= [decorators] "class" classname [inheritance] ":" suite\n inheritance ::= "(" [parameter_list] ")"\n classname ::= identifier\n\nA class definition is an executable statement. The inheritance list\nusually gives a list of base classes (see *Customizing class creation*\nfor more advanced uses), so each item in the list should evaluate to a\nclass object which allows subclassing. Classes without an inheritance\nlist inherit, by default, from the base class "object"; hence,\n\n class Foo:\n pass\n\nis equivalent to\n\n class Foo(object):\n pass\n\nThe class\'s suite is then executed in a new execution frame (see\n*Naming and binding*), using a newly created local namespace and the\noriginal global namespace. (Usually, the suite contains mostly\nfunction definitions.) When the class\'s suite finishes execution, its\nexecution frame is discarded but its local namespace is saved. [4] A\nclass object is then created using the inheritance list for the base\nclasses and the saved local namespace for the attribute dictionary.\nThe class name is bound to this class object in the original local\nnamespace.\n\nClass creation can be customized heavily using *metaclasses*.\n\nClasses can also be decorated: just like when decorating functions,\n\n @f1(arg)\n @f2\n class Foo: pass\n\nis equivalent to\n\n class Foo: pass\n Foo = f1(arg)(f2(Foo))\n\nThe evaluation rules for the decorator expressions are the same as for\nfunction decorators. The result must be a class object, which is then\nbound to the class name.\n\n**Programmer\'s note:** Variables defined in the class definition are\nclass attributes; they are shared by instances. Instance attributes\ncan be set in a method with "self.name = value". Both class and\ninstance attributes are accessible through the notation ""self.name"",\nand an instance attribute hides a class attribute with the same name\nwhen accessed in this way. Class attributes can be used as defaults\nfor instance attributes, but using mutable values there can lead to\nunexpected results. *Descriptors* can be used to create instance\nvariables with different implementation details.\n\nSee also: **PEP 3115** - Metaclasses in Python 3 **PEP 3129** -\n Class Decorators\n', - 'comparisons': u'\nComparisons\n***********\n\nUnlike C, all comparison operations in Python have the same priority,\nwhich is lower than that of any arithmetic, shifting or bitwise\noperation. Also unlike C, expressions like "a < b < c" have the\ninterpretation that is conventional in mathematics:\n\n comparison ::= or_expr ( comp_operator or_expr )*\n comp_operator ::= "<" | ">" | "==" | ">=" | "<=" | "!="\n | "is" ["not"] | ["not"] "in"\n\nComparisons yield boolean values: "True" or "False".\n\nComparisons can be chained arbitrarily, e.g., "x < y <= z" is\nequivalent to "x < y and y <= z", except that "y" is evaluated only\nonce (but in both cases "z" is not evaluated at all when "x < y" is\nfound to be false).\n\nFormally, if *a*, *b*, *c*, ..., *y*, *z* are expressions and *op1*,\n*op2*, ..., *opN* are comparison operators, then "a op1 b op2 c ... y\nopN z" is equivalent to "a op1 b and b op2 c and ... y opN z", except\nthat each expression is evaluated at most once.\n\nNote that "a op1 b op2 c" doesn\'t imply any kind of comparison between\n*a* and *c*, so that, e.g., "x < y > z" is perfectly legal (though\nperhaps not pretty).\n\nThe operators "<", ">", "==", ">=", "<=", and "!=" compare the values\nof two objects. The objects need not have the same type. If both are\nnumbers, they are converted to a common type. Otherwise, the "==" and\n"!=" operators *always* consider objects of different types to be\nunequal, while the "<", ">", ">=" and "<=" operators raise a\n"TypeError" when comparing objects of different types that do not\nimplement these operators for the given pair of types. You can\ncontrol comparison behavior of objects of non-built-in types by\ndefining rich comparison methods like "__gt__()", described in section\n*Basic customization*.\n\nComparison of objects of the same type depends on the type:\n\n* Numbers are compared arithmetically.\n\n* The values "float(\'NaN\')" and "Decimal(\'NaN\')" are special. They\n are identical to themselves, "x is x" but are not equal to\n themselves, "x != x". Additionally, comparing any value to a\n not-a-number value will return "False". For example, both "3 <\n float(\'NaN\')" and "float(\'NaN\') < 3" will return "False".\n\n* Bytes objects are compared lexicographically using the numeric\n values of their elements.\n\n* Strings are compared lexicographically using the numeric\n equivalents (the result of the built-in function "ord()") of their\n characters. [3] String and bytes object can\'t be compared!\n\n* Tuples and lists are compared lexicographically using comparison\n of corresponding elements. This means that to compare equal, each\n element must compare equal and the two sequences must be of the same\n type and have the same length.\n\n If not equal, the sequences are ordered the same as their first\n differing elements. For example, "[1,2,x] <= [1,2,y]" has the same\n value as "x <= y". If the corresponding element does not exist, the\n shorter sequence is ordered first (for example, "[1,2] < [1,2,3]").\n\n* Mappings (dictionaries) compare equal if and only if they have the\n same "(key, value)" pairs. Order comparisons "(\'<\', \'<=\', \'>=\',\n \'>\')" raise "TypeError".\n\n* Sets and frozensets define comparison operators to mean subset and\n superset tests. Those relations do not define total orderings (the\n two sets "{1,2}" and "{2,3}" are not equal, nor subsets of one\n another, nor supersets of one another). Accordingly, sets are not\n appropriate arguments for functions which depend on total ordering.\n For example, "min()", "max()", and "sorted()" produce undefined\n results given a list of sets as inputs.\n\n* Most other objects of built-in types compare unequal unless they\n are the same object; the choice whether one object is considered\n smaller or larger than another one is made arbitrarily but\n consistently within one execution of a program.\n\nComparison of objects of differing types depends on whether either of\nthe types provide explicit support for the comparison. Most numeric\ntypes can be compared with one another. When cross-type comparison is\nnot supported, the comparison method returns "NotImplemented".\n\nThe operators "in" and "not in" test for membership. "x in s"\nevaluates to true if *x* is a member of *s*, and false otherwise. "x\nnot in s" returns the negation of "x in s". All built-in sequences\nand set types support this as well as dictionary, for which "in" tests\nwhether the dictionary has a given key. For container types such as\nlist, tuple, set, frozenset, dict, or collections.deque, the\nexpression "x in y" is equivalent to "any(x is e or x == e for e in\ny)".\n\nFor the string and bytes types, "x in y" is true if and only if *x* is\na substring of *y*. An equivalent test is "y.find(x) != -1". Empty\nstrings are always considered to be a substring of any other string,\nso """ in "abc"" will return "True".\n\nFor user-defined classes which define the "__contains__()" method, "x\nin y" is true if and only if "y.__contains__(x)" is true.\n\nFor user-defined classes which do not define "__contains__()" but do\ndefine "__iter__()", "x in y" is true if some value "z" with "x == z"\nis produced while iterating over "y". If an exception is raised\nduring the iteration, it is as if "in" raised that exception.\n\nLastly, the old-style iteration protocol is tried: if a class defines\n"__getitem__()", "x in y" is true if and only if there is a non-\nnegative integer index *i* such that "x == y[i]", and all lower\ninteger indices do not raise "IndexError" exception. (If any other\nexception is raised, it is as if "in" raised that exception).\n\nThe operator "not in" is defined to have the inverse true value of\n"in".\n\nThe operators "is" and "is not" test for object identity: "x is y" is\ntrue if and only if *x* and *y* are the same object. "x is not y"\nyields the inverse truth value. [4]\n', - 'compound': u'\nCompound statements\n*******************\n\nCompound statements contain (groups of) other statements; they affect\nor control the execution of those other statements in some way. In\ngeneral, compound statements span multiple lines, although in simple\nincarnations a whole compound statement may be contained in one line.\n\nThe "if", "while" and "for" statements implement traditional control\nflow constructs. "try" specifies exception handlers and/or cleanup\ncode for a group of statements, while the "with" statement allows the\nexecution of initialization and finalization code around a block of\ncode. Function and class definitions are also syntactically compound\nstatements.\n\nA compound statement consists of one or more \'clauses.\' A clause\nconsists of a header and a \'suite.\' The clause headers of a\nparticular compound statement are all at the same indentation level.\nEach clause header begins with a uniquely identifying keyword and ends\nwith a colon. A suite is a group of statements controlled by a\nclause. A suite can be one or more semicolon-separated simple\nstatements on the same line as the header, following the header\'s\ncolon, or it can be one or more indented statements on subsequent\nlines. Only the latter form of a suite can contain nested compound\nstatements; the following is illegal, mostly because it wouldn\'t be\nclear to which "if" clause a following "else" clause would belong:\n\n if test1: if test2: print(x)\n\nAlso note that the semicolon binds tighter than the colon in this\ncontext, so that in the following example, either all or none of the\n"print()" calls are executed:\n\n if x < y < z: print(x); print(y); print(z)\n\nSummarizing:\n\n compound_stmt ::= if_stmt\n | while_stmt\n | for_stmt\n | try_stmt\n | with_stmt\n | funcdef\n | classdef\n | async_with_stmt\n | async_for_stmt\n | async_funcdef\n suite ::= stmt_list NEWLINE | NEWLINE INDENT statement+ DEDENT\n statement ::= stmt_list NEWLINE | compound_stmt\n stmt_list ::= simple_stmt (";" simple_stmt)* [";"]\n\nNote that statements always end in a "NEWLINE" possibly followed by a\n"DEDENT". Also note that optional continuation clauses always begin\nwith a keyword that cannot start a statement, thus there are no\nambiguities (the \'dangling "else"\' problem is solved in Python by\nrequiring nested "if" statements to be indented).\n\nThe formatting of the grammar rules in the following sections places\neach clause on a separate line for clarity.\n\n\nThe "if" statement\n==================\n\nThe "if" statement is used for conditional execution:\n\n if_stmt ::= "if" expression ":" suite\n ( "elif" expression ":" suite )*\n ["else" ":" suite]\n\nIt selects exactly one of the suites by evaluating the expressions one\nby one until one is found to be true (see section *Boolean operations*\nfor the definition of true and false); then that suite is executed\n(and no other part of the "if" statement is executed or evaluated).\nIf all expressions are false, the suite of the "else" clause, if\npresent, is executed.\n\n\nThe "while" statement\n=====================\n\nThe "while" statement is used for repeated execution as long as an\nexpression is true:\n\n while_stmt ::= "while" expression ":" suite\n ["else" ":" suite]\n\nThis repeatedly tests the expression and, if it is true, executes the\nfirst suite; if the expression is false (which may be the first time\nit is tested) the suite of the "else" clause, if present, is executed\nand the loop terminates.\n\nA "break" statement executed in the first suite terminates the loop\nwithout executing the "else" clause\'s suite. A "continue" statement\nexecuted in the first suite skips the rest of the suite and goes back\nto testing the expression.\n\n\nThe "for" statement\n===================\n\nThe "for" statement is used to iterate over the elements of a sequence\n(such as a string, tuple or list) or other iterable object:\n\n for_stmt ::= "for" target_list "in" expression_list ":" suite\n ["else" ":" suite]\n\nThe expression list is evaluated once; it should yield an iterable\nobject. An iterator is created for the result of the\n"expression_list". The suite is then executed once for each item\nprovided by the iterator, in the order returned by the iterator. Each\nitem in turn is assigned to the target list using the standard rules\nfor assignments (see *Assignment statements*), and then the suite is\nexecuted. When the items are exhausted (which is immediately when the\nsequence is empty or an iterator raises a "StopIteration" exception),\nthe suite in the "else" clause, if present, is executed, and the loop\nterminates.\n\nA "break" statement executed in the first suite terminates the loop\nwithout executing the "else" clause\'s suite. A "continue" statement\nexecuted in the first suite skips the rest of the suite and continues\nwith the next item, or with the "else" clause if there is no next\nitem.\n\nThe for-loop makes assignments to the variables(s) in the target list.\nThis overwrites all previous assignments to those variables including\nthose made in the suite of the for-loop:\n\n for i in range(10):\n print(i)\n i = 5 # this will not affect the for-loop\n # because i will be overwritten with the next\n # index in the range\n\nNames in the target list are not deleted when the loop is finished,\nbut if the sequence is empty, they will not have been assigned to at\nall by the loop. Hint: the built-in function "range()" returns an\niterator of integers suitable to emulate the effect of Pascal\'s "for i\n:= a to b do"; e.g., "list(range(3))" returns the list "[0, 1, 2]".\n\nNote: There is a subtlety when the sequence is being modified by the\n loop (this can only occur for mutable sequences, i.e. lists). An\n internal counter is used to keep track of which item is used next,\n and this is incremented on each iteration. When this counter has\n reached the length of the sequence the loop terminates. This means\n that if the suite deletes the current (or a previous) item from the\n sequence, the next item will be skipped (since it gets the index of\n the current item which has already been treated). Likewise, if the\n suite inserts an item in the sequence before the current item, the\n current item will be treated again the next time through the loop.\n This can lead to nasty bugs that can be avoided by making a\n temporary copy using a slice of the whole sequence, e.g.,\n\n for x in a[:]:\n if x < 0: a.remove(x)\n\n\nThe "try" statement\n===================\n\nThe "try" statement specifies exception handlers and/or cleanup code\nfor a group of statements:\n\n try_stmt ::= try1_stmt | try2_stmt\n try1_stmt ::= "try" ":" suite\n ("except" [expression ["as" identifier]] ":" suite)+\n ["else" ":" suite]\n ["finally" ":" suite]\n try2_stmt ::= "try" ":" suite\n "finally" ":" suite\n\nThe "except" clause(s) specify one or more exception handlers. When no\nexception occurs in the "try" clause, no exception handler is\nexecuted. When an exception occurs in the "try" suite, a search for an\nexception handler is started. This search inspects the except clauses\nin turn until one is found that matches the exception. An expression-\nless except clause, if present, must be last; it matches any\nexception. For an except clause with an expression, that expression\nis evaluated, and the clause matches the exception if the resulting\nobject is "compatible" with the exception. An object is compatible\nwith an exception if it is the class or a base class of the exception\nobject or a tuple containing an item compatible with the exception.\n\nIf no except clause matches the exception, the search for an exception\nhandler continues in the surrounding code and on the invocation stack.\n[1]\n\nIf the evaluation of an expression in the header of an except clause\nraises an exception, the original search for a handler is canceled and\na search starts for the new exception in the surrounding code and on\nthe call stack (it is treated as if the entire "try" statement raised\nthe exception).\n\nWhen a matching except clause is found, the exception is assigned to\nthe target specified after the "as" keyword in that except clause, if\npresent, and the except clause\'s suite is executed. All except\nclauses must have an executable block. When the end of this block is\nreached, execution continues normally after the entire try statement.\n(This means that if two nested handlers exist for the same exception,\nand the exception occurs in the try clause of the inner handler, the\nouter handler will not handle the exception.)\n\nWhen an exception has been assigned using "as target", it is cleared\nat the end of the except clause. This is as if\n\n except E as N:\n foo\n\nwas translated to\n\n except E as N:\n try:\n foo\n finally:\n del N\n\nThis means the exception must be assigned to a different name to be\nable to refer to it after the except clause. Exceptions are cleared\nbecause with the traceback attached to them, they form a reference\ncycle with the stack frame, keeping all locals in that frame alive\nuntil the next garbage collection occurs.\n\nBefore an except clause\'s suite is executed, details about the\nexception are stored in the "sys" module and can be accessed via\n"sys.exc_info()". "sys.exc_info()" returns a 3-tuple consisting of the\nexception class, the exception instance and a traceback object (see\nsection *The standard type hierarchy*) identifying the point in the\nprogram where the exception occurred. "sys.exc_info()" values are\nrestored to their previous values (before the call) when returning\nfrom a function that handled an exception.\n\nThe optional "else" clause is executed if and when control flows off\nthe end of the "try" clause. [2] Exceptions in the "else" clause are\nnot handled by the preceding "except" clauses.\n\nIf "finally" is present, it specifies a \'cleanup\' handler. The "try"\nclause is executed, including any "except" and "else" clauses. If an\nexception occurs in any of the clauses and is not handled, the\nexception is temporarily saved. The "finally" clause is executed. If\nthere is a saved exception it is re-raised at the end of the "finally"\nclause. If the "finally" clause raises another exception, the saved\nexception is set as the context of the new exception. If the "finally"\nclause executes a "return" or "break" statement, the saved exception\nis discarded:\n\n >>> def f():\n ... try:\n ... 1/0\n ... finally:\n ... return 42\n ...\n >>> f()\n 42\n\nThe exception information is not available to the program during\nexecution of the "finally" clause.\n\nWhen a "return", "break" or "continue" statement is executed in the\n"try" suite of a "try"..."finally" statement, the "finally" clause is\nalso executed \'on the way out.\' A "continue" statement is illegal in\nthe "finally" clause. (The reason is a problem with the current\nimplementation --- this restriction may be lifted in the future).\n\nThe return value of a function is determined by the last "return"\nstatement executed. Since the "finally" clause always executes, a\n"return" statement executed in the "finally" clause will always be the\nlast one executed:\n\n >>> def foo():\n ... try:\n ... return \'try\'\n ... finally:\n ... return \'finally\'\n ...\n >>> foo()\n \'finally\'\n\nAdditional information on exceptions can be found in section\n*Exceptions*, and information on using the "raise" statement to\ngenerate exceptions may be found in section *The raise statement*.\n\n\nThe "with" statement\n====================\n\nThe "with" statement is used to wrap the execution of a block with\nmethods defined by a context manager (see section *With Statement\nContext Managers*). This allows common "try"..."except"..."finally"\nusage patterns to be encapsulated for convenient reuse.\n\n with_stmt ::= "with" with_item ("," with_item)* ":" suite\n with_item ::= expression ["as" target]\n\nThe execution of the "with" statement with one "item" proceeds as\nfollows:\n\n1. The context expression (the expression given in the "with_item")\n is evaluated to obtain a context manager.\n\n2. The context manager\'s "__exit__()" is loaded for later use.\n\n3. The context manager\'s "__enter__()" method is invoked.\n\n4. If a target was included in the "with" statement, the return\n value from "__enter__()" is assigned to it.\n\n Note: The "with" statement guarantees that if the "__enter__()"\n method returns without an error, then "__exit__()" will always be\n called. Thus, if an error occurs during the assignment to the\n target list, it will be treated the same as an error occurring\n within the suite would be. See step 6 below.\n\n5. The suite is executed.\n\n6. The context manager\'s "__exit__()" method is invoked. If an\n exception caused the suite to be exited, its type, value, and\n traceback are passed as arguments to "__exit__()". Otherwise, three\n "None" arguments are supplied.\n\n If the suite was exited due to an exception, and the return value\n from the "__exit__()" method was false, the exception is reraised.\n If the return value was true, the exception is suppressed, and\n execution continues with the statement following the "with"\n statement.\n\n If the suite was exited for any reason other than an exception, the\n return value from "__exit__()" is ignored, and execution proceeds\n at the normal location for the kind of exit that was taken.\n\nWith more than one item, the context managers are processed as if\nmultiple "with" statements were nested:\n\n with A() as a, B() as b:\n suite\n\nis equivalent to\n\n with A() as a:\n with B() as b:\n suite\n\nChanged in version 3.1: Support for multiple context expressions.\n\nSee also: **PEP 0343** - The "with" statement\n\n The specification, background, and examples for the Python "with"\n statement.\n\n\nFunction definitions\n====================\n\nA function definition defines a user-defined function object (see\nsection *The standard type hierarchy*):\n\n funcdef ::= [decorators] "def" funcname "(" [parameter_list] ")" ["->" expression] ":" suite\n decorators ::= decorator+\n decorator ::= "@" dotted_name ["(" [parameter_list [","]] ")"] NEWLINE\n dotted_name ::= identifier ("." identifier)*\n parameter_list ::= (defparameter ",")*\n | "*" [parameter] ("," defparameter)* ["," "**" parameter]\n | "**" parameter\n | defparameter [","] )\n parameter ::= identifier [":" expression]\n defparameter ::= parameter ["=" expression]\n funcname ::= identifier\n\nA function definition is an executable statement. Its execution binds\nthe function name in the current local namespace to a function object\n(a wrapper around the executable code for the function). This\nfunction object contains a reference to the current global namespace\nas the global namespace to be used when the function is called.\n\nThe function definition does not execute the function body; this gets\nexecuted only when the function is called. [3]\n\nA function definition may be wrapped by one or more *decorator*\nexpressions. Decorator expressions are evaluated when the function is\ndefined, in the scope that contains the function definition. The\nresult must be a callable, which is invoked with the function object\nas the only argument. The returned value is bound to the function name\ninstead of the function object. Multiple decorators are applied in\nnested fashion. For example, the following code\n\n @f1(arg)\n @f2\n def func(): pass\n\nis equivalent to\n\n def func(): pass\n func = f1(arg)(f2(func))\n\nWhen one or more *parameters* have the form *parameter* "="\n*expression*, the function is said to have "default parameter values."\nFor a parameter with a default value, the corresponding *argument* may\nbe omitted from a call, in which case the parameter\'s default value is\nsubstituted. If a parameter has a default value, all following\nparameters up until the ""*"" must also have a default value --- this\nis a syntactic restriction that is not expressed by the grammar.\n\n**Default parameter values are evaluated from left to right when the\nfunction definition is executed.** This means that the expression is\nevaluated once, when the function is defined, and that the same "pre-\ncomputed" value is used for each call. This is especially important\nto understand when a default parameter is a mutable object, such as a\nlist or a dictionary: if the function modifies the object (e.g. by\nappending an item to a list), the default value is in effect modified.\nThis is generally not what was intended. A way around this is to use\n"None" as the default, and explicitly test for it in the body of the\nfunction, e.g.:\n\n def whats_on_the_telly(penguin=None):\n if penguin is None:\n penguin = []\n penguin.append("property of the zoo")\n return penguin\n\nFunction call semantics are described in more detail in section\n*Calls*. A function call always assigns values to all parameters\nmentioned in the parameter list, either from position arguments, from\nkeyword arguments, or from default values. If the form\n""*identifier"" is present, it is initialized to a tuple receiving any\nexcess positional parameters, defaulting to the empty tuple. If the\nform ""**identifier"" is present, it is initialized to a new\ndictionary receiving any excess keyword arguments, defaulting to a new\nempty dictionary. Parameters after ""*"" or ""*identifier"" are\nkeyword-only parameters and may only be passed used keyword arguments.\n\nParameters may have annotations of the form "": expression"" following\nthe parameter name. Any parameter may have an annotation even those\nof the form "*identifier" or "**identifier". Functions may have\n"return" annotation of the form ""-> expression"" after the parameter\nlist. These annotations can be any valid Python expression and are\nevaluated when the function definition is executed. Annotations may\nbe evaluated in a different order than they appear in the source code.\nThe presence of annotations does not change the semantics of a\nfunction. The annotation values are available as values of a\ndictionary keyed by the parameters\' names in the "__annotations__"\nattribute of the function object.\n\nIt is also possible to create anonymous functions (functions not bound\nto a name), for immediate use in expressions. This uses lambda\nexpressions, described in section *Lambdas*. Note that the lambda\nexpression is merely a shorthand for a simplified function definition;\na function defined in a ""def"" statement can be passed around or\nassigned to another name just like a function defined by a lambda\nexpression. The ""def"" form is actually more powerful since it\nallows the execution of multiple statements and annotations.\n\n**Programmer\'s note:** Functions are first-class objects. A ""def""\nstatement executed inside a function definition defines a local\nfunction that can be returned or passed around. Free variables used\nin the nested function can access the local variables of the function\ncontaining the def. See section *Naming and binding* for details.\n\nSee also: **PEP 3107** - Function Annotations\n\n The original specification for function annotations.\n\n\nClass definitions\n=================\n\nA class definition defines a class object (see section *The standard\ntype hierarchy*):\n\n classdef ::= [decorators] "class" classname [inheritance] ":" suite\n inheritance ::= "(" [parameter_list] ")"\n classname ::= identifier\n\nA class definition is an executable statement. The inheritance list\nusually gives a list of base classes (see *Customizing class creation*\nfor more advanced uses), so each item in the list should evaluate to a\nclass object which allows subclassing. Classes without an inheritance\nlist inherit, by default, from the base class "object"; hence,\n\n class Foo:\n pass\n\nis equivalent to\n\n class Foo(object):\n pass\n\nThe class\'s suite is then executed in a new execution frame (see\n*Naming and binding*), using a newly created local namespace and the\noriginal global namespace. (Usually, the suite contains mostly\nfunction definitions.) When the class\'s suite finishes execution, its\nexecution frame is discarded but its local namespace is saved. [4] A\nclass object is then created using the inheritance list for the base\nclasses and the saved local namespace for the attribute dictionary.\nThe class name is bound to this class object in the original local\nnamespace.\n\nClass creation can be customized heavily using *metaclasses*.\n\nClasses can also be decorated: just like when decorating functions,\n\n @f1(arg)\n @f2\n class Foo: pass\n\nis equivalent to\n\n class Foo: pass\n Foo = f1(arg)(f2(Foo))\n\nThe evaluation rules for the decorator expressions are the same as for\nfunction decorators. The result must be a class object, which is then\nbound to the class name.\n\n**Programmer\'s note:** Variables defined in the class definition are\nclass attributes; they are shared by instances. Instance attributes\ncan be set in a method with "self.name = value". Both class and\ninstance attributes are accessible through the notation ""self.name"",\nand an instance attribute hides a class attribute with the same name\nwhen accessed in this way. Class attributes can be used as defaults\nfor instance attributes, but using mutable values there can lead to\nunexpected results. *Descriptors* can be used to create instance\nvariables with different implementation details.\n\nSee also: **PEP 3115** - Metaclasses in Python 3 **PEP 3129** -\n Class Decorators\n\n\nCoroutines\n==========\n\nNew in version 3.5.\n\n\nCoroutine function definition\n-----------------------------\n\n async_funcdef ::= [decorators] "async" "def" funcname "(" [parameter_list] ")" ["->" expression] ":" suite\n\nExecution of Python coroutines can be suspended and resumed at many\npoints (see *coroutine*). In the body of a coroutine, any "await" and\n"async" identifiers become reserved keywords; "await" expressions,\n"async for" and "async with" can only be used in coroutine bodies.\n\nFunctions defined with "async def" syntax are always coroutine\nfunctions, even if they do not contain "await" or "async" keywords.\n\nIt is a "SyntaxError" to use "yield" expressions in "async def"\ncoroutines.\n\nAn example of a coroutine function:\n\n async def func(param1, param2):\n do_stuff()\n await some_coroutine()\n\n\nThe "async for" statement\n-------------------------\n\n async_for_stmt ::= "async" for_stmt\n\nAn *asynchronous iterable* is able to call asynchronous code in its\n*iter* implementation, and *asynchronous iterator* can call\nasynchronous code in its *next* method.\n\nThe "async for" statement allows convenient iteration over\nasynchronous iterators.\n\nThe following code:\n\n async for TARGET in ITER:\n BLOCK\n else:\n BLOCK2\n\nIs semantically equivalent to:\n\n iter = (ITER)\n iter = await type(iter).__aiter__(iter)\n running = True\n while running:\n try:\n TARGET = await type(iter).__anext__(iter)\n except StopAsyncIteration:\n running = False\n else:\n BLOCK\n else:\n BLOCK2\n\nSee also "__aiter__()" and "__anext__()" for details.\n\nIt is a "SyntaxError" to use "async for" statement outside of an\n"async def" function.\n\n\nThe "async with" statement\n--------------------------\n\n async_with_stmt ::= "async" with_stmt\n\nAn *asynchronous context manager* is a *context manager* that is able\nto suspend execution in its *enter* and *exit* methods.\n\nThe following code:\n\n async with EXPR as VAR:\n BLOCK\n\nIs semantically equivalent to:\n\n mgr = (EXPR)\n aexit = type(mgr).__aexit__\n aenter = type(mgr).__aenter__(mgr)\n exc = True\n\n VAR = await aenter\n try:\n BLOCK\n except:\n if not await aexit(mgr, *sys.exc_info()):\n raise\n else:\n await aexit(mgr, None, None, None)\n\nSee also "__aenter__()" and "__aexit__()" for details.\n\nIt is a "SyntaxError" to use "async with" statement outside of an\n"async def" function.\n\nSee also: **PEP 492** - Coroutines with async and await syntax\n\n-[ Footnotes ]-\n\n[1] The exception is propagated to the invocation stack unless\n there is a "finally" clause which happens to raise another\n exception. That new exception causes the old one to be lost.\n\n[2] Currently, control "flows off the end" except in the case of\n an exception or the execution of a "return", "continue", or\n "break" statement.\n\n[3] A string literal appearing as the first statement in the\n function body is transformed into the function\'s "__doc__"\n attribute and therefore the function\'s *docstring*.\n\n[4] A string literal appearing as the first statement in the class\n body is transformed into the namespace\'s "__doc__" item and\n therefore the class\'s *docstring*.\n', - 'context-managers': u'\nWith Statement Context Managers\n*******************************\n\nA *context manager* is an object that defines the runtime context to\nbe established when executing a "with" statement. The context manager\nhandles the entry into, and the exit from, the desired runtime context\nfor the execution of the block of code. Context managers are normally\ninvoked using the "with" statement (described in section *The with\nstatement*), but can also be used by directly invoking their methods.\n\nTypical uses of context managers include saving and restoring various\nkinds of global state, locking and unlocking resources, closing opened\nfiles, etc.\n\nFor more information on context managers, see *Context Manager Types*.\n\nobject.__enter__(self)\n\n Enter the runtime context related to this object. The "with"\n statement will bind this method\'s return value to the target(s)\n specified in the "as" clause of the statement, if any.\n\nobject.__exit__(self, exc_type, exc_value, traceback)\n\n Exit the runtime context related to this object. The parameters\n describe the exception that caused the context to be exited. If the\n context was exited without an exception, all three arguments will\n be "None".\n\n If an exception is supplied, and the method wishes to suppress the\n exception (i.e., prevent it from being propagated), it should\n return a true value. Otherwise, the exception will be processed\n normally upon exit from this method.\n\n Note that "__exit__()" methods should not reraise the passed-in\n exception; this is the caller\'s responsibility.\n\nSee also: **PEP 0343** - The "with" statement\n\n The specification, background, and examples for the Python "with"\n statement.\n', - 'continue': u'\nThe "continue" statement\n************************\n\n continue_stmt ::= "continue"\n\n"continue" may only occur syntactically nested in a "for" or "while"\nloop, but not nested in a function or class definition or "finally"\nclause within that loop. It continues with the next cycle of the\nnearest enclosing loop.\n\nWhen "continue" passes control out of a "try" statement with a\n"finally" clause, that "finally" clause is executed before really\nstarting the next loop cycle.\n', - 'conversions': u'\nArithmetic conversions\n**********************\n\nWhen a description of an arithmetic operator below uses the phrase\n"the numeric arguments are converted to a common type," this means\nthat the operator implementation for built-in types works as follows:\n\n* If either argument is a complex number, the other is converted to\n complex;\n\n* otherwise, if either argument is a floating point number, the\n other is converted to floating point;\n\n* otherwise, both must be integers and no conversion is necessary.\n\nSome additional rules apply for certain operators (e.g., a string as a\nleft argument to the \'%\' operator). Extensions must define their own\nconversion behavior.\n', - 'customization': u'\nBasic customization\n*******************\n\nobject.__new__(cls[, ...])\n\n Called to create a new instance of class *cls*. "__new__()" is a\n static method (special-cased so you need not declare it as such)\n that takes the class of which an instance was requested as its\n first argument. The remaining arguments are those passed to the\n object constructor expression (the call to the class). The return\n value of "__new__()" should be the new object instance (usually an\n instance of *cls*).\n\n Typical implementations create a new instance of the class by\n invoking the superclass\'s "__new__()" method using\n "super(currentclass, cls).__new__(cls[, ...])" with appropriate\n arguments and then modifying the newly-created instance as\n necessary before returning it.\n\n If "__new__()" returns an instance of *cls*, then the new\n instance\'s "__init__()" method will be invoked like\n "__init__(self[, ...])", where *self* is the new instance and the\n remaining arguments are the same as were passed to "__new__()".\n\n If "__new__()" does not return an instance of *cls*, then the new\n instance\'s "__init__()" method will not be invoked.\n\n "__new__()" is intended mainly to allow subclasses of immutable\n types (like int, str, or tuple) to customize instance creation. It\n is also commonly overridden in custom metaclasses in order to\n customize class creation.\n\nobject.__init__(self[, ...])\n\n Called after the instance has been created (by "__new__()"), but\n before it is returned to the caller. The arguments are those\n passed to the class constructor expression. If a base class has an\n "__init__()" method, the derived class\'s "__init__()" method, if\n any, must explicitly call it to ensure proper initialization of the\n base class part of the instance; for example:\n "BaseClass.__init__(self, [args...])".\n\n Because "__new__()" and "__init__()" work together in constructing\n objects ("__new__()" to create it, and "__init__()" to customise\n it), no non-"None" value may be returned by "__init__()"; doing so\n will cause a "TypeError" to be raised at runtime.\n\nobject.__del__(self)\n\n Called when the instance is about to be destroyed. This is also\n called a destructor. If a base class has a "__del__()" method, the\n derived class\'s "__del__()" method, if any, must explicitly call it\n to ensure proper deletion of the base class part of the instance.\n Note that it is possible (though not recommended!) for the\n "__del__()" method to postpone destruction of the instance by\n creating a new reference to it. It may then be called at a later\n time when this new reference is deleted. It is not guaranteed that\n "__del__()" methods are called for objects that still exist when\n the interpreter exits.\n\n Note: "del x" doesn\'t directly call "x.__del__()" --- the former\n decrements the reference count for "x" by one, and the latter is\n only called when "x"\'s reference count reaches zero. Some common\n situations that may prevent the reference count of an object from\n going to zero include: circular references between objects (e.g.,\n a doubly-linked list or a tree data structure with parent and\n child pointers); a reference to the object on the stack frame of\n a function that caught an exception (the traceback stored in\n "sys.exc_info()[2]" keeps the stack frame alive); or a reference\n to the object on the stack frame that raised an unhandled\n exception in interactive mode (the traceback stored in\n "sys.last_traceback" keeps the stack frame alive). The first\n situation can only be remedied by explicitly breaking the cycles;\n the second can be resolved by freeing the reference to the\n traceback object when it is no longer useful, and the third can\n be resolved by storing "None" in "sys.last_traceback". Circular\n references which are garbage are detected and cleaned up when the\n cyclic garbage collector is enabled (it\'s on by default). Refer\n to the documentation for the "gc" module for more information\n about this topic.\n\n Warning: Due to the precarious circumstances under which\n "__del__()" methods are invoked, exceptions that occur during\n their execution are ignored, and a warning is printed to\n "sys.stderr" instead. Also, when "__del__()" is invoked in\n response to a module being deleted (e.g., when execution of the\n program is done), other globals referenced by the "__del__()"\n method may already have been deleted or in the process of being\n torn down (e.g. the import machinery shutting down). For this\n reason, "__del__()" methods should do the absolute minimum needed\n to maintain external invariants. Starting with version 1.5,\n Python guarantees that globals whose name begins with a single\n underscore are deleted from their module before other globals are\n deleted; if no other references to such globals exist, this may\n help in assuring that imported modules are still available at the\n time when the "__del__()" method is called.\n\nobject.__repr__(self)\n\n Called by the "repr()" built-in function to compute the "official"\n string representation of an object. If at all possible, this\n should look like a valid Python expression that could be used to\n recreate an object with the same value (given an appropriate\n environment). If this is not possible, a string of the form\n "<...some useful description...>" should be returned. The return\n value must be a string object. If a class defines "__repr__()" but\n not "__str__()", then "__repr__()" is also used when an "informal"\n string representation of instances of that class is required.\n\n This is typically used for debugging, so it is important that the\n representation is information-rich and unambiguous.\n\nobject.__str__(self)\n\n Called by "str(object)" and the built-in functions "format()" and\n "print()" to compute the "informal" or nicely printable string\n representation of an object. The return value must be a *string*\n object.\n\n This method differs from "object.__repr__()" in that there is no\n expectation that "__str__()" return a valid Python expression: a\n more convenient or concise representation can be used.\n\n The default implementation defined by the built-in type "object"\n calls "object.__repr__()".\n\nobject.__bytes__(self)\n\n Called by "bytes()" to compute a byte-string representation of an\n object. This should return a "bytes" object.\n\nobject.__format__(self, format_spec)\n\n Called by the "format()" built-in function (and by extension, the\n "str.format()" method of class "str") to produce a "formatted"\n string representation of an object. The "format_spec" argument is a\n string that contains a description of the formatting options\n desired. The interpretation of the "format_spec" argument is up to\n the type implementing "__format__()", however most classes will\n either delegate formatting to one of the built-in types, or use a\n similar formatting option syntax.\n\n See *Format Specification Mini-Language* for a description of the\n standard formatting syntax.\n\n The return value must be a string object.\n\n Changed in version 3.4: The __format__ method of "object" itself\n raises a "TypeError" if passed any non-empty string.\n\nobject.__lt__(self, other)\nobject.__le__(self, other)\nobject.__eq__(self, other)\nobject.__ne__(self, other)\nobject.__gt__(self, other)\nobject.__ge__(self, other)\n\n These are the so-called "rich comparison" methods. The\n correspondence between operator symbols and method names is as\n follows: "xy" calls\n "x.__gt__(y)", and "x>=y" calls "x.__ge__(y)".\n\n A rich comparison method may return the singleton "NotImplemented"\n if it does not implement the operation for a given pair of\n arguments. By convention, "False" and "True" are returned for a\n successful comparison. However, these methods can return any value,\n so if the comparison operator is used in a Boolean context (e.g.,\n in the condition of an "if" statement), Python will call "bool()"\n on the value to determine if the result is true or false.\n\n By default, "__ne__()" delegates to "__eq__()" and inverts the\n result unless it is "NotImplemented". There are no other implied\n relationships among the comparison operators, for example, the\n truth of "(x.__hash__".\n\n If a class that does not override "__eq__()" wishes to suppress\n hash support, it should include "__hash__ = None" in the class\n definition. A class which defines its own "__hash__()" that\n explicitly raises a "TypeError" would be incorrectly identified as\n hashable by an "isinstance(obj, collections.Hashable)" call.\n\n Note: By default, the "__hash__()" values of str, bytes and\n datetime objects are "salted" with an unpredictable random value.\n Although they remain constant within an individual Python\n process, they are not predictable between repeated invocations of\n Python.This is intended to provide protection against a denial-\n of-service caused by carefully-chosen inputs that exploit the\n worst case performance of a dict insertion, O(n^2) complexity.\n See http://www.ocert.org/advisories/ocert-2011-003.html for\n details.Changing hash values affects the iteration order of\n dicts, sets and other mappings. Python has never made guarantees\n about this ordering (and it typically varies between 32-bit and\n 64-bit builds).See also "PYTHONHASHSEED".\n\n Changed in version 3.3: Hash randomization is enabled by default.\n\nobject.__bool__(self)\n\n Called to implement truth value testing and the built-in operation\n "bool()"; should return "False" or "True". When this method is not\n defined, "__len__()" is called, if it is defined, and the object is\n considered true if its result is nonzero. If a class defines\n neither "__len__()" nor "__bool__()", all its instances are\n considered true.\n', - 'debugger': u'\n"pdb" --- The Python Debugger\n*****************************\n\n**Source code:** Lib/pdb.py\n\n======================================================================\n\nThe module "pdb" defines an interactive source code debugger for\nPython programs. It supports setting (conditional) breakpoints and\nsingle stepping at the source line level, inspection of stack frames,\nsource code listing, and evaluation of arbitrary Python code in the\ncontext of any stack frame. It also supports post-mortem debugging\nand can be called under program control.\n\nThe debugger is extensible -- it is actually defined as the class\n"Pdb". This is currently undocumented but easily understood by reading\nthe source. The extension interface uses the modules "bdb" and "cmd".\n\nThe debugger\'s prompt is "(Pdb)". Typical usage to run a program under\ncontrol of the debugger is:\n\n >>> import pdb\n >>> import mymodule\n >>> pdb.run(\'mymodule.test()\')\n > (0)?()\n (Pdb) continue\n > (1)?()\n (Pdb) continue\n NameError: \'spam\'\n > (1)?()\n (Pdb)\n\nChanged in version 3.3: Tab-completion via the "readline" module is\navailable for commands and command arguments, e.g. the current global\nand local names are offered as arguments of the "p" command.\n\n"pdb.py" can also be invoked as a script to debug other scripts. For\nexample:\n\n python3 -m pdb myscript.py\n\nWhen invoked as a script, pdb will automatically enter post-mortem\ndebugging if the program being debugged exits abnormally. After post-\nmortem debugging (or after normal exit of the program), pdb will\nrestart the program. Automatic restarting preserves pdb\'s state (such\nas breakpoints) and in most cases is more useful than quitting the\ndebugger upon program\'s exit.\n\nNew in version 3.2: "pdb.py" now accepts a "-c" option that executes\ncommands as if given in a ".pdbrc" file, see *Debugger Commands*.\n\nThe typical usage to break into the debugger from a running program is\nto insert\n\n import pdb; pdb.set_trace()\n\nat the location you want to break into the debugger. You can then\nstep through the code following this statement, and continue running\nwithout the debugger using the "continue" command.\n\nThe typical usage to inspect a crashed program is:\n\n >>> import pdb\n >>> import mymodule\n >>> mymodule.test()\n Traceback (most recent call last):\n File "", line 1, in ?\n File "./mymodule.py", line 4, in test\n test2()\n File "./mymodule.py", line 3, in test2\n print(spam)\n NameError: spam\n >>> pdb.pm()\n > ./mymodule.py(3)test2()\n -> print(spam)\n (Pdb)\n\nThe module defines the following functions; each enters the debugger\nin a slightly different way:\n\npdb.run(statement, globals=None, locals=None)\n\n Execute the *statement* (given as a string or a code object) under\n debugger control. The debugger prompt appears before any code is\n executed; you can set breakpoints and type "continue", or you can\n step through the statement using "step" or "next" (all these\n commands are explained below). The optional *globals* and *locals*\n arguments specify the environment in which the code is executed; by\n default the dictionary of the module "__main__" is used. (See the\n explanation of the built-in "exec()" or "eval()" functions.)\n\npdb.runeval(expression, globals=None, locals=None)\n\n Evaluate the *expression* (given as a string or a code object)\n under debugger control. When "runeval()" returns, it returns the\n value of the expression. Otherwise this function is similar to\n "run()".\n\npdb.runcall(function, *args, **kwds)\n\n Call the *function* (a function or method object, not a string)\n with the given arguments. When "runcall()" returns, it returns\n whatever the function call returned. The debugger prompt appears\n as soon as the function is entered.\n\npdb.set_trace()\n\n Enter the debugger at the calling stack frame. This is useful to\n hard-code a breakpoint at a given point in a program, even if the\n code is not otherwise being debugged (e.g. when an assertion\n fails).\n\npdb.post_mortem(traceback=None)\n\n Enter post-mortem debugging of the given *traceback* object. If no\n *traceback* is given, it uses the one of the exception that is\n currently being handled (an exception must be being handled if the\n default is to be used).\n\npdb.pm()\n\n Enter post-mortem debugging of the traceback found in\n "sys.last_traceback".\n\nThe "run*" functions and "set_trace()" are aliases for instantiating\nthe "Pdb" class and calling the method of the same name. If you want\nto access further features, you have to do this yourself:\n\nclass class pdb.Pdb(completekey=\'tab\', stdin=None, stdout=None, skip=None, nosigint=False)\n\n "Pdb" is the debugger class.\n\n The *completekey*, *stdin* and *stdout* arguments are passed to the\n underlying "cmd.Cmd" class; see the description there.\n\n The *skip* argument, if given, must be an iterable of glob-style\n module name patterns. The debugger will not step into frames that\n originate in a module that matches one of these patterns. [1]\n\n By default, Pdb sets a handler for the SIGINT signal (which is sent\n when the user presses Ctrl-C on the console) when you give a\n "continue" command. This allows you to break into the debugger\n again by pressing Ctrl-C. If you want Pdb not to touch the SIGINT\n handler, set *nosigint* tot true.\n\n Example call to enable tracing with *skip*:\n\n import pdb; pdb.Pdb(skip=[\'django.*\']).set_trace()\n\n New in version 3.1: The *skip* argument.\n\n New in version 3.2: The *nosigint* argument. Previously, a SIGINT\n handler was never set by Pdb.\n\n run(statement, globals=None, locals=None)\n runeval(expression, globals=None, locals=None)\n runcall(function, *args, **kwds)\n set_trace()\n\n See the documentation for the functions explained above.\n\n\nDebugger Commands\n=================\n\nThe commands recognized by the debugger are listed below. Most\ncommands can be abbreviated to one or two letters as indicated; e.g.\n"h(elp)" means that either "h" or "help" can be used to enter the help\ncommand (but not "he" or "hel", nor "H" or "Help" or "HELP").\nArguments to commands must be separated by whitespace (spaces or\ntabs). Optional arguments are enclosed in square brackets ("[]") in\nthe command syntax; the square brackets must not be typed.\nAlternatives in the command syntax are separated by a vertical bar\n("|").\n\nEntering a blank line repeats the last command entered. Exception: if\nthe last command was a "list" command, the next 11 lines are listed.\n\nCommands that the debugger doesn\'t recognize are assumed to be Python\nstatements and are executed in the context of the program being\ndebugged. Python statements can also be prefixed with an exclamation\npoint ("!"). This is a powerful way to inspect the program being\ndebugged; it is even possible to change a variable or call a function.\nWhen an exception occurs in such a statement, the exception name is\nprinted but the debugger\'s state is not changed.\n\nThe debugger supports *aliases*. Aliases can have parameters which\nallows one a certain level of adaptability to the context under\nexamination.\n\nMultiple commands may be entered on a single line, separated by ";;".\n(A single ";" is not used as it is the separator for multiple commands\nin a line that is passed to the Python parser.) No intelligence is\napplied to separating the commands; the input is split at the first\n";;" pair, even if it is in the middle of a quoted string.\n\nIf a file ".pdbrc" exists in the user\'s home directory or in the\ncurrent directory, it is read in and executed as if it had been typed\nat the debugger prompt. This is particularly useful for aliases. If\nboth files exist, the one in the home directory is read first and\naliases defined there can be overridden by the local file.\n\nChanged in version 3.2: ".pdbrc" can now contain commands that\ncontinue debugging, such as "continue" or "next". Previously, these\ncommands had no effect.\n\nh(elp) [command]\n\n Without argument, print the list of available commands. With a\n *command* as argument, print help about that command. "help pdb"\n displays the full documentation (the docstring of the "pdb"\n module). Since the *command* argument must be an identifier, "help\n exec" must be entered to get help on the "!" command.\n\nw(here)\n\n Print a stack trace, with the most recent frame at the bottom. An\n arrow indicates the current frame, which determines the context of\n most commands.\n\nd(own) [count]\n\n Move the current frame *count* (default one) levels down in the\n stack trace (to a newer frame).\n\nu(p) [count]\n\n Move the current frame *count* (default one) levels up in the stack\n trace (to an older frame).\n\nb(reak) [([filename:]lineno | function) [, condition]]\n\n With a *lineno* argument, set a break there in the current file.\n With a *function* argument, set a break at the first executable\n statement within that function. The line number may be prefixed\n with a filename and a colon, to specify a breakpoint in another\n file (probably one that hasn\'t been loaded yet). The file is\n searched on "sys.path". Note that each breakpoint is assigned a\n number to which all the other breakpoint commands refer.\n\n If a second argument is present, it is an expression which must\n evaluate to true before the breakpoint is honored.\n\n Without argument, list all breaks, including for each breakpoint,\n the number of times that breakpoint has been hit, the current\n ignore count, and the associated condition if any.\n\ntbreak [([filename:]lineno | function) [, condition]]\n\n Temporary breakpoint, which is removed automatically when it is\n first hit. The arguments are the same as for "break".\n\ncl(ear) [filename:lineno | bpnumber [bpnumber ...]]\n\n With a *filename:lineno* argument, clear all the breakpoints at\n this line. With a space separated list of breakpoint numbers, clear\n those breakpoints. Without argument, clear all breaks (but first\n ask confirmation).\n\ndisable [bpnumber [bpnumber ...]]\n\n Disable the breakpoints given as a space separated list of\n breakpoint numbers. Disabling a breakpoint means it cannot cause\n the program to stop execution, but unlike clearing a breakpoint, it\n remains in the list of breakpoints and can be (re-)enabled.\n\nenable [bpnumber [bpnumber ...]]\n\n Enable the breakpoints specified.\n\nignore bpnumber [count]\n\n Set the ignore count for the given breakpoint number. If count is\n omitted, the ignore count is set to 0. A breakpoint becomes active\n when the ignore count is zero. When non-zero, the count is\n decremented each time the breakpoint is reached and the breakpoint\n is not disabled and any associated condition evaluates to true.\n\ncondition bpnumber [condition]\n\n Set a new *condition* for the breakpoint, an expression which must\n evaluate to true before the breakpoint is honored. If *condition*\n is absent, any existing condition is removed; i.e., the breakpoint\n is made unconditional.\n\ncommands [bpnumber]\n\n Specify a list of commands for breakpoint number *bpnumber*. The\n commands themselves appear on the following lines. Type a line\n containing just "end" to terminate the commands. An example:\n\n (Pdb) commands 1\n (com) p some_variable\n (com) end\n (Pdb)\n\n To remove all commands from a breakpoint, type commands and follow\n it immediately with "end"; that is, give no commands.\n\n With no *bpnumber* argument, commands refers to the last breakpoint\n set.\n\n You can use breakpoint commands to start your program up again.\n Simply use the continue command, or step, or any other command that\n resumes execution.\n\n Specifying any command resuming execution (currently continue,\n step, next, return, jump, quit and their abbreviations) terminates\n the command list (as if that command was immediately followed by\n end). This is because any time you resume execution (even with a\n simple next or step), you may encounter another breakpoint--which\n could have its own command list, leading to ambiguities about which\n list to execute.\n\n If you use the \'silent\' command in the command list, the usual\n message about stopping at a breakpoint is not printed. This may be\n desirable for breakpoints that are to print a specific message and\n then continue. If none of the other commands print anything, you\n see no sign that the breakpoint was reached.\n\ns(tep)\n\n Execute the current line, stop at the first possible occasion\n (either in a function that is called or on the next line in the\n current function).\n\nn(ext)\n\n Continue execution until the next line in the current function is\n reached or it returns. (The difference between "next" and "step"\n is that "step" stops inside a called function, while "next"\n executes called functions at (nearly) full speed, only stopping at\n the next line in the current function.)\n\nunt(il) [lineno]\n\n Without argument, continue execution until the line with a number\n greater than the current one is reached.\n\n With a line number, continue execution until a line with a number\n greater or equal to that is reached. In both cases, also stop when\n the current frame returns.\n\n Changed in version 3.2: Allow giving an explicit line number.\n\nr(eturn)\n\n Continue execution until the current function returns.\n\nc(ont(inue))\n\n Continue execution, only stop when a breakpoint is encountered.\n\nj(ump) lineno\n\n Set the next line that will be executed. Only available in the\n bottom-most frame. This lets you jump back and execute code again,\n or jump forward to skip code that you don\'t want to run.\n\n It should be noted that not all jumps are allowed -- for instance\n it is not possible to jump into the middle of a "for" loop or out\n of a "finally" clause.\n\nl(ist) [first[, last]]\n\n List source code for the current file. Without arguments, list 11\n lines around the current line or continue the previous listing.\n With "." as argument, list 11 lines around the current line. With\n one argument, list 11 lines around at that line. With two\n arguments, list the given range; if the second argument is less\n than the first, it is interpreted as a count.\n\n The current line in the current frame is indicated by "->". If an\n exception is being debugged, the line where the exception was\n originally raised or propagated is indicated by ">>", if it differs\n from the current line.\n\n New in version 3.2: The ">>" marker.\n\nll | longlist\n\n List all source code for the current function or frame.\n Interesting lines are marked as for "list".\n\n New in version 3.2.\n\na(rgs)\n\n Print the argument list of the current function.\n\np expression\n\n Evaluate the *expression* in the current context and print its\n value.\n\n Note: "print()" can also be used, but is not a debugger command\n --- this executes the Python "print()" function.\n\npp expression\n\n Like the "p" command, except the value of the expression is pretty-\n printed using the "pprint" module.\n\nwhatis expression\n\n Print the type of the *expression*.\n\nsource expression\n\n Try to get source code for the given object and display it.\n\n New in version 3.2.\n\ndisplay [expression]\n\n Display the value of the expression if it changed, each time\n execution stops in the current frame.\n\n Without expression, list all display expressions for the current\n frame.\n\n New in version 3.2.\n\nundisplay [expression]\n\n Do not display the expression any more in the current frame.\n Without expression, clear all display expressions for the current\n frame.\n\n New in version 3.2.\n\ninteract\n\n Start an interative interpreter (using the "code" module) whose\n global namespace contains all the (global and local) names found in\n the current scope.\n\n New in version 3.2.\n\nalias [name [command]]\n\n Create an alias called *name* that executes *command*. The command\n must *not* be enclosed in quotes. Replaceable parameters can be\n indicated by "%1", "%2", and so on, while "%*" is replaced by all\n the parameters. If no command is given, the current alias for\n *name* is shown. If no arguments are given, all aliases are listed.\n\n Aliases may be nested and can contain anything that can be legally\n typed at the pdb prompt. Note that internal pdb commands *can* be\n overridden by aliases. Such a command is then hidden until the\n alias is removed. Aliasing is recursively applied to the first\n word of the command line; all other words in the line are left\n alone.\n\n As an example, here are two useful aliases (especially when placed\n in the ".pdbrc" file):\n\n # Print instance variables (usage "pi classInst")\n alias pi for k in %1.__dict__.keys(): print("%1.",k,"=",%1.__dict__[k])\n # Print instance variables in self\n alias ps pi self\n\nunalias name\n\n Delete the specified alias.\n\n! statement\n\n Execute the (one-line) *statement* in the context of the current\n stack frame. The exclamation point can be omitted unless the first\n word of the statement resembles a debugger command. To set a\n global variable, you can prefix the assignment command with a\n "global" statement on the same line, e.g.:\n\n (Pdb) global list_options; list_options = [\'-l\']\n (Pdb)\n\nrun [args ...]\nrestart [args ...]\n\n Restart the debugged Python program. If an argument is supplied,\n it is split with "shlex" and the result is used as the new\n "sys.argv". History, breakpoints, actions and debugger options are\n preserved. "restart" is an alias for "run".\n\nq(uit)\n\n Quit from the debugger. The program being executed is aborted.\n\n-[ Footnotes ]-\n\n[1] Whether a frame is considered to originate in a certain module\n is determined by the "__name__" in the frame globals.\n', - 'del': u'\nThe "del" statement\n*******************\n\n del_stmt ::= "del" target_list\n\nDeletion is recursively defined very similar to the way assignment is\ndefined. Rather than spelling it out in full details, here are some\nhints.\n\nDeletion of a target list recursively deletes each target, from left\nto right.\n\nDeletion of a name removes the binding of that name from the local or\nglobal namespace, depending on whether the name occurs in a "global"\nstatement in the same code block. If the name is unbound, a\n"NameError" exception will be raised.\n\nDeletion of attribute references, subscriptions and slicings is passed\nto the primary object involved; deletion of a slicing is in general\nequivalent to assignment of an empty slice of the right type (but even\nthis is determined by the sliced object).\n\nChanged in version 3.2: Previously it was illegal to delete a name\nfrom the local namespace if it occurs as a free variable in a nested\nblock.\n', - 'dict': u'\nDictionary displays\n*******************\n\nA dictionary display is a possibly empty series of key/datum pairs\nenclosed in curly braces:\n\n dict_display ::= "{" [key_datum_list | dict_comprehension] "}"\n key_datum_list ::= key_datum ("," key_datum)* [","]\n key_datum ::= expression ":" expression\n dict_comprehension ::= expression ":" expression comp_for\n\nA dictionary display yields a new dictionary object.\n\nIf a comma-separated sequence of key/datum pairs is given, they are\nevaluated from left to right to define the entries of the dictionary:\neach key object is used as a key into the dictionary to store the\ncorresponding datum. This means that you can specify the same key\nmultiple times in the key/datum list, and the final dictionary\'s value\nfor that key will be the last one given.\n\nA dict comprehension, in contrast to list and set comprehensions,\nneeds two expressions separated with a colon followed by the usual\n"for" and "if" clauses. When the comprehension is run, the resulting\nkey and value elements are inserted in the new dictionary in the order\nthey are produced.\n\nRestrictions on the types of the key values are listed earlier in\nsection *The standard type hierarchy*. (To summarize, the key type\nshould be *hashable*, which excludes all mutable objects.) Clashes\nbetween duplicate keys are not detected; the last datum (textually\nrightmost in the display) stored for a given key value prevails.\n', - 'dynamic-features': u'\nInteraction with dynamic features\n*********************************\n\nName resolution of free variables occurs at runtime, not at compile\ntime. This means that the following code will print 42:\n\n i = 10\n def f():\n print(i)\n i = 42\n f()\n\nThere are several cases where Python statements are illegal when used\nin conjunction with nested scopes that contain free variables.\n\nIf a variable is referenced in an enclosing scope, it is illegal to\ndelete the name. An error will be reported at compile time.\n\nThe "eval()" and "exec()" functions do not have access to the full\nenvironment for resolving names. Names may be resolved in the local\nand global namespaces of the caller. Free variables are not resolved\nin the nearest enclosing namespace, but in the global namespace. [1]\nThe "exec()" and "eval()" functions have optional arguments to\noverride the global and local namespace. If only one namespace is\nspecified, it is used for both.\n', - 'else': u'\nThe "if" statement\n******************\n\nThe "if" statement is used for conditional execution:\n\n if_stmt ::= "if" expression ":" suite\n ( "elif" expression ":" suite )*\n ["else" ":" suite]\n\nIt selects exactly one of the suites by evaluating the expressions one\nby one until one is found to be true (see section *Boolean operations*\nfor the definition of true and false); then that suite is executed\n(and no other part of the "if" statement is executed or evaluated).\nIf all expressions are false, the suite of the "else" clause, if\npresent, is executed.\n', - 'exceptions': u'\nExceptions\n**********\n\nExceptions are a means of breaking out of the normal flow of control\nof a code block in order to handle errors or other exceptional\nconditions. An exception is *raised* at the point where the error is\ndetected; it may be *handled* by the surrounding code block or by any\ncode block that directly or indirectly invoked the code block where\nthe error occurred.\n\nThe Python interpreter raises an exception when it detects a run-time\nerror (such as division by zero). A Python program can also\nexplicitly raise an exception with the "raise" statement. Exception\nhandlers are specified with the "try" ... "except" statement. The\n"finally" clause of such a statement can be used to specify cleanup\ncode which does not handle the exception, but is executed whether an\nexception occurred or not in the preceding code.\n\nPython uses the "termination" model of error handling: an exception\nhandler can find out what happened and continue execution at an outer\nlevel, but it cannot repair the cause of the error and retry the\nfailing operation (except by re-entering the offending piece of code\nfrom the top).\n\nWhen an exception is not handled at all, the interpreter terminates\nexecution of the program, or returns to its interactive main loop. In\neither case, it prints a stack backtrace, except when the exception is\n"SystemExit".\n\nExceptions are identified by class instances. The "except" clause is\nselected depending on the class of the instance: it must reference the\nclass of the instance or a base class thereof. The instance can be\nreceived by the handler and can carry additional information about the\nexceptional condition.\n\nNote: Exception messages are not part of the Python API. Their\n contents may change from one version of Python to the next without\n warning and should not be relied on by code which will run under\n multiple versions of the interpreter.\n\nSee also the description of the "try" statement in section *The try\nstatement* and "raise" statement in section *The raise statement*.\n\n-[ Footnotes ]-\n\n[1] This limitation occurs because the code that is executed by\n these operations is not available at the time the module is\n compiled.\n', - 'execmodel': u'\nExecution model\n***************\n\n\nStructure of a programm\n=======================\n\nA Python program is constructed from code blocks. A *block* is a piece\nof Python program text that is executed as a unit. The following are\nblocks: a module, a function body, and a class definition. Each\ncommand typed interactively is a block. A script file (a file given\nas standard input to the interpreter or specified as a command line\nargument to the interpreter) is a code block. A script command (a\ncommand specified on the interpreter command line with the \'**-c**\'\noption) is a code block. The string argument passed to the built-in\nfunctions "eval()" and "exec()" is a code block.\n\nA code block is executed in an *execution frame*. A frame contains\nsome administrative information (used for debugging) and determines\nwhere and how execution continues after the code block\'s execution has\ncompleted.\n\n\nNaming and binding\n==================\n\n\nBinding of names\n----------------\n\n*Names* refer to objects. Names are introduced by name binding\noperations.\n\nThe following constructs bind names: formal parameters to functions,\n"import" statements, class and function definitions (these bind the\nclass or function name in the defining block), and targets that are\nidentifiers if occurring in an assignment, "for" loop header, or after\n"as" in a "with" statement or "except" clause. The "import" statement\nof the form "from ... import *" binds all names defined in the\nimported module, except those beginning with an underscore. This form\nmay only be used at the module level.\n\nA target occurring in a "del" statement is also considered bound for\nthis purpose (though the actual semantics are to unbind the name).\n\nEach assignment or import statement occurs within a block defined by a\nclass or function definition or at the module level (the top-level\ncode block).\n\nIf a name is bound in a block, it is a local variable of that block,\nunless declared as "nonlocal" or "global". If a name is bound at the\nmodule level, it is a global variable. (The variables of the module\ncode block are local and global.) If a variable is used in a code\nblock but not defined there, it is a *free variable*.\n\nEach occurrence of a name in the program text refers to the *binding*\nof that name established by the following name resolution rules.\n\n\nResolution of names\n-------------------\n\nA *scope* defines the visibility of a name within a block. If a local\nvariable is defined in a block, its scope includes that block. If the\ndefinition occurs in a function block, the scope extends to any blocks\ncontained within the defining one, unless a contained block introduces\na different binding for the name.\n\nWhen a name is used in a code block, it is resolved using the nearest\nenclosing scope. The set of all such scopes visible to a code block\nis called the block\'s *environment*.\n\nWhen a name is not found at all, a "NameError" exception is raised. If\nthe current scope is a function scope, and the name refers to a local\nvariable that has not yet been bound to a value at the point where the\nname is used, an "UnboundLocalError" exception is raised.\n"UnboundLocalError" is a subclass of "NameError".\n\nIf a name binding operation occurs anywhere within a code block, all\nuses of the name within the block are treated as references to the\ncurrent block. This can lead to errors when a name is used within a\nblock before it is bound. This rule is subtle. Python lacks\ndeclarations and allows name binding operations to occur anywhere\nwithin a code block. The local variables of a code block can be\ndetermined by scanning the entire text of the block for name binding\noperations.\n\nIf the "global" statement occurs within a block, all uses of the name\nspecified in the statement refer to the binding of that name in the\ntop-level namespace. Names are resolved in the top-level namespace by\nsearching the global namespace, i.e. the namespace of the module\ncontaining the code block, and the builtins namespace, the namespace\nof the module "builtins". The global namespace is searched first. If\nthe name is not found there, the builtins namespace is searched. The\n"global" statement must precede all uses of the name.\n\nThe "global" statement has the same scope as a name binding operation\nin the same block. If the nearest enclosing scope for a free variable\ncontains a global statement, the free variable is treated as a global.\n\nThe "nonlocal" statement causes corresponding names to refer to\npreviously bound variables in the nearest enclosing function scope.\n"SyntaxError" is raised at compile time if the given name does not\nexist in any enclosing function scope.\n\nThe namespace for a module is automatically created the first time a\nmodule is imported. The main module for a script is always called\n"__main__".\n\nClass definition blocks and arguments to "exec()" and "eval()" are\nspecial in the context of name resolution. A class definition is an\nexecutable statement that may use and define names. These references\nfollow the normal rules for name resolution with an exception that\nunbound local variables are looked up in the global namespace. The\nnamespace of the class definition becomes the attribute dictionary of\nthe class. The scope of names defined in a class block is limited to\nthe class block; it does not extend to the code blocks of methods --\nthis includes comprehensions and generator expressions since they are\nimplemented using a function scope. This means that the following\nwill fail:\n\n class A:\n a = 42\n b = list(a + i for i in range(10))\n\n\nBuiltins and restricted execution\n---------------------------------\n\nThe builtins namespace associated with the execution of a code block\nis actually found by looking up the name "__builtins__" in its global\nnamespace; this should be a dictionary or a module (in the latter case\nthe module\'s dictionary is used). By default, when in the "__main__"\nmodule, "__builtins__" is the built-in module "builtins"; when in any\nother module, "__builtins__" is an alias for the dictionary of the\n"builtins" module itself. "__builtins__" can be set to a user-created\ndictionary to create a weak form of restricted execution.\n\n**CPython implementation detail:** Users should not touch\n"__builtins__"; it is strictly an implementation detail. Users\nwanting to override values in the builtins namespace should "import"\nthe "builtins" module and modify its attributes appropriately.\n\n\nInteraction with dynamic features\n---------------------------------\n\nName resolution of free variables occurs at runtime, not at compile\ntime. This means that the following code will print 42:\n\n i = 10\n def f():\n print(i)\n i = 42\n f()\n\nThere are several cases where Python statements are illegal when used\nin conjunction with nested scopes that contain free variables.\n\nIf a variable is referenced in an enclosing scope, it is illegal to\ndelete the name. An error will be reported at compile time.\n\nThe "eval()" and "exec()" functions do not have access to the full\nenvironment for resolving names. Names may be resolved in the local\nand global namespaces of the caller. Free variables are not resolved\nin the nearest enclosing namespace, but in the global namespace. [1]\nThe "exec()" and "eval()" functions have optional arguments to\noverride the global and local namespace. If only one namespace is\nspecified, it is used for both.\n\n\nExceptions\n==========\n\nExceptions are a means of breaking out of the normal flow of control\nof a code block in order to handle errors or other exceptional\nconditions. An exception is *raised* at the point where the error is\ndetected; it may be *handled* by the surrounding code block or by any\ncode block that directly or indirectly invoked the code block where\nthe error occurred.\n\nThe Python interpreter raises an exception when it detects a run-time\nerror (such as division by zero). A Python program can also\nexplicitly raise an exception with the "raise" statement. Exception\nhandlers are specified with the "try" ... "except" statement. The\n"finally" clause of such a statement can be used to specify cleanup\ncode which does not handle the exception, but is executed whether an\nexception occurred or not in the preceding code.\n\nPython uses the "termination" model of error handling: an exception\nhandler can find out what happened and continue execution at an outer\nlevel, but it cannot repair the cause of the error and retry the\nfailing operation (except by re-entering the offending piece of code\nfrom the top).\n\nWhen an exception is not handled at all, the interpreter terminates\nexecution of the program, or returns to its interactive main loop. In\neither case, it prints a stack backtrace, except when the exception is\n"SystemExit".\n\nExceptions are identified by class instances. The "except" clause is\nselected depending on the class of the instance: it must reference the\nclass of the instance or a base class thereof. The instance can be\nreceived by the handler and can carry additional information about the\nexceptional condition.\n\nNote: Exception messages are not part of the Python API. Their\n contents may change from one version of Python to the next without\n warning and should not be relied on by code which will run under\n multiple versions of the interpreter.\n\nSee also the description of the "try" statement in section *The try\nstatement* and "raise" statement in section *The raise statement*.\n\n-[ Footnotes ]-\n\n[1] This limitation occurs because the code that is executed by\n these operations is not available at the time the module is\n compiled.\n', - 'exprlists': u'\nExpression lists\n****************\n\n expression_list ::= expression ( "," expression )* [","]\n\nAn expression list containing at least one comma yields a tuple. The\nlength of the tuple is the number of expressions in the list. The\nexpressions are evaluated from left to right.\n\nThe trailing comma is required only to create a single tuple (a.k.a. a\n*singleton*); it is optional in all other cases. A single expression\nwithout a trailing comma doesn\'t create a tuple, but rather yields the\nvalue of that expression. (To create an empty tuple, use an empty pair\nof parentheses: "()".)\n', - 'floating': u'\nFloating point literals\n***********************\n\nFloating point literals are described by the following lexical\ndefinitions:\n\n floatnumber ::= pointfloat | exponentfloat\n pointfloat ::= [intpart] fraction | intpart "."\n exponentfloat ::= (intpart | pointfloat) exponent\n intpart ::= digit+\n fraction ::= "." digit+\n exponent ::= ("e" | "E") ["+" | "-"] digit+\n\nNote that the integer and exponent parts are always interpreted using\nradix 10. For example, "077e010" is legal, and denotes the same number\nas "77e10". The allowed range of floating point literals is\nimplementation-dependent. Some examples of floating point literals:\n\n 3.14 10. .001 1e100 3.14e-10 0e0\n\nNote that numeric literals do not include a sign; a phrase like "-1"\nis actually an expression composed of the unary operator "-" and the\nliteral "1".\n', - 'for': u'\nThe "for" statement\n*******************\n\nThe "for" statement is used to iterate over the elements of a sequence\n(such as a string, tuple or list) or other iterable object:\n\n for_stmt ::= "for" target_list "in" expression_list ":" suite\n ["else" ":" suite]\n\nThe expression list is evaluated once; it should yield an iterable\nobject. An iterator is created for the result of the\n"expression_list". The suite is then executed once for each item\nprovided by the iterator, in the order returned by the iterator. Each\nitem in turn is assigned to the target list using the standard rules\nfor assignments (see *Assignment statements*), and then the suite is\nexecuted. When the items are exhausted (which is immediately when the\nsequence is empty or an iterator raises a "StopIteration" exception),\nthe suite in the "else" clause, if present, is executed, and the loop\nterminates.\n\nA "break" statement executed in the first suite terminates the loop\nwithout executing the "else" clause\'s suite. A "continue" statement\nexecuted in the first suite skips the rest of the suite and continues\nwith the next item, or with the "else" clause if there is no next\nitem.\n\nThe for-loop makes assignments to the variables(s) in the target list.\nThis overwrites all previous assignments to those variables including\nthose made in the suite of the for-loop:\n\n for i in range(10):\n print(i)\n i = 5 # this will not affect the for-loop\n # because i will be overwritten with the next\n # index in the range\n\nNames in the target list are not deleted when the loop is finished,\nbut if the sequence is empty, they will not have been assigned to at\nall by the loop. Hint: the built-in function "range()" returns an\niterator of integers suitable to emulate the effect of Pascal\'s "for i\n:= a to b do"; e.g., "list(range(3))" returns the list "[0, 1, 2]".\n\nNote: There is a subtlety when the sequence is being modified by the\n loop (this can only occur for mutable sequences, i.e. lists). An\n internal counter is used to keep track of which item is used next,\n and this is incremented on each iteration. When this counter has\n reached the length of the sequence the loop terminates. This means\n that if the suite deletes the current (or a previous) item from the\n sequence, the next item will be skipped (since it gets the index of\n the current item which has already been treated). Likewise, if the\n suite inserts an item in the sequence before the current item, the\n current item will be treated again the next time through the loop.\n This can lead to nasty bugs that can be avoided by making a\n temporary copy using a slice of the whole sequence, e.g.,\n\n for x in a[:]:\n if x < 0: a.remove(x)\n', - 'formatstrings': u'\nFormat String Syntax\n********************\n\nThe "str.format()" method and the "Formatter" class share the same\nsyntax for format strings (although in the case of "Formatter",\nsubclasses can define their own format string syntax).\n\nFormat strings contain "replacement fields" surrounded by curly braces\n"{}". Anything that is not contained in braces is considered literal\ntext, which is copied unchanged to the output. If you need to include\na brace character in the literal text, it can be escaped by doubling:\n"{{" and "}}".\n\nThe grammar for a replacement field is as follows:\n\n replacement_field ::= "{" [field_name] ["!" conversion] [":" format_spec] "}"\n field_name ::= arg_name ("." attribute_name | "[" element_index "]")*\n arg_name ::= [identifier | integer]\n attribute_name ::= identifier\n element_index ::= integer | index_string\n index_string ::= +\n conversion ::= "r" | "s" | "a"\n format_spec ::= \n\nIn less formal terms, the replacement field can start with a\n*field_name* that specifies the object whose value is to be formatted\nand inserted into the output instead of the replacement field. The\n*field_name* is optionally followed by a *conversion* field, which is\npreceded by an exclamation point "\'!\'", and a *format_spec*, which is\npreceded by a colon "\':\'". These specify a non-default format for the\nreplacement value.\n\nSee also the *Format Specification Mini-Language* section.\n\nThe *field_name* itself begins with an *arg_name* that is either a\nnumber or a keyword. If it\'s a number, it refers to a positional\nargument, and if it\'s a keyword, it refers to a named keyword\nargument. If the numerical arg_names in a format string are 0, 1, 2,\n... in sequence, they can all be omitted (not just some) and the\nnumbers 0, 1, 2, ... will be automatically inserted in that order.\nBecause *arg_name* is not quote-delimited, it is not possible to\nspecify arbitrary dictionary keys (e.g., the strings "\'10\'" or\n"\':-]\'") within a format string. The *arg_name* can be followed by any\nnumber of index or attribute expressions. An expression of the form\n"\'.name\'" selects the named attribute using "getattr()", while an\nexpression of the form "\'[index]\'" does an index lookup using\n"__getitem__()".\n\nChanged in version 3.1: The positional argument specifiers can be\nomitted, so "\'{} {}\'" is equivalent to "\'{0} {1}\'".\n\nSome simple format string examples:\n\n "First, thou shalt count to {0}" # References first positional argument\n "Bring me a {}" # Implicitly references the first positional argument\n "From {} to {}" # Same as "From {0} to {1}"\n "My quest is {name}" # References keyword argument \'name\'\n "Weight in tons {0.weight}" # \'weight\' attribute of first positional arg\n "Units destroyed: {players[0]}" # First element of keyword argument \'players\'.\n\nThe *conversion* field causes a type coercion before formatting.\nNormally, the job of formatting a value is done by the "__format__()"\nmethod of the value itself. However, in some cases it is desirable to\nforce a type to be formatted as a string, overriding its own\ndefinition of formatting. By converting the value to a string before\ncalling "__format__()", the normal formatting logic is bypassed.\n\nThree conversion flags are currently supported: "\'!s\'" which calls\n"str()" on the value, "\'!r\'" which calls "repr()" and "\'!a\'" which\ncalls "ascii()".\n\nSome examples:\n\n "Harold\'s a clever {0!s}" # Calls str() on the argument first\n "Bring out the holy {name!r}" # Calls repr() on the argument first\n "More {!a}" # Calls ascii() on the argument first\n\nThe *format_spec* field contains a specification of how the value\nshould be presented, including such details as field width, alignment,\npadding, decimal precision and so on. Each value type can define its\nown "formatting mini-language" or interpretation of the *format_spec*.\n\nMost built-in types support a common formatting mini-language, which\nis described in the next section.\n\nA *format_spec* field can also include nested replacement fields\nwithin it. These nested replacement fields can contain only a field\nname; conversion flags and format specifications are not allowed. The\nreplacement fields within the format_spec are substituted before the\n*format_spec* string is interpreted. This allows the formatting of a\nvalue to be dynamically specified.\n\nSee the *Format examples* section for some examples.\n\n\nFormat Specification Mini-Language\n==================================\n\n"Format specifications" are used within replacement fields contained\nwithin a format string to define how individual values are presented\n(see *Format String Syntax*). They can also be passed directly to the\nbuilt-in "format()" function. Each formattable type may define how\nthe format specification is to be interpreted.\n\nMost built-in types implement the following options for format\nspecifications, although some of the formatting options are only\nsupported by the numeric types.\n\nA general convention is that an empty format string ("""") produces\nthe same result as if you had called "str()" on the value. A non-empty\nformat string typically modifies the result.\n\nThe general form of a *standard format specifier* is:\n\n format_spec ::= [[fill]align][sign][#][0][width][,][.precision][type]\n fill ::= \n align ::= "<" | ">" | "=" | "^"\n sign ::= "+" | "-" | " "\n width ::= integer\n precision ::= integer\n type ::= "b" | "c" | "d" | "e" | "E" | "f" | "F" | "g" | "G" | "n" | "o" | "s" | "x" | "X" | "%"\n\nIf a valid *align* value is specified, it can be preceded by a *fill*\ncharacter that can be any character and defaults to a space if\nomitted. Note that it is not possible to use "{" and "}" as *fill*\nchar while using the "str.format()" method; this limitation however\ndoesn\'t affect the "format()" function.\n\nThe meaning of the various alignment options is as follows:\n\n +-----------+------------------------------------------------------------+\n | Option | Meaning |\n +===========+============================================================+\n | "\'<\'" | Forces the field to be left-aligned within the available |\n | | space (this is the default for most objects). |\n +-----------+------------------------------------------------------------+\n | "\'>\'" | Forces the field to be right-aligned within the available |\n | | space (this is the default for numbers). |\n +-----------+------------------------------------------------------------+\n | "\'=\'" | Forces the padding to be placed after the sign (if any) |\n | | but before the digits. This is used for printing fields |\n | | in the form \'+000000120\'. This alignment option is only |\n | | valid for numeric types. |\n +-----------+------------------------------------------------------------+\n | "\'^\'" | Forces the field to be centered within the available |\n | | space. |\n +-----------+------------------------------------------------------------+\n\nNote that unless a minimum field width is defined, the field width\nwill always be the same size as the data to fill it, so that the\nalignment option has no meaning in this case.\n\nThe *sign* option is only valid for number types, and can be one of\nthe following:\n\n +-----------+------------------------------------------------------------+\n | Option | Meaning |\n +===========+============================================================+\n | "\'+\'" | indicates that a sign should be used for both positive as |\n | | well as negative numbers. |\n +-----------+------------------------------------------------------------+\n | "\'-\'" | indicates that a sign should be used only for negative |\n | | numbers (this is the default behavior). |\n +-----------+------------------------------------------------------------+\n | space | indicates that a leading space should be used on positive |\n | | numbers, and a minus sign on negative numbers. |\n +-----------+------------------------------------------------------------+\n\nThe "\'#\'" option causes the "alternate form" to be used for the\nconversion. The alternate form is defined differently for different\ntypes. This option is only valid for integer, float, complex and\nDecimal types. For integers, when binary, octal, or hexadecimal output\nis used, this option adds the prefix respective "\'0b\'", "\'0o\'", or\n"\'0x\'" to the output value. For floats, complex and Decimal the\nalternate form causes the result of the conversion to always contain a\ndecimal-point character, even if no digits follow it. Normally, a\ndecimal-point character appears in the result of these conversions\nonly if a digit follows it. In addition, for "\'g\'" and "\'G\'"\nconversions, trailing zeros are not removed from the result.\n\nThe "\',\'" option signals the use of a comma for a thousands separator.\nFor a locale aware separator, use the "\'n\'" integer presentation type\ninstead.\n\nChanged in version 3.1: Added the "\',\'" option (see also **PEP 378**).\n\n*width* is a decimal integer defining the minimum field width. If not\nspecified, then the field width will be determined by the content.\n\nPreceding the *width* field by a zero ("\'0\'") character enables sign-\naware zero-padding for numeric types. This is equivalent to a *fill*\ncharacter of "\'0\'" with an *alignment* type of "\'=\'".\n\nThe *precision* is a decimal number indicating how many digits should\nbe displayed after the decimal point for a floating point value\nformatted with "\'f\'" and "\'F\'", or before and after the decimal point\nfor a floating point value formatted with "\'g\'" or "\'G\'". For non-\nnumber types the field indicates the maximum field size - in other\nwords, how many characters will be used from the field content. The\n*precision* is not allowed for integer values.\n\nFinally, the *type* determines how the data should be presented.\n\nThe available string presentation types are:\n\n +-----------+------------------------------------------------------------+\n | Type | Meaning |\n +===========+============================================================+\n | "\'s\'" | String format. This is the default type for strings and |\n | | may be omitted. |\n +-----------+------------------------------------------------------------+\n | None | The same as "\'s\'". |\n +-----------+------------------------------------------------------------+\n\nThe available integer presentation types are:\n\n +-----------+------------------------------------------------------------+\n | Type | Meaning |\n +===========+============================================================+\n | "\'b\'" | Binary format. Outputs the number in base 2. |\n +-----------+------------------------------------------------------------+\n | "\'c\'" | Character. Converts the integer to the corresponding |\n | | unicode character before printing. |\n +-----------+------------------------------------------------------------+\n | "\'d\'" | Decimal Integer. Outputs the number in base 10. |\n +-----------+------------------------------------------------------------+\n | "\'o\'" | Octal format. Outputs the number in base 8. |\n +-----------+------------------------------------------------------------+\n | "\'x\'" | Hex format. Outputs the number in base 16, using lower- |\n | | case letters for the digits above 9. |\n +-----------+------------------------------------------------------------+\n | "\'X\'" | Hex format. Outputs the number in base 16, using upper- |\n | | case letters for the digits above 9. |\n +-----------+------------------------------------------------------------+\n | "\'n\'" | Number. This is the same as "\'d\'", except that it uses the |\n | | current locale setting to insert the appropriate number |\n | | separator characters. |\n +-----------+------------------------------------------------------------+\n | None | The same as "\'d\'". |\n +-----------+------------------------------------------------------------+\n\nIn addition to the above presentation types, integers can be formatted\nwith the floating point presentation types listed below (except "\'n\'"\nand None). When doing so, "float()" is used to convert the integer to\na floating point number before formatting.\n\nThe available presentation types for floating point and decimal values\nare:\n\n +-----------+------------------------------------------------------------+\n | Type | Meaning |\n +===========+============================================================+\n | "\'e\'" | Exponent notation. Prints the number in scientific |\n | | notation using the letter \'e\' to indicate the exponent. |\n | | The default precision is "6". |\n +-----------+------------------------------------------------------------+\n | "\'E\'" | Exponent notation. Same as "\'e\'" except it uses an upper |\n | | case \'E\' as the separator character. |\n +-----------+------------------------------------------------------------+\n | "\'f\'" | Fixed point. Displays the number as a fixed-point number. |\n | | The default precision is "6". |\n +-----------+------------------------------------------------------------+\n | "\'F\'" | Fixed point. Same as "\'f\'", but converts "nan" to "NAN" |\n | | and "inf" to "INF". |\n +-----------+------------------------------------------------------------+\n | "\'g\'" | General format. For a given precision "p >= 1", this |\n | | rounds the number to "p" significant digits and then |\n | | formats the result in either fixed-point format or in |\n | | scientific notation, depending on its magnitude. The |\n | | precise rules are as follows: suppose that the result |\n | | formatted with presentation type "\'e\'" and precision "p-1" |\n | | would have exponent "exp". Then if "-4 <= exp < p", the |\n | | number is formatted with presentation type "\'f\'" and |\n | | precision "p-1-exp". Otherwise, the number is formatted |\n | | with presentation type "\'e\'" and precision "p-1". In both |\n | | cases insignificant trailing zeros are removed from the |\n | | significand, and the decimal point is also removed if |\n | | there are no remaining digits following it. Positive and |\n | | negative infinity, positive and negative zero, and nans, |\n | | are formatted as "inf", "-inf", "0", "-0" and "nan" |\n | | respectively, regardless of the precision. A precision of |\n | | "0" is treated as equivalent to a precision of "1". The |\n | | default precision is "6". |\n +-----------+------------------------------------------------------------+\n | "\'G\'" | General format. Same as "\'g\'" except switches to "\'E\'" if |\n | | the number gets too large. The representations of infinity |\n | | and NaN are uppercased, too. |\n +-----------+------------------------------------------------------------+\n | "\'n\'" | Number. This is the same as "\'g\'", except that it uses the |\n | | current locale setting to insert the appropriate number |\n | | separator characters. |\n +-----------+------------------------------------------------------------+\n | "\'%\'" | Percentage. Multiplies the number by 100 and displays in |\n | | fixed ("\'f\'") format, followed by a percent sign. |\n +-----------+------------------------------------------------------------+\n | None | Similar to "\'g\'", except that fixed-point notation, when |\n | | used, has at least one digit past the decimal point. The |\n | | default precision is as high as needed to represent the |\n | | particular value. The overall effect is to match the |\n | | output of "str()" as altered by the other format |\n | | modifiers. |\n +-----------+------------------------------------------------------------+\n\n\nFormat examples\n===============\n\nThis section contains examples of the new format syntax and comparison\nwith the old "%"-formatting.\n\nIn most of the cases the syntax is similar to the old "%"-formatting,\nwith the addition of the "{}" and with ":" used instead of "%". For\nexample, "\'%03.2f\'" can be translated to "\'{:03.2f}\'".\n\nThe new format syntax also supports new and different options, shown\nin the follow examples.\n\nAccessing arguments by position:\n\n >>> \'{0}, {1}, {2}\'.format(\'a\', \'b\', \'c\')\n \'a, b, c\'\n >>> \'{}, {}, {}\'.format(\'a\', \'b\', \'c\') # 3.1+ only\n \'a, b, c\'\n >>> \'{2}, {1}, {0}\'.format(\'a\', \'b\', \'c\')\n \'c, b, a\'\n >>> \'{2}, {1}, {0}\'.format(*\'abc\') # unpacking argument sequence\n \'c, b, a\'\n >>> \'{0}{1}{0}\'.format(\'abra\', \'cad\') # arguments\' indices can be repeated\n \'abracadabra\'\n\nAccessing arguments by name:\n\n >>> \'Coordinates: {latitude}, {longitude}\'.format(latitude=\'37.24N\', longitude=\'-115.81W\')\n \'Coordinates: 37.24N, -115.81W\'\n >>> coord = {\'latitude\': \'37.24N\', \'longitude\': \'-115.81W\'}\n >>> \'Coordinates: {latitude}, {longitude}\'.format(**coord)\n \'Coordinates: 37.24N, -115.81W\'\n\nAccessing arguments\' attributes:\n\n >>> c = 3-5j\n >>> (\'The complex number {0} is formed from the real part {0.real} \'\n ... \'and the imaginary part {0.imag}.\').format(c)\n \'The complex number (3-5j) is formed from the real part 3.0 and the imaginary part -5.0.\'\n >>> class Point:\n ... def __init__(self, x, y):\n ... self.x, self.y = x, y\n ... def __str__(self):\n ... return \'Point({self.x}, {self.y})\'.format(self=self)\n ...\n >>> str(Point(4, 2))\n \'Point(4, 2)\'\n\nAccessing arguments\' items:\n\n >>> coord = (3, 5)\n >>> \'X: {0[0]}; Y: {0[1]}\'.format(coord)\n \'X: 3; Y: 5\'\n\nReplacing "%s" and "%r":\n\n >>> "repr() shows quotes: {!r}; str() doesn\'t: {!s}".format(\'test1\', \'test2\')\n "repr() shows quotes: \'test1\'; str() doesn\'t: test2"\n\nAligning the text and specifying a width:\n\n >>> \'{:<30}\'.format(\'left aligned\')\n \'left aligned \'\n >>> \'{:>30}\'.format(\'right aligned\')\n \' right aligned\'\n >>> \'{:^30}\'.format(\'centered\')\n \' centered \'\n >>> \'{:*^30}\'.format(\'centered\') # use \'*\' as a fill char\n \'***********centered***********\'\n\nReplacing "%+f", "%-f", and "% f" and specifying a sign:\n\n >>> \'{:+f}; {:+f}\'.format(3.14, -3.14) # show it always\n \'+3.140000; -3.140000\'\n >>> \'{: f}; {: f}\'.format(3.14, -3.14) # show a space for positive numbers\n \' 3.140000; -3.140000\'\n >>> \'{:-f}; {:-f}\'.format(3.14, -3.14) # show only the minus -- same as \'{:f}; {:f}\'\n \'3.140000; -3.140000\'\n\nReplacing "%x" and "%o" and converting the value to different bases:\n\n >>> # format also supports binary numbers\n >>> "int: {0:d}; hex: {0:x}; oct: {0:o}; bin: {0:b}".format(42)\n \'int: 42; hex: 2a; oct: 52; bin: 101010\'\n >>> # with 0x, 0o, or 0b as prefix:\n >>> "int: {0:d}; hex: {0:#x}; oct: {0:#o}; bin: {0:#b}".format(42)\n \'int: 42; hex: 0x2a; oct: 0o52; bin: 0b101010\'\n\nUsing the comma as a thousands separator:\n\n >>> \'{:,}\'.format(1234567890)\n \'1,234,567,890\'\n\nExpressing a percentage:\n\n >>> points = 19\n >>> total = 22\n >>> \'Correct answers: {:.2%}\'.format(points/total)\n \'Correct answers: 86.36%\'\n\nUsing type-specific formatting:\n\n >>> import datetime\n >>> d = datetime.datetime(2010, 7, 4, 12, 15, 58)\n >>> \'{:%Y-%m-%d %H:%M:%S}\'.format(d)\n \'2010-07-04 12:15:58\'\n\nNesting arguments and more complex examples:\n\n >>> for align, text in zip(\'<^>\', [\'left\', \'center\', \'right\']):\n ... \'{0:{fill}{align}16}\'.format(text, fill=align, align=align)\n ...\n \'left<<<<<<<<<<<<\'\n \'^^^^^center^^^^^\'\n \'>>>>>>>>>>>right\'\n >>>\n >>> octets = [192, 168, 0, 1]\n >>> \'{:02X}{:02X}{:02X}{:02X}\'.format(*octets)\n \'C0A80001\'\n >>> int(_, 16)\n 3232235521\n >>>\n >>> width = 5\n >>> for num in range(5,12): #doctest: +NORMALIZE_WHITESPACE\n ... for base in \'dXob\':\n ... print(\'{0:{width}{base}}\'.format(num, base=base, width=width), end=\' \')\n ... print()\n ...\n 5 5 5 101\n 6 6 6 110\n 7 7 7 111\n 8 8 10 1000\n 9 9 11 1001\n 10 A 12 1010\n 11 B 13 1011\n', - 'function': u'\nFunction definitions\n********************\n\nA function definition defines a user-defined function object (see\nsection *The standard type hierarchy*):\n\n funcdef ::= [decorators] "def" funcname "(" [parameter_list] ")" ["->" expression] ":" suite\n decorators ::= decorator+\n decorator ::= "@" dotted_name ["(" [parameter_list [","]] ")"] NEWLINE\n dotted_name ::= identifier ("." identifier)*\n parameter_list ::= (defparameter ",")*\n | "*" [parameter] ("," defparameter)* ["," "**" parameter]\n | "**" parameter\n | defparameter [","] )\n parameter ::= identifier [":" expression]\n defparameter ::= parameter ["=" expression]\n funcname ::= identifier\n\nA function definition is an executable statement. Its execution binds\nthe function name in the current local namespace to a function object\n(a wrapper around the executable code for the function). This\nfunction object contains a reference to the current global namespace\nas the global namespace to be used when the function is called.\n\nThe function definition does not execute the function body; this gets\nexecuted only when the function is called. [3]\n\nA function definition may be wrapped by one or more *decorator*\nexpressions. Decorator expressions are evaluated when the function is\ndefined, in the scope that contains the function definition. The\nresult must be a callable, which is invoked with the function object\nas the only argument. The returned value is bound to the function name\ninstead of the function object. Multiple decorators are applied in\nnested fashion. For example, the following code\n\n @f1(arg)\n @f2\n def func(): pass\n\nis equivalent to\n\n def func(): pass\n func = f1(arg)(f2(func))\n\nWhen one or more *parameters* have the form *parameter* "="\n*expression*, the function is said to have "default parameter values."\nFor a parameter with a default value, the corresponding *argument* may\nbe omitted from a call, in which case the parameter\'s default value is\nsubstituted. If a parameter has a default value, all following\nparameters up until the ""*"" must also have a default value --- this\nis a syntactic restriction that is not expressed by the grammar.\n\n**Default parameter values are evaluated from left to right when the\nfunction definition is executed.** This means that the expression is\nevaluated once, when the function is defined, and that the same "pre-\ncomputed" value is used for each call. This is especially important\nto understand when a default parameter is a mutable object, such as a\nlist or a dictionary: if the function modifies the object (e.g. by\nappending an item to a list), the default value is in effect modified.\nThis is generally not what was intended. A way around this is to use\n"None" as the default, and explicitly test for it in the body of the\nfunction, e.g.:\n\n def whats_on_the_telly(penguin=None):\n if penguin is None:\n penguin = []\n penguin.append("property of the zoo")\n return penguin\n\nFunction call semantics are described in more detail in section\n*Calls*. A function call always assigns values to all parameters\nmentioned in the parameter list, either from position arguments, from\nkeyword arguments, or from default values. If the form\n""*identifier"" is present, it is initialized to a tuple receiving any\nexcess positional parameters, defaulting to the empty tuple. If the\nform ""**identifier"" is present, it is initialized to a new\ndictionary receiving any excess keyword arguments, defaulting to a new\nempty dictionary. Parameters after ""*"" or ""*identifier"" are\nkeyword-only parameters and may only be passed used keyword arguments.\n\nParameters may have annotations of the form "": expression"" following\nthe parameter name. Any parameter may have an annotation even those\nof the form "*identifier" or "**identifier". Functions may have\n"return" annotation of the form ""-> expression"" after the parameter\nlist. These annotations can be any valid Python expression and are\nevaluated when the function definition is executed. Annotations may\nbe evaluated in a different order than they appear in the source code.\nThe presence of annotations does not change the semantics of a\nfunction. The annotation values are available as values of a\ndictionary keyed by the parameters\' names in the "__annotations__"\nattribute of the function object.\n\nIt is also possible to create anonymous functions (functions not bound\nto a name), for immediate use in expressions. This uses lambda\nexpressions, described in section *Lambdas*. Note that the lambda\nexpression is merely a shorthand for a simplified function definition;\na function defined in a ""def"" statement can be passed around or\nassigned to another name just like a function defined by a lambda\nexpression. The ""def"" form is actually more powerful since it\nallows the execution of multiple statements and annotations.\n\n**Programmer\'s note:** Functions are first-class objects. A ""def""\nstatement executed inside a function definition defines a local\nfunction that can be returned or passed around. Free variables used\nin the nested function can access the local variables of the function\ncontaining the def. See section *Naming and binding* for details.\n\nSee also: **PEP 3107** - Function Annotations\n\n The original specification for function annotations.\n', - 'global': u'\nThe "global" statement\n**********************\n\n global_stmt ::= "global" identifier ("," identifier)*\n\nThe "global" statement is a declaration which holds for the entire\ncurrent code block. It means that the listed identifiers are to be\ninterpreted as globals. It would be impossible to assign to a global\nvariable without "global", although free variables may refer to\nglobals without being declared global.\n\nNames listed in a "global" statement must not be used in the same code\nblock textually preceding that "global" statement.\n\nNames listed in a "global" statement must not be defined as formal\nparameters or in a "for" loop control target, "class" definition,\nfunction definition, or "import" statement.\n\n**CPython implementation detail:** The current implementation does not\nenforce the two restrictions, but programs should not abuse this\nfreedom, as future implementations may enforce them or silently change\nthe meaning of the program.\n\n**Programmer\'s note:** the "global" is a directive to the parser. It\napplies only to code parsed at the same time as the "global"\nstatement. In particular, a "global" statement contained in a string\nor code object supplied to the built-in "exec()" function does not\naffect the code block *containing* the function call, and code\ncontained in such a string is unaffected by "global" statements in the\ncode containing the function call. The same applies to the "eval()"\nand "compile()" functions.\n', - 'id-classes': u'\nReserved classes of identifiers\n*******************************\n\nCertain classes of identifiers (besides keywords) have special\nmeanings. These classes are identified by the patterns of leading and\ntrailing underscore characters:\n\n"_*"\n Not imported by "from module import *". The special identifier "_"\n is used in the interactive interpreter to store the result of the\n last evaluation; it is stored in the "builtins" module. When not\n in interactive mode, "_" has no special meaning and is not defined.\n See section *The import statement*.\n\n Note: The name "_" is often used in conjunction with\n internationalization; refer to the documentation for the\n "gettext" module for more information on this convention.\n\n"__*__"\n System-defined names. These names are defined by the interpreter\n and its implementation (including the standard library). Current\n system names are discussed in the *Special method names* section\n and elsewhere. More will likely be defined in future versions of\n Python. *Any* use of "__*__" names, in any context, that does not\n follow explicitly documented use, is subject to breakage without\n warning.\n\n"__*"\n Class-private names. Names in this category, when used within the\n context of a class definition, are re-written to use a mangled form\n to help avoid name clashes between "private" attributes of base and\n derived classes. See section *Identifiers (Names)*.\n', - 'identifiers': u'\nIdentifiers and keywords\n************************\n\nIdentifiers (also referred to as *names*) are described by the\nfollowing lexical definitions.\n\nThe syntax of identifiers in Python is based on the Unicode standard\nannex UAX-31, with elaboration and changes as defined below; see also\n**PEP 3131** for further details.\n\nWithin the ASCII range (U+0001..U+007F), the valid characters for\nidentifiers are the same as in Python 2.x: the uppercase and lowercase\nletters "A" through "Z", the underscore "_" and, except for the first\ncharacter, the digits "0" through "9".\n\nPython 3.0 introduces additional characters from outside the ASCII\nrange (see **PEP 3131**). For these characters, the classification\nuses the version of the Unicode Character Database as included in the\n"unicodedata" module.\n\nIdentifiers are unlimited in length. Case is significant.\n\n identifier ::= xid_start xid_continue*\n id_start ::= \n id_continue ::= \n xid_start ::= \n xid_continue ::= \n\nThe Unicode category codes mentioned above stand for:\n\n* *Lu* - uppercase letters\n\n* *Ll* - lowercase letters\n\n* *Lt* - titlecase letters\n\n* *Lm* - modifier letters\n\n* *Lo* - other letters\n\n* *Nl* - letter numbers\n\n* *Mn* - nonspacing marks\n\n* *Mc* - spacing combining marks\n\n* *Nd* - decimal numbers\n\n* *Pc* - connector punctuations\n\n* *Other_ID_Start* - explicit list of characters in PropList.txt to\n support backwards compatibility\n\n* *Other_ID_Continue* - likewise\n\nAll identifiers are converted into the normal form NFKC while parsing;\ncomparison of identifiers is based on NFKC.\n\nA non-normative HTML file listing all valid identifier characters for\nUnicode 4.1 can be found at http://www.dcl.hpi.uni-\npotsdam.de/home/loewis/table-3131.html.\n\n\nKeywords\n========\n\nThe following identifiers are used as reserved words, or *keywords* of\nthe language, and cannot be used as ordinary identifiers. They must\nbe spelled exactly as written here:\n\n False class finally is return\n None continue for lambda try\n True def from nonlocal while\n and del global not with\n as elif if or yield\n assert else import pass\n break except in raise\n\n\nReserved classes of identifiers\n===============================\n\nCertain classes of identifiers (besides keywords) have special\nmeanings. These classes are identified by the patterns of leading and\ntrailing underscore characters:\n\n"_*"\n Not imported by "from module import *". The special identifier "_"\n is used in the interactive interpreter to store the result of the\n last evaluation; it is stored in the "builtins" module. When not\n in interactive mode, "_" has no special meaning and is not defined.\n See section *The import statement*.\n\n Note: The name "_" is often used in conjunction with\n internationalization; refer to the documentation for the\n "gettext" module for more information on this convention.\n\n"__*__"\n System-defined names. These names are defined by the interpreter\n and its implementation (including the standard library). Current\n system names are discussed in the *Special method names* section\n and elsewhere. More will likely be defined in future versions of\n Python. *Any* use of "__*__" names, in any context, that does not\n follow explicitly documented use, is subject to breakage without\n warning.\n\n"__*"\n Class-private names. Names in this category, when used within the\n context of a class definition, are re-written to use a mangled form\n to help avoid name clashes between "private" attributes of base and\n derived classes. See section *Identifiers (Names)*.\n', - 'if': u'\nThe "if" statement\n******************\n\nThe "if" statement is used for conditional execution:\n\n if_stmt ::= "if" expression ":" suite\n ( "elif" expression ":" suite )*\n ["else" ":" suite]\n\nIt selects exactly one of the suites by evaluating the expressions one\nby one until one is found to be true (see section *Boolean operations*\nfor the definition of true and false); then that suite is executed\n(and no other part of the "if" statement is executed or evaluated).\nIf all expressions are false, the suite of the "else" clause, if\npresent, is executed.\n', - 'imaginary': u'\nImaginary literals\n******************\n\nImaginary literals are described by the following lexical definitions:\n\n imagnumber ::= (floatnumber | intpart) ("j" | "J")\n\nAn imaginary literal yields a complex number with a real part of 0.0.\nComplex numbers are represented as a pair of floating point numbers\nand have the same restrictions on their range. To create a complex\nnumber with a nonzero real part, add a floating point number to it,\ne.g., "(3+4j)". Some examples of imaginary literals:\n\n 3.14j 10.j 10j .001j 1e100j 3.14e-10j\n', - 'import': u'\nThe "import" statement\n**********************\n\n import_stmt ::= "import" module ["as" name] ( "," module ["as" name] )*\n | "from" relative_module "import" identifier ["as" name]\n ( "," identifier ["as" name] )*\n | "from" relative_module "import" "(" identifier ["as" name]\n ( "," identifier ["as" name] )* [","] ")"\n | "from" module "import" "*"\n module ::= (identifier ".")* identifier\n relative_module ::= "."* module | "."+\n name ::= identifier\n\nThe basic import statement (no "from" clause) is executed in two\nsteps:\n\n1. find a module, loading and initializing it if necessary\n\n2. define a name or names in the local namespace for the scope\n where the "import" statement occurs.\n\nWhen the statement contains multiple clauses (separated by commas) the\ntwo steps are carried out separately for each clause, just as though\nthe clauses had been separated out into individiual import statements.\n\nThe details of the first step, finding and loading modules are\ndescribed in greater detail in the section on the *import system*,\nwhich also describes the various types of packages and modules that\ncan be imported, as well as all the hooks that can be used to\ncustomize the import system. Note that failures in this step may\nindicate either that the module could not be located, *or* that an\nerror occurred while initializing the module, which includes execution\nof the module\'s code.\n\nIf the requested module is retrieved successfully, it will be made\navailable in the local namespace in one of three ways:\n\n* If the module name is followed by "as", then the name following\n "as" is bound directly to the imported module.\n\n* If no other name is specified, and the module being imported is a\n top level module, the module\'s name is bound in the local namespace\n as a reference to the imported module\n\n* If the module being imported is *not* a top level module, then the\n name of the top level package that contains the module is bound in\n the local namespace as a reference to the top level package. The\n imported module must be accessed using its full qualified name\n rather than directly\n\nThe "from" form uses a slightly more complex process:\n\n1. find the module specified in the "from" clause, loading and\n initializing it if necessary;\n\n2. for each of the identifiers specified in the "import" clauses:\n\n 1. check if the imported module has an attribute by that name\n\n 2. if not, attempt to import a submodule with that name and then\n check the imported module again for that attribute\n\n 3. if the attribute is not found, "ImportError" is raised.\n\n 4. otherwise, a reference to that value is stored in the local\n namespace, using the name in the "as" clause if it is present,\n otherwise using the attribute name\n\nExamples:\n\n import foo # foo imported and bound locally\n import foo.bar.baz # foo.bar.baz imported, foo bound locally\n import foo.bar.baz as fbb # foo.bar.baz imported and bound as fbb\n from foo.bar import baz # foo.bar.baz imported and bound as baz\n from foo import attr # foo imported and foo.attr bound as attr\n\nIf the list of identifiers is replaced by a star ("\'*\'"), all public\nnames defined in the module are bound in the local namespace for the\nscope where the "import" statement occurs.\n\nThe *public names* defined by a module are determined by checking the\nmodule\'s namespace for a variable named "__all__"; if defined, it must\nbe a sequence of strings which are names defined or imported by that\nmodule. The names given in "__all__" are all considered public and\nare required to exist. If "__all__" is not defined, the set of public\nnames includes all names found in the module\'s namespace which do not\nbegin with an underscore character ("\'_\'"). "__all__" should contain\nthe entire public API. It is intended to avoid accidentally exporting\nitems that are not part of the API (such as library modules which were\nimported and used within the module).\n\nThe wild card form of import --- "from module import *" --- is only\nallowed at the module level. Attempting to use it in class or\nfunction definitions will raise a "SyntaxError".\n\nWhen specifying what module to import you do not have to specify the\nabsolute name of the module. When a module or package is contained\nwithin another package it is possible to make a relative import within\nthe same top package without having to mention the package name. By\nusing leading dots in the specified module or package after "from" you\ncan specify how high to traverse up the current package hierarchy\nwithout specifying exact names. One leading dot means the current\npackage where the module making the import exists. Two dots means up\none package level. Three dots is up two levels, etc. So if you execute\n"from . import mod" from a module in the "pkg" package then you will\nend up importing "pkg.mod". If you execute "from ..subpkg2 import mod"\nfrom within "pkg.subpkg1" you will import "pkg.subpkg2.mod". The\nspecification for relative imports is contained within **PEP 328**.\n\n"importlib.import_module()" is provided to support applications that\ndetermine dynamically the modules to be loaded.\n\n\nFuture statements\n=================\n\nA *future statement* is a directive to the compiler that a particular\nmodule should be compiled using syntax or semantics that will be\navailable in a specified future release of Python where the feature\nbecomes standard.\n\nThe future statement is intended to ease migration to future versions\nof Python that introduce incompatible changes to the language. It\nallows use of the new features on a per-module basis before the\nrelease in which the feature becomes standard.\n\n future_statement ::= "from" "__future__" "import" feature ["as" name]\n ("," feature ["as" name])*\n | "from" "__future__" "import" "(" feature ["as" name]\n ("," feature ["as" name])* [","] ")"\n feature ::= identifier\n name ::= identifier\n\nA future statement must appear near the top of the module. The only\nlines that can appear before a future statement are:\n\n* the module docstring (if any),\n\n* comments,\n\n* blank lines, and\n\n* other future statements.\n\nThe features recognized by Python 3.0 are "absolute_import",\n"division", "generators", "unicode_literals", "print_function",\n"nested_scopes" and "with_statement". They are all redundant because\nthey are always enabled, and only kept for backwards compatibility.\n\nA future statement is recognized and treated specially at compile\ntime: Changes to the semantics of core constructs are often\nimplemented by generating different code. It may even be the case\nthat a new feature introduces new incompatible syntax (such as a new\nreserved word), in which case the compiler may need to parse the\nmodule differently. Such decisions cannot be pushed off until\nruntime.\n\nFor any given release, the compiler knows which feature names have\nbeen defined, and raises a compile-time error if a future statement\ncontains a feature not known to it.\n\nThe direct runtime semantics are the same as for any import statement:\nthere is a standard module "__future__", described later, and it will\nbe imported in the usual way at the time the future statement is\nexecuted.\n\nThe interesting runtime semantics depend on the specific feature\nenabled by the future statement.\n\nNote that there is nothing special about the statement:\n\n import __future__ [as name]\n\nThat is not a future statement; it\'s an ordinary import statement with\nno special semantics or syntax restrictions.\n\nCode compiled by calls to the built-in functions "exec()" and\n"compile()" that occur in a module "M" containing a future statement\nwill, by default, use the new syntax or semantics associated with the\nfuture statement. This can be controlled by optional arguments to\n"compile()" --- see the documentation of that function for details.\n\nA future statement typed at an interactive interpreter prompt will\ntake effect for the rest of the interpreter session. If an\ninterpreter is started with the *-i* option, is passed a script name\nto execute, and the script includes a future statement, it will be in\neffect in the interactive session started after the script is\nexecuted.\n\nSee also: **PEP 236** - Back to the __future__\n\n The original proposal for the __future__ mechanism.\n', - 'in': u'\nComparisons\n***********\n\nUnlike C, all comparison operations in Python have the same priority,\nwhich is lower than that of any arithmetic, shifting or bitwise\noperation. Also unlike C, expressions like "a < b < c" have the\ninterpretation that is conventional in mathematics:\n\n comparison ::= or_expr ( comp_operator or_expr )*\n comp_operator ::= "<" | ">" | "==" | ">=" | "<=" | "!="\n | "is" ["not"] | ["not"] "in"\n\nComparisons yield boolean values: "True" or "False".\n\nComparisons can be chained arbitrarily, e.g., "x < y <= z" is\nequivalent to "x < y and y <= z", except that "y" is evaluated only\nonce (but in both cases "z" is not evaluated at all when "x < y" is\nfound to be false).\n\nFormally, if *a*, *b*, *c*, ..., *y*, *z* are expressions and *op1*,\n*op2*, ..., *opN* are comparison operators, then "a op1 b op2 c ... y\nopN z" is equivalent to "a op1 b and b op2 c and ... y opN z", except\nthat each expression is evaluated at most once.\n\nNote that "a op1 b op2 c" doesn\'t imply any kind of comparison between\n*a* and *c*, so that, e.g., "x < y > z" is perfectly legal (though\nperhaps not pretty).\n\nThe operators "<", ">", "==", ">=", "<=", and "!=" compare the values\nof two objects. The objects need not have the same type. If both are\nnumbers, they are converted to a common type. Otherwise, the "==" and\n"!=" operators *always* consider objects of different types to be\nunequal, while the "<", ">", ">=" and "<=" operators raise a\n"TypeError" when comparing objects of different types that do not\nimplement these operators for the given pair of types. You can\ncontrol comparison behavior of objects of non-built-in types by\ndefining rich comparison methods like "__gt__()", described in section\n*Basic customization*.\n\nComparison of objects of the same type depends on the type:\n\n* Numbers are compared arithmetically.\n\n* The values "float(\'NaN\')" and "Decimal(\'NaN\')" are special. They\n are identical to themselves, "x is x" but are not equal to\n themselves, "x != x". Additionally, comparing any value to a\n not-a-number value will return "False". For example, both "3 <\n float(\'NaN\')" and "float(\'NaN\') < 3" will return "False".\n\n* Bytes objects are compared lexicographically using the numeric\n values of their elements.\n\n* Strings are compared lexicographically using the numeric\n equivalents (the result of the built-in function "ord()") of their\n characters. [3] String and bytes object can\'t be compared!\n\n* Tuples and lists are compared lexicographically using comparison\n of corresponding elements. This means that to compare equal, each\n element must compare equal and the two sequences must be of the same\n type and have the same length.\n\n If not equal, the sequences are ordered the same as their first\n differing elements. For example, "[1,2,x] <= [1,2,y]" has the same\n value as "x <= y". If the corresponding element does not exist, the\n shorter sequence is ordered first (for example, "[1,2] < [1,2,3]").\n\n* Mappings (dictionaries) compare equal if and only if they have the\n same "(key, value)" pairs. Order comparisons "(\'<\', \'<=\', \'>=\',\n \'>\')" raise "TypeError".\n\n* Sets and frozensets define comparison operators to mean subset and\n superset tests. Those relations do not define total orderings (the\n two sets "{1,2}" and "{2,3}" are not equal, nor subsets of one\n another, nor supersets of one another). Accordingly, sets are not\n appropriate arguments for functions which depend on total ordering.\n For example, "min()", "max()", and "sorted()" produce undefined\n results given a list of sets as inputs.\n\n* Most other objects of built-in types compare unequal unless they\n are the same object; the choice whether one object is considered\n smaller or larger than another one is made arbitrarily but\n consistently within one execution of a program.\n\nComparison of objects of differing types depends on whether either of\nthe types provide explicit support for the comparison. Most numeric\ntypes can be compared with one another. When cross-type comparison is\nnot supported, the comparison method returns "NotImplemented".\n\nThe operators "in" and "not in" test for membership. "x in s"\nevaluates to true if *x* is a member of *s*, and false otherwise. "x\nnot in s" returns the negation of "x in s". All built-in sequences\nand set types support this as well as dictionary, for which "in" tests\nwhether the dictionary has a given key. For container types such as\nlist, tuple, set, frozenset, dict, or collections.deque, the\nexpression "x in y" is equivalent to "any(x is e or x == e for e in\ny)".\n\nFor the string and bytes types, "x in y" is true if and only if *x* is\na substring of *y*. An equivalent test is "y.find(x) != -1". Empty\nstrings are always considered to be a substring of any other string,\nso """ in "abc"" will return "True".\n\nFor user-defined classes which define the "__contains__()" method, "x\nin y" is true if and only if "y.__contains__(x)" is true.\n\nFor user-defined classes which do not define "__contains__()" but do\ndefine "__iter__()", "x in y" is true if some value "z" with "x == z"\nis produced while iterating over "y". If an exception is raised\nduring the iteration, it is as if "in" raised that exception.\n\nLastly, the old-style iteration protocol is tried: if a class defines\n"__getitem__()", "x in y" is true if and only if there is a non-\nnegative integer index *i* such that "x == y[i]", and all lower\ninteger indices do not raise "IndexError" exception. (If any other\nexception is raised, it is as if "in" raised that exception).\n\nThe operator "not in" is defined to have the inverse true value of\n"in".\n\nThe operators "is" and "is not" test for object identity: "x is y" is\ntrue if and only if *x* and *y* are the same object. "x is not y"\nyields the inverse truth value. [4]\n', - 'integers': u'\nInteger literals\n****************\n\nInteger literals are described by the following lexical definitions:\n\n integer ::= decimalinteger | octinteger | hexinteger | bininteger\n decimalinteger ::= nonzerodigit digit* | "0"+\n nonzerodigit ::= "1"..."9"\n digit ::= "0"..."9"\n octinteger ::= "0" ("o" | "O") octdigit+\n hexinteger ::= "0" ("x" | "X") hexdigit+\n bininteger ::= "0" ("b" | "B") bindigit+\n octdigit ::= "0"..."7"\n hexdigit ::= digit | "a"..."f" | "A"..."F"\n bindigit ::= "0" | "1"\n\nThere is no limit for the length of integer literals apart from what\ncan be stored in available memory.\n\nNote that leading zeros in a non-zero decimal number are not allowed.\nThis is for disambiguation with C-style octal literals, which Python\nused before version 3.0.\n\nSome examples of integer literals:\n\n 7 2147483647 0o177 0b100110111\n 3 79228162514264337593543950336 0o377 0xdeadbeef\n', - 'lambda': u'\nLambdas\n*******\n\n lambda_expr ::= "lambda" [parameter_list]: expression\n lambda_expr_nocond ::= "lambda" [parameter_list]: expression_nocond\n\nLambda expressions (sometimes called lambda forms) are used to create\nanonymous functions. The expression "lambda arguments: expression"\nyields a function object. The unnamed object behaves like a function\nobject defined with\n\n def (arguments):\n return expression\n\nSee section *Function definitions* for the syntax of parameter lists.\nNote that functions created with lambda expressions cannot contain\nstatements or annotations.\n', - 'lists': u'\nList displays\n*************\n\nA list display is a possibly empty series of expressions enclosed in\nsquare brackets:\n\n list_display ::= "[" [expression_list | comprehension] "]"\n\nA list display yields a new list object, the contents being specified\nby either a list of expressions or a comprehension. When a comma-\nseparated list of expressions is supplied, its elements are evaluated\nfrom left to right and placed into the list object in that order.\nWhen a comprehension is supplied, the list is constructed from the\nelements resulting from the comprehension.\n', - 'naming': u'\nNaming and binding\n******************\n\n\nBinding of names\n================\n\n*Names* refer to objects. Names are introduced by name binding\noperations.\n\nThe following constructs bind names: formal parameters to functions,\n"import" statements, class and function definitions (these bind the\nclass or function name in the defining block), and targets that are\nidentifiers if occurring in an assignment, "for" loop header, or after\n"as" in a "with" statement or "except" clause. The "import" statement\nof the form "from ... import *" binds all names defined in the\nimported module, except those beginning with an underscore. This form\nmay only be used at the module level.\n\nA target occurring in a "del" statement is also considered bound for\nthis purpose (though the actual semantics are to unbind the name).\n\nEach assignment or import statement occurs within a block defined by a\nclass or function definition or at the module level (the top-level\ncode block).\n\nIf a name is bound in a block, it is a local variable of that block,\nunless declared as "nonlocal" or "global". If a name is bound at the\nmodule level, it is a global variable. (The variables of the module\ncode block are local and global.) If a variable is used in a code\nblock but not defined there, it is a *free variable*.\n\nEach occurrence of a name in the program text refers to the *binding*\nof that name established by the following name resolution rules.\n\n\nResolution of names\n===================\n\nA *scope* defines the visibility of a name within a block. If a local\nvariable is defined in a block, its scope includes that block. If the\ndefinition occurs in a function block, the scope extends to any blocks\ncontained within the defining one, unless a contained block introduces\na different binding for the name.\n\nWhen a name is used in a code block, it is resolved using the nearest\nenclosing scope. The set of all such scopes visible to a code block\nis called the block\'s *environment*.\n\nWhen a name is not found at all, a "NameError" exception is raised. If\nthe current scope is a function scope, and the name refers to a local\nvariable that has not yet been bound to a value at the point where the\nname is used, an "UnboundLocalError" exception is raised.\n"UnboundLocalError" is a subclass of "NameError".\n\nIf a name binding operation occurs anywhere within a code block, all\nuses of the name within the block are treated as references to the\ncurrent block. This can lead to errors when a name is used within a\nblock before it is bound. This rule is subtle. Python lacks\ndeclarations and allows name binding operations to occur anywhere\nwithin a code block. The local variables of a code block can be\ndetermined by scanning the entire text of the block for name binding\noperations.\n\nIf the "global" statement occurs within a block, all uses of the name\nspecified in the statement refer to the binding of that name in the\ntop-level namespace. Names are resolved in the top-level namespace by\nsearching the global namespace, i.e. the namespace of the module\ncontaining the code block, and the builtins namespace, the namespace\nof the module "builtins". The global namespace is searched first. If\nthe name is not found there, the builtins namespace is searched. The\n"global" statement must precede all uses of the name.\n\nThe "global" statement has the same scope as a name binding operation\nin the same block. If the nearest enclosing scope for a free variable\ncontains a global statement, the free variable is treated as a global.\n\nThe "nonlocal" statement causes corresponding names to refer to\npreviously bound variables in the nearest enclosing function scope.\n"SyntaxError" is raised at compile time if the given name does not\nexist in any enclosing function scope.\n\nThe namespace for a module is automatically created the first time a\nmodule is imported. The main module for a script is always called\n"__main__".\n\nClass definition blocks and arguments to "exec()" and "eval()" are\nspecial in the context of name resolution. A class definition is an\nexecutable statement that may use and define names. These references\nfollow the normal rules for name resolution with an exception that\nunbound local variables are looked up in the global namespace. The\nnamespace of the class definition becomes the attribute dictionary of\nthe class. The scope of names defined in a class block is limited to\nthe class block; it does not extend to the code blocks of methods --\nthis includes comprehensions and generator expressions since they are\nimplemented using a function scope. This means that the following\nwill fail:\n\n class A:\n a = 42\n b = list(a + i for i in range(10))\n\n\nBuiltins and restricted execution\n=================================\n\nThe builtins namespace associated with the execution of a code block\nis actually found by looking up the name "__builtins__" in its global\nnamespace; this should be a dictionary or a module (in the latter case\nthe module\'s dictionary is used). By default, when in the "__main__"\nmodule, "__builtins__" is the built-in module "builtins"; when in any\nother module, "__builtins__" is an alias for the dictionary of the\n"builtins" module itself. "__builtins__" can be set to a user-created\ndictionary to create a weak form of restricted execution.\n\n**CPython implementation detail:** Users should not touch\n"__builtins__"; it is strictly an implementation detail. Users\nwanting to override values in the builtins namespace should "import"\nthe "builtins" module and modify its attributes appropriately.\n\n\nInteraction with dynamic features\n=================================\n\nName resolution of free variables occurs at runtime, not at compile\ntime. This means that the following code will print 42:\n\n i = 10\n def f():\n print(i)\n i = 42\n f()\n\nThere are several cases where Python statements are illegal when used\nin conjunction with nested scopes that contain free variables.\n\nIf a variable is referenced in an enclosing scope, it is illegal to\ndelete the name. An error will be reported at compile time.\n\nThe "eval()" and "exec()" functions do not have access to the full\nenvironment for resolving names. Names may be resolved in the local\nand global namespaces of the caller. Free variables are not resolved\nin the nearest enclosing namespace, but in the global namespace. [1]\nThe "exec()" and "eval()" functions have optional arguments to\noverride the global and local namespace. If only one namespace is\nspecified, it is used for both.\n', - 'nonlocal': u'\nThe "nonlocal" statement\n************************\n\n nonlocal_stmt ::= "nonlocal" identifier ("," identifier)*\n\nThe "nonlocal" statement causes the listed identifiers to refer to\npreviously bound variables in the nearest enclosing scope excluding\nglobals. This is important because the default behavior for binding is\nto search the local namespace first. The statement allows\nencapsulated code to rebind variables outside of the local scope\nbesides the global (module) scope.\n\nNames listed in a "nonlocal" statement, unlike those listed in a\n"global" statement, must refer to pre-existing bindings in an\nenclosing scope (the scope in which a new binding should be created\ncannot be determined unambiguously).\n\nNames listed in a "nonlocal" statement must not collide with pre-\nexisting bindings in the local scope.\n\nSee also: **PEP 3104** - Access to Names in Outer Scopes\n\n The specification for the "nonlocal" statement.\n', - 'numbers': u'\nNumeric literals\n****************\n\nThere are three types of numeric literals: integers, floating point\nnumbers, and imaginary numbers. There are no complex literals\n(complex numbers can be formed by adding a real number and an\nimaginary number).\n\nNote that numeric literals do not include a sign; a phrase like "-1"\nis actually an expression composed of the unary operator \'"-"\' and the\nliteral "1".\n', - 'numeric-types': u'\nEmulating numeric types\n***********************\n\nThe following methods can be defined to emulate numeric objects.\nMethods corresponding to operations that are not supported by the\nparticular kind of number implemented (e.g., bitwise operations for\nnon-integral numbers) should be left undefined.\n\nobject.__add__(self, other)\nobject.__sub__(self, other)\nobject.__mul__(self, other)\nobject.__matmul__(self, other)\nobject.__truediv__(self, other)\nobject.__floordiv__(self, other)\nobject.__mod__(self, other)\nobject.__divmod__(self, other)\nobject.__pow__(self, other[, modulo])\nobject.__lshift__(self, other)\nobject.__rshift__(self, other)\nobject.__and__(self, other)\nobject.__xor__(self, other)\nobject.__or__(self, other)\n\n These methods are called to implement the binary arithmetic\n operations ("+", "-", "*", "@", "/", "//", "%", "divmod()",\n "pow()", "**", "<<", ">>", "&", "^", "|"). For instance, to\n evaluate the expression "x + y", where *x* is an instance of a\n class that has an "__add__()" method, "x.__add__(y)" is called.\n The "__divmod__()" method should be the equivalent to using\n "__floordiv__()" and "__mod__()"; it should not be related to\n "__truediv__()". Note that "__pow__()" should be defined to accept\n an optional third argument if the ternary version of the built-in\n "pow()" function is to be supported.\n\n If one of those methods does not support the operation with the\n supplied arguments, it should return "NotImplemented".\n\nobject.__radd__(self, other)\nobject.__rsub__(self, other)\nobject.__rmul__(self, other)\nobject.__rmatmul__(self, other)\nobject.__rtruediv__(self, other)\nobject.__rfloordiv__(self, other)\nobject.__rmod__(self, other)\nobject.__rdivmod__(self, other)\nobject.__rpow__(self, other)\nobject.__rlshift__(self, other)\nobject.__rrshift__(self, other)\nobject.__rand__(self, other)\nobject.__rxor__(self, other)\nobject.__ror__(self, other)\n\n These methods are called to implement the binary arithmetic\n operations ("+", "-", "*", "@", "/", "//", "%", "divmod()",\n "pow()", "**", "<<", ">>", "&", "^", "|") with reflected (swapped)\n operands. These functions are only called if the left operand does\n not support the corresponding operation and the operands are of\n different types. [2] For instance, to evaluate the expression "x -\n y", where *y* is an instance of a class that has an "__rsub__()"\n method, "y.__rsub__(x)" is called if "x.__sub__(y)" returns\n *NotImplemented*.\n\n Note that ternary "pow()" will not try calling "__rpow__()" (the\n coercion rules would become too complicated).\n\n Note: If the right operand\'s type is a subclass of the left\n operand\'s type and that subclass provides the reflected method\n for the operation, this method will be called before the left\n operand\'s non-reflected method. This behavior allows subclasses\n to override their ancestors\' operations.\n\nobject.__iadd__(self, other)\nobject.__isub__(self, other)\nobject.__imul__(self, other)\nobject.__imatmul__(self, other)\nobject.__itruediv__(self, other)\nobject.__ifloordiv__(self, other)\nobject.__imod__(self, other)\nobject.__ipow__(self, other[, modulo])\nobject.__ilshift__(self, other)\nobject.__irshift__(self, other)\nobject.__iand__(self, other)\nobject.__ixor__(self, other)\nobject.__ior__(self, other)\n\n These methods are called to implement the augmented arithmetic\n assignments ("+=", "-=", "*=", "@=", "/=", "//=", "%=", "**=",\n "<<=", ">>=", "&=", "^=", "|="). These methods should attempt to\n do the operation in-place (modifying *self*) and return the result\n (which could be, but does not have to be, *self*). If a specific\n method is not defined, the augmented assignment falls back to the\n normal methods. For instance, if *x* is an instance of a class\n with an "__iadd__()" method, "x += y" is equivalent to "x =\n x.__iadd__(y)" . Otherwise, "x.__add__(y)" and "y.__radd__(x)" are\n considered, as with the evaluation of "x + y". In certain\n situations, augmented assignment can result in unexpected errors\n (see *Why does a_tuple[i] += [\'item\'] raise an exception when the\n addition works?*), but this behavior is in fact part of the data\n model.\n\nobject.__neg__(self)\nobject.__pos__(self)\nobject.__abs__(self)\nobject.__invert__(self)\n\n Called to implement the unary arithmetic operations ("-", "+",\n "abs()" and "~").\n\nobject.__complex__(self)\nobject.__int__(self)\nobject.__float__(self)\nobject.__round__(self[, n])\n\n Called to implement the built-in functions "complex()", "int()",\n "float()" and "round()". Should return a value of the appropriate\n type.\n\nobject.__index__(self)\n\n Called to implement "operator.index()", and whenever Python needs\n to losslessly convert the numeric object to an integer object (such\n as in slicing, or in the built-in "bin()", "hex()" and "oct()"\n functions). Presence of this method indicates that the numeric\n object is an integer type. Must return an integer.\n\n Note: In order to have a coherent integer type class, when\n "__index__()" is defined "__int__()" should also be defined, and\n both should return the same value.\n', - 'objects': u'\nObjects, values and types\n*************************\n\n*Objects* are Python\'s abstraction for data. All data in a Python\nprogram is represented by objects or by relations between objects. (In\na sense, and in conformance to Von Neumann\'s model of a "stored\nprogram computer," code is also represented by objects.)\n\nEvery object has an identity, a type and a value. An object\'s\n*identity* never changes once it has been created; you may think of it\nas the object\'s address in memory. The \'"is"\' operator compares the\nidentity of two objects; the "id()" function returns an integer\nrepresenting its identity.\n\n**CPython implementation detail:** For CPython, "id(x)" is the memory\naddress where "x" is stored.\n\nAn object\'s type determines the operations that the object supports\n(e.g., "does it have a length?") and also defines the possible values\nfor objects of that type. The "type()" function returns an object\'s\ntype (which is an object itself). Like its identity, an object\'s\n*type* is also unchangeable. [1]\n\nThe *value* of some objects can change. Objects whose value can\nchange are said to be *mutable*; objects whose value is unchangeable\nonce they are created are called *immutable*. (The value of an\nimmutable container object that contains a reference to a mutable\nobject can change when the latter\'s value is changed; however the\ncontainer is still considered immutable, because the collection of\nobjects it contains cannot be changed. So, immutability is not\nstrictly the same as having an unchangeable value, it is more subtle.)\nAn object\'s mutability is determined by its type; for instance,\nnumbers, strings and tuples are immutable, while dictionaries and\nlists are mutable.\n\nObjects are never explicitly destroyed; however, when they become\nunreachable they may be garbage-collected. An implementation is\nallowed to postpone garbage collection or omit it altogether --- it is\na matter of implementation quality how garbage collection is\nimplemented, as long as no objects are collected that are still\nreachable.\n\n**CPython implementation detail:** CPython currently uses a reference-\ncounting scheme with (optional) delayed detection of cyclically linked\ngarbage, which collects most objects as soon as they become\nunreachable, but is not guaranteed to collect garbage containing\ncircular references. See the documentation of the "gc" module for\ninformation on controlling the collection of cyclic garbage. Other\nimplementations act differently and CPython may change. Do not depend\non immediate finalization of objects when they become unreachable (so\nyou should always close files explicitly).\n\nNote that the use of the implementation\'s tracing or debugging\nfacilities may keep objects alive that would normally be collectable.\nAlso note that catching an exception with a \'"try"..."except"\'\nstatement may keep objects alive.\n\nSome objects contain references to "external" resources such as open\nfiles or windows. It is understood that these resources are freed\nwhen the object is garbage-collected, but since garbage collection is\nnot guaranteed to happen, such objects also provide an explicit way to\nrelease the external resource, usually a "close()" method. Programs\nare strongly recommended to explicitly close such objects. The\n\'"try"..."finally"\' statement and the \'"with"\' statement provide\nconvenient ways to do this.\n\nSome objects contain references to other objects; these are called\n*containers*. Examples of containers are tuples, lists and\ndictionaries. The references are part of a container\'s value. In\nmost cases, when we talk about the value of a container, we imply the\nvalues, not the identities of the contained objects; however, when we\ntalk about the mutability of a container, only the identities of the\nimmediately contained objects are implied. So, if an immutable\ncontainer (like a tuple) contains a reference to a mutable object, its\nvalue changes if that mutable object is changed.\n\nTypes affect almost all aspects of object behavior. Even the\nimportance of object identity is affected in some sense: for immutable\ntypes, operations that compute new values may actually return a\nreference to any existing object with the same type and value, while\nfor mutable objects this is not allowed. E.g., after "a = 1; b = 1",\n"a" and "b" may or may not refer to the same object with the value\none, depending on the implementation, but after "c = []; d = []", "c"\nand "d" are guaranteed to refer to two different, unique, newly\ncreated empty lists. (Note that "c = d = []" assigns the same object\nto both "c" and "d".)\n', - 'operator-summary': u'\nOperator precedence\n*******************\n\nThe following table summarizes the operator precedence in Python, from\nlowest precedence (least binding) to highest precedence (most\nbinding). Operators in the same box have the same precedence. Unless\nthe syntax is explicitly given, operators are binary. Operators in\nthe same box group left to right (except for exponentiation, which\ngroups from right to left).\n\nNote that comparisons, membership tests, and identity tests, all have\nthe same precedence and have a left-to-right chaining feature as\ndescribed in the *Comparisons* section.\n\n+-------------------------------------------------+---------------------------------------+\n| Operator | Description |\n+=================================================+=======================================+\n| "lambda" | Lambda expression |\n+-------------------------------------------------+---------------------------------------+\n| "if" -- "else" | Conditional expression |\n+-------------------------------------------------+---------------------------------------+\n| "or" | Boolean OR |\n+-------------------------------------------------+---------------------------------------+\n| "and" | Boolean AND |\n+-------------------------------------------------+---------------------------------------+\n| "not" "x" | Boolean NOT |\n+-------------------------------------------------+---------------------------------------+\n| "in", "not in", "is", "is not", "<", "<=", ">", | Comparisons, including membership |\n| ">=", "!=", "==" | tests and identity tests |\n+-------------------------------------------------+---------------------------------------+\n| "|" | Bitwise OR |\n+-------------------------------------------------+---------------------------------------+\n| "^" | Bitwise XOR |\n+-------------------------------------------------+---------------------------------------+\n| "&" | Bitwise AND |\n+-------------------------------------------------+---------------------------------------+\n| "<<", ">>" | Shifts |\n+-------------------------------------------------+---------------------------------------+\n| "+", "-" | Addition and subtraction |\n+-------------------------------------------------+---------------------------------------+\n| "*", "@", "/", "//", "%" | Multiplication, matrix multiplication |\n| | division, remainder [5] |\n+-------------------------------------------------+---------------------------------------+\n| "+x", "-x", "~x" | Positive, negative, bitwise NOT |\n+-------------------------------------------------+---------------------------------------+\n| "**" | Exponentiation [6] |\n+-------------------------------------------------+---------------------------------------+\n| "await" "x" | Await expression |\n+-------------------------------------------------+---------------------------------------+\n| "x[index]", "x[index:index]", | Subscription, slicing, call, |\n| "x(arguments...)", "x.attribute" | attribute reference |\n+-------------------------------------------------+---------------------------------------+\n| "(expressions...)", "[expressions...]", "{key: | Binding or tuple display, list |\n| value...}", "{expressions...}" | display, dictionary display, set |\n| | display |\n+-------------------------------------------------+---------------------------------------+\n\n-[ Footnotes ]-\n\n[1] While "abs(x%y) < abs(y)" is true mathematically, for floats\n it may not be true numerically due to roundoff. For example, and\n assuming a platform on which a Python float is an IEEE 754 double-\n precision number, in order that "-1e-100 % 1e100" have the same\n sign as "1e100", the computed result is "-1e-100 + 1e100", which\n is numerically exactly equal to "1e100". The function\n "math.fmod()" returns a result whose sign matches the sign of the\n first argument instead, and so returns "-1e-100" in this case.\n Which approach is more appropriate depends on the application.\n\n[2] If x is very close to an exact integer multiple of y, it\'s\n possible for "x//y" to be one larger than "(x-x%y)//y" due to\n rounding. In such cases, Python returns the latter result, in\n order to preserve that "divmod(x,y)[0] * y + x % y" be very close\n to "x".\n\n[3] While comparisons between strings make sense at the byte\n level, they may be counter-intuitive to users. For example, the\n strings ""\\u00C7"" and ""\\u0043\\u0327"" compare differently, even\n though they both represent the same unicode character (LATIN\n CAPITAL LETTER C WITH CEDILLA). To compare strings in a human\n recognizable way, compare using "unicodedata.normalize()".\n\n[4] Due to automatic garbage-collection, free lists, and the\n dynamic nature of descriptors, you may notice seemingly unusual\n behaviour in certain uses of the "is" operator, like those\n involving comparisons between instance methods, or constants.\n Check their documentation for more info.\n\n[5] The "%" operator is also used for string formatting; the same\n precedence applies.\n\n[6] The power operator "**" binds less tightly than an arithmetic\n or bitwise unary operator on its right, that is, "2**-1" is "0.5".\n', - 'pass': u'\nThe "pass" statement\n********************\n\n pass_stmt ::= "pass"\n\n"pass" is a null operation --- when it is executed, nothing happens.\nIt is useful as a placeholder when a statement is required\nsyntactically, but no code needs to be executed, for example:\n\n def f(arg): pass # a function that does nothing (yet)\n\n class C: pass # a class with no methods (yet)\n', - 'power': u'\nThe power operator\n******************\n\nThe power operator binds more tightly than unary operators on its\nleft; it binds less tightly than unary operators on its right. The\nsyntax is:\n\n power ::= await ["**" u_expr]\n\nThus, in an unparenthesized sequence of power and unary operators, the\noperators are evaluated from right to left (this does not constrain\nthe evaluation order for the operands): "-1**2" results in "-1".\n\nThe power operator has the same semantics as the built-in "pow()"\nfunction, when called with two arguments: it yields its left argument\nraised to the power of its right argument. The numeric arguments are\nfirst converted to a common type, and the result is of that type.\n\nFor int operands, the result has the same type as the operands unless\nthe second argument is negative; in that case, all arguments are\nconverted to float and a float result is delivered. For example,\n"10**2" returns "100", but "10**-2" returns "0.01".\n\nRaising "0.0" to a negative power results in a "ZeroDivisionError".\nRaising a negative number to a fractional power results in a "complex"\nnumber. (In earlier versions it raised a "ValueError".)\n', - 'raise': u'\nThe "raise" statement\n*********************\n\n raise_stmt ::= "raise" [expression ["from" expression]]\n\nIf no expressions are present, "raise" re-raises the last exception\nthat was active in the current scope. If no exception is active in\nthe current scope, a "RuntimeError" exception is raised indicating\nthat this is an error.\n\nOtherwise, "raise" evaluates the first expression as the exception\nobject. It must be either a subclass or an instance of\n"BaseException". If it is a class, the exception instance will be\nobtained when needed by instantiating the class with no arguments.\n\nThe *type* of the exception is the exception instance\'s class, the\n*value* is the instance itself.\n\nA traceback object is normally created automatically when an exception\nis raised and attached to it as the "__traceback__" attribute, which\nis writable. You can create an exception and set your own traceback in\none step using the "with_traceback()" exception method (which returns\nthe same exception instance, with its traceback set to its argument),\nlike so:\n\n raise Exception("foo occurred").with_traceback(tracebackobj)\n\nThe "from" clause is used for exception chaining: if given, the second\n*expression* must be another exception class or instance, which will\nthen be attached to the raised exception as the "__cause__" attribute\n(which is writable). If the raised exception is not handled, both\nexceptions will be printed:\n\n >>> try:\n ... print(1 / 0)\n ... except Exception as exc:\n ... raise RuntimeError("Something bad happened") from exc\n ...\n Traceback (most recent call last):\n File "", line 2, in \n ZeroDivisionError: int division or modulo by zero\n\n The above exception was the direct cause of the following exception:\n\n Traceback (most recent call last):\n File "", line 4, in \n RuntimeError: Something bad happened\n\nA similar mechanism works implicitly if an exception is raised inside\nan exception handler or a "finally" clause: the previous exception is\nthen attached as the new exception\'s "__context__" attribute:\n\n >>> try:\n ... print(1 / 0)\n ... except:\n ... raise RuntimeError("Something bad happened")\n ...\n Traceback (most recent call last):\n File "", line 2, in \n ZeroDivisionError: int division or modulo by zero\n\n During handling of the above exception, another exception occurred:\n\n Traceback (most recent call last):\n File "", line 4, in \n RuntimeError: Something bad happened\n\nAdditional information on exceptions can be found in section\n*Exceptions*, and information about handling exceptions is in section\n*The try statement*.\n', - 'return': u'\nThe "return" statement\n**********************\n\n return_stmt ::= "return" [expression_list]\n\n"return" may only occur syntactically nested in a function definition,\nnot within a nested class definition.\n\nIf an expression list is present, it is evaluated, else "None" is\nsubstituted.\n\n"return" leaves the current function call with the expression list (or\n"None") as return value.\n\nWhen "return" passes control out of a "try" statement with a "finally"\nclause, that "finally" clause is executed before really leaving the\nfunction.\n\nIn a generator function, the "return" statement indicates that the\ngenerator is done and will cause "StopIteration" to be raised. The\nreturned value (if any) is used as an argument to construct\n"StopIteration" and becomes the "StopIteration.value" attribute.\n', - 'sequence-types': u'\nEmulating container types\n*************************\n\nThe following methods can be defined to implement container objects.\nContainers usually are sequences (such as lists or tuples) or mappings\n(like dictionaries), but can represent other containers as well. The\nfirst set of methods is used either to emulate a sequence or to\nemulate a mapping; the difference is that for a sequence, the\nallowable keys should be the integers *k* for which "0 <= k < N" where\n*N* is the length of the sequence, or slice objects, which define a\nrange of items. It is also recommended that mappings provide the\nmethods "keys()", "values()", "items()", "get()", "clear()",\n"setdefault()", "pop()", "popitem()", "copy()", and "update()"\nbehaving similar to those for Python\'s standard dictionary objects.\nThe "collections" module provides a "MutableMapping" abstract base\nclass to help create those methods from a base set of "__getitem__()",\n"__setitem__()", "__delitem__()", and "keys()". Mutable sequences\nshould provide methods "append()", "count()", "index()", "extend()",\n"insert()", "pop()", "remove()", "reverse()" and "sort()", like Python\nstandard list objects. Finally, sequence types should implement\naddition (meaning concatenation) and multiplication (meaning\nrepetition) by defining the methods "__add__()", "__radd__()",\n"__iadd__()", "__mul__()", "__rmul__()" and "__imul__()" described\nbelow; they should not define other numerical operators. It is\nrecommended that both mappings and sequences implement the\n"__contains__()" method to allow efficient use of the "in" operator;\nfor mappings, "in" should search the mapping\'s keys; for sequences, it\nshould search through the values. It is further recommended that both\nmappings and sequences implement the "__iter__()" method to allow\nefficient iteration through the container; for mappings, "__iter__()"\nshould be the same as "keys()"; for sequences, it should iterate\nthrough the values.\n\nobject.__len__(self)\n\n Called to implement the built-in function "len()". Should return\n the length of the object, an integer ">=" 0. Also, an object that\n doesn\'t define a "__bool__()" method and whose "__len__()" method\n returns zero is considered to be false in a Boolean context.\n\nobject.__length_hint__(self)\n\n Called to implement "operator.length_hint()". Should return an\n estimated length for the object (which may be greater or less than\n the actual length). The length must be an integer ">=" 0. This\n method is purely an optimization and is never required for\n correctness.\n\n New in version 3.4.\n\nNote: Slicing is done exclusively with the following three methods.\n A call like\n\n a[1:2] = b\n\n is translated to\n\n a[slice(1, 2, None)] = b\n\n and so forth. Missing slice items are always filled in with "None".\n\nobject.__getitem__(self, key)\n\n Called to implement evaluation of "self[key]". For sequence types,\n the accepted keys should be integers and slice objects. Note that\n the special interpretation of negative indexes (if the class wishes\n to emulate a sequence type) is up to the "__getitem__()" method. If\n *key* is of an inappropriate type, "TypeError" may be raised; if of\n a value outside the set of indexes for the sequence (after any\n special interpretation of negative values), "IndexError" should be\n raised. For mapping types, if *key* is missing (not in the\n container), "KeyError" should be raised.\n\n Note: "for" loops expect that an "IndexError" will be raised for\n illegal indexes to allow proper detection of the end of the\n sequence.\n\nobject.__missing__(self, key)\n\n Called by "dict"."__getitem__()" to implement "self[key]" for dict\n subclasses when key is not in the dictionary.\n\nobject.__setitem__(self, key, value)\n\n Called to implement assignment to "self[key]". Same note as for\n "__getitem__()". This should only be implemented for mappings if\n the objects support changes to the values for keys, or if new keys\n can be added, or for sequences if elements can be replaced. The\n same exceptions should be raised for improper *key* values as for\n the "__getitem__()" method.\n\nobject.__delitem__(self, key)\n\n Called to implement deletion of "self[key]". Same note as for\n "__getitem__()". This should only be implemented for mappings if\n the objects support removal of keys, or for sequences if elements\n can be removed from the sequence. The same exceptions should be\n raised for improper *key* values as for the "__getitem__()" method.\n\nobject.__iter__(self)\n\n This method is called when an iterator is required for a container.\n This method should return a new iterator object that can iterate\n over all the objects in the container. For mappings, it should\n iterate over the keys of the container.\n\n Iterator objects also need to implement this method; they are\n required to return themselves. For more information on iterator\n objects, see *Iterator Types*.\n\nobject.__reversed__(self)\n\n Called (if present) by the "reversed()" built-in to implement\n reverse iteration. It should return a new iterator object that\n iterates over all the objects in the container in reverse order.\n\n If the "__reversed__()" method is not provided, the "reversed()"\n built-in will fall back to using the sequence protocol ("__len__()"\n and "__getitem__()"). Objects that support the sequence protocol\n should only provide "__reversed__()" if they can provide an\n implementation that is more efficient than the one provided by\n "reversed()".\n\nThe membership test operators ("in" and "not in") are normally\nimplemented as an iteration through a sequence. However, container\nobjects can supply the following special method with a more efficient\nimplementation, which also does not require the object be a sequence.\n\nobject.__contains__(self, item)\n\n Called to implement membership test operators. Should return true\n if *item* is in *self*, false otherwise. For mapping objects, this\n should consider the keys of the mapping rather than the values or\n the key-item pairs.\n\n For objects that don\'t define "__contains__()", the membership test\n first tries iteration via "__iter__()", then the old sequence\n iteration protocol via "__getitem__()", see *this section in the\n language reference*.\n', - 'shifting': u'\nShifting operations\n*******************\n\nThe shifting operations have lower priority than the arithmetic\noperations:\n\n shift_expr ::= a_expr | shift_expr ( "<<" | ">>" ) a_expr\n\nThese operators accept integers as arguments. They shift the first\nargument to the left or right by the number of bits given by the\nsecond argument.\n\nA right shift by *n* bits is defined as floor division by "pow(2,n)".\nA left shift by *n* bits is defined as multiplication with "pow(2,n)".\n\nNote: In the current implementation, the right-hand operand is\n required to be at most "sys.maxsize". If the right-hand operand is\n larger than "sys.maxsize" an "OverflowError" exception is raised.\n', - 'slicings': u'\nSlicings\n********\n\nA slicing selects a range of items in a sequence object (e.g., a\nstring, tuple or list). Slicings may be used as expressions or as\ntargets in assignment or "del" statements. The syntax for a slicing:\n\n slicing ::= primary "[" slice_list "]"\n slice_list ::= slice_item ("," slice_item)* [","]\n slice_item ::= expression | proper_slice\n proper_slice ::= [lower_bound] ":" [upper_bound] [ ":" [stride] ]\n lower_bound ::= expression\n upper_bound ::= expression\n stride ::= expression\n\nThere is ambiguity in the formal syntax here: anything that looks like\nan expression list also looks like a slice list, so any subscription\ncan be interpreted as a slicing. Rather than further complicating the\nsyntax, this is disambiguated by defining that in this case the\ninterpretation as a subscription takes priority over the\ninterpretation as a slicing (this is the case if the slice list\ncontains no proper slice).\n\nThe semantics for a slicing are as follows. The primary is indexed\n(using the same "__getitem__()" method as normal subscription) with a\nkey that is constructed from the slice list, as follows. If the slice\nlist contains at least one comma, the key is a tuple containing the\nconversion of the slice items; otherwise, the conversion of the lone\nslice item is the key. The conversion of a slice item that is an\nexpression is that expression. The conversion of a proper slice is a\nslice object (see section *The standard type hierarchy*) whose\n"start", "stop" and "step" attributes are the values of the\nexpressions given as lower bound, upper bound and stride,\nrespectively, substituting "None" for missing expressions.\n', - 'specialattrs': u'\nSpecial Attributes\n******************\n\nThe implementation adds a few special read-only attributes to several\nobject types, where they are relevant. Some of these are not reported\nby the "dir()" built-in function.\n\nobject.__dict__\n\n A dictionary or other mapping object used to store an object\'s\n (writable) attributes.\n\ninstance.__class__\n\n The class to which a class instance belongs.\n\nclass.__bases__\n\n The tuple of base classes of a class object.\n\nclass.__name__\n\n The name of the class or type.\n\nclass.__qualname__\n\n The *qualified name* of the class or type.\n\n New in version 3.3.\n\nclass.__mro__\n\n This attribute is a tuple of classes that are considered when\n looking for base classes during method resolution.\n\nclass.mro()\n\n This method can be overridden by a metaclass to customize the\n method resolution order for its instances. It is called at class\n instantiation, and its result is stored in "__mro__".\n\nclass.__subclasses__()\n\n Each class keeps a list of weak references to its immediate\n subclasses. This method returns a list of all those references\n still alive. Example:\n\n >>> int.__subclasses__()\n []\n\n-[ Footnotes ]-\n\n[1] Additional information on these special methods may be found\n in the Python Reference Manual (*Basic customization*).\n\n[2] As a consequence, the list "[1, 2]" is considered equal to\n "[1.0, 2.0]", and similarly for tuples.\n\n[3] They must have since the parser can\'t tell the type of the\n operands.\n\n[4] Cased characters are those with general category property\n being one of "Lu" (Letter, uppercase), "Ll" (Letter, lowercase),\n or "Lt" (Letter, titlecase).\n\n[5] To format only a tuple you should therefore provide a\n singleton tuple whose only element is the tuple to be formatted.\n', - 'specialnames': u'\nSpecial method names\n********************\n\nA class can implement certain operations that are invoked by special\nsyntax (such as arithmetic operations or subscripting and slicing) by\ndefining methods with special names. This is Python\'s approach to\n*operator overloading*, allowing classes to define their own behavior\nwith respect to language operators. For instance, if a class defines\na method named "__getitem__()", and "x" is an instance of this class,\nthen "x[i]" is roughly equivalent to "type(x).__getitem__(x, i)".\nExcept where mentioned, attempts to execute an operation raise an\nexception when no appropriate method is defined (typically\n"AttributeError" or "TypeError").\n\nWhen implementing a class that emulates any built-in type, it is\nimportant that the emulation only be implemented to the degree that it\nmakes sense for the object being modelled. For example, some\nsequences may work well with retrieval of individual elements, but\nextracting a slice may not make sense. (One example of this is the\n"NodeList" interface in the W3C\'s Document Object Model.)\n\n\nBasic customization\n===================\n\nobject.__new__(cls[, ...])\n\n Called to create a new instance of class *cls*. "__new__()" is a\n static method (special-cased so you need not declare it as such)\n that takes the class of which an instance was requested as its\n first argument. The remaining arguments are those passed to the\n object constructor expression (the call to the class). The return\n value of "__new__()" should be the new object instance (usually an\n instance of *cls*).\n\n Typical implementations create a new instance of the class by\n invoking the superclass\'s "__new__()" method using\n "super(currentclass, cls).__new__(cls[, ...])" with appropriate\n arguments and then modifying the newly-created instance as\n necessary before returning it.\n\n If "__new__()" returns an instance of *cls*, then the new\n instance\'s "__init__()" method will be invoked like\n "__init__(self[, ...])", where *self* is the new instance and the\n remaining arguments are the same as were passed to "__new__()".\n\n If "__new__()" does not return an instance of *cls*, then the new\n instance\'s "__init__()" method will not be invoked.\n\n "__new__()" is intended mainly to allow subclasses of immutable\n types (like int, str, or tuple) to customize instance creation. It\n is also commonly overridden in custom metaclasses in order to\n customize class creation.\n\nobject.__init__(self[, ...])\n\n Called after the instance has been created (by "__new__()"), but\n before it is returned to the caller. The arguments are those\n passed to the class constructor expression. If a base class has an\n "__init__()" method, the derived class\'s "__init__()" method, if\n any, must explicitly call it to ensure proper initialization of the\n base class part of the instance; for example:\n "BaseClass.__init__(self, [args...])".\n\n Because "__new__()" and "__init__()" work together in constructing\n objects ("__new__()" to create it, and "__init__()" to customise\n it), no non-"None" value may be returned by "__init__()"; doing so\n will cause a "TypeError" to be raised at runtime.\n\nobject.__del__(self)\n\n Called when the instance is about to be destroyed. This is also\n called a destructor. If a base class has a "__del__()" method, the\n derived class\'s "__del__()" method, if any, must explicitly call it\n to ensure proper deletion of the base class part of the instance.\n Note that it is possible (though not recommended!) for the\n "__del__()" method to postpone destruction of the instance by\n creating a new reference to it. It may then be called at a later\n time when this new reference is deleted. It is not guaranteed that\n "__del__()" methods are called for objects that still exist when\n the interpreter exits.\n\n Note: "del x" doesn\'t directly call "x.__del__()" --- the former\n decrements the reference count for "x" by one, and the latter is\n only called when "x"\'s reference count reaches zero. Some common\n situations that may prevent the reference count of an object from\n going to zero include: circular references between objects (e.g.,\n a doubly-linked list or a tree data structure with parent and\n child pointers); a reference to the object on the stack frame of\n a function that caught an exception (the traceback stored in\n "sys.exc_info()[2]" keeps the stack frame alive); or a reference\n to the object on the stack frame that raised an unhandled\n exception in interactive mode (the traceback stored in\n "sys.last_traceback" keeps the stack frame alive). The first\n situation can only be remedied by explicitly breaking the cycles;\n the second can be resolved by freeing the reference to the\n traceback object when it is no longer useful, and the third can\n be resolved by storing "None" in "sys.last_traceback". Circular\n references which are garbage are detected and cleaned up when the\n cyclic garbage collector is enabled (it\'s on by default). Refer\n to the documentation for the "gc" module for more information\n about this topic.\n\n Warning: Due to the precarious circumstances under which\n "__del__()" methods are invoked, exceptions that occur during\n their execution are ignored, and a warning is printed to\n "sys.stderr" instead. Also, when "__del__()" is invoked in\n response to a module being deleted (e.g., when execution of the\n program is done), other globals referenced by the "__del__()"\n method may already have been deleted or in the process of being\n torn down (e.g. the import machinery shutting down). For this\n reason, "__del__()" methods should do the absolute minimum needed\n to maintain external invariants. Starting with version 1.5,\n Python guarantees that globals whose name begins with a single\n underscore are deleted from their module before other globals are\n deleted; if no other references to such globals exist, this may\n help in assuring that imported modules are still available at the\n time when the "__del__()" method is called.\n\nobject.__repr__(self)\n\n Called by the "repr()" built-in function to compute the "official"\n string representation of an object. If at all possible, this\n should look like a valid Python expression that could be used to\n recreate an object with the same value (given an appropriate\n environment). If this is not possible, a string of the form\n "<...some useful description...>" should be returned. The return\n value must be a string object. If a class defines "__repr__()" but\n not "__str__()", then "__repr__()" is also used when an "informal"\n string representation of instances of that class is required.\n\n This is typically used for debugging, so it is important that the\n representation is information-rich and unambiguous.\n\nobject.__str__(self)\n\n Called by "str(object)" and the built-in functions "format()" and\n "print()" to compute the "informal" or nicely printable string\n representation of an object. The return value must be a *string*\n object.\n\n This method differs from "object.__repr__()" in that there is no\n expectation that "__str__()" return a valid Python expression: a\n more convenient or concise representation can be used.\n\n The default implementation defined by the built-in type "object"\n calls "object.__repr__()".\n\nobject.__bytes__(self)\n\n Called by "bytes()" to compute a byte-string representation of an\n object. This should return a "bytes" object.\n\nobject.__format__(self, format_spec)\n\n Called by the "format()" built-in function (and by extension, the\n "str.format()" method of class "str") to produce a "formatted"\n string representation of an object. The "format_spec" argument is a\n string that contains a description of the formatting options\n desired. The interpretation of the "format_spec" argument is up to\n the type implementing "__format__()", however most classes will\n either delegate formatting to one of the built-in types, or use a\n similar formatting option syntax.\n\n See *Format Specification Mini-Language* for a description of the\n standard formatting syntax.\n\n The return value must be a string object.\n\n Changed in version 3.4: The __format__ method of "object" itself\n raises a "TypeError" if passed any non-empty string.\n\nobject.__lt__(self, other)\nobject.__le__(self, other)\nobject.__eq__(self, other)\nobject.__ne__(self, other)\nobject.__gt__(self, other)\nobject.__ge__(self, other)\n\n These are the so-called "rich comparison" methods. The\n correspondence between operator symbols and method names is as\n follows: "xy" calls\n "x.__gt__(y)", and "x>=y" calls "x.__ge__(y)".\n\n A rich comparison method may return the singleton "NotImplemented"\n if it does not implement the operation for a given pair of\n arguments. By convention, "False" and "True" are returned for a\n successful comparison. However, these methods can return any value,\n so if the comparison operator is used in a Boolean context (e.g.,\n in the condition of an "if" statement), Python will call "bool()"\n on the value to determine if the result is true or false.\n\n By default, "__ne__()" delegates to "__eq__()" and inverts the\n result unless it is "NotImplemented". There are no other implied\n relationships among the comparison operators, for example, the\n truth of "(x.__hash__".\n\n If a class that does not override "__eq__()" wishes to suppress\n hash support, it should include "__hash__ = None" in the class\n definition. A class which defines its own "__hash__()" that\n explicitly raises a "TypeError" would be incorrectly identified as\n hashable by an "isinstance(obj, collections.Hashable)" call.\n\n Note: By default, the "__hash__()" values of str, bytes and\n datetime objects are "salted" with an unpredictable random value.\n Although they remain constant within an individual Python\n process, they are not predictable between repeated invocations of\n Python.This is intended to provide protection against a denial-\n of-service caused by carefully-chosen inputs that exploit the\n worst case performance of a dict insertion, O(n^2) complexity.\n See http://www.ocert.org/advisories/ocert-2011-003.html for\n details.Changing hash values affects the iteration order of\n dicts, sets and other mappings. Python has never made guarantees\n about this ordering (and it typically varies between 32-bit and\n 64-bit builds).See also "PYTHONHASHSEED".\n\n Changed in version 3.3: Hash randomization is enabled by default.\n\nobject.__bool__(self)\n\n Called to implement truth value testing and the built-in operation\n "bool()"; should return "False" or "True". When this method is not\n defined, "__len__()" is called, if it is defined, and the object is\n considered true if its result is nonzero. If a class defines\n neither "__len__()" nor "__bool__()", all its instances are\n considered true.\n\n\nCustomizing attribute access\n============================\n\nThe following methods can be defined to customize the meaning of\nattribute access (use of, assignment to, or deletion of "x.name") for\nclass instances.\n\nobject.__getattr__(self, name)\n\n Called when an attribute lookup has not found the attribute in the\n usual places (i.e. it is not an instance attribute nor is it found\n in the class tree for "self"). "name" is the attribute name. This\n method should return the (computed) attribute value or raise an\n "AttributeError" exception.\n\n Note that if the attribute is found through the normal mechanism,\n "__getattr__()" is not called. (This is an intentional asymmetry\n between "__getattr__()" and "__setattr__()".) This is done both for\n efficiency reasons and because otherwise "__getattr__()" would have\n no way to access other attributes of the instance. Note that at\n least for instance variables, you can fake total control by not\n inserting any values in the instance attribute dictionary (but\n instead inserting them in another object). See the\n "__getattribute__()" method below for a way to actually get total\n control over attribute access.\n\nobject.__getattribute__(self, name)\n\n Called unconditionally to implement attribute accesses for\n instances of the class. If the class also defines "__getattr__()",\n the latter will not be called unless "__getattribute__()" either\n calls it explicitly or raises an "AttributeError". This method\n should return the (computed) attribute value or raise an\n "AttributeError" exception. In order to avoid infinite recursion in\n this method, its implementation should always call the base class\n method with the same name to access any attributes it needs, for\n example, "object.__getattribute__(self, name)".\n\n Note: This method may still be bypassed when looking up special\n methods as the result of implicit invocation via language syntax\n or built-in functions. See *Special method lookup*.\n\nobject.__setattr__(self, name, value)\n\n Called when an attribute assignment is attempted. This is called\n instead of the normal mechanism (i.e. store the value in the\n instance dictionary). *name* is the attribute name, *value* is the\n value to be assigned to it.\n\n If "__setattr__()" wants to assign to an instance attribute, it\n should call the base class method with the same name, for example,\n "object.__setattr__(self, name, value)".\n\nobject.__delattr__(self, name)\n\n Like "__setattr__()" but for attribute deletion instead of\n assignment. This should only be implemented if "del obj.name" is\n meaningful for the object.\n\nobject.__dir__(self)\n\n Called when "dir()" is called on the object. A sequence must be\n returned. "dir()" converts the returned sequence to a list and\n sorts it.\n\n\nImplementing Descriptors\n------------------------\n\nThe following methods only apply when an instance of the class\ncontaining the method (a so-called *descriptor* class) appears in an\n*owner* class (the descriptor must be in either the owner\'s class\ndictionary or in the class dictionary for one of its parents). In the\nexamples below, "the attribute" refers to the attribute whose name is\nthe key of the property in the owner class\' "__dict__".\n\nobject.__get__(self, instance, owner)\n\n Called to get the attribute of the owner class (class attribute\n access) or of an instance of that class (instance attribute\n access). *owner* is always the owner class, while *instance* is the\n instance that the attribute was accessed through, or "None" when\n the attribute is accessed through the *owner*. This method should\n return the (computed) attribute value or raise an "AttributeError"\n exception.\n\nobject.__set__(self, instance, value)\n\n Called to set the attribute on an instance *instance* of the owner\n class to a new value, *value*.\n\nobject.__delete__(self, instance)\n\n Called to delete the attribute on an instance *instance* of the\n owner class.\n\nThe attribute "__objclass__" is interpreted by the "inspect" module as\nspecifying the class where this object was defined (setting this\nappropriately can assist in runtime introspection of dynamic class\nattributes). For callables, it may indicate that an instance of the\ngiven type (or a subclass) is expected or required as the first\npositional argument (for example, CPython sets this attribute for\nunbound methods that are implemented in C).\n\n\nInvoking Descriptors\n--------------------\n\nIn general, a descriptor is an object attribute with "binding\nbehavior", one whose attribute access has been overridden by methods\nin the descriptor protocol: "__get__()", "__set__()", and\n"__delete__()". If any of those methods are defined for an object, it\nis said to be a descriptor.\n\nThe default behavior for attribute access is to get, set, or delete\nthe attribute from an object\'s dictionary. For instance, "a.x" has a\nlookup chain starting with "a.__dict__[\'x\']", then\n"type(a).__dict__[\'x\']", and continuing through the base classes of\n"type(a)" excluding metaclasses.\n\nHowever, if the looked-up value is an object defining one of the\ndescriptor methods, then Python may override the default behavior and\ninvoke the descriptor method instead. Where this occurs in the\nprecedence chain depends on which descriptor methods were defined and\nhow they were called.\n\nThe starting point for descriptor invocation is a binding, "a.x". How\nthe arguments are assembled depends on "a":\n\nDirect Call\n The simplest and least common call is when user code directly\n invokes a descriptor method: "x.__get__(a)".\n\nInstance Binding\n If binding to an object instance, "a.x" is transformed into the\n call: "type(a).__dict__[\'x\'].__get__(a, type(a))".\n\nClass Binding\n If binding to a class, "A.x" is transformed into the call:\n "A.__dict__[\'x\'].__get__(None, A)".\n\nSuper Binding\n If "a" is an instance of "super", then the binding "super(B,\n obj).m()" searches "obj.__class__.__mro__" for the base class "A"\n immediately preceding "B" and then invokes the descriptor with the\n call: "A.__dict__[\'m\'].__get__(obj, obj.__class__)".\n\nFor instance bindings, the precedence of descriptor invocation depends\non the which descriptor methods are defined. A descriptor can define\nany combination of "__get__()", "__set__()" and "__delete__()". If it\ndoes not define "__get__()", then accessing the attribute will return\nthe descriptor object itself unless there is a value in the object\'s\ninstance dictionary. If the descriptor defines "__set__()" and/or\n"__delete__()", it is a data descriptor; if it defines neither, it is\na non-data descriptor. Normally, data descriptors define both\n"__get__()" and "__set__()", while non-data descriptors have just the\n"__get__()" method. Data descriptors with "__set__()" and "__get__()"\ndefined always override a redefinition in an instance dictionary. In\ncontrast, non-data descriptors can be overridden by instances.\n\nPython methods (including "staticmethod()" and "classmethod()") are\nimplemented as non-data descriptors. Accordingly, instances can\nredefine and override methods. This allows individual instances to\nacquire behaviors that differ from other instances of the same class.\n\nThe "property()" function is implemented as a data descriptor.\nAccordingly, instances cannot override the behavior of a property.\n\n\n__slots__\n---------\n\nBy default, instances of classes have a dictionary for attribute\nstorage. This wastes space for objects having very few instance\nvariables. The space consumption can become acute when creating large\nnumbers of instances.\n\nThe default can be overridden by defining *__slots__* in a class\ndefinition. The *__slots__* declaration takes a sequence of instance\nvariables and reserves just enough space in each instance to hold a\nvalue for each variable. Space is saved because *__dict__* is not\ncreated for each instance.\n\nobject.__slots__\n\n This class variable can be assigned a string, iterable, or sequence\n of strings with variable names used by instances. *__slots__*\n reserves space for the declared variables and prevents the\n automatic creation of *__dict__* and *__weakref__* for each\n instance.\n\n\nNotes on using *__slots__*\n~~~~~~~~~~~~~~~~~~~~~~~~~~\n\n* When inheriting from a class without *__slots__*, the *__dict__*\n attribute of that class will always be accessible, so a *__slots__*\n definition in the subclass is meaningless.\n\n* Without a *__dict__* variable, instances cannot be assigned new\n variables not listed in the *__slots__* definition. Attempts to\n assign to an unlisted variable name raises "AttributeError". If\n dynamic assignment of new variables is desired, then add\n "\'__dict__\'" to the sequence of strings in the *__slots__*\n declaration.\n\n* Without a *__weakref__* variable for each instance, classes\n defining *__slots__* do not support weak references to its\n instances. If weak reference support is needed, then add\n "\'__weakref__\'" to the sequence of strings in the *__slots__*\n declaration.\n\n* *__slots__* are implemented at the class level by creating\n descriptors (*Implementing Descriptors*) for each variable name. As\n a result, class attributes cannot be used to set default values for\n instance variables defined by *__slots__*; otherwise, the class\n attribute would overwrite the descriptor assignment.\n\n* The action of a *__slots__* declaration is limited to the class\n where it is defined. As a result, subclasses will have a *__dict__*\n unless they also define *__slots__* (which must only contain names\n of any *additional* slots).\n\n* If a class defines a slot also defined in a base class, the\n instance variable defined by the base class slot is inaccessible\n (except by retrieving its descriptor directly from the base class).\n This renders the meaning of the program undefined. In the future, a\n check may be added to prevent this.\n\n* Nonempty *__slots__* does not work for classes derived from\n "variable-length" built-in types such as "int", "bytes" and "tuple".\n\n* Any non-string iterable may be assigned to *__slots__*. Mappings\n may also be used; however, in the future, special meaning may be\n assigned to the values corresponding to each key.\n\n* *__class__* assignment works only if both classes have the same\n *__slots__*.\n\n\nCustomizing class creation\n==========================\n\nBy default, classes are constructed using "type()". The class body is\nexecuted in a new namespace and the class name is bound locally to the\nresult of "type(name, bases, namespace)".\n\nThe class creation process can be customised by passing the\n"metaclass" keyword argument in the class definition line, or by\ninheriting from an existing class that included such an argument. In\nthe following example, both "MyClass" and "MySubclass" are instances\nof "Meta":\n\n class Meta(type):\n pass\n\n class MyClass(metaclass=Meta):\n pass\n\n class MySubclass(MyClass):\n pass\n\nAny other keyword arguments that are specified in the class definition\nare passed through to all metaclass operations described below.\n\nWhen a class definition is executed, the following steps occur:\n\n* the appropriate metaclass is determined\n\n* the class namespace is prepared\n\n* the class body is executed\n\n* the class object is created\n\n\nDetermining the appropriate metaclass\n-------------------------------------\n\nThe appropriate metaclass for a class definition is determined as\nfollows:\n\n* if no bases and no explicit metaclass are given, then "type()" is\n used\n\n* if an explicit metaclass is given and it is *not* an instance of\n "type()", then it is used directly as the metaclass\n\n* if an instance of "type()" is given as the explicit metaclass, or\n bases are defined, then the most derived metaclass is used\n\nThe most derived metaclass is selected from the explicitly specified\nmetaclass (if any) and the metaclasses (i.e. "type(cls)") of all\nspecified base classes. The most derived metaclass is one which is a\nsubtype of *all* of these candidate metaclasses. If none of the\ncandidate metaclasses meets that criterion, then the class definition\nwill fail with "TypeError".\n\n\nPreparing the class namespace\n-----------------------------\n\nOnce the appropriate metaclass has been identified, then the class\nnamespace is prepared. If the metaclass has a "__prepare__" attribute,\nit is called as "namespace = metaclass.__prepare__(name, bases,\n**kwds)" (where the additional keyword arguments, if any, come from\nthe class definition).\n\nIf the metaclass has no "__prepare__" attribute, then the class\nnamespace is initialised as an empty "dict()" instance.\n\nSee also: **PEP 3115** - Metaclasses in Python 3000\n\n Introduced the "__prepare__" namespace hook\n\n\nExecuting the class body\n------------------------\n\nThe class body is executed (approximately) as "exec(body, globals(),\nnamespace)". The key difference from a normal call to "exec()" is that\nlexical scoping allows the class body (including any methods) to\nreference names from the current and outer scopes when the class\ndefinition occurs inside a function.\n\nHowever, even when the class definition occurs inside the function,\nmethods defined inside the class still cannot see names defined at the\nclass scope. Class variables must be accessed through the first\nparameter of instance or class methods, and cannot be accessed at all\nfrom static methods.\n\n\nCreating the class object\n-------------------------\n\nOnce the class namespace has been populated by executing the class\nbody, the class object is created by calling "metaclass(name, bases,\nnamespace, **kwds)" (the additional keywords passed here are the same\nas those passed to "__prepare__").\n\nThis class object is the one that will be referenced by the zero-\nargument form of "super()". "__class__" is an implicit closure\nreference created by the compiler if any methods in a class body refer\nto either "__class__" or "super". This allows the zero argument form\nof "super()" to correctly identify the class being defined based on\nlexical scoping, while the class or instance that was used to make the\ncurrent call is identified based on the first argument passed to the\nmethod.\n\nAfter the class object is created, it is passed to the class\ndecorators included in the class definition (if any) and the resulting\nobject is bound in the local namespace as the defined class.\n\nSee also: **PEP 3135** - New super\n\n Describes the implicit "__class__" closure reference\n\n\nMetaclass example\n-----------------\n\nThe potential uses for metaclasses are boundless. Some ideas that have\nbeen explored include logging, interface checking, automatic\ndelegation, automatic property creation, proxies, frameworks, and\nautomatic resource locking/synchronization.\n\nHere is an example of a metaclass that uses an\n"collections.OrderedDict" to remember the order that class variables\nare defined:\n\n class OrderedClass(type):\n\n @classmethod\n def __prepare__(metacls, name, bases, **kwds):\n return collections.OrderedDict()\n\n def __new__(cls, name, bases, namespace, **kwds):\n result = type.__new__(cls, name, bases, dict(namespace))\n result.members = tuple(namespace)\n return result\n\n class A(metaclass=OrderedClass):\n def one(self): pass\n def two(self): pass\n def three(self): pass\n def four(self): pass\n\n >>> A.members\n (\'__module__\', \'one\', \'two\', \'three\', \'four\')\n\nWhen the class definition for *A* gets executed, the process begins\nwith calling the metaclass\'s "__prepare__()" method which returns an\nempty "collections.OrderedDict". That mapping records the methods and\nattributes of *A* as they are defined within the body of the class\nstatement. Once those definitions are executed, the ordered dictionary\nis fully populated and the metaclass\'s "__new__()" method gets\ninvoked. That method builds the new type and it saves the ordered\ndictionary keys in an attribute called "members".\n\n\nCustomizing instance and subclass checks\n========================================\n\nThe following methods are used to override the default behavior of the\n"isinstance()" and "issubclass()" built-in functions.\n\nIn particular, the metaclass "abc.ABCMeta" implements these methods in\norder to allow the addition of Abstract Base Classes (ABCs) as\n"virtual base classes" to any class or type (including built-in\ntypes), including other ABCs.\n\nclass.__instancecheck__(self, instance)\n\n Return true if *instance* should be considered a (direct or\n indirect) instance of *class*. If defined, called to implement\n "isinstance(instance, class)".\n\nclass.__subclasscheck__(self, subclass)\n\n Return true if *subclass* should be considered a (direct or\n indirect) subclass of *class*. If defined, called to implement\n "issubclass(subclass, class)".\n\nNote that these methods are looked up on the type (metaclass) of a\nclass. They cannot be defined as class methods in the actual class.\nThis is consistent with the lookup of special methods that are called\non instances, only in this case the instance is itself a class.\n\nSee also: **PEP 3119** - Introducing Abstract Base Classes\n\n Includes the specification for customizing "isinstance()" and\n "issubclass()" behavior through "__instancecheck__()" and\n "__subclasscheck__()", with motivation for this functionality in\n the context of adding Abstract Base Classes (see the "abc"\n module) to the language.\n\n\nEmulating callable objects\n==========================\n\nobject.__call__(self[, args...])\n\n Called when the instance is "called" as a function; if this method\n is defined, "x(arg1, arg2, ...)" is a shorthand for\n "x.__call__(arg1, arg2, ...)".\n\n\nEmulating container types\n=========================\n\nThe following methods can be defined to implement container objects.\nContainers usually are sequences (such as lists or tuples) or mappings\n(like dictionaries), but can represent other containers as well. The\nfirst set of methods is used either to emulate a sequence or to\nemulate a mapping; the difference is that for a sequence, the\nallowable keys should be the integers *k* for which "0 <= k < N" where\n*N* is the length of the sequence, or slice objects, which define a\nrange of items. It is also recommended that mappings provide the\nmethods "keys()", "values()", "items()", "get()", "clear()",\n"setdefault()", "pop()", "popitem()", "copy()", and "update()"\nbehaving similar to those for Python\'s standard dictionary objects.\nThe "collections" module provides a "MutableMapping" abstract base\nclass to help create those methods from a base set of "__getitem__()",\n"__setitem__()", "__delitem__()", and "keys()". Mutable sequences\nshould provide methods "append()", "count()", "index()", "extend()",\n"insert()", "pop()", "remove()", "reverse()" and "sort()", like Python\nstandard list objects. Finally, sequence types should implement\naddition (meaning concatenation) and multiplication (meaning\nrepetition) by defining the methods "__add__()", "__radd__()",\n"__iadd__()", "__mul__()", "__rmul__()" and "__imul__()" described\nbelow; they should not define other numerical operators. It is\nrecommended that both mappings and sequences implement the\n"__contains__()" method to allow efficient use of the "in" operator;\nfor mappings, "in" should search the mapping\'s keys; for sequences, it\nshould search through the values. It is further recommended that both\nmappings and sequences implement the "__iter__()" method to allow\nefficient iteration through the container; for mappings, "__iter__()"\nshould be the same as "keys()"; for sequences, it should iterate\nthrough the values.\n\nobject.__len__(self)\n\n Called to implement the built-in function "len()". Should return\n the length of the object, an integer ">=" 0. Also, an object that\n doesn\'t define a "__bool__()" method and whose "__len__()" method\n returns zero is considered to be false in a Boolean context.\n\nobject.__length_hint__(self)\n\n Called to implement "operator.length_hint()". Should return an\n estimated length for the object (which may be greater or less than\n the actual length). The length must be an integer ">=" 0. This\n method is purely an optimization and is never required for\n correctness.\n\n New in version 3.4.\n\nNote: Slicing is done exclusively with the following three methods.\n A call like\n\n a[1:2] = b\n\n is translated to\n\n a[slice(1, 2, None)] = b\n\n and so forth. Missing slice items are always filled in with "None".\n\nobject.__getitem__(self, key)\n\n Called to implement evaluation of "self[key]". For sequence types,\n the accepted keys should be integers and slice objects. Note that\n the special interpretation of negative indexes (if the class wishes\n to emulate a sequence type) is up to the "__getitem__()" method. If\n *key* is of an inappropriate type, "TypeError" may be raised; if of\n a value outside the set of indexes for the sequence (after any\n special interpretation of negative values), "IndexError" should be\n raised. For mapping types, if *key* is missing (not in the\n container), "KeyError" should be raised.\n\n Note: "for" loops expect that an "IndexError" will be raised for\n illegal indexes to allow proper detection of the end of the\n sequence.\n\nobject.__missing__(self, key)\n\n Called by "dict"."__getitem__()" to implement "self[key]" for dict\n subclasses when key is not in the dictionary.\n\nobject.__setitem__(self, key, value)\n\n Called to implement assignment to "self[key]". Same note as for\n "__getitem__()". This should only be implemented for mappings if\n the objects support changes to the values for keys, or if new keys\n can be added, or for sequences if elements can be replaced. The\n same exceptions should be raised for improper *key* values as for\n the "__getitem__()" method.\n\nobject.__delitem__(self, key)\n\n Called to implement deletion of "self[key]". Same note as for\n "__getitem__()". This should only be implemented for mappings if\n the objects support removal of keys, or for sequences if elements\n can be removed from the sequence. The same exceptions should be\n raised for improper *key* values as for the "__getitem__()" method.\n\nobject.__iter__(self)\n\n This method is called when an iterator is required for a container.\n This method should return a new iterator object that can iterate\n over all the objects in the container. For mappings, it should\n iterate over the keys of the container.\n\n Iterator objects also need to implement this method; they are\n required to return themselves. For more information on iterator\n objects, see *Iterator Types*.\n\nobject.__reversed__(self)\n\n Called (if present) by the "reversed()" built-in to implement\n reverse iteration. It should return a new iterator object that\n iterates over all the objects in the container in reverse order.\n\n If the "__reversed__()" method is not provided, the "reversed()"\n built-in will fall back to using the sequence protocol ("__len__()"\n and "__getitem__()"). Objects that support the sequence protocol\n should only provide "__reversed__()" if they can provide an\n implementation that is more efficient than the one provided by\n "reversed()".\n\nThe membership test operators ("in" and "not in") are normally\nimplemented as an iteration through a sequence. However, container\nobjects can supply the following special method with a more efficient\nimplementation, which also does not require the object be a sequence.\n\nobject.__contains__(self, item)\n\n Called to implement membership test operators. Should return true\n if *item* is in *self*, false otherwise. For mapping objects, this\n should consider the keys of the mapping rather than the values or\n the key-item pairs.\n\n For objects that don\'t define "__contains__()", the membership test\n first tries iteration via "__iter__()", then the old sequence\n iteration protocol via "__getitem__()", see *this section in the\n language reference*.\n\n\nEmulating numeric types\n=======================\n\nThe following methods can be defined to emulate numeric objects.\nMethods corresponding to operations that are not supported by the\nparticular kind of number implemented (e.g., bitwise operations for\nnon-integral numbers) should be left undefined.\n\nobject.__add__(self, other)\nobject.__sub__(self, other)\nobject.__mul__(self, other)\nobject.__matmul__(self, other)\nobject.__truediv__(self, other)\nobject.__floordiv__(self, other)\nobject.__mod__(self, other)\nobject.__divmod__(self, other)\nobject.__pow__(self, other[, modulo])\nobject.__lshift__(self, other)\nobject.__rshift__(self, other)\nobject.__and__(self, other)\nobject.__xor__(self, other)\nobject.__or__(self, other)\n\n These methods are called to implement the binary arithmetic\n operations ("+", "-", "*", "@", "/", "//", "%", "divmod()",\n "pow()", "**", "<<", ">>", "&", "^", "|"). For instance, to\n evaluate the expression "x + y", where *x* is an instance of a\n class that has an "__add__()" method, "x.__add__(y)" is called.\n The "__divmod__()" method should be the equivalent to using\n "__floordiv__()" and "__mod__()"; it should not be related to\n "__truediv__()". Note that "__pow__()" should be defined to accept\n an optional third argument if the ternary version of the built-in\n "pow()" function is to be supported.\n\n If one of those methods does not support the operation with the\n supplied arguments, it should return "NotImplemented".\n\nobject.__radd__(self, other)\nobject.__rsub__(self, other)\nobject.__rmul__(self, other)\nobject.__rmatmul__(self, other)\nobject.__rtruediv__(self, other)\nobject.__rfloordiv__(self, other)\nobject.__rmod__(self, other)\nobject.__rdivmod__(self, other)\nobject.__rpow__(self, other)\nobject.__rlshift__(self, other)\nobject.__rrshift__(self, other)\nobject.__rand__(self, other)\nobject.__rxor__(self, other)\nobject.__ror__(self, other)\n\n These methods are called to implement the binary arithmetic\n operations ("+", "-", "*", "@", "/", "//", "%", "divmod()",\n "pow()", "**", "<<", ">>", "&", "^", "|") with reflected (swapped)\n operands. These functions are only called if the left operand does\n not support the corresponding operation and the operands are of\n different types. [2] For instance, to evaluate the expression "x -\n y", where *y* is an instance of a class that has an "__rsub__()"\n method, "y.__rsub__(x)" is called if "x.__sub__(y)" returns\n *NotImplemented*.\n\n Note that ternary "pow()" will not try calling "__rpow__()" (the\n coercion rules would become too complicated).\n\n Note: If the right operand\'s type is a subclass of the left\n operand\'s type and that subclass provides the reflected method\n for the operation, this method will be called before the left\n operand\'s non-reflected method. This behavior allows subclasses\n to override their ancestors\' operations.\n\nobject.__iadd__(self, other)\nobject.__isub__(self, other)\nobject.__imul__(self, other)\nobject.__imatmul__(self, other)\nobject.__itruediv__(self, other)\nobject.__ifloordiv__(self, other)\nobject.__imod__(self, other)\nobject.__ipow__(self, other[, modulo])\nobject.__ilshift__(self, other)\nobject.__irshift__(self, other)\nobject.__iand__(self, other)\nobject.__ixor__(self, other)\nobject.__ior__(self, other)\n\n These methods are called to implement the augmented arithmetic\n assignments ("+=", "-=", "*=", "@=", "/=", "//=", "%=", "**=",\n "<<=", ">>=", "&=", "^=", "|="). These methods should attempt to\n do the operation in-place (modifying *self*) and return the result\n (which could be, but does not have to be, *self*). If a specific\n method is not defined, the augmented assignment falls back to the\n normal methods. For instance, if *x* is an instance of a class\n with an "__iadd__()" method, "x += y" is equivalent to "x =\n x.__iadd__(y)" . Otherwise, "x.__add__(y)" and "y.__radd__(x)" are\n considered, as with the evaluation of "x + y". In certain\n situations, augmented assignment can result in unexpected errors\n (see *Why does a_tuple[i] += [\'item\'] raise an exception when the\n addition works?*), but this behavior is in fact part of the data\n model.\n\nobject.__neg__(self)\nobject.__pos__(self)\nobject.__abs__(self)\nobject.__invert__(self)\n\n Called to implement the unary arithmetic operations ("-", "+",\n "abs()" and "~").\n\nobject.__complex__(self)\nobject.__int__(self)\nobject.__float__(self)\nobject.__round__(self[, n])\n\n Called to implement the built-in functions "complex()", "int()",\n "float()" and "round()". Should return a value of the appropriate\n type.\n\nobject.__index__(self)\n\n Called to implement "operator.index()", and whenever Python needs\n to losslessly convert the numeric object to an integer object (such\n as in slicing, or in the built-in "bin()", "hex()" and "oct()"\n functions). Presence of this method indicates that the numeric\n object is an integer type. Must return an integer.\n\n Note: In order to have a coherent integer type class, when\n "__index__()" is defined "__int__()" should also be defined, and\n both should return the same value.\n\n\nWith Statement Context Managers\n===============================\n\nA *context manager* is an object that defines the runtime context to\nbe established when executing a "with" statement. The context manager\nhandles the entry into, and the exit from, the desired runtime context\nfor the execution of the block of code. Context managers are normally\ninvoked using the "with" statement (described in section *The with\nstatement*), but can also be used by directly invoking their methods.\n\nTypical uses of context managers include saving and restoring various\nkinds of global state, locking and unlocking resources, closing opened\nfiles, etc.\n\nFor more information on context managers, see *Context Manager Types*.\n\nobject.__enter__(self)\n\n Enter the runtime context related to this object. The "with"\n statement will bind this method\'s return value to the target(s)\n specified in the "as" clause of the statement, if any.\n\nobject.__exit__(self, exc_type, exc_value, traceback)\n\n Exit the runtime context related to this object. The parameters\n describe the exception that caused the context to be exited. If the\n context was exited without an exception, all three arguments will\n be "None".\n\n If an exception is supplied, and the method wishes to suppress the\n exception (i.e., prevent it from being propagated), it should\n return a true value. Otherwise, the exception will be processed\n normally upon exit from this method.\n\n Note that "__exit__()" methods should not reraise the passed-in\n exception; this is the caller\'s responsibility.\n\nSee also: **PEP 0343** - The "with" statement\n\n The specification, background, and examples for the Python "with"\n statement.\n\n\nSpecial method lookup\n=====================\n\nFor custom classes, implicit invocations of special methods are only\nguaranteed to work correctly if defined on an object\'s type, not in\nthe object\'s instance dictionary. That behaviour is the reason why\nthe following code raises an exception:\n\n >>> class C:\n ... pass\n ...\n >>> c = C()\n >>> c.__len__ = lambda: 5\n >>> len(c)\n Traceback (most recent call last):\n File "", line 1, in \n TypeError: object of type \'C\' has no len()\n\nThe rationale behind this behaviour lies with a number of special\nmethods such as "__hash__()" and "__repr__()" that are implemented by\nall objects, including type objects. If the implicit lookup of these\nmethods used the conventional lookup process, they would fail when\ninvoked on the type object itself:\n\n >>> 1 .__hash__() == hash(1)\n True\n >>> int.__hash__() == hash(int)\n Traceback (most recent call last):\n File "", line 1, in \n TypeError: descriptor \'__hash__\' of \'int\' object needs an argument\n\nIncorrectly attempting to invoke an unbound method of a class in this\nway is sometimes referred to as \'metaclass confusion\', and is avoided\nby bypassing the instance when looking up special methods:\n\n >>> type(1).__hash__(1) == hash(1)\n True\n >>> type(int).__hash__(int) == hash(int)\n True\n\nIn addition to bypassing any instance attributes in the interest of\ncorrectness, implicit special method lookup generally also bypasses\nthe "__getattribute__()" method even of the object\'s metaclass:\n\n >>> class Meta(type):\n ... def __getattribute__(*args):\n ... print("Metaclass getattribute invoked")\n ... return type.__getattribute__(*args)\n ...\n >>> class C(object, metaclass=Meta):\n ... def __len__(self):\n ... return 10\n ... def __getattribute__(*args):\n ... print("Class getattribute invoked")\n ... return object.__getattribute__(*args)\n ...\n >>> c = C()\n >>> c.__len__() # Explicit lookup via instance\n Class getattribute invoked\n 10\n >>> type(c).__len__(c) # Explicit lookup via type\n Metaclass getattribute invoked\n 10\n >>> len(c) # Implicit lookup\n 10\n\nBypassing the "__getattribute__()" machinery in this fashion provides\nsignificant scope for speed optimisations within the interpreter, at\nthe cost of some flexibility in the handling of special methods (the\nspecial method *must* be set on the class object itself in order to be\nconsistently invoked by the interpreter).\n', - 'string-methods': u'\nString Methods\n**************\n\nStrings implement all of the *common* sequence operations, along with\nthe additional methods described below.\n\nStrings also support two styles of string formatting, one providing a\nlarge degree of flexibility and customization (see "str.format()",\n*Format String Syntax* and *String Formatting*) and the other based on\nC "printf" style formatting that handles a narrower range of types and\nis slightly harder to use correctly, but is often faster for the cases\nit can handle (*printf-style String Formatting*).\n\nThe *Text Processing Services* section of the standard library covers\na number of other modules that provide various text related utilities\n(including regular expression support in the "re" module).\n\nstr.capitalize()\n\n Return a copy of the string with its first character capitalized\n and the rest lowercased.\n\nstr.casefold()\n\n Return a casefolded copy of the string. Casefolded strings may be\n used for caseless matching.\n\n Casefolding is similar to lowercasing but more aggressive because\n it is intended to remove all case distinctions in a string. For\n example, the German lowercase letter "\'\xdf\'" is equivalent to ""ss"".\n Since it is already lowercase, "lower()" would do nothing to "\'\xdf\'";\n "casefold()" converts it to ""ss"".\n\n The casefolding algorithm is described in section 3.13 of the\n Unicode Standard.\n\n New in version 3.3.\n\nstr.center(width[, fillchar])\n\n Return centered in a string of length *width*. Padding is done\n using the specified *fillchar* (default is an ASCII space). The\n original string is returned if *width* is less than or equal to\n "len(s)".\n\nstr.count(sub[, start[, end]])\n\n Return the number of non-overlapping occurrences of substring *sub*\n in the range [*start*, *end*]. Optional arguments *start* and\n *end* are interpreted as in slice notation.\n\nstr.encode(encoding="utf-8", errors="strict")\n\n Return an encoded version of the string as a bytes object. Default\n encoding is "\'utf-8\'". *errors* may be given to set a different\n error handling scheme. The default for *errors* is "\'strict\'",\n meaning that encoding errors raise a "UnicodeError". Other possible\n values are "\'ignore\'", "\'replace\'", "\'xmlcharrefreplace\'",\n "\'backslashreplace\'" and any other name registered via\n "codecs.register_error()", see section *Error Handlers*. For a list\n of possible encodings, see section *Standard Encodings*.\n\n Changed in version 3.1: Support for keyword arguments added.\n\nstr.endswith(suffix[, start[, end]])\n\n Return "True" if the string ends with the specified *suffix*,\n otherwise return "False". *suffix* can also be a tuple of suffixes\n to look for. With optional *start*, test beginning at that\n position. With optional *end*, stop comparing at that position.\n\nstr.expandtabs(tabsize=8)\n\n Return a copy of the string where all tab characters are replaced\n by one or more spaces, depending on the current column and the\n given tab size. Tab positions occur every *tabsize* characters\n (default is 8, giving tab positions at columns 0, 8, 16 and so on).\n To expand the string, the current column is set to zero and the\n string is examined character by character. If the character is a\n tab ("\\t"), one or more space characters are inserted in the result\n until the current column is equal to the next tab position. (The\n tab character itself is not copied.) If the character is a newline\n ("\\n") or return ("\\r"), it is copied and the current column is\n reset to zero. Any other character is copied unchanged and the\n current column is incremented by one regardless of how the\n character is represented when printed.\n\n >>> \'01\\t012\\t0123\\t01234\'.expandtabs()\n \'01 012 0123 01234\'\n >>> \'01\\t012\\t0123\\t01234\'.expandtabs(4)\n \'01 012 0123 01234\'\n\nstr.find(sub[, start[, end]])\n\n Return the lowest index in the string where substring *sub* is\n found, such that *sub* is contained in the slice "s[start:end]".\n Optional arguments *start* and *end* are interpreted as in slice\n notation. Return "-1" if *sub* is not found.\n\n Note: The "find()" method should be used only if you need to know\n the position of *sub*. To check if *sub* is a substring or not,\n use the "in" operator:\n\n >>> \'Py\' in \'Python\'\n True\n\nstr.format(*args, **kwargs)\n\n Perform a string formatting operation. The string on which this\n method is called can contain literal text or replacement fields\n delimited by braces "{}". Each replacement field contains either\n the numeric index of a positional argument, or the name of a\n keyword argument. Returns a copy of the string where each\n replacement field is replaced with the string value of the\n corresponding argument.\n\n >>> "The sum of 1 + 2 is {0}".format(1+2)\n \'The sum of 1 + 2 is 3\'\n\n See *Format String Syntax* for a description of the various\n formatting options that can be specified in format strings.\n\nstr.format_map(mapping)\n\n Similar to "str.format(**mapping)", except that "mapping" is used\n directly and not copied to a "dict". This is useful if for example\n "mapping" is a dict subclass:\n\n >>> class Default(dict):\n ... def __missing__(self, key):\n ... return key\n ...\n >>> \'{name} was born in {country}\'.format_map(Default(name=\'Guido\'))\n \'Guido was born in country\'\n\n New in version 3.2.\n\nstr.index(sub[, start[, end]])\n\n Like "find()", but raise "ValueError" when the substring is not\n found.\n\nstr.isalnum()\n\n Return true if all characters in the string are alphanumeric and\n there is at least one character, false otherwise. A character "c"\n is alphanumeric if one of the following returns "True":\n "c.isalpha()", "c.isdecimal()", "c.isdigit()", or "c.isnumeric()".\n\nstr.isalpha()\n\n Return true if all characters in the string are alphabetic and\n there is at least one character, false otherwise. Alphabetic\n characters are those characters defined in the Unicode character\n database as "Letter", i.e., those with general category property\n being one of "Lm", "Lt", "Lu", "Ll", or "Lo". Note that this is\n different from the "Alphabetic" property defined in the Unicode\n Standard.\n\nstr.isdecimal()\n\n Return true if all characters in the string are decimal characters\n and there is at least one character, false otherwise. Decimal\n characters are those from general category "Nd". This category\n includes digit characters, and all characters that can be used to\n form decimal-radix numbers, e.g. U+0660, ARABIC-INDIC DIGIT ZERO.\n\nstr.isdigit()\n\n Return true if all characters in the string are digits and there is\n at least one character, false otherwise. Digits include decimal\n characters and digits that need special handling, such as the\n compatibility superscript digits. Formally, a digit is a character\n that has the property value Numeric_Type=Digit or\n Numeric_Type=Decimal.\n\nstr.isidentifier()\n\n Return true if the string is a valid identifier according to the\n language definition, section *Identifiers and keywords*.\n\n Use "keyword.iskeyword()" to test for reserved identifiers such as\n "def" and "class".\n\nstr.islower()\n\n Return true if all cased characters [4] in the string are lowercase\n and there is at least one cased character, false otherwise.\n\nstr.isnumeric()\n\n Return true if all characters in the string are numeric characters,\n and there is at least one character, false otherwise. Numeric\n characters include digit characters, and all characters that have\n the Unicode numeric value property, e.g. U+2155, VULGAR FRACTION\n ONE FIFTH. Formally, numeric characters are those with the\n property value Numeric_Type=Digit, Numeric_Type=Decimal or\n Numeric_Type=Numeric.\n\nstr.isprintable()\n\n Return true if all characters in the string are printable or the\n string is empty, false otherwise. Nonprintable characters are\n those characters defined in the Unicode character database as\n "Other" or "Separator", excepting the ASCII space (0x20) which is\n considered printable. (Note that printable characters in this\n context are those which should not be escaped when "repr()" is\n invoked on a string. It has no bearing on the handling of strings\n written to "sys.stdout" or "sys.stderr".)\n\nstr.isspace()\n\n Return true if there are only whitespace characters in the string\n and there is at least one character, false otherwise. Whitespace\n characters are those characters defined in the Unicode character\n database as "Other" or "Separator" and those with bidirectional\n property being one of "WS", "B", or "S".\n\nstr.istitle()\n\n Return true if the string is a titlecased string and there is at\n least one character, for example uppercase characters may only\n follow uncased characters and lowercase characters only cased ones.\n Return false otherwise.\n\nstr.isupper()\n\n Return true if all cased characters [4] in the string are uppercase\n and there is at least one cased character, false otherwise.\n\nstr.join(iterable)\n\n Return a string which is the concatenation of the strings in the\n *iterable* *iterable*. A "TypeError" will be raised if there are\n any non-string values in *iterable*, including "bytes" objects.\n The separator between elements is the string providing this method.\n\nstr.ljust(width[, fillchar])\n\n Return the string left justified in a string of length *width*.\n Padding is done using the specified *fillchar* (default is an ASCII\n space). The original string is returned if *width* is less than or\n equal to "len(s)".\n\nstr.lower()\n\n Return a copy of the string with all the cased characters [4]\n converted to lowercase.\n\n The lowercasing algorithm used is described in section 3.13 of the\n Unicode Standard.\n\nstr.lstrip([chars])\n\n Return a copy of the string with leading characters removed. The\n *chars* argument is a string specifying the set of characters to be\n removed. If omitted or "None", the *chars* argument defaults to\n removing whitespace. The *chars* argument is not a prefix; rather,\n all combinations of its values are stripped:\n\n >>> \' spacious \'.lstrip()\n \'spacious \'\n >>> \'www.example.com\'.lstrip(\'cmowz.\')\n \'example.com\'\n\nstatic str.maketrans(x[, y[, z]])\n\n This static method returns a translation table usable for\n "str.translate()".\n\n If there is only one argument, it must be a dictionary mapping\n Unicode ordinals (integers) or characters (strings of length 1) to\n Unicode ordinals, strings (of arbitrary lengths) or None.\n Character keys will then be converted to ordinals.\n\n If there are two arguments, they must be strings of equal length,\n and in the resulting dictionary, each character in x will be mapped\n to the character at the same position in y. If there is a third\n argument, it must be a string, whose characters will be mapped to\n None in the result.\n\nstr.partition(sep)\n\n Split the string at the first occurrence of *sep*, and return a\n 3-tuple containing the part before the separator, the separator\n itself, and the part after the separator. If the separator is not\n found, return a 3-tuple containing the string itself, followed by\n two empty strings.\n\nstr.replace(old, new[, count])\n\n Return a copy of the string with all occurrences of substring *old*\n replaced by *new*. If the optional argument *count* is given, only\n the first *count* occurrences are replaced.\n\nstr.rfind(sub[, start[, end]])\n\n Return the highest index in the string where substring *sub* is\n found, such that *sub* is contained within "s[start:end]".\n Optional arguments *start* and *end* are interpreted as in slice\n notation. Return "-1" on failure.\n\nstr.rindex(sub[, start[, end]])\n\n Like "rfind()" but raises "ValueError" when the substring *sub* is\n not found.\n\nstr.rjust(width[, fillchar])\n\n Return the string right justified in a string of length *width*.\n Padding is done using the specified *fillchar* (default is an ASCII\n space). The original string is returned if *width* is less than or\n equal to "len(s)".\n\nstr.rpartition(sep)\n\n Split the string at the last occurrence of *sep*, and return a\n 3-tuple containing the part before the separator, the separator\n itself, and the part after the separator. If the separator is not\n found, return a 3-tuple containing two empty strings, followed by\n the string itself.\n\nstr.rsplit(sep=None, maxsplit=-1)\n\n Return a list of the words in the string, using *sep* as the\n delimiter string. If *maxsplit* is given, at most *maxsplit* splits\n are done, the *rightmost* ones. If *sep* is not specified or\n "None", any whitespace string is a separator. Except for splitting\n from the right, "rsplit()" behaves like "split()" which is\n described in detail below.\n\nstr.rstrip([chars])\n\n Return a copy of the string with trailing characters removed. The\n *chars* argument is a string specifying the set of characters to be\n removed. If omitted or "None", the *chars* argument defaults to\n removing whitespace. The *chars* argument is not a suffix; rather,\n all combinations of its values are stripped:\n\n >>> \' spacious \'.rstrip()\n \' spacious\'\n >>> \'mississippi\'.rstrip(\'ipz\')\n \'mississ\'\n\nstr.split(sep=None, maxsplit=-1)\n\n Return a list of the words in the string, using *sep* as the\n delimiter string. If *maxsplit* is given, at most *maxsplit*\n splits are done (thus, the list will have at most "maxsplit+1"\n elements). If *maxsplit* is not specified or "-1", then there is\n no limit on the number of splits (all possible splits are made).\n\n If *sep* is given, consecutive delimiters are not grouped together\n and are deemed to delimit empty strings (for example,\n "\'1,,2\'.split(\',\')" returns "[\'1\', \'\', \'2\']"). The *sep* argument\n may consist of multiple characters (for example,\n "\'1<>2<>3\'.split(\'<>\')" returns "[\'1\', \'2\', \'3\']"). Splitting an\n empty string with a specified separator returns "[\'\']".\n\n For example:\n\n >>> \'1,2,3\'.split(\',\')\n [\'1\', \'2\', \'3\']\n >>> \'1,2,3\'.split(\',\', maxsplit=1)\n [\'1\', \'2,3\']\n >>> \'1,2,,3,\'.split(\',\')\n [\'1\', \'2\', \'\', \'3\', \'\']\n\n If *sep* is not specified or is "None", a different splitting\n algorithm is applied: runs of consecutive whitespace are regarded\n as a single separator, and the result will contain no empty strings\n at the start or end if the string has leading or trailing\n whitespace. Consequently, splitting an empty string or a string\n consisting of just whitespace with a "None" separator returns "[]".\n\n For example:\n\n >>> \'1 2 3\'.split()\n [\'1\', \'2\', \'3\']\n >>> \'1 2 3\'.split(maxsplit=1)\n [\'1\', \'2 3\']\n >>> \' 1 2 3 \'.split()\n [\'1\', \'2\', \'3\']\n\nstr.splitlines([keepends])\n\n Return a list of the lines in the string, breaking at line\n boundaries. Line breaks are not included in the resulting list\n unless *keepends* is given and true.\n\n This method splits on the following line boundaries. In\n particular, the boundaries are a superset of *universal newlines*.\n\n +-------------------------+-------------------------------+\n | Representation | Description |\n +=========================+===============================+\n | "\\n" | Line Feed |\n +-------------------------+-------------------------------+\n | "\\r" | Carriage Return |\n +-------------------------+-------------------------------+\n | "\\r\\n" | Carriage Return + Line Feed |\n +-------------------------+-------------------------------+\n | "\\v" or "\\x0b" | Line Tabulation |\n +-------------------------+-------------------------------+\n | "\\f" or "\\x0c" | Form Feed |\n +-------------------------+-------------------------------+\n | "\\x1c" | File Separator |\n +-------------------------+-------------------------------+\n | "\\x1d" | Group Separator |\n +-------------------------+-------------------------------+\n | "\\x1e" | Record Separator |\n +-------------------------+-------------------------------+\n | "\\x85" | Next Line (C1 Control Code) |\n +-------------------------+-------------------------------+\n | "\\u2028" | Line Separator |\n +-------------------------+-------------------------------+\n | "\\u2029" | Paragraph Separator |\n +-------------------------+-------------------------------+\n\n Changed in version 3.2: "\\v" and "\\f" added to list of line\n boundaries.\n\n For example:\n\n >>> \'ab c\\n\\nde fg\\rkl\\r\\n\'.splitlines()\n [\'ab c\', \'\', \'de fg\', \'kl\']\n >>> \'ab c\\n\\nde fg\\rkl\\r\\n\'.splitlines(keepends=True)\n [\'ab c\\n\', \'\\n\', \'de fg\\r\', \'kl\\r\\n\']\n\n Unlike "split()" when a delimiter string *sep* is given, this\n method returns an empty list for the empty string, and a terminal\n line break does not result in an extra line:\n\n >>> "".splitlines()\n []\n >>> "One line\\n".splitlines()\n [\'One line\']\n\n For comparison, "split(\'\\n\')" gives:\n\n >>> \'\'.split(\'\\n\')\n [\'\']\n >>> \'Two lines\\n\'.split(\'\\n\')\n [\'Two lines\', \'\']\n\nstr.startswith(prefix[, start[, end]])\n\n Return "True" if string starts with the *prefix*, otherwise return\n "False". *prefix* can also be a tuple of prefixes to look for.\n With optional *start*, test string beginning at that position.\n With optional *end*, stop comparing string at that position.\n\nstr.strip([chars])\n\n Return a copy of the string with the leading and trailing\n characters removed. The *chars* argument is a string specifying the\n set of characters to be removed. If omitted or "None", the *chars*\n argument defaults to removing whitespace. The *chars* argument is\n not a prefix or suffix; rather, all combinations of its values are\n stripped:\n\n >>> \' spacious \'.strip()\n \'spacious\'\n >>> \'www.example.com\'.strip(\'cmowz.\')\n \'example\'\n\n The outermost leading and trailing *chars* argument values are\n stripped from the string. Characters are removed from the leading\n end until reaching a string character that is not contained in the\n set of characters in *chars*. A similar action takes place on the\n trailing end. For example:\n\n >>> comment_string = \'#....... Section 3.2.1 Issue #32 .......\'\n >>> comment_string.strip(\'.#! \')\n \'Section 3.2.1 Issue #32\'\n\nstr.swapcase()\n\n Return a copy of the string with uppercase characters converted to\n lowercase and vice versa. Note that it is not necessarily true that\n "s.swapcase().swapcase() == s".\n\nstr.title()\n\n Return a titlecased version of the string where words start with an\n uppercase character and the remaining characters are lowercase.\n\n For example:\n\n >>> \'Hello world\'.title()\n \'Hello World\'\n\n The algorithm uses a simple language-independent definition of a\n word as groups of consecutive letters. The definition works in\n many contexts but it means that apostrophes in contractions and\n possessives form word boundaries, which may not be the desired\n result:\n\n >>> "they\'re bill\'s friends from the UK".title()\n "They\'Re Bill\'S Friends From The Uk"\n\n A workaround for apostrophes can be constructed using regular\n expressions:\n\n >>> import re\n >>> def titlecase(s):\n ... return re.sub(r"[A-Za-z]+(\'[A-Za-z]+)?",\n ... lambda mo: mo.group(0)[0].upper() +\n ... mo.group(0)[1:].lower(),\n ... s)\n ...\n >>> titlecase("they\'re bill\'s friends.")\n "They\'re Bill\'s Friends."\n\nstr.translate(table)\n\n Return a copy of the string in which each character has been mapped\n through the given translation table. The table must be an object\n that implements indexing via "__getitem__()", typically a *mapping*\n or *sequence*. When indexed by a Unicode ordinal (an integer), the\n table object can do any of the following: return a Unicode ordinal\n or a string, to map the character to one or more other characters;\n return "None", to delete the character from the return string; or\n raise a "LookupError" exception, to map the character to itself.\n\n You can use "str.maketrans()" to create a translation map from\n character-to-character mappings in different formats.\n\n See also the "codecs" module for a more flexible approach to custom\n character mappings.\n\nstr.upper()\n\n Return a copy of the string with all the cased characters [4]\n converted to uppercase. Note that "str.upper().isupper()" might be\n "False" if "s" contains uncased characters or if the Unicode\n category of the resulting character(s) is not "Lu" (Letter,\n uppercase), but e.g. "Lt" (Letter, titlecase).\n\n The uppercasing algorithm used is described in section 3.13 of the\n Unicode Standard.\n\nstr.zfill(width)\n\n Return a copy of the string left filled with ASCII "\'0\'" digits to\n make a string of length *width*. A leading sign prefix\n ("\'+\'"/"\'-\'") is handled by inserting the padding *after* the sign\n character rather than before. The original string is returned if\n *width* is less than or equal to "len(s)".\n\n For example:\n\n >>> "42".zfill(5)\n \'00042\'\n >>> "-42".zfill(5)\n \'-0042\'\n', - 'strings': u'\nString and Bytes literals\n*************************\n\nString literals are described by the following lexical definitions:\n\n stringliteral ::= [stringprefix](shortstring | longstring)\n stringprefix ::= "r" | "u" | "R" | "U"\n shortstring ::= "\'" shortstringitem* "\'" | \'"\' shortstringitem* \'"\'\n longstring ::= "\'\'\'" longstringitem* "\'\'\'" | \'"""\' longstringitem* \'"""\'\n shortstringitem ::= shortstringchar | stringescapeseq\n longstringitem ::= longstringchar | stringescapeseq\n shortstringchar ::= \n longstringchar ::= \n stringescapeseq ::= "\\" \n\n bytesliteral ::= bytesprefix(shortbytes | longbytes)\n bytesprefix ::= "b" | "B" | "br" | "Br" | "bR" | "BR" | "rb" | "rB" | "Rb" | "RB"\n shortbytes ::= "\'" shortbytesitem* "\'" | \'"\' shortbytesitem* \'"\'\n longbytes ::= "\'\'\'" longbytesitem* "\'\'\'" | \'"""\' longbytesitem* \'"""\'\n shortbytesitem ::= shortbyteschar | bytesescapeseq\n longbytesitem ::= longbyteschar | bytesescapeseq\n shortbyteschar ::= \n longbyteschar ::= \n bytesescapeseq ::= "\\" \n\nOne syntactic restriction not indicated by these productions is that\nwhitespace is not allowed between the "stringprefix" or "bytesprefix"\nand the rest of the literal. The source character set is defined by\nthe encoding declaration; it is UTF-8 if no encoding declaration is\ngiven in the source file; see section *Encoding declarations*.\n\nIn plain English: Both types of literals can be enclosed in matching\nsingle quotes ("\'") or double quotes ("""). They can also be enclosed\nin matching groups of three single or double quotes (these are\ngenerally referred to as *triple-quoted strings*). The backslash\n("\\") character is used to escape characters that otherwise have a\nspecial meaning, such as newline, backslash itself, or the quote\ncharacter.\n\nBytes literals are always prefixed with "\'b\'" or "\'B\'"; they produce\nan instance of the "bytes" type instead of the "str" type. They may\nonly contain ASCII characters; bytes with a numeric value of 128 or\ngreater must be expressed with escapes.\n\nAs of Python 3.3 it is possible again to prefix string literals with a\n"u" prefix to simplify maintenance of dual 2.x and 3.x codebases.\n\nBoth string and bytes literals may optionally be prefixed with a\nletter "\'r\'" or "\'R\'"; such strings are called *raw strings* and treat\nbackslashes as literal characters. As a result, in string literals,\n"\'\\U\'" and "\'\\u\'" escapes in raw strings are not treated specially.\nGiven that Python 2.x\'s raw unicode literals behave differently than\nPython 3.x\'s the "\'ur\'" syntax is not supported.\n\nNew in version 3.3: The "\'rb\'" prefix of raw bytes literals has been\nadded as a synonym of "\'br\'".\n\nNew in version 3.3: Support for the unicode legacy literal\n("u\'value\'") was reintroduced to simplify the maintenance of dual\nPython 2.x and 3.x codebases. See **PEP 414** for more information.\n\nIn triple-quoted literals, unescaped newlines and quotes are allowed\n(and are retained), except that three unescaped quotes in a row\nterminate the literal. (A "quote" is the character used to open the\nliteral, i.e. either "\'" or """.)\n\nUnless an "\'r\'" or "\'R\'" prefix is present, escape sequences in string\nand bytes literals are interpreted according to rules similar to those\nused by Standard C. The recognized escape sequences are:\n\n+-------------------+-----------------------------------+---------+\n| Escape Sequence | Meaning | Notes |\n+===================+===================================+=========+\n| "\\newline" | Backslash and newline ignored | |\n+-------------------+-----------------------------------+---------+\n| "\\\\" | Backslash ("\\") | |\n+-------------------+-----------------------------------+---------+\n| "\\\'" | Single quote ("\'") | |\n+-------------------+-----------------------------------+---------+\n| "\\"" | Double quote (""") | |\n+-------------------+-----------------------------------+---------+\n| "\\a" | ASCII Bell (BEL) | |\n+-------------------+-----------------------------------+---------+\n| "\\b" | ASCII Backspace (BS) | |\n+-------------------+-----------------------------------+---------+\n| "\\f" | ASCII Formfeed (FF) | |\n+-------------------+-----------------------------------+---------+\n| "\\n" | ASCII Linefeed (LF) | |\n+-------------------+-----------------------------------+---------+\n| "\\r" | ASCII Carriage Return (CR) | |\n+-------------------+-----------------------------------+---------+\n| "\\t" | ASCII Horizontal Tab (TAB) | |\n+-------------------+-----------------------------------+---------+\n| "\\v" | ASCII Vertical Tab (VT) | |\n+-------------------+-----------------------------------+---------+\n| "\\ooo" | Character with octal value *ooo* | (1,3) |\n+-------------------+-----------------------------------+---------+\n| "\\xhh" | Character with hex value *hh* | (2,3) |\n+-------------------+-----------------------------------+---------+\n\nEscape sequences only recognized in string literals are:\n\n+-------------------+-----------------------------------+---------+\n| Escape Sequence | Meaning | Notes |\n+===================+===================================+=========+\n| "\\N{name}" | Character named *name* in the | (4) |\n| | Unicode database | |\n+-------------------+-----------------------------------+---------+\n| "\\uxxxx" | Character with 16-bit hex value | (5) |\n| | *xxxx* | |\n+-------------------+-----------------------------------+---------+\n| "\\Uxxxxxxxx" | Character with 32-bit hex value | (6) |\n| | *xxxxxxxx* | |\n+-------------------+-----------------------------------+---------+\n\nNotes:\n\n1. As in Standard C, up to three octal digits are accepted.\n\n2. Unlike in Standard C, exactly two hex digits are required.\n\n3. In a bytes literal, hexadecimal and octal escapes denote the\n byte with the given value. In a string literal, these escapes\n denote a Unicode character with the given value.\n\n4. Changed in version 3.3: Support for name aliases [1] has been\n added.\n\n5. Individual code units which form parts of a surrogate pair can\n be encoded using this escape sequence. Exactly four hex digits are\n required.\n\n6. Any Unicode character can be encoded this way. Exactly eight\n hex digits are required.\n\nUnlike Standard C, all unrecognized escape sequences are left in the\nstring unchanged, i.e., *the backslash is left in the result*. (This\nbehavior is useful when debugging: if an escape sequence is mistyped,\nthe resulting output is more easily recognized as broken.) It is also\nimportant to note that the escape sequences only recognized in string\nliterals fall into the category of unrecognized escapes for bytes\nliterals.\n\nEven in a raw literal, quotes can be escaped with a backslash, but the\nbackslash remains in the result; for example, "r"\\""" is a valid\nstring literal consisting of two characters: a backslash and a double\nquote; "r"\\"" is not a valid string literal (even a raw string cannot\nend in an odd number of backslashes). Specifically, *a raw literal\ncannot end in a single backslash* (since the backslash would escape\nthe following quote character). Note also that a single backslash\nfollowed by a newline is interpreted as those two characters as part\nof the literal, *not* as a line continuation.\n', - 'subscriptions': u'\nSubscriptions\n*************\n\nA subscription selects an item of a sequence (string, tuple or list)\nor mapping (dictionary) object:\n\n subscription ::= primary "[" expression_list "]"\n\nThe primary must evaluate to an object that supports subscription\n(lists or dictionaries for example). User-defined objects can support\nsubscription by defining a "__getitem__()" method.\n\nFor built-in objects, there are two types of objects that support\nsubscription:\n\nIf the primary is a mapping, the expression list must evaluate to an\nobject whose value is one of the keys of the mapping, and the\nsubscription selects the value in the mapping that corresponds to that\nkey. (The expression list is a tuple except if it has exactly one\nitem.)\n\nIf the primary is a sequence, the expression (list) must evaluate to\nan integer or a slice (as discussed in the following section).\n\nThe formal syntax makes no special provision for negative indices in\nsequences; however, built-in sequences all provide a "__getitem__()"\nmethod that interprets negative indices by adding the length of the\nsequence to the index (so that "x[-1]" selects the last item of "x").\nThe resulting value must be a nonnegative integer less than the number\nof items in the sequence, and the subscription selects the item whose\nindex is that value (counting from zero). Since the support for\nnegative indices and slicing occurs in the object\'s "__getitem__()"\nmethod, subclasses overriding this method will need to explicitly add\nthat support.\n\nA string\'s items are characters. A character is not a separate data\ntype but a string of exactly one character.\n', - 'truth': u'\nTruth Value Testing\n*******************\n\nAny object can be tested for truth value, for use in an "if" or\n"while" condition or as operand of the Boolean operations below. The\nfollowing values are considered false:\n\n* "None"\n\n* "False"\n\n* zero of any numeric type, for example, "0", "0.0", "0j".\n\n* any empty sequence, for example, "\'\'", "()", "[]".\n\n* any empty mapping, for example, "{}".\n\n* instances of user-defined classes, if the class defines a\n "__bool__()" or "__len__()" method, when that method returns the\n integer zero or "bool" value "False". [1]\n\nAll other values are considered true --- so objects of many types are\nalways true.\n\nOperations and built-in functions that have a Boolean result always\nreturn "0" or "False" for false and "1" or "True" for true, unless\notherwise stated. (Important exception: the Boolean operations "or"\nand "and" always return one of their operands.)\n', - 'try': u'\nThe "try" statement\n*******************\n\nThe "try" statement specifies exception handlers and/or cleanup code\nfor a group of statements:\n\n try_stmt ::= try1_stmt | try2_stmt\n try1_stmt ::= "try" ":" suite\n ("except" [expression ["as" identifier]] ":" suite)+\n ["else" ":" suite]\n ["finally" ":" suite]\n try2_stmt ::= "try" ":" suite\n "finally" ":" suite\n\nThe "except" clause(s) specify one or more exception handlers. When no\nexception occurs in the "try" clause, no exception handler is\nexecuted. When an exception occurs in the "try" suite, a search for an\nexception handler is started. This search inspects the except clauses\nin turn until one is found that matches the exception. An expression-\nless except clause, if present, must be last; it matches any\nexception. For an except clause with an expression, that expression\nis evaluated, and the clause matches the exception if the resulting\nobject is "compatible" with the exception. An object is compatible\nwith an exception if it is the class or a base class of the exception\nobject or a tuple containing an item compatible with the exception.\n\nIf no except clause matches the exception, the search for an exception\nhandler continues in the surrounding code and on the invocation stack.\n[1]\n\nIf the evaluation of an expression in the header of an except clause\nraises an exception, the original search for a handler is canceled and\na search starts for the new exception in the surrounding code and on\nthe call stack (it is treated as if the entire "try" statement raised\nthe exception).\n\nWhen a matching except clause is found, the exception is assigned to\nthe target specified after the "as" keyword in that except clause, if\npresent, and the except clause\'s suite is executed. All except\nclauses must have an executable block. When the end of this block is\nreached, execution continues normally after the entire try statement.\n(This means that if two nested handlers exist for the same exception,\nand the exception occurs in the try clause of the inner handler, the\nouter handler will not handle the exception.)\n\nWhen an exception has been assigned using "as target", it is cleared\nat the end of the except clause. This is as if\n\n except E as N:\n foo\n\nwas translated to\n\n except E as N:\n try:\n foo\n finally:\n del N\n\nThis means the exception must be assigned to a different name to be\nable to refer to it after the except clause. Exceptions are cleared\nbecause with the traceback attached to them, they form a reference\ncycle with the stack frame, keeping all locals in that frame alive\nuntil the next garbage collection occurs.\n\nBefore an except clause\'s suite is executed, details about the\nexception are stored in the "sys" module and can be accessed via\n"sys.exc_info()". "sys.exc_info()" returns a 3-tuple consisting of the\nexception class, the exception instance and a traceback object (see\nsection *The standard type hierarchy*) identifying the point in the\nprogram where the exception occurred. "sys.exc_info()" values are\nrestored to their previous values (before the call) when returning\nfrom a function that handled an exception.\n\nThe optional "else" clause is executed if and when control flows off\nthe end of the "try" clause. [2] Exceptions in the "else" clause are\nnot handled by the preceding "except" clauses.\n\nIf "finally" is present, it specifies a \'cleanup\' handler. The "try"\nclause is executed, including any "except" and "else" clauses. If an\nexception occurs in any of the clauses and is not handled, the\nexception is temporarily saved. The "finally" clause is executed. If\nthere is a saved exception it is re-raised at the end of the "finally"\nclause. If the "finally" clause raises another exception, the saved\nexception is set as the context of the new exception. If the "finally"\nclause executes a "return" or "break" statement, the saved exception\nis discarded:\n\n >>> def f():\n ... try:\n ... 1/0\n ... finally:\n ... return 42\n ...\n >>> f()\n 42\n\nThe exception information is not available to the program during\nexecution of the "finally" clause.\n\nWhen a "return", "break" or "continue" statement is executed in the\n"try" suite of a "try"..."finally" statement, the "finally" clause is\nalso executed \'on the way out.\' A "continue" statement is illegal in\nthe "finally" clause. (The reason is a problem with the current\nimplementation --- this restriction may be lifted in the future).\n\nThe return value of a function is determined by the last "return"\nstatement executed. Since the "finally" clause always executes, a\n"return" statement executed in the "finally" clause will always be the\nlast one executed:\n\n >>> def foo():\n ... try:\n ... return \'try\'\n ... finally:\n ... return \'finally\'\n ...\n >>> foo()\n \'finally\'\n\nAdditional information on exceptions can be found in section\n*Exceptions*, and information on using the "raise" statement to\ngenerate exceptions may be found in section *The raise statement*.\n', - 'types': u'\nThe standard type hierarchy\n***************************\n\nBelow is a list of the types that are built into Python. Extension\nmodules (written in C, Java, or other languages, depending on the\nimplementation) can define additional types. Future versions of\nPython may add types to the type hierarchy (e.g., rational numbers,\nefficiently stored arrays of integers, etc.), although such additions\nwill often be provided via the standard library instead.\n\nSome of the type descriptions below contain a paragraph listing\n\'special attributes.\' These are attributes that provide access to the\nimplementation and are not intended for general use. Their definition\nmay change in the future.\n\nNone\n This type has a single value. There is a single object with this\n value. This object is accessed through the built-in name "None". It\n is used to signify the absence of a value in many situations, e.g.,\n it is returned from functions that don\'t explicitly return\n anything. Its truth value is false.\n\nNotImplemented\n This type has a single value. There is a single object with this\n value. This object is accessed through the built-in name\n "NotImplemented". Numeric methods and rich comparison methods\n should return this value if they do not implement the operation for\n the operands provided. (The interpreter will then try the\n reflected operation, or some other fallback, depending on the\n operator.) Its truth value is true.\n\n See *Implementing the arithmetic operations* for more details.\n\nEllipsis\n This type has a single value. There is a single object with this\n value. This object is accessed through the literal "..." or the\n built-in name "Ellipsis". Its truth value is true.\n\n"numbers.Number"\n These are created by numeric literals and returned as results by\n arithmetic operators and arithmetic built-in functions. Numeric\n objects are immutable; once created their value never changes.\n Python numbers are of course strongly related to mathematical\n numbers, but subject to the limitations of numerical representation\n in computers.\n\n Python distinguishes between integers, floating point numbers, and\n complex numbers:\n\n "numbers.Integral"\n These represent elements from the mathematical set of integers\n (positive and negative).\n\n There are two types of integers:\n\n Integers ("int")\n\n These represent numbers in an unlimited range, subject to\n available (virtual) memory only. For the purpose of shift\n and mask operations, a binary representation is assumed, and\n negative numbers are represented in a variant of 2\'s\n complement which gives the illusion of an infinite string of\n sign bits extending to the left.\n\n Booleans ("bool")\n These represent the truth values False and True. The two\n objects representing the values "False" and "True" are the\n only Boolean objects. The Boolean type is a subtype of the\n integer type, and Boolean values behave like the values 0 and\n 1, respectively, in almost all contexts, the exception being\n that when converted to a string, the strings ""False"" or\n ""True"" are returned, respectively.\n\n The rules for integer representation are intended to give the\n most meaningful interpretation of shift and mask operations\n involving negative integers.\n\n "numbers.Real" ("float")\n These represent machine-level double precision floating point\n numbers. You are at the mercy of the underlying machine\n architecture (and C or Java implementation) for the accepted\n range and handling of overflow. Python does not support single-\n precision floating point numbers; the savings in processor and\n memory usage that are usually the reason for using these are\n dwarfed by the overhead of using objects in Python, so there is\n no reason to complicate the language with two kinds of floating\n point numbers.\n\n "numbers.Complex" ("complex")\n These represent complex numbers as a pair of machine-level\n double precision floating point numbers. The same caveats apply\n as for floating point numbers. The real and imaginary parts of a\n complex number "z" can be retrieved through the read-only\n attributes "z.real" and "z.imag".\n\nSequences\n These represent finite ordered sets indexed by non-negative\n numbers. The built-in function "len()" returns the number of items\n of a sequence. When the length of a sequence is *n*, the index set\n contains the numbers 0, 1, ..., *n*-1. Item *i* of sequence *a* is\n selected by "a[i]".\n\n Sequences also support slicing: "a[i:j]" selects all items with\n index *k* such that *i* "<=" *k* "<" *j*. When used as an\n expression, a slice is a sequence of the same type. This implies\n that the index set is renumbered so that it starts at 0.\n\n Some sequences also support "extended slicing" with a third "step"\n parameter: "a[i:j:k]" selects all items of *a* with index *x* where\n "x = i + n*k", *n* ">=" "0" and *i* "<=" *x* "<" *j*.\n\n Sequences are distinguished according to their mutability:\n\n Immutable sequences\n An object of an immutable sequence type cannot change once it is\n created. (If the object contains references to other objects,\n these other objects may be mutable and may be changed; however,\n the collection of objects directly referenced by an immutable\n object cannot change.)\n\n The following types are immutable sequences:\n\n Strings\n A string is a sequence of values that represent Unicode code\n points. All the code points in the range "U+0000 - U+10FFFF"\n can be represented in a string. Python doesn\'t have a "char"\n type; instead, every code point in the string is represented\n as a string object with length "1". The built-in function\n "ord()" converts a code point from its string form to an\n integer in the range "0 - 10FFFF"; "chr()" converts an\n integer in the range "0 - 10FFFF" to the corresponding length\n "1" string object. "str.encode()" can be used to convert a\n "str" to "bytes" using the given text encoding, and\n "bytes.decode()" can be used to achieve the opposite.\n\n Tuples\n The items of a tuple are arbitrary Python objects. Tuples of\n two or more items are formed by comma-separated lists of\n expressions. A tuple of one item (a \'singleton\') can be\n formed by affixing a comma to an expression (an expression by\n itself does not create a tuple, since parentheses must be\n usable for grouping of expressions). An empty tuple can be\n formed by an empty pair of parentheses.\n\n Bytes\n A bytes object is an immutable array. The items are 8-bit\n bytes, represented by integers in the range 0 <= x < 256.\n Bytes literals (like "b\'abc\'") and the built-in function\n "bytes()" can be used to construct bytes objects. Also,\n bytes objects can be decoded to strings via the "decode()"\n method.\n\n Mutable sequences\n Mutable sequences can be changed after they are created. The\n subscription and slicing notations can be used as the target of\n assignment and "del" (delete) statements.\n\n There are currently two intrinsic mutable sequence types:\n\n Lists\n The items of a list are arbitrary Python objects. Lists are\n formed by placing a comma-separated list of expressions in\n square brackets. (Note that there are no special cases needed\n to form lists of length 0 or 1.)\n\n Byte Arrays\n A bytearray object is a mutable array. They are created by\n the built-in "bytearray()" constructor. Aside from being\n mutable (and hence unhashable), byte arrays otherwise provide\n the same interface and functionality as immutable bytes\n objects.\n\n The extension module "array" provides an additional example of a\n mutable sequence type, as does the "collections" module.\n\nSet types\n These represent unordered, finite sets of unique, immutable\n objects. As such, they cannot be indexed by any subscript. However,\n they can be iterated over, and the built-in function "len()"\n returns the number of items in a set. Common uses for sets are fast\n membership testing, removing duplicates from a sequence, and\n computing mathematical operations such as intersection, union,\n difference, and symmetric difference.\n\n For set elements, the same immutability rules apply as for\n dictionary keys. Note that numeric types obey the normal rules for\n numeric comparison: if two numbers compare equal (e.g., "1" and\n "1.0"), only one of them can be contained in a set.\n\n There are currently two intrinsic set types:\n\n Sets\n These represent a mutable set. They are created by the built-in\n "set()" constructor and can be modified afterwards by several\n methods, such as "add()".\n\n Frozen sets\n These represent an immutable set. They are created by the\n built-in "frozenset()" constructor. As a frozenset is immutable\n and *hashable*, it can be used again as an element of another\n set, or as a dictionary key.\n\nMappings\n These represent finite sets of objects indexed by arbitrary index\n sets. The subscript notation "a[k]" selects the item indexed by "k"\n from the mapping "a"; this can be used in expressions and as the\n target of assignments or "del" statements. The built-in function\n "len()" returns the number of items in a mapping.\n\n There is currently a single intrinsic mapping type:\n\n Dictionaries\n These represent finite sets of objects indexed by nearly\n arbitrary values. The only types of values not acceptable as\n keys are values containing lists or dictionaries or other\n mutable types that are compared by value rather than by object\n identity, the reason being that the efficient implementation of\n dictionaries requires a key\'s hash value to remain constant.\n Numeric types used for keys obey the normal rules for numeric\n comparison: if two numbers compare equal (e.g., "1" and "1.0")\n then they can be used interchangeably to index the same\n dictionary entry.\n\n Dictionaries are mutable; they can be created by the "{...}"\n notation (see section *Dictionary displays*).\n\n The extension modules "dbm.ndbm" and "dbm.gnu" provide\n additional examples of mapping types, as does the "collections"\n module.\n\nCallable types\n These are the types to which the function call operation (see\n section *Calls*) can be applied:\n\n User-defined functions\n A user-defined function object is created by a function\n definition (see section *Function definitions*). It should be\n called with an argument list containing the same number of items\n as the function\'s formal parameter list.\n\n Special attributes:\n\n +---------------------------+---------------------------------+-------------+\n | Attribute | Meaning | |\n +===========================+=================================+=============+\n | "__doc__" | The function\'s documentation | Writable |\n | | string, or "None" if | |\n | | unavailable; not inherited by | |\n | | subclasses | |\n +---------------------------+---------------------------------+-------------+\n | "__name__" | The function\'s name | Writable |\n +---------------------------+---------------------------------+-------------+\n | "__qualname__" | The function\'s *qualified name* | Writable |\n | | New in version 3.3. | |\n +---------------------------+---------------------------------+-------------+\n | "__module__" | The name of the module the | Writable |\n | | function was defined in, or | |\n | | "None" if unavailable. | |\n +---------------------------+---------------------------------+-------------+\n | "__defaults__" | A tuple containing default | Writable |\n | | argument values for those | |\n | | arguments that have defaults, | |\n | | or "None" if no arguments have | |\n | | a default value | |\n +---------------------------+---------------------------------+-------------+\n | "__code__" | The code object representing | Writable |\n | | the compiled function body. | |\n +---------------------------+---------------------------------+-------------+\n | "__globals__" | A reference to the dictionary | Read-only |\n | | that holds the function\'s | |\n | | global variables --- the global | |\n | | namespace of the module in | |\n | | which the function was defined. | |\n +---------------------------+---------------------------------+-------------+\n | "__dict__" | The namespace supporting | Writable |\n | | arbitrary function attributes. | |\n +---------------------------+---------------------------------+-------------+\n | "__closure__" | "None" or a tuple of cells that | Read-only |\n | | contain bindings for the | |\n | | function\'s free variables. | |\n +---------------------------+---------------------------------+-------------+\n | "__annotations__" | A dict containing annotations | Writable |\n | | of parameters. The keys of the | |\n | | dict are the parameter names, | |\n | | and "\'return\'" for the return | |\n | | annotation, if provided. | |\n +---------------------------+---------------------------------+-------------+\n | "__kwdefaults__" | A dict containing defaults for | Writable |\n | | keyword-only parameters. | |\n +---------------------------+---------------------------------+-------------+\n\n Most of the attributes labelled "Writable" check the type of the\n assigned value.\n\n Function objects also support getting and setting arbitrary\n attributes, which can be used, for example, to attach metadata\n to functions. Regular attribute dot-notation is used to get and\n set such attributes. *Note that the current implementation only\n supports function attributes on user-defined functions. Function\n attributes on built-in functions may be supported in the\n future.*\n\n Additional information about a function\'s definition can be\n retrieved from its code object; see the description of internal\n types below.\n\n Instance methods\n An instance method object combines a class, a class instance and\n any callable object (normally a user-defined function).\n\n Special read-only attributes: "__self__" is the class instance\n object, "__func__" is the function object; "__doc__" is the\n method\'s documentation (same as "__func__.__doc__"); "__name__"\n is the method name (same as "__func__.__name__"); "__module__"\n is the name of the module the method was defined in, or "None"\n if unavailable.\n\n Methods also support accessing (but not setting) the arbitrary\n function attributes on the underlying function object.\n\n User-defined method objects may be created when getting an\n attribute of a class (perhaps via an instance of that class), if\n that attribute is a user-defined function object or a class\n method object.\n\n When an instance method object is created by retrieving a user-\n defined function object from a class via one of its instances,\n its "__self__" attribute is the instance, and the method object\n is said to be bound. The new method\'s "__func__" attribute is\n the original function object.\n\n When a user-defined method object is created by retrieving\n another method object from a class or instance, the behaviour is\n the same as for a function object, except that the "__func__"\n attribute of the new instance is not the original method object\n but its "__func__" attribute.\n\n When an instance method object is created by retrieving a class\n method object from a class or instance, its "__self__" attribute\n is the class itself, and its "__func__" attribute is the\n function object underlying the class method.\n\n When an instance method object is called, the underlying\n function ("__func__") is called, inserting the class instance\n ("__self__") in front of the argument list. For instance, when\n "C" is a class which contains a definition for a function "f()",\n and "x" is an instance of "C", calling "x.f(1)" is equivalent to\n calling "C.f(x, 1)".\n\n When an instance method object is derived from a class method\n object, the "class instance" stored in "__self__" will actually\n be the class itself, so that calling either "x.f(1)" or "C.f(1)"\n is equivalent to calling "f(C,1)" where "f" is the underlying\n function.\n\n Note that the transformation from function object to instance\n method object happens each time the attribute is retrieved from\n the instance. In some cases, a fruitful optimization is to\n assign the attribute to a local variable and call that local\n variable. Also notice that this transformation only happens for\n user-defined functions; other callable objects (and all non-\n callable objects) are retrieved without transformation. It is\n also important to note that user-defined functions which are\n attributes of a class instance are not converted to bound\n methods; this *only* happens when the function is an attribute\n of the class.\n\n Generator functions\n A function or method which uses the "yield" statement (see\n section *The yield statement*) is called a *generator function*.\n Such a function, when called, always returns an iterator object\n which can be used to execute the body of the function: calling\n the iterator\'s "iterator.__next__()" method will cause the\n function to execute until it provides a value using the "yield"\n statement. When the function executes a "return" statement or\n falls off the end, a "StopIteration" exception is raised and the\n iterator will have reached the end of the set of values to be\n returned.\n\n Coroutine functions\n A function or method which is defined using "async def" is\n called a *coroutine function*. Such a function, when called,\n returns a *coroutine* object. It may contain "await"\n expressions, as well as "async with" and "async for" statements.\n See also the *Coroutine Objects* section.\n\n Built-in functions\n A built-in function object is a wrapper around a C function.\n Examples of built-in functions are "len()" and "math.sin()"\n ("math" is a standard built-in module). The number and type of\n the arguments are determined by the C function. Special read-\n only attributes: "__doc__" is the function\'s documentation\n string, or "None" if unavailable; "__name__" is the function\'s\n name; "__self__" is set to "None" (but see the next item);\n "__module__" is the name of the module the function was defined\n in or "None" if unavailable.\n\n Built-in methods\n This is really a different disguise of a built-in function, this\n time containing an object passed to the C function as an\n implicit extra argument. An example of a built-in method is\n "alist.append()", assuming *alist* is a list object. In this\n case, the special read-only attribute "__self__" is set to the\n object denoted by *alist*.\n\n Classes\n Classes are callable. These objects normally act as factories\n for new instances of themselves, but variations are possible for\n class types that override "__new__()". The arguments of the\n call are passed to "__new__()" and, in the typical case, to\n "__init__()" to initialize the new instance.\n\n Class Instances\n Instances of arbitrary classes can be made callable by defining\n a "__call__()" method in their class.\n\nModules\n Modules are a basic organizational unit of Python code, and are\n created by the *import system* as invoked either by the "import"\n statement (see "import"), or by calling functions such as\n "importlib.import_module()" and built-in "__import__()". A module\n object has a namespace implemented by a dictionary object (this is\n the dictionary referenced by the "__globals__" attribute of\n functions defined in the module). Attribute references are\n translated to lookups in this dictionary, e.g., "m.x" is equivalent\n to "m.__dict__["x"]". A module object does not contain the code\n object used to initialize the module (since it isn\'t needed once\n the initialization is done).\n\n Attribute assignment updates the module\'s namespace dictionary,\n e.g., "m.x = 1" is equivalent to "m.__dict__["x"] = 1".\n\n Special read-only attribute: "__dict__" is the module\'s namespace\n as a dictionary object.\n\n **CPython implementation detail:** Because of the way CPython\n clears module dictionaries, the module dictionary will be cleared\n when the module falls out of scope even if the dictionary still has\n live references. To avoid this, copy the dictionary or keep the\n module around while using its dictionary directly.\n\n Predefined (writable) attributes: "__name__" is the module\'s name;\n "__doc__" is the module\'s documentation string, or "None" if\n unavailable; "__file__" is the pathname of the file from which the\n module was loaded, if it was loaded from a file. The "__file__"\n attribute may be missing for certain types of modules, such as C\n modules that are statically linked into the interpreter; for\n extension modules loaded dynamically from a shared library, it is\n the pathname of the shared library file.\n\nCustom classes\n Custom class types are typically created by class definitions (see\n section *Class definitions*). A class has a namespace implemented\n by a dictionary object. Class attribute references are translated\n to lookups in this dictionary, e.g., "C.x" is translated to\n "C.__dict__["x"]" (although there are a number of hooks which allow\n for other means of locating attributes). When the attribute name is\n not found there, the attribute search continues in the base\n classes. This search of the base classes uses the C3 method\n resolution order which behaves correctly even in the presence of\n \'diamond\' inheritance structures where there are multiple\n inheritance paths leading back to a common ancestor. Additional\n details on the C3 MRO used by Python can be found in the\n documentation accompanying the 2.3 release at\n https://www.python.org/download/releases/2.3/mro/.\n\n When a class attribute reference (for class "C", say) would yield a\n class method object, it is transformed into an instance method\n object whose "__self__" attributes is "C". When it would yield a\n static method object, it is transformed into the object wrapped by\n the static method object. See section *Implementing Descriptors*\n for another way in which attributes retrieved from a class may\n differ from those actually contained in its "__dict__".\n\n Class attribute assignments update the class\'s dictionary, never\n the dictionary of a base class.\n\n A class object can be called (see above) to yield a class instance\n (see below).\n\n Special attributes: "__name__" is the class name; "__module__" is\n the module name in which the class was defined; "__dict__" is the\n dictionary containing the class\'s namespace; "__bases__" is a tuple\n (possibly empty or a singleton) containing the base classes, in the\n order of their occurrence in the base class list; "__doc__" is the\n class\'s documentation string, or None if undefined.\n\nClass instances\n A class instance is created by calling a class object (see above).\n A class instance has a namespace implemented as a dictionary which\n is the first place in which attribute references are searched.\n When an attribute is not found there, and the instance\'s class has\n an attribute by that name, the search continues with the class\n attributes. If a class attribute is found that is a user-defined\n function object, it is transformed into an instance method object\n whose "__self__" attribute is the instance. Static method and\n class method objects are also transformed; see above under\n "Classes". See section *Implementing Descriptors* for another way\n in which attributes of a class retrieved via its instances may\n differ from the objects actually stored in the class\'s "__dict__".\n If no class attribute is found, and the object\'s class has a\n "__getattr__()" method, that is called to satisfy the lookup.\n\n Attribute assignments and deletions update the instance\'s\n dictionary, never a class\'s dictionary. If the class has a\n "__setattr__()" or "__delattr__()" method, this is called instead\n of updating the instance dictionary directly.\n\n Class instances can pretend to be numbers, sequences, or mappings\n if they have methods with certain special names. See section\n *Special method names*.\n\n Special attributes: "__dict__" is the attribute dictionary;\n "__class__" is the instance\'s class.\n\nI/O objects (also known as file objects)\n A *file object* represents an open file. Various shortcuts are\n available to create file objects: the "open()" built-in function,\n and also "os.popen()", "os.fdopen()", and the "makefile()" method\n of socket objects (and perhaps by other functions or methods\n provided by extension modules).\n\n The objects "sys.stdin", "sys.stdout" and "sys.stderr" are\n initialized to file objects corresponding to the interpreter\'s\n standard input, output and error streams; they are all open in text\n mode and therefore follow the interface defined by the\n "io.TextIOBase" abstract class.\n\nInternal types\n A few types used internally by the interpreter are exposed to the\n user. Their definitions may change with future versions of the\n interpreter, but they are mentioned here for completeness.\n\n Code objects\n Code objects represent *byte-compiled* executable Python code,\n or *bytecode*. The difference between a code object and a\n function object is that the function object contains an explicit\n reference to the function\'s globals (the module in which it was\n defined), while a code object contains no context; also the\n default argument values are stored in the function object, not\n in the code object (because they represent values calculated at\n run-time). Unlike function objects, code objects are immutable\n and contain no references (directly or indirectly) to mutable\n objects.\n\n Special read-only attributes: "co_name" gives the function name;\n "co_argcount" is the number of positional arguments (including\n arguments with default values); "co_nlocals" is the number of\n local variables used by the function (including arguments);\n "co_varnames" is a tuple containing the names of the local\n variables (starting with the argument names); "co_cellvars" is a\n tuple containing the names of local variables that are\n referenced by nested functions; "co_freevars" is a tuple\n containing the names of free variables; "co_code" is a string\n representing the sequence of bytecode instructions; "co_consts"\n is a tuple containing the literals used by the bytecode;\n "co_names" is a tuple containing the names used by the bytecode;\n "co_filename" is the filename from which the code was compiled;\n "co_firstlineno" is the first line number of the function;\n "co_lnotab" is a string encoding the mapping from bytecode\n offsets to line numbers (for details see the source code of the\n interpreter); "co_stacksize" is the required stack size\n (including local variables); "co_flags" is an integer encoding a\n number of flags for the interpreter.\n\n The following flag bits are defined for "co_flags": bit "0x04"\n is set if the function uses the "*arguments" syntax to accept an\n arbitrary number of positional arguments; bit "0x08" is set if\n the function uses the "**keywords" syntax to accept arbitrary\n keyword arguments; bit "0x20" is set if the function is a\n generator.\n\n Future feature declarations ("from __future__ import division")\n also use bits in "co_flags" to indicate whether a code object\n was compiled with a particular feature enabled: bit "0x2000" is\n set if the function was compiled with future division enabled;\n bits "0x10" and "0x1000" were used in earlier versions of\n Python.\n\n Other bits in "co_flags" are reserved for internal use.\n\n If a code object represents a function, the first item in\n "co_consts" is the documentation string of the function, or\n "None" if undefined.\n\n Frame objects\n Frame objects represent execution frames. They may occur in\n traceback objects (see below).\n\n Special read-only attributes: "f_back" is to the previous stack\n frame (towards the caller), or "None" if this is the bottom\n stack frame; "f_code" is the code object being executed in this\n frame; "f_locals" is the dictionary used to look up local\n variables; "f_globals" is used for global variables;\n "f_builtins" is used for built-in (intrinsic) names; "f_lasti"\n gives the precise instruction (this is an index into the\n bytecode string of the code object).\n\n Special writable attributes: "f_trace", if not "None", is a\n function called at the start of each source code line (this is\n used by the debugger); "f_lineno" is the current line number of\n the frame --- writing to this from within a trace function jumps\n to the given line (only for the bottom-most frame). A debugger\n can implement a Jump command (aka Set Next Statement) by writing\n to f_lineno.\n\n Frame objects support one method:\n\n frame.clear()\n\n This method clears all references to local variables held by\n the frame. Also, if the frame belonged to a generator, the\n generator is finalized. This helps break reference cycles\n involving frame objects (for example when catching an\n exception and storing its traceback for later use).\n\n "RuntimeError" is raised if the frame is currently executing.\n\n New in version 3.4.\n\n Traceback objects\n Traceback objects represent a stack trace of an exception. A\n traceback object is created when an exception occurs. When the\n search for an exception handler unwinds the execution stack, at\n each unwound level a traceback object is inserted in front of\n the current traceback. When an exception handler is entered,\n the stack trace is made available to the program. (See section\n *The try statement*.) It is accessible as the third item of the\n tuple returned by "sys.exc_info()". When the program contains no\n suitable handler, the stack trace is written (nicely formatted)\n to the standard error stream; if the interpreter is interactive,\n it is also made available to the user as "sys.last_traceback".\n\n Special read-only attributes: "tb_next" is the next level in the\n stack trace (towards the frame where the exception occurred), or\n "None" if there is no next level; "tb_frame" points to the\n execution frame of the current level; "tb_lineno" gives the line\n number where the exception occurred; "tb_lasti" indicates the\n precise instruction. The line number and last instruction in\n the traceback may differ from the line number of its frame\n object if the exception occurred in a "try" statement with no\n matching except clause or with a finally clause.\n\n Slice objects\n Slice objects are used to represent slices for "__getitem__()"\n methods. They are also created by the built-in "slice()"\n function.\n\n Special read-only attributes: "start" is the lower bound; "stop"\n is the upper bound; "step" is the step value; each is "None" if\n omitted. These attributes can have any type.\n\n Slice objects support one method:\n\n slice.indices(self, length)\n\n This method takes a single integer argument *length* and\n computes information about the slice that the slice object\n would describe if applied to a sequence of *length* items.\n It returns a tuple of three integers; respectively these are\n the *start* and *stop* indices and the *step* or stride\n length of the slice. Missing or out-of-bounds indices are\n handled in a manner consistent with regular slices.\n\n Static method objects\n Static method objects provide a way of defeating the\n transformation of function objects to method objects described\n above. A static method object is a wrapper around any other\n object, usually a user-defined method object. When a static\n method object is retrieved from a class or a class instance, the\n object actually returned is the wrapped object, which is not\n subject to any further transformation. Static method objects are\n not themselves callable, although the objects they wrap usually\n are. Static method objects are created by the built-in\n "staticmethod()" constructor.\n\n Class method objects\n A class method object, like a static method object, is a wrapper\n around another object that alters the way in which that object\n is retrieved from classes and class instances. The behaviour of\n class method objects upon such retrieval is described above,\n under "User-defined methods". Class method objects are created\n by the built-in "classmethod()" constructor.\n', - 'typesfunctions': u'\nFunctions\n*********\n\nFunction objects are created by function definitions. The only\noperation on a function object is to call it: "func(argument-list)".\n\nThere are really two flavors of function objects: built-in functions\nand user-defined functions. Both support the same operation (to call\nthe function), but the implementation is different, hence the\ndifferent object types.\n\nSee *Function definitions* for more information.\n', - 'typesmapping': u'\nMapping Types --- "dict"\n************************\n\nA *mapping* object maps *hashable* values to arbitrary objects.\nMappings are mutable objects. There is currently only one standard\nmapping type, the *dictionary*. (For other containers see the built-\nin "list", "set", and "tuple" classes, and the "collections" module.)\n\nA dictionary\'s keys are *almost* arbitrary values. Values that are\nnot *hashable*, that is, values containing lists, dictionaries or\nother mutable types (that are compared by value rather than by object\nidentity) may not be used as keys. Numeric types used for keys obey\nthe normal rules for numeric comparison: if two numbers compare equal\n(such as "1" and "1.0") then they can be used interchangeably to index\nthe same dictionary entry. (Note however, that since computers store\nfloating-point numbers as approximations it is usually unwise to use\nthem as dictionary keys.)\n\nDictionaries can be created by placing a comma-separated list of "key:\nvalue" pairs within braces, for example: "{\'jack\': 4098, \'sjoerd\':\n4127}" or "{4098: \'jack\', 4127: \'sjoerd\'}", or by the "dict"\nconstructor.\n\nclass class dict(**kwarg)\nclass class dict(mapping, **kwarg)\nclass class dict(iterable, **kwarg)\n\n Return a new dictionary initialized from an optional positional\n argument and a possibly empty set of keyword arguments.\n\n If no positional argument is given, an empty dictionary is created.\n If a positional argument is given and it is a mapping object, a\n dictionary is created with the same key-value pairs as the mapping\n object. Otherwise, the positional argument must be an *iterable*\n object. Each item in the iterable must itself be an iterable with\n exactly two objects. The first object of each item becomes a key\n in the new dictionary, and the second object the corresponding\n value. If a key occurs more than once, the last value for that key\n becomes the corresponding value in the new dictionary.\n\n If keyword arguments are given, the keyword arguments and their\n values are added to the dictionary created from the positional\n argument. If a key being added is already present, the value from\n the keyword argument replaces the value from the positional\n argument.\n\n To illustrate, the following examples all return a dictionary equal\n to "{"one": 1, "two": 2, "three": 3}":\n\n >>> a = dict(one=1, two=2, three=3)\n >>> b = {\'one\': 1, \'two\': 2, \'three\': 3}\n >>> c = dict(zip([\'one\', \'two\', \'three\'], [1, 2, 3]))\n >>> d = dict([(\'two\', 2), (\'one\', 1), (\'three\', 3)])\n >>> e = dict({\'three\': 3, \'one\': 1, \'two\': 2})\n >>> a == b == c == d == e\n True\n\n Providing keyword arguments as in the first example only works for\n keys that are valid Python identifiers. Otherwise, any valid keys\n can be used.\n\n These are the operations that dictionaries support (and therefore,\n custom mapping types should support too):\n\n len(d)\n\n Return the number of items in the dictionary *d*.\n\n d[key]\n\n Return the item of *d* with key *key*. Raises a "KeyError" if\n *key* is not in the map.\n\n If a subclass of dict defines a method "__missing__()" and *key*\n is not present, the "d[key]" operation calls that method with\n the key *key* as argument. The "d[key]" operation then returns\n or raises whatever is returned or raised by the\n "__missing__(key)" call. No other operations or methods invoke\n "__missing__()". If "__missing__()" is not defined, "KeyError"\n is raised. "__missing__()" must be a method; it cannot be an\n instance variable:\n\n >>> class Counter(dict):\n ... def __missing__(self, key):\n ... return 0\n >>> c = Counter()\n >>> c[\'red\']\n 0\n >>> c[\'red\'] += 1\n >>> c[\'red\']\n 1\n\n The example above shows part of the implementation of\n "collections.Counter". A different "__missing__" method is used\n by "collections.defaultdict".\n\n d[key] = value\n\n Set "d[key]" to *value*.\n\n del d[key]\n\n Remove "d[key]" from *d*. Raises a "KeyError" if *key* is not\n in the map.\n\n key in d\n\n Return "True" if *d* has a key *key*, else "False".\n\n key not in d\n\n Equivalent to "not key in d".\n\n iter(d)\n\n Return an iterator over the keys of the dictionary. This is a\n shortcut for "iter(d.keys())".\n\n clear()\n\n Remove all items from the dictionary.\n\n copy()\n\n Return a shallow copy of the dictionary.\n\n classmethod fromkeys(seq[, value])\n\n Create a new dictionary with keys from *seq* and values set to\n *value*.\n\n "fromkeys()" is a class method that returns a new dictionary.\n *value* defaults to "None".\n\n get(key[, default])\n\n Return the value for *key* if *key* is in the dictionary, else\n *default*. If *default* is not given, it defaults to "None", so\n that this method never raises a "KeyError".\n\n items()\n\n Return a new view of the dictionary\'s items ("(key, value)"\n pairs). See the *documentation of view objects*.\n\n keys()\n\n Return a new view of the dictionary\'s keys. See the\n *documentation of view objects*.\n\n pop(key[, default])\n\n If *key* is in the dictionary, remove it and return its value,\n else return *default*. If *default* is not given and *key* is\n not in the dictionary, a "KeyError" is raised.\n\n popitem()\n\n Remove and return an arbitrary "(key, value)" pair from the\n dictionary.\n\n "popitem()" is useful to destructively iterate over a\n dictionary, as often used in set algorithms. If the dictionary\n is empty, calling "popitem()" raises a "KeyError".\n\n setdefault(key[, default])\n\n If *key* is in the dictionary, return its value. If not, insert\n *key* with a value of *default* and return *default*. *default*\n defaults to "None".\n\n update([other])\n\n Update the dictionary with the key/value pairs from *other*,\n overwriting existing keys. Return "None".\n\n "update()" accepts either another dictionary object or an\n iterable of key/value pairs (as tuples or other iterables of\n length two). If keyword arguments are specified, the dictionary\n is then updated with those key/value pairs: "d.update(red=1,\n blue=2)".\n\n values()\n\n Return a new view of the dictionary\'s values. See the\n *documentation of view objects*.\n\n Dictionaries compare equal if and only if they have the same "(key,\n value)" pairs. Order comparisons (\'<\', \'<=\', \'>=\', \'>\') raise\n "TypeError".\n\nSee also: "types.MappingProxyType" can be used to create a read-only\n view of a "dict".\n\n\nDictionary view objects\n=======================\n\nThe objects returned by "dict.keys()", "dict.values()" and\n"dict.items()" are *view objects*. They provide a dynamic view on the\ndictionary\'s entries, which means that when the dictionary changes,\nthe view reflects these changes.\n\nDictionary views can be iterated over to yield their respective data,\nand support membership tests:\n\nlen(dictview)\n\n Return the number of entries in the dictionary.\n\niter(dictview)\n\n Return an iterator over the keys, values or items (represented as\n tuples of "(key, value)") in the dictionary.\n\n Keys and values are iterated over in an arbitrary order which is\n non-random, varies across Python implementations, and depends on\n the dictionary\'s history of insertions and deletions. If keys,\n values and items views are iterated over with no intervening\n modifications to the dictionary, the order of items will directly\n correspond. This allows the creation of "(value, key)" pairs using\n "zip()": "pairs = zip(d.values(), d.keys())". Another way to\n create the same list is "pairs = [(v, k) for (k, v) in d.items()]".\n\n Iterating views while adding or deleting entries in the dictionary\n may raise a "RuntimeError" or fail to iterate over all entries.\n\nx in dictview\n\n Return "True" if *x* is in the underlying dictionary\'s keys, values\n or items (in the latter case, *x* should be a "(key, value)"\n tuple).\n\nKeys views are set-like since their entries are unique and hashable.\nIf all values are hashable, so that "(key, value)" pairs are unique\nand hashable, then the items view is also set-like. (Values views are\nnot treated as set-like since the entries are generally not unique.)\nFor set-like views, all of the operations defined for the abstract\nbase class "collections.abc.Set" are available (for example, "==",\n"<", or "^").\n\nAn example of dictionary view usage:\n\n >>> dishes = {\'eggs\': 2, \'sausage\': 1, \'bacon\': 1, \'spam\': 500}\n >>> keys = dishes.keys()\n >>> values = dishes.values()\n\n >>> # iteration\n >>> n = 0\n >>> for val in values:\n ... n += val\n >>> print(n)\n 504\n\n >>> # keys and values are iterated over in the same order\n >>> list(keys)\n [\'eggs\', \'bacon\', \'sausage\', \'spam\']\n >>> list(values)\n [2, 1, 1, 500]\n\n >>> # view objects are dynamic and reflect dict changes\n >>> del dishes[\'eggs\']\n >>> del dishes[\'sausage\']\n >>> list(keys)\n [\'spam\', \'bacon\']\n\n >>> # set operations\n >>> keys & {\'eggs\', \'bacon\', \'salad\'}\n {\'bacon\'}\n >>> keys ^ {\'sausage\', \'juice\'}\n {\'juice\', \'sausage\', \'bacon\', \'spam\'}\n', - 'typesmethods': u'\nMethods\n*******\n\nMethods are functions that are called using the attribute notation.\nThere are two flavors: built-in methods (such as "append()" on lists)\nand class instance methods. Built-in methods are described with the\ntypes that support them.\n\nIf you access a method (a function defined in a class namespace)\nthrough an instance, you get a special object: a *bound method* (also\ncalled *instance method*) object. When called, it will add the "self"\nargument to the argument list. Bound methods have two special read-\nonly attributes: "m.__self__" is the object on which the method\noperates, and "m.__func__" is the function implementing the method.\nCalling "m(arg-1, arg-2, ..., arg-n)" is completely equivalent to\ncalling "m.__func__(m.__self__, arg-1, arg-2, ..., arg-n)".\n\nLike function objects, bound method objects support getting arbitrary\nattributes. However, since method attributes are actually stored on\nthe underlying function object ("meth.__func__"), setting method\nattributes on bound methods is disallowed. Attempting to set an\nattribute on a method results in an "AttributeError" being raised. In\norder to set a method attribute, you need to explicitly set it on the\nunderlying function object:\n\n >>> class C:\n ... def method(self):\n ... pass\n ...\n >>> c = C()\n >>> c.method.whoami = \'my name is method\' # can\'t set on the method\n Traceback (most recent call last):\n File "", line 1, in \n AttributeError: \'method\' object has no attribute \'whoami\'\n >>> c.method.__func__.whoami = \'my name is method\'\n >>> c.method.whoami\n \'my name is method\'\n\nSee *The standard type hierarchy* for more information.\n', - 'typesmodules': u'\nModules\n*******\n\nThe only special operation on a module is attribute access: "m.name",\nwhere *m* is a module and *name* accesses a name defined in *m*\'s\nsymbol table. Module attributes can be assigned to. (Note that the\n"import" statement is not, strictly speaking, an operation on a module\nobject; "import foo" does not require a module object named *foo* to\nexist, rather it requires an (external) *definition* for a module\nnamed *foo* somewhere.)\n\nA special attribute of every module is "__dict__". This is the\ndictionary containing the module\'s symbol table. Modifying this\ndictionary will actually change the module\'s symbol table, but direct\nassignment to the "__dict__" attribute is not possible (you can write\n"m.__dict__[\'a\'] = 1", which defines "m.a" to be "1", but you can\'t\nwrite "m.__dict__ = {}"). Modifying "__dict__" directly is not\nrecommended.\n\nModules built into the interpreter are written like this: "". If loaded from a file, they are written as\n"".\n', - 'typesseq': u'\nSequence Types --- "list", "tuple", "range"\n*******************************************\n\nThere are three basic sequence types: lists, tuples, and range\nobjects. Additional sequence types tailored for processing of *binary\ndata* and *text strings* are described in dedicated sections.\n\n\nCommon Sequence Operations\n==========================\n\nThe operations in the following table are supported by most sequence\ntypes, both mutable and immutable. The "collections.abc.Sequence" ABC\nis provided to make it easier to correctly implement these operations\non custom sequence types.\n\nThis table lists the sequence operations sorted in ascending priority.\nIn the table, *s* and *t* are sequences of the same type, *n*, *i*,\n*j* and *k* are integers and *x* is an arbitrary object that meets any\ntype and value restrictions imposed by *s*.\n\nThe "in" and "not in" operations have the same priorities as the\ncomparison operations. The "+" (concatenation) and "*" (repetition)\noperations have the same priority as the corresponding numeric\noperations.\n\n+----------------------------+----------------------------------+------------+\n| Operation | Result | Notes |\n+============================+==================================+============+\n| "x in s" | "True" if an item of *s* is | (1) |\n| | equal to *x*, else "False" | |\n+----------------------------+----------------------------------+------------+\n| "x not in s" | "False" if an item of *s* is | (1) |\n| | equal to *x*, else "True" | |\n+----------------------------+----------------------------------+------------+\n| "s + t" | the concatenation of *s* and *t* | (6)(7) |\n+----------------------------+----------------------------------+------------+\n| "s * n" or "n * s" | *n* shallow copies of *s* | (2)(7) |\n| | concatenated | |\n+----------------------------+----------------------------------+------------+\n| "s[i]" | *i*th item of *s*, origin 0 | (3) |\n+----------------------------+----------------------------------+------------+\n| "s[i:j]" | slice of *s* from *i* to *j* | (3)(4) |\n+----------------------------+----------------------------------+------------+\n| "s[i:j:k]" | slice of *s* from *i* to *j* | (3)(5) |\n| | with step *k* | |\n+----------------------------+----------------------------------+------------+\n| "len(s)" | length of *s* | |\n+----------------------------+----------------------------------+------------+\n| "min(s)" | smallest item of *s* | |\n+----------------------------+----------------------------------+------------+\n| "max(s)" | largest item of *s* | |\n+----------------------------+----------------------------------+------------+\n| "s.index(x[, i[, j]])" | index of the first occurrence of | (8) |\n| | *x* in *s* (at or after index | |\n| | *i* and before index *j*) | |\n+----------------------------+----------------------------------+------------+\n| "s.count(x)" | total number of occurrences of | |\n| | *x* in *s* | |\n+----------------------------+----------------------------------+------------+\n\nSequences of the same type also support comparisons. In particular,\ntuples and lists are compared lexicographically by comparing\ncorresponding elements. This means that to compare equal, every\nelement must compare equal and the two sequences must be of the same\ntype and have the same length. (For full details see *Comparisons* in\nthe language reference.)\n\nNotes:\n\n1. While the "in" and "not in" operations are used only for simple\n containment testing in the general case, some specialised sequences\n (such as "str", "bytes" and "bytearray") also use them for\n subsequence testing:\n\n >>> "gg" in "eggs"\n True\n\n2. Values of *n* less than "0" are treated as "0" (which yields an\n empty sequence of the same type as *s*). Note also that the copies\n are shallow; nested structures are not copied. This often haunts\n new Python programmers; consider:\n\n >>> lists = [[]] * 3\n >>> lists\n [[], [], []]\n >>> lists[0].append(3)\n >>> lists\n [[3], [3], [3]]\n\n What has happened is that "[[]]" is a one-element list containing\n an empty list, so all three elements of "[[]] * 3" are (pointers\n to) this single empty list. Modifying any of the elements of\n "lists" modifies this single list. You can create a list of\n different lists this way:\n\n >>> lists = [[] for i in range(3)]\n >>> lists[0].append(3)\n >>> lists[1].append(5)\n >>> lists[2].append(7)\n >>> lists\n [[3], [5], [7]]\n\n3. If *i* or *j* is negative, the index is relative to the end of\n the string: "len(s) + i" or "len(s) + j" is substituted. But note\n that "-0" is still "0".\n\n4. The slice of *s* from *i* to *j* is defined as the sequence of\n items with index *k* such that "i <= k < j". If *i* or *j* is\n greater than "len(s)", use "len(s)". If *i* is omitted or "None",\n use "0". If *j* is omitted or "None", use "len(s)". If *i* is\n greater than or equal to *j*, the slice is empty.\n\n5. The slice of *s* from *i* to *j* with step *k* is defined as the\n sequence of items with index "x = i + n*k" such that "0 <= n <\n (j-i)/k". In other words, the indices are "i", "i+k", "i+2*k",\n "i+3*k" and so on, stopping when *j* is reached (but never\n including *j*). If *i* or *j* is greater than "len(s)", use\n "len(s)". If *i* or *j* are omitted or "None", they become "end"\n values (which end depends on the sign of *k*). Note, *k* cannot be\n zero. If *k* is "None", it is treated like "1".\n\n6. Concatenating immutable sequences always results in a new\n object. This means that building up a sequence by repeated\n concatenation will have a quadratic runtime cost in the total\n sequence length. To get a linear runtime cost, you must switch to\n one of the alternatives below:\n\n * if concatenating "str" objects, you can build a list and use\n "str.join()" at the end or else write to a "io.StringIO" instance\n and retrieve its value when complete\n\n * if concatenating "bytes" objects, you can similarly use\n "bytes.join()" or "io.BytesIO", or you can do in-place\n concatenation with a "bytearray" object. "bytearray" objects are\n mutable and have an efficient overallocation mechanism\n\n * if concatenating "tuple" objects, extend a "list" instead\n\n * for other types, investigate the relevant class documentation\n\n7. Some sequence types (such as "range") only support item\n sequences that follow specific patterns, and hence don\'t support\n sequence concatenation or repetition.\n\n8. "index" raises "ValueError" when *x* is not found in *s*. When\n supported, the additional arguments to the index method allow\n efficient searching of subsections of the sequence. Passing the\n extra arguments is roughly equivalent to using "s[i:j].index(x)",\n only without copying any data and with the returned index being\n relative to the start of the sequence rather than the start of the\n slice.\n\n\nImmutable Sequence Types\n========================\n\nThe only operation that immutable sequence types generally implement\nthat is not also implemented by mutable sequence types is support for\nthe "hash()" built-in.\n\nThis support allows immutable sequences, such as "tuple" instances, to\nbe used as "dict" keys and stored in "set" and "frozenset" instances.\n\nAttempting to hash an immutable sequence that contains unhashable\nvalues will result in "TypeError".\n\n\nMutable Sequence Types\n======================\n\nThe operations in the following table are defined on mutable sequence\ntypes. The "collections.abc.MutableSequence" ABC is provided to make\nit easier to correctly implement these operations on custom sequence\ntypes.\n\nIn the table *s* is an instance of a mutable sequence type, *t* is any\niterable object and *x* is an arbitrary object that meets any type and\nvalue restrictions imposed by *s* (for example, "bytearray" only\naccepts integers that meet the value restriction "0 <= x <= 255").\n\n+--------------------------------+----------------------------------+-----------------------+\n| Operation | Result | Notes |\n+================================+==================================+=======================+\n| "s[i] = x" | item *i* of *s* is replaced by | |\n| | *x* | |\n+--------------------------------+----------------------------------+-----------------------+\n| "s[i:j] = t" | slice of *s* from *i* to *j* is | |\n| | replaced by the contents of the | |\n| | iterable *t* | |\n+--------------------------------+----------------------------------+-----------------------+\n| "del s[i:j]" | same as "s[i:j] = []" | |\n+--------------------------------+----------------------------------+-----------------------+\n| "s[i:j:k] = t" | the elements of "s[i:j:k]" are | (1) |\n| | replaced by those of *t* | |\n+--------------------------------+----------------------------------+-----------------------+\n| "del s[i:j:k]" | removes the elements of | |\n| | "s[i:j:k]" from the list | |\n+--------------------------------+----------------------------------+-----------------------+\n| "s.append(x)" | appends *x* to the end of the | |\n| | sequence (same as | |\n| | "s[len(s):len(s)] = [x]") | |\n+--------------------------------+----------------------------------+-----------------------+\n| "s.clear()" | removes all items from "s" (same | (5) |\n| | as "del s[:]") | |\n+--------------------------------+----------------------------------+-----------------------+\n| "s.copy()" | creates a shallow copy of "s" | (5) |\n| | (same as "s[:]") | |\n+--------------------------------+----------------------------------+-----------------------+\n| "s.extend(t)" | extends *s* with the contents of | |\n| | *t* (same as "s[len(s):len(s)] = | |\n| | t") | |\n+--------------------------------+----------------------------------+-----------------------+\n| "s.insert(i, x)" | inserts *x* into *s* at the | |\n| | index given by *i* (same as | |\n| | "s[i:i] = [x]") | |\n+--------------------------------+----------------------------------+-----------------------+\n| "s.pop([i])" | retrieves the item at *i* and | (2) |\n| | also removes it from *s* | |\n+--------------------------------+----------------------------------+-----------------------+\n| "s.remove(x)" | remove the first item from *s* | (3) |\n| | where "s[i] == x" | |\n+--------------------------------+----------------------------------+-----------------------+\n| "s.reverse()" | reverses the items of *s* in | (4) |\n| | place | |\n+--------------------------------+----------------------------------+-----------------------+\n\nNotes:\n\n1. *t* must have the same length as the slice it is replacing.\n\n2. The optional argument *i* defaults to "-1", so that by default\n the last item is removed and returned.\n\n3. "remove" raises "ValueError" when *x* is not found in *s*.\n\n4. The "reverse()" method modifies the sequence in place for\n economy of space when reversing a large sequence. To remind users\n that it operates by side effect, it does not return the reversed\n sequence.\n\n5. "clear()" and "copy()" are included for consistency with the\n interfaces of mutable containers that don\'t support slicing\n operations (such as "dict" and "set")\n\n New in version 3.3: "clear()" and "copy()" methods.\n\n\nLists\n=====\n\nLists are mutable sequences, typically used to store collections of\nhomogeneous items (where the precise degree of similarity will vary by\napplication).\n\nclass class list([iterable])\n\n Lists may be constructed in several ways:\n\n * Using a pair of square brackets to denote the empty list: "[]"\n\n * Using square brackets, separating items with commas: "[a]",\n "[a, b, c]"\n\n * Using a list comprehension: "[x for x in iterable]"\n\n * Using the type constructor: "list()" or "list(iterable)"\n\n The constructor builds a list whose items are the same and in the\n same order as *iterable*\'s items. *iterable* may be either a\n sequence, a container that supports iteration, or an iterator\n object. If *iterable* is already a list, a copy is made and\n returned, similar to "iterable[:]". For example, "list(\'abc\')"\n returns "[\'a\', \'b\', \'c\']" and "list( (1, 2, 3) )" returns "[1, 2,\n 3]". If no argument is given, the constructor creates a new empty\n list, "[]".\n\n Many other operations also produce lists, including the "sorted()"\n built-in.\n\n Lists implement all of the *common* and *mutable* sequence\n operations. Lists also provide the following additional method:\n\n sort(*, key=None, reverse=None)\n\n This method sorts the list in place, using only "<" comparisons\n between items. Exceptions are not suppressed - if any comparison\n operations fail, the entire sort operation will fail (and the\n list will likely be left in a partially modified state).\n\n "sort()" accepts two arguments that can only be passed by\n keyword (*keyword-only arguments*):\n\n *key* specifies a function of one argument that is used to\n extract a comparison key from each list element (for example,\n "key=str.lower"). The key corresponding to each item in the list\n is calculated once and then used for the entire sorting process.\n The default value of "None" means that list items are sorted\n directly without calculating a separate key value.\n\n The "functools.cmp_to_key()" utility is available to convert a\n 2.x style *cmp* function to a *key* function.\n\n *reverse* is a boolean value. If set to "True", then the list\n elements are sorted as if each comparison were reversed.\n\n This method modifies the sequence in place for economy of space\n when sorting a large sequence. To remind users that it operates\n by side effect, it does not return the sorted sequence (use\n "sorted()" to explicitly request a new sorted list instance).\n\n The "sort()" method is guaranteed to be stable. A sort is\n stable if it guarantees not to change the relative order of\n elements that compare equal --- this is helpful for sorting in\n multiple passes (for example, sort by department, then by salary\n grade).\n\n **CPython implementation detail:** While a list is being sorted,\n the effect of attempting to mutate, or even inspect, the list is\n undefined. The C implementation of Python makes the list appear\n empty for the duration, and raises "ValueError" if it can detect\n that the list has been mutated during a sort.\n\n\nTuples\n======\n\nTuples are immutable sequences, typically used to store collections of\nheterogeneous data (such as the 2-tuples produced by the "enumerate()"\nbuilt-in). Tuples are also used for cases where an immutable sequence\nof homogeneous data is needed (such as allowing storage in a "set" or\n"dict" instance).\n\nclass class tuple([iterable])\n\n Tuples may be constructed in a number of ways:\n\n * Using a pair of parentheses to denote the empty tuple: "()"\n\n * Using a trailing comma for a singleton tuple: "a," or "(a,)"\n\n * Separating items with commas: "a, b, c" or "(a, b, c)"\n\n * Using the "tuple()" built-in: "tuple()" or "tuple(iterable)"\n\n The constructor builds a tuple whose items are the same and in the\n same order as *iterable*\'s items. *iterable* may be either a\n sequence, a container that supports iteration, or an iterator\n object. If *iterable* is already a tuple, it is returned\n unchanged. For example, "tuple(\'abc\')" returns "(\'a\', \'b\', \'c\')"\n and "tuple( [1, 2, 3] )" returns "(1, 2, 3)". If no argument is\n given, the constructor creates a new empty tuple, "()".\n\n Note that it is actually the comma which makes a tuple, not the\n parentheses. The parentheses are optional, except in the empty\n tuple case, or when they are needed to avoid syntactic ambiguity.\n For example, "f(a, b, c)" is a function call with three arguments,\n while "f((a, b, c))" is a function call with a 3-tuple as the sole\n argument.\n\n Tuples implement all of the *common* sequence operations.\n\nFor heterogeneous collections of data where access by name is clearer\nthan access by index, "collections.namedtuple()" may be a more\nappropriate choice than a simple tuple object.\n\n\nRanges\n======\n\nThe "range" type represents an immutable sequence of numbers and is\ncommonly used for looping a specific number of times in "for" loops.\n\nclass class range(stop)\nclass class range(start, stop[, step])\n\n The arguments to the range constructor must be integers (either\n built-in "int" or any object that implements the "__index__"\n special method). If the *step* argument is omitted, it defaults to\n "1". If the *start* argument is omitted, it defaults to "0". If\n *step* is zero, "ValueError" is raised.\n\n For a positive *step*, the contents of a range "r" are determined\n by the formula "r[i] = start + step*i" where "i >= 0" and "r[i] <\n stop".\n\n For a negative *step*, the contents of the range are still\n determined by the formula "r[i] = start + step*i", but the\n constraints are "i >= 0" and "r[i] > stop".\n\n A range object will be empty if "r[0]" does not meet the value\n constraint. Ranges do support negative indices, but these are\n interpreted as indexing from the end of the sequence determined by\n the positive indices.\n\n Ranges containing absolute values larger than "sys.maxsize" are\n permitted but some features (such as "len()") may raise\n "OverflowError".\n\n Range examples:\n\n >>> list(range(10))\n [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]\n >>> list(range(1, 11))\n [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]\n >>> list(range(0, 30, 5))\n [0, 5, 10, 15, 20, 25]\n >>> list(range(0, 10, 3))\n [0, 3, 6, 9]\n >>> list(range(0, -10, -1))\n [0, -1, -2, -3, -4, -5, -6, -7, -8, -9]\n >>> list(range(0))\n []\n >>> list(range(1, 0))\n []\n\n Ranges implement all of the *common* sequence operations except\n concatenation and repetition (due to the fact that range objects\n can only represent sequences that follow a strict pattern and\n repetition and concatenation will usually violate that pattern).\n\nThe advantage of the "range" type over a regular "list" or "tuple" is\nthat a "range" object will always take the same (small) amount of\nmemory, no matter the size of the range it represents (as it only\nstores the "start", "stop" and "step" values, calculating individual\nitems and subranges as needed).\n\nRange objects implement the "collections.abc.Sequence" ABC, and\nprovide features such as containment tests, element index lookup,\nslicing and support for negative indices (see *Sequence Types ---\nlist, tuple, range*):\n\n>>> r = range(0, 20, 2)\n>>> r\nrange(0, 20, 2)\n>>> 11 in r\nFalse\n>>> 10 in r\nTrue\n>>> r.index(10)\n5\n>>> r[5]\n10\n>>> r[:5]\nrange(0, 10, 2)\n>>> r[-1]\n18\n\nTesting range objects for equality with "==" and "!=" compares them as\nsequences. That is, two range objects are considered equal if they\nrepresent the same sequence of values. (Note that two range objects\nthat compare equal might have different "start", "stop" and "step"\nattributes, for example "range(0) == range(2, 1, 3)" or "range(0, 3,\n2) == range(0, 4, 2)".)\n\nChanged in version 3.2: Implement the Sequence ABC. Support slicing\nand negative indices. Test "int" objects for membership in constant\ntime instead of iterating through all items.\n\nChanged in version 3.3: Define \'==\' and \'!=\' to compare range objects\nbased on the sequence of values they define (instead of comparing\nbased on object identity).\n\nNew in version 3.3: The "start", "stop" and "step" attributes.\n', - 'typesseq-mutable': u'\nMutable Sequence Types\n**********************\n\nThe operations in the following table are defined on mutable sequence\ntypes. The "collections.abc.MutableSequence" ABC is provided to make\nit easier to correctly implement these operations on custom sequence\ntypes.\n\nIn the table *s* is an instance of a mutable sequence type, *t* is any\niterable object and *x* is an arbitrary object that meets any type and\nvalue restrictions imposed by *s* (for example, "bytearray" only\naccepts integers that meet the value restriction "0 <= x <= 255").\n\n+--------------------------------+----------------------------------+-----------------------+\n| Operation | Result | Notes |\n+================================+==================================+=======================+\n| "s[i] = x" | item *i* of *s* is replaced by | |\n| | *x* | |\n+--------------------------------+----------------------------------+-----------------------+\n| "s[i:j] = t" | slice of *s* from *i* to *j* is | |\n| | replaced by the contents of the | |\n| | iterable *t* | |\n+--------------------------------+----------------------------------+-----------------------+\n| "del s[i:j]" | same as "s[i:j] = []" | |\n+--------------------------------+----------------------------------+-----------------------+\n| "s[i:j:k] = t" | the elements of "s[i:j:k]" are | (1) |\n| | replaced by those of *t* | |\n+--------------------------------+----------------------------------+-----------------------+\n| "del s[i:j:k]" | removes the elements of | |\n| | "s[i:j:k]" from the list | |\n+--------------------------------+----------------------------------+-----------------------+\n| "s.append(x)" | appends *x* to the end of the | |\n| | sequence (same as | |\n| | "s[len(s):len(s)] = [x]") | |\n+--------------------------------+----------------------------------+-----------------------+\n| "s.clear()" | removes all items from "s" (same | (5) |\n| | as "del s[:]") | |\n+--------------------------------+----------------------------------+-----------------------+\n| "s.copy()" | creates a shallow copy of "s" | (5) |\n| | (same as "s[:]") | |\n+--------------------------------+----------------------------------+-----------------------+\n| "s.extend(t)" | extends *s* with the contents of | |\n| | *t* (same as "s[len(s):len(s)] = | |\n| | t") | |\n+--------------------------------+----------------------------------+-----------------------+\n| "s.insert(i, x)" | inserts *x* into *s* at the | |\n| | index given by *i* (same as | |\n| | "s[i:i] = [x]") | |\n+--------------------------------+----------------------------------+-----------------------+\n| "s.pop([i])" | retrieves the item at *i* and | (2) |\n| | also removes it from *s* | |\n+--------------------------------+----------------------------------+-----------------------+\n| "s.remove(x)" | remove the first item from *s* | (3) |\n| | where "s[i] == x" | |\n+--------------------------------+----------------------------------+-----------------------+\n| "s.reverse()" | reverses the items of *s* in | (4) |\n| | place | |\n+--------------------------------+----------------------------------+-----------------------+\n\nNotes:\n\n1. *t* must have the same length as the slice it is replacing.\n\n2. The optional argument *i* defaults to "-1", so that by default\n the last item is removed and returned.\n\n3. "remove" raises "ValueError" when *x* is not found in *s*.\n\n4. The "reverse()" method modifies the sequence in place for\n economy of space when reversing a large sequence. To remind users\n that it operates by side effect, it does not return the reversed\n sequence.\n\n5. "clear()" and "copy()" are included for consistency with the\n interfaces of mutable containers that don\'t support slicing\n operations (such as "dict" and "set")\n\n New in version 3.3: "clear()" and "copy()" methods.\n', - 'unary': u'\nUnary arithmetic and bitwise operations\n***************************************\n\nAll unary arithmetic and bitwise operations have the same priority:\n\n u_expr ::= power | "-" u_expr | "+" u_expr | "~" u_expr\n\nThe unary "-" (minus) operator yields the negation of its numeric\nargument.\n\nThe unary "+" (plus) operator yields its numeric argument unchanged.\n\nThe unary "~" (invert) operator yields the bitwise inversion of its\ninteger argument. The bitwise inversion of "x" is defined as\n"-(x+1)". It only applies to integral numbers.\n\nIn all three cases, if the argument does not have the proper type, a\n"TypeError" exception is raised.\n', - 'while': u'\nThe "while" statement\n*********************\n\nThe "while" statement is used for repeated execution as long as an\nexpression is true:\n\n while_stmt ::= "while" expression ":" suite\n ["else" ":" suite]\n\nThis repeatedly tests the expression and, if it is true, executes the\nfirst suite; if the expression is false (which may be the first time\nit is tested) the suite of the "else" clause, if present, is executed\nand the loop terminates.\n\nA "break" statement executed in the first suite terminates the loop\nwithout executing the "else" clause\'s suite. A "continue" statement\nexecuted in the first suite skips the rest of the suite and goes back\nto testing the expression.\n', - 'with': u'\nThe "with" statement\n********************\n\nThe "with" statement is used to wrap the execution of a block with\nmethods defined by a context manager (see section *With Statement\nContext Managers*). This allows common "try"..."except"..."finally"\nusage patterns to be encapsulated for convenient reuse.\n\n with_stmt ::= "with" with_item ("," with_item)* ":" suite\n with_item ::= expression ["as" target]\n\nThe execution of the "with" statement with one "item" proceeds as\nfollows:\n\n1. The context expression (the expression given in the "with_item")\n is evaluated to obtain a context manager.\n\n2. The context manager\'s "__exit__()" is loaded for later use.\n\n3. The context manager\'s "__enter__()" method is invoked.\n\n4. If a target was included in the "with" statement, the return\n value from "__enter__()" is assigned to it.\n\n Note: The "with" statement guarantees that if the "__enter__()"\n method returns without an error, then "__exit__()" will always be\n called. Thus, if an error occurs during the assignment to the\n target list, it will be treated the same as an error occurring\n within the suite would be. See step 6 below.\n\n5. The suite is executed.\n\n6. The context manager\'s "__exit__()" method is invoked. If an\n exception caused the suite to be exited, its type, value, and\n traceback are passed as arguments to "__exit__()". Otherwise, three\n "None" arguments are supplied.\n\n If the suite was exited due to an exception, and the return value\n from the "__exit__()" method was false, the exception is reraised.\n If the return value was true, the exception is suppressed, and\n execution continues with the statement following the "with"\n statement.\n\n If the suite was exited for any reason other than an exception, the\n return value from "__exit__()" is ignored, and execution proceeds\n at the normal location for the kind of exit that was taken.\n\nWith more than one item, the context managers are processed as if\nmultiple "with" statements were nested:\n\n with A() as a, B() as b:\n suite\n\nis equivalent to\n\n with A() as a:\n with B() as b:\n suite\n\nChanged in version 3.1: Support for multiple context expressions.\n\nSee also: **PEP 0343** - The "with" statement\n\n The specification, background, and examples for the Python "with"\n statement.\n', - 'yield': u'\nThe "yield" statement\n*********************\n\n yield_stmt ::= yield_expression\n\nA "yield" statement is semantically equivalent to a *yield\nexpression*. The yield statement can be used to omit the parentheses\nthat would otherwise be required in the equivalent yield expression\nstatement. For example, the yield statements\n\n yield \n yield from \n\nare equivalent to the yield expression statements\n\n (yield )\n (yield from )\n\nYield expressions and statements are only used when defining a\n*generator* function, and are only used in the body of the generator\nfunction. Using yield in a function definition is sufficient to cause\nthat definition to create a generator function instead of a normal\nfunction.\n\nFor full details of "yield" semantics, refer to the *Yield\nexpressions* section.\n'} +# Autogenerated by Sphinx on Sat Sep 12 17:22:24 2015 +topics = {'assert': '\n' + 'The "assert" statement\n' + '**********************\n' + '\n' + 'Assert statements are a convenient way to insert debugging ' + 'assertions\n' + 'into a program:\n' + '\n' + ' assert_stmt ::= "assert" expression ["," expression]\n' + '\n' + 'The simple form, "assert expression", is equivalent to\n' + '\n' + ' if __debug__:\n' + ' if not expression: raise AssertionError\n' + '\n' + 'The extended form, "assert expression1, expression2", is ' + 'equivalent to\n' + '\n' + ' if __debug__:\n' + ' if not expression1: raise AssertionError(expression2)\n' + '\n' + 'These equivalences assume that "__debug__" and "AssertionError" ' + 'refer\n' + 'to the built-in variables with those names. In the current\n' + 'implementation, the built-in variable "__debug__" is "True" ' + 'under\n' + 'normal circumstances, "False" when optimization is requested ' + '(command\n' + 'line option -O). The current code generator emits no code for ' + 'an\n' + 'assert statement when optimization is requested at compile ' + 'time. Note\n' + 'that it is unnecessary to include the source code for the ' + 'expression\n' + 'that failed in the error message; it will be displayed as part ' + 'of the\n' + 'stack trace.\n' + '\n' + 'Assignments to "__debug__" are illegal. The value for the ' + 'built-in\n' + 'variable is determined when the interpreter starts.\n', + 'assignment': '\n' + 'Assignment statements\n' + '*********************\n' + '\n' + 'Assignment statements are used to (re)bind names to values ' + 'and to\n' + 'modify attributes or items of mutable objects:\n' + '\n' + ' assignment_stmt ::= (target_list "=")+ (expression_list | ' + 'yield_expression)\n' + ' target_list ::= target ("," target)* [","]\n' + ' target ::= identifier\n' + ' | "(" target_list ")"\n' + ' | "[" target_list "]"\n' + ' | attributeref\n' + ' | subscription\n' + ' | slicing\n' + ' | "*" target\n' + '\n' + '(See section *Primaries* for the syntax definitions for\n' + '*attributeref*, *subscription*, and *slicing*.)\n' + '\n' + 'An assignment statement evaluates the expression list ' + '(remember that\n' + 'this can be a single expression or a comma-separated list, ' + 'the latter\n' + 'yielding a tuple) and assigns the single resulting object to ' + 'each of\n' + 'the target lists, from left to right.\n' + '\n' + 'Assignment is defined recursively depending on the form of ' + 'the target\n' + '(list). When a target is part of a mutable object (an ' + 'attribute\n' + 'reference, subscription or slicing), the mutable object ' + 'must\n' + 'ultimately perform the assignment and decide about its ' + 'validity, and\n' + 'may raise an exception if the assignment is unacceptable. ' + 'The rules\n' + 'observed by various types and the exceptions raised are ' + 'given with the\n' + 'definition of the object types (see section *The standard ' + 'type\n' + 'hierarchy*).\n' + '\n' + 'Assignment of an object to a target list, optionally ' + 'enclosed in\n' + 'parentheses or square brackets, is recursively defined as ' + 'follows.\n' + '\n' + '* If the target list is a single target: The object is ' + 'assigned to\n' + ' that target.\n' + '\n' + '* If the target list is a comma-separated list of targets: ' + 'The\n' + ' object must be an iterable with the same number of items ' + 'as there\n' + ' are targets in the target list, and the items are ' + 'assigned, from\n' + ' left to right, to the corresponding targets.\n' + '\n' + ' * If the target list contains one target prefixed with an\n' + ' asterisk, called a "starred" target: The object must be ' + 'a sequence\n' + ' with at least as many items as there are targets in the ' + 'target\n' + ' list, minus one. The first items of the sequence are ' + 'assigned,\n' + ' from left to right, to the targets before the starred ' + 'target. The\n' + ' final items of the sequence are assigned to the targets ' + 'after the\n' + ' starred target. A list of the remaining items in the ' + 'sequence is\n' + ' then assigned to the starred target (the list can be ' + 'empty).\n' + '\n' + ' * Else: The object must be a sequence with the same number ' + 'of\n' + ' items as there are targets in the target list, and the ' + 'items are\n' + ' assigned, from left to right, to the corresponding ' + 'targets.\n' + '\n' + 'Assignment of an object to a single target is recursively ' + 'defined as\n' + 'follows.\n' + '\n' + '* If the target is an identifier (name):\n' + '\n' + ' * If the name does not occur in a "global" or "nonlocal" ' + 'statement\n' + ' in the current code block: the name is bound to the ' + 'object in the\n' + ' current local namespace.\n' + '\n' + ' * Otherwise: the name is bound to the object in the ' + 'global\n' + ' namespace or the outer namespace determined by ' + '"nonlocal",\n' + ' respectively.\n' + '\n' + ' The name is rebound if it was already bound. This may ' + 'cause the\n' + ' reference count for the object previously bound to the ' + 'name to reach\n' + ' zero, causing the object to be deallocated and its ' + 'destructor (if it\n' + ' has one) to be called.\n' + '\n' + '* If the target is a target list enclosed in parentheses or ' + 'in\n' + ' square brackets: The object must be an iterable with the ' + 'same number\n' + ' of items as there are targets in the target list, and its ' + 'items are\n' + ' assigned, from left to right, to the corresponding ' + 'targets.\n' + '\n' + '* If the target is an attribute reference: The primary ' + 'expression in\n' + ' the reference is evaluated. It should yield an object ' + 'with\n' + ' assignable attributes; if this is not the case, ' + '"TypeError" is\n' + ' raised. That object is then asked to assign the assigned ' + 'object to\n' + ' the given attribute; if it cannot perform the assignment, ' + 'it raises\n' + ' an exception (usually but not necessarily ' + '"AttributeError").\n' + '\n' + ' Note: If the object is a class instance and the attribute ' + 'reference\n' + ' occurs on both sides of the assignment operator, the RHS ' + 'expression,\n' + ' "a.x" can access either an instance attribute or (if no ' + 'instance\n' + ' attribute exists) a class attribute. The LHS target "a.x" ' + 'is always\n' + ' set as an instance attribute, creating it if necessary. ' + 'Thus, the\n' + ' two occurrences of "a.x" do not necessarily refer to the ' + 'same\n' + ' attribute: if the RHS expression refers to a class ' + 'attribute, the\n' + ' LHS creates a new instance attribute as the target of the\n' + ' assignment:\n' + '\n' + ' class Cls:\n' + ' x = 3 # class variable\n' + ' inst = Cls()\n' + ' inst.x = inst.x + 1 # writes inst.x as 4 leaving ' + 'Cls.x as 3\n' + '\n' + ' This description does not necessarily apply to descriptor\n' + ' attributes, such as properties created with "property()".\n' + '\n' + '* If the target is a subscription: The primary expression in ' + 'the\n' + ' reference is evaluated. It should yield either a mutable ' + 'sequence\n' + ' object (such as a list) or a mapping object (such as a ' + 'dictionary).\n' + ' Next, the subscript expression is evaluated.\n' + '\n' + ' If the primary is a mutable sequence object (such as a ' + 'list), the\n' + ' subscript must yield an integer. If it is negative, the ' + "sequence's\n" + ' length is added to it. The resulting value must be a ' + 'nonnegative\n' + " integer less than the sequence's length, and the sequence " + 'is asked\n' + ' to assign the assigned object to its item with that ' + 'index. If the\n' + ' index is out of range, "IndexError" is raised (assignment ' + 'to a\n' + ' subscripted sequence cannot add new items to a list).\n' + '\n' + ' If the primary is a mapping object (such as a dictionary), ' + 'the\n' + " subscript must have a type compatible with the mapping's " + 'key type,\n' + ' and the mapping is then asked to create a key/datum pair ' + 'which maps\n' + ' the subscript to the assigned object. This can either ' + 'replace an\n' + ' existing key/value pair with the same key value, or insert ' + 'a new\n' + ' key/value pair (if no key with the same value existed).\n' + '\n' + ' For user-defined objects, the "__setitem__()" method is ' + 'called with\n' + ' appropriate arguments.\n' + '\n' + '* If the target is a slicing: The primary expression in the\n' + ' reference is evaluated. It should yield a mutable ' + 'sequence object\n' + ' (such as a list). The assigned object should be a ' + 'sequence object\n' + ' of the same type. Next, the lower and upper bound ' + 'expressions are\n' + ' evaluated, insofar they are present; defaults are zero and ' + 'the\n' + " sequence's length. The bounds should evaluate to " + 'integers. If\n' + " either bound is negative, the sequence's length is added " + 'to it. The\n' + ' resulting bounds are clipped to lie between zero and the ' + "sequence's\n" + ' length, inclusive. Finally, the sequence object is asked ' + 'to replace\n' + ' the slice with the items of the assigned sequence. The ' + 'length of\n' + ' the slice may be different from the length of the assigned ' + 'sequence,\n' + ' thus changing the length of the target sequence, if the ' + 'target\n' + ' sequence allows it.\n' + '\n' + '**CPython implementation detail:** In the current ' + 'implementation, the\n' + 'syntax for targets is taken to be the same as for ' + 'expressions, and\n' + 'invalid syntax is rejected during the code generation phase, ' + 'causing\n' + 'less detailed error messages.\n' + '\n' + 'Although the definition of assignment implies that overlaps ' + 'between\n' + 'the left-hand side and the right-hand side are ' + "'simultanenous' (for\n" + 'example "a, b = b, a" swaps two variables), overlaps ' + '*within* the\n' + 'collection of assigned-to variables occur left-to-right, ' + 'sometimes\n' + 'resulting in confusion. For instance, the following program ' + 'prints\n' + '"[0, 2]":\n' + '\n' + ' x = [0, 1]\n' + ' i = 0\n' + ' i, x[i] = 1, 2 # i is updated, then x[i] is ' + 'updated\n' + ' print(x)\n' + '\n' + 'See also: **PEP 3132** - Extended Iterable Unpacking\n' + '\n' + ' The specification for the "*target" feature.\n' + '\n' + '\n' + 'Augmented assignment statements\n' + '===============================\n' + '\n' + 'Augmented assignment is the combination, in a single ' + 'statement, of a\n' + 'binary operation and an assignment statement:\n' + '\n' + ' augmented_assignment_stmt ::= augtarget augop ' + '(expression_list | yield_expression)\n' + ' augtarget ::= identifier | attributeref | ' + 'subscription | slicing\n' + ' augop ::= "+=" | "-=" | "*=" | "@=" | ' + '"/=" | "//=" | "%=" | "**="\n' + ' | ">>=" | "<<=" | "&=" | "^=" | "|="\n' + '\n' + '(See section *Primaries* for the syntax definitions of the ' + 'last three\n' + 'symbols.)\n' + '\n' + 'An augmented assignment evaluates the target (which, unlike ' + 'normal\n' + 'assignment statements, cannot be an unpacking) and the ' + 'expression\n' + 'list, performs the binary operation specific to the type of ' + 'assignment\n' + 'on the two operands, and assigns the result to the original ' + 'target.\n' + 'The target is only evaluated once.\n' + '\n' + 'An augmented assignment expression like "x += 1" can be ' + 'rewritten as\n' + '"x = x + 1" to achieve a similar, but not exactly equal ' + 'effect. In the\n' + 'augmented version, "x" is only evaluated once. Also, when ' + 'possible,\n' + 'the actual operation is performed *in-place*, meaning that ' + 'rather than\n' + 'creating a new object and assigning that to the target, the ' + 'old object\n' + 'is modified instead.\n' + '\n' + 'Unlike normal assignments, augmented assignments evaluate ' + 'the left-\n' + 'hand side *before* evaluating the right-hand side. For ' + 'example, "a[i]\n' + '+= f(x)" first looks-up "a[i]", then it evaluates "f(x)" and ' + 'performs\n' + 'the addition, and lastly, it writes the result back to ' + '"a[i]".\n' + '\n' + 'With the exception of assigning to tuples and multiple ' + 'targets in a\n' + 'single statement, the assignment done by augmented ' + 'assignment\n' + 'statements is handled the same way as normal assignments. ' + 'Similarly,\n' + 'with the exception of the possible *in-place* behavior, the ' + 'binary\n' + 'operation performed by augmented assignment is the same as ' + 'the normal\n' + 'binary operations.\n' + '\n' + 'For targets which are attribute references, the same *caveat ' + 'about\n' + 'class and instance attributes* applies as for regular ' + 'assignments.\n', + 'atom-identifiers': '\n' + 'Identifiers (Names)\n' + '*******************\n' + '\n' + 'An identifier occurring as an atom is a name. See ' + 'section\n' + '*Identifiers and keywords* for lexical definition and ' + 'section *Naming\n' + 'and binding* for documentation of naming and binding.\n' + '\n' + 'When the name is bound to an object, evaluation of the ' + 'atom yields\n' + 'that object. When a name is not bound, an attempt to ' + 'evaluate it\n' + 'raises a "NameError" exception.\n' + '\n' + '**Private name mangling:** When an identifier that ' + 'textually occurs in\n' + 'a class definition begins with two or more underscore ' + 'characters and\n' + 'does not end in two or more underscores, it is ' + 'considered a *private\n' + 'name* of that class. Private names are transformed to ' + 'a longer form\n' + 'before code is generated for them. The transformation ' + 'inserts the\n' + 'class name, with leading underscores removed and a ' + 'single underscore\n' + 'inserted, in front of the name. For example, the ' + 'identifier "__spam"\n' + 'occurring in a class named "Ham" will be transformed ' + 'to "_Ham__spam".\n' + 'This transformation is independent of the syntactical ' + 'context in which\n' + 'the identifier is used. If the transformed name is ' + 'extremely long\n' + '(longer than 255 characters), implementation defined ' + 'truncation may\n' + 'happen. If the class name consists only of ' + 'underscores, no\n' + 'transformation is done.\n', + 'atom-literals': '\n' + 'Literals\n' + '********\n' + '\n' + 'Python supports string and bytes literals and various ' + 'numeric\n' + 'literals:\n' + '\n' + ' literal ::= stringliteral | bytesliteral\n' + ' | integer | floatnumber | imagnumber\n' + '\n' + 'Evaluation of a literal yields an object of the given ' + 'type (string,\n' + 'bytes, integer, floating point number, complex number) ' + 'with the given\n' + 'value. The value may be approximated in the case of ' + 'floating point\n' + 'and imaginary (complex) literals. See section *Literals* ' + 'for details.\n' + '\n' + 'All literals correspond to immutable data types, and ' + 'hence the\n' + "object's identity is less important than its value. " + 'Multiple\n' + 'evaluations of literals with the same value (either the ' + 'same\n' + 'occurrence in the program text or a different occurrence) ' + 'may obtain\n' + 'the same object or a different object with the same ' + 'value.\n', + 'attribute-access': '\n' + 'Customizing attribute access\n' + '****************************\n' + '\n' + 'The following methods can be defined to customize the ' + 'meaning of\n' + 'attribute access (use of, assignment to, or deletion ' + 'of "x.name") for\n' + 'class instances.\n' + '\n' + 'object.__getattr__(self, name)\n' + '\n' + ' Called when an attribute lookup has not found the ' + 'attribute in the\n' + ' usual places (i.e. it is not an instance attribute ' + 'nor is it found\n' + ' in the class tree for "self"). "name" is the ' + 'attribute name. This\n' + ' method should return the (computed) attribute value ' + 'or raise an\n' + ' "AttributeError" exception.\n' + '\n' + ' Note that if the attribute is found through the ' + 'normal mechanism,\n' + ' "__getattr__()" is not called. (This is an ' + 'intentional asymmetry\n' + ' between "__getattr__()" and "__setattr__()".) This ' + 'is done both for\n' + ' efficiency reasons and because otherwise ' + '"__getattr__()" would have\n' + ' no way to access other attributes of the instance. ' + 'Note that at\n' + ' least for instance variables, you can fake total ' + 'control by not\n' + ' inserting any values in the instance attribute ' + 'dictionary (but\n' + ' instead inserting them in another object). See ' + 'the\n' + ' "__getattribute__()" method below for a way to ' + 'actually get total\n' + ' control over attribute access.\n' + '\n' + 'object.__getattribute__(self, name)\n' + '\n' + ' Called unconditionally to implement attribute ' + 'accesses for\n' + ' instances of the class. If the class also defines ' + '"__getattr__()",\n' + ' the latter will not be called unless ' + '"__getattribute__()" either\n' + ' calls it explicitly or raises an "AttributeError". ' + 'This method\n' + ' should return the (computed) attribute value or ' + 'raise an\n' + ' "AttributeError" exception. In order to avoid ' + 'infinite recursion in\n' + ' this method, its implementation should always call ' + 'the base class\n' + ' method with the same name to access any attributes ' + 'it needs, for\n' + ' example, "object.__getattribute__(self, name)".\n' + '\n' + ' Note: This method may still be bypassed when ' + 'looking up special\n' + ' methods as the result of implicit invocation via ' + 'language syntax\n' + ' or built-in functions. See *Special method ' + 'lookup*.\n' + '\n' + 'object.__setattr__(self, name, value)\n' + '\n' + ' Called when an attribute assignment is attempted. ' + 'This is called\n' + ' instead of the normal mechanism (i.e. store the ' + 'value in the\n' + ' instance dictionary). *name* is the attribute name, ' + '*value* is the\n' + ' value to be assigned to it.\n' + '\n' + ' If "__setattr__()" wants to assign to an instance ' + 'attribute, it\n' + ' should call the base class method with the same ' + 'name, for example,\n' + ' "object.__setattr__(self, name, value)".\n' + '\n' + 'object.__delattr__(self, name)\n' + '\n' + ' Like "__setattr__()" but for attribute deletion ' + 'instead of\n' + ' assignment. This should only be implemented if ' + '"del obj.name" is\n' + ' meaningful for the object.\n' + '\n' + 'object.__dir__(self)\n' + '\n' + ' Called when "dir()" is called on the object. A ' + 'sequence must be\n' + ' returned. "dir()" converts the returned sequence to ' + 'a list and\n' + ' sorts it.\n' + '\n' + '\n' + 'Implementing Descriptors\n' + '========================\n' + '\n' + 'The following methods only apply when an instance of ' + 'the class\n' + 'containing the method (a so-called *descriptor* class) ' + 'appears in an\n' + '*owner* class (the descriptor must be in either the ' + "owner's class\n" + 'dictionary or in the class dictionary for one of its ' + 'parents). In the\n' + 'examples below, "the attribute" refers to the ' + 'attribute whose name is\n' + "the key of the property in the owner class' " + '"__dict__".\n' + '\n' + 'object.__get__(self, instance, owner)\n' + '\n' + ' Called to get the attribute of the owner class ' + '(class attribute\n' + ' access) or of an instance of that class (instance ' + 'attribute\n' + ' access). *owner* is always the owner class, while ' + '*instance* is the\n' + ' instance that the attribute was accessed through, ' + 'or "None" when\n' + ' the attribute is accessed through the *owner*. ' + 'This method should\n' + ' return the (computed) attribute value or raise an ' + '"AttributeError"\n' + ' exception.\n' + '\n' + 'object.__set__(self, instance, value)\n' + '\n' + ' Called to set the attribute on an instance ' + '*instance* of the owner\n' + ' class to a new value, *value*.\n' + '\n' + 'object.__delete__(self, instance)\n' + '\n' + ' Called to delete the attribute on an instance ' + '*instance* of the\n' + ' owner class.\n' + '\n' + 'The attribute "__objclass__" is interpreted by the ' + '"inspect" module as\n' + 'specifying the class where this object was defined ' + '(setting this\n' + 'appropriately can assist in runtime introspection of ' + 'dynamic class\n' + 'attributes). For callables, it may indicate that an ' + 'instance of the\n' + 'given type (or a subclass) is expected or required as ' + 'the first\n' + 'positional argument (for example, CPython sets this ' + 'attribute for\n' + 'unbound methods that are implemented in C).\n' + '\n' + '\n' + 'Invoking Descriptors\n' + '====================\n' + '\n' + 'In general, a descriptor is an object attribute with ' + '"binding\n' + 'behavior", one whose attribute access has been ' + 'overridden by methods\n' + 'in the descriptor protocol: "__get__()", "__set__()", ' + 'and\n' + '"__delete__()". If any of those methods are defined ' + 'for an object, it\n' + 'is said to be a descriptor.\n' + '\n' + 'The default behavior for attribute access is to get, ' + 'set, or delete\n' + "the attribute from an object's dictionary. For " + 'instance, "a.x" has a\n' + 'lookup chain starting with "a.__dict__[\'x\']", then\n' + '"type(a).__dict__[\'x\']", and continuing through the ' + 'base classes of\n' + '"type(a)" excluding metaclasses.\n' + '\n' + 'However, if the looked-up value is an object defining ' + 'one of the\n' + 'descriptor methods, then Python may override the ' + 'default behavior and\n' + 'invoke the descriptor method instead. Where this ' + 'occurs in the\n' + 'precedence chain depends on which descriptor methods ' + 'were defined and\n' + 'how they were called.\n' + '\n' + 'The starting point for descriptor invocation is a ' + 'binding, "a.x". How\n' + 'the arguments are assembled depends on "a":\n' + '\n' + 'Direct Call\n' + ' The simplest and least common call is when user ' + 'code directly\n' + ' invokes a descriptor method: "x.__get__(a)".\n' + '\n' + 'Instance Binding\n' + ' If binding to an object instance, "a.x" is ' + 'transformed into the\n' + ' call: "type(a).__dict__[\'x\'].__get__(a, ' + 'type(a))".\n' + '\n' + 'Class Binding\n' + ' If binding to a class, "A.x" is transformed into ' + 'the call:\n' + ' "A.__dict__[\'x\'].__get__(None, A)".\n' + '\n' + 'Super Binding\n' + ' If "a" is an instance of "super", then the binding ' + '"super(B,\n' + ' obj).m()" searches "obj.__class__.__mro__" for the ' + 'base class "A"\n' + ' immediately preceding "B" and then invokes the ' + 'descriptor with the\n' + ' call: "A.__dict__[\'m\'].__get__(obj, ' + 'obj.__class__)".\n' + '\n' + 'For instance bindings, the precedence of descriptor ' + 'invocation depends\n' + 'on the which descriptor methods are defined. A ' + 'descriptor can define\n' + 'any combination of "__get__()", "__set__()" and ' + '"__delete__()". If it\n' + 'does not define "__get__()", then accessing the ' + 'attribute will return\n' + 'the descriptor object itself unless there is a value ' + "in the object's\n" + 'instance dictionary. If the descriptor defines ' + '"__set__()" and/or\n' + '"__delete__()", it is a data descriptor; if it defines ' + 'neither, it is\n' + 'a non-data descriptor. Normally, data descriptors ' + 'define both\n' + '"__get__()" and "__set__()", while non-data ' + 'descriptors have just the\n' + '"__get__()" method. Data descriptors with "__set__()" ' + 'and "__get__()"\n' + 'defined always override a redefinition in an instance ' + 'dictionary. In\n' + 'contrast, non-data descriptors can be overridden by ' + 'instances.\n' + '\n' + 'Python methods (including "staticmethod()" and ' + '"classmethod()") are\n' + 'implemented as non-data descriptors. Accordingly, ' + 'instances can\n' + 'redefine and override methods. This allows individual ' + 'instances to\n' + 'acquire behaviors that differ from other instances of ' + 'the same class.\n' + '\n' + 'The "property()" function is implemented as a data ' + 'descriptor.\n' + 'Accordingly, instances cannot override the behavior of ' + 'a property.\n' + '\n' + '\n' + '__slots__\n' + '=========\n' + '\n' + 'By default, instances of classes have a dictionary for ' + 'attribute\n' + 'storage. This wastes space for objects having very ' + 'few instance\n' + 'variables. The space consumption can become acute ' + 'when creating large\n' + 'numbers of instances.\n' + '\n' + 'The default can be overridden by defining *__slots__* ' + 'in a class\n' + 'definition. The *__slots__* declaration takes a ' + 'sequence of instance\n' + 'variables and reserves just enough space in each ' + 'instance to hold a\n' + 'value for each variable. Space is saved because ' + '*__dict__* is not\n' + 'created for each instance.\n' + '\n' + 'object.__slots__\n' + '\n' + ' This class variable can be assigned a string, ' + 'iterable, or sequence\n' + ' of strings with variable names used by instances. ' + '*__slots__*\n' + ' reserves space for the declared variables and ' + 'prevents the\n' + ' automatic creation of *__dict__* and *__weakref__* ' + 'for each\n' + ' instance.\n' + '\n' + '\n' + 'Notes on using *__slots__*\n' + '--------------------------\n' + '\n' + '* When inheriting from a class without *__slots__*, ' + 'the *__dict__*\n' + ' attribute of that class will always be accessible, ' + 'so a *__slots__*\n' + ' definition in the subclass is meaningless.\n' + '\n' + '* Without a *__dict__* variable, instances cannot be ' + 'assigned new\n' + ' variables not listed in the *__slots__* definition. ' + 'Attempts to\n' + ' assign to an unlisted variable name raises ' + '"AttributeError". If\n' + ' dynamic assignment of new variables is desired, then ' + 'add\n' + ' "\'__dict__\'" to the sequence of strings in the ' + '*__slots__*\n' + ' declaration.\n' + '\n' + '* Without a *__weakref__* variable for each instance, ' + 'classes\n' + ' defining *__slots__* do not support weak references ' + 'to its\n' + ' instances. If weak reference support is needed, then ' + 'add\n' + ' "\'__weakref__\'" to the sequence of strings in the ' + '*__slots__*\n' + ' declaration.\n' + '\n' + '* *__slots__* are implemented at the class level by ' + 'creating\n' + ' descriptors (*Implementing Descriptors*) for each ' + 'variable name. As\n' + ' a result, class attributes cannot be used to set ' + 'default values for\n' + ' instance variables defined by *__slots__*; ' + 'otherwise, the class\n' + ' attribute would overwrite the descriptor ' + 'assignment.\n' + '\n' + '* The action of a *__slots__* declaration is limited ' + 'to the class\n' + ' where it is defined. As a result, subclasses will ' + 'have a *__dict__*\n' + ' unless they also define *__slots__* (which must only ' + 'contain names\n' + ' of any *additional* slots).\n' + '\n' + '* If a class defines a slot also defined in a base ' + 'class, the\n' + ' instance variable defined by the base class slot is ' + 'inaccessible\n' + ' (except by retrieving its descriptor directly from ' + 'the base class).\n' + ' This renders the meaning of the program undefined. ' + 'In the future, a\n' + ' check may be added to prevent this.\n' + '\n' + '* Nonempty *__slots__* does not work for classes ' + 'derived from\n' + ' "variable-length" built-in types such as "int", ' + '"bytes" and "tuple".\n' + '\n' + '* Any non-string iterable may be assigned to ' + '*__slots__*. Mappings\n' + ' may also be used; however, in the future, special ' + 'meaning may be\n' + ' assigned to the values corresponding to each key.\n' + '\n' + '* *__class__* assignment works only if both classes ' + 'have the same\n' + ' *__slots__*.\n', + 'attribute-references': '\n' + 'Attribute references\n' + '********************\n' + '\n' + 'An attribute reference is a primary followed by a ' + 'period and a name:\n' + '\n' + ' attributeref ::= primary "." identifier\n' + '\n' + 'The primary must evaluate to an object of a type ' + 'that supports\n' + 'attribute references, which most objects do. This ' + 'object is then\n' + 'asked to produce the attribute whose name is the ' + 'identifier. This\n' + 'production can be customized by overriding the ' + '"__getattr__()" method.\n' + 'If this attribute is not available, the exception ' + '"AttributeError" is\n' + 'raised. Otherwise, the type and value of the ' + 'object produced is\n' + 'determined by the object. Multiple evaluations of ' + 'the same attribute\n' + 'reference may yield different objects.\n', + 'augassign': '\n' + 'Augmented assignment statements\n' + '*******************************\n' + '\n' + 'Augmented assignment is the combination, in a single ' + 'statement, of a\n' + 'binary operation and an assignment statement:\n' + '\n' + ' augmented_assignment_stmt ::= augtarget augop ' + '(expression_list | yield_expression)\n' + ' augtarget ::= identifier | attributeref | ' + 'subscription | slicing\n' + ' augop ::= "+=" | "-=" | "*=" | "@=" | ' + '"/=" | "//=" | "%=" | "**="\n' + ' | ">>=" | "<<=" | "&=" | "^=" | "|="\n' + '\n' + '(See section *Primaries* for the syntax definitions of the ' + 'last three\n' + 'symbols.)\n' + '\n' + 'An augmented assignment evaluates the target (which, unlike ' + 'normal\n' + 'assignment statements, cannot be an unpacking) and the ' + 'expression\n' + 'list, performs the binary operation specific to the type of ' + 'assignment\n' + 'on the two operands, and assigns the result to the original ' + 'target.\n' + 'The target is only evaluated once.\n' + '\n' + 'An augmented assignment expression like "x += 1" can be ' + 'rewritten as\n' + '"x = x + 1" to achieve a similar, but not exactly equal ' + 'effect. In the\n' + 'augmented version, "x" is only evaluated once. Also, when ' + 'possible,\n' + 'the actual operation is performed *in-place*, meaning that ' + 'rather than\n' + 'creating a new object and assigning that to the target, the ' + 'old object\n' + 'is modified instead.\n' + '\n' + 'Unlike normal assignments, augmented assignments evaluate the ' + 'left-\n' + 'hand side *before* evaluating the right-hand side. For ' + 'example, "a[i]\n' + '+= f(x)" first looks-up "a[i]", then it evaluates "f(x)" and ' + 'performs\n' + 'the addition, and lastly, it writes the result back to ' + '"a[i]".\n' + '\n' + 'With the exception of assigning to tuples and multiple ' + 'targets in a\n' + 'single statement, the assignment done by augmented ' + 'assignment\n' + 'statements is handled the same way as normal assignments. ' + 'Similarly,\n' + 'with the exception of the possible *in-place* behavior, the ' + 'binary\n' + 'operation performed by augmented assignment is the same as ' + 'the normal\n' + 'binary operations.\n' + '\n' + 'For targets which are attribute references, the same *caveat ' + 'about\n' + 'class and instance attributes* applies as for regular ' + 'assignments.\n', + 'binary': '\n' + 'Binary arithmetic operations\n' + '****************************\n' + '\n' + 'The binary arithmetic operations have the conventional priority\n' + 'levels. Note that some of these operations also apply to ' + 'certain non-\n' + 'numeric types. Apart from the power operator, there are only ' + 'two\n' + 'levels, one for multiplicative operators and one for additive\n' + 'operators:\n' + '\n' + ' m_expr ::= u_expr | m_expr "*" u_expr | m_expr "@" m_expr |\n' + ' m_expr "//" u_expr| m_expr "/" u_expr |\n' + ' m_expr "%" u_expr\n' + ' a_expr ::= m_expr | a_expr "+" m_expr | a_expr "-" m_expr\n' + '\n' + 'The "*" (multiplication) operator yields the product of its ' + 'arguments.\n' + 'The arguments must either both be numbers, or one argument must ' + 'be an\n' + 'integer and the other must be a sequence. In the former case, ' + 'the\n' + 'numbers are converted to a common type and then multiplied ' + 'together.\n' + 'In the latter case, sequence repetition is performed; a ' + 'negative\n' + 'repetition factor yields an empty sequence.\n' + '\n' + 'The "@" (at) operator is intended to be used for matrix\n' + 'multiplication. No builtin Python types implement this ' + 'operator.\n' + '\n' + 'New in version 3.5.\n' + '\n' + 'The "/" (division) and "//" (floor division) operators yield ' + 'the\n' + 'quotient of their arguments. The numeric arguments are first\n' + 'converted to a common type. Division of integers yields a float, ' + 'while\n' + 'floor division of integers results in an integer; the result is ' + 'that\n' + "of mathematical division with the 'floor' function applied to " + 'the\n' + 'result. Division by zero raises the "ZeroDivisionError" ' + 'exception.\n' + '\n' + 'The "%" (modulo) operator yields the remainder from the division ' + 'of\n' + 'the first argument by the second. The numeric arguments are ' + 'first\n' + 'converted to a common type. A zero right argument raises the\n' + '"ZeroDivisionError" exception. The arguments may be floating ' + 'point\n' + 'numbers, e.g., "3.14%0.7" equals "0.34" (since "3.14" equals ' + '"4*0.7 +\n' + '0.34".) The modulo operator always yields a result with the ' + 'same sign\n' + 'as its second operand (or zero); the absolute value of the ' + 'result is\n' + 'strictly smaller than the absolute value of the second operand ' + '[1].\n' + '\n' + 'The floor division and modulo operators are connected by the ' + 'following\n' + 'identity: "x == (x//y)*y + (x%y)". Floor division and modulo ' + 'are also\n' + 'connected with the built-in function "divmod()": "divmod(x, y) ' + '==\n' + '(x//y, x%y)". [2].\n' + '\n' + 'In addition to performing the modulo operation on numbers, the ' + '"%"\n' + 'operator is also overloaded by string objects to perform ' + 'old-style\n' + 'string formatting (also known as interpolation). The syntax ' + 'for\n' + 'string formatting is described in the Python Library Reference,\n' + 'section *printf-style String Formatting*.\n' + '\n' + 'The floor division operator, the modulo operator, and the ' + '"divmod()"\n' + 'function are not defined for complex numbers. Instead, convert ' + 'to a\n' + 'floating point number using the "abs()" function if ' + 'appropriate.\n' + '\n' + 'The "+" (addition) operator yields the sum of its arguments. ' + 'The\n' + 'arguments must either both be numbers or both be sequences of ' + 'the same\n' + 'type. In the former case, the numbers are converted to a common ' + 'type\n' + 'and then added together. In the latter case, the sequences are\n' + 'concatenated.\n' + '\n' + 'The "-" (subtraction) operator yields the difference of its ' + 'arguments.\n' + 'The numeric arguments are first converted to a common type.\n', + 'bitwise': '\n' + 'Binary bitwise operations\n' + '*************************\n' + '\n' + 'Each of the three bitwise operations has a different priority ' + 'level:\n' + '\n' + ' and_expr ::= shift_expr | and_expr "&" shift_expr\n' + ' xor_expr ::= and_expr | xor_expr "^" and_expr\n' + ' or_expr ::= xor_expr | or_expr "|" xor_expr\n' + '\n' + 'The "&" operator yields the bitwise AND of its arguments, which ' + 'must\n' + 'be integers.\n' + '\n' + 'The "^" operator yields the bitwise XOR (exclusive OR) of its\n' + 'arguments, which must be integers.\n' + '\n' + 'The "|" operator yields the bitwise (inclusive) OR of its ' + 'arguments,\n' + 'which must be integers.\n', + 'bltin-code-objects': '\n' + 'Code Objects\n' + '************\n' + '\n' + 'Code objects are used by the implementation to ' + 'represent "pseudo-\n' + 'compiled" executable Python code such as a function ' + 'body. They differ\n' + "from function objects because they don't contain a " + 'reference to their\n' + 'global execution environment. Code objects are ' + 'returned by the built-\n' + 'in "compile()" function and can be extracted from ' + 'function objects\n' + 'through their "__code__" attribute. See also the ' + '"code" module.\n' + '\n' + 'A code object can be executed or evaluated by ' + 'passing it (instead of a\n' + 'source string) to the "exec()" or "eval()" built-in ' + 'functions.\n' + '\n' + 'See *The standard type hierarchy* for more ' + 'information.\n', + 'bltin-ellipsis-object': '\n' + 'The Ellipsis Object\n' + '*******************\n' + '\n' + 'This object is commonly used by slicing (see ' + '*Slicings*). It supports\n' + 'no special operations. There is exactly one ' + 'ellipsis object, named\n' + '"Ellipsis" (a built-in name). "type(Ellipsis)()" ' + 'produces the\n' + '"Ellipsis" singleton.\n' + '\n' + 'It is written as "Ellipsis" or "...".\n', + 'bltin-null-object': '\n' + 'The Null Object\n' + '***************\n' + '\n' + "This object is returned by functions that don't " + 'explicitly return a\n' + 'value. It supports no special operations. There is ' + 'exactly one null\n' + 'object, named "None" (a built-in name). ' + '"type(None)()" produces the\n' + 'same singleton.\n' + '\n' + 'It is written as "None".\n', + 'bltin-type-objects': '\n' + 'Type Objects\n' + '************\n' + '\n' + 'Type objects represent the various object types. An ' + "object's type is\n" + 'accessed by the built-in function "type()". There ' + 'are no special\n' + 'operations on types. The standard module "types" ' + 'defines names for\n' + 'all standard built-in types.\n' + '\n' + 'Types are written like this: "".\n', + 'booleans': '\n' + 'Boolean operations\n' + '******************\n' + '\n' + ' or_test ::= and_test | or_test "or" and_test\n' + ' and_test ::= not_test | and_test "and" not_test\n' + ' not_test ::= comparison | "not" not_test\n' + '\n' + 'In the context of Boolean operations, and also when ' + 'expressions are\n' + 'used by control flow statements, the following values are ' + 'interpreted\n' + 'as false: "False", "None", numeric zero of all types, and ' + 'empty\n' + 'strings and containers (including strings, tuples, lists,\n' + 'dictionaries, sets and frozensets). All other values are ' + 'interpreted\n' + 'as true. User-defined objects can customize their truth value ' + 'by\n' + 'providing a "__bool__()" method.\n' + '\n' + 'The operator "not" yields "True" if its argument is false, ' + '"False"\n' + 'otherwise.\n' + '\n' + 'The expression "x and y" first evaluates *x*; if *x* is false, ' + 'its\n' + 'value is returned; otherwise, *y* is evaluated and the ' + 'resulting value\n' + 'is returned.\n' + '\n' + 'The expression "x or y" first evaluates *x*; if *x* is true, ' + 'its value\n' + 'is returned; otherwise, *y* is evaluated and the resulting ' + 'value is\n' + 'returned.\n' + '\n' + '(Note that neither "and" nor "or" restrict the value and type ' + 'they\n' + 'return to "False" and "True", but rather return the last ' + 'evaluated\n' + 'argument. This is sometimes useful, e.g., if "s" is a string ' + 'that\n' + 'should be replaced by a default value if it is empty, the ' + 'expression\n' + '"s or \'foo\'" yields the desired value. Because "not" has to ' + 'create a\n' + 'new value, it returns a boolean value regardless of the type ' + 'of its\n' + 'argument (for example, "not \'foo\'" produces "False" rather ' + 'than "\'\'".)\n', + 'break': '\n' + 'The "break" statement\n' + '*********************\n' + '\n' + ' break_stmt ::= "break"\n' + '\n' + '"break" may only occur syntactically nested in a "for" or ' + '"while"\n' + 'loop, but not nested in a function or class definition within ' + 'that\n' + 'loop.\n' + '\n' + 'It terminates the nearest enclosing loop, skipping the optional ' + '"else"\n' + 'clause if the loop has one.\n' + '\n' + 'If a "for" loop is terminated by "break", the loop control ' + 'target\n' + 'keeps its current value.\n' + '\n' + 'When "break" passes control out of a "try" statement with a ' + '"finally"\n' + 'clause, that "finally" clause is executed before really leaving ' + 'the\n' + 'loop.\n', + 'callable-types': '\n' + 'Emulating callable objects\n' + '**************************\n' + '\n' + 'object.__call__(self[, args...])\n' + '\n' + ' Called when the instance is "called" as a function; ' + 'if this method\n' + ' is defined, "x(arg1, arg2, ...)" is a shorthand for\n' + ' "x.__call__(arg1, arg2, ...)".\n', + 'calls': '\n' + 'Calls\n' + '*****\n' + '\n' + 'A call calls a callable object (e.g., a *function*) with a ' + 'possibly\n' + 'empty series of *arguments*:\n' + '\n' + ' call ::= primary "(" [argument_list [","] | ' + 'comprehension] ")"\n' + ' argument_list ::= positional_arguments ["," ' + 'keyword_arguments]\n' + ' ["," "*" expression] ["," ' + 'keyword_arguments]\n' + ' ["," "**" expression]\n' + ' | keyword_arguments ["," "*" expression]\n' + ' ["," keyword_arguments] ["," "**" ' + 'expression]\n' + ' | "*" expression ["," keyword_arguments] ' + '["," "**" expression]\n' + ' | "**" expression\n' + ' positional_arguments ::= expression ("," expression)*\n' + ' keyword_arguments ::= keyword_item ("," keyword_item)*\n' + ' keyword_item ::= identifier "=" expression\n' + '\n' + 'An optional trailing comma may be present after the positional ' + 'and\n' + 'keyword arguments but does not affect the semantics.\n' + '\n' + 'The primary must evaluate to a callable object (user-defined\n' + 'functions, built-in functions, methods of built-in objects, ' + 'class\n' + 'objects, methods of class instances, and all objects having a\n' + '"__call__()" method are callable). All argument expressions are\n' + 'evaluated before the call is attempted. Please refer to section\n' + '*Function definitions* for the syntax of formal *parameter* ' + 'lists.\n' + '\n' + 'If keyword arguments are present, they are first converted to\n' + 'positional arguments, as follows. First, a list of unfilled ' + 'slots is\n' + 'created for the formal parameters. If there are N positional\n' + 'arguments, they are placed in the first N slots. Next, for each\n' + 'keyword argument, the identifier is used to determine the\n' + 'corresponding slot (if the identifier is the same as the first ' + 'formal\n' + 'parameter name, the first slot is used, and so on). If the slot ' + 'is\n' + 'already filled, a "TypeError" exception is raised. Otherwise, ' + 'the\n' + 'value of the argument is placed in the slot, filling it (even if ' + 'the\n' + 'expression is "None", it fills the slot). When all arguments ' + 'have\n' + 'been processed, the slots that are still unfilled are filled with ' + 'the\n' + 'corresponding default value from the function definition. ' + '(Default\n' + 'values are calculated, once, when the function is defined; thus, ' + 'a\n' + 'mutable object such as a list or dictionary used as default value ' + 'will\n' + "be shared by all calls that don't specify an argument value for " + 'the\n' + 'corresponding slot; this should usually be avoided.) If there ' + 'are any\n' + 'unfilled slots for which no default value is specified, a ' + '"TypeError"\n' + 'exception is raised. Otherwise, the list of filled slots is used ' + 'as\n' + 'the argument list for the call.\n' + '\n' + '**CPython implementation detail:** An implementation may provide\n' + 'built-in functions whose positional parameters do not have names, ' + 'even\n' + "if they are 'named' for the purpose of documentation, and which\n" + 'therefore cannot be supplied by keyword. In CPython, this is the ' + 'case\n' + 'for functions implemented in C that use "PyArg_ParseTuple()" to ' + 'parse\n' + 'their arguments.\n' + '\n' + 'If there are more positional arguments than there are formal ' + 'parameter\n' + 'slots, a "TypeError" exception is raised, unless a formal ' + 'parameter\n' + 'using the syntax "*identifier" is present; in this case, that ' + 'formal\n' + 'parameter receives a tuple containing the excess positional ' + 'arguments\n' + '(or an empty tuple if there were no excess positional ' + 'arguments).\n' + '\n' + 'If any keyword argument does not correspond to a formal ' + 'parameter\n' + 'name, a "TypeError" exception is raised, unless a formal ' + 'parameter\n' + 'using the syntax "**identifier" is present; in this case, that ' + 'formal\n' + 'parameter receives a dictionary containing the excess keyword\n' + 'arguments (using the keywords as keys and the argument values as\n' + 'corresponding values), or a (new) empty dictionary if there were ' + 'no\n' + 'excess keyword arguments.\n' + '\n' + 'If the syntax "*expression" appears in the function call, ' + '"expression"\n' + 'must evaluate to an iterable. Elements from this iterable are ' + 'treated\n' + 'as if they were additional positional arguments; if there are\n' + 'positional arguments *x1*, ..., *xN*, and "expression" evaluates ' + 'to a\n' + 'sequence *y1*, ..., *yM*, this is equivalent to a call with M+N\n' + 'positional arguments *x1*, ..., *xN*, *y1*, ..., *yM*.\n' + '\n' + 'A consequence of this is that although the "*expression" syntax ' + 'may\n' + 'appear *after* some keyword arguments, it is processed *before* ' + 'the\n' + 'keyword arguments (and the "**expression" argument, if any -- ' + 'see\n' + 'below). So:\n' + '\n' + ' >>> def f(a, b):\n' + ' ... print(a, b)\n' + ' ...\n' + ' >>> f(b=1, *(2,))\n' + ' 2 1\n' + ' >>> f(a=1, *(2,))\n' + ' Traceback (most recent call last):\n' + ' File "", line 1, in ?\n' + " TypeError: f() got multiple values for keyword argument 'a'\n" + ' >>> f(1, *(2,))\n' + ' 1 2\n' + '\n' + 'It is unusual for both keyword arguments and the "*expression" ' + 'syntax\n' + 'to be used in the same call, so in practice this confusion does ' + 'not\n' + 'arise.\n' + '\n' + 'If the syntax "**expression" appears in the function call,\n' + '"expression" must evaluate to a mapping, the contents of which ' + 'are\n' + 'treated as additional keyword arguments. In the case of a ' + 'keyword\n' + 'appearing in both "expression" and as an explicit keyword ' + 'argument, a\n' + '"TypeError" exception is raised.\n' + '\n' + 'Formal parameters using the syntax "*identifier" or ' + '"**identifier"\n' + 'cannot be used as positional argument slots or as keyword ' + 'argument\n' + 'names.\n' + '\n' + 'A call always returns some value, possibly "None", unless it ' + 'raises an\n' + 'exception. How this value is computed depends on the type of ' + 'the\n' + 'callable object.\n' + '\n' + 'If it is---\n' + '\n' + 'a user-defined function:\n' + ' The code block for the function is executed, passing it the\n' + ' argument list. The first thing the code block will do is bind ' + 'the\n' + ' formal parameters to the arguments; this is described in ' + 'section\n' + ' *Function definitions*. When the code block executes a ' + '"return"\n' + ' statement, this specifies the return value of the function ' + 'call.\n' + '\n' + 'a built-in function or method:\n' + ' The result is up to the interpreter; see *Built-in Functions* ' + 'for\n' + ' the descriptions of built-in functions and methods.\n' + '\n' + 'a class object:\n' + ' A new instance of that class is returned.\n' + '\n' + 'a class instance method:\n' + ' The corresponding user-defined function is called, with an ' + 'argument\n' + ' list that is one longer than the argument list of the call: ' + 'the\n' + ' instance becomes the first argument.\n' + '\n' + 'a class instance:\n' + ' The class must define a "__call__()" method; the effect is ' + 'then the\n' + ' same as if that method was called.\n', + 'class': '\n' + 'Class definitions\n' + '*****************\n' + '\n' + 'A class definition defines a class object (see section *The ' + 'standard\n' + 'type hierarchy*):\n' + '\n' + ' classdef ::= [decorators] "class" classname [inheritance] ' + '":" suite\n' + ' inheritance ::= "(" [parameter_list] ")"\n' + ' classname ::= identifier\n' + '\n' + 'A class definition is an executable statement. The inheritance ' + 'list\n' + 'usually gives a list of base classes (see *Customizing class ' + 'creation*\n' + 'for more advanced uses), so each item in the list should evaluate ' + 'to a\n' + 'class object which allows subclassing. Classes without an ' + 'inheritance\n' + 'list inherit, by default, from the base class "object"; hence,\n' + '\n' + ' class Foo:\n' + ' pass\n' + '\n' + 'is equivalent to\n' + '\n' + ' class Foo(object):\n' + ' pass\n' + '\n' + "The class's suite is then executed in a new execution frame (see\n" + '*Naming and binding*), using a newly created local namespace and ' + 'the\n' + 'original global namespace. (Usually, the suite contains mostly\n' + "function definitions.) When the class's suite finishes " + 'execution, its\n' + 'execution frame is discarded but its local namespace is saved. ' + '[4] A\n' + 'class object is then created using the inheritance list for the ' + 'base\n' + 'classes and the saved local namespace for the attribute ' + 'dictionary.\n' + 'The class name is bound to this class object in the original ' + 'local\n' + 'namespace.\n' + '\n' + 'Class creation can be customized heavily using *metaclasses*.\n' + '\n' + 'Classes can also be decorated: just like when decorating ' + 'functions,\n' + '\n' + ' @f1(arg)\n' + ' @f2\n' + ' class Foo: pass\n' + '\n' + 'is equivalent to\n' + '\n' + ' class Foo: pass\n' + ' Foo = f1(arg)(f2(Foo))\n' + '\n' + 'The evaluation rules for the decorator expressions are the same ' + 'as for\n' + 'function decorators. The result must be a class object, which is ' + 'then\n' + 'bound to the class name.\n' + '\n' + "**Programmer's note:** Variables defined in the class definition " + 'are\n' + 'class attributes; they are shared by instances. Instance ' + 'attributes\n' + 'can be set in a method with "self.name = value". Both class and\n' + 'instance attributes are accessible through the notation ' + '""self.name"",\n' + 'and an instance attribute hides a class attribute with the same ' + 'name\n' + 'when accessed in this way. Class attributes can be used as ' + 'defaults\n' + 'for instance attributes, but using mutable values there can lead ' + 'to\n' + 'unexpected results. *Descriptors* can be used to create ' + 'instance\n' + 'variables with different implementation details.\n' + '\n' + 'See also: **PEP 3115** - Metaclasses in Python 3 **PEP 3129** -\n' + ' Class Decorators\n', + 'comparisons': '\n' + 'Comparisons\n' + '***********\n' + '\n' + 'Unlike C, all comparison operations in Python have the same ' + 'priority,\n' + 'which is lower than that of any arithmetic, shifting or ' + 'bitwise\n' + 'operation. Also unlike C, expressions like "a < b < c" ' + 'have the\n' + 'interpretation that is conventional in mathematics:\n' + '\n' + ' comparison ::= or_expr ( comp_operator or_expr )*\n' + ' comp_operator ::= "<" | ">" | "==" | ">=" | "<=" | "!="\n' + ' | "is" ["not"] | ["not"] "in"\n' + '\n' + 'Comparisons yield boolean values: "True" or "False".\n' + '\n' + 'Comparisons can be chained arbitrarily, e.g., "x < y <= z" ' + 'is\n' + 'equivalent to "x < y and y <= z", except that "y" is ' + 'evaluated only\n' + 'once (but in both cases "z" is not evaluated at all when "x ' + '< y" is\n' + 'found to be false).\n' + '\n' + 'Formally, if *a*, *b*, *c*, ..., *y*, *z* are expressions ' + 'and *op1*,\n' + '*op2*, ..., *opN* are comparison operators, then "a op1 b ' + 'op2 c ... y\n' + 'opN z" is equivalent to "a op1 b and b op2 c and ... y opN ' + 'z", except\n' + 'that each expression is evaluated at most once.\n' + '\n' + 'Note that "a op1 b op2 c" doesn\'t imply any kind of ' + 'comparison between\n' + '*a* and *c*, so that, e.g., "x < y > z" is perfectly legal ' + '(though\n' + 'perhaps not pretty).\n' + '\n' + 'The operators "<", ">", "==", ">=", "<=", and "!=" compare ' + 'the values\n' + 'of two objects. The objects need not have the same type. ' + 'If both are\n' + 'numbers, they are converted to a common type. Otherwise, ' + 'the "==" and\n' + '"!=" operators *always* consider objects of different types ' + 'to be\n' + 'unequal, while the "<", ">", ">=" and "<=" operators raise ' + 'a\n' + '"TypeError" when comparing objects of different types that ' + 'do not\n' + 'implement these operators for the given pair of types. You ' + 'can\n' + 'control comparison behavior of objects of non-built-in ' + 'types by\n' + 'defining rich comparison methods like "__gt__()", described ' + 'in section\n' + '*Basic customization*.\n' + '\n' + 'Comparison of objects of the same type depends on the ' + 'type:\n' + '\n' + '* Numbers are compared arithmetically.\n' + '\n' + '* The values "float(\'NaN\')" and "Decimal(\'NaN\')" are ' + 'special. They\n' + ' are identical to themselves, "x is x" but are not equal ' + 'to\n' + ' themselves, "x != x". Additionally, comparing any value ' + 'to a\n' + ' not-a-number value will return "False". For example, ' + 'both "3 <\n' + ' float(\'NaN\')" and "float(\'NaN\') < 3" will return ' + '"False".\n' + '\n' + '* Bytes objects are compared lexicographically using the ' + 'numeric\n' + ' values of their elements.\n' + '\n' + '* Strings are compared lexicographically using the numeric\n' + ' equivalents (the result of the built-in function "ord()") ' + 'of their\n' + " characters. [3] String and bytes object can't be " + 'compared!\n' + '\n' + '* Tuples and lists are compared lexicographically using ' + 'comparison\n' + ' of corresponding elements. This means that to compare ' + 'equal, each\n' + ' element must compare equal and the two sequences must be ' + 'of the same\n' + ' type and have the same length.\n' + '\n' + ' If not equal, the sequences are ordered the same as their ' + 'first\n' + ' differing elements. For example, "[1,2,x] <= [1,2,y]" ' + 'has the same\n' + ' value as "x <= y". If the corresponding element does not ' + 'exist, the\n' + ' shorter sequence is ordered first (for example, "[1,2] < ' + '[1,2,3]").\n' + '\n' + '* Mappings (dictionaries) compare equal if and only if they ' + 'have the\n' + ' same "(key, value)" pairs. Order comparisons "(\'<\', ' + "'<=', '>=',\n" + ' \'>\')" raise "TypeError".\n' + '\n' + '* Sets and frozensets define comparison operators to mean ' + 'subset and\n' + ' superset tests. Those relations do not define total ' + 'orderings (the\n' + ' two sets "{1,2}" and "{2,3}" are not equal, nor subsets ' + 'of one\n' + ' another, nor supersets of one another). Accordingly, ' + 'sets are not\n' + ' appropriate arguments for functions which depend on total ' + 'ordering.\n' + ' For example, "min()", "max()", and "sorted()" produce ' + 'undefined\n' + ' results given a list of sets as inputs.\n' + '\n' + '* Most other objects of built-in types compare unequal ' + 'unless they\n' + ' are the same object; the choice whether one object is ' + 'considered\n' + ' smaller or larger than another one is made arbitrarily ' + 'but\n' + ' consistently within one execution of a program.\n' + '\n' + 'Comparison of objects of differing types depends on whether ' + 'either of\n' + 'the types provide explicit support for the comparison. ' + 'Most numeric\n' + 'types can be compared with one another. When cross-type ' + 'comparison is\n' + 'not supported, the comparison method returns ' + '"NotImplemented".\n' + '\n' + 'The operators "in" and "not in" test for membership. "x in ' + 's"\n' + 'evaluates to true if *x* is a member of *s*, and false ' + 'otherwise. "x\n' + 'not in s" returns the negation of "x in s". All built-in ' + 'sequences\n' + 'and set types support this as well as dictionary, for which ' + '"in" tests\n' + 'whether the dictionary has a given key. For container types ' + 'such as\n' + 'list, tuple, set, frozenset, dict, or collections.deque, ' + 'the\n' + 'expression "x in y" is equivalent to "any(x is e or x == e ' + 'for e in\n' + 'y)".\n' + '\n' + 'For the string and bytes types, "x in y" is true if and ' + 'only if *x* is\n' + 'a substring of *y*. An equivalent test is "y.find(x) != ' + '-1". Empty\n' + 'strings are always considered to be a substring of any ' + 'other string,\n' + 'so """ in "abc"" will return "True".\n' + '\n' + 'For user-defined classes which define the "__contains__()" ' + 'method, "x\n' + 'in y" is true if and only if "y.__contains__(x)" is true.\n' + '\n' + 'For user-defined classes which do not define ' + '"__contains__()" but do\n' + 'define "__iter__()", "x in y" is true if some value "z" ' + 'with "x == z"\n' + 'is produced while iterating over "y". If an exception is ' + 'raised\n' + 'during the iteration, it is as if "in" raised that ' + 'exception.\n' + '\n' + 'Lastly, the old-style iteration protocol is tried: if a ' + 'class defines\n' + '"__getitem__()", "x in y" is true if and only if there is a ' + 'non-\n' + 'negative integer index *i* such that "x == y[i]", and all ' + 'lower\n' + 'integer indices do not raise "IndexError" exception. (If ' + 'any other\n' + 'exception is raised, it is as if "in" raised that ' + 'exception).\n' + '\n' + 'The operator "not in" is defined to have the inverse true ' + 'value of\n' + '"in".\n' + '\n' + 'The operators "is" and "is not" test for object identity: ' + '"x is y" is\n' + 'true if and only if *x* and *y* are the same object. "x is ' + 'not y"\n' + 'yields the inverse truth value. [4]\n', + 'compound': '\n' + 'Compound statements\n' + '*******************\n' + '\n' + 'Compound statements contain (groups of) other statements; they ' + 'affect\n' + 'or control the execution of those other statements in some ' + 'way. In\n' + 'general, compound statements span multiple lines, although in ' + 'simple\n' + 'incarnations a whole compound statement may be contained in ' + 'one line.\n' + '\n' + 'The "if", "while" and "for" statements implement traditional ' + 'control\n' + 'flow constructs. "try" specifies exception handlers and/or ' + 'cleanup\n' + 'code for a group of statements, while the "with" statement ' + 'allows the\n' + 'execution of initialization and finalization code around a ' + 'block of\n' + 'code. Function and class definitions are also syntactically ' + 'compound\n' + 'statements.\n' + '\n' + "A compound statement consists of one or more 'clauses.' A " + 'clause\n' + "consists of a header and a 'suite.' The clause headers of a\n" + 'particular compound statement are all at the same indentation ' + 'level.\n' + 'Each clause header begins with a uniquely identifying keyword ' + 'and ends\n' + 'with a colon. A suite is a group of statements controlled by ' + 'a\n' + 'clause. A suite can be one or more semicolon-separated ' + 'simple\n' + 'statements on the same line as the header, following the ' + "header's\n" + 'colon, or it can be one or more indented statements on ' + 'subsequent\n' + 'lines. Only the latter form of a suite can contain nested ' + 'compound\n' + 'statements; the following is illegal, mostly because it ' + "wouldn't be\n" + 'clear to which "if" clause a following "else" clause would ' + 'belong:\n' + '\n' + ' if test1: if test2: print(x)\n' + '\n' + 'Also note that the semicolon binds tighter than the colon in ' + 'this\n' + 'context, so that in the following example, either all or none ' + 'of the\n' + '"print()" calls are executed:\n' + '\n' + ' if x < y < z: print(x); print(y); print(z)\n' + '\n' + 'Summarizing:\n' + '\n' + ' compound_stmt ::= if_stmt\n' + ' | while_stmt\n' + ' | for_stmt\n' + ' | try_stmt\n' + ' | with_stmt\n' + ' | funcdef\n' + ' | classdef\n' + ' | async_with_stmt\n' + ' | async_for_stmt\n' + ' | async_funcdef\n' + ' suite ::= stmt_list NEWLINE | NEWLINE INDENT ' + 'statement+ DEDENT\n' + ' statement ::= stmt_list NEWLINE | compound_stmt\n' + ' stmt_list ::= simple_stmt (";" simple_stmt)* [";"]\n' + '\n' + 'Note that statements always end in a "NEWLINE" possibly ' + 'followed by a\n' + '"DEDENT". Also note that optional continuation clauses always ' + 'begin\n' + 'with a keyword that cannot start a statement, thus there are ' + 'no\n' + 'ambiguities (the \'dangling "else"\' problem is solved in ' + 'Python by\n' + 'requiring nested "if" statements to be indented).\n' + '\n' + 'The formatting of the grammar rules in the following sections ' + 'places\n' + 'each clause on a separate line for clarity.\n' + '\n' + '\n' + 'The "if" statement\n' + '==================\n' + '\n' + 'The "if" statement is used for conditional execution:\n' + '\n' + ' if_stmt ::= "if" expression ":" suite\n' + ' ( "elif" expression ":" suite )*\n' + ' ["else" ":" suite]\n' + '\n' + 'It selects exactly one of the suites by evaluating the ' + 'expressions one\n' + 'by one until one is found to be true (see section *Boolean ' + 'operations*\n' + 'for the definition of true and false); then that suite is ' + 'executed\n' + '(and no other part of the "if" statement is executed or ' + 'evaluated).\n' + 'If all expressions are false, the suite of the "else" clause, ' + 'if\n' + 'present, is executed.\n' + '\n' + '\n' + 'The "while" statement\n' + '=====================\n' + '\n' + 'The "while" statement is used for repeated execution as long ' + 'as an\n' + 'expression is true:\n' + '\n' + ' while_stmt ::= "while" expression ":" suite\n' + ' ["else" ":" suite]\n' + '\n' + 'This repeatedly tests the expression and, if it is true, ' + 'executes the\n' + 'first suite; if the expression is false (which may be the ' + 'first time\n' + 'it is tested) the suite of the "else" clause, if present, is ' + 'executed\n' + 'and the loop terminates.\n' + '\n' + 'A "break" statement executed in the first suite terminates the ' + 'loop\n' + 'without executing the "else" clause\'s suite. A "continue" ' + 'statement\n' + 'executed in the first suite skips the rest of the suite and ' + 'goes back\n' + 'to testing the expression.\n' + '\n' + '\n' + 'The "for" statement\n' + '===================\n' + '\n' + 'The "for" statement is used to iterate over the elements of a ' + 'sequence\n' + '(such as a string, tuple or list) or other iterable object:\n' + '\n' + ' for_stmt ::= "for" target_list "in" expression_list ":" ' + 'suite\n' + ' ["else" ":" suite]\n' + '\n' + 'The expression list is evaluated once; it should yield an ' + 'iterable\n' + 'object. An iterator is created for the result of the\n' + '"expression_list". The suite is then executed once for each ' + 'item\n' + 'provided by the iterator, in the order returned by the ' + 'iterator. Each\n' + 'item in turn is assigned to the target list using the standard ' + 'rules\n' + 'for assignments (see *Assignment statements*), and then the ' + 'suite is\n' + 'executed. When the items are exhausted (which is immediately ' + 'when the\n' + 'sequence is empty or an iterator raises a "StopIteration" ' + 'exception),\n' + 'the suite in the "else" clause, if present, is executed, and ' + 'the loop\n' + 'terminates.\n' + '\n' + 'A "break" statement executed in the first suite terminates the ' + 'loop\n' + 'without executing the "else" clause\'s suite. A "continue" ' + 'statement\n' + 'executed in the first suite skips the rest of the suite and ' + 'continues\n' + 'with the next item, or with the "else" clause if there is no ' + 'next\n' + 'item.\n' + '\n' + 'The for-loop makes assignments to the variables(s) in the ' + 'target list.\n' + 'This overwrites all previous assignments to those variables ' + 'including\n' + 'those made in the suite of the for-loop:\n' + '\n' + ' for i in range(10):\n' + ' print(i)\n' + ' i = 5 # this will not affect the for-loop\n' + ' # because i will be overwritten with ' + 'the next\n' + ' # index in the range\n' + '\n' + 'Names in the target list are not deleted when the loop is ' + 'finished,\n' + 'but if the sequence is empty, they will not have been assigned ' + 'to at\n' + 'all by the loop. Hint: the built-in function "range()" ' + 'returns an\n' + 'iterator of integers suitable to emulate the effect of ' + 'Pascal\'s "for i\n' + ':= a to b do"; e.g., "list(range(3))" returns the list "[0, 1, ' + '2]".\n' + '\n' + 'Note: There is a subtlety when the sequence is being modified ' + 'by the\n' + ' loop (this can only occur for mutable sequences, i.e. ' + 'lists). An\n' + ' internal counter is used to keep track of which item is used ' + 'next,\n' + ' and this is incremented on each iteration. When this ' + 'counter has\n' + ' reached the length of the sequence the loop terminates. ' + 'This means\n' + ' that if the suite deletes the current (or a previous) item ' + 'from the\n' + ' sequence, the next item will be skipped (since it gets the ' + 'index of\n' + ' the current item which has already been treated). Likewise, ' + 'if the\n' + ' suite inserts an item in the sequence before the current ' + 'item, the\n' + ' current item will be treated again the next time through the ' + 'loop.\n' + ' This can lead to nasty bugs that can be avoided by making a\n' + ' temporary copy using a slice of the whole sequence, e.g.,\n' + '\n' + ' for x in a[:]:\n' + ' if x < 0: a.remove(x)\n' + '\n' + '\n' + 'The "try" statement\n' + '===================\n' + '\n' + 'The "try" statement specifies exception handlers and/or ' + 'cleanup code\n' + 'for a group of statements:\n' + '\n' + ' try_stmt ::= try1_stmt | try2_stmt\n' + ' try1_stmt ::= "try" ":" suite\n' + ' ("except" [expression ["as" identifier]] ":" ' + 'suite)+\n' + ' ["else" ":" suite]\n' + ' ["finally" ":" suite]\n' + ' try2_stmt ::= "try" ":" suite\n' + ' "finally" ":" suite\n' + '\n' + 'The "except" clause(s) specify one or more exception handlers. ' + 'When no\n' + 'exception occurs in the "try" clause, no exception handler is\n' + 'executed. When an exception occurs in the "try" suite, a ' + 'search for an\n' + 'exception handler is started. This search inspects the except ' + 'clauses\n' + 'in turn until one is found that matches the exception. An ' + 'expression-\n' + 'less except clause, if present, must be last; it matches any\n' + 'exception. For an except clause with an expression, that ' + 'expression\n' + 'is evaluated, and the clause matches the exception if the ' + 'resulting\n' + 'object is "compatible" with the exception. An object is ' + 'compatible\n' + 'with an exception if it is the class or a base class of the ' + 'exception\n' + 'object or a tuple containing an item compatible with the ' + 'exception.\n' + '\n' + 'If no except clause matches the exception, the search for an ' + 'exception\n' + 'handler continues in the surrounding code and on the ' + 'invocation stack.\n' + '[1]\n' + '\n' + 'If the evaluation of an expression in the header of an except ' + 'clause\n' + 'raises an exception, the original search for a handler is ' + 'canceled and\n' + 'a search starts for the new exception in the surrounding code ' + 'and on\n' + 'the call stack (it is treated as if the entire "try" statement ' + 'raised\n' + 'the exception).\n' + '\n' + 'When a matching except clause is found, the exception is ' + 'assigned to\n' + 'the target specified after the "as" keyword in that except ' + 'clause, if\n' + "present, and the except clause's suite is executed. All " + 'except\n' + 'clauses must have an executable block. When the end of this ' + 'block is\n' + 'reached, execution continues normally after the entire try ' + 'statement.\n' + '(This means that if two nested handlers exist for the same ' + 'exception,\n' + 'and the exception occurs in the try clause of the inner ' + 'handler, the\n' + 'outer handler will not handle the exception.)\n' + '\n' + 'When an exception has been assigned using "as target", it is ' + 'cleared\n' + 'at the end of the except clause. This is as if\n' + '\n' + ' except E as N:\n' + ' foo\n' + '\n' + 'was translated to\n' + '\n' + ' except E as N:\n' + ' try:\n' + ' foo\n' + ' finally:\n' + ' del N\n' + '\n' + 'This means the exception must be assigned to a different name ' + 'to be\n' + 'able to refer to it after the except clause. Exceptions are ' + 'cleared\n' + 'because with the traceback attached to them, they form a ' + 'reference\n' + 'cycle with the stack frame, keeping all locals in that frame ' + 'alive\n' + 'until the next garbage collection occurs.\n' + '\n' + "Before an except clause's suite is executed, details about " + 'the\n' + 'exception are stored in the "sys" module and can be accessed ' + 'via\n' + '"sys.exc_info()". "sys.exc_info()" returns a 3-tuple ' + 'consisting of the\n' + 'exception class, the exception instance and a traceback object ' + '(see\n' + 'section *The standard type hierarchy*) identifying the point ' + 'in the\n' + 'program where the exception occurred. "sys.exc_info()" values ' + 'are\n' + 'restored to their previous values (before the call) when ' + 'returning\n' + 'from a function that handled an exception.\n' + '\n' + 'The optional "else" clause is executed if and when control ' + 'flows off\n' + 'the end of the "try" clause. [2] Exceptions in the "else" ' + 'clause are\n' + 'not handled by the preceding "except" clauses.\n' + '\n' + 'If "finally" is present, it specifies a \'cleanup\' handler. ' + 'The "try"\n' + 'clause is executed, including any "except" and "else" ' + 'clauses. If an\n' + 'exception occurs in any of the clauses and is not handled, ' + 'the\n' + 'exception is temporarily saved. The "finally" clause is ' + 'executed. If\n' + 'there is a saved exception it is re-raised at the end of the ' + '"finally"\n' + 'clause. If the "finally" clause raises another exception, the ' + 'saved\n' + 'exception is set as the context of the new exception. If the ' + '"finally"\n' + 'clause executes a "return" or "break" statement, the saved ' + 'exception\n' + 'is discarded:\n' + '\n' + ' >>> def f():\n' + ' ... try:\n' + ' ... 1/0\n' + ' ... finally:\n' + ' ... return 42\n' + ' ...\n' + ' >>> f()\n' + ' 42\n' + '\n' + 'The exception information is not available to the program ' + 'during\n' + 'execution of the "finally" clause.\n' + '\n' + 'When a "return", "break" or "continue" statement is executed ' + 'in the\n' + '"try" suite of a "try"..."finally" statement, the "finally" ' + 'clause is\n' + 'also executed \'on the way out.\' A "continue" statement is ' + 'illegal in\n' + 'the "finally" clause. (The reason is a problem with the ' + 'current\n' + 'implementation --- this restriction may be lifted in the ' + 'future).\n' + '\n' + 'The return value of a function is determined by the last ' + '"return"\n' + 'statement executed. Since the "finally" clause always ' + 'executes, a\n' + '"return" statement executed in the "finally" clause will ' + 'always be the\n' + 'last one executed:\n' + '\n' + ' >>> def foo():\n' + ' ... try:\n' + " ... return 'try'\n" + ' ... finally:\n' + " ... return 'finally'\n" + ' ...\n' + ' >>> foo()\n' + " 'finally'\n" + '\n' + 'Additional information on exceptions can be found in section\n' + '*Exceptions*, and information on using the "raise" statement ' + 'to\n' + 'generate exceptions may be found in section *The raise ' + 'statement*.\n' + '\n' + '\n' + 'The "with" statement\n' + '====================\n' + '\n' + 'The "with" statement is used to wrap the execution of a block ' + 'with\n' + 'methods defined by a context manager (see section *With ' + 'Statement\n' + 'Context Managers*). This allows common ' + '"try"..."except"..."finally"\n' + 'usage patterns to be encapsulated for convenient reuse.\n' + '\n' + ' with_stmt ::= "with" with_item ("," with_item)* ":" suite\n' + ' with_item ::= expression ["as" target]\n' + '\n' + 'The execution of the "with" statement with one "item" proceeds ' + 'as\n' + 'follows:\n' + '\n' + '1. The context expression (the expression given in the ' + '"with_item")\n' + ' is evaluated to obtain a context manager.\n' + '\n' + '2. The context manager\'s "__exit__()" is loaded for later ' + 'use.\n' + '\n' + '3. The context manager\'s "__enter__()" method is invoked.\n' + '\n' + '4. If a target was included in the "with" statement, the ' + 'return\n' + ' value from "__enter__()" is assigned to it.\n' + '\n' + ' Note: The "with" statement guarantees that if the ' + '"__enter__()"\n' + ' method returns without an error, then "__exit__()" will ' + 'always be\n' + ' called. Thus, if an error occurs during the assignment to ' + 'the\n' + ' target list, it will be treated the same as an error ' + 'occurring\n' + ' within the suite would be. See step 6 below.\n' + '\n' + '5. The suite is executed.\n' + '\n' + '6. The context manager\'s "__exit__()" method is invoked. If ' + 'an\n' + ' exception caused the suite to be exited, its type, value, ' + 'and\n' + ' traceback are passed as arguments to "__exit__()". ' + 'Otherwise, three\n' + ' "None" arguments are supplied.\n' + '\n' + ' If the suite was exited due to an exception, and the return ' + 'value\n' + ' from the "__exit__()" method was false, the exception is ' + 'reraised.\n' + ' If the return value was true, the exception is suppressed, ' + 'and\n' + ' execution continues with the statement following the ' + '"with"\n' + ' statement.\n' + '\n' + ' If the suite was exited for any reason other than an ' + 'exception, the\n' + ' return value from "__exit__()" is ignored, and execution ' + 'proceeds\n' + ' at the normal location for the kind of exit that was ' + 'taken.\n' + '\n' + 'With more than one item, the context managers are processed as ' + 'if\n' + 'multiple "with" statements were nested:\n' + '\n' + ' with A() as a, B() as b:\n' + ' suite\n' + '\n' + 'is equivalent to\n' + '\n' + ' with A() as a:\n' + ' with B() as b:\n' + ' suite\n' + '\n' + 'Changed in version 3.1: Support for multiple context ' + 'expressions.\n' + '\n' + 'See also: **PEP 0343** - The "with" statement\n' + '\n' + ' The specification, background, and examples for the ' + 'Python "with"\n' + ' statement.\n' + '\n' + '\n' + 'Function definitions\n' + '====================\n' + '\n' + 'A function definition defines a user-defined function object ' + '(see\n' + 'section *The standard type hierarchy*):\n' + '\n' + ' funcdef ::= [decorators] "def" funcname "(" ' + '[parameter_list] ")" ["->" expression] ":" suite\n' + ' decorators ::= decorator+\n' + ' decorator ::= "@" dotted_name ["(" [parameter_list ' + '[","]] ")"] NEWLINE\n' + ' dotted_name ::= identifier ("." identifier)*\n' + ' parameter_list ::= (defparameter ",")*\n' + ' | "*" [parameter] ("," defparameter)* ' + '["," "**" parameter]\n' + ' | "**" parameter\n' + ' | defparameter [","] )\n' + ' parameter ::= identifier [":" expression]\n' + ' defparameter ::= parameter ["=" expression]\n' + ' funcname ::= identifier\n' + '\n' + 'A function definition is an executable statement. Its ' + 'execution binds\n' + 'the function name in the current local namespace to a function ' + 'object\n' + '(a wrapper around the executable code for the function). ' + 'This\n' + 'function object contains a reference to the current global ' + 'namespace\n' + 'as the global namespace to be used when the function is ' + 'called.\n' + '\n' + 'The function definition does not execute the function body; ' + 'this gets\n' + 'executed only when the function is called. [3]\n' + '\n' + 'A function definition may be wrapped by one or more ' + '*decorator*\n' + 'expressions. Decorator expressions are evaluated when the ' + 'function is\n' + 'defined, in the scope that contains the function definition. ' + 'The\n' + 'result must be a callable, which is invoked with the function ' + 'object\n' + 'as the only argument. The returned value is bound to the ' + 'function name\n' + 'instead of the function object. Multiple decorators are ' + 'applied in\n' + 'nested fashion. For example, the following code\n' + '\n' + ' @f1(arg)\n' + ' @f2\n' + ' def func(): pass\n' + '\n' + 'is equivalent to\n' + '\n' + ' def func(): pass\n' + ' func = f1(arg)(f2(func))\n' + '\n' + 'When one or more *parameters* have the form *parameter* "="\n' + '*expression*, the function is said to have "default parameter ' + 'values."\n' + 'For a parameter with a default value, the corresponding ' + '*argument* may\n' + "be omitted from a call, in which case the parameter's default " + 'value is\n' + 'substituted. If a parameter has a default value, all ' + 'following\n' + 'parameters up until the ""*"" must also have a default value ' + '--- this\n' + 'is a syntactic restriction that is not expressed by the ' + 'grammar.\n' + '\n' + '**Default parameter values are evaluated from left to right ' + 'when the\n' + 'function definition is executed.** This means that the ' + 'expression is\n' + 'evaluated once, when the function is defined, and that the ' + 'same "pre-\n' + 'computed" value is used for each call. This is especially ' + 'important\n' + 'to understand when a default parameter is a mutable object, ' + 'such as a\n' + 'list or a dictionary: if the function modifies the object ' + '(e.g. by\n' + 'appending an item to a list), the default value is in effect ' + 'modified.\n' + 'This is generally not what was intended. A way around this is ' + 'to use\n' + '"None" as the default, and explicitly test for it in the body ' + 'of the\n' + 'function, e.g.:\n' + '\n' + ' def whats_on_the_telly(penguin=None):\n' + ' if penguin is None:\n' + ' penguin = []\n' + ' penguin.append("property of the zoo")\n' + ' return penguin\n' + '\n' + 'Function call semantics are described in more detail in ' + 'section\n' + '*Calls*. A function call always assigns values to all ' + 'parameters\n' + 'mentioned in the parameter list, either from position ' + 'arguments, from\n' + 'keyword arguments, or from default values. If the form\n' + '""*identifier"" is present, it is initialized to a tuple ' + 'receiving any\n' + 'excess positional parameters, defaulting to the empty tuple. ' + 'If the\n' + 'form ""**identifier"" is present, it is initialized to a new\n' + 'dictionary receiving any excess keyword arguments, defaulting ' + 'to a new\n' + 'empty dictionary. Parameters after ""*"" or ""*identifier"" ' + 'are\n' + 'keyword-only parameters and may only be passed used keyword ' + 'arguments.\n' + '\n' + 'Parameters may have annotations of the form "": expression"" ' + 'following\n' + 'the parameter name. Any parameter may have an annotation even ' + 'those\n' + 'of the form "*identifier" or "**identifier". Functions may ' + 'have\n' + '"return" annotation of the form ""-> expression"" after the ' + 'parameter\n' + 'list. These annotations can be any valid Python expression ' + 'and are\n' + 'evaluated when the function definition is executed. ' + 'Annotations may\n' + 'be evaluated in a different order than they appear in the ' + 'source code.\n' + 'The presence of annotations does not change the semantics of ' + 'a\n' + 'function. The annotation values are available as values of a\n' + "dictionary keyed by the parameters' names in the " + '"__annotations__"\n' + 'attribute of the function object.\n' + '\n' + 'It is also possible to create anonymous functions (functions ' + 'not bound\n' + 'to a name), for immediate use in expressions. This uses ' + 'lambda\n' + 'expressions, described in section *Lambdas*. Note that the ' + 'lambda\n' + 'expression is merely a shorthand for a simplified function ' + 'definition;\n' + 'a function defined in a ""def"" statement can be passed around ' + 'or\n' + 'assigned to another name just like a function defined by a ' + 'lambda\n' + 'expression. The ""def"" form is actually more powerful since ' + 'it\n' + 'allows the execution of multiple statements and annotations.\n' + '\n' + "**Programmer's note:** Functions are first-class objects. A " + '""def""\n' + 'statement executed inside a function definition defines a ' + 'local\n' + 'function that can be returned or passed around. Free ' + 'variables used\n' + 'in the nested function can access the local variables of the ' + 'function\n' + 'containing the def. See section *Naming and binding* for ' + 'details.\n' + '\n' + 'See also: **PEP 3107** - Function Annotations\n' + '\n' + ' The original specification for function annotations.\n' + '\n' + '\n' + 'Class definitions\n' + '=================\n' + '\n' + 'A class definition defines a class object (see section *The ' + 'standard\n' + 'type hierarchy*):\n' + '\n' + ' classdef ::= [decorators] "class" classname ' + '[inheritance] ":" suite\n' + ' inheritance ::= "(" [parameter_list] ")"\n' + ' classname ::= identifier\n' + '\n' + 'A class definition is an executable statement. The ' + 'inheritance list\n' + 'usually gives a list of base classes (see *Customizing class ' + 'creation*\n' + 'for more advanced uses), so each item in the list should ' + 'evaluate to a\n' + 'class object which allows subclassing. Classes without an ' + 'inheritance\n' + 'list inherit, by default, from the base class "object"; ' + 'hence,\n' + '\n' + ' class Foo:\n' + ' pass\n' + '\n' + 'is equivalent to\n' + '\n' + ' class Foo(object):\n' + ' pass\n' + '\n' + "The class's suite is then executed in a new execution frame " + '(see\n' + '*Naming and binding*), using a newly created local namespace ' + 'and the\n' + 'original global namespace. (Usually, the suite contains ' + 'mostly\n' + "function definitions.) When the class's suite finishes " + 'execution, its\n' + 'execution frame is discarded but its local namespace is saved. ' + '[4] A\n' + 'class object is then created using the inheritance list for ' + 'the base\n' + 'classes and the saved local namespace for the attribute ' + 'dictionary.\n' + 'The class name is bound to this class object in the original ' + 'local\n' + 'namespace.\n' + '\n' + 'Class creation can be customized heavily using *metaclasses*.\n' + '\n' + 'Classes can also be decorated: just like when decorating ' + 'functions,\n' + '\n' + ' @f1(arg)\n' + ' @f2\n' + ' class Foo: pass\n' + '\n' + 'is equivalent to\n' + '\n' + ' class Foo: pass\n' + ' Foo = f1(arg)(f2(Foo))\n' + '\n' + 'The evaluation rules for the decorator expressions are the ' + 'same as for\n' + 'function decorators. The result must be a class object, which ' + 'is then\n' + 'bound to the class name.\n' + '\n' + "**Programmer's note:** Variables defined in the class " + 'definition are\n' + 'class attributes; they are shared by instances. Instance ' + 'attributes\n' + 'can be set in a method with "self.name = value". Both class ' + 'and\n' + 'instance attributes are accessible through the notation ' + '""self.name"",\n' + 'and an instance attribute hides a class attribute with the ' + 'same name\n' + 'when accessed in this way. Class attributes can be used as ' + 'defaults\n' + 'for instance attributes, but using mutable values there can ' + 'lead to\n' + 'unexpected results. *Descriptors* can be used to create ' + 'instance\n' + 'variables with different implementation details.\n' + '\n' + 'See also: **PEP 3115** - Metaclasses in Python 3 **PEP 3129** ' + '-\n' + ' Class Decorators\n' + '\n' + '\n' + 'Coroutines\n' + '==========\n' + '\n' + 'New in version 3.5.\n' + '\n' + '\n' + 'Coroutine function definition\n' + '-----------------------------\n' + '\n' + ' async_funcdef ::= [decorators] "async" "def" funcname "(" ' + '[parameter_list] ")" ["->" expression] ":" suite\n' + '\n' + 'Execution of Python coroutines can be suspended and resumed at ' + 'many\n' + 'points (see *coroutine*). In the body of a coroutine, any ' + '"await" and\n' + '"async" identifiers become reserved keywords; "await" ' + 'expressions,\n' + '"async for" and "async with" can only be used in coroutine ' + 'bodies.\n' + '\n' + 'Functions defined with "async def" syntax are always ' + 'coroutine\n' + 'functions, even if they do not contain "await" or "async" ' + 'keywords.\n' + '\n' + 'It is a "SyntaxError" to use "yield" expressions in "async ' + 'def"\n' + 'coroutines.\n' + '\n' + 'An example of a coroutine function:\n' + '\n' + ' async def func(param1, param2):\n' + ' do_stuff()\n' + ' await some_coroutine()\n' + '\n' + '\n' + 'The "async for" statement\n' + '-------------------------\n' + '\n' + ' async_for_stmt ::= "async" for_stmt\n' + '\n' + 'An *asynchronous iterable* is able to call asynchronous code ' + 'in its\n' + '*iter* implementation, and *asynchronous iterator* can call\n' + 'asynchronous code in its *next* method.\n' + '\n' + 'The "async for" statement allows convenient iteration over\n' + 'asynchronous iterators.\n' + '\n' + 'The following code:\n' + '\n' + ' async for TARGET in ITER:\n' + ' BLOCK\n' + ' else:\n' + ' BLOCK2\n' + '\n' + 'Is semantically equivalent to:\n' + '\n' + ' iter = (ITER)\n' + ' iter = await type(iter).__aiter__(iter)\n' + ' running = True\n' + ' while running:\n' + ' try:\n' + ' TARGET = await type(iter).__anext__(iter)\n' + ' except StopAsyncIteration:\n' + ' running = False\n' + ' else:\n' + ' BLOCK\n' + ' else:\n' + ' BLOCK2\n' + '\n' + 'See also "__aiter__()" and "__anext__()" for details.\n' + '\n' + 'It is a "SyntaxError" to use "async for" statement outside of ' + 'an\n' + '"async def" function.\n' + '\n' + '\n' + 'The "async with" statement\n' + '--------------------------\n' + '\n' + ' async_with_stmt ::= "async" with_stmt\n' + '\n' + 'An *asynchronous context manager* is a *context manager* that ' + 'is able\n' + 'to suspend execution in its *enter* and *exit* methods.\n' + '\n' + 'The following code:\n' + '\n' + ' async with EXPR as VAR:\n' + ' BLOCK\n' + '\n' + 'Is semantically equivalent to:\n' + '\n' + ' mgr = (EXPR)\n' + ' aexit = type(mgr).__aexit__\n' + ' aenter = type(mgr).__aenter__(mgr)\n' + ' exc = True\n' + '\n' + ' VAR = await aenter\n' + ' try:\n' + ' BLOCK\n' + ' except:\n' + ' if not await aexit(mgr, *sys.exc_info()):\n' + ' raise\n' + ' else:\n' + ' await aexit(mgr, None, None, None)\n' + '\n' + 'See also "__aenter__()" and "__aexit__()" for details.\n' + '\n' + 'It is a "SyntaxError" to use "async with" statement outside of ' + 'an\n' + '"async def" function.\n' + '\n' + 'See also: **PEP 492** - Coroutines with async and await ' + 'syntax\n' + '\n' + '-[ Footnotes ]-\n' + '\n' + '[1] The exception is propagated to the invocation stack ' + 'unless\n' + ' there is a "finally" clause which happens to raise ' + 'another\n' + ' exception. That new exception causes the old one to be ' + 'lost.\n' + '\n' + '[2] Currently, control "flows off the end" except in the case ' + 'of\n' + ' an exception or the execution of a "return", "continue", ' + 'or\n' + ' "break" statement.\n' + '\n' + '[3] A string literal appearing as the first statement in the\n' + " function body is transformed into the function's " + '"__doc__"\n' + " attribute and therefore the function's *docstring*.\n" + '\n' + '[4] A string literal appearing as the first statement in the ' + 'class\n' + ' body is transformed into the namespace\'s "__doc__" item ' + 'and\n' + " therefore the class's *docstring*.\n", + 'context-managers': '\n' + 'With Statement Context Managers\n' + '*******************************\n' + '\n' + 'A *context manager* is an object that defines the ' + 'runtime context to\n' + 'be established when executing a "with" statement. The ' + 'context manager\n' + 'handles the entry into, and the exit from, the desired ' + 'runtime context\n' + 'for the execution of the block of code. Context ' + 'managers are normally\n' + 'invoked using the "with" statement (described in ' + 'section *The with\n' + 'statement*), but can also be used by directly invoking ' + 'their methods.\n' + '\n' + 'Typical uses of context managers include saving and ' + 'restoring various\n' + 'kinds of global state, locking and unlocking ' + 'resources, closing opened\n' + 'files, etc.\n' + '\n' + 'For more information on context managers, see *Context ' + 'Manager Types*.\n' + '\n' + 'object.__enter__(self)\n' + '\n' + ' Enter the runtime context related to this object. ' + 'The "with"\n' + " statement will bind this method's return value to " + 'the target(s)\n' + ' specified in the "as" clause of the statement, if ' + 'any.\n' + '\n' + 'object.__exit__(self, exc_type, exc_value, traceback)\n' + '\n' + ' Exit the runtime context related to this object. ' + 'The parameters\n' + ' describe the exception that caused the context to ' + 'be exited. If the\n' + ' context was exited without an exception, all three ' + 'arguments will\n' + ' be "None".\n' + '\n' + ' If an exception is supplied, and the method wishes ' + 'to suppress the\n' + ' exception (i.e., prevent it from being propagated), ' + 'it should\n' + ' return a true value. Otherwise, the exception will ' + 'be processed\n' + ' normally upon exit from this method.\n' + '\n' + ' Note that "__exit__()" methods should not reraise ' + 'the passed-in\n' + " exception; this is the caller's responsibility.\n" + '\n' + 'See also: **PEP 0343** - The "with" statement\n' + '\n' + ' The specification, background, and examples for ' + 'the Python "with"\n' + ' statement.\n', + 'continue': '\n' + 'The "continue" statement\n' + '************************\n' + '\n' + ' continue_stmt ::= "continue"\n' + '\n' + '"continue" may only occur syntactically nested in a "for" or ' + '"while"\n' + 'loop, but not nested in a function or class definition or ' + '"finally"\n' + 'clause within that loop. It continues with the next cycle of ' + 'the\n' + 'nearest enclosing loop.\n' + '\n' + 'When "continue" passes control out of a "try" statement with ' + 'a\n' + '"finally" clause, that "finally" clause is executed before ' + 'really\n' + 'starting the next loop cycle.\n', + 'conversions': '\n' + 'Arithmetic conversions\n' + '**********************\n' + '\n' + 'When a description of an arithmetic operator below uses the ' + 'phrase\n' + '"the numeric arguments are converted to a common type," ' + 'this means\n' + 'that the operator implementation for built-in types works ' + 'as follows:\n' + '\n' + '* If either argument is a complex number, the other is ' + 'converted to\n' + ' complex;\n' + '\n' + '* otherwise, if either argument is a floating point number, ' + 'the\n' + ' other is converted to floating point;\n' + '\n' + '* otherwise, both must be integers and no conversion is ' + 'necessary.\n' + '\n' + 'Some additional rules apply for certain operators (e.g., a ' + 'string as a\n' + "left argument to the '%' operator). Extensions must define " + 'their own\n' + 'conversion behavior.\n', + 'customization': '\n' + 'Basic customization\n' + '*******************\n' + '\n' + 'object.__new__(cls[, ...])\n' + '\n' + ' Called to create a new instance of class *cls*. ' + '"__new__()" is a\n' + ' static method (special-cased so you need not declare ' + 'it as such)\n' + ' that takes the class of which an instance was ' + 'requested as its\n' + ' first argument. The remaining arguments are those ' + 'passed to the\n' + ' object constructor expression (the call to the ' + 'class). The return\n' + ' value of "__new__()" should be the new object instance ' + '(usually an\n' + ' instance of *cls*).\n' + '\n' + ' Typical implementations create a new instance of the ' + 'class by\n' + ' invoking the superclass\'s "__new__()" method using\n' + ' "super(currentclass, cls).__new__(cls[, ...])" with ' + 'appropriate\n' + ' arguments and then modifying the newly-created ' + 'instance as\n' + ' necessary before returning it.\n' + '\n' + ' If "__new__()" returns an instance of *cls*, then the ' + 'new\n' + ' instance\'s "__init__()" method will be invoked like\n' + ' "__init__(self[, ...])", where *self* is the new ' + 'instance and the\n' + ' remaining arguments are the same as were passed to ' + '"__new__()".\n' + '\n' + ' If "__new__()" does not return an instance of *cls*, ' + 'then the new\n' + ' instance\'s "__init__()" method will not be invoked.\n' + '\n' + ' "__new__()" is intended mainly to allow subclasses of ' + 'immutable\n' + ' types (like int, str, or tuple) to customize instance ' + 'creation. It\n' + ' is also commonly overridden in custom metaclasses in ' + 'order to\n' + ' customize class creation.\n' + '\n' + 'object.__init__(self[, ...])\n' + '\n' + ' Called after the instance has been created (by ' + '"__new__()"), but\n' + ' before it is returned to the caller. The arguments ' + 'are those\n' + ' passed to the class constructor expression. If a base ' + 'class has an\n' + ' "__init__()" method, the derived class\'s "__init__()" ' + 'method, if\n' + ' any, must explicitly call it to ensure proper ' + 'initialization of the\n' + ' base class part of the instance; for example:\n' + ' "BaseClass.__init__(self, [args...])".\n' + '\n' + ' Because "__new__()" and "__init__()" work together in ' + 'constructing\n' + ' objects ("__new__()" to create it, and "__init__()" to ' + 'customise\n' + ' it), no non-"None" value may be returned by ' + '"__init__()"; doing so\n' + ' will cause a "TypeError" to be raised at runtime.\n' + '\n' + 'object.__del__(self)\n' + '\n' + ' Called when the instance is about to be destroyed. ' + 'This is also\n' + ' called a destructor. If a base class has a ' + '"__del__()" method, the\n' + ' derived class\'s "__del__()" method, if any, must ' + 'explicitly call it\n' + ' to ensure proper deletion of the base class part of ' + 'the instance.\n' + ' Note that it is possible (though not recommended!) for ' + 'the\n' + ' "__del__()" method to postpone destruction of the ' + 'instance by\n' + ' creating a new reference to it. It may then be called ' + 'at a later\n' + ' time when this new reference is deleted. It is not ' + 'guaranteed that\n' + ' "__del__()" methods are called for objects that still ' + 'exist when\n' + ' the interpreter exits.\n' + '\n' + ' Note: "del x" doesn\'t directly call "x.__del__()" --- ' + 'the former\n' + ' decrements the reference count for "x" by one, and ' + 'the latter is\n' + ' only called when "x"\'s reference count reaches ' + 'zero. Some common\n' + ' situations that may prevent the reference count of ' + 'an object from\n' + ' going to zero include: circular references between ' + 'objects (e.g.,\n' + ' a doubly-linked list or a tree data structure with ' + 'parent and\n' + ' child pointers); a reference to the object on the ' + 'stack frame of\n' + ' a function that caught an exception (the traceback ' + 'stored in\n' + ' "sys.exc_info()[2]" keeps the stack frame alive); or ' + 'a reference\n' + ' to the object on the stack frame that raised an ' + 'unhandled\n' + ' exception in interactive mode (the traceback stored ' + 'in\n' + ' "sys.last_traceback" keeps the stack frame alive). ' + 'The first\n' + ' situation can only be remedied by explicitly ' + 'breaking the cycles;\n' + ' the second can be resolved by freeing the reference ' + 'to the\n' + ' traceback object when it is no longer useful, and ' + 'the third can\n' + ' be resolved by storing "None" in ' + '"sys.last_traceback". Circular\n' + ' references which are garbage are detected and ' + 'cleaned up when the\n' + " cyclic garbage collector is enabled (it's on by " + 'default). Refer\n' + ' to the documentation for the "gc" module for more ' + 'information\n' + ' about this topic.\n' + '\n' + ' Warning: Due to the precarious circumstances under ' + 'which\n' + ' "__del__()" methods are invoked, exceptions that ' + 'occur during\n' + ' their execution are ignored, and a warning is ' + 'printed to\n' + ' "sys.stderr" instead. Also, when "__del__()" is ' + 'invoked in\n' + ' response to a module being deleted (e.g., when ' + 'execution of the\n' + ' program is done), other globals referenced by the ' + '"__del__()"\n' + ' method may already have been deleted or in the ' + 'process of being\n' + ' torn down (e.g. the import machinery shutting ' + 'down). For this\n' + ' reason, "__del__()" methods should do the absolute ' + 'minimum needed\n' + ' to maintain external invariants. Starting with ' + 'version 1.5,\n' + ' Python guarantees that globals whose name begins ' + 'with a single\n' + ' underscore are deleted from their module before ' + 'other globals are\n' + ' deleted; if no other references to such globals ' + 'exist, this may\n' + ' help in assuring that imported modules are still ' + 'available at the\n' + ' time when the "__del__()" method is called.\n' + '\n' + 'object.__repr__(self)\n' + '\n' + ' Called by the "repr()" built-in function to compute ' + 'the "official"\n' + ' string representation of an object. If at all ' + 'possible, this\n' + ' should look like a valid Python expression that could ' + 'be used to\n' + ' recreate an object with the same value (given an ' + 'appropriate\n' + ' environment). If this is not possible, a string of ' + 'the form\n' + ' "<...some useful description...>" should be returned. ' + 'The return\n' + ' value must be a string object. If a class defines ' + '"__repr__()" but\n' + ' not "__str__()", then "__repr__()" is also used when ' + 'an "informal"\n' + ' string representation of instances of that class is ' + 'required.\n' + '\n' + ' This is typically used for debugging, so it is ' + 'important that the\n' + ' representation is information-rich and unambiguous.\n' + '\n' + 'object.__str__(self)\n' + '\n' + ' Called by "str(object)" and the built-in functions ' + '"format()" and\n' + ' "print()" to compute the "informal" or nicely ' + 'printable string\n' + ' representation of an object. The return value must be ' + 'a *string*\n' + ' object.\n' + '\n' + ' This method differs from "object.__repr__()" in that ' + 'there is no\n' + ' expectation that "__str__()" return a valid Python ' + 'expression: a\n' + ' more convenient or concise representation can be ' + 'used.\n' + '\n' + ' The default implementation defined by the built-in ' + 'type "object"\n' + ' calls "object.__repr__()".\n' + '\n' + 'object.__bytes__(self)\n' + '\n' + ' Called by "bytes()" to compute a byte-string ' + 'representation of an\n' + ' object. This should return a "bytes" object.\n' + '\n' + 'object.__format__(self, format_spec)\n' + '\n' + ' Called by the "format()" built-in function (and by ' + 'extension, the\n' + ' "str.format()" method of class "str") to produce a ' + '"formatted"\n' + ' string representation of an object. The "format_spec" ' + 'argument is a\n' + ' string that contains a description of the formatting ' + 'options\n' + ' desired. The interpretation of the "format_spec" ' + 'argument is up to\n' + ' the type implementing "__format__()", however most ' + 'classes will\n' + ' either delegate formatting to one of the built-in ' + 'types, or use a\n' + ' similar formatting option syntax.\n' + '\n' + ' See *Format Specification Mini-Language* for a ' + 'description of the\n' + ' standard formatting syntax.\n' + '\n' + ' The return value must be a string object.\n' + '\n' + ' Changed in version 3.4: The __format__ method of ' + '"object" itself\n' + ' raises a "TypeError" if passed any non-empty string.\n' + '\n' + 'object.__lt__(self, other)\n' + 'object.__le__(self, other)\n' + 'object.__eq__(self, other)\n' + 'object.__ne__(self, other)\n' + 'object.__gt__(self, other)\n' + 'object.__ge__(self, other)\n' + '\n' + ' These are the so-called "rich comparison" methods. ' + 'The\n' + ' correspondence between operator symbols and method ' + 'names is as\n' + ' follows: "xy" calls\n' + ' "x.__gt__(y)", and "x>=y" calls "x.__ge__(y)".\n' + '\n' + ' A rich comparison method may return the singleton ' + '"NotImplemented"\n' + ' if it does not implement the operation for a given ' + 'pair of\n' + ' arguments. By convention, "False" and "True" are ' + 'returned for a\n' + ' successful comparison. However, these methods can ' + 'return any value,\n' + ' so if the comparison operator is used in a Boolean ' + 'context (e.g.,\n' + ' in the condition of an "if" statement), Python will ' + 'call "bool()"\n' + ' on the value to determine if the result is true or ' + 'false.\n' + '\n' + ' By default, "__ne__()" delegates to "__eq__()" and ' + 'inverts the\n' + ' result unless it is "NotImplemented". There are no ' + 'other implied\n' + ' relationships among the comparison operators, for ' + 'example, the\n' + ' truth of "(x.__hash__".\n' + '\n' + ' If a class that does not override "__eq__()" wishes to ' + 'suppress\n' + ' hash support, it should include "__hash__ = None" in ' + 'the class\n' + ' definition. A class which defines its own "__hash__()" ' + 'that\n' + ' explicitly raises a "TypeError" would be incorrectly ' + 'identified as\n' + ' hashable by an "isinstance(obj, collections.Hashable)" ' + 'call.\n' + '\n' + ' Note: By default, the "__hash__()" values of str, ' + 'bytes and\n' + ' datetime objects are "salted" with an unpredictable ' + 'random value.\n' + ' Although they remain constant within an individual ' + 'Python\n' + ' process, they are not predictable between repeated ' + 'invocations of\n' + ' Python.This is intended to provide protection ' + 'against a denial-\n' + ' of-service caused by carefully-chosen inputs that ' + 'exploit the\n' + ' worst case performance of a dict insertion, O(n^2) ' + 'complexity.\n' + ' See ' + 'http://www.ocert.org/advisories/ocert-2011-003.html for\n' + ' details.Changing hash values affects the iteration ' + 'order of\n' + ' dicts, sets and other mappings. Python has never ' + 'made guarantees\n' + ' about this ordering (and it typically varies between ' + '32-bit and\n' + ' 64-bit builds).See also "PYTHONHASHSEED".\n' + '\n' + ' Changed in version 3.3: Hash randomization is enabled ' + 'by default.\n' + '\n' + 'object.__bool__(self)\n' + '\n' + ' Called to implement truth value testing and the ' + 'built-in operation\n' + ' "bool()"; should return "False" or "True". When this ' + 'method is not\n' + ' defined, "__len__()" is called, if it is defined, and ' + 'the object is\n' + ' considered true if its result is nonzero. If a class ' + 'defines\n' + ' neither "__len__()" nor "__bool__()", all its ' + 'instances are\n' + ' considered true.\n', + 'debugger': '\n' + '"pdb" --- The Python Debugger\n' + '*****************************\n' + '\n' + '**Source code:** Lib/pdb.py\n' + '\n' + '======================================================================\n' + '\n' + 'The module "pdb" defines an interactive source code debugger ' + 'for\n' + 'Python programs. It supports setting (conditional) ' + 'breakpoints and\n' + 'single stepping at the source line level, inspection of stack ' + 'frames,\n' + 'source code listing, and evaluation of arbitrary Python code ' + 'in the\n' + 'context of any stack frame. It also supports post-mortem ' + 'debugging\n' + 'and can be called under program control.\n' + '\n' + 'The debugger is extensible -- it is actually defined as the ' + 'class\n' + '"Pdb". This is currently undocumented but easily understood by ' + 'reading\n' + 'the source. The extension interface uses the modules "bdb" ' + 'and "cmd".\n' + '\n' + 'The debugger\'s prompt is "(Pdb)". Typical usage to run a ' + 'program under\n' + 'control of the debugger is:\n' + '\n' + ' >>> import pdb\n' + ' >>> import mymodule\n' + " >>> pdb.run('mymodule.test()')\n" + ' > (0)?()\n' + ' (Pdb) continue\n' + ' > (1)?()\n' + ' (Pdb) continue\n' + " NameError: 'spam'\n" + ' > (1)?()\n' + ' (Pdb)\n' + '\n' + 'Changed in version 3.3: Tab-completion via the "readline" ' + 'module is\n' + 'available for commands and command arguments, e.g. the current ' + 'global\n' + 'and local names are offered as arguments of the "p" command.\n' + '\n' + '"pdb.py" can also be invoked as a script to debug other ' + 'scripts. For\n' + 'example:\n' + '\n' + ' python3 -m pdb myscript.py\n' + '\n' + 'When invoked as a script, pdb will automatically enter ' + 'post-mortem\n' + 'debugging if the program being debugged exits abnormally. ' + 'After post-\n' + 'mortem debugging (or after normal exit of the program), pdb ' + 'will\n' + "restart the program. Automatic restarting preserves pdb's " + 'state (such\n' + 'as breakpoints) and in most cases is more useful than quitting ' + 'the\n' + "debugger upon program's exit.\n" + '\n' + 'New in version 3.2: "pdb.py" now accepts a "-c" option that ' + 'executes\n' + 'commands as if given in a ".pdbrc" file, see *Debugger ' + 'Commands*.\n' + '\n' + 'The typical usage to break into the debugger from a running ' + 'program is\n' + 'to insert\n' + '\n' + ' import pdb; pdb.set_trace()\n' + '\n' + 'at the location you want to break into the debugger. You can ' + 'then\n' + 'step through the code following this statement, and continue ' + 'running\n' + 'without the debugger using the "continue" command.\n' + '\n' + 'The typical usage to inspect a crashed program is:\n' + '\n' + ' >>> import pdb\n' + ' >>> import mymodule\n' + ' >>> mymodule.test()\n' + ' Traceback (most recent call last):\n' + ' File "", line 1, in ?\n' + ' File "./mymodule.py", line 4, in test\n' + ' test2()\n' + ' File "./mymodule.py", line 3, in test2\n' + ' print(spam)\n' + ' NameError: spam\n' + ' >>> pdb.pm()\n' + ' > ./mymodule.py(3)test2()\n' + ' -> print(spam)\n' + ' (Pdb)\n' + '\n' + 'The module defines the following functions; each enters the ' + 'debugger\n' + 'in a slightly different way:\n' + '\n' + 'pdb.run(statement, globals=None, locals=None)\n' + '\n' + ' Execute the *statement* (given as a string or a code ' + 'object) under\n' + ' debugger control. The debugger prompt appears before any ' + 'code is\n' + ' executed; you can set breakpoints and type "continue", or ' + 'you can\n' + ' step through the statement using "step" or "next" (all ' + 'these\n' + ' commands are explained below). The optional *globals* and ' + '*locals*\n' + ' arguments specify the environment in which the code is ' + 'executed; by\n' + ' default the dictionary of the module "__main__" is used. ' + '(See the\n' + ' explanation of the built-in "exec()" or "eval()" ' + 'functions.)\n' + '\n' + 'pdb.runeval(expression, globals=None, locals=None)\n' + '\n' + ' Evaluate the *expression* (given as a string or a code ' + 'object)\n' + ' under debugger control. When "runeval()" returns, it ' + 'returns the\n' + ' value of the expression. Otherwise this function is ' + 'similar to\n' + ' "run()".\n' + '\n' + 'pdb.runcall(function, *args, **kwds)\n' + '\n' + ' Call the *function* (a function or method object, not a ' + 'string)\n' + ' with the given arguments. When "runcall()" returns, it ' + 'returns\n' + ' whatever the function call returned. The debugger prompt ' + 'appears\n' + ' as soon as the function is entered.\n' + '\n' + 'pdb.set_trace()\n' + '\n' + ' Enter the debugger at the calling stack frame. This is ' + 'useful to\n' + ' hard-code a breakpoint at a given point in a program, even ' + 'if the\n' + ' code is not otherwise being debugged (e.g. when an ' + 'assertion\n' + ' fails).\n' + '\n' + 'pdb.post_mortem(traceback=None)\n' + '\n' + ' Enter post-mortem debugging of the given *traceback* ' + 'object. If no\n' + ' *traceback* is given, it uses the one of the exception that ' + 'is\n' + ' currently being handled (an exception must be being handled ' + 'if the\n' + ' default is to be used).\n' + '\n' + 'pdb.pm()\n' + '\n' + ' Enter post-mortem debugging of the traceback found in\n' + ' "sys.last_traceback".\n' + '\n' + 'The "run*" functions and "set_trace()" are aliases for ' + 'instantiating\n' + 'the "Pdb" class and calling the method of the same name. If ' + 'you want\n' + 'to access further features, you have to do this yourself:\n' + '\n' + "class class pdb.Pdb(completekey='tab', stdin=None, " + 'stdout=None, skip=None, nosigint=False)\n' + '\n' + ' "Pdb" is the debugger class.\n' + '\n' + ' The *completekey*, *stdin* and *stdout* arguments are ' + 'passed to the\n' + ' underlying "cmd.Cmd" class; see the description there.\n' + '\n' + ' The *skip* argument, if given, must be an iterable of ' + 'glob-style\n' + ' module name patterns. The debugger will not step into ' + 'frames that\n' + ' originate in a module that matches one of these patterns. ' + '[1]\n' + '\n' + ' By default, Pdb sets a handler for the SIGINT signal (which ' + 'is sent\n' + ' when the user presses Ctrl-C on the console) when you give ' + 'a\n' + ' "continue" command. This allows you to break into the ' + 'debugger\n' + ' again by pressing Ctrl-C. If you want Pdb not to touch the ' + 'SIGINT\n' + ' handler, set *nosigint* tot true.\n' + '\n' + ' Example call to enable tracing with *skip*:\n' + '\n' + " import pdb; pdb.Pdb(skip=['django.*']).set_trace()\n" + '\n' + ' New in version 3.1: The *skip* argument.\n' + '\n' + ' New in version 3.2: The *nosigint* argument. Previously, a ' + 'SIGINT\n' + ' handler was never set by Pdb.\n' + '\n' + ' run(statement, globals=None, locals=None)\n' + ' runeval(expression, globals=None, locals=None)\n' + ' runcall(function, *args, **kwds)\n' + ' set_trace()\n' + '\n' + ' See the documentation for the functions explained ' + 'above.\n' + '\n' + '\n' + 'Debugger Commands\n' + '=================\n' + '\n' + 'The commands recognized by the debugger are listed below. ' + 'Most\n' + 'commands can be abbreviated to one or two letters as ' + 'indicated; e.g.\n' + '"h(elp)" means that either "h" or "help" can be used to enter ' + 'the help\n' + 'command (but not "he" or "hel", nor "H" or "Help" or "HELP").\n' + 'Arguments to commands must be separated by whitespace (spaces ' + 'or\n' + 'tabs). Optional arguments are enclosed in square brackets ' + '("[]") in\n' + 'the command syntax; the square brackets must not be typed.\n' + 'Alternatives in the command syntax are separated by a vertical ' + 'bar\n' + '("|").\n' + '\n' + 'Entering a blank line repeats the last command entered. ' + 'Exception: if\n' + 'the last command was a "list" command, the next 11 lines are ' + 'listed.\n' + '\n' + "Commands that the debugger doesn't recognize are assumed to be " + 'Python\n' + 'statements and are executed in the context of the program ' + 'being\n' + 'debugged. Python statements can also be prefixed with an ' + 'exclamation\n' + 'point ("!"). This is a powerful way to inspect the program ' + 'being\n' + 'debugged; it is even possible to change a variable or call a ' + 'function.\n' + 'When an exception occurs in such a statement, the exception ' + 'name is\n' + "printed but the debugger's state is not changed.\n" + '\n' + 'The debugger supports *aliases*. Aliases can have parameters ' + 'which\n' + 'allows one a certain level of adaptability to the context ' + 'under\n' + 'examination.\n' + '\n' + 'Multiple commands may be entered on a single line, separated ' + 'by ";;".\n' + '(A single ";" is not used as it is the separator for multiple ' + 'commands\n' + 'in a line that is passed to the Python parser.) No ' + 'intelligence is\n' + 'applied to separating the commands; the input is split at the ' + 'first\n' + '";;" pair, even if it is in the middle of a quoted string.\n' + '\n' + 'If a file ".pdbrc" exists in the user\'s home directory or in ' + 'the\n' + 'current directory, it is read in and executed as if it had ' + 'been typed\n' + 'at the debugger prompt. This is particularly useful for ' + 'aliases. If\n' + 'both files exist, the one in the home directory is read first ' + 'and\n' + 'aliases defined there can be overridden by the local file.\n' + '\n' + 'Changed in version 3.2: ".pdbrc" can now contain commands ' + 'that\n' + 'continue debugging, such as "continue" or "next". Previously, ' + 'these\n' + 'commands had no effect.\n' + '\n' + 'h(elp) [command]\n' + '\n' + ' Without argument, print the list of available commands. ' + 'With a\n' + ' *command* as argument, print help about that command. ' + '"help pdb"\n' + ' displays the full documentation (the docstring of the ' + '"pdb"\n' + ' module). Since the *command* argument must be an ' + 'identifier, "help\n' + ' exec" must be entered to get help on the "!" command.\n' + '\n' + 'w(here)\n' + '\n' + ' Print a stack trace, with the most recent frame at the ' + 'bottom. An\n' + ' arrow indicates the current frame, which determines the ' + 'context of\n' + ' most commands.\n' + '\n' + 'd(own) [count]\n' + '\n' + ' Move the current frame *count* (default one) levels down in ' + 'the\n' + ' stack trace (to a newer frame).\n' + '\n' + 'u(p) [count]\n' + '\n' + ' Move the current frame *count* (default one) levels up in ' + 'the stack\n' + ' trace (to an older frame).\n' + '\n' + 'b(reak) [([filename:]lineno | function) [, condition]]\n' + '\n' + ' With a *lineno* argument, set a break there in the current ' + 'file.\n' + ' With a *function* argument, set a break at the first ' + 'executable\n' + ' statement within that function. The line number may be ' + 'prefixed\n' + ' with a filename and a colon, to specify a breakpoint in ' + 'another\n' + " file (probably one that hasn't been loaded yet). The file " + 'is\n' + ' searched on "sys.path". Note that each breakpoint is ' + 'assigned a\n' + ' number to which all the other breakpoint commands refer.\n' + '\n' + ' If a second argument is present, it is an expression which ' + 'must\n' + ' evaluate to true before the breakpoint is honored.\n' + '\n' + ' Without argument, list all breaks, including for each ' + 'breakpoint,\n' + ' the number of times that breakpoint has been hit, the ' + 'current\n' + ' ignore count, and the associated condition if any.\n' + '\n' + 'tbreak [([filename:]lineno | function) [, condition]]\n' + '\n' + ' Temporary breakpoint, which is removed automatically when ' + 'it is\n' + ' first hit. The arguments are the same as for "break".\n' + '\n' + 'cl(ear) [filename:lineno | bpnumber [bpnumber ...]]\n' + '\n' + ' With a *filename:lineno* argument, clear all the ' + 'breakpoints at\n' + ' this line. With a space separated list of breakpoint ' + 'numbers, clear\n' + ' those breakpoints. Without argument, clear all breaks (but ' + 'first\n' + ' ask confirmation).\n' + '\n' + 'disable [bpnumber [bpnumber ...]]\n' + '\n' + ' Disable the breakpoints given as a space separated list of\n' + ' breakpoint numbers. Disabling a breakpoint means it cannot ' + 'cause\n' + ' the program to stop execution, but unlike clearing a ' + 'breakpoint, it\n' + ' remains in the list of breakpoints and can be ' + '(re-)enabled.\n' + '\n' + 'enable [bpnumber [bpnumber ...]]\n' + '\n' + ' Enable the breakpoints specified.\n' + '\n' + 'ignore bpnumber [count]\n' + '\n' + ' Set the ignore count for the given breakpoint number. If ' + 'count is\n' + ' omitted, the ignore count is set to 0. A breakpoint ' + 'becomes active\n' + ' when the ignore count is zero. When non-zero, the count ' + 'is\n' + ' decremented each time the breakpoint is reached and the ' + 'breakpoint\n' + ' is not disabled and any associated condition evaluates to ' + 'true.\n' + '\n' + 'condition bpnumber [condition]\n' + '\n' + ' Set a new *condition* for the breakpoint, an expression ' + 'which must\n' + ' evaluate to true before the breakpoint is honored. If ' + '*condition*\n' + ' is absent, any existing condition is removed; i.e., the ' + 'breakpoint\n' + ' is made unconditional.\n' + '\n' + 'commands [bpnumber]\n' + '\n' + ' Specify a list of commands for breakpoint number ' + '*bpnumber*. The\n' + ' commands themselves appear on the following lines. Type a ' + 'line\n' + ' containing just "end" to terminate the commands. An ' + 'example:\n' + '\n' + ' (Pdb) commands 1\n' + ' (com) p some_variable\n' + ' (com) end\n' + ' (Pdb)\n' + '\n' + ' To remove all commands from a breakpoint, type commands and ' + 'follow\n' + ' it immediately with "end"; that is, give no commands.\n' + '\n' + ' With no *bpnumber* argument, commands refers to the last ' + 'breakpoint\n' + ' set.\n' + '\n' + ' You can use breakpoint commands to start your program up ' + 'again.\n' + ' Simply use the continue command, or step, or any other ' + 'command that\n' + ' resumes execution.\n' + '\n' + ' Specifying any command resuming execution (currently ' + 'continue,\n' + ' step, next, return, jump, quit and their abbreviations) ' + 'terminates\n' + ' the command list (as if that command was immediately ' + 'followed by\n' + ' end). This is because any time you resume execution (even ' + 'with a\n' + ' simple next or step), you may encounter another ' + 'breakpoint--which\n' + ' could have its own command list, leading to ambiguities ' + 'about which\n' + ' list to execute.\n' + '\n' + " If you use the 'silent' command in the command list, the " + 'usual\n' + ' message about stopping at a breakpoint is not printed. ' + 'This may be\n' + ' desirable for breakpoints that are to print a specific ' + 'message and\n' + ' then continue. If none of the other commands print ' + 'anything, you\n' + ' see no sign that the breakpoint was reached.\n' + '\n' + 's(tep)\n' + '\n' + ' Execute the current line, stop at the first possible ' + 'occasion\n' + ' (either in a function that is called or on the next line in ' + 'the\n' + ' current function).\n' + '\n' + 'n(ext)\n' + '\n' + ' Continue execution until the next line in the current ' + 'function is\n' + ' reached or it returns. (The difference between "next" and ' + '"step"\n' + ' is that "step" stops inside a called function, while ' + '"next"\n' + ' executes called functions at (nearly) full speed, only ' + 'stopping at\n' + ' the next line in the current function.)\n' + '\n' + 'unt(il) [lineno]\n' + '\n' + ' Without argument, continue execution until the line with a ' + 'number\n' + ' greater than the current one is reached.\n' + '\n' + ' With a line number, continue execution until a line with a ' + 'number\n' + ' greater or equal to that is reached. In both cases, also ' + 'stop when\n' + ' the current frame returns.\n' + '\n' + ' Changed in version 3.2: Allow giving an explicit line ' + 'number.\n' + '\n' + 'r(eturn)\n' + '\n' + ' Continue execution until the current function returns.\n' + '\n' + 'c(ont(inue))\n' + '\n' + ' Continue execution, only stop when a breakpoint is ' + 'encountered.\n' + '\n' + 'j(ump) lineno\n' + '\n' + ' Set the next line that will be executed. Only available in ' + 'the\n' + ' bottom-most frame. This lets you jump back and execute ' + 'code again,\n' + " or jump forward to skip code that you don't want to run.\n" + '\n' + ' It should be noted that not all jumps are allowed -- for ' + 'instance\n' + ' it is not possible to jump into the middle of a "for" loop ' + 'or out\n' + ' of a "finally" clause.\n' + '\n' + 'l(ist) [first[, last]]\n' + '\n' + ' List source code for the current file. Without arguments, ' + 'list 11\n' + ' lines around the current line or continue the previous ' + 'listing.\n' + ' With "." as argument, list 11 lines around the current ' + 'line. With\n' + ' one argument, list 11 lines around at that line. With two\n' + ' arguments, list the given range; if the second argument is ' + 'less\n' + ' than the first, it is interpreted as a count.\n' + '\n' + ' The current line in the current frame is indicated by ' + '"->". If an\n' + ' exception is being debugged, the line where the exception ' + 'was\n' + ' originally raised or propagated is indicated by ">>", if it ' + 'differs\n' + ' from the current line.\n' + '\n' + ' New in version 3.2: The ">>" marker.\n' + '\n' + 'll | longlist\n' + '\n' + ' List all source code for the current function or frame.\n' + ' Interesting lines are marked as for "list".\n' + '\n' + ' New in version 3.2.\n' + '\n' + 'a(rgs)\n' + '\n' + ' Print the argument list of the current function.\n' + '\n' + 'p expression\n' + '\n' + ' Evaluate the *expression* in the current context and print ' + 'its\n' + ' value.\n' + '\n' + ' Note: "print()" can also be used, but is not a debugger ' + 'command\n' + ' --- this executes the Python "print()" function.\n' + '\n' + 'pp expression\n' + '\n' + ' Like the "p" command, except the value of the expression is ' + 'pretty-\n' + ' printed using the "pprint" module.\n' + '\n' + 'whatis expression\n' + '\n' + ' Print the type of the *expression*.\n' + '\n' + 'source expression\n' + '\n' + ' Try to get source code for the given object and display ' + 'it.\n' + '\n' + ' New in version 3.2.\n' + '\n' + 'display [expression]\n' + '\n' + ' Display the value of the expression if it changed, each ' + 'time\n' + ' execution stops in the current frame.\n' + '\n' + ' Without expression, list all display expressions for the ' + 'current\n' + ' frame.\n' + '\n' + ' New in version 3.2.\n' + '\n' + 'undisplay [expression]\n' + '\n' + ' Do not display the expression any more in the current ' + 'frame.\n' + ' Without expression, clear all display expressions for the ' + 'current\n' + ' frame.\n' + '\n' + ' New in version 3.2.\n' + '\n' + 'interact\n' + '\n' + ' Start an interative interpreter (using the "code" module) ' + 'whose\n' + ' global namespace contains all the (global and local) names ' + 'found in\n' + ' the current scope.\n' + '\n' + ' New in version 3.2.\n' + '\n' + 'alias [name [command]]\n' + '\n' + ' Create an alias called *name* that executes *command*. The ' + 'command\n' + ' must *not* be enclosed in quotes. Replaceable parameters ' + 'can be\n' + ' indicated by "%1", "%2", and so on, while "%*" is replaced ' + 'by all\n' + ' the parameters. If no command is given, the current alias ' + 'for\n' + ' *name* is shown. If no arguments are given, all aliases are ' + 'listed.\n' + '\n' + ' Aliases may be nested and can contain anything that can be ' + 'legally\n' + ' typed at the pdb prompt. Note that internal pdb commands ' + '*can* be\n' + ' overridden by aliases. Such a command is then hidden until ' + 'the\n' + ' alias is removed. Aliasing is recursively applied to the ' + 'first\n' + ' word of the command line; all other words in the line are ' + 'left\n' + ' alone.\n' + '\n' + ' As an example, here are two useful aliases (especially when ' + 'placed\n' + ' in the ".pdbrc" file):\n' + '\n' + ' # Print instance variables (usage "pi classInst")\n' + ' alias pi for k in %1.__dict__.keys(): ' + 'print("%1.",k,"=",%1.__dict__[k])\n' + ' # Print instance variables in self\n' + ' alias ps pi self\n' + '\n' + 'unalias name\n' + '\n' + ' Delete the specified alias.\n' + '\n' + '! statement\n' + '\n' + ' Execute the (one-line) *statement* in the context of the ' + 'current\n' + ' stack frame. The exclamation point can be omitted unless ' + 'the first\n' + ' word of the statement resembles a debugger command. To set ' + 'a\n' + ' global variable, you can prefix the assignment command with ' + 'a\n' + ' "global" statement on the same line, e.g.:\n' + '\n' + " (Pdb) global list_options; list_options = ['-l']\n" + ' (Pdb)\n' + '\n' + 'run [args ...]\n' + 'restart [args ...]\n' + '\n' + ' Restart the debugged Python program. If an argument is ' + 'supplied,\n' + ' it is split with "shlex" and the result is used as the new\n' + ' "sys.argv". History, breakpoints, actions and debugger ' + 'options are\n' + ' preserved. "restart" is an alias for "run".\n' + '\n' + 'q(uit)\n' + '\n' + ' Quit from the debugger. The program being executed is ' + 'aborted.\n' + '\n' + '-[ Footnotes ]-\n' + '\n' + '[1] Whether a frame is considered to originate in a certain ' + 'module\n' + ' is determined by the "__name__" in the frame globals.\n', + 'del': '\n' + 'The "del" statement\n' + '*******************\n' + '\n' + ' del_stmt ::= "del" target_list\n' + '\n' + 'Deletion is recursively defined very similar to the way assignment ' + 'is\n' + 'defined. Rather than spelling it out in full details, here are ' + 'some\n' + 'hints.\n' + '\n' + 'Deletion of a target list recursively deletes each target, from ' + 'left\n' + 'to right.\n' + '\n' + 'Deletion of a name removes the binding of that name from the local ' + 'or\n' + 'global namespace, depending on whether the name occurs in a ' + '"global"\n' + 'statement in the same code block. If the name is unbound, a\n' + '"NameError" exception will be raised.\n' + '\n' + 'Deletion of attribute references, subscriptions and slicings is ' + 'passed\n' + 'to the primary object involved; deletion of a slicing is in ' + 'general\n' + 'equivalent to assignment of an empty slice of the right type (but ' + 'even\n' + 'this is determined by the sliced object).\n' + '\n' + 'Changed in version 3.2: Previously it was illegal to delete a name\n' + 'from the local namespace if it occurs as a free variable in a ' + 'nested\n' + 'block.\n', + 'dict': '\n' + 'Dictionary displays\n' + '*******************\n' + '\n' + 'A dictionary display is a possibly empty series of key/datum ' + 'pairs\n' + 'enclosed in curly braces:\n' + '\n' + ' dict_display ::= "{" [key_datum_list | ' + 'dict_comprehension] "}"\n' + ' key_datum_list ::= key_datum ("," key_datum)* [","]\n' + ' key_datum ::= expression ":" expression\n' + ' dict_comprehension ::= expression ":" expression comp_for\n' + '\n' + 'A dictionary display yields a new dictionary object.\n' + '\n' + 'If a comma-separated sequence of key/datum pairs is given, they ' + 'are\n' + 'evaluated from left to right to define the entries of the ' + 'dictionary:\n' + 'each key object is used as a key into the dictionary to store the\n' + 'corresponding datum. This means that you can specify the same ' + 'key\n' + "multiple times in the key/datum list, and the final dictionary's " + 'value\n' + 'for that key will be the last one given.\n' + '\n' + 'A dict comprehension, in contrast to list and set comprehensions,\n' + 'needs two expressions separated with a colon followed by the ' + 'usual\n' + '"for" and "if" clauses. When the comprehension is run, the ' + 'resulting\n' + 'key and value elements are inserted in the new dictionary in the ' + 'order\n' + 'they are produced.\n' + '\n' + 'Restrictions on the types of the key values are listed earlier in\n' + 'section *The standard type hierarchy*. (To summarize, the key ' + 'type\n' + 'should be *hashable*, which excludes all mutable objects.) ' + 'Clashes\n' + 'between duplicate keys are not detected; the last datum ' + '(textually\n' + 'rightmost in the display) stored for a given key value prevails.\n', + 'dynamic-features': '\n' + 'Interaction with dynamic features\n' + '*********************************\n' + '\n' + 'Name resolution of free variables occurs at runtime, ' + 'not at compile\n' + 'time. This means that the following code will print ' + '42:\n' + '\n' + ' i = 10\n' + ' def f():\n' + ' print(i)\n' + ' i = 42\n' + ' f()\n' + '\n' + 'There are several cases where Python statements are ' + 'illegal when used\n' + 'in conjunction with nested scopes that contain free ' + 'variables.\n' + '\n' + 'If a variable is referenced in an enclosing scope, it ' + 'is illegal to\n' + 'delete the name. An error will be reported at compile ' + 'time.\n' + '\n' + 'The "eval()" and "exec()" functions do not have access ' + 'to the full\n' + 'environment for resolving names. Names may be ' + 'resolved in the local\n' + 'and global namespaces of the caller. Free variables ' + 'are not resolved\n' + 'in the nearest enclosing namespace, but in the global ' + 'namespace. [1]\n' + 'The "exec()" and "eval()" functions have optional ' + 'arguments to\n' + 'override the global and local namespace. If only one ' + 'namespace is\n' + 'specified, it is used for both.\n', + 'else': '\n' + 'The "if" statement\n' + '******************\n' + '\n' + 'The "if" statement is used for conditional execution:\n' + '\n' + ' if_stmt ::= "if" expression ":" suite\n' + ' ( "elif" expression ":" suite )*\n' + ' ["else" ":" suite]\n' + '\n' + 'It selects exactly one of the suites by evaluating the expressions ' + 'one\n' + 'by one until one is found to be true (see section *Boolean ' + 'operations*\n' + 'for the definition of true and false); then that suite is ' + 'executed\n' + '(and no other part of the "if" statement is executed or ' + 'evaluated).\n' + 'If all expressions are false, the suite of the "else" clause, if\n' + 'present, is executed.\n', + 'exceptions': '\n' + 'Exceptions\n' + '**********\n' + '\n' + 'Exceptions are a means of breaking out of the normal flow of ' + 'control\n' + 'of a code block in order to handle errors or other ' + 'exceptional\n' + 'conditions. An exception is *raised* at the point where the ' + 'error is\n' + 'detected; it may be *handled* by the surrounding code block ' + 'or by any\n' + 'code block that directly or indirectly invoked the code ' + 'block where\n' + 'the error occurred.\n' + '\n' + 'The Python interpreter raises an exception when it detects a ' + 'run-time\n' + 'error (such as division by zero). A Python program can ' + 'also\n' + 'explicitly raise an exception with the "raise" statement. ' + 'Exception\n' + 'handlers are specified with the "try" ... "except" ' + 'statement. The\n' + '"finally" clause of such a statement can be used to specify ' + 'cleanup\n' + 'code which does not handle the exception, but is executed ' + 'whether an\n' + 'exception occurred or not in the preceding code.\n' + '\n' + 'Python uses the "termination" model of error handling: an ' + 'exception\n' + 'handler can find out what happened and continue execution at ' + 'an outer\n' + 'level, but it cannot repair the cause of the error and retry ' + 'the\n' + 'failing operation (except by re-entering the offending piece ' + 'of code\n' + 'from the top).\n' + '\n' + 'When an exception is not handled at all, the interpreter ' + 'terminates\n' + 'execution of the program, or returns to its interactive main ' + 'loop. In\n' + 'either case, it prints a stack backtrace, except when the ' + 'exception is\n' + '"SystemExit".\n' + '\n' + 'Exceptions are identified by class instances. The "except" ' + 'clause is\n' + 'selected depending on the class of the instance: it must ' + 'reference the\n' + 'class of the instance or a base class thereof. The instance ' + 'can be\n' + 'received by the handler and can carry additional information ' + 'about the\n' + 'exceptional condition.\n' + '\n' + 'Note: Exception messages are not part of the Python API. ' + 'Their\n' + ' contents may change from one version of Python to the next ' + 'without\n' + ' warning and should not be relied on by code which will run ' + 'under\n' + ' multiple versions of the interpreter.\n' + '\n' + 'See also the description of the "try" statement in section ' + '*The try\n' + 'statement* and "raise" statement in section *The raise ' + 'statement*.\n' + '\n' + '-[ Footnotes ]-\n' + '\n' + '[1] This limitation occurs because the code that is executed ' + 'by\n' + ' these operations is not available at the time the module ' + 'is\n' + ' compiled.\n', + 'execmodel': '\n' + 'Execution model\n' + '***************\n' + '\n' + '\n' + 'Structure of a programm\n' + '=======================\n' + '\n' + 'A Python program is constructed from code blocks. A *block* ' + 'is a piece\n' + 'of Python program text that is executed as a unit. The ' + 'following are\n' + 'blocks: a module, a function body, and a class definition. ' + 'Each\n' + 'command typed interactively is a block. A script file (a ' + 'file given\n' + 'as standard input to the interpreter or specified as a ' + 'command line\n' + 'argument to the interpreter) is a code block. A script ' + 'command (a\n' + 'command specified on the interpreter command line with the ' + "'**-c**'\n" + 'option) is a code block. The string argument passed to the ' + 'built-in\n' + 'functions "eval()" and "exec()" is a code block.\n' + '\n' + 'A code block is executed in an *execution frame*. A frame ' + 'contains\n' + 'some administrative information (used for debugging) and ' + 'determines\n' + "where and how execution continues after the code block's " + 'execution has\n' + 'completed.\n' + '\n' + '\n' + 'Naming and binding\n' + '==================\n' + '\n' + '\n' + 'Binding of names\n' + '----------------\n' + '\n' + '*Names* refer to objects. Names are introduced by name ' + 'binding\n' + 'operations.\n' + '\n' + 'The following constructs bind names: formal parameters to ' + 'functions,\n' + '"import" statements, class and function definitions (these ' + 'bind the\n' + 'class or function name in the defining block), and targets ' + 'that are\n' + 'identifiers if occurring in an assignment, "for" loop header, ' + 'or after\n' + '"as" in a "with" statement or "except" clause. The "import" ' + 'statement\n' + 'of the form "from ... import *" binds all names defined in ' + 'the\n' + 'imported module, except those beginning with an underscore. ' + 'This form\n' + 'may only be used at the module level.\n' + '\n' + 'A target occurring in a "del" statement is also considered ' + 'bound for\n' + 'this purpose (though the actual semantics are to unbind the ' + 'name).\n' + '\n' + 'Each assignment or import statement occurs within a block ' + 'defined by a\n' + 'class or function definition or at the module level (the ' + 'top-level\n' + 'code block).\n' + '\n' + 'If a name is bound in a block, it is a local variable of that ' + 'block,\n' + 'unless declared as "nonlocal" or "global". If a name is ' + 'bound at the\n' + 'module level, it is a global variable. (The variables of the ' + 'module\n' + 'code block are local and global.) If a variable is used in a ' + 'code\n' + 'block but not defined there, it is a *free variable*.\n' + '\n' + 'Each occurrence of a name in the program text refers to the ' + '*binding*\n' + 'of that name established by the following name resolution ' + 'rules.\n' + '\n' + '\n' + 'Resolution of names\n' + '-------------------\n' + '\n' + 'A *scope* defines the visibility of a name within a block. ' + 'If a local\n' + 'variable is defined in a block, its scope includes that ' + 'block. If the\n' + 'definition occurs in a function block, the scope extends to ' + 'any blocks\n' + 'contained within the defining one, unless a contained block ' + 'introduces\n' + 'a different binding for the name.\n' + '\n' + 'When a name is used in a code block, it is resolved using the ' + 'nearest\n' + 'enclosing scope. The set of all such scopes visible to a ' + 'code block\n' + "is called the block's *environment*.\n" + '\n' + 'When a name is not found at all, a "NameError" exception is ' + 'raised. If\n' + 'the current scope is a function scope, and the name refers to ' + 'a local\n' + 'variable that has not yet been bound to a value at the point ' + 'where the\n' + 'name is used, an "UnboundLocalError" exception is raised.\n' + '"UnboundLocalError" is a subclass of "NameError".\n' + '\n' + 'If a name binding operation occurs anywhere within a code ' + 'block, all\n' + 'uses of the name within the block are treated as references ' + 'to the\n' + 'current block. This can lead to errors when a name is used ' + 'within a\n' + 'block before it is bound. This rule is subtle. Python ' + 'lacks\n' + 'declarations and allows name binding operations to occur ' + 'anywhere\n' + 'within a code block. The local variables of a code block can ' + 'be\n' + 'determined by scanning the entire text of the block for name ' + 'binding\n' + 'operations.\n' + '\n' + 'If the "global" statement occurs within a block, all uses of ' + 'the name\n' + 'specified in the statement refer to the binding of that name ' + 'in the\n' + 'top-level namespace. Names are resolved in the top-level ' + 'namespace by\n' + 'searching the global namespace, i.e. the namespace of the ' + 'module\n' + 'containing the code block, and the builtins namespace, the ' + 'namespace\n' + 'of the module "builtins". The global namespace is searched ' + 'first. If\n' + 'the name is not found there, the builtins namespace is ' + 'searched. The\n' + '"global" statement must precede all uses of the name.\n' + '\n' + 'The "global" statement has the same scope as a name binding ' + 'operation\n' + 'in the same block. If the nearest enclosing scope for a free ' + 'variable\n' + 'contains a global statement, the free variable is treated as ' + 'a global.\n' + '\n' + 'The "nonlocal" statement causes corresponding names to refer ' + 'to\n' + 'previously bound variables in the nearest enclosing function ' + 'scope.\n' + '"SyntaxError" is raised at compile time if the given name ' + 'does not\n' + 'exist in any enclosing function scope.\n' + '\n' + 'The namespace for a module is automatically created the first ' + 'time a\n' + 'module is imported. The main module for a script is always ' + 'called\n' + '"__main__".\n' + '\n' + 'Class definition blocks and arguments to "exec()" and ' + '"eval()" are\n' + 'special in the context of name resolution. A class definition ' + 'is an\n' + 'executable statement that may use and define names. These ' + 'references\n' + 'follow the normal rules for name resolution with an exception ' + 'that\n' + 'unbound local variables are looked up in the global ' + 'namespace. The\n' + 'namespace of the class definition becomes the attribute ' + 'dictionary of\n' + 'the class. The scope of names defined in a class block is ' + 'limited to\n' + 'the class block; it does not extend to the code blocks of ' + 'methods --\n' + 'this includes comprehensions and generator expressions since ' + 'they are\n' + 'implemented using a function scope. This means that the ' + 'following\n' + 'will fail:\n' + '\n' + ' class A:\n' + ' a = 42\n' + ' b = list(a + i for i in range(10))\n' + '\n' + '\n' + 'Builtins and restricted execution\n' + '---------------------------------\n' + '\n' + 'The builtins namespace associated with the execution of a ' + 'code block\n' + 'is actually found by looking up the name "__builtins__" in ' + 'its global\n' + 'namespace; this should be a dictionary or a module (in the ' + 'latter case\n' + "the module's dictionary is used). By default, when in the " + '"__main__"\n' + 'module, "__builtins__" is the built-in module "builtins"; ' + 'when in any\n' + 'other module, "__builtins__" is an alias for the dictionary ' + 'of the\n' + '"builtins" module itself. "__builtins__" can be set to a ' + 'user-created\n' + 'dictionary to create a weak form of restricted execution.\n' + '\n' + '**CPython implementation detail:** Users should not touch\n' + '"__builtins__"; it is strictly an implementation detail. ' + 'Users\n' + 'wanting to override values in the builtins namespace should ' + '"import"\n' + 'the "builtins" module and modify its attributes ' + 'appropriately.\n' + '\n' + '\n' + 'Interaction with dynamic features\n' + '---------------------------------\n' + '\n' + 'Name resolution of free variables occurs at runtime, not at ' + 'compile\n' + 'time. This means that the following code will print 42:\n' + '\n' + ' i = 10\n' + ' def f():\n' + ' print(i)\n' + ' i = 42\n' + ' f()\n' + '\n' + 'There are several cases where Python statements are illegal ' + 'when used\n' + 'in conjunction with nested scopes that contain free ' + 'variables.\n' + '\n' + 'If a variable is referenced in an enclosing scope, it is ' + 'illegal to\n' + 'delete the name. An error will be reported at compile time.\n' + '\n' + 'The "eval()" and "exec()" functions do not have access to the ' + 'full\n' + 'environment for resolving names. Names may be resolved in ' + 'the local\n' + 'and global namespaces of the caller. Free variables are not ' + 'resolved\n' + 'in the nearest enclosing namespace, but in the global ' + 'namespace. [1]\n' + 'The "exec()" and "eval()" functions have optional arguments ' + 'to\n' + 'override the global and local namespace. If only one ' + 'namespace is\n' + 'specified, it is used for both.\n' + '\n' + '\n' + 'Exceptions\n' + '==========\n' + '\n' + 'Exceptions are a means of breaking out of the normal flow of ' + 'control\n' + 'of a code block in order to handle errors or other ' + 'exceptional\n' + 'conditions. An exception is *raised* at the point where the ' + 'error is\n' + 'detected; it may be *handled* by the surrounding code block ' + 'or by any\n' + 'code block that directly or indirectly invoked the code block ' + 'where\n' + 'the error occurred.\n' + '\n' + 'The Python interpreter raises an exception when it detects a ' + 'run-time\n' + 'error (such as division by zero). A Python program can also\n' + 'explicitly raise an exception with the "raise" statement. ' + 'Exception\n' + 'handlers are specified with the "try" ... "except" ' + 'statement. The\n' + '"finally" clause of such a statement can be used to specify ' + 'cleanup\n' + 'code which does not handle the exception, but is executed ' + 'whether an\n' + 'exception occurred or not in the preceding code.\n' + '\n' + 'Python uses the "termination" model of error handling: an ' + 'exception\n' + 'handler can find out what happened and continue execution at ' + 'an outer\n' + 'level, but it cannot repair the cause of the error and retry ' + 'the\n' + 'failing operation (except by re-entering the offending piece ' + 'of code\n' + 'from the top).\n' + '\n' + 'When an exception is not handled at all, the interpreter ' + 'terminates\n' + 'execution of the program, or returns to its interactive main ' + 'loop. In\n' + 'either case, it prints a stack backtrace, except when the ' + 'exception is\n' + '"SystemExit".\n' + '\n' + 'Exceptions are identified by class instances. The "except" ' + 'clause is\n' + 'selected depending on the class of the instance: it must ' + 'reference the\n' + 'class of the instance or a base class thereof. The instance ' + 'can be\n' + 'received by the handler and can carry additional information ' + 'about the\n' + 'exceptional condition.\n' + '\n' + 'Note: Exception messages are not part of the Python API. ' + 'Their\n' + ' contents may change from one version of Python to the next ' + 'without\n' + ' warning and should not be relied on by code which will run ' + 'under\n' + ' multiple versions of the interpreter.\n' + '\n' + 'See also the description of the "try" statement in section ' + '*The try\n' + 'statement* and "raise" statement in section *The raise ' + 'statement*.\n' + '\n' + '-[ Footnotes ]-\n' + '\n' + '[1] This limitation occurs because the code that is executed ' + 'by\n' + ' these operations is not available at the time the module ' + 'is\n' + ' compiled.\n', + 'exprlists': '\n' + 'Expression lists\n' + '****************\n' + '\n' + ' expression_list ::= expression ( "," expression )* [","]\n' + '\n' + 'An expression list containing at least one comma yields a ' + 'tuple. The\n' + 'length of the tuple is the number of expressions in the ' + 'list. The\n' + 'expressions are evaluated from left to right.\n' + '\n' + 'The trailing comma is required only to create a single tuple ' + '(a.k.a. a\n' + '*singleton*); it is optional in all other cases. A single ' + 'expression\n' + "without a trailing comma doesn't create a tuple, but rather " + 'yields the\n' + 'value of that expression. (To create an empty tuple, use an ' + 'empty pair\n' + 'of parentheses: "()".)\n', + 'floating': '\n' + 'Floating point literals\n' + '***********************\n' + '\n' + 'Floating point literals are described by the following ' + 'lexical\n' + 'definitions:\n' + '\n' + ' floatnumber ::= pointfloat | exponentfloat\n' + ' pointfloat ::= [intpart] fraction | intpart "."\n' + ' exponentfloat ::= (intpart | pointfloat) exponent\n' + ' intpart ::= digit+\n' + ' fraction ::= "." digit+\n' + ' exponent ::= ("e" | "E") ["+" | "-"] digit+\n' + '\n' + 'Note that the integer and exponent parts are always ' + 'interpreted using\n' + 'radix 10. For example, "077e010" is legal, and denotes the ' + 'same number\n' + 'as "77e10". The allowed range of floating point literals is\n' + 'implementation-dependent. Some examples of floating point ' + 'literals:\n' + '\n' + ' 3.14 10. .001 1e100 3.14e-10 0e0\n' + '\n' + 'Note that numeric literals do not include a sign; a phrase ' + 'like "-1"\n' + 'is actually an expression composed of the unary operator "-" ' + 'and the\n' + 'literal "1".\n', + 'for': '\n' + 'The "for" statement\n' + '*******************\n' + '\n' + 'The "for" statement is used to iterate over the elements of a ' + 'sequence\n' + '(such as a string, tuple or list) or other iterable object:\n' + '\n' + ' for_stmt ::= "for" target_list "in" expression_list ":" suite\n' + ' ["else" ":" suite]\n' + '\n' + 'The expression list is evaluated once; it should yield an iterable\n' + 'object. An iterator is created for the result of the\n' + '"expression_list". The suite is then executed once for each item\n' + 'provided by the iterator, in the order returned by the iterator. ' + 'Each\n' + 'item in turn is assigned to the target list using the standard ' + 'rules\n' + 'for assignments (see *Assignment statements*), and then the suite ' + 'is\n' + 'executed. When the items are exhausted (which is immediately when ' + 'the\n' + 'sequence is empty or an iterator raises a "StopIteration" ' + 'exception),\n' + 'the suite in the "else" clause, if present, is executed, and the ' + 'loop\n' + 'terminates.\n' + '\n' + 'A "break" statement executed in the first suite terminates the ' + 'loop\n' + 'without executing the "else" clause\'s suite. A "continue" ' + 'statement\n' + 'executed in the first suite skips the rest of the suite and ' + 'continues\n' + 'with the next item, or with the "else" clause if there is no next\n' + 'item.\n' + '\n' + 'The for-loop makes assignments to the variables(s) in the target ' + 'list.\n' + 'This overwrites all previous assignments to those variables ' + 'including\n' + 'those made in the suite of the for-loop:\n' + '\n' + ' for i in range(10):\n' + ' print(i)\n' + ' i = 5 # this will not affect the for-loop\n' + ' # because i will be overwritten with the ' + 'next\n' + ' # index in the range\n' + '\n' + 'Names in the target list are not deleted when the loop is ' + 'finished,\n' + 'but if the sequence is empty, they will not have been assigned to ' + 'at\n' + 'all by the loop. Hint: the built-in function "range()" returns an\n' + "iterator of integers suitable to emulate the effect of Pascal's " + '"for i\n' + ':= a to b do"; e.g., "list(range(3))" returns the list "[0, 1, ' + '2]".\n' + '\n' + 'Note: There is a subtlety when the sequence is being modified by ' + 'the\n' + ' loop (this can only occur for mutable sequences, i.e. lists). ' + 'An\n' + ' internal counter is used to keep track of which item is used ' + 'next,\n' + ' and this is incremented on each iteration. When this counter ' + 'has\n' + ' reached the length of the sequence the loop terminates. This ' + 'means\n' + ' that if the suite deletes the current (or a previous) item from ' + 'the\n' + ' sequence, the next item will be skipped (since it gets the index ' + 'of\n' + ' the current item which has already been treated). Likewise, if ' + 'the\n' + ' suite inserts an item in the sequence before the current item, ' + 'the\n' + ' current item will be treated again the next time through the ' + 'loop.\n' + ' This can lead to nasty bugs that can be avoided by making a\n' + ' temporary copy using a slice of the whole sequence, e.g.,\n' + '\n' + ' for x in a[:]:\n' + ' if x < 0: a.remove(x)\n', + 'formatstrings': '\n' + 'Format String Syntax\n' + '********************\n' + '\n' + 'The "str.format()" method and the "Formatter" class share ' + 'the same\n' + 'syntax for format strings (although in the case of ' + '"Formatter",\n' + 'subclasses can define their own format string syntax).\n' + '\n' + 'Format strings contain "replacement fields" surrounded by ' + 'curly braces\n' + '"{}". Anything that is not contained in braces is ' + 'considered literal\n' + 'text, which is copied unchanged to the output. If you ' + 'need to include\n' + 'a brace character in the literal text, it can be escaped ' + 'by doubling:\n' + '"{{" and "}}".\n' + '\n' + 'The grammar for a replacement field is as follows:\n' + '\n' + ' replacement_field ::= "{" [field_name] ["!" ' + 'conversion] [":" format_spec] "}"\n' + ' field_name ::= arg_name ("." attribute_name ' + '| "[" element_index "]")*\n' + ' arg_name ::= [identifier | integer]\n' + ' attribute_name ::= identifier\n' + ' element_index ::= integer | index_string\n' + ' index_string ::= +\n' + ' conversion ::= "r" | "s" | "a"\n' + ' format_spec ::= \n' + '\n' + 'In less formal terms, the replacement field can start ' + 'with a\n' + '*field_name* that specifies the object whose value is to ' + 'be formatted\n' + 'and inserted into the output instead of the replacement ' + 'field. The\n' + '*field_name* is optionally followed by a *conversion* ' + 'field, which is\n' + 'preceded by an exclamation point "\'!\'", and a ' + '*format_spec*, which is\n' + 'preceded by a colon "\':\'". These specify a non-default ' + 'format for the\n' + 'replacement value.\n' + '\n' + 'See also the *Format Specification Mini-Language* ' + 'section.\n' + '\n' + 'The *field_name* itself begins with an *arg_name* that is ' + 'either a\n' + "number or a keyword. If it's a number, it refers to a " + 'positional\n' + "argument, and if it's a keyword, it refers to a named " + 'keyword\n' + 'argument. If the numerical arg_names in a format string ' + 'are 0, 1, 2,\n' + '... in sequence, they can all be omitted (not just some) ' + 'and the\n' + 'numbers 0, 1, 2, ... will be automatically inserted in ' + 'that order.\n' + 'Because *arg_name* is not quote-delimited, it is not ' + 'possible to\n' + 'specify arbitrary dictionary keys (e.g., the strings ' + '"\'10\'" or\n' + '"\':-]\'") within a format string. The *arg_name* can be ' + 'followed by any\n' + 'number of index or attribute expressions. An expression ' + 'of the form\n' + '"\'.name\'" selects the named attribute using ' + '"getattr()", while an\n' + 'expression of the form "\'[index]\'" does an index lookup ' + 'using\n' + '"__getitem__()".\n' + '\n' + 'Changed in version 3.1: The positional argument ' + 'specifiers can be\n' + 'omitted, so "\'{} {}\'" is equivalent to "\'{0} {1}\'".\n' + '\n' + 'Some simple format string examples:\n' + '\n' + ' "First, thou shalt count to {0}" # References first ' + 'positional argument\n' + ' "Bring me a {}" # Implicitly ' + 'references the first positional argument\n' + ' "From {} to {}" # Same as "From {0} ' + 'to {1}"\n' + ' "My quest is {name}" # References keyword ' + "argument 'name'\n" + ' "Weight in tons {0.weight}" # \'weight\' ' + 'attribute of first positional arg\n' + ' "Units destroyed: {players[0]}" # First element of ' + "keyword argument 'players'.\n" + '\n' + 'The *conversion* field causes a type coercion before ' + 'formatting.\n' + 'Normally, the job of formatting a value is done by the ' + '"__format__()"\n' + 'method of the value itself. However, in some cases it is ' + 'desirable to\n' + 'force a type to be formatted as a string, overriding its ' + 'own\n' + 'definition of formatting. By converting the value to a ' + 'string before\n' + 'calling "__format__()", the normal formatting logic is ' + 'bypassed.\n' + '\n' + 'Three conversion flags are currently supported: "\'!s\'" ' + 'which calls\n' + '"str()" on the value, "\'!r\'" which calls "repr()" and ' + '"\'!a\'" which\n' + 'calls "ascii()".\n' + '\n' + 'Some examples:\n' + '\n' + ' "Harold\'s a clever {0!s}" # Calls str() on the ' + 'argument first\n' + ' "Bring out the holy {name!r}" # Calls repr() on the ' + 'argument first\n' + ' "More {!a}" # Calls ascii() on ' + 'the argument first\n' + '\n' + 'The *format_spec* field contains a specification of how ' + 'the value\n' + 'should be presented, including such details as field ' + 'width, alignment,\n' + 'padding, decimal precision and so on. Each value type ' + 'can define its\n' + 'own "formatting mini-language" or interpretation of the ' + '*format_spec*.\n' + '\n' + 'Most built-in types support a common formatting ' + 'mini-language, which\n' + 'is described in the next section.\n' + '\n' + 'A *format_spec* field can also include nested replacement ' + 'fields\n' + 'within it. These nested replacement fields can contain ' + 'only a field\n' + 'name; conversion flags and format specifications are not ' + 'allowed. The\n' + 'replacement fields within the format_spec are substituted ' + 'before the\n' + '*format_spec* string is interpreted. This allows the ' + 'formatting of a\n' + 'value to be dynamically specified.\n' + '\n' + 'See the *Format examples* section for some examples.\n' + '\n' + '\n' + 'Format Specification Mini-Language\n' + '==================================\n' + '\n' + '"Format specifications" are used within replacement ' + 'fields contained\n' + 'within a format string to define how individual values ' + 'are presented\n' + '(see *Format String Syntax*). They can also be passed ' + 'directly to the\n' + 'built-in "format()" function. Each formattable type may ' + 'define how\n' + 'the format specification is to be interpreted.\n' + '\n' + 'Most built-in types implement the following options for ' + 'format\n' + 'specifications, although some of the formatting options ' + 'are only\n' + 'supported by the numeric types.\n' + '\n' + 'A general convention is that an empty format string ' + '("""") produces\n' + 'the same result as if you had called "str()" on the ' + 'value. A non-empty\n' + 'format string typically modifies the result.\n' + '\n' + 'The general form of a *standard format specifier* is:\n' + '\n' + ' format_spec ::= ' + '[[fill]align][sign][#][0][width][,][.precision][type]\n' + ' fill ::= \n' + ' align ::= "<" | ">" | "=" | "^"\n' + ' sign ::= "+" | "-" | " "\n' + ' width ::= integer\n' + ' precision ::= integer\n' + ' type ::= "b" | "c" | "d" | "e" | "E" | "f" | ' + '"F" | "g" | "G" | "n" | "o" | "s" | "x" | "X" | "%"\n' + '\n' + 'If a valid *align* value is specified, it can be preceded ' + 'by a *fill*\n' + 'character that can be any character and defaults to a ' + 'space if\n' + 'omitted. Note that it is not possible to use "{" and "}" ' + 'as *fill*\n' + 'char while using the "str.format()" method; this ' + 'limitation however\n' + 'doesn\'t affect the "format()" function.\n' + '\n' + 'The meaning of the various alignment options is as ' + 'follows:\n' + '\n' + ' ' + '+-----------+------------------------------------------------------------+\n' + ' | Option | ' + 'Meaning ' + '|\n' + ' ' + '+===========+============================================================+\n' + ' | "\'<\'" | Forces the field to be left-aligned ' + 'within the available |\n' + ' | | space (this is the default for most ' + 'objects). |\n' + ' ' + '+-----------+------------------------------------------------------------+\n' + ' | "\'>\'" | Forces the field to be right-aligned ' + 'within the available |\n' + ' | | space (this is the default for ' + 'numbers). |\n' + ' ' + '+-----------+------------------------------------------------------------+\n' + ' | "\'=\'" | Forces the padding to be placed after ' + 'the sign (if any) |\n' + ' | | but before the digits. This is used for ' + 'printing fields |\n' + " | | in the form '+000000120'. This alignment " + 'option is only |\n' + ' | | valid for numeric ' + 'types. |\n' + ' ' + '+-----------+------------------------------------------------------------+\n' + ' | "\'^\'" | Forces the field to be centered within ' + 'the available |\n' + ' | | ' + 'space. ' + '|\n' + ' ' + '+-----------+------------------------------------------------------------+\n' + '\n' + 'Note that unless a minimum field width is defined, the ' + 'field width\n' + 'will always be the same size as the data to fill it, so ' + 'that the\n' + 'alignment option has no meaning in this case.\n' + '\n' + 'The *sign* option is only valid for number types, and can ' + 'be one of\n' + 'the following:\n' + '\n' + ' ' + '+-----------+------------------------------------------------------------+\n' + ' | Option | ' + 'Meaning ' + '|\n' + ' ' + '+===========+============================================================+\n' + ' | "\'+\'" | indicates that a sign should be used ' + 'for both positive as |\n' + ' | | well as negative ' + 'numbers. |\n' + ' ' + '+-----------+------------------------------------------------------------+\n' + ' | "\'-\'" | indicates that a sign should be used ' + 'only for negative |\n' + ' | | numbers (this is the default ' + 'behavior). |\n' + ' ' + '+-----------+------------------------------------------------------------+\n' + ' | space | indicates that a leading space should be ' + 'used on positive |\n' + ' | | numbers, and a minus sign on negative ' + 'numbers. |\n' + ' ' + '+-----------+------------------------------------------------------------+\n' + '\n' + 'The "\'#\'" option causes the "alternate form" to be used ' + 'for the\n' + 'conversion. The alternate form is defined differently ' + 'for different\n' + 'types. This option is only valid for integer, float, ' + 'complex and\n' + 'Decimal types. For integers, when binary, octal, or ' + 'hexadecimal output\n' + 'is used, this option adds the prefix respective "\'0b\'", ' + '"\'0o\'", or\n' + '"\'0x\'" to the output value. For floats, complex and ' + 'Decimal the\n' + 'alternate form causes the result of the conversion to ' + 'always contain a\n' + 'decimal-point character, even if no digits follow it. ' + 'Normally, a\n' + 'decimal-point character appears in the result of these ' + 'conversions\n' + 'only if a digit follows it. In addition, for "\'g\'" and ' + '"\'G\'"\n' + 'conversions, trailing zeros are not removed from the ' + 'result.\n' + '\n' + 'The "\',\'" option signals the use of a comma for a ' + 'thousands separator.\n' + 'For a locale aware separator, use the "\'n\'" integer ' + 'presentation type\n' + 'instead.\n' + '\n' + 'Changed in version 3.1: Added the "\',\'" option (see ' + 'also **PEP 378**).\n' + '\n' + '*width* is a decimal integer defining the minimum field ' + 'width. If not\n' + 'specified, then the field width will be determined by the ' + 'content.\n' + '\n' + 'Preceding the *width* field by a zero ("\'0\'") character ' + 'enables sign-\n' + 'aware zero-padding for numeric types. This is equivalent ' + 'to a *fill*\n' + 'character of "\'0\'" with an *alignment* type of ' + '"\'=\'".\n' + '\n' + 'The *precision* is a decimal number indicating how many ' + 'digits should\n' + 'be displayed after the decimal point for a floating point ' + 'value\n' + 'formatted with "\'f\'" and "\'F\'", or before and after ' + 'the decimal point\n' + 'for a floating point value formatted with "\'g\'" or ' + '"\'G\'". For non-\n' + 'number types the field indicates the maximum field size - ' + 'in other\n' + 'words, how many characters will be used from the field ' + 'content. The\n' + '*precision* is not allowed for integer values.\n' + '\n' + 'Finally, the *type* determines how the data should be ' + 'presented.\n' + '\n' + 'The available string presentation types are:\n' + '\n' + ' ' + '+-----------+------------------------------------------------------------+\n' + ' | Type | ' + 'Meaning ' + '|\n' + ' ' + '+===========+============================================================+\n' + ' | "\'s\'" | String format. This is the default ' + 'type for strings and |\n' + ' | | may be ' + 'omitted. |\n' + ' ' + '+-----------+------------------------------------------------------------+\n' + ' | None | The same as ' + '"\'s\'". |\n' + ' ' + '+-----------+------------------------------------------------------------+\n' + '\n' + 'The available integer presentation types are:\n' + '\n' + ' ' + '+-----------+------------------------------------------------------------+\n' + ' | Type | ' + 'Meaning ' + '|\n' + ' ' + '+===========+============================================================+\n' + ' | "\'b\'" | Binary format. Outputs the number in ' + 'base 2. |\n' + ' ' + '+-----------+------------------------------------------------------------+\n' + ' | "\'c\'" | Character. Converts the integer to the ' + 'corresponding |\n' + ' | | unicode character before ' + 'printing. |\n' + ' ' + '+-----------+------------------------------------------------------------+\n' + ' | "\'d\'" | Decimal Integer. Outputs the number in ' + 'base 10. |\n' + ' ' + '+-----------+------------------------------------------------------------+\n' + ' | "\'o\'" | Octal format. Outputs the number in ' + 'base 8. |\n' + ' ' + '+-----------+------------------------------------------------------------+\n' + ' | "\'x\'" | Hex format. Outputs the number in base ' + '16, using lower- |\n' + ' | | case letters for the digits above ' + '9. |\n' + ' ' + '+-----------+------------------------------------------------------------+\n' + ' | "\'X\'" | Hex format. Outputs the number in base ' + '16, using upper- |\n' + ' | | case letters for the digits above ' + '9. |\n' + ' ' + '+-----------+------------------------------------------------------------+\n' + ' | "\'n\'" | Number. This is the same as "\'d\'", ' + 'except that it uses the |\n' + ' | | current locale setting to insert the ' + 'appropriate number |\n' + ' | | separator ' + 'characters. |\n' + ' ' + '+-----------+------------------------------------------------------------+\n' + ' | None | The same as ' + '"\'d\'". |\n' + ' ' + '+-----------+------------------------------------------------------------+\n' + '\n' + 'In addition to the above presentation types, integers can ' + 'be formatted\n' + 'with the floating point presentation types listed below ' + '(except "\'n\'"\n' + 'and None). When doing so, "float()" is used to convert ' + 'the integer to\n' + 'a floating point number before formatting.\n' + '\n' + 'The available presentation types for floating point and ' + 'decimal values\n' + 'are:\n' + '\n' + ' ' + '+-----------+------------------------------------------------------------+\n' + ' | Type | ' + 'Meaning ' + '|\n' + ' ' + '+===========+============================================================+\n' + ' | "\'e\'" | Exponent notation. Prints the number ' + 'in scientific |\n' + " | | notation using the letter 'e' to " + 'indicate the exponent. |\n' + ' | | The default precision is ' + '"6". |\n' + ' ' + '+-----------+------------------------------------------------------------+\n' + ' | "\'E\'" | Exponent notation. Same as "\'e\'" ' + 'except it uses an upper |\n' + " | | case 'E' as the separator " + 'character. |\n' + ' ' + '+-----------+------------------------------------------------------------+\n' + ' | "\'f\'" | Fixed point. Displays the number as a ' + 'fixed-point number. |\n' + ' | | The default precision is ' + '"6". |\n' + ' ' + '+-----------+------------------------------------------------------------+\n' + ' | "\'F\'" | Fixed point. Same as "\'f\'", but ' + 'converts "nan" to "NAN" |\n' + ' | | and "inf" to ' + '"INF". |\n' + ' ' + '+-----------+------------------------------------------------------------+\n' + ' | "\'g\'" | General format. For a given precision ' + '"p >= 1", this |\n' + ' | | rounds the number to "p" significant ' + 'digits and then |\n' + ' | | formats the result in either fixed-point ' + 'format or in |\n' + ' | | scientific notation, depending on its ' + 'magnitude. The |\n' + ' | | precise rules are as follows: suppose ' + 'that the result |\n' + ' | | formatted with presentation type "\'e\'" ' + 'and precision "p-1" |\n' + ' | | would have exponent "exp". Then if "-4 ' + '<= exp < p", the |\n' + ' | | number is formatted with presentation ' + 'type "\'f\'" and |\n' + ' | | precision "p-1-exp". Otherwise, the ' + 'number is formatted |\n' + ' | | with presentation type "\'e\'" and ' + 'precision "p-1". In both |\n' + ' | | cases insignificant trailing zeros are ' + 'removed from the |\n' + ' | | significand, and the decimal point is ' + 'also removed if |\n' + ' | | there are no remaining digits following ' + 'it. Positive and |\n' + ' | | negative infinity, positive and negative ' + 'zero, and nans, |\n' + ' | | are formatted as "inf", "-inf", "0", ' + '"-0" and "nan" |\n' + ' | | respectively, regardless of the ' + 'precision. A precision of |\n' + ' | | "0" is treated as equivalent to a ' + 'precision of "1". The |\n' + ' | | default precision is ' + '"6". |\n' + ' ' + '+-----------+------------------------------------------------------------+\n' + ' | "\'G\'" | General format. Same as "\'g\'" except ' + 'switches to "\'E\'" if |\n' + ' | | the number gets too large. The ' + 'representations of infinity |\n' + ' | | and NaN are uppercased, ' + 'too. |\n' + ' ' + '+-----------+------------------------------------------------------------+\n' + ' | "\'n\'" | Number. This is the same as "\'g\'", ' + 'except that it uses the |\n' + ' | | current locale setting to insert the ' + 'appropriate number |\n' + ' | | separator ' + 'characters. |\n' + ' ' + '+-----------+------------------------------------------------------------+\n' + ' | "\'%\'" | Percentage. Multiplies the number by ' + '100 and displays in |\n' + ' | | fixed ("\'f\'") format, followed by a ' + 'percent sign. |\n' + ' ' + '+-----------+------------------------------------------------------------+\n' + ' | None | Similar to "\'g\'", except that ' + 'fixed-point notation, when |\n' + ' | | used, has at least one digit past the ' + 'decimal point. The |\n' + ' | | default precision is as high as needed ' + 'to represent the |\n' + ' | | particular value. The overall effect is ' + 'to match the |\n' + ' | | output of "str()" as altered by the ' + 'other format |\n' + ' | | ' + 'modifiers. ' + '|\n' + ' ' + '+-----------+------------------------------------------------------------+\n' + '\n' + '\n' + 'Format examples\n' + '===============\n' + '\n' + 'This section contains examples of the new format syntax ' + 'and comparison\n' + 'with the old "%"-formatting.\n' + '\n' + 'In most of the cases the syntax is similar to the old ' + '"%"-formatting,\n' + 'with the addition of the "{}" and with ":" used instead ' + 'of "%". For\n' + 'example, "\'%03.2f\'" can be translated to ' + '"\'{:03.2f}\'".\n' + '\n' + 'The new format syntax also supports new and different ' + 'options, shown\n' + 'in the follow examples.\n' + '\n' + 'Accessing arguments by position:\n' + '\n' + " >>> '{0}, {1}, {2}'.format('a', 'b', 'c')\n" + " 'a, b, c'\n" + " >>> '{}, {}, {}'.format('a', 'b', 'c') # 3.1+ only\n" + " 'a, b, c'\n" + " >>> '{2}, {1}, {0}'.format('a', 'b', 'c')\n" + " 'c, b, a'\n" + " >>> '{2}, {1}, {0}'.format(*'abc') # unpacking " + 'argument sequence\n' + " 'c, b, a'\n" + " >>> '{0}{1}{0}'.format('abra', 'cad') # arguments' " + 'indices can be repeated\n' + " 'abracadabra'\n" + '\n' + 'Accessing arguments by name:\n' + '\n' + " >>> 'Coordinates: {latitude}, " + "{longitude}'.format(latitude='37.24N', " + "longitude='-115.81W')\n" + " 'Coordinates: 37.24N, -115.81W'\n" + " >>> coord = {'latitude': '37.24N', 'longitude': " + "'-115.81W'}\n" + " >>> 'Coordinates: {latitude}, " + "{longitude}'.format(**coord)\n" + " 'Coordinates: 37.24N, -115.81W'\n" + '\n' + "Accessing arguments' attributes:\n" + '\n' + ' >>> c = 3-5j\n' + " >>> ('The complex number {0} is formed from the real " + "part {0.real} '\n" + " ... 'and the imaginary part {0.imag}.').format(c)\n" + " 'The complex number (3-5j) is formed from the real " + "part 3.0 and the imaginary part -5.0.'\n" + ' >>> class Point:\n' + ' ... def __init__(self, x, y):\n' + ' ... self.x, self.y = x, y\n' + ' ... def __str__(self):\n' + " ... return 'Point({self.x}, " + "{self.y})'.format(self=self)\n" + ' ...\n' + ' >>> str(Point(4, 2))\n' + " 'Point(4, 2)'\n" + '\n' + "Accessing arguments' items:\n" + '\n' + ' >>> coord = (3, 5)\n' + " >>> 'X: {0[0]}; Y: {0[1]}'.format(coord)\n" + " 'X: 3; Y: 5'\n" + '\n' + 'Replacing "%s" and "%r":\n' + '\n' + ' >>> "repr() shows quotes: {!r}; str() doesn\'t: ' + '{!s}".format(\'test1\', \'test2\')\n' + ' "repr() shows quotes: \'test1\'; str() doesn\'t: ' + 'test2"\n' + '\n' + 'Aligning the text and specifying a width:\n' + '\n' + " >>> '{:<30}'.format('left aligned')\n" + " 'left aligned '\n" + " >>> '{:>30}'.format('right aligned')\n" + " ' right aligned'\n" + " >>> '{:^30}'.format('centered')\n" + " ' centered '\n" + " >>> '{:*^30}'.format('centered') # use '*' as a fill " + 'char\n' + " '***********centered***********'\n" + '\n' + 'Replacing "%+f", "%-f", and "% f" and specifying a sign:\n' + '\n' + " >>> '{:+f}; {:+f}'.format(3.14, -3.14) # show it " + 'always\n' + " '+3.140000; -3.140000'\n" + " >>> '{: f}; {: f}'.format(3.14, -3.14) # show a space " + 'for positive numbers\n' + " ' 3.140000; -3.140000'\n" + " >>> '{:-f}; {:-f}'.format(3.14, -3.14) # show only " + "the minus -- same as '{:f}; {:f}'\n" + " '3.140000; -3.140000'\n" + '\n' + 'Replacing "%x" and "%o" and converting the value to ' + 'different bases:\n' + '\n' + ' >>> # format also supports binary numbers\n' + ' >>> "int: {0:d}; hex: {0:x}; oct: {0:o}; bin: ' + '{0:b}".format(42)\n' + " 'int: 42; hex: 2a; oct: 52; bin: 101010'\n" + ' >>> # with 0x, 0o, or 0b as prefix:\n' + ' >>> "int: {0:d}; hex: {0:#x}; oct: {0:#o}; bin: ' + '{0:#b}".format(42)\n' + " 'int: 42; hex: 0x2a; oct: 0o52; bin: 0b101010'\n" + '\n' + 'Using the comma as a thousands separator:\n' + '\n' + " >>> '{:,}'.format(1234567890)\n" + " '1,234,567,890'\n" + '\n' + 'Expressing a percentage:\n' + '\n' + ' >>> points = 19\n' + ' >>> total = 22\n' + " >>> 'Correct answers: {:.2%}'.format(points/total)\n" + " 'Correct answers: 86.36%'\n" + '\n' + 'Using type-specific formatting:\n' + '\n' + ' >>> import datetime\n' + ' >>> d = datetime.datetime(2010, 7, 4, 12, 15, 58)\n' + " >>> '{:%Y-%m-%d %H:%M:%S}'.format(d)\n" + " '2010-07-04 12:15:58'\n" + '\n' + 'Nesting arguments and more complex examples:\n' + '\n' + " >>> for align, text in zip('<^>', ['left', 'center', " + "'right']):\n" + " ... '{0:{fill}{align}16}'.format(text, fill=align, " + 'align=align)\n' + ' ...\n' + " 'left<<<<<<<<<<<<'\n" + " '^^^^^center^^^^^'\n" + " '>>>>>>>>>>>right'\n" + ' >>>\n' + ' >>> octets = [192, 168, 0, 1]\n' + " >>> '{:02X}{:02X}{:02X}{:02X}'.format(*octets)\n" + " 'C0A80001'\n" + ' >>> int(_, 16)\n' + ' 3232235521\n' + ' >>>\n' + ' >>> width = 5\n' + ' >>> for num in range(5,12): #doctest: ' + '+NORMALIZE_WHITESPACE\n' + " ... for base in 'dXob':\n" + " ... print('{0:{width}{base}}'.format(num, " + "base=base, width=width), end=' ')\n" + ' ... print()\n' + ' ...\n' + ' 5 5 5 101\n' + ' 6 6 6 110\n' + ' 7 7 7 111\n' + ' 8 8 10 1000\n' + ' 9 9 11 1001\n' + ' 10 A 12 1010\n' + ' 11 B 13 1011\n', + 'function': '\n' + 'Function definitions\n' + '********************\n' + '\n' + 'A function definition defines a user-defined function object ' + '(see\n' + 'section *The standard type hierarchy*):\n' + '\n' + ' funcdef ::= [decorators] "def" funcname "(" ' + '[parameter_list] ")" ["->" expression] ":" suite\n' + ' decorators ::= decorator+\n' + ' decorator ::= "@" dotted_name ["(" [parameter_list ' + '[","]] ")"] NEWLINE\n' + ' dotted_name ::= identifier ("." identifier)*\n' + ' parameter_list ::= (defparameter ",")*\n' + ' | "*" [parameter] ("," defparameter)* ' + '["," "**" parameter]\n' + ' | "**" parameter\n' + ' | defparameter [","] )\n' + ' parameter ::= identifier [":" expression]\n' + ' defparameter ::= parameter ["=" expression]\n' + ' funcname ::= identifier\n' + '\n' + 'A function definition is an executable statement. Its ' + 'execution binds\n' + 'the function name in the current local namespace to a function ' + 'object\n' + '(a wrapper around the executable code for the function). ' + 'This\n' + 'function object contains a reference to the current global ' + 'namespace\n' + 'as the global namespace to be used when the function is ' + 'called.\n' + '\n' + 'The function definition does not execute the function body; ' + 'this gets\n' + 'executed only when the function is called. [3]\n' + '\n' + 'A function definition may be wrapped by one or more ' + '*decorator*\n' + 'expressions. Decorator expressions are evaluated when the ' + 'function is\n' + 'defined, in the scope that contains the function definition. ' + 'The\n' + 'result must be a callable, which is invoked with the function ' + 'object\n' + 'as the only argument. The returned value is bound to the ' + 'function name\n' + 'instead of the function object. Multiple decorators are ' + 'applied in\n' + 'nested fashion. For example, the following code\n' + '\n' + ' @f1(arg)\n' + ' @f2\n' + ' def func(): pass\n' + '\n' + 'is equivalent to\n' + '\n' + ' def func(): pass\n' + ' func = f1(arg)(f2(func))\n' + '\n' + 'When one or more *parameters* have the form *parameter* "="\n' + '*expression*, the function is said to have "default parameter ' + 'values."\n' + 'For a parameter with a default value, the corresponding ' + '*argument* may\n' + "be omitted from a call, in which case the parameter's default " + 'value is\n' + 'substituted. If a parameter has a default value, all ' + 'following\n' + 'parameters up until the ""*"" must also have a default value ' + '--- this\n' + 'is a syntactic restriction that is not expressed by the ' + 'grammar.\n' + '\n' + '**Default parameter values are evaluated from left to right ' + 'when the\n' + 'function definition is executed.** This means that the ' + 'expression is\n' + 'evaluated once, when the function is defined, and that the ' + 'same "pre-\n' + 'computed" value is used for each call. This is especially ' + 'important\n' + 'to understand when a default parameter is a mutable object, ' + 'such as a\n' + 'list or a dictionary: if the function modifies the object ' + '(e.g. by\n' + 'appending an item to a list), the default value is in effect ' + 'modified.\n' + 'This is generally not what was intended. A way around this is ' + 'to use\n' + '"None" as the default, and explicitly test for it in the body ' + 'of the\n' + 'function, e.g.:\n' + '\n' + ' def whats_on_the_telly(penguin=None):\n' + ' if penguin is None:\n' + ' penguin = []\n' + ' penguin.append("property of the zoo")\n' + ' return penguin\n' + '\n' + 'Function call semantics are described in more detail in ' + 'section\n' + '*Calls*. A function call always assigns values to all ' + 'parameters\n' + 'mentioned in the parameter list, either from position ' + 'arguments, from\n' + 'keyword arguments, or from default values. If the form\n' + '""*identifier"" is present, it is initialized to a tuple ' + 'receiving any\n' + 'excess positional parameters, defaulting to the empty tuple. ' + 'If the\n' + 'form ""**identifier"" is present, it is initialized to a new\n' + 'dictionary receiving any excess keyword arguments, defaulting ' + 'to a new\n' + 'empty dictionary. Parameters after ""*"" or ""*identifier"" ' + 'are\n' + 'keyword-only parameters and may only be passed used keyword ' + 'arguments.\n' + '\n' + 'Parameters may have annotations of the form "": expression"" ' + 'following\n' + 'the parameter name. Any parameter may have an annotation even ' + 'those\n' + 'of the form "*identifier" or "**identifier". Functions may ' + 'have\n' + '"return" annotation of the form ""-> expression"" after the ' + 'parameter\n' + 'list. These annotations can be any valid Python expression ' + 'and are\n' + 'evaluated when the function definition is executed. ' + 'Annotations may\n' + 'be evaluated in a different order than they appear in the ' + 'source code.\n' + 'The presence of annotations does not change the semantics of ' + 'a\n' + 'function. The annotation values are available as values of a\n' + "dictionary keyed by the parameters' names in the " + '"__annotations__"\n' + 'attribute of the function object.\n' + '\n' + 'It is also possible to create anonymous functions (functions ' + 'not bound\n' + 'to a name), for immediate use in expressions. This uses ' + 'lambda\n' + 'expressions, described in section *Lambdas*. Note that the ' + 'lambda\n' + 'expression is merely a shorthand for a simplified function ' + 'definition;\n' + 'a function defined in a ""def"" statement can be passed around ' + 'or\n' + 'assigned to another name just like a function defined by a ' + 'lambda\n' + 'expression. The ""def"" form is actually more powerful since ' + 'it\n' + 'allows the execution of multiple statements and annotations.\n' + '\n' + "**Programmer's note:** Functions are first-class objects. A " + '""def""\n' + 'statement executed inside a function definition defines a ' + 'local\n' + 'function that can be returned or passed around. Free ' + 'variables used\n' + 'in the nested function can access the local variables of the ' + 'function\n' + 'containing the def. See section *Naming and binding* for ' + 'details.\n' + '\n' + 'See also: **PEP 3107** - Function Annotations\n' + '\n' + ' The original specification for function annotations.\n', + 'global': '\n' + 'The "global" statement\n' + '**********************\n' + '\n' + ' global_stmt ::= "global" identifier ("," identifier)*\n' + '\n' + 'The "global" statement is a declaration which holds for the ' + 'entire\n' + 'current code block. It means that the listed identifiers are to ' + 'be\n' + 'interpreted as globals. It would be impossible to assign to a ' + 'global\n' + 'variable without "global", although free variables may refer to\n' + 'globals without being declared global.\n' + '\n' + 'Names listed in a "global" statement must not be used in the ' + 'same code\n' + 'block textually preceding that "global" statement.\n' + '\n' + 'Names listed in a "global" statement must not be defined as ' + 'formal\n' + 'parameters or in a "for" loop control target, "class" ' + 'definition,\n' + 'function definition, or "import" statement.\n' + '\n' + '**CPython implementation detail:** The current implementation ' + 'does not\n' + 'enforce the two restrictions, but programs should not abuse ' + 'this\n' + 'freedom, as future implementations may enforce them or silently ' + 'change\n' + 'the meaning of the program.\n' + '\n' + '**Programmer\'s note:** the "global" is a directive to the ' + 'parser. It\n' + 'applies only to code parsed at the same time as the "global"\n' + 'statement. In particular, a "global" statement contained in a ' + 'string\n' + 'or code object supplied to the built-in "exec()" function does ' + 'not\n' + 'affect the code block *containing* the function call, and code\n' + 'contained in such a string is unaffected by "global" statements ' + 'in the\n' + 'code containing the function call. The same applies to the ' + '"eval()"\n' + 'and "compile()" functions.\n', + 'id-classes': '\n' + 'Reserved classes of identifiers\n' + '*******************************\n' + '\n' + 'Certain classes of identifiers (besides keywords) have ' + 'special\n' + 'meanings. These classes are identified by the patterns of ' + 'leading and\n' + 'trailing underscore characters:\n' + '\n' + '"_*"\n' + ' Not imported by "from module import *". The special ' + 'identifier "_"\n' + ' is used in the interactive interpreter to store the ' + 'result of the\n' + ' last evaluation; it is stored in the "builtins" module. ' + 'When not\n' + ' in interactive mode, "_" has no special meaning and is ' + 'not defined.\n' + ' See section *The import statement*.\n' + '\n' + ' Note: The name "_" is often used in conjunction with\n' + ' internationalization; refer to the documentation for ' + 'the\n' + ' "gettext" module for more information on this ' + 'convention.\n' + '\n' + '"__*__"\n' + ' System-defined names. These names are defined by the ' + 'interpreter\n' + ' and its implementation (including the standard library). ' + 'Current\n' + ' system names are discussed in the *Special method names* ' + 'section\n' + ' and elsewhere. More will likely be defined in future ' + 'versions of\n' + ' Python. *Any* use of "__*__" names, in any context, that ' + 'does not\n' + ' follow explicitly documented use, is subject to breakage ' + 'without\n' + ' warning.\n' + '\n' + '"__*"\n' + ' Class-private names. Names in this category, when used ' + 'within the\n' + ' context of a class definition, are re-written to use a ' + 'mangled form\n' + ' to help avoid name clashes between "private" attributes ' + 'of base and\n' + ' derived classes. See section *Identifiers (Names)*.\n', + 'identifiers': '\n' + 'Identifiers and keywords\n' + '************************\n' + '\n' + 'Identifiers (also referred to as *names*) are described by ' + 'the\n' + 'following lexical definitions.\n' + '\n' + 'The syntax of identifiers in Python is based on the Unicode ' + 'standard\n' + 'annex UAX-31, with elaboration and changes as defined ' + 'below; see also\n' + '**PEP 3131** for further details.\n' + '\n' + 'Within the ASCII range (U+0001..U+007F), the valid ' + 'characters for\n' + 'identifiers are the same as in Python 2.x: the uppercase ' + 'and lowercase\n' + 'letters "A" through "Z", the underscore "_" and, except for ' + 'the first\n' + 'character, the digits "0" through "9".\n' + '\n' + 'Python 3.0 introduces additional characters from outside ' + 'the ASCII\n' + 'range (see **PEP 3131**). For these characters, the ' + 'classification\n' + 'uses the version of the Unicode Character Database as ' + 'included in the\n' + '"unicodedata" module.\n' + '\n' + 'Identifiers are unlimited in length. Case is significant.\n' + '\n' + ' identifier ::= xid_start xid_continue*\n' + ' id_start ::= \n' + ' id_continue ::= \n' + ' xid_start ::= \n' + ' xid_continue ::= \n' + '\n' + 'The Unicode category codes mentioned above stand for:\n' + '\n' + '* *Lu* - uppercase letters\n' + '\n' + '* *Ll* - lowercase letters\n' + '\n' + '* *Lt* - titlecase letters\n' + '\n' + '* *Lm* - modifier letters\n' + '\n' + '* *Lo* - other letters\n' + '\n' + '* *Nl* - letter numbers\n' + '\n' + '* *Mn* - nonspacing marks\n' + '\n' + '* *Mc* - spacing combining marks\n' + '\n' + '* *Nd* - decimal numbers\n' + '\n' + '* *Pc* - connector punctuations\n' + '\n' + '* *Other_ID_Start* - explicit list of characters in ' + 'PropList.txt to\n' + ' support backwards compatibility\n' + '\n' + '* *Other_ID_Continue* - likewise\n' + '\n' + 'All identifiers are converted into the normal form NFKC ' + 'while parsing;\n' + 'comparison of identifiers is based on NFKC.\n' + '\n' + 'A non-normative HTML file listing all valid identifier ' + 'characters for\n' + 'Unicode 4.1 can be found at http://www.dcl.hpi.uni-\n' + 'potsdam.de/home/loewis/table-3131.html.\n' + '\n' + '\n' + 'Keywords\n' + '========\n' + '\n' + 'The following identifiers are used as reserved words, or ' + '*keywords* of\n' + 'the language, and cannot be used as ordinary identifiers. ' + 'They must\n' + 'be spelled exactly as written here:\n' + '\n' + ' False class finally is return\n' + ' None continue for lambda try\n' + ' True def from nonlocal while\n' + ' and del global not with\n' + ' as elif if or yield\n' + ' assert else import pass\n' + ' break except in raise\n' + '\n' + '\n' + 'Reserved classes of identifiers\n' + '===============================\n' + '\n' + 'Certain classes of identifiers (besides keywords) have ' + 'special\n' + 'meanings. These classes are identified by the patterns of ' + 'leading and\n' + 'trailing underscore characters:\n' + '\n' + '"_*"\n' + ' Not imported by "from module import *". The special ' + 'identifier "_"\n' + ' is used in the interactive interpreter to store the ' + 'result of the\n' + ' last evaluation; it is stored in the "builtins" module. ' + 'When not\n' + ' in interactive mode, "_" has no special meaning and is ' + 'not defined.\n' + ' See section *The import statement*.\n' + '\n' + ' Note: The name "_" is often used in conjunction with\n' + ' internationalization; refer to the documentation for ' + 'the\n' + ' "gettext" module for more information on this ' + 'convention.\n' + '\n' + '"__*__"\n' + ' System-defined names. These names are defined by the ' + 'interpreter\n' + ' and its implementation (including the standard ' + 'library). Current\n' + ' system names are discussed in the *Special method names* ' + 'section\n' + ' and elsewhere. More will likely be defined in future ' + 'versions of\n' + ' Python. *Any* use of "__*__" names, in any context, ' + 'that does not\n' + ' follow explicitly documented use, is subject to breakage ' + 'without\n' + ' warning.\n' + '\n' + '"__*"\n' + ' Class-private names. Names in this category, when used ' + 'within the\n' + ' context of a class definition, are re-written to use a ' + 'mangled form\n' + ' to help avoid name clashes between "private" attributes ' + 'of base and\n' + ' derived classes. See section *Identifiers (Names)*.\n', + 'if': '\n' + 'The "if" statement\n' + '******************\n' + '\n' + 'The "if" statement is used for conditional execution:\n' + '\n' + ' if_stmt ::= "if" expression ":" suite\n' + ' ( "elif" expression ":" suite )*\n' + ' ["else" ":" suite]\n' + '\n' + 'It selects exactly one of the suites by evaluating the expressions ' + 'one\n' + 'by one until one is found to be true (see section *Boolean ' + 'operations*\n' + 'for the definition of true and false); then that suite is executed\n' + '(and no other part of the "if" statement is executed or evaluated).\n' + 'If all expressions are false, the suite of the "else" clause, if\n' + 'present, is executed.\n', + 'imaginary': '\n' + 'Imaginary literals\n' + '******************\n' + '\n' + 'Imaginary literals are described by the following lexical ' + 'definitions:\n' + '\n' + ' imagnumber ::= (floatnumber | intpart) ("j" | "J")\n' + '\n' + 'An imaginary literal yields a complex number with a real part ' + 'of 0.0.\n' + 'Complex numbers are represented as a pair of floating point ' + 'numbers\n' + 'and have the same restrictions on their range. To create a ' + 'complex\n' + 'number with a nonzero real part, add a floating point number ' + 'to it,\n' + 'e.g., "(3+4j)". Some examples of imaginary literals:\n' + '\n' + ' 3.14j 10.j 10j .001j 1e100j 3.14e-10j\n', + 'import': '\n' + 'The "import" statement\n' + '**********************\n' + '\n' + ' import_stmt ::= "import" module ["as" name] ( "," module ' + '["as" name] )*\n' + ' | "from" relative_module "import" identifier ' + '["as" name]\n' + ' ( "," identifier ["as" name] )*\n' + ' | "from" relative_module "import" "(" ' + 'identifier ["as" name]\n' + ' ( "," identifier ["as" name] )* [","] ")"\n' + ' | "from" module "import" "*"\n' + ' module ::= (identifier ".")* identifier\n' + ' relative_module ::= "."* module | "."+\n' + ' name ::= identifier\n' + '\n' + 'The basic import statement (no "from" clause) is executed in ' + 'two\n' + 'steps:\n' + '\n' + '1. find a module, loading and initializing it if necessary\n' + '\n' + '2. define a name or names in the local namespace for the scope\n' + ' where the "import" statement occurs.\n' + '\n' + 'When the statement contains multiple clauses (separated by ' + 'commas) the\n' + 'two steps are carried out separately for each clause, just as ' + 'though\n' + 'the clauses had been separated out into individiual import ' + 'statements.\n' + '\n' + 'The details of the first step, finding and loading modules are\n' + 'described in greater detail in the section on the *import ' + 'system*,\n' + 'which also describes the various types of packages and modules ' + 'that\n' + 'can be imported, as well as all the hooks that can be used to\n' + 'customize the import system. Note that failures in this step ' + 'may\n' + 'indicate either that the module could not be located, *or* that ' + 'an\n' + 'error occurred while initializing the module, which includes ' + 'execution\n' + "of the module's code.\n" + '\n' + 'If the requested module is retrieved successfully, it will be ' + 'made\n' + 'available in the local namespace in one of three ways:\n' + '\n' + '* If the module name is followed by "as", then the name ' + 'following\n' + ' "as" is bound directly to the imported module.\n' + '\n' + '* If no other name is specified, and the module being imported ' + 'is a\n' + " top level module, the module's name is bound in the local " + 'namespace\n' + ' as a reference to the imported module\n' + '\n' + '* If the module being imported is *not* a top level module, then ' + 'the\n' + ' name of the top level package that contains the module is ' + 'bound in\n' + ' the local namespace as a reference to the top level package. ' + 'The\n' + ' imported module must be accessed using its full qualified ' + 'name\n' + ' rather than directly\n' + '\n' + 'The "from" form uses a slightly more complex process:\n' + '\n' + '1. find the module specified in the "from" clause, loading and\n' + ' initializing it if necessary;\n' + '\n' + '2. for each of the identifiers specified in the "import" ' + 'clauses:\n' + '\n' + ' 1. check if the imported module has an attribute by that ' + 'name\n' + '\n' + ' 2. if not, attempt to import a submodule with that name and ' + 'then\n' + ' check the imported module again for that attribute\n' + '\n' + ' 3. if the attribute is not found, "ImportError" is raised.\n' + '\n' + ' 4. otherwise, a reference to that value is stored in the ' + 'local\n' + ' namespace, using the name in the "as" clause if it is ' + 'present,\n' + ' otherwise using the attribute name\n' + '\n' + 'Examples:\n' + '\n' + ' import foo # foo imported and bound locally\n' + ' import foo.bar.baz # foo.bar.baz imported, foo bound ' + 'locally\n' + ' import foo.bar.baz as fbb # foo.bar.baz imported and bound ' + 'as fbb\n' + ' from foo.bar import baz # foo.bar.baz imported and bound ' + 'as baz\n' + ' from foo import attr # foo imported and foo.attr bound ' + 'as attr\n' + '\n' + 'If the list of identifiers is replaced by a star ("\'*\'"), all ' + 'public\n' + 'names defined in the module are bound in the local namespace for ' + 'the\n' + 'scope where the "import" statement occurs.\n' + '\n' + 'The *public names* defined by a module are determined by ' + 'checking the\n' + 'module\'s namespace for a variable named "__all__"; if defined, ' + 'it must\n' + 'be a sequence of strings which are names defined or imported by ' + 'that\n' + 'module. The names given in "__all__" are all considered public ' + 'and\n' + 'are required to exist. If "__all__" is not defined, the set of ' + 'public\n' + "names includes all names found in the module's namespace which " + 'do not\n' + 'begin with an underscore character ("\'_\'"). "__all__" should ' + 'contain\n' + 'the entire public API. It is intended to avoid accidentally ' + 'exporting\n' + 'items that are not part of the API (such as library modules ' + 'which were\n' + 'imported and used within the module).\n' + '\n' + 'The wild card form of import --- "from module import *" --- is ' + 'only\n' + 'allowed at the module level. Attempting to use it in class or\n' + 'function definitions will raise a "SyntaxError".\n' + '\n' + 'When specifying what module to import you do not have to specify ' + 'the\n' + 'absolute name of the module. When a module or package is ' + 'contained\n' + 'within another package it is possible to make a relative import ' + 'within\n' + 'the same top package without having to mention the package name. ' + 'By\n' + 'using leading dots in the specified module or package after ' + '"from" you\n' + 'can specify how high to traverse up the current package ' + 'hierarchy\n' + 'without specifying exact names. One leading dot means the ' + 'current\n' + 'package where the module making the import exists. Two dots ' + 'means up\n' + 'one package level. Three dots is up two levels, etc. So if you ' + 'execute\n' + '"from . import mod" from a module in the "pkg" package then you ' + 'will\n' + 'end up importing "pkg.mod". If you execute "from ..subpkg2 ' + 'import mod"\n' + 'from within "pkg.subpkg1" you will import "pkg.subpkg2.mod". ' + 'The\n' + 'specification for relative imports is contained within **PEP ' + '328**.\n' + '\n' + '"importlib.import_module()" is provided to support applications ' + 'that\n' + 'determine dynamically the modules to be loaded.\n' + '\n' + '\n' + 'Future statements\n' + '=================\n' + '\n' + 'A *future statement* is a directive to the compiler that a ' + 'particular\n' + 'module should be compiled using syntax or semantics that will ' + 'be\n' + 'available in a specified future release of Python where the ' + 'feature\n' + 'becomes standard.\n' + '\n' + 'The future statement is intended to ease migration to future ' + 'versions\n' + 'of Python that introduce incompatible changes to the language. ' + 'It\n' + 'allows use of the new features on a per-module basis before the\n' + 'release in which the feature becomes standard.\n' + '\n' + ' future_statement ::= "from" "__future__" "import" feature ' + '["as" name]\n' + ' ("," feature ["as" name])*\n' + ' | "from" "__future__" "import" "(" ' + 'feature ["as" name]\n' + ' ("," feature ["as" name])* [","] ")"\n' + ' feature ::= identifier\n' + ' name ::= identifier\n' + '\n' + 'A future statement must appear near the top of the module. The ' + 'only\n' + 'lines that can appear before a future statement are:\n' + '\n' + '* the module docstring (if any),\n' + '\n' + '* comments,\n' + '\n' + '* blank lines, and\n' + '\n' + '* other future statements.\n' + '\n' + 'The features recognized by Python 3.0 are "absolute_import",\n' + '"division", "generators", "unicode_literals", "print_function",\n' + '"nested_scopes" and "with_statement". They are all redundant ' + 'because\n' + 'they are always enabled, and only kept for backwards ' + 'compatibility.\n' + '\n' + 'A future statement is recognized and treated specially at ' + 'compile\n' + 'time: Changes to the semantics of core constructs are often\n' + 'implemented by generating different code. It may even be the ' + 'case\n' + 'that a new feature introduces new incompatible syntax (such as a ' + 'new\n' + 'reserved word), in which case the compiler may need to parse ' + 'the\n' + 'module differently. Such decisions cannot be pushed off until\n' + 'runtime.\n' + '\n' + 'For any given release, the compiler knows which feature names ' + 'have\n' + 'been defined, and raises a compile-time error if a future ' + 'statement\n' + 'contains a feature not known to it.\n' + '\n' + 'The direct runtime semantics are the same as for any import ' + 'statement:\n' + 'there is a standard module "__future__", described later, and it ' + 'will\n' + 'be imported in the usual way at the time the future statement ' + 'is\n' + 'executed.\n' + '\n' + 'The interesting runtime semantics depend on the specific ' + 'feature\n' + 'enabled by the future statement.\n' + '\n' + 'Note that there is nothing special about the statement:\n' + '\n' + ' import __future__ [as name]\n' + '\n' + "That is not a future statement; it's an ordinary import " + 'statement with\n' + 'no special semantics or syntax restrictions.\n' + '\n' + 'Code compiled by calls to the built-in functions "exec()" and\n' + '"compile()" that occur in a module "M" containing a future ' + 'statement\n' + 'will, by default, use the new syntax or semantics associated ' + 'with the\n' + 'future statement. This can be controlled by optional arguments ' + 'to\n' + '"compile()" --- see the documentation of that function for ' + 'details.\n' + '\n' + 'A future statement typed at an interactive interpreter prompt ' + 'will\n' + 'take effect for the rest of the interpreter session. If an\n' + 'interpreter is started with the *-i* option, is passed a script ' + 'name\n' + 'to execute, and the script includes a future statement, it will ' + 'be in\n' + 'effect in the interactive session started after the script is\n' + 'executed.\n' + '\n' + 'See also: **PEP 236** - Back to the __future__\n' + '\n' + ' The original proposal for the __future__ mechanism.\n', + 'in': '\n' + 'Comparisons\n' + '***********\n' + '\n' + 'Unlike C, all comparison operations in Python have the same ' + 'priority,\n' + 'which is lower than that of any arithmetic, shifting or bitwise\n' + 'operation. Also unlike C, expressions like "a < b < c" have the\n' + 'interpretation that is conventional in mathematics:\n' + '\n' + ' comparison ::= or_expr ( comp_operator or_expr )*\n' + ' comp_operator ::= "<" | ">" | "==" | ">=" | "<=" | "!="\n' + ' | "is" ["not"] | ["not"] "in"\n' + '\n' + 'Comparisons yield boolean values: "True" or "False".\n' + '\n' + 'Comparisons can be chained arbitrarily, e.g., "x < y <= z" is\n' + 'equivalent to "x < y and y <= z", except that "y" is evaluated only\n' + 'once (but in both cases "z" is not evaluated at all when "x < y" is\n' + 'found to be false).\n' + '\n' + 'Formally, if *a*, *b*, *c*, ..., *y*, *z* are expressions and ' + '*op1*,\n' + '*op2*, ..., *opN* are comparison operators, then "a op1 b op2 c ... ' + 'y\n' + 'opN z" is equivalent to "a op1 b and b op2 c and ... y opN z", ' + 'except\n' + 'that each expression is evaluated at most once.\n' + '\n' + 'Note that "a op1 b op2 c" doesn\'t imply any kind of comparison ' + 'between\n' + '*a* and *c*, so that, e.g., "x < y > z" is perfectly legal (though\n' + 'perhaps not pretty).\n' + '\n' + 'The operators "<", ">", "==", ">=", "<=", and "!=" compare the ' + 'values\n' + 'of two objects. The objects need not have the same type. If both ' + 'are\n' + 'numbers, they are converted to a common type. Otherwise, the "==" ' + 'and\n' + '"!=" operators *always* consider objects of different types to be\n' + 'unequal, while the "<", ">", ">=" and "<=" operators raise a\n' + '"TypeError" when comparing objects of different types that do not\n' + 'implement these operators for the given pair of types. You can\n' + 'control comparison behavior of objects of non-built-in types by\n' + 'defining rich comparison methods like "__gt__()", described in ' + 'section\n' + '*Basic customization*.\n' + '\n' + 'Comparison of objects of the same type depends on the type:\n' + '\n' + '* Numbers are compared arithmetically.\n' + '\n' + '* The values "float(\'NaN\')" and "Decimal(\'NaN\')" are special. ' + 'They\n' + ' are identical to themselves, "x is x" but are not equal to\n' + ' themselves, "x != x". Additionally, comparing any value to a\n' + ' not-a-number value will return "False". For example, both "3 <\n' + ' float(\'NaN\')" and "float(\'NaN\') < 3" will return "False".\n' + '\n' + '* Bytes objects are compared lexicographically using the numeric\n' + ' values of their elements.\n' + '\n' + '* Strings are compared lexicographically using the numeric\n' + ' equivalents (the result of the built-in function "ord()") of ' + 'their\n' + " characters. [3] String and bytes object can't be compared!\n" + '\n' + '* Tuples and lists are compared lexicographically using comparison\n' + ' of corresponding elements. This means that to compare equal, ' + 'each\n' + ' element must compare equal and the two sequences must be of the ' + 'same\n' + ' type and have the same length.\n' + '\n' + ' If not equal, the sequences are ordered the same as their first\n' + ' differing elements. For example, "[1,2,x] <= [1,2,y]" has the ' + 'same\n' + ' value as "x <= y". If the corresponding element does not exist, ' + 'the\n' + ' shorter sequence is ordered first (for example, "[1,2] < ' + '[1,2,3]").\n' + '\n' + '* Mappings (dictionaries) compare equal if and only if they have ' + 'the\n' + ' same "(key, value)" pairs. Order comparisons "(\'<\', \'<=\', ' + "'>=',\n" + ' \'>\')" raise "TypeError".\n' + '\n' + '* Sets and frozensets define comparison operators to mean subset ' + 'and\n' + ' superset tests. Those relations do not define total orderings ' + '(the\n' + ' two sets "{1,2}" and "{2,3}" are not equal, nor subsets of one\n' + ' another, nor supersets of one another). Accordingly, sets are ' + 'not\n' + ' appropriate arguments for functions which depend on total ' + 'ordering.\n' + ' For example, "min()", "max()", and "sorted()" produce undefined\n' + ' results given a list of sets as inputs.\n' + '\n' + '* Most other objects of built-in types compare unequal unless they\n' + ' are the same object; the choice whether one object is considered\n' + ' smaller or larger than another one is made arbitrarily but\n' + ' consistently within one execution of a program.\n' + '\n' + 'Comparison of objects of differing types depends on whether either ' + 'of\n' + 'the types provide explicit support for the comparison. Most ' + 'numeric\n' + 'types can be compared with one another. When cross-type comparison ' + 'is\n' + 'not supported, the comparison method returns "NotImplemented".\n' + '\n' + 'The operators "in" and "not in" test for membership. "x in s"\n' + 'evaluates to true if *x* is a member of *s*, and false otherwise. ' + '"x\n' + 'not in s" returns the negation of "x in s". All built-in sequences\n' + 'and set types support this as well as dictionary, for which "in" ' + 'tests\n' + 'whether the dictionary has a given key. For container types such as\n' + 'list, tuple, set, frozenset, dict, or collections.deque, the\n' + 'expression "x in y" is equivalent to "any(x is e or x == e for e in\n' + 'y)".\n' + '\n' + 'For the string and bytes types, "x in y" is true if and only if *x* ' + 'is\n' + 'a substring of *y*. An equivalent test is "y.find(x) != -1". ' + 'Empty\n' + 'strings are always considered to be a substring of any other ' + 'string,\n' + 'so """ in "abc"" will return "True".\n' + '\n' + 'For user-defined classes which define the "__contains__()" method, ' + '"x\n' + 'in y" is true if and only if "y.__contains__(x)" is true.\n' + '\n' + 'For user-defined classes which do not define "__contains__()" but ' + 'do\n' + 'define "__iter__()", "x in y" is true if some value "z" with "x == ' + 'z"\n' + 'is produced while iterating over "y". If an exception is raised\n' + 'during the iteration, it is as if "in" raised that exception.\n' + '\n' + 'Lastly, the old-style iteration protocol is tried: if a class ' + 'defines\n' + '"__getitem__()", "x in y" is true if and only if there is a non-\n' + 'negative integer index *i* such that "x == y[i]", and all lower\n' + 'integer indices do not raise "IndexError" exception. (If any other\n' + 'exception is raised, it is as if "in" raised that exception).\n' + '\n' + 'The operator "not in" is defined to have the inverse true value of\n' + '"in".\n' + '\n' + 'The operators "is" and "is not" test for object identity: "x is y" ' + 'is\n' + 'true if and only if *x* and *y* are the same object. "x is not y"\n' + 'yields the inverse truth value. [4]\n', + 'integers': '\n' + 'Integer literals\n' + '****************\n' + '\n' + 'Integer literals are described by the following lexical ' + 'definitions:\n' + '\n' + ' integer ::= decimalinteger | octinteger | hexinteger ' + '| bininteger\n' + ' decimalinteger ::= nonzerodigit digit* | "0"+\n' + ' nonzerodigit ::= "1"..."9"\n' + ' digit ::= "0"..."9"\n' + ' octinteger ::= "0" ("o" | "O") octdigit+\n' + ' hexinteger ::= "0" ("x" | "X") hexdigit+\n' + ' bininteger ::= "0" ("b" | "B") bindigit+\n' + ' octdigit ::= "0"..."7"\n' + ' hexdigit ::= digit | "a"..."f" | "A"..."F"\n' + ' bindigit ::= "0" | "1"\n' + '\n' + 'There is no limit for the length of integer literals apart ' + 'from what\n' + 'can be stored in available memory.\n' + '\n' + 'Note that leading zeros in a non-zero decimal number are not ' + 'allowed.\n' + 'This is for disambiguation with C-style octal literals, which ' + 'Python\n' + 'used before version 3.0.\n' + '\n' + 'Some examples of integer literals:\n' + '\n' + ' 7 2147483647 0o177 ' + '0b100110111\n' + ' 3 79228162514264337593543950336 0o377 ' + '0xdeadbeef\n', + 'lambda': '\n' + 'Lambdas\n' + '*******\n' + '\n' + ' lambda_expr ::= "lambda" [parameter_list]: expression\n' + ' lambda_expr_nocond ::= "lambda" [parameter_list]: ' + 'expression_nocond\n' + '\n' + 'Lambda expressions (sometimes called lambda forms) are used to ' + 'create\n' + 'anonymous functions. The expression "lambda arguments: ' + 'expression"\n' + 'yields a function object. The unnamed object behaves like a ' + 'function\n' + 'object defined with\n' + '\n' + ' def (arguments):\n' + ' return expression\n' + '\n' + 'See section *Function definitions* for the syntax of parameter ' + 'lists.\n' + 'Note that functions created with lambda expressions cannot ' + 'contain\n' + 'statements or annotations.\n', + 'lists': '\n' + 'List displays\n' + '*************\n' + '\n' + 'A list display is a possibly empty series of expressions enclosed ' + 'in\n' + 'square brackets:\n' + '\n' + ' list_display ::= "[" [expression_list | comprehension] "]"\n' + '\n' + 'A list display yields a new list object, the contents being ' + 'specified\n' + 'by either a list of expressions or a comprehension. When a ' + 'comma-\n' + 'separated list of expressions is supplied, its elements are ' + 'evaluated\n' + 'from left to right and placed into the list object in that ' + 'order.\n' + 'When a comprehension is supplied, the list is constructed from ' + 'the\n' + 'elements resulting from the comprehension.\n', + 'naming': '\n' + 'Naming and binding\n' + '******************\n' + '\n' + '\n' + 'Binding of names\n' + '================\n' + '\n' + '*Names* refer to objects. Names are introduced by name binding\n' + 'operations.\n' + '\n' + 'The following constructs bind names: formal parameters to ' + 'functions,\n' + '"import" statements, class and function definitions (these bind ' + 'the\n' + 'class or function name in the defining block), and targets that ' + 'are\n' + 'identifiers if occurring in an assignment, "for" loop header, or ' + 'after\n' + '"as" in a "with" statement or "except" clause. The "import" ' + 'statement\n' + 'of the form "from ... import *" binds all names defined in the\n' + 'imported module, except those beginning with an underscore. ' + 'This form\n' + 'may only be used at the module level.\n' + '\n' + 'A target occurring in a "del" statement is also considered bound ' + 'for\n' + 'this purpose (though the actual semantics are to unbind the ' + 'name).\n' + '\n' + 'Each assignment or import statement occurs within a block ' + 'defined by a\n' + 'class or function definition or at the module level (the ' + 'top-level\n' + 'code block).\n' + '\n' + 'If a name is bound in a block, it is a local variable of that ' + 'block,\n' + 'unless declared as "nonlocal" or "global". If a name is bound ' + 'at the\n' + 'module level, it is a global variable. (The variables of the ' + 'module\n' + 'code block are local and global.) If a variable is used in a ' + 'code\n' + 'block but not defined there, it is a *free variable*.\n' + '\n' + 'Each occurrence of a name in the program text refers to the ' + '*binding*\n' + 'of that name established by the following name resolution ' + 'rules.\n' + '\n' + '\n' + 'Resolution of names\n' + '===================\n' + '\n' + 'A *scope* defines the visibility of a name within a block. If a ' + 'local\n' + 'variable is defined in a block, its scope includes that block. ' + 'If the\n' + 'definition occurs in a function block, the scope extends to any ' + 'blocks\n' + 'contained within the defining one, unless a contained block ' + 'introduces\n' + 'a different binding for the name.\n' + '\n' + 'When a name is used in a code block, it is resolved using the ' + 'nearest\n' + 'enclosing scope. The set of all such scopes visible to a code ' + 'block\n' + "is called the block's *environment*.\n" + '\n' + 'When a name is not found at all, a "NameError" exception is ' + 'raised. If\n' + 'the current scope is a function scope, and the name refers to a ' + 'local\n' + 'variable that has not yet been bound to a value at the point ' + 'where the\n' + 'name is used, an "UnboundLocalError" exception is raised.\n' + '"UnboundLocalError" is a subclass of "NameError".\n' + '\n' + 'If a name binding operation occurs anywhere within a code block, ' + 'all\n' + 'uses of the name within the block are treated as references to ' + 'the\n' + 'current block. This can lead to errors when a name is used ' + 'within a\n' + 'block before it is bound. This rule is subtle. Python lacks\n' + 'declarations and allows name binding operations to occur ' + 'anywhere\n' + 'within a code block. The local variables of a code block can ' + 'be\n' + 'determined by scanning the entire text of the block for name ' + 'binding\n' + 'operations.\n' + '\n' + 'If the "global" statement occurs within a block, all uses of the ' + 'name\n' + 'specified in the statement refer to the binding of that name in ' + 'the\n' + 'top-level namespace. Names are resolved in the top-level ' + 'namespace by\n' + 'searching the global namespace, i.e. the namespace of the ' + 'module\n' + 'containing the code block, and the builtins namespace, the ' + 'namespace\n' + 'of the module "builtins". The global namespace is searched ' + 'first. If\n' + 'the name is not found there, the builtins namespace is ' + 'searched. The\n' + '"global" statement must precede all uses of the name.\n' + '\n' + 'The "global" statement has the same scope as a name binding ' + 'operation\n' + 'in the same block. If the nearest enclosing scope for a free ' + 'variable\n' + 'contains a global statement, the free variable is treated as a ' + 'global.\n' + '\n' + 'The "nonlocal" statement causes corresponding names to refer to\n' + 'previously bound variables in the nearest enclosing function ' + 'scope.\n' + '"SyntaxError" is raised at compile time if the given name does ' + 'not\n' + 'exist in any enclosing function scope.\n' + '\n' + 'The namespace for a module is automatically created the first ' + 'time a\n' + 'module is imported. The main module for a script is always ' + 'called\n' + '"__main__".\n' + '\n' + 'Class definition blocks and arguments to "exec()" and "eval()" ' + 'are\n' + 'special in the context of name resolution. A class definition is ' + 'an\n' + 'executable statement that may use and define names. These ' + 'references\n' + 'follow the normal rules for name resolution with an exception ' + 'that\n' + 'unbound local variables are looked up in the global namespace. ' + 'The\n' + 'namespace of the class definition becomes the attribute ' + 'dictionary of\n' + 'the class. The scope of names defined in a class block is ' + 'limited to\n' + 'the class block; it does not extend to the code blocks of ' + 'methods --\n' + 'this includes comprehensions and generator expressions since ' + 'they are\n' + 'implemented using a function scope. This means that the ' + 'following\n' + 'will fail:\n' + '\n' + ' class A:\n' + ' a = 42\n' + ' b = list(a + i for i in range(10))\n' + '\n' + '\n' + 'Builtins and restricted execution\n' + '=================================\n' + '\n' + 'The builtins namespace associated with the execution of a code ' + 'block\n' + 'is actually found by looking up the name "__builtins__" in its ' + 'global\n' + 'namespace; this should be a dictionary or a module (in the ' + 'latter case\n' + "the module's dictionary is used). By default, when in the " + '"__main__"\n' + 'module, "__builtins__" is the built-in module "builtins"; when ' + 'in any\n' + 'other module, "__builtins__" is an alias for the dictionary of ' + 'the\n' + '"builtins" module itself. "__builtins__" can be set to a ' + 'user-created\n' + 'dictionary to create a weak form of restricted execution.\n' + '\n' + '**CPython implementation detail:** Users should not touch\n' + '"__builtins__"; it is strictly an implementation detail. Users\n' + 'wanting to override values in the builtins namespace should ' + '"import"\n' + 'the "builtins" module and modify its attributes appropriately.\n' + '\n' + '\n' + 'Interaction with dynamic features\n' + '=================================\n' + '\n' + 'Name resolution of free variables occurs at runtime, not at ' + 'compile\n' + 'time. This means that the following code will print 42:\n' + '\n' + ' i = 10\n' + ' def f():\n' + ' print(i)\n' + ' i = 42\n' + ' f()\n' + '\n' + 'There are several cases where Python statements are illegal when ' + 'used\n' + 'in conjunction with nested scopes that contain free variables.\n' + '\n' + 'If a variable is referenced in an enclosing scope, it is illegal ' + 'to\n' + 'delete the name. An error will be reported at compile time.\n' + '\n' + 'The "eval()" and "exec()" functions do not have access to the ' + 'full\n' + 'environment for resolving names. Names may be resolved in the ' + 'local\n' + 'and global namespaces of the caller. Free variables are not ' + 'resolved\n' + 'in the nearest enclosing namespace, but in the global ' + 'namespace. [1]\n' + 'The "exec()" and "eval()" functions have optional arguments to\n' + 'override the global and local namespace. If only one namespace ' + 'is\n' + 'specified, it is used for both.\n', + 'nonlocal': '\n' + 'The "nonlocal" statement\n' + '************************\n' + '\n' + ' nonlocal_stmt ::= "nonlocal" identifier ("," identifier)*\n' + '\n' + 'The "nonlocal" statement causes the listed identifiers to ' + 'refer to\n' + 'previously bound variables in the nearest enclosing scope ' + 'excluding\n' + 'globals. This is important because the default behavior for ' + 'binding is\n' + 'to search the local namespace first. The statement allows\n' + 'encapsulated code to rebind variables outside of the local ' + 'scope\n' + 'besides the global (module) scope.\n' + '\n' + 'Names listed in a "nonlocal" statement, unlike those listed in ' + 'a\n' + '"global" statement, must refer to pre-existing bindings in an\n' + 'enclosing scope (the scope in which a new binding should be ' + 'created\n' + 'cannot be determined unambiguously).\n' + '\n' + 'Names listed in a "nonlocal" statement must not collide with ' + 'pre-\n' + 'existing bindings in the local scope.\n' + '\n' + 'See also: **PEP 3104** - Access to Names in Outer Scopes\n' + '\n' + ' The specification for the "nonlocal" statement.\n', + 'numbers': '\n' + 'Numeric literals\n' + '****************\n' + '\n' + 'There are three types of numeric literals: integers, floating ' + 'point\n' + 'numbers, and imaginary numbers. There are no complex literals\n' + '(complex numbers can be formed by adding a real number and an\n' + 'imaginary number).\n' + '\n' + 'Note that numeric literals do not include a sign; a phrase like ' + '"-1"\n' + 'is actually an expression composed of the unary operator ' + '\'"-"\' and the\n' + 'literal "1".\n', + 'numeric-types': '\n' + 'Emulating numeric types\n' + '***********************\n' + '\n' + 'The following methods can be defined to emulate numeric ' + 'objects.\n' + 'Methods corresponding to operations that are not ' + 'supported by the\n' + 'particular kind of number implemented (e.g., bitwise ' + 'operations for\n' + 'non-integral numbers) should be left undefined.\n' + '\n' + 'object.__add__(self, other)\n' + 'object.__sub__(self, other)\n' + 'object.__mul__(self, other)\n' + 'object.__matmul__(self, other)\n' + 'object.__truediv__(self, other)\n' + 'object.__floordiv__(self, other)\n' + 'object.__mod__(self, other)\n' + 'object.__divmod__(self, other)\n' + 'object.__pow__(self, other[, modulo])\n' + 'object.__lshift__(self, other)\n' + 'object.__rshift__(self, other)\n' + 'object.__and__(self, other)\n' + 'object.__xor__(self, other)\n' + 'object.__or__(self, other)\n' + '\n' + ' These methods are called to implement the binary ' + 'arithmetic\n' + ' operations ("+", "-", "*", "@", "/", "//", "%", ' + '"divmod()",\n' + ' "pow()", "**", "<<", ">>", "&", "^", "|"). For ' + 'instance, to\n' + ' evaluate the expression "x + y", where *x* is an ' + 'instance of a\n' + ' class that has an "__add__()" method, "x.__add__(y)" ' + 'is called.\n' + ' The "__divmod__()" method should be the equivalent to ' + 'using\n' + ' "__floordiv__()" and "__mod__()"; it should not be ' + 'related to\n' + ' "__truediv__()". Note that "__pow__()" should be ' + 'defined to accept\n' + ' an optional third argument if the ternary version of ' + 'the built-in\n' + ' "pow()" function is to be supported.\n' + '\n' + ' If one of those methods does not support the operation ' + 'with the\n' + ' supplied arguments, it should return ' + '"NotImplemented".\n' + '\n' + 'object.__radd__(self, other)\n' + 'object.__rsub__(self, other)\n' + 'object.__rmul__(self, other)\n' + 'object.__rmatmul__(self, other)\n' + 'object.__rtruediv__(self, other)\n' + 'object.__rfloordiv__(self, other)\n' + 'object.__rmod__(self, other)\n' + 'object.__rdivmod__(self, other)\n' + 'object.__rpow__(self, other)\n' + 'object.__rlshift__(self, other)\n' + 'object.__rrshift__(self, other)\n' + 'object.__rand__(self, other)\n' + 'object.__rxor__(self, other)\n' + 'object.__ror__(self, other)\n' + '\n' + ' These methods are called to implement the binary ' + 'arithmetic\n' + ' operations ("+", "-", "*", "@", "/", "//", "%", ' + '"divmod()",\n' + ' "pow()", "**", "<<", ">>", "&", "^", "|") with ' + 'reflected (swapped)\n' + ' operands. These functions are only called if the left ' + 'operand does\n' + ' not support the corresponding operation and the ' + 'operands are of\n' + ' different types. [2] For instance, to evaluate the ' + 'expression "x -\n' + ' y", where *y* is an instance of a class that has an ' + '"__rsub__()"\n' + ' method, "y.__rsub__(x)" is called if "x.__sub__(y)" ' + 'returns\n' + ' *NotImplemented*.\n' + '\n' + ' Note that ternary "pow()" will not try calling ' + '"__rpow__()" (the\n' + ' coercion rules would become too complicated).\n' + '\n' + " Note: If the right operand's type is a subclass of the " + 'left\n' + " operand's type and that subclass provides the " + 'reflected method\n' + ' for the operation, this method will be called before ' + 'the left\n' + " operand's non-reflected method. This behavior " + 'allows subclasses\n' + " to override their ancestors' operations.\n" + '\n' + 'object.__iadd__(self, other)\n' + 'object.__isub__(self, other)\n' + 'object.__imul__(self, other)\n' + 'object.__imatmul__(self, other)\n' + 'object.__itruediv__(self, other)\n' + 'object.__ifloordiv__(self, other)\n' + 'object.__imod__(self, other)\n' + 'object.__ipow__(self, other[, modulo])\n' + 'object.__ilshift__(self, other)\n' + 'object.__irshift__(self, other)\n' + 'object.__iand__(self, other)\n' + 'object.__ixor__(self, other)\n' + 'object.__ior__(self, other)\n' + '\n' + ' These methods are called to implement the augmented ' + 'arithmetic\n' + ' assignments ("+=", "-=", "*=", "@=", "/=", "//=", ' + '"%=", "**=",\n' + ' "<<=", ">>=", "&=", "^=", "|="). These methods should ' + 'attempt to\n' + ' do the operation in-place (modifying *self*) and ' + 'return the result\n' + ' (which could be, but does not have to be, *self*). If ' + 'a specific\n' + ' method is not defined, the augmented assignment falls ' + 'back to the\n' + ' normal methods. For instance, if *x* is an instance ' + 'of a class\n' + ' with an "__iadd__()" method, "x += y" is equivalent to ' + '"x =\n' + ' x.__iadd__(y)" . Otherwise, "x.__add__(y)" and ' + '"y.__radd__(x)" are\n' + ' considered, as with the evaluation of "x + y". In ' + 'certain\n' + ' situations, augmented assignment can result in ' + 'unexpected errors\n' + " (see *Why does a_tuple[i] += ['item'] raise an " + 'exception when the\n' + ' addition works?*), but this behavior is in fact part ' + 'of the data\n' + ' model.\n' + '\n' + 'object.__neg__(self)\n' + 'object.__pos__(self)\n' + 'object.__abs__(self)\n' + 'object.__invert__(self)\n' + '\n' + ' Called to implement the unary arithmetic operations ' + '("-", "+",\n' + ' "abs()" and "~").\n' + '\n' + 'object.__complex__(self)\n' + 'object.__int__(self)\n' + 'object.__float__(self)\n' + 'object.__round__(self[, n])\n' + '\n' + ' Called to implement the built-in functions ' + '"complex()", "int()",\n' + ' "float()" and "round()". Should return a value of the ' + 'appropriate\n' + ' type.\n' + '\n' + 'object.__index__(self)\n' + '\n' + ' Called to implement "operator.index()", and whenever ' + 'Python needs\n' + ' to losslessly convert the numeric object to an integer ' + 'object (such\n' + ' as in slicing, or in the built-in "bin()", "hex()" and ' + '"oct()"\n' + ' functions). Presence of this method indicates that the ' + 'numeric\n' + ' object is an integer type. Must return an integer.\n' + '\n' + ' Note: In order to have a coherent integer type class, ' + 'when\n' + ' "__index__()" is defined "__int__()" should also be ' + 'defined, and\n' + ' both should return the same value.\n', + 'objects': '\n' + 'Objects, values and types\n' + '*************************\n' + '\n' + "*Objects* are Python's abstraction for data. All data in a " + 'Python\n' + 'program is represented by objects or by relations between ' + 'objects. (In\n' + "a sense, and in conformance to Von Neumann's model of a " + '"stored\n' + 'program computer," code is also represented by objects.)\n' + '\n' + "Every object has an identity, a type and a value. An object's\n" + '*identity* never changes once it has been created; you may ' + 'think of it\n' + 'as the object\'s address in memory. The \'"is"\' operator ' + 'compares the\n' + 'identity of two objects; the "id()" function returns an ' + 'integer\n' + 'representing its identity.\n' + '\n' + '**CPython implementation detail:** For CPython, "id(x)" is the ' + 'memory\n' + 'address where "x" is stored.\n' + '\n' + "An object's type determines the operations that the object " + 'supports\n' + '(e.g., "does it have a length?") and also defines the possible ' + 'values\n' + 'for objects of that type. The "type()" function returns an ' + "object's\n" + 'type (which is an object itself). Like its identity, an ' + "object's\n" + '*type* is also unchangeable. [1]\n' + '\n' + 'The *value* of some objects can change. Objects whose value ' + 'can\n' + 'change are said to be *mutable*; objects whose value is ' + 'unchangeable\n' + 'once they are created are called *immutable*. (The value of an\n' + 'immutable container object that contains a reference to a ' + 'mutable\n' + "object can change when the latter's value is changed; however " + 'the\n' + 'container is still considered immutable, because the collection ' + 'of\n' + 'objects it contains cannot be changed. So, immutability is ' + 'not\n' + 'strictly the same as having an unchangeable value, it is more ' + 'subtle.)\n' + "An object's mutability is determined by its type; for " + 'instance,\n' + 'numbers, strings and tuples are immutable, while dictionaries ' + 'and\n' + 'lists are mutable.\n' + '\n' + 'Objects are never explicitly destroyed; however, when they ' + 'become\n' + 'unreachable they may be garbage-collected. An implementation ' + 'is\n' + 'allowed to postpone garbage collection or omit it altogether ' + '--- it is\n' + 'a matter of implementation quality how garbage collection is\n' + 'implemented, as long as no objects are collected that are ' + 'still\n' + 'reachable.\n' + '\n' + '**CPython implementation detail:** CPython currently uses a ' + 'reference-\n' + 'counting scheme with (optional) delayed detection of cyclically ' + 'linked\n' + 'garbage, which collects most objects as soon as they become\n' + 'unreachable, but is not guaranteed to collect garbage ' + 'containing\n' + 'circular references. See the documentation of the "gc" module ' + 'for\n' + 'information on controlling the collection of cyclic garbage. ' + 'Other\n' + 'implementations act differently and CPython may change. Do not ' + 'depend\n' + 'on immediate finalization of objects when they become ' + 'unreachable (so\n' + 'you should always close files explicitly).\n' + '\n' + "Note that the use of the implementation's tracing or debugging\n" + 'facilities may keep objects alive that would normally be ' + 'collectable.\n' + 'Also note that catching an exception with a ' + '\'"try"..."except"\'\n' + 'statement may keep objects alive.\n' + '\n' + 'Some objects contain references to "external" resources such as ' + 'open\n' + 'files or windows. It is understood that these resources are ' + 'freed\n' + 'when the object is garbage-collected, but since garbage ' + 'collection is\n' + 'not guaranteed to happen, such objects also provide an explicit ' + 'way to\n' + 'release the external resource, usually a "close()" method. ' + 'Programs\n' + 'are strongly recommended to explicitly close such objects. ' + 'The\n' + '\'"try"..."finally"\' statement and the \'"with"\' statement ' + 'provide\n' + 'convenient ways to do this.\n' + '\n' + 'Some objects contain references to other objects; these are ' + 'called\n' + '*containers*. Examples of containers are tuples, lists and\n' + "dictionaries. The references are part of a container's value. " + 'In\n' + 'most cases, when we talk about the value of a container, we ' + 'imply the\n' + 'values, not the identities of the contained objects; however, ' + 'when we\n' + 'talk about the mutability of a container, only the identities ' + 'of the\n' + 'immediately contained objects are implied. So, if an ' + 'immutable\n' + 'container (like a tuple) contains a reference to a mutable ' + 'object, its\n' + 'value changes if that mutable object is changed.\n' + '\n' + 'Types affect almost all aspects of object behavior. Even the\n' + 'importance of object identity is affected in some sense: for ' + 'immutable\n' + 'types, operations that compute new values may actually return ' + 'a\n' + 'reference to any existing object with the same type and value, ' + 'while\n' + 'for mutable objects this is not allowed. E.g., after "a = 1; b ' + '= 1",\n' + '"a" and "b" may or may not refer to the same object with the ' + 'value\n' + 'one, depending on the implementation, but after "c = []; d = ' + '[]", "c"\n' + 'and "d" are guaranteed to refer to two different, unique, ' + 'newly\n' + 'created empty lists. (Note that "c = d = []" assigns the same ' + 'object\n' + 'to both "c" and "d".)\n', + 'operator-summary': '\n' + 'Operator precedence\n' + '*******************\n' + '\n' + 'The following table summarizes the operator precedence ' + 'in Python, from\n' + 'lowest precedence (least binding) to highest ' + 'precedence (most\n' + 'binding). Operators in the same box have the same ' + 'precedence. Unless\n' + 'the syntax is explicitly given, operators are binary. ' + 'Operators in\n' + 'the same box group left to right (except for ' + 'exponentiation, which\n' + 'groups from right to left).\n' + '\n' + 'Note that comparisons, membership tests, and identity ' + 'tests, all have\n' + 'the same precedence and have a left-to-right chaining ' + 'feature as\n' + 'described in the *Comparisons* section.\n' + '\n' + '+-------------------------------------------------+---------------------------------------+\n' + '| Operator | ' + 'Description |\n' + '+=================================================+=======================================+\n' + '| "lambda" | ' + 'Lambda expression |\n' + '+-------------------------------------------------+---------------------------------------+\n' + '| "if" -- "else" | ' + 'Conditional expression |\n' + '+-------------------------------------------------+---------------------------------------+\n' + '| "or" | ' + 'Boolean OR |\n' + '+-------------------------------------------------+---------------------------------------+\n' + '| "and" | ' + 'Boolean AND |\n' + '+-------------------------------------------------+---------------------------------------+\n' + '| "not" "x" | ' + 'Boolean NOT |\n' + '+-------------------------------------------------+---------------------------------------+\n' + '| "in", "not in", "is", "is not", "<", "<=", ">", | ' + 'Comparisons, including membership |\n' + '| ">=", "!=", "==" | ' + 'tests and identity tests |\n' + '+-------------------------------------------------+---------------------------------------+\n' + '| "|" | ' + 'Bitwise OR |\n' + '+-------------------------------------------------+---------------------------------------+\n' + '| "^" | ' + 'Bitwise XOR |\n' + '+-------------------------------------------------+---------------------------------------+\n' + '| "&" | ' + 'Bitwise AND |\n' + '+-------------------------------------------------+---------------------------------------+\n' + '| "<<", ">>" | ' + 'Shifts |\n' + '+-------------------------------------------------+---------------------------------------+\n' + '| "+", "-" | ' + 'Addition and subtraction |\n' + '+-------------------------------------------------+---------------------------------------+\n' + '| "*", "@", "/", "//", "%" | ' + 'Multiplication, matrix multiplication |\n' + '| | ' + 'division, remainder [5] |\n' + '+-------------------------------------------------+---------------------------------------+\n' + '| "+x", "-x", "~x" | ' + 'Positive, negative, bitwise NOT |\n' + '+-------------------------------------------------+---------------------------------------+\n' + '| "**" | ' + 'Exponentiation [6] |\n' + '+-------------------------------------------------+---------------------------------------+\n' + '| "await" "x" | ' + 'Await expression |\n' + '+-------------------------------------------------+---------------------------------------+\n' + '| "x[index]", "x[index:index]", | ' + 'Subscription, slicing, call, |\n' + '| "x(arguments...)", "x.attribute" | ' + 'attribute reference |\n' + '+-------------------------------------------------+---------------------------------------+\n' + '| "(expressions...)", "[expressions...]", "{key: | ' + 'Binding or tuple display, list |\n' + '| value...}", "{expressions...}" | ' + 'display, dictionary display, set |\n' + '| | ' + 'display |\n' + '+-------------------------------------------------+---------------------------------------+\n' + '\n' + '-[ Footnotes ]-\n' + '\n' + '[1] While "abs(x%y) < abs(y)" is true mathematically, ' + 'for floats\n' + ' it may not be true numerically due to roundoff. ' + 'For example, and\n' + ' assuming a platform on which a Python float is an ' + 'IEEE 754 double-\n' + ' precision number, in order that "-1e-100 % 1e100" ' + 'have the same\n' + ' sign as "1e100", the computed result is "-1e-100 + ' + '1e100", which\n' + ' is numerically exactly equal to "1e100". The ' + 'function\n' + ' "math.fmod()" returns a result whose sign matches ' + 'the sign of the\n' + ' first argument instead, and so returns "-1e-100" ' + 'in this case.\n' + ' Which approach is more appropriate depends on the ' + 'application.\n' + '\n' + '[2] If x is very close to an exact integer multiple of ' + "y, it's\n" + ' possible for "x//y" to be one larger than ' + '"(x-x%y)//y" due to\n' + ' rounding. In such cases, Python returns the ' + 'latter result, in\n' + ' order to preserve that "divmod(x,y)[0] * y + x % ' + 'y" be very close\n' + ' to "x".\n' + '\n' + '[3] While comparisons between strings make sense at ' + 'the byte\n' + ' level, they may be counter-intuitive to users. ' + 'For example, the\n' + ' strings ""\\u00C7"" and ""\\u0043\\u0327"" compare ' + 'differently, even\n' + ' though they both represent the same unicode ' + 'character (LATIN\n' + ' CAPITAL LETTER C WITH CEDILLA). To compare ' + 'strings in a human\n' + ' recognizable way, compare using ' + '"unicodedata.normalize()".\n' + '\n' + '[4] Due to automatic garbage-collection, free lists, ' + 'and the\n' + ' dynamic nature of descriptors, you may notice ' + 'seemingly unusual\n' + ' behaviour in certain uses of the "is" operator, ' + 'like those\n' + ' involving comparisons between instance methods, or ' + 'constants.\n' + ' Check their documentation for more info.\n' + '\n' + '[5] The "%" operator is also used for string ' + 'formatting; the same\n' + ' precedence applies.\n' + '\n' + '[6] The power operator "**" binds less tightly than an ' + 'arithmetic\n' + ' or bitwise unary operator on its right, that is, ' + '"2**-1" is "0.5".\n', + 'pass': '\n' + 'The "pass" statement\n' + '********************\n' + '\n' + ' pass_stmt ::= "pass"\n' + '\n' + '"pass" is a null operation --- when it is executed, nothing ' + 'happens.\n' + 'It is useful as a placeholder when a statement is required\n' + 'syntactically, but no code needs to be executed, for example:\n' + '\n' + ' def f(arg): pass # a function that does nothing (yet)\n' + '\n' + ' class C: pass # a class with no methods (yet)\n', + 'power': '\n' + 'The power operator\n' + '******************\n' + '\n' + 'The power operator binds more tightly than unary operators on ' + 'its\n' + 'left; it binds less tightly than unary operators on its right. ' + 'The\n' + 'syntax is:\n' + '\n' + ' power ::= await ["**" u_expr]\n' + '\n' + 'Thus, in an unparenthesized sequence of power and unary ' + 'operators, the\n' + 'operators are evaluated from right to left (this does not ' + 'constrain\n' + 'the evaluation order for the operands): "-1**2" results in "-1".\n' + '\n' + 'The power operator has the same semantics as the built-in ' + '"pow()"\n' + 'function, when called with two arguments: it yields its left ' + 'argument\n' + 'raised to the power of its right argument. The numeric arguments ' + 'are\n' + 'first converted to a common type, and the result is of that ' + 'type.\n' + '\n' + 'For int operands, the result has the same type as the operands ' + 'unless\n' + 'the second argument is negative; in that case, all arguments are\n' + 'converted to float and a float result is delivered. For example,\n' + '"10**2" returns "100", but "10**-2" returns "0.01".\n' + '\n' + 'Raising "0.0" to a negative power results in a ' + '"ZeroDivisionError".\n' + 'Raising a negative number to a fractional power results in a ' + '"complex"\n' + 'number. (In earlier versions it raised a "ValueError".)\n', + 'raise': '\n' + 'The "raise" statement\n' + '*********************\n' + '\n' + ' raise_stmt ::= "raise" [expression ["from" expression]]\n' + '\n' + 'If no expressions are present, "raise" re-raises the last ' + 'exception\n' + 'that was active in the current scope. If no exception is active ' + 'in\n' + 'the current scope, a "RuntimeError" exception is raised ' + 'indicating\n' + 'that this is an error.\n' + '\n' + 'Otherwise, "raise" evaluates the first expression as the ' + 'exception\n' + 'object. It must be either a subclass or an instance of\n' + '"BaseException". If it is a class, the exception instance will ' + 'be\n' + 'obtained when needed by instantiating the class with no ' + 'arguments.\n' + '\n' + "The *type* of the exception is the exception instance's class, " + 'the\n' + '*value* is the instance itself.\n' + '\n' + 'A traceback object is normally created automatically when an ' + 'exception\n' + 'is raised and attached to it as the "__traceback__" attribute, ' + 'which\n' + 'is writable. You can create an exception and set your own ' + 'traceback in\n' + 'one step using the "with_traceback()" exception method (which ' + 'returns\n' + 'the same exception instance, with its traceback set to its ' + 'argument),\n' + 'like so:\n' + '\n' + ' raise Exception("foo occurred").with_traceback(tracebackobj)\n' + '\n' + 'The "from" clause is used for exception chaining: if given, the ' + 'second\n' + '*expression* must be another exception class or instance, which ' + 'will\n' + 'then be attached to the raised exception as the "__cause__" ' + 'attribute\n' + '(which is writable). If the raised exception is not handled, ' + 'both\n' + 'exceptions will be printed:\n' + '\n' + ' >>> try:\n' + ' ... print(1 / 0)\n' + ' ... except Exception as exc:\n' + ' ... raise RuntimeError("Something bad happened") from exc\n' + ' ...\n' + ' Traceback (most recent call last):\n' + ' File "", line 2, in \n' + ' ZeroDivisionError: int division or modulo by zero\n' + '\n' + ' The above exception was the direct cause of the following ' + 'exception:\n' + '\n' + ' Traceback (most recent call last):\n' + ' File "", line 4, in \n' + ' RuntimeError: Something bad happened\n' + '\n' + 'A similar mechanism works implicitly if an exception is raised ' + 'inside\n' + 'an exception handler or a "finally" clause: the previous ' + 'exception is\n' + 'then attached as the new exception\'s "__context__" attribute:\n' + '\n' + ' >>> try:\n' + ' ... print(1 / 0)\n' + ' ... except:\n' + ' ... raise RuntimeError("Something bad happened")\n' + ' ...\n' + ' Traceback (most recent call last):\n' + ' File "", line 2, in \n' + ' ZeroDivisionError: int division or modulo by zero\n' + '\n' + ' During handling of the above exception, another exception ' + 'occurred:\n' + '\n' + ' Traceback (most recent call last):\n' + ' File "", line 4, in \n' + ' RuntimeError: Something bad happened\n' + '\n' + 'Additional information on exceptions can be found in section\n' + '*Exceptions*, and information about handling exceptions is in ' + 'section\n' + '*The try statement*.\n', + 'return': '\n' + 'The "return" statement\n' + '**********************\n' + '\n' + ' return_stmt ::= "return" [expression_list]\n' + '\n' + '"return" may only occur syntactically nested in a function ' + 'definition,\n' + 'not within a nested class definition.\n' + '\n' + 'If an expression list is present, it is evaluated, else "None" ' + 'is\n' + 'substituted.\n' + '\n' + '"return" leaves the current function call with the expression ' + 'list (or\n' + '"None") as return value.\n' + '\n' + 'When "return" passes control out of a "try" statement with a ' + '"finally"\n' + 'clause, that "finally" clause is executed before really leaving ' + 'the\n' + 'function.\n' + '\n' + 'In a generator function, the "return" statement indicates that ' + 'the\n' + 'generator is done and will cause "StopIteration" to be raised. ' + 'The\n' + 'returned value (if any) is used as an argument to construct\n' + '"StopIteration" and becomes the "StopIteration.value" ' + 'attribute.\n', + 'sequence-types': '\n' + 'Emulating container types\n' + '*************************\n' + '\n' + 'The following methods can be defined to implement ' + 'container objects.\n' + 'Containers usually are sequences (such as lists or ' + 'tuples) or mappings\n' + '(like dictionaries), but can represent other containers ' + 'as well. The\n' + 'first set of methods is used either to emulate a ' + 'sequence or to\n' + 'emulate a mapping; the difference is that for a ' + 'sequence, the\n' + 'allowable keys should be the integers *k* for which "0 ' + '<= k < N" where\n' + '*N* is the length of the sequence, or slice objects, ' + 'which define a\n' + 'range of items. It is also recommended that mappings ' + 'provide the\n' + 'methods "keys()", "values()", "items()", "get()", ' + '"clear()",\n' + '"setdefault()", "pop()", "popitem()", "copy()", and ' + '"update()"\n' + "behaving similar to those for Python's standard " + 'dictionary objects.\n' + 'The "collections" module provides a "MutableMapping" ' + 'abstract base\n' + 'class to help create those methods from a base set of ' + '"__getitem__()",\n' + '"__setitem__()", "__delitem__()", and "keys()". Mutable ' + 'sequences\n' + 'should provide methods "append()", "count()", "index()", ' + '"extend()",\n' + '"insert()", "pop()", "remove()", "reverse()" and ' + '"sort()", like Python\n' + 'standard list objects. Finally, sequence types should ' + 'implement\n' + 'addition (meaning concatenation) and multiplication ' + '(meaning\n' + 'repetition) by defining the methods "__add__()", ' + '"__radd__()",\n' + '"__iadd__()", "__mul__()", "__rmul__()" and "__imul__()" ' + 'described\n' + 'below; they should not define other numerical ' + 'operators. It is\n' + 'recommended that both mappings and sequences implement ' + 'the\n' + '"__contains__()" method to allow efficient use of the ' + '"in" operator;\n' + 'for mappings, "in" should search the mapping\'s keys; ' + 'for sequences, it\n' + 'should search through the values. It is further ' + 'recommended that both\n' + 'mappings and sequences implement the "__iter__()" method ' + 'to allow\n' + 'efficient iteration through the container; for mappings, ' + '"__iter__()"\n' + 'should be the same as "keys()"; for sequences, it should ' + 'iterate\n' + 'through the values.\n' + '\n' + 'object.__len__(self)\n' + '\n' + ' Called to implement the built-in function "len()". ' + 'Should return\n' + ' the length of the object, an integer ">=" 0. Also, ' + 'an object that\n' + ' doesn\'t define a "__bool__()" method and whose ' + '"__len__()" method\n' + ' returns zero is considered to be false in a Boolean ' + 'context.\n' + '\n' + 'object.__length_hint__(self)\n' + '\n' + ' Called to implement "operator.length_hint()". Should ' + 'return an\n' + ' estimated length for the object (which may be greater ' + 'or less than\n' + ' the actual length). The length must be an integer ' + '">=" 0. This\n' + ' method is purely an optimization and is never ' + 'required for\n' + ' correctness.\n' + '\n' + ' New in version 3.4.\n' + '\n' + 'Note: Slicing is done exclusively with the following ' + 'three methods.\n' + ' A call like\n' + '\n' + ' a[1:2] = b\n' + '\n' + ' is translated to\n' + '\n' + ' a[slice(1, 2, None)] = b\n' + '\n' + ' and so forth. Missing slice items are always filled ' + 'in with "None".\n' + '\n' + 'object.__getitem__(self, key)\n' + '\n' + ' Called to implement evaluation of "self[key]". For ' + 'sequence types,\n' + ' the accepted keys should be integers and slice ' + 'objects. Note that\n' + ' the special interpretation of negative indexes (if ' + 'the class wishes\n' + ' to emulate a sequence type) is up to the ' + '"__getitem__()" method. If\n' + ' *key* is of an inappropriate type, "TypeError" may be ' + 'raised; if of\n' + ' a value outside the set of indexes for the sequence ' + '(after any\n' + ' special interpretation of negative values), ' + '"IndexError" should be\n' + ' raised. For mapping types, if *key* is missing (not ' + 'in the\n' + ' container), "KeyError" should be raised.\n' + '\n' + ' Note: "for" loops expect that an "IndexError" will be ' + 'raised for\n' + ' illegal indexes to allow proper detection of the ' + 'end of the\n' + ' sequence.\n' + '\n' + 'object.__missing__(self, key)\n' + '\n' + ' Called by "dict"."__getitem__()" to implement ' + '"self[key]" for dict\n' + ' subclasses when key is not in the dictionary.\n' + '\n' + 'object.__setitem__(self, key, value)\n' + '\n' + ' Called to implement assignment to "self[key]". Same ' + 'note as for\n' + ' "__getitem__()". This should only be implemented for ' + 'mappings if\n' + ' the objects support changes to the values for keys, ' + 'or if new keys\n' + ' can be added, or for sequences if elements can be ' + 'replaced. The\n' + ' same exceptions should be raised for improper *key* ' + 'values as for\n' + ' the "__getitem__()" method.\n' + '\n' + 'object.__delitem__(self, key)\n' + '\n' + ' Called to implement deletion of "self[key]". Same ' + 'note as for\n' + ' "__getitem__()". This should only be implemented for ' + 'mappings if\n' + ' the objects support removal of keys, or for sequences ' + 'if elements\n' + ' can be removed from the sequence. The same ' + 'exceptions should be\n' + ' raised for improper *key* values as for the ' + '"__getitem__()" method.\n' + '\n' + 'object.__iter__(self)\n' + '\n' + ' This method is called when an iterator is required ' + 'for a container.\n' + ' This method should return a new iterator object that ' + 'can iterate\n' + ' over all the objects in the container. For mappings, ' + 'it should\n' + ' iterate over the keys of the container.\n' + '\n' + ' Iterator objects also need to implement this method; ' + 'they are\n' + ' required to return themselves. For more information ' + 'on iterator\n' + ' objects, see *Iterator Types*.\n' + '\n' + 'object.__reversed__(self)\n' + '\n' + ' Called (if present) by the "reversed()" built-in to ' + 'implement\n' + ' reverse iteration. It should return a new iterator ' + 'object that\n' + ' iterates over all the objects in the container in ' + 'reverse order.\n' + '\n' + ' If the "__reversed__()" method is not provided, the ' + '"reversed()"\n' + ' built-in will fall back to using the sequence ' + 'protocol ("__len__()"\n' + ' and "__getitem__()"). Objects that support the ' + 'sequence protocol\n' + ' should only provide "__reversed__()" if they can ' + 'provide an\n' + ' implementation that is more efficient than the one ' + 'provided by\n' + ' "reversed()".\n' + '\n' + 'The membership test operators ("in" and "not in") are ' + 'normally\n' + 'implemented as an iteration through a sequence. ' + 'However, container\n' + 'objects can supply the following special method with a ' + 'more efficient\n' + 'implementation, which also does not require the object ' + 'be a sequence.\n' + '\n' + 'object.__contains__(self, item)\n' + '\n' + ' Called to implement membership test operators. ' + 'Should return true\n' + ' if *item* is in *self*, false otherwise. For mapping ' + 'objects, this\n' + ' should consider the keys of the mapping rather than ' + 'the values or\n' + ' the key-item pairs.\n' + '\n' + ' For objects that don\'t define "__contains__()", the ' + 'membership test\n' + ' first tries iteration via "__iter__()", then the old ' + 'sequence\n' + ' iteration protocol via "__getitem__()", see *this ' + 'section in the\n' + ' language reference*.\n', + 'shifting': '\n' + 'Shifting operations\n' + '*******************\n' + '\n' + 'The shifting operations have lower priority than the ' + 'arithmetic\n' + 'operations:\n' + '\n' + ' shift_expr ::= a_expr | shift_expr ( "<<" | ">>" ) a_expr\n' + '\n' + 'These operators accept integers as arguments. They shift the ' + 'first\n' + 'argument to the left or right by the number of bits given by ' + 'the\n' + 'second argument.\n' + '\n' + 'A right shift by *n* bits is defined as floor division by ' + '"pow(2,n)".\n' + 'A left shift by *n* bits is defined as multiplication with ' + '"pow(2,n)".\n' + '\n' + 'Note: In the current implementation, the right-hand operand ' + 'is\n' + ' required to be at most "sys.maxsize". If the right-hand ' + 'operand is\n' + ' larger than "sys.maxsize" an "OverflowError" exception is ' + 'raised.\n', + 'slicings': '\n' + 'Slicings\n' + '********\n' + '\n' + 'A slicing selects a range of items in a sequence object (e.g., ' + 'a\n' + 'string, tuple or list). Slicings may be used as expressions ' + 'or as\n' + 'targets in assignment or "del" statements. The syntax for a ' + 'slicing:\n' + '\n' + ' slicing ::= primary "[" slice_list "]"\n' + ' slice_list ::= slice_item ("," slice_item)* [","]\n' + ' slice_item ::= expression | proper_slice\n' + ' proper_slice ::= [lower_bound] ":" [upper_bound] [ ":" ' + '[stride] ]\n' + ' lower_bound ::= expression\n' + ' upper_bound ::= expression\n' + ' stride ::= expression\n' + '\n' + 'There is ambiguity in the formal syntax here: anything that ' + 'looks like\n' + 'an expression list also looks like a slice list, so any ' + 'subscription\n' + 'can be interpreted as a slicing. Rather than further ' + 'complicating the\n' + 'syntax, this is disambiguated by defining that in this case ' + 'the\n' + 'interpretation as a subscription takes priority over the\n' + 'interpretation as a slicing (this is the case if the slice ' + 'list\n' + 'contains no proper slice).\n' + '\n' + 'The semantics for a slicing are as follows. The primary is ' + 'indexed\n' + '(using the same "__getitem__()" method as normal subscription) ' + 'with a\n' + 'key that is constructed from the slice list, as follows. If ' + 'the slice\n' + 'list contains at least one comma, the key is a tuple ' + 'containing the\n' + 'conversion of the slice items; otherwise, the conversion of ' + 'the lone\n' + 'slice item is the key. The conversion of a slice item that is ' + 'an\n' + 'expression is that expression. The conversion of a proper ' + 'slice is a\n' + 'slice object (see section *The standard type hierarchy*) ' + 'whose\n' + '"start", "stop" and "step" attributes are the values of the\n' + 'expressions given as lower bound, upper bound and stride,\n' + 'respectively, substituting "None" for missing expressions.\n', + 'specialattrs': '\n' + 'Special Attributes\n' + '******************\n' + '\n' + 'The implementation adds a few special read-only attributes ' + 'to several\n' + 'object types, where they are relevant. Some of these are ' + 'not reported\n' + 'by the "dir()" built-in function.\n' + '\n' + 'object.__dict__\n' + '\n' + ' A dictionary or other mapping object used to store an ' + "object's\n" + ' (writable) attributes.\n' + '\n' + 'instance.__class__\n' + '\n' + ' The class to which a class instance belongs.\n' + '\n' + 'class.__bases__\n' + '\n' + ' The tuple of base classes of a class object.\n' + '\n' + 'class.__name__\n' + '\n' + ' The name of the class or type.\n' + '\n' + 'class.__qualname__\n' + '\n' + ' The *qualified name* of the class or type.\n' + '\n' + ' New in version 3.3.\n' + '\n' + 'class.__mro__\n' + '\n' + ' This attribute is a tuple of classes that are ' + 'considered when\n' + ' looking for base classes during method resolution.\n' + '\n' + 'class.mro()\n' + '\n' + ' This method can be overridden by a metaclass to ' + 'customize the\n' + ' method resolution order for its instances. It is ' + 'called at class\n' + ' instantiation, and its result is stored in "__mro__".\n' + '\n' + 'class.__subclasses__()\n' + '\n' + ' Each class keeps a list of weak references to its ' + 'immediate\n' + ' subclasses. This method returns a list of all those ' + 'references\n' + ' still alive. Example:\n' + '\n' + ' >>> int.__subclasses__()\n' + " []\n" + '\n' + '-[ Footnotes ]-\n' + '\n' + '[1] Additional information on these special methods may be ' + 'found\n' + ' in the Python Reference Manual (*Basic ' + 'customization*).\n' + '\n' + '[2] As a consequence, the list "[1, 2]" is considered ' + 'equal to\n' + ' "[1.0, 2.0]", and similarly for tuples.\n' + '\n' + "[3] They must have since the parser can't tell the type of " + 'the\n' + ' operands.\n' + '\n' + '[4] Cased characters are those with general category ' + 'property\n' + ' being one of "Lu" (Letter, uppercase), "Ll" (Letter, ' + 'lowercase),\n' + ' or "Lt" (Letter, titlecase).\n' + '\n' + '[5] To format only a tuple you should therefore provide a\n' + ' singleton tuple whose only element is the tuple to be ' + 'formatted.\n', + 'specialnames': '\n' + 'Special method names\n' + '********************\n' + '\n' + 'A class can implement certain operations that are invoked ' + 'by special\n' + 'syntax (such as arithmetic operations or subscripting and ' + 'slicing) by\n' + "defining methods with special names. This is Python's " + 'approach to\n' + '*operator overloading*, allowing classes to define their ' + 'own behavior\n' + 'with respect to language operators. For instance, if a ' + 'class defines\n' + 'a method named "__getitem__()", and "x" is an instance of ' + 'this class,\n' + 'then "x[i]" is roughly equivalent to ' + '"type(x).__getitem__(x, i)".\n' + 'Except where mentioned, attempts to execute an operation ' + 'raise an\n' + 'exception when no appropriate method is defined ' + '(typically\n' + '"AttributeError" or "TypeError").\n' + '\n' + 'When implementing a class that emulates any built-in type, ' + 'it is\n' + 'important that the emulation only be implemented to the ' + 'degree that it\n' + 'makes sense for the object being modelled. For example, ' + 'some\n' + 'sequences may work well with retrieval of individual ' + 'elements, but\n' + 'extracting a slice may not make sense. (One example of ' + 'this is the\n' + '"NodeList" interface in the W3C\'s Document Object ' + 'Model.)\n' + '\n' + '\n' + 'Basic customization\n' + '===================\n' + '\n' + 'object.__new__(cls[, ...])\n' + '\n' + ' Called to create a new instance of class *cls*. ' + '"__new__()" is a\n' + ' static method (special-cased so you need not declare it ' + 'as such)\n' + ' that takes the class of which an instance was requested ' + 'as its\n' + ' first argument. The remaining arguments are those ' + 'passed to the\n' + ' object constructor expression (the call to the class). ' + 'The return\n' + ' value of "__new__()" should be the new object instance ' + '(usually an\n' + ' instance of *cls*).\n' + '\n' + ' Typical implementations create a new instance of the ' + 'class by\n' + ' invoking the superclass\'s "__new__()" method using\n' + ' "super(currentclass, cls).__new__(cls[, ...])" with ' + 'appropriate\n' + ' arguments and then modifying the newly-created instance ' + 'as\n' + ' necessary before returning it.\n' + '\n' + ' If "__new__()" returns an instance of *cls*, then the ' + 'new\n' + ' instance\'s "__init__()" method will be invoked like\n' + ' "__init__(self[, ...])", where *self* is the new ' + 'instance and the\n' + ' remaining arguments are the same as were passed to ' + '"__new__()".\n' + '\n' + ' If "__new__()" does not return an instance of *cls*, ' + 'then the new\n' + ' instance\'s "__init__()" method will not be invoked.\n' + '\n' + ' "__new__()" is intended mainly to allow subclasses of ' + 'immutable\n' + ' types (like int, str, or tuple) to customize instance ' + 'creation. It\n' + ' is also commonly overridden in custom metaclasses in ' + 'order to\n' + ' customize class creation.\n' + '\n' + 'object.__init__(self[, ...])\n' + '\n' + ' Called after the instance has been created (by ' + '"__new__()"), but\n' + ' before it is returned to the caller. The arguments are ' + 'those\n' + ' passed to the class constructor expression. If a base ' + 'class has an\n' + ' "__init__()" method, the derived class\'s "__init__()" ' + 'method, if\n' + ' any, must explicitly call it to ensure proper ' + 'initialization of the\n' + ' base class part of the instance; for example:\n' + ' "BaseClass.__init__(self, [args...])".\n' + '\n' + ' Because "__new__()" and "__init__()" work together in ' + 'constructing\n' + ' objects ("__new__()" to create it, and "__init__()" to ' + 'customise\n' + ' it), no non-"None" value may be returned by ' + '"__init__()"; doing so\n' + ' will cause a "TypeError" to be raised at runtime.\n' + '\n' + 'object.__del__(self)\n' + '\n' + ' Called when the instance is about to be destroyed. ' + 'This is also\n' + ' called a destructor. If a base class has a "__del__()" ' + 'method, the\n' + ' derived class\'s "__del__()" method, if any, must ' + 'explicitly call it\n' + ' to ensure proper deletion of the base class part of the ' + 'instance.\n' + ' Note that it is possible (though not recommended!) for ' + 'the\n' + ' "__del__()" method to postpone destruction of the ' + 'instance by\n' + ' creating a new reference to it. It may then be called ' + 'at a later\n' + ' time when this new reference is deleted. It is not ' + 'guaranteed that\n' + ' "__del__()" methods are called for objects that still ' + 'exist when\n' + ' the interpreter exits.\n' + '\n' + ' Note: "del x" doesn\'t directly call "x.__del__()" --- ' + 'the former\n' + ' decrements the reference count for "x" by one, and ' + 'the latter is\n' + ' only called when "x"\'s reference count reaches ' + 'zero. Some common\n' + ' situations that may prevent the reference count of an ' + 'object from\n' + ' going to zero include: circular references between ' + 'objects (e.g.,\n' + ' a doubly-linked list or a tree data structure with ' + 'parent and\n' + ' child pointers); a reference to the object on the ' + 'stack frame of\n' + ' a function that caught an exception (the traceback ' + 'stored in\n' + ' "sys.exc_info()[2]" keeps the stack frame alive); or ' + 'a reference\n' + ' to the object on the stack frame that raised an ' + 'unhandled\n' + ' exception in interactive mode (the traceback stored ' + 'in\n' + ' "sys.last_traceback" keeps the stack frame alive). ' + 'The first\n' + ' situation can only be remedied by explicitly breaking ' + 'the cycles;\n' + ' the second can be resolved by freeing the reference ' + 'to the\n' + ' traceback object when it is no longer useful, and the ' + 'third can\n' + ' be resolved by storing "None" in ' + '"sys.last_traceback". Circular\n' + ' references which are garbage are detected and cleaned ' + 'up when the\n' + " cyclic garbage collector is enabled (it's on by " + 'default). Refer\n' + ' to the documentation for the "gc" module for more ' + 'information\n' + ' about this topic.\n' + '\n' + ' Warning: Due to the precarious circumstances under ' + 'which\n' + ' "__del__()" methods are invoked, exceptions that ' + 'occur during\n' + ' their execution are ignored, and a warning is printed ' + 'to\n' + ' "sys.stderr" instead. Also, when "__del__()" is ' + 'invoked in\n' + ' response to a module being deleted (e.g., when ' + 'execution of the\n' + ' program is done), other globals referenced by the ' + '"__del__()"\n' + ' method may already have been deleted or in the ' + 'process of being\n' + ' torn down (e.g. the import machinery shutting down). ' + 'For this\n' + ' reason, "__del__()" methods should do the absolute ' + 'minimum needed\n' + ' to maintain external invariants. Starting with ' + 'version 1.5,\n' + ' Python guarantees that globals whose name begins with ' + 'a single\n' + ' underscore are deleted from their module before other ' + 'globals are\n' + ' deleted; if no other references to such globals ' + 'exist, this may\n' + ' help in assuring that imported modules are still ' + 'available at the\n' + ' time when the "__del__()" method is called.\n' + '\n' + 'object.__repr__(self)\n' + '\n' + ' Called by the "repr()" built-in function to compute the ' + '"official"\n' + ' string representation of an object. If at all ' + 'possible, this\n' + ' should look like a valid Python expression that could ' + 'be used to\n' + ' recreate an object with the same value (given an ' + 'appropriate\n' + ' environment). If this is not possible, a string of the ' + 'form\n' + ' "<...some useful description...>" should be returned. ' + 'The return\n' + ' value must be a string object. If a class defines ' + '"__repr__()" but\n' + ' not "__str__()", then "__repr__()" is also used when an ' + '"informal"\n' + ' string representation of instances of that class is ' + 'required.\n' + '\n' + ' This is typically used for debugging, so it is ' + 'important that the\n' + ' representation is information-rich and unambiguous.\n' + '\n' + 'object.__str__(self)\n' + '\n' + ' Called by "str(object)" and the built-in functions ' + '"format()" and\n' + ' "print()" to compute the "informal" or nicely printable ' + 'string\n' + ' representation of an object. The return value must be ' + 'a *string*\n' + ' object.\n' + '\n' + ' This method differs from "object.__repr__()" in that ' + 'there is no\n' + ' expectation that "__str__()" return a valid Python ' + 'expression: a\n' + ' more convenient or concise representation can be used.\n' + '\n' + ' The default implementation defined by the built-in type ' + '"object"\n' + ' calls "object.__repr__()".\n' + '\n' + 'object.__bytes__(self)\n' + '\n' + ' Called by "bytes()" to compute a byte-string ' + 'representation of an\n' + ' object. This should return a "bytes" object.\n' + '\n' + 'object.__format__(self, format_spec)\n' + '\n' + ' Called by the "format()" built-in function (and by ' + 'extension, the\n' + ' "str.format()" method of class "str") to produce a ' + '"formatted"\n' + ' string representation of an object. The "format_spec" ' + 'argument is a\n' + ' string that contains a description of the formatting ' + 'options\n' + ' desired. The interpretation of the "format_spec" ' + 'argument is up to\n' + ' the type implementing "__format__()", however most ' + 'classes will\n' + ' either delegate formatting to one of the built-in ' + 'types, or use a\n' + ' similar formatting option syntax.\n' + '\n' + ' See *Format Specification Mini-Language* for a ' + 'description of the\n' + ' standard formatting syntax.\n' + '\n' + ' The return value must be a string object.\n' + '\n' + ' Changed in version 3.4: The __format__ method of ' + '"object" itself\n' + ' raises a "TypeError" if passed any non-empty string.\n' + '\n' + 'object.__lt__(self, other)\n' + 'object.__le__(self, other)\n' + 'object.__eq__(self, other)\n' + 'object.__ne__(self, other)\n' + 'object.__gt__(self, other)\n' + 'object.__ge__(self, other)\n' + '\n' + ' These are the so-called "rich comparison" methods. The\n' + ' correspondence between operator symbols and method ' + 'names is as\n' + ' follows: "xy" calls\n' + ' "x.__gt__(y)", and "x>=y" calls "x.__ge__(y)".\n' + '\n' + ' A rich comparison method may return the singleton ' + '"NotImplemented"\n' + ' if it does not implement the operation for a given pair ' + 'of\n' + ' arguments. By convention, "False" and "True" are ' + 'returned for a\n' + ' successful comparison. However, these methods can ' + 'return any value,\n' + ' so if the comparison operator is used in a Boolean ' + 'context (e.g.,\n' + ' in the condition of an "if" statement), Python will ' + 'call "bool()"\n' + ' on the value to determine if the result is true or ' + 'false.\n' + '\n' + ' By default, "__ne__()" delegates to "__eq__()" and ' + 'inverts the\n' + ' result unless it is "NotImplemented". There are no ' + 'other implied\n' + ' relationships among the comparison operators, for ' + 'example, the\n' + ' truth of "(x.__hash__".\n' + '\n' + ' If a class that does not override "__eq__()" wishes to ' + 'suppress\n' + ' hash support, it should include "__hash__ = None" in ' + 'the class\n' + ' definition. A class which defines its own "__hash__()" ' + 'that\n' + ' explicitly raises a "TypeError" would be incorrectly ' + 'identified as\n' + ' hashable by an "isinstance(obj, collections.Hashable)" ' + 'call.\n' + '\n' + ' Note: By default, the "__hash__()" values of str, bytes ' + 'and\n' + ' datetime objects are "salted" with an unpredictable ' + 'random value.\n' + ' Although they remain constant within an individual ' + 'Python\n' + ' process, they are not predictable between repeated ' + 'invocations of\n' + ' Python.This is intended to provide protection against ' + 'a denial-\n' + ' of-service caused by carefully-chosen inputs that ' + 'exploit the\n' + ' worst case performance of a dict insertion, O(n^2) ' + 'complexity.\n' + ' See ' + 'http://www.ocert.org/advisories/ocert-2011-003.html for\n' + ' details.Changing hash values affects the iteration ' + 'order of\n' + ' dicts, sets and other mappings. Python has never ' + 'made guarantees\n' + ' about this ordering (and it typically varies between ' + '32-bit and\n' + ' 64-bit builds).See also "PYTHONHASHSEED".\n' + '\n' + ' Changed in version 3.3: Hash randomization is enabled ' + 'by default.\n' + '\n' + 'object.__bool__(self)\n' + '\n' + ' Called to implement truth value testing and the ' + 'built-in operation\n' + ' "bool()"; should return "False" or "True". When this ' + 'method is not\n' + ' defined, "__len__()" is called, if it is defined, and ' + 'the object is\n' + ' considered true if its result is nonzero. If a class ' + 'defines\n' + ' neither "__len__()" nor "__bool__()", all its instances ' + 'are\n' + ' considered true.\n' + '\n' + '\n' + 'Customizing attribute access\n' + '============================\n' + '\n' + 'The following methods can be defined to customize the ' + 'meaning of\n' + 'attribute access (use of, assignment to, or deletion of ' + '"x.name") for\n' + 'class instances.\n' + '\n' + 'object.__getattr__(self, name)\n' + '\n' + ' Called when an attribute lookup has not found the ' + 'attribute in the\n' + ' usual places (i.e. it is not an instance attribute nor ' + 'is it found\n' + ' in the class tree for "self"). "name" is the attribute ' + 'name. This\n' + ' method should return the (computed) attribute value or ' + 'raise an\n' + ' "AttributeError" exception.\n' + '\n' + ' Note that if the attribute is found through the normal ' + 'mechanism,\n' + ' "__getattr__()" is not called. (This is an intentional ' + 'asymmetry\n' + ' between "__getattr__()" and "__setattr__()".) This is ' + 'done both for\n' + ' efficiency reasons and because otherwise ' + '"__getattr__()" would have\n' + ' no way to access other attributes of the instance. ' + 'Note that at\n' + ' least for instance variables, you can fake total ' + 'control by not\n' + ' inserting any values in the instance attribute ' + 'dictionary (but\n' + ' instead inserting them in another object). See the\n' + ' "__getattribute__()" method below for a way to actually ' + 'get total\n' + ' control over attribute access.\n' + '\n' + 'object.__getattribute__(self, name)\n' + '\n' + ' Called unconditionally to implement attribute accesses ' + 'for\n' + ' instances of the class. If the class also defines ' + '"__getattr__()",\n' + ' the latter will not be called unless ' + '"__getattribute__()" either\n' + ' calls it explicitly or raises an "AttributeError". This ' + 'method\n' + ' should return the (computed) attribute value or raise ' + 'an\n' + ' "AttributeError" exception. In order to avoid infinite ' + 'recursion in\n' + ' this method, its implementation should always call the ' + 'base class\n' + ' method with the same name to access any attributes it ' + 'needs, for\n' + ' example, "object.__getattribute__(self, name)".\n' + '\n' + ' Note: This method may still be bypassed when looking up ' + 'special\n' + ' methods as the result of implicit invocation via ' + 'language syntax\n' + ' or built-in functions. See *Special method lookup*.\n' + '\n' + 'object.__setattr__(self, name, value)\n' + '\n' + ' Called when an attribute assignment is attempted. This ' + 'is called\n' + ' instead of the normal mechanism (i.e. store the value ' + 'in the\n' + ' instance dictionary). *name* is the attribute name, ' + '*value* is the\n' + ' value to be assigned to it.\n' + '\n' + ' If "__setattr__()" wants to assign to an instance ' + 'attribute, it\n' + ' should call the base class method with the same name, ' + 'for example,\n' + ' "object.__setattr__(self, name, value)".\n' + '\n' + 'object.__delattr__(self, name)\n' + '\n' + ' Like "__setattr__()" but for attribute deletion instead ' + 'of\n' + ' assignment. This should only be implemented if "del ' + 'obj.name" is\n' + ' meaningful for the object.\n' + '\n' + 'object.__dir__(self)\n' + '\n' + ' Called when "dir()" is called on the object. A sequence ' + 'must be\n' + ' returned. "dir()" converts the returned sequence to a ' + 'list and\n' + ' sorts it.\n' + '\n' + '\n' + 'Implementing Descriptors\n' + '------------------------\n' + '\n' + 'The following methods only apply when an instance of the ' + 'class\n' + 'containing the method (a so-called *descriptor* class) ' + 'appears in an\n' + '*owner* class (the descriptor must be in either the ' + "owner's class\n" + 'dictionary or in the class dictionary for one of its ' + 'parents). In the\n' + 'examples below, "the attribute" refers to the attribute ' + 'whose name is\n' + 'the key of the property in the owner class\' "__dict__".\n' + '\n' + 'object.__get__(self, instance, owner)\n' + '\n' + ' Called to get the attribute of the owner class (class ' + 'attribute\n' + ' access) or of an instance of that class (instance ' + 'attribute\n' + ' access). *owner* is always the owner class, while ' + '*instance* is the\n' + ' instance that the attribute was accessed through, or ' + '"None" when\n' + ' the attribute is accessed through the *owner*. This ' + 'method should\n' + ' return the (computed) attribute value or raise an ' + '"AttributeError"\n' + ' exception.\n' + '\n' + 'object.__set__(self, instance, value)\n' + '\n' + ' Called to set the attribute on an instance *instance* ' + 'of the owner\n' + ' class to a new value, *value*.\n' + '\n' + 'object.__delete__(self, instance)\n' + '\n' + ' Called to delete the attribute on an instance ' + '*instance* of the\n' + ' owner class.\n' + '\n' + 'The attribute "__objclass__" is interpreted by the ' + '"inspect" module as\n' + 'specifying the class where this object was defined ' + '(setting this\n' + 'appropriately can assist in runtime introspection of ' + 'dynamic class\n' + 'attributes). For callables, it may indicate that an ' + 'instance of the\n' + 'given type (or a subclass) is expected or required as the ' + 'first\n' + 'positional argument (for example, CPython sets this ' + 'attribute for\n' + 'unbound methods that are implemented in C).\n' + '\n' + '\n' + 'Invoking Descriptors\n' + '--------------------\n' + '\n' + 'In general, a descriptor is an object attribute with ' + '"binding\n' + 'behavior", one whose attribute access has been overridden ' + 'by methods\n' + 'in the descriptor protocol: "__get__()", "__set__()", ' + 'and\n' + '"__delete__()". If any of those methods are defined for an ' + 'object, it\n' + 'is said to be a descriptor.\n' + '\n' + 'The default behavior for attribute access is to get, set, ' + 'or delete\n' + "the attribute from an object's dictionary. For instance, " + '"a.x" has a\n' + 'lookup chain starting with "a.__dict__[\'x\']", then\n' + '"type(a).__dict__[\'x\']", and continuing through the base ' + 'classes of\n' + '"type(a)" excluding metaclasses.\n' + '\n' + 'However, if the looked-up value is an object defining one ' + 'of the\n' + 'descriptor methods, then Python may override the default ' + 'behavior and\n' + 'invoke the descriptor method instead. Where this occurs ' + 'in the\n' + 'precedence chain depends on which descriptor methods were ' + 'defined and\n' + 'how they were called.\n' + '\n' + 'The starting point for descriptor invocation is a binding, ' + '"a.x". How\n' + 'the arguments are assembled depends on "a":\n' + '\n' + 'Direct Call\n' + ' The simplest and least common call is when user code ' + 'directly\n' + ' invokes a descriptor method: "x.__get__(a)".\n' + '\n' + 'Instance Binding\n' + ' If binding to an object instance, "a.x" is transformed ' + 'into the\n' + ' call: "type(a).__dict__[\'x\'].__get__(a, type(a))".\n' + '\n' + 'Class Binding\n' + ' If binding to a class, "A.x" is transformed into the ' + 'call:\n' + ' "A.__dict__[\'x\'].__get__(None, A)".\n' + '\n' + 'Super Binding\n' + ' If "a" is an instance of "super", then the binding ' + '"super(B,\n' + ' obj).m()" searches "obj.__class__.__mro__" for the base ' + 'class "A"\n' + ' immediately preceding "B" and then invokes the ' + 'descriptor with the\n' + ' call: "A.__dict__[\'m\'].__get__(obj, obj.__class__)".\n' + '\n' + 'For instance bindings, the precedence of descriptor ' + 'invocation depends\n' + 'on the which descriptor methods are defined. A descriptor ' + 'can define\n' + 'any combination of "__get__()", "__set__()" and ' + '"__delete__()". If it\n' + 'does not define "__get__()", then accessing the attribute ' + 'will return\n' + 'the descriptor object itself unless there is a value in ' + "the object's\n" + 'instance dictionary. If the descriptor defines ' + '"__set__()" and/or\n' + '"__delete__()", it is a data descriptor; if it defines ' + 'neither, it is\n' + 'a non-data descriptor. Normally, data descriptors define ' + 'both\n' + '"__get__()" and "__set__()", while non-data descriptors ' + 'have just the\n' + '"__get__()" method. Data descriptors with "__set__()" and ' + '"__get__()"\n' + 'defined always override a redefinition in an instance ' + 'dictionary. In\n' + 'contrast, non-data descriptors can be overridden by ' + 'instances.\n' + '\n' + 'Python methods (including "staticmethod()" and ' + '"classmethod()") are\n' + 'implemented as non-data descriptors. Accordingly, ' + 'instances can\n' + 'redefine and override methods. This allows individual ' + 'instances to\n' + 'acquire behaviors that differ from other instances of the ' + 'same class.\n' + '\n' + 'The "property()" function is implemented as a data ' + 'descriptor.\n' + 'Accordingly, instances cannot override the behavior of a ' + 'property.\n' + '\n' + '\n' + '__slots__\n' + '---------\n' + '\n' + 'By default, instances of classes have a dictionary for ' + 'attribute\n' + 'storage. This wastes space for objects having very few ' + 'instance\n' + 'variables. The space consumption can become acute when ' + 'creating large\n' + 'numbers of instances.\n' + '\n' + 'The default can be overridden by defining *__slots__* in a ' + 'class\n' + 'definition. The *__slots__* declaration takes a sequence ' + 'of instance\n' + 'variables and reserves just enough space in each instance ' + 'to hold a\n' + 'value for each variable. Space is saved because ' + '*__dict__* is not\n' + 'created for each instance.\n' + '\n' + 'object.__slots__\n' + '\n' + ' This class variable can be assigned a string, iterable, ' + 'or sequence\n' + ' of strings with variable names used by instances. ' + '*__slots__*\n' + ' reserves space for the declared variables and prevents ' + 'the\n' + ' automatic creation of *__dict__* and *__weakref__* for ' + 'each\n' + ' instance.\n' + '\n' + '\n' + 'Notes on using *__slots__*\n' + '~~~~~~~~~~~~~~~~~~~~~~~~~~\n' + '\n' + '* When inheriting from a class without *__slots__*, the ' + '*__dict__*\n' + ' attribute of that class will always be accessible, so a ' + '*__slots__*\n' + ' definition in the subclass is meaningless.\n' + '\n' + '* Without a *__dict__* variable, instances cannot be ' + 'assigned new\n' + ' variables not listed in the *__slots__* definition. ' + 'Attempts to\n' + ' assign to an unlisted variable name raises ' + '"AttributeError". If\n' + ' dynamic assignment of new variables is desired, then ' + 'add\n' + ' "\'__dict__\'" to the sequence of strings in the ' + '*__slots__*\n' + ' declaration.\n' + '\n' + '* Without a *__weakref__* variable for each instance, ' + 'classes\n' + ' defining *__slots__* do not support weak references to ' + 'its\n' + ' instances. If weak reference support is needed, then ' + 'add\n' + ' "\'__weakref__\'" to the sequence of strings in the ' + '*__slots__*\n' + ' declaration.\n' + '\n' + '* *__slots__* are implemented at the class level by ' + 'creating\n' + ' descriptors (*Implementing Descriptors*) for each ' + 'variable name. As\n' + ' a result, class attributes cannot be used to set default ' + 'values for\n' + ' instance variables defined by *__slots__*; otherwise, ' + 'the class\n' + ' attribute would overwrite the descriptor assignment.\n' + '\n' + '* The action of a *__slots__* declaration is limited to ' + 'the class\n' + ' where it is defined. As a result, subclasses will have ' + 'a *__dict__*\n' + ' unless they also define *__slots__* (which must only ' + 'contain names\n' + ' of any *additional* slots).\n' + '\n' + '* If a class defines a slot also defined in a base class, ' + 'the\n' + ' instance variable defined by the base class slot is ' + 'inaccessible\n' + ' (except by retrieving its descriptor directly from the ' + 'base class).\n' + ' This renders the meaning of the program undefined. In ' + 'the future, a\n' + ' check may be added to prevent this.\n' + '\n' + '* Nonempty *__slots__* does not work for classes derived ' + 'from\n' + ' "variable-length" built-in types such as "int", "bytes" ' + 'and "tuple".\n' + '\n' + '* Any non-string iterable may be assigned to *__slots__*. ' + 'Mappings\n' + ' may also be used; however, in the future, special ' + 'meaning may be\n' + ' assigned to the values corresponding to each key.\n' + '\n' + '* *__class__* assignment works only if both classes have ' + 'the same\n' + ' *__slots__*.\n' + '\n' + '\n' + 'Customizing class creation\n' + '==========================\n' + '\n' + 'By default, classes are constructed using "type()". The ' + 'class body is\n' + 'executed in a new namespace and the class name is bound ' + 'locally to the\n' + 'result of "type(name, bases, namespace)".\n' + '\n' + 'The class creation process can be customised by passing ' + 'the\n' + '"metaclass" keyword argument in the class definition line, ' + 'or by\n' + 'inheriting from an existing class that included such an ' + 'argument. In\n' + 'the following example, both "MyClass" and "MySubclass" are ' + 'instances\n' + 'of "Meta":\n' + '\n' + ' class Meta(type):\n' + ' pass\n' + '\n' + ' class MyClass(metaclass=Meta):\n' + ' pass\n' + '\n' + ' class MySubclass(MyClass):\n' + ' pass\n' + '\n' + 'Any other keyword arguments that are specified in the ' + 'class definition\n' + 'are passed through to all metaclass operations described ' + 'below.\n' + '\n' + 'When a class definition is executed, the following steps ' + 'occur:\n' + '\n' + '* the appropriate metaclass is determined\n' + '\n' + '* the class namespace is prepared\n' + '\n' + '* the class body is executed\n' + '\n' + '* the class object is created\n' + '\n' + '\n' + 'Determining the appropriate metaclass\n' + '-------------------------------------\n' + '\n' + 'The appropriate metaclass for a class definition is ' + 'determined as\n' + 'follows:\n' + '\n' + '* if no bases and no explicit metaclass are given, then ' + '"type()" is\n' + ' used\n' + '\n' + '* if an explicit metaclass is given and it is *not* an ' + 'instance of\n' + ' "type()", then it is used directly as the metaclass\n' + '\n' + '* if an instance of "type()" is given as the explicit ' + 'metaclass, or\n' + ' bases are defined, then the most derived metaclass is ' + 'used\n' + '\n' + 'The most derived metaclass is selected from the explicitly ' + 'specified\n' + 'metaclass (if any) and the metaclasses (i.e. "type(cls)") ' + 'of all\n' + 'specified base classes. The most derived metaclass is one ' + 'which is a\n' + 'subtype of *all* of these candidate metaclasses. If none ' + 'of the\n' + 'candidate metaclasses meets that criterion, then the class ' + 'definition\n' + 'will fail with "TypeError".\n' + '\n' + '\n' + 'Preparing the class namespace\n' + '-----------------------------\n' + '\n' + 'Once the appropriate metaclass has been identified, then ' + 'the class\n' + 'namespace is prepared. If the metaclass has a ' + '"__prepare__" attribute,\n' + 'it is called as "namespace = metaclass.__prepare__(name, ' + 'bases,\n' + '**kwds)" (where the additional keyword arguments, if any, ' + 'come from\n' + 'the class definition).\n' + '\n' + 'If the metaclass has no "__prepare__" attribute, then the ' + 'class\n' + 'namespace is initialised as an empty "dict()" instance.\n' + '\n' + 'See also: **PEP 3115** - Metaclasses in Python 3000\n' + '\n' + ' Introduced the "__prepare__" namespace hook\n' + '\n' + '\n' + 'Executing the class body\n' + '------------------------\n' + '\n' + 'The class body is executed (approximately) as "exec(body, ' + 'globals(),\n' + 'namespace)". The key difference from a normal call to ' + '"exec()" is that\n' + 'lexical scoping allows the class body (including any ' + 'methods) to\n' + 'reference names from the current and outer scopes when the ' + 'class\n' + 'definition occurs inside a function.\n' + '\n' + 'However, even when the class definition occurs inside the ' + 'function,\n' + 'methods defined inside the class still cannot see names ' + 'defined at the\n' + 'class scope. Class variables must be accessed through the ' + 'first\n' + 'parameter of instance or class methods, and cannot be ' + 'accessed at all\n' + 'from static methods.\n' + '\n' + '\n' + 'Creating the class object\n' + '-------------------------\n' + '\n' + 'Once the class namespace has been populated by executing ' + 'the class\n' + 'body, the class object is created by calling ' + '"metaclass(name, bases,\n' + 'namespace, **kwds)" (the additional keywords passed here ' + 'are the same\n' + 'as those passed to "__prepare__").\n' + '\n' + 'This class object is the one that will be referenced by ' + 'the zero-\n' + 'argument form of "super()". "__class__" is an implicit ' + 'closure\n' + 'reference created by the compiler if any methods in a ' + 'class body refer\n' + 'to either "__class__" or "super". This allows the zero ' + 'argument form\n' + 'of "super()" to correctly identify the class being defined ' + 'based on\n' + 'lexical scoping, while the class or instance that was used ' + 'to make the\n' + 'current call is identified based on the first argument ' + 'passed to the\n' + 'method.\n' + '\n' + 'After the class object is created, it is passed to the ' + 'class\n' + 'decorators included in the class definition (if any) and ' + 'the resulting\n' + 'object is bound in the local namespace as the defined ' + 'class.\n' + '\n' + 'See also: **PEP 3135** - New super\n' + '\n' + ' Describes the implicit "__class__" closure reference\n' + '\n' + '\n' + 'Metaclass example\n' + '-----------------\n' + '\n' + 'The potential uses for metaclasses are boundless. Some ' + 'ideas that have\n' + 'been explored include logging, interface checking, ' + 'automatic\n' + 'delegation, automatic property creation, proxies, ' + 'frameworks, and\n' + 'automatic resource locking/synchronization.\n' + '\n' + 'Here is an example of a metaclass that uses an\n' + '"collections.OrderedDict" to remember the order that class ' + 'variables\n' + 'are defined:\n' + '\n' + ' class OrderedClass(type):\n' + '\n' + ' @classmethod\n' + ' def __prepare__(metacls, name, bases, **kwds):\n' + ' return collections.OrderedDict()\n' + '\n' + ' def __new__(cls, name, bases, namespace, **kwds):\n' + ' result = type.__new__(cls, name, bases, ' + 'dict(namespace))\n' + ' result.members = tuple(namespace)\n' + ' return result\n' + '\n' + ' class A(metaclass=OrderedClass):\n' + ' def one(self): pass\n' + ' def two(self): pass\n' + ' def three(self): pass\n' + ' def four(self): pass\n' + '\n' + ' >>> A.members\n' + " ('__module__', 'one', 'two', 'three', 'four')\n" + '\n' + 'When the class definition for *A* gets executed, the ' + 'process begins\n' + 'with calling the metaclass\'s "__prepare__()" method which ' + 'returns an\n' + 'empty "collections.OrderedDict". That mapping records the ' + 'methods and\n' + 'attributes of *A* as they are defined within the body of ' + 'the class\n' + 'statement. Once those definitions are executed, the ' + 'ordered dictionary\n' + 'is fully populated and the metaclass\'s "__new__()" method ' + 'gets\n' + 'invoked. That method builds the new type and it saves the ' + 'ordered\n' + 'dictionary keys in an attribute called "members".\n' + '\n' + '\n' + 'Customizing instance and subclass checks\n' + '========================================\n' + '\n' + 'The following methods are used to override the default ' + 'behavior of the\n' + '"isinstance()" and "issubclass()" built-in functions.\n' + '\n' + 'In particular, the metaclass "abc.ABCMeta" implements ' + 'these methods in\n' + 'order to allow the addition of Abstract Base Classes ' + '(ABCs) as\n' + '"virtual base classes" to any class or type (including ' + 'built-in\n' + 'types), including other ABCs.\n' + '\n' + 'class.__instancecheck__(self, instance)\n' + '\n' + ' Return true if *instance* should be considered a ' + '(direct or\n' + ' indirect) instance of *class*. If defined, called to ' + 'implement\n' + ' "isinstance(instance, class)".\n' + '\n' + 'class.__subclasscheck__(self, subclass)\n' + '\n' + ' Return true if *subclass* should be considered a ' + '(direct or\n' + ' indirect) subclass of *class*. If defined, called to ' + 'implement\n' + ' "issubclass(subclass, class)".\n' + '\n' + 'Note that these methods are looked up on the type ' + '(metaclass) of a\n' + 'class. They cannot be defined as class methods in the ' + 'actual class.\n' + 'This is consistent with the lookup of special methods that ' + 'are called\n' + 'on instances, only in this case the instance is itself a ' + 'class.\n' + '\n' + 'See also: **PEP 3119** - Introducing Abstract Base ' + 'Classes\n' + '\n' + ' Includes the specification for customizing ' + '"isinstance()" and\n' + ' "issubclass()" behavior through "__instancecheck__()" ' + 'and\n' + ' "__subclasscheck__()", with motivation for this ' + 'functionality in\n' + ' the context of adding Abstract Base Classes (see the ' + '"abc"\n' + ' module) to the language.\n' + '\n' + '\n' + 'Emulating callable objects\n' + '==========================\n' + '\n' + 'object.__call__(self[, args...])\n' + '\n' + ' Called when the instance is "called" as a function; if ' + 'this method\n' + ' is defined, "x(arg1, arg2, ...)" is a shorthand for\n' + ' "x.__call__(arg1, arg2, ...)".\n' + '\n' + '\n' + 'Emulating container types\n' + '=========================\n' + '\n' + 'The following methods can be defined to implement ' + 'container objects.\n' + 'Containers usually are sequences (such as lists or tuples) ' + 'or mappings\n' + '(like dictionaries), but can represent other containers as ' + 'well. The\n' + 'first set of methods is used either to emulate a sequence ' + 'or to\n' + 'emulate a mapping; the difference is that for a sequence, ' + 'the\n' + 'allowable keys should be the integers *k* for which "0 <= ' + 'k < N" where\n' + '*N* is the length of the sequence, or slice objects, which ' + 'define a\n' + 'range of items. It is also recommended that mappings ' + 'provide the\n' + 'methods "keys()", "values()", "items()", "get()", ' + '"clear()",\n' + '"setdefault()", "pop()", "popitem()", "copy()", and ' + '"update()"\n' + "behaving similar to those for Python's standard dictionary " + 'objects.\n' + 'The "collections" module provides a "MutableMapping" ' + 'abstract base\n' + 'class to help create those methods from a base set of ' + '"__getitem__()",\n' + '"__setitem__()", "__delitem__()", and "keys()". Mutable ' + 'sequences\n' + 'should provide methods "append()", "count()", "index()", ' + '"extend()",\n' + '"insert()", "pop()", "remove()", "reverse()" and "sort()", ' + 'like Python\n' + 'standard list objects. Finally, sequence types should ' + 'implement\n' + 'addition (meaning concatenation) and multiplication ' + '(meaning\n' + 'repetition) by defining the methods "__add__()", ' + '"__radd__()",\n' + '"__iadd__()", "__mul__()", "__rmul__()" and "__imul__()" ' + 'described\n' + 'below; they should not define other numerical operators. ' + 'It is\n' + 'recommended that both mappings and sequences implement ' + 'the\n' + '"__contains__()" method to allow efficient use of the "in" ' + 'operator;\n' + 'for mappings, "in" should search the mapping\'s keys; for ' + 'sequences, it\n' + 'should search through the values. It is further ' + 'recommended that both\n' + 'mappings and sequences implement the "__iter__()" method ' + 'to allow\n' + 'efficient iteration through the container; for mappings, ' + '"__iter__()"\n' + 'should be the same as "keys()"; for sequences, it should ' + 'iterate\n' + 'through the values.\n' + '\n' + 'object.__len__(self)\n' + '\n' + ' Called to implement the built-in function "len()". ' + 'Should return\n' + ' the length of the object, an integer ">=" 0. Also, an ' + 'object that\n' + ' doesn\'t define a "__bool__()" method and whose ' + '"__len__()" method\n' + ' returns zero is considered to be false in a Boolean ' + 'context.\n' + '\n' + 'object.__length_hint__(self)\n' + '\n' + ' Called to implement "operator.length_hint()". Should ' + 'return an\n' + ' estimated length for the object (which may be greater ' + 'or less than\n' + ' the actual length). The length must be an integer ">=" ' + '0. This\n' + ' method is purely an optimization and is never required ' + 'for\n' + ' correctness.\n' + '\n' + ' New in version 3.4.\n' + '\n' + 'Note: Slicing is done exclusively with the following three ' + 'methods.\n' + ' A call like\n' + '\n' + ' a[1:2] = b\n' + '\n' + ' is translated to\n' + '\n' + ' a[slice(1, 2, None)] = b\n' + '\n' + ' and so forth. Missing slice items are always filled in ' + 'with "None".\n' + '\n' + 'object.__getitem__(self, key)\n' + '\n' + ' Called to implement evaluation of "self[key]". For ' + 'sequence types,\n' + ' the accepted keys should be integers and slice ' + 'objects. Note that\n' + ' the special interpretation of negative indexes (if the ' + 'class wishes\n' + ' to emulate a sequence type) is up to the ' + '"__getitem__()" method. If\n' + ' *key* is of an inappropriate type, "TypeError" may be ' + 'raised; if of\n' + ' a value outside the set of indexes for the sequence ' + '(after any\n' + ' special interpretation of negative values), ' + '"IndexError" should be\n' + ' raised. For mapping types, if *key* is missing (not in ' + 'the\n' + ' container), "KeyError" should be raised.\n' + '\n' + ' Note: "for" loops expect that an "IndexError" will be ' + 'raised for\n' + ' illegal indexes to allow proper detection of the end ' + 'of the\n' + ' sequence.\n' + '\n' + 'object.__missing__(self, key)\n' + '\n' + ' Called by "dict"."__getitem__()" to implement ' + '"self[key]" for dict\n' + ' subclasses when key is not in the dictionary.\n' + '\n' + 'object.__setitem__(self, key, value)\n' + '\n' + ' Called to implement assignment to "self[key]". Same ' + 'note as for\n' + ' "__getitem__()". This should only be implemented for ' + 'mappings if\n' + ' the objects support changes to the values for keys, or ' + 'if new keys\n' + ' can be added, or for sequences if elements can be ' + 'replaced. The\n' + ' same exceptions should be raised for improper *key* ' + 'values as for\n' + ' the "__getitem__()" method.\n' + '\n' + 'object.__delitem__(self, key)\n' + '\n' + ' Called to implement deletion of "self[key]". Same note ' + 'as for\n' + ' "__getitem__()". This should only be implemented for ' + 'mappings if\n' + ' the objects support removal of keys, or for sequences ' + 'if elements\n' + ' can be removed from the sequence. The same exceptions ' + 'should be\n' + ' raised for improper *key* values as for the ' + '"__getitem__()" method.\n' + '\n' + 'object.__iter__(self)\n' + '\n' + ' This method is called when an iterator is required for ' + 'a container.\n' + ' This method should return a new iterator object that ' + 'can iterate\n' + ' over all the objects in the container. For mappings, ' + 'it should\n' + ' iterate over the keys of the container.\n' + '\n' + ' Iterator objects also need to implement this method; ' + 'they are\n' + ' required to return themselves. For more information on ' + 'iterator\n' + ' objects, see *Iterator Types*.\n' + '\n' + 'object.__reversed__(self)\n' + '\n' + ' Called (if present) by the "reversed()" built-in to ' + 'implement\n' + ' reverse iteration. It should return a new iterator ' + 'object that\n' + ' iterates over all the objects in the container in ' + 'reverse order.\n' + '\n' + ' If the "__reversed__()" method is not provided, the ' + '"reversed()"\n' + ' built-in will fall back to using the sequence protocol ' + '("__len__()"\n' + ' and "__getitem__()"). Objects that support the ' + 'sequence protocol\n' + ' should only provide "__reversed__()" if they can ' + 'provide an\n' + ' implementation that is more efficient than the one ' + 'provided by\n' + ' "reversed()".\n' + '\n' + 'The membership test operators ("in" and "not in") are ' + 'normally\n' + 'implemented as an iteration through a sequence. However, ' + 'container\n' + 'objects can supply the following special method with a ' + 'more efficient\n' + 'implementation, which also does not require the object be ' + 'a sequence.\n' + '\n' + 'object.__contains__(self, item)\n' + '\n' + ' Called to implement membership test operators. Should ' + 'return true\n' + ' if *item* is in *self*, false otherwise. For mapping ' + 'objects, this\n' + ' should consider the keys of the mapping rather than the ' + 'values or\n' + ' the key-item pairs.\n' + '\n' + ' For objects that don\'t define "__contains__()", the ' + 'membership test\n' + ' first tries iteration via "__iter__()", then the old ' + 'sequence\n' + ' iteration protocol via "__getitem__()", see *this ' + 'section in the\n' + ' language reference*.\n' + '\n' + '\n' + 'Emulating numeric types\n' + '=======================\n' + '\n' + 'The following methods can be defined to emulate numeric ' + 'objects.\n' + 'Methods corresponding to operations that are not supported ' + 'by the\n' + 'particular kind of number implemented (e.g., bitwise ' + 'operations for\n' + 'non-integral numbers) should be left undefined.\n' + '\n' + 'object.__add__(self, other)\n' + 'object.__sub__(self, other)\n' + 'object.__mul__(self, other)\n' + 'object.__matmul__(self, other)\n' + 'object.__truediv__(self, other)\n' + 'object.__floordiv__(self, other)\n' + 'object.__mod__(self, other)\n' + 'object.__divmod__(self, other)\n' + 'object.__pow__(self, other[, modulo])\n' + 'object.__lshift__(self, other)\n' + 'object.__rshift__(self, other)\n' + 'object.__and__(self, other)\n' + 'object.__xor__(self, other)\n' + 'object.__or__(self, other)\n' + '\n' + ' These methods are called to implement the binary ' + 'arithmetic\n' + ' operations ("+", "-", "*", "@", "/", "//", "%", ' + '"divmod()",\n' + ' "pow()", "**", "<<", ">>", "&", "^", "|"). For ' + 'instance, to\n' + ' evaluate the expression "x + y", where *x* is an ' + 'instance of a\n' + ' class that has an "__add__()" method, "x.__add__(y)" is ' + 'called.\n' + ' The "__divmod__()" method should be the equivalent to ' + 'using\n' + ' "__floordiv__()" and "__mod__()"; it should not be ' + 'related to\n' + ' "__truediv__()". Note that "__pow__()" should be ' + 'defined to accept\n' + ' an optional third argument if the ternary version of ' + 'the built-in\n' + ' "pow()" function is to be supported.\n' + '\n' + ' If one of those methods does not support the operation ' + 'with the\n' + ' supplied arguments, it should return "NotImplemented".\n' + '\n' + 'object.__radd__(self, other)\n' + 'object.__rsub__(self, other)\n' + 'object.__rmul__(self, other)\n' + 'object.__rmatmul__(self, other)\n' + 'object.__rtruediv__(self, other)\n' + 'object.__rfloordiv__(self, other)\n' + 'object.__rmod__(self, other)\n' + 'object.__rdivmod__(self, other)\n' + 'object.__rpow__(self, other)\n' + 'object.__rlshift__(self, other)\n' + 'object.__rrshift__(self, other)\n' + 'object.__rand__(self, other)\n' + 'object.__rxor__(self, other)\n' + 'object.__ror__(self, other)\n' + '\n' + ' These methods are called to implement the binary ' + 'arithmetic\n' + ' operations ("+", "-", "*", "@", "/", "//", "%", ' + '"divmod()",\n' + ' "pow()", "**", "<<", ">>", "&", "^", "|") with ' + 'reflected (swapped)\n' + ' operands. These functions are only called if the left ' + 'operand does\n' + ' not support the corresponding operation and the ' + 'operands are of\n' + ' different types. [2] For instance, to evaluate the ' + 'expression "x -\n' + ' y", where *y* is an instance of a class that has an ' + '"__rsub__()"\n' + ' method, "y.__rsub__(x)" is called if "x.__sub__(y)" ' + 'returns\n' + ' *NotImplemented*.\n' + '\n' + ' Note that ternary "pow()" will not try calling ' + '"__rpow__()" (the\n' + ' coercion rules would become too complicated).\n' + '\n' + " Note: If the right operand's type is a subclass of the " + 'left\n' + " operand's type and that subclass provides the " + 'reflected method\n' + ' for the operation, this method will be called before ' + 'the left\n' + " operand's non-reflected method. This behavior allows " + 'subclasses\n' + " to override their ancestors' operations.\n" + '\n' + 'object.__iadd__(self, other)\n' + 'object.__isub__(self, other)\n' + 'object.__imul__(self, other)\n' + 'object.__imatmul__(self, other)\n' + 'object.__itruediv__(self, other)\n' + 'object.__ifloordiv__(self, other)\n' + 'object.__imod__(self, other)\n' + 'object.__ipow__(self, other[, modulo])\n' + 'object.__ilshift__(self, other)\n' + 'object.__irshift__(self, other)\n' + 'object.__iand__(self, other)\n' + 'object.__ixor__(self, other)\n' + 'object.__ior__(self, other)\n' + '\n' + ' These methods are called to implement the augmented ' + 'arithmetic\n' + ' assignments ("+=", "-=", "*=", "@=", "/=", "//=", "%=", ' + '"**=",\n' + ' "<<=", ">>=", "&=", "^=", "|="). These methods should ' + 'attempt to\n' + ' do the operation in-place (modifying *self*) and return ' + 'the result\n' + ' (which could be, but does not have to be, *self*). If ' + 'a specific\n' + ' method is not defined, the augmented assignment falls ' + 'back to the\n' + ' normal methods. For instance, if *x* is an instance of ' + 'a class\n' + ' with an "__iadd__()" method, "x += y" is equivalent to ' + '"x =\n' + ' x.__iadd__(y)" . Otherwise, "x.__add__(y)" and ' + '"y.__radd__(x)" are\n' + ' considered, as with the evaluation of "x + y". In ' + 'certain\n' + ' situations, augmented assignment can result in ' + 'unexpected errors\n' + " (see *Why does a_tuple[i] += ['item'] raise an " + 'exception when the\n' + ' addition works?*), but this behavior is in fact part of ' + 'the data\n' + ' model.\n' + '\n' + 'object.__neg__(self)\n' + 'object.__pos__(self)\n' + 'object.__abs__(self)\n' + 'object.__invert__(self)\n' + '\n' + ' Called to implement the unary arithmetic operations ' + '("-", "+",\n' + ' "abs()" and "~").\n' + '\n' + 'object.__complex__(self)\n' + 'object.__int__(self)\n' + 'object.__float__(self)\n' + 'object.__round__(self[, n])\n' + '\n' + ' Called to implement the built-in functions "complex()", ' + '"int()",\n' + ' "float()" and "round()". Should return a value of the ' + 'appropriate\n' + ' type.\n' + '\n' + 'object.__index__(self)\n' + '\n' + ' Called to implement "operator.index()", and whenever ' + 'Python needs\n' + ' to losslessly convert the numeric object to an integer ' + 'object (such\n' + ' as in slicing, or in the built-in "bin()", "hex()" and ' + '"oct()"\n' + ' functions). Presence of this method indicates that the ' + 'numeric\n' + ' object is an integer type. Must return an integer.\n' + '\n' + ' Note: In order to have a coherent integer type class, ' + 'when\n' + ' "__index__()" is defined "__int__()" should also be ' + 'defined, and\n' + ' both should return the same value.\n' + '\n' + '\n' + 'With Statement Context Managers\n' + '===============================\n' + '\n' + 'A *context manager* is an object that defines the runtime ' + 'context to\n' + 'be established when executing a "with" statement. The ' + 'context manager\n' + 'handles the entry into, and the exit from, the desired ' + 'runtime context\n' + 'for the execution of the block of code. Context managers ' + 'are normally\n' + 'invoked using the "with" statement (described in section ' + '*The with\n' + 'statement*), but can also be used by directly invoking ' + 'their methods.\n' + '\n' + 'Typical uses of context managers include saving and ' + 'restoring various\n' + 'kinds of global state, locking and unlocking resources, ' + 'closing opened\n' + 'files, etc.\n' + '\n' + 'For more information on context managers, see *Context ' + 'Manager Types*.\n' + '\n' + 'object.__enter__(self)\n' + '\n' + ' Enter the runtime context related to this object. The ' + '"with"\n' + " statement will bind this method's return value to the " + 'target(s)\n' + ' specified in the "as" clause of the statement, if any.\n' + '\n' + 'object.__exit__(self, exc_type, exc_value, traceback)\n' + '\n' + ' Exit the runtime context related to this object. The ' + 'parameters\n' + ' describe the exception that caused the context to be ' + 'exited. If the\n' + ' context was exited without an exception, all three ' + 'arguments will\n' + ' be "None".\n' + '\n' + ' If an exception is supplied, and the method wishes to ' + 'suppress the\n' + ' exception (i.e., prevent it from being propagated), it ' + 'should\n' + ' return a true value. Otherwise, the exception will be ' + 'processed\n' + ' normally upon exit from this method.\n' + '\n' + ' Note that "__exit__()" methods should not reraise the ' + 'passed-in\n' + " exception; this is the caller's responsibility.\n" + '\n' + 'See also: **PEP 0343** - The "with" statement\n' + '\n' + ' The specification, background, and examples for the ' + 'Python "with"\n' + ' statement.\n' + '\n' + '\n' + 'Special method lookup\n' + '=====================\n' + '\n' + 'For custom classes, implicit invocations of special ' + 'methods are only\n' + "guaranteed to work correctly if defined on an object's " + 'type, not in\n' + "the object's instance dictionary. That behaviour is the " + 'reason why\n' + 'the following code raises an exception:\n' + '\n' + ' >>> class C:\n' + ' ... pass\n' + ' ...\n' + ' >>> c = C()\n' + ' >>> c.__len__ = lambda: 5\n' + ' >>> len(c)\n' + ' Traceback (most recent call last):\n' + ' File "", line 1, in \n' + " TypeError: object of type 'C' has no len()\n" + '\n' + 'The rationale behind this behaviour lies with a number of ' + 'special\n' + 'methods such as "__hash__()" and "__repr__()" that are ' + 'implemented by\n' + 'all objects, including type objects. If the implicit ' + 'lookup of these\n' + 'methods used the conventional lookup process, they would ' + 'fail when\n' + 'invoked on the type object itself:\n' + '\n' + ' >>> 1 .__hash__() == hash(1)\n' + ' True\n' + ' >>> int.__hash__() == hash(int)\n' + ' Traceback (most recent call last):\n' + ' File "", line 1, in \n' + " TypeError: descriptor '__hash__' of 'int' object needs " + 'an argument\n' + '\n' + 'Incorrectly attempting to invoke an unbound method of a ' + 'class in this\n' + "way is sometimes referred to as 'metaclass confusion', and " + 'is avoided\n' + 'by bypassing the instance when looking up special ' + 'methods:\n' + '\n' + ' >>> type(1).__hash__(1) == hash(1)\n' + ' True\n' + ' >>> type(int).__hash__(int) == hash(int)\n' + ' True\n' + '\n' + 'In addition to bypassing any instance attributes in the ' + 'interest of\n' + 'correctness, implicit special method lookup generally also ' + 'bypasses\n' + 'the "__getattribute__()" method even of the object\'s ' + 'metaclass:\n' + '\n' + ' >>> class Meta(type):\n' + ' ... def __getattribute__(*args):\n' + ' ... print("Metaclass getattribute invoked")\n' + ' ... return type.__getattribute__(*args)\n' + ' ...\n' + ' >>> class C(object, metaclass=Meta):\n' + ' ... def __len__(self):\n' + ' ... return 10\n' + ' ... def __getattribute__(*args):\n' + ' ... print("Class getattribute invoked")\n' + ' ... return object.__getattribute__(*args)\n' + ' ...\n' + ' >>> c = C()\n' + ' >>> c.__len__() # Explicit lookup via ' + 'instance\n' + ' Class getattribute invoked\n' + ' 10\n' + ' >>> type(c).__len__(c) # Explicit lookup via ' + 'type\n' + ' Metaclass getattribute invoked\n' + ' 10\n' + ' >>> len(c) # Implicit lookup\n' + ' 10\n' + '\n' + 'Bypassing the "__getattribute__()" machinery in this ' + 'fashion provides\n' + 'significant scope for speed optimisations within the ' + 'interpreter, at\n' + 'the cost of some flexibility in the handling of special ' + 'methods (the\n' + 'special method *must* be set on the class object itself in ' + 'order to be\n' + 'consistently invoked by the interpreter).\n', + 'string-methods': '\n' + 'String Methods\n' + '**************\n' + '\n' + 'Strings implement all of the *common* sequence ' + 'operations, along with\n' + 'the additional methods described below.\n' + '\n' + 'Strings also support two styles of string formatting, ' + 'one providing a\n' + 'large degree of flexibility and customization (see ' + '"str.format()",\n' + '*Format String Syntax* and *String Formatting*) and the ' + 'other based on\n' + 'C "printf" style formatting that handles a narrower ' + 'range of types and\n' + 'is slightly harder to use correctly, but is often faster ' + 'for the cases\n' + 'it can handle (*printf-style String Formatting*).\n' + '\n' + 'The *Text Processing Services* section of the standard ' + 'library covers\n' + 'a number of other modules that provide various text ' + 'related utilities\n' + '(including regular expression support in the "re" ' + 'module).\n' + '\n' + 'str.capitalize()\n' + '\n' + ' Return a copy of the string with its first character ' + 'capitalized\n' + ' and the rest lowercased.\n' + '\n' + 'str.casefold()\n' + '\n' + ' Return a casefolded copy of the string. Casefolded ' + 'strings may be\n' + ' used for caseless matching.\n' + '\n' + ' Casefolding is similar to lowercasing but more ' + 'aggressive because\n' + ' it is intended to remove all case distinctions in a ' + 'string. For\n' + ' example, the German lowercase letter "\'?\'" is ' + 'equivalent to ""ss"".\n' + ' Since it is already lowercase, "lower()" would do ' + 'nothing to "\'?\'";\n' + ' "casefold()" converts it to ""ss"".\n' + '\n' + ' The casefolding algorithm is described in section ' + '3.13 of the\n' + ' Unicode Standard.\n' + '\n' + ' New in version 3.3.\n' + '\n' + 'str.center(width[, fillchar])\n' + '\n' + ' Return centered in a string of length *width*. ' + 'Padding is done\n' + ' using the specified *fillchar* (default is an ASCII ' + 'space). The\n' + ' original string is returned if *width* is less than ' + 'or equal to\n' + ' "len(s)".\n' + '\n' + 'str.count(sub[, start[, end]])\n' + '\n' + ' Return the number of non-overlapping occurrences of ' + 'substring *sub*\n' + ' in the range [*start*, *end*]. Optional arguments ' + '*start* and\n' + ' *end* are interpreted as in slice notation.\n' + '\n' + 'str.encode(encoding="utf-8", errors="strict")\n' + '\n' + ' Return an encoded version of the string as a bytes ' + 'object. Default\n' + ' encoding is "\'utf-8\'". *errors* may be given to set ' + 'a different\n' + ' error handling scheme. The default for *errors* is ' + '"\'strict\'",\n' + ' meaning that encoding errors raise a "UnicodeError". ' + 'Other possible\n' + ' values are "\'ignore\'", "\'replace\'", ' + '"\'xmlcharrefreplace\'",\n' + ' "\'backslashreplace\'" and any other name registered ' + 'via\n' + ' "codecs.register_error()", see section *Error ' + 'Handlers*. For a list\n' + ' of possible encodings, see section *Standard ' + 'Encodings*.\n' + '\n' + ' Changed in version 3.1: Support for keyword arguments ' + 'added.\n' + '\n' + 'str.endswith(suffix[, start[, end]])\n' + '\n' + ' Return "True" if the string ends with the specified ' + '*suffix*,\n' + ' otherwise return "False". *suffix* can also be a ' + 'tuple of suffixes\n' + ' to look for. With optional *start*, test beginning ' + 'at that\n' + ' position. With optional *end*, stop comparing at ' + 'that position.\n' + '\n' + 'str.expandtabs(tabsize=8)\n' + '\n' + ' Return a copy of the string where all tab characters ' + 'are replaced\n' + ' by one or more spaces, depending on the current ' + 'column and the\n' + ' given tab size. Tab positions occur every *tabsize* ' + 'characters\n' + ' (default is 8, giving tab positions at columns 0, 8, ' + '16 and so on).\n' + ' To expand the string, the current column is set to ' + 'zero and the\n' + ' string is examined character by character. If the ' + 'character is a\n' + ' tab ("\\t"), one or more space characters are ' + 'inserted in the result\n' + ' until the current column is equal to the next tab ' + 'position. (The\n' + ' tab character itself is not copied.) If the ' + 'character is a newline\n' + ' ("\\n") or return ("\\r"), it is copied and the ' + 'current column is\n' + ' reset to zero. Any other character is copied ' + 'unchanged and the\n' + ' current column is incremented by one regardless of ' + 'how the\n' + ' character is represented when printed.\n' + '\n' + " >>> '01\\t012\\t0123\\t01234'.expandtabs()\n" + " '01 012 0123 01234'\n" + " >>> '01\\t012\\t0123\\t01234'.expandtabs(4)\n" + " '01 012 0123 01234'\n" + '\n' + 'str.find(sub[, start[, end]])\n' + '\n' + ' Return the lowest index in the string where substring ' + '*sub* is\n' + ' found, such that *sub* is contained in the slice ' + '"s[start:end]".\n' + ' Optional arguments *start* and *end* are interpreted ' + 'as in slice\n' + ' notation. Return "-1" if *sub* is not found.\n' + '\n' + ' Note: The "find()" method should be used only if you ' + 'need to know\n' + ' the position of *sub*. To check if *sub* is a ' + 'substring or not,\n' + ' use the "in" operator:\n' + '\n' + " >>> 'Py' in 'Python'\n" + ' True\n' + '\n' + 'str.format(*args, **kwargs)\n' + '\n' + ' Perform a string formatting operation. The string on ' + 'which this\n' + ' method is called can contain literal text or ' + 'replacement fields\n' + ' delimited by braces "{}". Each replacement field ' + 'contains either\n' + ' the numeric index of a positional argument, or the ' + 'name of a\n' + ' keyword argument. Returns a copy of the string where ' + 'each\n' + ' replacement field is replaced with the string value ' + 'of the\n' + ' corresponding argument.\n' + '\n' + ' >>> "The sum of 1 + 2 is {0}".format(1+2)\n' + " 'The sum of 1 + 2 is 3'\n" + '\n' + ' See *Format String Syntax* for a description of the ' + 'various\n' + ' formatting options that can be specified in format ' + 'strings.\n' + '\n' + 'str.format_map(mapping)\n' + '\n' + ' Similar to "str.format(**mapping)", except that ' + '"mapping" is used\n' + ' directly and not copied to a "dict". This is useful ' + 'if for example\n' + ' "mapping" is a dict subclass:\n' + '\n' + ' >>> class Default(dict):\n' + ' ... def __missing__(self, key):\n' + ' ... return key\n' + ' ...\n' + " >>> '{name} was born in " + "{country}'.format_map(Default(name='Guido'))\n" + " 'Guido was born in country'\n" + '\n' + ' New in version 3.2.\n' + '\n' + 'str.index(sub[, start[, end]])\n' + '\n' + ' Like "find()", but raise "ValueError" when the ' + 'substring is not\n' + ' found.\n' + '\n' + 'str.isalnum()\n' + '\n' + ' Return true if all characters in the string are ' + 'alphanumeric and\n' + ' there is at least one character, false otherwise. A ' + 'character "c"\n' + ' is alphanumeric if one of the following returns ' + '"True":\n' + ' "c.isalpha()", "c.isdecimal()", "c.isdigit()", or ' + '"c.isnumeric()".\n' + '\n' + 'str.isalpha()\n' + '\n' + ' Return true if all characters in the string are ' + 'alphabetic and\n' + ' there is at least one character, false otherwise. ' + 'Alphabetic\n' + ' characters are those characters defined in the ' + 'Unicode character\n' + ' database as "Letter", i.e., those with general ' + 'category property\n' + ' being one of "Lm", "Lt", "Lu", "Ll", or "Lo". Note ' + 'that this is\n' + ' different from the "Alphabetic" property defined in ' + 'the Unicode\n' + ' Standard.\n' + '\n' + 'str.isdecimal()\n' + '\n' + ' Return true if all characters in the string are ' + 'decimal characters\n' + ' and there is at least one character, false otherwise. ' + 'Decimal\n' + ' characters are those from general category "Nd". This ' + 'category\n' + ' includes digit characters, and all characters that ' + 'can be used to\n' + ' form decimal-radix numbers, e.g. U+0660, ARABIC-INDIC ' + 'DIGIT ZERO.\n' + '\n' + 'str.isdigit()\n' + '\n' + ' Return true if all characters in the string are ' + 'digits and there is\n' + ' at least one character, false otherwise. Digits ' + 'include decimal\n' + ' characters and digits that need special handling, ' + 'such as the\n' + ' compatibility superscript digits. Formally, a digit ' + 'is a character\n' + ' that has the property value Numeric_Type=Digit or\n' + ' Numeric_Type=Decimal.\n' + '\n' + 'str.isidentifier()\n' + '\n' + ' Return true if the string is a valid identifier ' + 'according to the\n' + ' language definition, section *Identifiers and ' + 'keywords*.\n' + '\n' + ' Use "keyword.iskeyword()" to test for reserved ' + 'identifiers such as\n' + ' "def" and "class".\n' + '\n' + 'str.islower()\n' + '\n' + ' Return true if all cased characters [4] in the string ' + 'are lowercase\n' + ' and there is at least one cased character, false ' + 'otherwise.\n' + '\n' + 'str.isnumeric()\n' + '\n' + ' Return true if all characters in the string are ' + 'numeric characters,\n' + ' and there is at least one character, false otherwise. ' + 'Numeric\n' + ' characters include digit characters, and all ' + 'characters that have\n' + ' the Unicode numeric value property, e.g. U+2155, ' + 'VULGAR FRACTION\n' + ' ONE FIFTH. Formally, numeric characters are those ' + 'with the\n' + ' property value Numeric_Type=Digit, ' + 'Numeric_Type=Decimal or\n' + ' Numeric_Type=Numeric.\n' + '\n' + 'str.isprintable()\n' + '\n' + ' Return true if all characters in the string are ' + 'printable or the\n' + ' string is empty, false otherwise. Nonprintable ' + 'characters are\n' + ' those characters defined in the Unicode character ' + 'database as\n' + ' "Other" or "Separator", excepting the ASCII space ' + '(0x20) which is\n' + ' considered printable. (Note that printable ' + 'characters in this\n' + ' context are those which should not be escaped when ' + '"repr()" is\n' + ' invoked on a string. It has no bearing on the ' + 'handling of strings\n' + ' written to "sys.stdout" or "sys.stderr".)\n' + '\n' + 'str.isspace()\n' + '\n' + ' Return true if there are only whitespace characters ' + 'in the string\n' + ' and there is at least one character, false ' + 'otherwise. Whitespace\n' + ' characters are those characters defined in the ' + 'Unicode character\n' + ' database as "Other" or "Separator" and those with ' + 'bidirectional\n' + ' property being one of "WS", "B", or "S".\n' + '\n' + 'str.istitle()\n' + '\n' + ' Return true if the string is a titlecased string and ' + 'there is at\n' + ' least one character, for example uppercase characters ' + 'may only\n' + ' follow uncased characters and lowercase characters ' + 'only cased ones.\n' + ' Return false otherwise.\n' + '\n' + 'str.isupper()\n' + '\n' + ' Return true if all cased characters [4] in the string ' + 'are uppercase\n' + ' and there is at least one cased character, false ' + 'otherwise.\n' + '\n' + 'str.join(iterable)\n' + '\n' + ' Return a string which is the concatenation of the ' + 'strings in the\n' + ' *iterable* *iterable*. A "TypeError" will be raised ' + 'if there are\n' + ' any non-string values in *iterable*, including ' + '"bytes" objects.\n' + ' The separator between elements is the string ' + 'providing this method.\n' + '\n' + 'str.ljust(width[, fillchar])\n' + '\n' + ' Return the string left justified in a string of ' + 'length *width*.\n' + ' Padding is done using the specified *fillchar* ' + '(default is an ASCII\n' + ' space). The original string is returned if *width* is ' + 'less than or\n' + ' equal to "len(s)".\n' + '\n' + 'str.lower()\n' + '\n' + ' Return a copy of the string with all the cased ' + 'characters [4]\n' + ' converted to lowercase.\n' + '\n' + ' The lowercasing algorithm used is described in ' + 'section 3.13 of the\n' + ' Unicode Standard.\n' + '\n' + 'str.lstrip([chars])\n' + '\n' + ' Return a copy of the string with leading characters ' + 'removed. The\n' + ' *chars* argument is a string specifying the set of ' + 'characters to be\n' + ' removed. If omitted or "None", the *chars* argument ' + 'defaults to\n' + ' removing whitespace. The *chars* argument is not a ' + 'prefix; rather,\n' + ' all combinations of its values are stripped:\n' + '\n' + " >>> ' spacious '.lstrip()\n" + " 'spacious '\n" + " >>> 'www.example.com'.lstrip('cmowz.')\n" + " 'example.com'\n" + '\n' + 'static str.maketrans(x[, y[, z]])\n' + '\n' + ' This static method returns a translation table usable ' + 'for\n' + ' "str.translate()".\n' + '\n' + ' If there is only one argument, it must be a ' + 'dictionary mapping\n' + ' Unicode ordinals (integers) or characters (strings of ' + 'length 1) to\n' + ' Unicode ordinals, strings (of arbitrary lengths) or ' + 'None.\n' + ' Character keys will then be converted to ordinals.\n' + '\n' + ' If there are two arguments, they must be strings of ' + 'equal length,\n' + ' and in the resulting dictionary, each character in x ' + 'will be mapped\n' + ' to the character at the same position in y. If there ' + 'is a third\n' + ' argument, it must be a string, whose characters will ' + 'be mapped to\n' + ' None in the result.\n' + '\n' + 'str.partition(sep)\n' + '\n' + ' Split the string at the first occurrence of *sep*, ' + 'and return a\n' + ' 3-tuple containing the part before the separator, the ' + 'separator\n' + ' itself, and the part after the separator. If the ' + 'separator is not\n' + ' found, return a 3-tuple containing the string itself, ' + 'followed by\n' + ' two empty strings.\n' + '\n' + 'str.replace(old, new[, count])\n' + '\n' + ' Return a copy of the string with all occurrences of ' + 'substring *old*\n' + ' replaced by *new*. If the optional argument *count* ' + 'is given, only\n' + ' the first *count* occurrences are replaced.\n' + '\n' + 'str.rfind(sub[, start[, end]])\n' + '\n' + ' Return the highest index in the string where ' + 'substring *sub* is\n' + ' found, such that *sub* is contained within ' + '"s[start:end]".\n' + ' Optional arguments *start* and *end* are interpreted ' + 'as in slice\n' + ' notation. Return "-1" on failure.\n' + '\n' + 'str.rindex(sub[, start[, end]])\n' + '\n' + ' Like "rfind()" but raises "ValueError" when the ' + 'substring *sub* is\n' + ' not found.\n' + '\n' + 'str.rjust(width[, fillchar])\n' + '\n' + ' Return the string right justified in a string of ' + 'length *width*.\n' + ' Padding is done using the specified *fillchar* ' + '(default is an ASCII\n' + ' space). The original string is returned if *width* is ' + 'less than or\n' + ' equal to "len(s)".\n' + '\n' + 'str.rpartition(sep)\n' + '\n' + ' Split the string at the last occurrence of *sep*, and ' + 'return a\n' + ' 3-tuple containing the part before the separator, the ' + 'separator\n' + ' itself, and the part after the separator. If the ' + 'separator is not\n' + ' found, return a 3-tuple containing two empty strings, ' + 'followed by\n' + ' the string itself.\n' + '\n' + 'str.rsplit(sep=None, maxsplit=-1)\n' + '\n' + ' Return a list of the words in the string, using *sep* ' + 'as the\n' + ' delimiter string. If *maxsplit* is given, at most ' + '*maxsplit* splits\n' + ' are done, the *rightmost* ones. If *sep* is not ' + 'specified or\n' + ' "None", any whitespace string is a separator. Except ' + 'for splitting\n' + ' from the right, "rsplit()" behaves like "split()" ' + 'which is\n' + ' described in detail below.\n' + '\n' + 'str.rstrip([chars])\n' + '\n' + ' Return a copy of the string with trailing characters ' + 'removed. The\n' + ' *chars* argument is a string specifying the set of ' + 'characters to be\n' + ' removed. If omitted or "None", the *chars* argument ' + 'defaults to\n' + ' removing whitespace. The *chars* argument is not a ' + 'suffix; rather,\n' + ' all combinations of its values are stripped:\n' + '\n' + " >>> ' spacious '.rstrip()\n" + " ' spacious'\n" + " >>> 'mississippi'.rstrip('ipz')\n" + " 'mississ'\n" + '\n' + 'str.split(sep=None, maxsplit=-1)\n' + '\n' + ' Return a list of the words in the string, using *sep* ' + 'as the\n' + ' delimiter string. If *maxsplit* is given, at most ' + '*maxsplit*\n' + ' splits are done (thus, the list will have at most ' + '"maxsplit+1"\n' + ' elements). If *maxsplit* is not specified or "-1", ' + 'then there is\n' + ' no limit on the number of splits (all possible splits ' + 'are made).\n' + '\n' + ' If *sep* is given, consecutive delimiters are not ' + 'grouped together\n' + ' and are deemed to delimit empty strings (for ' + 'example,\n' + ' "\'1,,2\'.split(\',\')" returns "[\'1\', \'\', ' + '\'2\']"). The *sep* argument\n' + ' may consist of multiple characters (for example,\n' + ' "\'1<>2<>3\'.split(\'<>\')" returns "[\'1\', \'2\', ' + '\'3\']"). Splitting an\n' + ' empty string with a specified separator returns ' + '"[\'\']".\n' + '\n' + ' For example:\n' + '\n' + " >>> '1,2,3'.split(',')\n" + " ['1', '2', '3']\n" + " >>> '1,2,3'.split(',', maxsplit=1)\n" + " ['1', '2,3']\n" + " >>> '1,2,,3,'.split(',')\n" + " ['1', '2', '', '3', '']\n" + '\n' + ' If *sep* is not specified or is "None", a different ' + 'splitting\n' + ' algorithm is applied: runs of consecutive whitespace ' + 'are regarded\n' + ' as a single separator, and the result will contain no ' + 'empty strings\n' + ' at the start or end if the string has leading or ' + 'trailing\n' + ' whitespace. Consequently, splitting an empty string ' + 'or a string\n' + ' consisting of just whitespace with a "None" separator ' + 'returns "[]".\n' + '\n' + ' For example:\n' + '\n' + " >>> '1 2 3'.split()\n" + " ['1', '2', '3']\n" + " >>> '1 2 3'.split(maxsplit=1)\n" + " ['1', '2 3']\n" + " >>> ' 1 2 3 '.split()\n" + " ['1', '2', '3']\n" + '\n' + 'str.splitlines([keepends])\n' + '\n' + ' Return a list of the lines in the string, breaking at ' + 'line\n' + ' boundaries. Line breaks are not included in the ' + 'resulting list\n' + ' unless *keepends* is given and true.\n' + '\n' + ' This method splits on the following line boundaries. ' + 'In\n' + ' particular, the boundaries are a superset of ' + '*universal newlines*.\n' + '\n' + ' ' + '+-------------------------+-------------------------------+\n' + ' | Representation | ' + 'Description |\n' + ' ' + '+=========================+===============================+\n' + ' | "\\n" | Line ' + 'Feed |\n' + ' ' + '+-------------------------+-------------------------------+\n' + ' | "\\r" | Carriage ' + 'Return |\n' + ' ' + '+-------------------------+-------------------------------+\n' + ' | "\\r\\n" | Carriage Return + Line ' + 'Feed |\n' + ' ' + '+-------------------------+-------------------------------+\n' + ' | "\\v" or "\\x0b" | Line ' + 'Tabulation |\n' + ' ' + '+-------------------------+-------------------------------+\n' + ' | "\\f" or "\\x0c" | Form ' + 'Feed |\n' + ' ' + '+-------------------------+-------------------------------+\n' + ' | "\\x1c" | File ' + 'Separator |\n' + ' ' + '+-------------------------+-------------------------------+\n' + ' | "\\x1d" | Group ' + 'Separator |\n' + ' ' + '+-------------------------+-------------------------------+\n' + ' | "\\x1e" | Record ' + 'Separator |\n' + ' ' + '+-------------------------+-------------------------------+\n' + ' | "\\x85" | Next Line (C1 Control ' + 'Code) |\n' + ' ' + '+-------------------------+-------------------------------+\n' + ' | "\\u2028" | Line ' + 'Separator |\n' + ' ' + '+-------------------------+-------------------------------+\n' + ' | "\\u2029" | Paragraph ' + 'Separator |\n' + ' ' + '+-------------------------+-------------------------------+\n' + '\n' + ' Changed in version 3.2: "\\v" and "\\f" added to list ' + 'of line\n' + ' boundaries.\n' + '\n' + ' For example:\n' + '\n' + " >>> 'ab c\\n\\nde fg\\rkl\\r\\n'.splitlines()\n" + " ['ab c', '', 'de fg', 'kl']\n" + " >>> 'ab c\\n\\nde " + "fg\\rkl\\r\\n'.splitlines(keepends=True)\n" + " ['ab c\\n', '\\n', 'de fg\\r', 'kl\\r\\n']\n" + '\n' + ' Unlike "split()" when a delimiter string *sep* is ' + 'given, this\n' + ' method returns an empty list for the empty string, ' + 'and a terminal\n' + ' line break does not result in an extra line:\n' + '\n' + ' >>> "".splitlines()\n' + ' []\n' + ' >>> "One line\\n".splitlines()\n' + " ['One line']\n" + '\n' + ' For comparison, "split(\'\\n\')" gives:\n' + '\n' + " >>> ''.split('\\n')\n" + " ['']\n" + " >>> 'Two lines\\n'.split('\\n')\n" + " ['Two lines', '']\n" + '\n' + 'str.startswith(prefix[, start[, end]])\n' + '\n' + ' Return "True" if string starts with the *prefix*, ' + 'otherwise return\n' + ' "False". *prefix* can also be a tuple of prefixes to ' + 'look for.\n' + ' With optional *start*, test string beginning at that ' + 'position.\n' + ' With optional *end*, stop comparing string at that ' + 'position.\n' + '\n' + 'str.strip([chars])\n' + '\n' + ' Return a copy of the string with the leading and ' + 'trailing\n' + ' characters removed. The *chars* argument is a string ' + 'specifying the\n' + ' set of characters to be removed. If omitted or ' + '"None", the *chars*\n' + ' argument defaults to removing whitespace. The *chars* ' + 'argument is\n' + ' not a prefix or suffix; rather, all combinations of ' + 'its values are\n' + ' stripped:\n' + '\n' + " >>> ' spacious '.strip()\n" + " 'spacious'\n" + " >>> 'www.example.com'.strip('cmowz.')\n" + " 'example'\n" + '\n' + ' The outermost leading and trailing *chars* argument ' + 'values are\n' + ' stripped from the string. Characters are removed from ' + 'the leading\n' + ' end until reaching a string character that is not ' + 'contained in the\n' + ' set of characters in *chars*. A similar action takes ' + 'place on the\n' + ' trailing end. For example:\n' + '\n' + " >>> comment_string = '#....... Section 3.2.1 Issue " + "#32 .......'\n" + " >>> comment_string.strip('.#! ')\n" + " 'Section 3.2.1 Issue #32'\n" + '\n' + 'str.swapcase()\n' + '\n' + ' Return a copy of the string with uppercase characters ' + 'converted to\n' + ' lowercase and vice versa. Note that it is not ' + 'necessarily true that\n' + ' "s.swapcase().swapcase() == s".\n' + '\n' + 'str.title()\n' + '\n' + ' Return a titlecased version of the string where words ' + 'start with an\n' + ' uppercase character and the remaining characters are ' + 'lowercase.\n' + '\n' + ' For example:\n' + '\n' + " >>> 'Hello world'.title()\n" + " 'Hello World'\n" + '\n' + ' The algorithm uses a simple language-independent ' + 'definition of a\n' + ' word as groups of consecutive letters. The ' + 'definition works in\n' + ' many contexts but it means that apostrophes in ' + 'contractions and\n' + ' possessives form word boundaries, which may not be ' + 'the desired\n' + ' result:\n' + '\n' + ' >>> "they\'re bill\'s friends from the ' + 'UK".title()\n' + ' "They\'Re Bill\'S Friends From The Uk"\n' + '\n' + ' A workaround for apostrophes can be constructed using ' + 'regular\n' + ' expressions:\n' + '\n' + ' >>> import re\n' + ' >>> def titlecase(s):\n' + ' ... return re.sub(r"[A-Za-z]+(\'[A-Za-z]+)?",\n' + ' ... lambda mo: ' + 'mo.group(0)[0].upper() +\n' + ' ... ' + 'mo.group(0)[1:].lower(),\n' + ' ... s)\n' + ' ...\n' + ' >>> titlecase("they\'re bill\'s friends.")\n' + ' "They\'re Bill\'s Friends."\n' + '\n' + 'str.translate(table)\n' + '\n' + ' Return a copy of the string in which each character ' + 'has been mapped\n' + ' through the given translation table. The table must ' + 'be an object\n' + ' that implements indexing via "__getitem__()", ' + 'typically a *mapping*\n' + ' or *sequence*. When indexed by a Unicode ordinal (an ' + 'integer), the\n' + ' table object can do any of the following: return a ' + 'Unicode ordinal\n' + ' or a string, to map the character to one or more ' + 'other characters;\n' + ' return "None", to delete the character from the ' + 'return string; or\n' + ' raise a "LookupError" exception, to map the character ' + 'to itself.\n' + '\n' + ' You can use "str.maketrans()" to create a translation ' + 'map from\n' + ' character-to-character mappings in different ' + 'formats.\n' + '\n' + ' See also the "codecs" module for a more flexible ' + 'approach to custom\n' + ' character mappings.\n' + '\n' + 'str.upper()\n' + '\n' + ' Return a copy of the string with all the cased ' + 'characters [4]\n' + ' converted to uppercase. Note that ' + '"str.upper().isupper()" might be\n' + ' "False" if "s" contains uncased characters or if the ' + 'Unicode\n' + ' category of the resulting character(s) is not "Lu" ' + '(Letter,\n' + ' uppercase), but e.g. "Lt" (Letter, titlecase).\n' + '\n' + ' The uppercasing algorithm used is described in ' + 'section 3.13 of the\n' + ' Unicode Standard.\n' + '\n' + 'str.zfill(width)\n' + '\n' + ' Return a copy of the string left filled with ASCII ' + '"\'0\'" digits to\n' + ' make a string of length *width*. A leading sign ' + 'prefix\n' + ' ("\'+\'"/"\'-\'") is handled by inserting the padding ' + '*after* the sign\n' + ' character rather than before. The original string is ' + 'returned if\n' + ' *width* is less than or equal to "len(s)".\n' + '\n' + ' For example:\n' + '\n' + ' >>> "42".zfill(5)\n' + " '00042'\n" + ' >>> "-42".zfill(5)\n' + " '-0042'\n", + 'strings': '\n' + 'String and Bytes literals\n' + '*************************\n' + '\n' + 'String literals are described by the following lexical ' + 'definitions:\n' + '\n' + ' stringliteral ::= [stringprefix](shortstring | ' + 'longstring)\n' + ' stringprefix ::= "r" | "u" | "R" | "U"\n' + ' shortstring ::= "\'" shortstringitem* "\'" | \'"\' ' + 'shortstringitem* \'"\'\n' + ' longstring ::= "\'\'\'" longstringitem* "\'\'\'" | ' + '\'"""\' longstringitem* \'"""\'\n' + ' shortstringitem ::= shortstringchar | stringescapeseq\n' + ' longstringitem ::= longstringchar | stringescapeseq\n' + ' shortstringchar ::= \n' + ' longstringchar ::= \n' + ' stringescapeseq ::= "\\" \n' + '\n' + ' bytesliteral ::= bytesprefix(shortbytes | longbytes)\n' + ' bytesprefix ::= "b" | "B" | "br" | "Br" | "bR" | "BR" | ' + '"rb" | "rB" | "Rb" | "RB"\n' + ' shortbytes ::= "\'" shortbytesitem* "\'" | \'"\' ' + 'shortbytesitem* \'"\'\n' + ' longbytes ::= "\'\'\'" longbytesitem* "\'\'\'" | ' + '\'"""\' longbytesitem* \'"""\'\n' + ' shortbytesitem ::= shortbyteschar | bytesescapeseq\n' + ' longbytesitem ::= longbyteschar | bytesescapeseq\n' + ' shortbyteschar ::= \n' + ' longbyteschar ::= \n' + ' bytesescapeseq ::= "\\" \n' + '\n' + 'One syntactic restriction not indicated by these productions is ' + 'that\n' + 'whitespace is not allowed between the "stringprefix" or ' + '"bytesprefix"\n' + 'and the rest of the literal. The source character set is ' + 'defined by\n' + 'the encoding declaration; it is UTF-8 if no encoding ' + 'declaration is\n' + 'given in the source file; see section *Encoding declarations*.\n' + '\n' + 'In plain English: Both types of literals can be enclosed in ' + 'matching\n' + 'single quotes ("\'") or double quotes ("""). They can also be ' + 'enclosed\n' + 'in matching groups of three single or double quotes (these are\n' + 'generally referred to as *triple-quoted strings*). The ' + 'backslash\n' + '("\\") character is used to escape characters that otherwise ' + 'have a\n' + 'special meaning, such as newline, backslash itself, or the ' + 'quote\n' + 'character.\n' + '\n' + 'Bytes literals are always prefixed with "\'b\'" or "\'B\'"; ' + 'they produce\n' + 'an instance of the "bytes" type instead of the "str" type. ' + 'They may\n' + 'only contain ASCII characters; bytes with a numeric value of ' + '128 or\n' + 'greater must be expressed with escapes.\n' + '\n' + 'As of Python 3.3 it is possible again to prefix string literals ' + 'with a\n' + '"u" prefix to simplify maintenance of dual 2.x and 3.x ' + 'codebases.\n' + '\n' + 'Both string and bytes literals may optionally be prefixed with ' + 'a\n' + 'letter "\'r\'" or "\'R\'"; such strings are called *raw ' + 'strings* and treat\n' + 'backslashes as literal characters. As a result, in string ' + 'literals,\n' + '"\'\\U\'" and "\'\\u\'" escapes in raw strings are not treated ' + 'specially.\n' + "Given that Python 2.x's raw unicode literals behave differently " + 'than\n' + 'Python 3.x\'s the "\'ur\'" syntax is not supported.\n' + '\n' + 'New in version 3.3: The "\'rb\'" prefix of raw bytes literals ' + 'has been\n' + 'added as a synonym of "\'br\'".\n' + '\n' + 'New in version 3.3: Support for the unicode legacy literal\n' + '("u\'value\'") was reintroduced to simplify the maintenance of ' + 'dual\n' + 'Python 2.x and 3.x codebases. See **PEP 414** for more ' + 'information.\n' + '\n' + 'In triple-quoted literals, unescaped newlines and quotes are ' + 'allowed\n' + '(and are retained), except that three unescaped quotes in a ' + 'row\n' + 'terminate the literal. (A "quote" is the character used to ' + 'open the\n' + 'literal, i.e. either "\'" or """.)\n' + '\n' + 'Unless an "\'r\'" or "\'R\'" prefix is present, escape ' + 'sequences in string\n' + 'and bytes literals are interpreted according to rules similar ' + 'to those\n' + 'used by Standard C. The recognized escape sequences are:\n' + '\n' + '+-------------------+-----------------------------------+---------+\n' + '| Escape Sequence | Meaning | ' + 'Notes |\n' + '+===================+===================================+=========+\n' + '| "\\newline" | Backslash and newline ignored ' + '| |\n' + '+-------------------+-----------------------------------+---------+\n' + '| "\\\\" | Backslash ("\\") ' + '| |\n' + '+-------------------+-----------------------------------+---------+\n' + '| "\\\'" | Single quote ("\'") ' + '| |\n' + '+-------------------+-----------------------------------+---------+\n' + '| "\\"" | Double quote (""") ' + '| |\n' + '+-------------------+-----------------------------------+---------+\n' + '| "\\a" | ASCII Bell (BEL) ' + '| |\n' + '+-------------------+-----------------------------------+---------+\n' + '| "\\b" | ASCII Backspace (BS) ' + '| |\n' + '+-------------------+-----------------------------------+---------+\n' + '| "\\f" | ASCII Formfeed (FF) ' + '| |\n' + '+-------------------+-----------------------------------+---------+\n' + '| "\\n" | ASCII Linefeed (LF) ' + '| |\n' + '+-------------------+-----------------------------------+---------+\n' + '| "\\r" | ASCII Carriage Return (CR) ' + '| |\n' + '+-------------------+-----------------------------------+---------+\n' + '| "\\t" | ASCII Horizontal Tab (TAB) ' + '| |\n' + '+-------------------+-----------------------------------+---------+\n' + '| "\\v" | ASCII Vertical Tab (VT) ' + '| |\n' + '+-------------------+-----------------------------------+---------+\n' + '| "\\ooo" | Character with octal value *ooo* | ' + '(1,3) |\n' + '+-------------------+-----------------------------------+---------+\n' + '| "\\xhh" | Character with hex value *hh* | ' + '(2,3) |\n' + '+-------------------+-----------------------------------+---------+\n' + '\n' + 'Escape sequences only recognized in string literals are:\n' + '\n' + '+-------------------+-----------------------------------+---------+\n' + '| Escape Sequence | Meaning | ' + 'Notes |\n' + '+===================+===================================+=========+\n' + '| "\\N{name}" | Character named *name* in the | ' + '(4) |\n' + '| | Unicode database ' + '| |\n' + '+-------------------+-----------------------------------+---------+\n' + '| "\\uxxxx" | Character with 16-bit hex value | ' + '(5) |\n' + '| | *xxxx* ' + '| |\n' + '+-------------------+-----------------------------------+---------+\n' + '| "\\Uxxxxxxxx" | Character with 32-bit hex value | ' + '(6) |\n' + '| | *xxxxxxxx* ' + '| |\n' + '+-------------------+-----------------------------------+---------+\n' + '\n' + 'Notes:\n' + '\n' + '1. As in Standard C, up to three octal digits are accepted.\n' + '\n' + '2. Unlike in Standard C, exactly two hex digits are required.\n' + '\n' + '3. In a bytes literal, hexadecimal and octal escapes denote ' + 'the\n' + ' byte with the given value. In a string literal, these ' + 'escapes\n' + ' denote a Unicode character with the given value.\n' + '\n' + '4. Changed in version 3.3: Support for name aliases [1] has ' + 'been\n' + ' added.\n' + '\n' + '5. Individual code units which form parts of a surrogate pair ' + 'can\n' + ' be encoded using this escape sequence. Exactly four hex ' + 'digits are\n' + ' required.\n' + '\n' + '6. Any Unicode character can be encoded this way. Exactly ' + 'eight\n' + ' hex digits are required.\n' + '\n' + 'Unlike Standard C, all unrecognized escape sequences are left ' + 'in the\n' + 'string unchanged, i.e., *the backslash is left in the result*. ' + '(This\n' + 'behavior is useful when debugging: if an escape sequence is ' + 'mistyped,\n' + 'the resulting output is more easily recognized as broken.) It ' + 'is also\n' + 'important to note that the escape sequences only recognized in ' + 'string\n' + 'literals fall into the category of unrecognized escapes for ' + 'bytes\n' + 'literals.\n' + '\n' + 'Even in a raw literal, quotes can be escaped with a backslash, ' + 'but the\n' + 'backslash remains in the result; for example, "r"\\""" is a ' + 'valid\n' + 'string literal consisting of two characters: a backslash and a ' + 'double\n' + 'quote; "r"\\"" is not a valid string literal (even a raw string ' + 'cannot\n' + 'end in an odd number of backslashes). Specifically, *a raw ' + 'literal\n' + 'cannot end in a single backslash* (since the backslash would ' + 'escape\n' + 'the following quote character). Note also that a single ' + 'backslash\n' + 'followed by a newline is interpreted as those two characters as ' + 'part\n' + 'of the literal, *not* as a line continuation.\n', + 'subscriptions': '\n' + 'Subscriptions\n' + '*************\n' + '\n' + 'A subscription selects an item of a sequence (string, ' + 'tuple or list)\n' + 'or mapping (dictionary) object:\n' + '\n' + ' subscription ::= primary "[" expression_list "]"\n' + '\n' + 'The primary must evaluate to an object that supports ' + 'subscription\n' + '(lists or dictionaries for example). User-defined ' + 'objects can support\n' + 'subscription by defining a "__getitem__()" method.\n' + '\n' + 'For built-in objects, there are two types of objects that ' + 'support\n' + 'subscription:\n' + '\n' + 'If the primary is a mapping, the expression list must ' + 'evaluate to an\n' + 'object whose value is one of the keys of the mapping, and ' + 'the\n' + 'subscription selects the value in the mapping that ' + 'corresponds to that\n' + 'key. (The expression list is a tuple except if it has ' + 'exactly one\n' + 'item.)\n' + '\n' + 'If the primary is a sequence, the expression (list) must ' + 'evaluate to\n' + 'an integer or a slice (as discussed in the following ' + 'section).\n' + '\n' + 'The formal syntax makes no special provision for negative ' + 'indices in\n' + 'sequences; however, built-in sequences all provide a ' + '"__getitem__()"\n' + 'method that interprets negative indices by adding the ' + 'length of the\n' + 'sequence to the index (so that "x[-1]" selects the last ' + 'item of "x").\n' + 'The resulting value must be a nonnegative integer less ' + 'than the number\n' + 'of items in the sequence, and the subscription selects ' + 'the item whose\n' + 'index is that value (counting from zero). Since the ' + 'support for\n' + "negative indices and slicing occurs in the object's " + '"__getitem__()"\n' + 'method, subclasses overriding this method will need to ' + 'explicitly add\n' + 'that support.\n' + '\n' + "A string's items are characters. A character is not a " + 'separate data\n' + 'type but a string of exactly one character.\n', + 'truth': '\n' + 'Truth Value Testing\n' + '*******************\n' + '\n' + 'Any object can be tested for truth value, for use in an "if" or\n' + '"while" condition or as operand of the Boolean operations below. ' + 'The\n' + 'following values are considered false:\n' + '\n' + '* "None"\n' + '\n' + '* "False"\n' + '\n' + '* zero of any numeric type, for example, "0", "0.0", "0j".\n' + '\n' + '* any empty sequence, for example, "\'\'", "()", "[]".\n' + '\n' + '* any empty mapping, for example, "{}".\n' + '\n' + '* instances of user-defined classes, if the class defines a\n' + ' "__bool__()" or "__len__()" method, when that method returns ' + 'the\n' + ' integer zero or "bool" value "False". [1]\n' + '\n' + 'All other values are considered true --- so objects of many types ' + 'are\n' + 'always true.\n' + '\n' + 'Operations and built-in functions that have a Boolean result ' + 'always\n' + 'return "0" or "False" for false and "1" or "True" for true, ' + 'unless\n' + 'otherwise stated. (Important exception: the Boolean operations ' + '"or"\n' + 'and "and" always return one of their operands.)\n', + 'try': '\n' + 'The "try" statement\n' + '*******************\n' + '\n' + 'The "try" statement specifies exception handlers and/or cleanup ' + 'code\n' + 'for a group of statements:\n' + '\n' + ' try_stmt ::= try1_stmt | try2_stmt\n' + ' try1_stmt ::= "try" ":" suite\n' + ' ("except" [expression ["as" identifier]] ":" ' + 'suite)+\n' + ' ["else" ":" suite]\n' + ' ["finally" ":" suite]\n' + ' try2_stmt ::= "try" ":" suite\n' + ' "finally" ":" suite\n' + '\n' + 'The "except" clause(s) specify one or more exception handlers. When ' + 'no\n' + 'exception occurs in the "try" clause, no exception handler is\n' + 'executed. When an exception occurs in the "try" suite, a search for ' + 'an\n' + 'exception handler is started. This search inspects the except ' + 'clauses\n' + 'in turn until one is found that matches the exception. An ' + 'expression-\n' + 'less except clause, if present, must be last; it matches any\n' + 'exception. For an except clause with an expression, that ' + 'expression\n' + 'is evaluated, and the clause matches the exception if the ' + 'resulting\n' + 'object is "compatible" with the exception. An object is ' + 'compatible\n' + 'with an exception if it is the class or a base class of the ' + 'exception\n' + 'object or a tuple containing an item compatible with the ' + 'exception.\n' + '\n' + 'If no except clause matches the exception, the search for an ' + 'exception\n' + 'handler continues in the surrounding code and on the invocation ' + 'stack.\n' + '[1]\n' + '\n' + 'If the evaluation of an expression in the header of an except ' + 'clause\n' + 'raises an exception, the original search for a handler is canceled ' + 'and\n' + 'a search starts for the new exception in the surrounding code and ' + 'on\n' + 'the call stack (it is treated as if the entire "try" statement ' + 'raised\n' + 'the exception).\n' + '\n' + 'When a matching except clause is found, the exception is assigned ' + 'to\n' + 'the target specified after the "as" keyword in that except clause, ' + 'if\n' + "present, and the except clause's suite is executed. All except\n" + 'clauses must have an executable block. When the end of this block ' + 'is\n' + 'reached, execution continues normally after the entire try ' + 'statement.\n' + '(This means that if two nested handlers exist for the same ' + 'exception,\n' + 'and the exception occurs in the try clause of the inner handler, ' + 'the\n' + 'outer handler will not handle the exception.)\n' + '\n' + 'When an exception has been assigned using "as target", it is ' + 'cleared\n' + 'at the end of the except clause. This is as if\n' + '\n' + ' except E as N:\n' + ' foo\n' + '\n' + 'was translated to\n' + '\n' + ' except E as N:\n' + ' try:\n' + ' foo\n' + ' finally:\n' + ' del N\n' + '\n' + 'This means the exception must be assigned to a different name to ' + 'be\n' + 'able to refer to it after the except clause. Exceptions are ' + 'cleared\n' + 'because with the traceback attached to them, they form a reference\n' + 'cycle with the stack frame, keeping all locals in that frame alive\n' + 'until the next garbage collection occurs.\n' + '\n' + "Before an except clause's suite is executed, details about the\n" + 'exception are stored in the "sys" module and can be accessed via\n' + '"sys.exc_info()". "sys.exc_info()" returns a 3-tuple consisting of ' + 'the\n' + 'exception class, the exception instance and a traceback object ' + '(see\n' + 'section *The standard type hierarchy*) identifying the point in ' + 'the\n' + 'program where the exception occurred. "sys.exc_info()" values are\n' + 'restored to their previous values (before the call) when returning\n' + 'from a function that handled an exception.\n' + '\n' + 'The optional "else" clause is executed if and when control flows ' + 'off\n' + 'the end of the "try" clause. [2] Exceptions in the "else" clause ' + 'are\n' + 'not handled by the preceding "except" clauses.\n' + '\n' + 'If "finally" is present, it specifies a \'cleanup\' handler. The ' + '"try"\n' + 'clause is executed, including any "except" and "else" clauses. If ' + 'an\n' + 'exception occurs in any of the clauses and is not handled, the\n' + 'exception is temporarily saved. The "finally" clause is executed. ' + 'If\n' + 'there is a saved exception it is re-raised at the end of the ' + '"finally"\n' + 'clause. If the "finally" clause raises another exception, the ' + 'saved\n' + 'exception is set as the context of the new exception. If the ' + '"finally"\n' + 'clause executes a "return" or "break" statement, the saved ' + 'exception\n' + 'is discarded:\n' + '\n' + ' >>> def f():\n' + ' ... try:\n' + ' ... 1/0\n' + ' ... finally:\n' + ' ... return 42\n' + ' ...\n' + ' >>> f()\n' + ' 42\n' + '\n' + 'The exception information is not available to the program during\n' + 'execution of the "finally" clause.\n' + '\n' + 'When a "return", "break" or "continue" statement is executed in ' + 'the\n' + '"try" suite of a "try"..."finally" statement, the "finally" clause ' + 'is\n' + 'also executed \'on the way out.\' A "continue" statement is illegal ' + 'in\n' + 'the "finally" clause. (The reason is a problem with the current\n' + 'implementation --- this restriction may be lifted in the future).\n' + '\n' + 'The return value of a function is determined by the last "return"\n' + 'statement executed. Since the "finally" clause always executes, a\n' + '"return" statement executed in the "finally" clause will always be ' + 'the\n' + 'last one executed:\n' + '\n' + ' >>> def foo():\n' + ' ... try:\n' + " ... return 'try'\n" + ' ... finally:\n' + " ... return 'finally'\n" + ' ...\n' + ' >>> foo()\n' + " 'finally'\n" + '\n' + 'Additional information on exceptions can be found in section\n' + '*Exceptions*, and information on using the "raise" statement to\n' + 'generate exceptions may be found in section *The raise statement*.\n', + 'types': '\n' + 'The standard type hierarchy\n' + '***************************\n' + '\n' + 'Below is a list of the types that are built into Python. ' + 'Extension\n' + 'modules (written in C, Java, or other languages, depending on ' + 'the\n' + 'implementation) can define additional types. Future versions of\n' + 'Python may add types to the type hierarchy (e.g., rational ' + 'numbers,\n' + 'efficiently stored arrays of integers, etc.), although such ' + 'additions\n' + 'will often be provided via the standard library instead.\n' + '\n' + 'Some of the type descriptions below contain a paragraph listing\n' + "'special attributes.' These are attributes that provide access " + 'to the\n' + 'implementation and are not intended for general use. Their ' + 'definition\n' + 'may change in the future.\n' + '\n' + 'None\n' + ' This type has a single value. There is a single object with ' + 'this\n' + ' value. This object is accessed through the built-in name ' + '"None". It\n' + ' is used to signify the absence of a value in many situations, ' + 'e.g.,\n' + " it is returned from functions that don't explicitly return\n" + ' anything. Its truth value is false.\n' + '\n' + 'NotImplemented\n' + ' This type has a single value. There is a single object with ' + 'this\n' + ' value. This object is accessed through the built-in name\n' + ' "NotImplemented". Numeric methods and rich comparison methods\n' + ' should return this value if they do not implement the ' + 'operation for\n' + ' the operands provided. (The interpreter will then try the\n' + ' reflected operation, or some other fallback, depending on the\n' + ' operator.) Its truth value is true.\n' + '\n' + ' See *Implementing the arithmetic operations* for more ' + 'details.\n' + '\n' + 'Ellipsis\n' + ' This type has a single value. There is a single object with ' + 'this\n' + ' value. This object is accessed through the literal "..." or ' + 'the\n' + ' built-in name "Ellipsis". Its truth value is true.\n' + '\n' + '"numbers.Number"\n' + ' These are created by numeric literals and returned as results ' + 'by\n' + ' arithmetic operators and arithmetic built-in functions. ' + 'Numeric\n' + ' objects are immutable; once created their value never ' + 'changes.\n' + ' Python numbers are of course strongly related to mathematical\n' + ' numbers, but subject to the limitations of numerical ' + 'representation\n' + ' in computers.\n' + '\n' + ' Python distinguishes between integers, floating point numbers, ' + 'and\n' + ' complex numbers:\n' + '\n' + ' "numbers.Integral"\n' + ' These represent elements from the mathematical set of ' + 'integers\n' + ' (positive and negative).\n' + '\n' + ' There are two types of integers:\n' + '\n' + ' Integers ("int")\n' + '\n' + ' These represent numbers in an unlimited range, subject ' + 'to\n' + ' available (virtual) memory only. For the purpose of ' + 'shift\n' + ' and mask operations, a binary representation is assumed, ' + 'and\n' + " negative numbers are represented in a variant of 2's\n" + ' complement which gives the illusion of an infinite ' + 'string of\n' + ' sign bits extending to the left.\n' + '\n' + ' Booleans ("bool")\n' + ' These represent the truth values False and True. The ' + 'two\n' + ' objects representing the values "False" and "True" are ' + 'the\n' + ' only Boolean objects. The Boolean type is a subtype of ' + 'the\n' + ' integer type, and Boolean values behave like the values ' + '0 and\n' + ' 1, respectively, in almost all contexts, the exception ' + 'being\n' + ' that when converted to a string, the strings ""False"" ' + 'or\n' + ' ""True"" are returned, respectively.\n' + '\n' + ' The rules for integer representation are intended to give ' + 'the\n' + ' most meaningful interpretation of shift and mask ' + 'operations\n' + ' involving negative integers.\n' + '\n' + ' "numbers.Real" ("float")\n' + ' These represent machine-level double precision floating ' + 'point\n' + ' numbers. You are at the mercy of the underlying machine\n' + ' architecture (and C or Java implementation) for the ' + 'accepted\n' + ' range and handling of overflow. Python does not support ' + 'single-\n' + ' precision floating point numbers; the savings in processor ' + 'and\n' + ' memory usage that are usually the reason for using these ' + 'are\n' + ' dwarfed by the overhead of using objects in Python, so ' + 'there is\n' + ' no reason to complicate the language with two kinds of ' + 'floating\n' + ' point numbers.\n' + '\n' + ' "numbers.Complex" ("complex")\n' + ' These represent complex numbers as a pair of machine-level\n' + ' double precision floating point numbers. The same caveats ' + 'apply\n' + ' as for floating point numbers. The real and imaginary parts ' + 'of a\n' + ' complex number "z" can be retrieved through the read-only\n' + ' attributes "z.real" and "z.imag".\n' + '\n' + 'Sequences\n' + ' These represent finite ordered sets indexed by non-negative\n' + ' numbers. The built-in function "len()" returns the number of ' + 'items\n' + ' of a sequence. When the length of a sequence is *n*, the index ' + 'set\n' + ' contains the numbers 0, 1, ..., *n*-1. Item *i* of sequence ' + '*a* is\n' + ' selected by "a[i]".\n' + '\n' + ' Sequences also support slicing: "a[i:j]" selects all items ' + 'with\n' + ' index *k* such that *i* "<=" *k* "<" *j*. When used as an\n' + ' expression, a slice is a sequence of the same type. This ' + 'implies\n' + ' that the index set is renumbered so that it starts at 0.\n' + '\n' + ' Some sequences also support "extended slicing" with a third ' + '"step"\n' + ' parameter: "a[i:j:k]" selects all items of *a* with index *x* ' + 'where\n' + ' "x = i + n*k", *n* ">=" "0" and *i* "<=" *x* "<" *j*.\n' + '\n' + ' Sequences are distinguished according to their mutability:\n' + '\n' + ' Immutable sequences\n' + ' An object of an immutable sequence type cannot change once ' + 'it is\n' + ' created. (If the object contains references to other ' + 'objects,\n' + ' these other objects may be mutable and may be changed; ' + 'however,\n' + ' the collection of objects directly referenced by an ' + 'immutable\n' + ' object cannot change.)\n' + '\n' + ' The following types are immutable sequences:\n' + '\n' + ' Strings\n' + ' A string is a sequence of values that represent Unicode ' + 'code\n' + ' points. All the code points in the range "U+0000 - ' + 'U+10FFFF"\n' + " can be represented in a string. Python doesn't have a " + '"char"\n' + ' type; instead, every code point in the string is ' + 'represented\n' + ' as a string object with length "1". The built-in ' + 'function\n' + ' "ord()" converts a code point from its string form to ' + 'an\n' + ' integer in the range "0 - 10FFFF"; "chr()" converts an\n' + ' integer in the range "0 - 10FFFF" to the corresponding ' + 'length\n' + ' "1" string object. "str.encode()" can be used to convert ' + 'a\n' + ' "str" to "bytes" using the given text encoding, and\n' + ' "bytes.decode()" can be used to achieve the opposite.\n' + '\n' + ' Tuples\n' + ' The items of a tuple are arbitrary Python objects. ' + 'Tuples of\n' + ' two or more items are formed by comma-separated lists ' + 'of\n' + " expressions. A tuple of one item (a 'singleton') can " + 'be\n' + ' formed by affixing a comma to an expression (an ' + 'expression by\n' + ' itself does not create a tuple, since parentheses must ' + 'be\n' + ' usable for grouping of expressions). An empty tuple can ' + 'be\n' + ' formed by an empty pair of parentheses.\n' + '\n' + ' Bytes\n' + ' A bytes object is an immutable array. The items are ' + '8-bit\n' + ' bytes, represented by integers in the range 0 <= x < ' + '256.\n' + ' Bytes literals (like "b\'abc\'") and the built-in ' + 'function\n' + ' "bytes()" can be used to construct bytes objects. ' + 'Also,\n' + ' bytes objects can be decoded to strings via the ' + '"decode()"\n' + ' method.\n' + '\n' + ' Mutable sequences\n' + ' Mutable sequences can be changed after they are created. ' + 'The\n' + ' subscription and slicing notations can be used as the ' + 'target of\n' + ' assignment and "del" (delete) statements.\n' + '\n' + ' There are currently two intrinsic mutable sequence types:\n' + '\n' + ' Lists\n' + ' The items of a list are arbitrary Python objects. Lists ' + 'are\n' + ' formed by placing a comma-separated list of expressions ' + 'in\n' + ' square brackets. (Note that there are no special cases ' + 'needed\n' + ' to form lists of length 0 or 1.)\n' + '\n' + ' Byte Arrays\n' + ' A bytearray object is a mutable array. They are created ' + 'by\n' + ' the built-in "bytearray()" constructor. Aside from ' + 'being\n' + ' mutable (and hence unhashable), byte arrays otherwise ' + 'provide\n' + ' the same interface and functionality as immutable bytes\n' + ' objects.\n' + '\n' + ' The extension module "array" provides an additional example ' + 'of a\n' + ' mutable sequence type, as does the "collections" module.\n' + '\n' + 'Set types\n' + ' These represent unordered, finite sets of unique, immutable\n' + ' objects. As such, they cannot be indexed by any subscript. ' + 'However,\n' + ' they can be iterated over, and the built-in function "len()"\n' + ' returns the number of items in a set. Common uses for sets are ' + 'fast\n' + ' membership testing, removing duplicates from a sequence, and\n' + ' computing mathematical operations such as intersection, ' + 'union,\n' + ' difference, and symmetric difference.\n' + '\n' + ' For set elements, the same immutability rules apply as for\n' + ' dictionary keys. Note that numeric types obey the normal rules ' + 'for\n' + ' numeric comparison: if two numbers compare equal (e.g., "1" ' + 'and\n' + ' "1.0"), only one of them can be contained in a set.\n' + '\n' + ' There are currently two intrinsic set types:\n' + '\n' + ' Sets\n' + ' These represent a mutable set. They are created by the ' + 'built-in\n' + ' "set()" constructor and can be modified afterwards by ' + 'several\n' + ' methods, such as "add()".\n' + '\n' + ' Frozen sets\n' + ' These represent an immutable set. They are created by the\n' + ' built-in "frozenset()" constructor. As a frozenset is ' + 'immutable\n' + ' and *hashable*, it can be used again as an element of ' + 'another\n' + ' set, or as a dictionary key.\n' + '\n' + 'Mappings\n' + ' These represent finite sets of objects indexed by arbitrary ' + 'index\n' + ' sets. The subscript notation "a[k]" selects the item indexed ' + 'by "k"\n' + ' from the mapping "a"; this can be used in expressions and as ' + 'the\n' + ' target of assignments or "del" statements. The built-in ' + 'function\n' + ' "len()" returns the number of items in a mapping.\n' + '\n' + ' There is currently a single intrinsic mapping type:\n' + '\n' + ' Dictionaries\n' + ' These represent finite sets of objects indexed by nearly\n' + ' arbitrary values. The only types of values not acceptable ' + 'as\n' + ' keys are values containing lists or dictionaries or other\n' + ' mutable types that are compared by value rather than by ' + 'object\n' + ' identity, the reason being that the efficient ' + 'implementation of\n' + " dictionaries requires a key's hash value to remain " + 'constant.\n' + ' Numeric types used for keys obey the normal rules for ' + 'numeric\n' + ' comparison: if two numbers compare equal (e.g., "1" and ' + '"1.0")\n' + ' then they can be used interchangeably to index the same\n' + ' dictionary entry.\n' + '\n' + ' Dictionaries are mutable; they can be created by the ' + '"{...}"\n' + ' notation (see section *Dictionary displays*).\n' + '\n' + ' The extension modules "dbm.ndbm" and "dbm.gnu" provide\n' + ' additional examples of mapping types, as does the ' + '"collections"\n' + ' module.\n' + '\n' + 'Callable types\n' + ' These are the types to which the function call operation (see\n' + ' section *Calls*) can be applied:\n' + '\n' + ' User-defined functions\n' + ' A user-defined function object is created by a function\n' + ' definition (see section *Function definitions*). It should ' + 'be\n' + ' called with an argument list containing the same number of ' + 'items\n' + " as the function's formal parameter list.\n" + '\n' + ' Special attributes:\n' + '\n' + ' ' + '+---------------------------+---------------------------------+-------------+\n' + ' | Attribute | ' + 'Meaning | |\n' + ' ' + '+===========================+=================================+=============+\n' + ' | "__doc__" | The function\'s ' + 'documentation | Writable |\n' + ' | | string, or "None" ' + 'if | |\n' + ' | | unavailable; not inherited ' + 'by | |\n' + ' | | ' + 'subclasses | |\n' + ' ' + '+---------------------------+---------------------------------+-------------+\n' + ' | "__name__" | The function\'s ' + 'name | Writable |\n' + ' ' + '+---------------------------+---------------------------------+-------------+\n' + ' | "__qualname__" | The function\'s *qualified ' + 'name* | Writable |\n' + ' | | New in version ' + '3.3. | |\n' + ' ' + '+---------------------------+---------------------------------+-------------+\n' + ' | "__module__" | The name of the module ' + 'the | Writable |\n' + ' | | function was defined in, ' + 'or | |\n' + ' | | "None" if ' + 'unavailable. | |\n' + ' ' + '+---------------------------+---------------------------------+-------------+\n' + ' | "__defaults__" | A tuple containing ' + 'default | Writable |\n' + ' | | argument values for ' + 'those | |\n' + ' | | arguments that have ' + 'defaults, | |\n' + ' | | or "None" if no arguments ' + 'have | |\n' + ' | | a default ' + 'value | |\n' + ' ' + '+---------------------------+---------------------------------+-------------+\n' + ' | "__code__" | The code object ' + 'representing | Writable |\n' + ' | | the compiled function ' + 'body. | |\n' + ' ' + '+---------------------------+---------------------------------+-------------+\n' + ' | "__globals__" | A reference to the ' + 'dictionary | Read-only |\n' + ' | | that holds the ' + "function's | |\n" + ' | | global variables --- the ' + 'global | |\n' + ' | | namespace of the module ' + 'in | |\n' + ' | | which the function was ' + 'defined. | |\n' + ' ' + '+---------------------------+---------------------------------+-------------+\n' + ' | "__dict__" | The namespace ' + 'supporting | Writable |\n' + ' | | arbitrary function ' + 'attributes. | |\n' + ' ' + '+---------------------------+---------------------------------+-------------+\n' + ' | "__closure__" | "None" or a tuple of cells ' + 'that | Read-only |\n' + ' | | contain bindings for ' + 'the | |\n' + " | | function's free " + 'variables. | |\n' + ' ' + '+---------------------------+---------------------------------+-------------+\n' + ' | "__annotations__" | A dict containing ' + 'annotations | Writable |\n' + ' | | of parameters. The keys of ' + 'the | |\n' + ' | | dict are the parameter ' + 'names, | |\n' + ' | | and "\'return\'" for the ' + 'return | |\n' + ' | | annotation, if ' + 'provided. | |\n' + ' ' + '+---------------------------+---------------------------------+-------------+\n' + ' | "__kwdefaults__" | A dict containing defaults ' + 'for | Writable |\n' + ' | | keyword-only ' + 'parameters. | |\n' + ' ' + '+---------------------------+---------------------------------+-------------+\n' + '\n' + ' Most of the attributes labelled "Writable" check the type ' + 'of the\n' + ' assigned value.\n' + '\n' + ' Function objects also support getting and setting ' + 'arbitrary\n' + ' attributes, which can be used, for example, to attach ' + 'metadata\n' + ' to functions. Regular attribute dot-notation is used to ' + 'get and\n' + ' set such attributes. *Note that the current implementation ' + 'only\n' + ' supports function attributes on user-defined functions. ' + 'Function\n' + ' attributes on built-in functions may be supported in the\n' + ' future.*\n' + '\n' + " Additional information about a function's definition can " + 'be\n' + ' retrieved from its code object; see the description of ' + 'internal\n' + ' types below.\n' + '\n' + ' Instance methods\n' + ' An instance method object combines a class, a class ' + 'instance and\n' + ' any callable object (normally a user-defined function).\n' + '\n' + ' Special read-only attributes: "__self__" is the class ' + 'instance\n' + ' object, "__func__" is the function object; "__doc__" is ' + 'the\n' + ' method\'s documentation (same as "__func__.__doc__"); ' + '"__name__"\n' + ' is the method name (same as "__func__.__name__"); ' + '"__module__"\n' + ' is the name of the module the method was defined in, or ' + '"None"\n' + ' if unavailable.\n' + '\n' + ' Methods also support accessing (but not setting) the ' + 'arbitrary\n' + ' function attributes on the underlying function object.\n' + '\n' + ' User-defined method objects may be created when getting an\n' + ' attribute of a class (perhaps via an instance of that ' + 'class), if\n' + ' that attribute is a user-defined function object or a ' + 'class\n' + ' method object.\n' + '\n' + ' When an instance method object is created by retrieving a ' + 'user-\n' + ' defined function object from a class via one of its ' + 'instances,\n' + ' its "__self__" attribute is the instance, and the method ' + 'object\n' + ' is said to be bound. The new method\'s "__func__" ' + 'attribute is\n' + ' the original function object.\n' + '\n' + ' When a user-defined method object is created by retrieving\n' + ' another method object from a class or instance, the ' + 'behaviour is\n' + ' the same as for a function object, except that the ' + '"__func__"\n' + ' attribute of the new instance is not the original method ' + 'object\n' + ' but its "__func__" attribute.\n' + '\n' + ' When an instance method object is created by retrieving a ' + 'class\n' + ' method object from a class or instance, its "__self__" ' + 'attribute\n' + ' is the class itself, and its "__func__" attribute is the\n' + ' function object underlying the class method.\n' + '\n' + ' When an instance method object is called, the underlying\n' + ' function ("__func__") is called, inserting the class ' + 'instance\n' + ' ("__self__") in front of the argument list. For instance, ' + 'when\n' + ' "C" is a class which contains a definition for a function ' + '"f()",\n' + ' and "x" is an instance of "C", calling "x.f(1)" is ' + 'equivalent to\n' + ' calling "C.f(x, 1)".\n' + '\n' + ' When an instance method object is derived from a class ' + 'method\n' + ' object, the "class instance" stored in "__self__" will ' + 'actually\n' + ' be the class itself, so that calling either "x.f(1)" or ' + '"C.f(1)"\n' + ' is equivalent to calling "f(C,1)" where "f" is the ' + 'underlying\n' + ' function.\n' + '\n' + ' Note that the transformation from function object to ' + 'instance\n' + ' method object happens each time the attribute is retrieved ' + 'from\n' + ' the instance. In some cases, a fruitful optimization is ' + 'to\n' + ' assign the attribute to a local variable and call that ' + 'local\n' + ' variable. Also notice that this transformation only happens ' + 'for\n' + ' user-defined functions; other callable objects (and all ' + 'non-\n' + ' callable objects) are retrieved without transformation. It ' + 'is\n' + ' also important to note that user-defined functions which ' + 'are\n' + ' attributes of a class instance are not converted to bound\n' + ' methods; this *only* happens when the function is an ' + 'attribute\n' + ' of the class.\n' + '\n' + ' Generator functions\n' + ' A function or method which uses the "yield" statement (see\n' + ' section *The yield statement*) is called a *generator ' + 'function*.\n' + ' Such a function, when called, always returns an iterator ' + 'object\n' + ' which can be used to execute the body of the function: ' + 'calling\n' + ' the iterator\'s "iterator.__next__()" method will cause ' + 'the\n' + ' function to execute until it provides a value using the ' + '"yield"\n' + ' statement. When the function executes a "return" statement ' + 'or\n' + ' falls off the end, a "StopIteration" exception is raised ' + 'and the\n' + ' iterator will have reached the end of the set of values to ' + 'be\n' + ' returned.\n' + '\n' + ' Coroutine functions\n' + ' A function or method which is defined using "async def" is\n' + ' called a *coroutine function*. Such a function, when ' + 'called,\n' + ' returns a *coroutine* object. It may contain "await"\n' + ' expressions, as well as "async with" and "async for" ' + 'statements.\n' + ' See also the *Coroutine Objects* section.\n' + '\n' + ' Built-in functions\n' + ' A built-in function object is a wrapper around a C ' + 'function.\n' + ' Examples of built-in functions are "len()" and ' + '"math.sin()"\n' + ' ("math" is a standard built-in module). The number and type ' + 'of\n' + ' the arguments are determined by the C function. Special ' + 'read-\n' + ' only attributes: "__doc__" is the function\'s ' + 'documentation\n' + ' string, or "None" if unavailable; "__name__" is the ' + "function's\n" + ' name; "__self__" is set to "None" (but see the next item);\n' + ' "__module__" is the name of the module the function was ' + 'defined\n' + ' in or "None" if unavailable.\n' + '\n' + ' Built-in methods\n' + ' This is really a different disguise of a built-in function, ' + 'this\n' + ' time containing an object passed to the C function as an\n' + ' implicit extra argument. An example of a built-in method ' + 'is\n' + ' "alist.append()", assuming *alist* is a list object. In ' + 'this\n' + ' case, the special read-only attribute "__self__" is set to ' + 'the\n' + ' object denoted by *alist*.\n' + '\n' + ' Classes\n' + ' Classes are callable. These objects normally act as ' + 'factories\n' + ' for new instances of themselves, but variations are ' + 'possible for\n' + ' class types that override "__new__()". The arguments of ' + 'the\n' + ' call are passed to "__new__()" and, in the typical case, ' + 'to\n' + ' "__init__()" to initialize the new instance.\n' + '\n' + ' Class Instances\n' + ' Instances of arbitrary classes can be made callable by ' + 'defining\n' + ' a "__call__()" method in their class.\n' + '\n' + 'Modules\n' + ' Modules are a basic organizational unit of Python code, and ' + 'are\n' + ' created by the *import system* as invoked either by the ' + '"import"\n' + ' statement (see "import"), or by calling functions such as\n' + ' "importlib.import_module()" and built-in "__import__()". A ' + 'module\n' + ' object has a namespace implemented by a dictionary object ' + '(this is\n' + ' the dictionary referenced by the "__globals__" attribute of\n' + ' functions defined in the module). Attribute references are\n' + ' translated to lookups in this dictionary, e.g., "m.x" is ' + 'equivalent\n' + ' to "m.__dict__["x"]". A module object does not contain the ' + 'code\n' + " object used to initialize the module (since it isn't needed " + 'once\n' + ' the initialization is done).\n' + '\n' + " Attribute assignment updates the module's namespace " + 'dictionary,\n' + ' e.g., "m.x = 1" is equivalent to "m.__dict__["x"] = 1".\n' + '\n' + ' Special read-only attribute: "__dict__" is the module\'s ' + 'namespace\n' + ' as a dictionary object.\n' + '\n' + ' **CPython implementation detail:** Because of the way CPython\n' + ' clears module dictionaries, the module dictionary will be ' + 'cleared\n' + ' when the module falls out of scope even if the dictionary ' + 'still has\n' + ' live references. To avoid this, copy the dictionary or keep ' + 'the\n' + ' module around while using its dictionary directly.\n' + '\n' + ' Predefined (writable) attributes: "__name__" is the module\'s ' + 'name;\n' + ' "__doc__" is the module\'s documentation string, or "None" if\n' + ' unavailable; "__file__" is the pathname of the file from which ' + 'the\n' + ' module was loaded, if it was loaded from a file. The ' + '"__file__"\n' + ' attribute may be missing for certain types of modules, such as ' + 'C\n' + ' modules that are statically linked into the interpreter; for\n' + ' extension modules loaded dynamically from a shared library, it ' + 'is\n' + ' the pathname of the shared library file.\n' + '\n' + 'Custom classes\n' + ' Custom class types are typically created by class definitions ' + '(see\n' + ' section *Class definitions*). A class has a namespace ' + 'implemented\n' + ' by a dictionary object. Class attribute references are ' + 'translated\n' + ' to lookups in this dictionary, e.g., "C.x" is translated to\n' + ' "C.__dict__["x"]" (although there are a number of hooks which ' + 'allow\n' + ' for other means of locating attributes). When the attribute ' + 'name is\n' + ' not found there, the attribute search continues in the base\n' + ' classes. This search of the base classes uses the C3 method\n' + ' resolution order which behaves correctly even in the presence ' + 'of\n' + " 'diamond' inheritance structures where there are multiple\n" + ' inheritance paths leading back to a common ancestor. ' + 'Additional\n' + ' details on the C3 MRO used by Python can be found in the\n' + ' documentation accompanying the 2.3 release at\n' + ' https://www.python.org/download/releases/2.3/mro/.\n' + '\n' + ' When a class attribute reference (for class "C", say) would ' + 'yield a\n' + ' class method object, it is transformed into an instance ' + 'method\n' + ' object whose "__self__" attributes is "C". When it would ' + 'yield a\n' + ' static method object, it is transformed into the object ' + 'wrapped by\n' + ' the static method object. See section *Implementing ' + 'Descriptors*\n' + ' for another way in which attributes retrieved from a class ' + 'may\n' + ' differ from those actually contained in its "__dict__".\n' + '\n' + " Class attribute assignments update the class's dictionary, " + 'never\n' + ' the dictionary of a base class.\n' + '\n' + ' A class object can be called (see above) to yield a class ' + 'instance\n' + ' (see below).\n' + '\n' + ' Special attributes: "__name__" is the class name; "__module__" ' + 'is\n' + ' the module name in which the class was defined; "__dict__" is ' + 'the\n' + ' dictionary containing the class\'s namespace; "__bases__" is a ' + 'tuple\n' + ' (possibly empty or a singleton) containing the base classes, ' + 'in the\n' + ' order of their occurrence in the base class list; "__doc__" is ' + 'the\n' + " class's documentation string, or None if undefined.\n" + '\n' + 'Class instances\n' + ' A class instance is created by calling a class object (see ' + 'above).\n' + ' A class instance has a namespace implemented as a dictionary ' + 'which\n' + ' is the first place in which attribute references are ' + 'searched.\n' + " When an attribute is not found there, and the instance's class " + 'has\n' + ' an attribute by that name, the search continues with the ' + 'class\n' + ' attributes. If a class attribute is found that is a ' + 'user-defined\n' + ' function object, it is transformed into an instance method ' + 'object\n' + ' whose "__self__" attribute is the instance. Static method ' + 'and\n' + ' class method objects are also transformed; see above under\n' + ' "Classes". See section *Implementing Descriptors* for another ' + 'way\n' + ' in which attributes of a class retrieved via its instances ' + 'may\n' + " differ from the objects actually stored in the class's " + '"__dict__".\n' + " If no class attribute is found, and the object's class has a\n" + ' "__getattr__()" method, that is called to satisfy the lookup.\n' + '\n' + " Attribute assignments and deletions update the instance's\n" + " dictionary, never a class's dictionary. If the class has a\n" + ' "__setattr__()" or "__delattr__()" method, this is called ' + 'instead\n' + ' of updating the instance dictionary directly.\n' + '\n' + ' Class instances can pretend to be numbers, sequences, or ' + 'mappings\n' + ' if they have methods with certain special names. See section\n' + ' *Special method names*.\n' + '\n' + ' Special attributes: "__dict__" is the attribute dictionary;\n' + ' "__class__" is the instance\'s class.\n' + '\n' + 'I/O objects (also known as file objects)\n' + ' A *file object* represents an open file. Various shortcuts ' + 'are\n' + ' available to create file objects: the "open()" built-in ' + 'function,\n' + ' and also "os.popen()", "os.fdopen()", and the "makefile()" ' + 'method\n' + ' of socket objects (and perhaps by other functions or methods\n' + ' provided by extension modules).\n' + '\n' + ' The objects "sys.stdin", "sys.stdout" and "sys.stderr" are\n' + ' initialized to file objects corresponding to the ' + "interpreter's\n" + ' standard input, output and error streams; they are all open in ' + 'text\n' + ' mode and therefore follow the interface defined by the\n' + ' "io.TextIOBase" abstract class.\n' + '\n' + 'Internal types\n' + ' A few types used internally by the interpreter are exposed to ' + 'the\n' + ' user. Their definitions may change with future versions of ' + 'the\n' + ' interpreter, but they are mentioned here for completeness.\n' + '\n' + ' Code objects\n' + ' Code objects represent *byte-compiled* executable Python ' + 'code,\n' + ' or *bytecode*. The difference between a code object and a\n' + ' function object is that the function object contains an ' + 'explicit\n' + " reference to the function's globals (the module in which it " + 'was\n' + ' defined), while a code object contains no context; also ' + 'the\n' + ' default argument values are stored in the function object, ' + 'not\n' + ' in the code object (because they represent values ' + 'calculated at\n' + ' run-time). Unlike function objects, code objects are ' + 'immutable\n' + ' and contain no references (directly or indirectly) to ' + 'mutable\n' + ' objects.\n' + '\n' + ' Special read-only attributes: "co_name" gives the function ' + 'name;\n' + ' "co_argcount" is the number of positional arguments ' + '(including\n' + ' arguments with default values); "co_nlocals" is the number ' + 'of\n' + ' local variables used by the function (including ' + 'arguments);\n' + ' "co_varnames" is a tuple containing the names of the local\n' + ' variables (starting with the argument names); "co_cellvars" ' + 'is a\n' + ' tuple containing the names of local variables that are\n' + ' referenced by nested functions; "co_freevars" is a tuple\n' + ' containing the names of free variables; "co_code" is a ' + 'string\n' + ' representing the sequence of bytecode instructions; ' + '"co_consts"\n' + ' is a tuple containing the literals used by the bytecode;\n' + ' "co_names" is a tuple containing the names used by the ' + 'bytecode;\n' + ' "co_filename" is the filename from which the code was ' + 'compiled;\n' + ' "co_firstlineno" is the first line number of the function;\n' + ' "co_lnotab" is a string encoding the mapping from bytecode\n' + ' offsets to line numbers (for details see the source code of ' + 'the\n' + ' interpreter); "co_stacksize" is the required stack size\n' + ' (including local variables); "co_flags" is an integer ' + 'encoding a\n' + ' number of flags for the interpreter.\n' + '\n' + ' The following flag bits are defined for "co_flags": bit ' + '"0x04"\n' + ' is set if the function uses the "*arguments" syntax to ' + 'accept an\n' + ' arbitrary number of positional arguments; bit "0x08" is set ' + 'if\n' + ' the function uses the "**keywords" syntax to accept ' + 'arbitrary\n' + ' keyword arguments; bit "0x20" is set if the function is a\n' + ' generator.\n' + '\n' + ' Future feature declarations ("from __future__ import ' + 'division")\n' + ' also use bits in "co_flags" to indicate whether a code ' + 'object\n' + ' was compiled with a particular feature enabled: bit ' + '"0x2000" is\n' + ' set if the function was compiled with future division ' + 'enabled;\n' + ' bits "0x10" and "0x1000" were used in earlier versions of\n' + ' Python.\n' + '\n' + ' Other bits in "co_flags" are reserved for internal use.\n' + '\n' + ' If a code object represents a function, the first item in\n' + ' "co_consts" is the documentation string of the function, ' + 'or\n' + ' "None" if undefined.\n' + '\n' + ' Frame objects\n' + ' Frame objects represent execution frames. They may occur ' + 'in\n' + ' traceback objects (see below).\n' + '\n' + ' Special read-only attributes: "f_back" is to the previous ' + 'stack\n' + ' frame (towards the caller), or "None" if this is the ' + 'bottom\n' + ' stack frame; "f_code" is the code object being executed in ' + 'this\n' + ' frame; "f_locals" is the dictionary used to look up local\n' + ' variables; "f_globals" is used for global variables;\n' + ' "f_builtins" is used for built-in (intrinsic) names; ' + '"f_lasti"\n' + ' gives the precise instruction (this is an index into the\n' + ' bytecode string of the code object).\n' + '\n' + ' Special writable attributes: "f_trace", if not "None", is ' + 'a\n' + ' function called at the start of each source code line (this ' + 'is\n' + ' used by the debugger); "f_lineno" is the current line ' + 'number of\n' + ' the frame --- writing to this from within a trace function ' + 'jumps\n' + ' to the given line (only for the bottom-most frame). A ' + 'debugger\n' + ' can implement a Jump command (aka Set Next Statement) by ' + 'writing\n' + ' to f_lineno.\n' + '\n' + ' Frame objects support one method:\n' + '\n' + ' frame.clear()\n' + '\n' + ' This method clears all references to local variables ' + 'held by\n' + ' the frame. Also, if the frame belonged to a generator, ' + 'the\n' + ' generator is finalized. This helps break reference ' + 'cycles\n' + ' involving frame objects (for example when catching an\n' + ' exception and storing its traceback for later use).\n' + '\n' + ' "RuntimeError" is raised if the frame is currently ' + 'executing.\n' + '\n' + ' New in version 3.4.\n' + '\n' + ' Traceback objects\n' + ' Traceback objects represent a stack trace of an exception. ' + 'A\n' + ' traceback object is created when an exception occurs. When ' + 'the\n' + ' search for an exception handler unwinds the execution ' + 'stack, at\n' + ' each unwound level a traceback object is inserted in front ' + 'of\n' + ' the current traceback. When an exception handler is ' + 'entered,\n' + ' the stack trace is made available to the program. (See ' + 'section\n' + ' *The try statement*.) It is accessible as the third item of ' + 'the\n' + ' tuple returned by "sys.exc_info()". When the program ' + 'contains no\n' + ' suitable handler, the stack trace is written (nicely ' + 'formatted)\n' + ' to the standard error stream; if the interpreter is ' + 'interactive,\n' + ' it is also made available to the user as ' + '"sys.last_traceback".\n' + '\n' + ' Special read-only attributes: "tb_next" is the next level ' + 'in the\n' + ' stack trace (towards the frame where the exception ' + 'occurred), or\n' + ' "None" if there is no next level; "tb_frame" points to the\n' + ' execution frame of the current level; "tb_lineno" gives the ' + 'line\n' + ' number where the exception occurred; "tb_lasti" indicates ' + 'the\n' + ' precise instruction. The line number and last instruction ' + 'in\n' + ' the traceback may differ from the line number of its frame\n' + ' object if the exception occurred in a "try" statement with ' + 'no\n' + ' matching except clause or with a finally clause.\n' + '\n' + ' Slice objects\n' + ' Slice objects are used to represent slices for ' + '"__getitem__()"\n' + ' methods. They are also created by the built-in "slice()"\n' + ' function.\n' + '\n' + ' Special read-only attributes: "start" is the lower bound; ' + '"stop"\n' + ' is the upper bound; "step" is the step value; each is ' + '"None" if\n' + ' omitted. These attributes can have any type.\n' + '\n' + ' Slice objects support one method:\n' + '\n' + ' slice.indices(self, length)\n' + '\n' + ' This method takes a single integer argument *length* ' + 'and\n' + ' computes information about the slice that the slice ' + 'object\n' + ' would describe if applied to a sequence of *length* ' + 'items.\n' + ' It returns a tuple of three integers; respectively these ' + 'are\n' + ' the *start* and *stop* indices and the *step* or stride\n' + ' length of the slice. Missing or out-of-bounds indices ' + 'are\n' + ' handled in a manner consistent with regular slices.\n' + '\n' + ' Static method objects\n' + ' Static method objects provide a way of defeating the\n' + ' transformation of function objects to method objects ' + 'described\n' + ' above. A static method object is a wrapper around any ' + 'other\n' + ' object, usually a user-defined method object. When a ' + 'static\n' + ' method object is retrieved from a class or a class ' + 'instance, the\n' + ' object actually returned is the wrapped object, which is ' + 'not\n' + ' subject to any further transformation. Static method ' + 'objects are\n' + ' not themselves callable, although the objects they wrap ' + 'usually\n' + ' are. Static method objects are created by the built-in\n' + ' "staticmethod()" constructor.\n' + '\n' + ' Class method objects\n' + ' A class method object, like a static method object, is a ' + 'wrapper\n' + ' around another object that alters the way in which that ' + 'object\n' + ' is retrieved from classes and class instances. The ' + 'behaviour of\n' + ' class method objects upon such retrieval is described ' + 'above,\n' + ' under "User-defined methods". Class method objects are ' + 'created\n' + ' by the built-in "classmethod()" constructor.\n', + 'typesfunctions': '\n' + 'Functions\n' + '*********\n' + '\n' + 'Function objects are created by function definitions. ' + 'The only\n' + 'operation on a function object is to call it: ' + '"func(argument-list)".\n' + '\n' + 'There are really two flavors of function objects: ' + 'built-in functions\n' + 'and user-defined functions. Both support the same ' + 'operation (to call\n' + 'the function), but the implementation is different, ' + 'hence the\n' + 'different object types.\n' + '\n' + 'See *Function definitions* for more information.\n', + 'typesmapping': '\n' + 'Mapping Types --- "dict"\n' + '************************\n' + '\n' + 'A *mapping* object maps *hashable* values to arbitrary ' + 'objects.\n' + 'Mappings are mutable objects. There is currently only one ' + 'standard\n' + 'mapping type, the *dictionary*. (For other containers see ' + 'the built-\n' + 'in "list", "set", and "tuple" classes, and the ' + '"collections" module.)\n' + '\n' + "A dictionary's keys are *almost* arbitrary values. Values " + 'that are\n' + 'not *hashable*, that is, values containing lists, ' + 'dictionaries or\n' + 'other mutable types (that are compared by value rather ' + 'than by object\n' + 'identity) may not be used as keys. Numeric types used for ' + 'keys obey\n' + 'the normal rules for numeric comparison: if two numbers ' + 'compare equal\n' + '(such as "1" and "1.0") then they can be used ' + 'interchangeably to index\n' + 'the same dictionary entry. (Note however, that since ' + 'computers store\n' + 'floating-point numbers as approximations it is usually ' + 'unwise to use\n' + 'them as dictionary keys.)\n' + '\n' + 'Dictionaries can be created by placing a comma-separated ' + 'list of "key:\n' + 'value" pairs within braces, for example: "{\'jack\': 4098, ' + "'sjoerd':\n" + '4127}" or "{4098: \'jack\', 4127: \'sjoerd\'}", or by the ' + '"dict"\n' + 'constructor.\n' + '\n' + 'class class dict(**kwarg)\n' + 'class class dict(mapping, **kwarg)\n' + 'class class dict(iterable, **kwarg)\n' + '\n' + ' Return a new dictionary initialized from an optional ' + 'positional\n' + ' argument and a possibly empty set of keyword ' + 'arguments.\n' + '\n' + ' If no positional argument is given, an empty dictionary ' + 'is created.\n' + ' If a positional argument is given and it is a mapping ' + 'object, a\n' + ' dictionary is created with the same key-value pairs as ' + 'the mapping\n' + ' object. Otherwise, the positional argument must be an ' + '*iterable*\n' + ' object. Each item in the iterable must itself be an ' + 'iterable with\n' + ' exactly two objects. The first object of each item ' + 'becomes a key\n' + ' in the new dictionary, and the second object the ' + 'corresponding\n' + ' value. If a key occurs more than once, the last value ' + 'for that key\n' + ' becomes the corresponding value in the new dictionary.\n' + '\n' + ' If keyword arguments are given, the keyword arguments ' + 'and their\n' + ' values are added to the dictionary created from the ' + 'positional\n' + ' argument. If a key being added is already present, the ' + 'value from\n' + ' the keyword argument replaces the value from the ' + 'positional\n' + ' argument.\n' + '\n' + ' To illustrate, the following examples all return a ' + 'dictionary equal\n' + ' to "{"one": 1, "two": 2, "three": 3}":\n' + '\n' + ' >>> a = dict(one=1, two=2, three=3)\n' + " >>> b = {'one': 1, 'two': 2, 'three': 3}\n" + " >>> c = dict(zip(['one', 'two', 'three'], [1, 2, " + '3]))\n' + " >>> d = dict([('two', 2), ('one', 1), ('three', " + '3)])\n' + " >>> e = dict({'three': 3, 'one': 1, 'two': 2})\n" + ' >>> a == b == c == d == e\n' + ' True\n' + '\n' + ' Providing keyword arguments as in the first example ' + 'only works for\n' + ' keys that are valid Python identifiers. Otherwise, any ' + 'valid keys\n' + ' can be used.\n' + '\n' + ' These are the operations that dictionaries support (and ' + 'therefore,\n' + ' custom mapping types should support too):\n' + '\n' + ' len(d)\n' + '\n' + ' Return the number of items in the dictionary *d*.\n' + '\n' + ' d[key]\n' + '\n' + ' Return the item of *d* with key *key*. Raises a ' + '"KeyError" if\n' + ' *key* is not in the map.\n' + '\n' + ' If a subclass of dict defines a method ' + '"__missing__()" and *key*\n' + ' is not present, the "d[key]" operation calls that ' + 'method with\n' + ' the key *key* as argument. The "d[key]" operation ' + 'then returns\n' + ' or raises whatever is returned or raised by the\n' + ' "__missing__(key)" call. No other operations or ' + 'methods invoke\n' + ' "__missing__()". If "__missing__()" is not defined, ' + '"KeyError"\n' + ' is raised. "__missing__()" must be a method; it ' + 'cannot be an\n' + ' instance variable:\n' + '\n' + ' >>> class Counter(dict):\n' + ' ... def __missing__(self, key):\n' + ' ... return 0\n' + ' >>> c = Counter()\n' + " >>> c['red']\n" + ' 0\n' + " >>> c['red'] += 1\n" + " >>> c['red']\n" + ' 1\n' + '\n' + ' The example above shows part of the implementation ' + 'of\n' + ' "collections.Counter". A different "__missing__" ' + 'method is used\n' + ' by "collections.defaultdict".\n' + '\n' + ' d[key] = value\n' + '\n' + ' Set "d[key]" to *value*.\n' + '\n' + ' del d[key]\n' + '\n' + ' Remove "d[key]" from *d*. Raises a "KeyError" if ' + '*key* is not\n' + ' in the map.\n' + '\n' + ' key in d\n' + '\n' + ' Return "True" if *d* has a key *key*, else "False".\n' + '\n' + ' key not in d\n' + '\n' + ' Equivalent to "not key in d".\n' + '\n' + ' iter(d)\n' + '\n' + ' Return an iterator over the keys of the dictionary. ' + 'This is a\n' + ' shortcut for "iter(d.keys())".\n' + '\n' + ' clear()\n' + '\n' + ' Remove all items from the dictionary.\n' + '\n' + ' copy()\n' + '\n' + ' Return a shallow copy of the dictionary.\n' + '\n' + ' classmethod fromkeys(seq[, value])\n' + '\n' + ' Create a new dictionary with keys from *seq* and ' + 'values set to\n' + ' *value*.\n' + '\n' + ' "fromkeys()" is a class method that returns a new ' + 'dictionary.\n' + ' *value* defaults to "None".\n' + '\n' + ' get(key[, default])\n' + '\n' + ' Return the value for *key* if *key* is in the ' + 'dictionary, else\n' + ' *default*. If *default* is not given, it defaults to ' + '"None", so\n' + ' that this method never raises a "KeyError".\n' + '\n' + ' items()\n' + '\n' + ' Return a new view of the dictionary\'s items ("(key, ' + 'value)"\n' + ' pairs). See the *documentation of view objects*.\n' + '\n' + ' keys()\n' + '\n' + " Return a new view of the dictionary's keys. See " + 'the\n' + ' *documentation of view objects*.\n' + '\n' + ' pop(key[, default])\n' + '\n' + ' If *key* is in the dictionary, remove it and return ' + 'its value,\n' + ' else return *default*. If *default* is not given ' + 'and *key* is\n' + ' not in the dictionary, a "KeyError" is raised.\n' + '\n' + ' popitem()\n' + '\n' + ' Remove and return an arbitrary "(key, value)" pair ' + 'from the\n' + ' dictionary.\n' + '\n' + ' "popitem()" is useful to destructively iterate over ' + 'a\n' + ' dictionary, as often used in set algorithms. If the ' + 'dictionary\n' + ' is empty, calling "popitem()" raises a "KeyError".\n' + '\n' + ' setdefault(key[, default])\n' + '\n' + ' If *key* is in the dictionary, return its value. If ' + 'not, insert\n' + ' *key* with a value of *default* and return ' + '*default*. *default*\n' + ' defaults to "None".\n' + '\n' + ' update([other])\n' + '\n' + ' Update the dictionary with the key/value pairs from ' + '*other*,\n' + ' overwriting existing keys. Return "None".\n' + '\n' + ' "update()" accepts either another dictionary object ' + 'or an\n' + ' iterable of key/value pairs (as tuples or other ' + 'iterables of\n' + ' length two). If keyword arguments are specified, ' + 'the dictionary\n' + ' is then updated with those key/value pairs: ' + '"d.update(red=1,\n' + ' blue=2)".\n' + '\n' + ' values()\n' + '\n' + " Return a new view of the dictionary's values. See " + 'the\n' + ' *documentation of view objects*.\n' + '\n' + ' Dictionaries compare equal if and only if they have the ' + 'same "(key,\n' + ' value)" pairs. Order comparisons (\'<\', \'<=\', ' + "'>=', '>') raise\n" + ' "TypeError".\n' + '\n' + 'See also: "types.MappingProxyType" can be used to create a ' + 'read-only\n' + ' view of a "dict".\n' + '\n' + '\n' + 'Dictionary view objects\n' + '=======================\n' + '\n' + 'The objects returned by "dict.keys()", "dict.values()" ' + 'and\n' + '"dict.items()" are *view objects*. They provide a dynamic ' + 'view on the\n' + "dictionary's entries, which means that when the dictionary " + 'changes,\n' + 'the view reflects these changes.\n' + '\n' + 'Dictionary views can be iterated over to yield their ' + 'respective data,\n' + 'and support membership tests:\n' + '\n' + 'len(dictview)\n' + '\n' + ' Return the number of entries in the dictionary.\n' + '\n' + 'iter(dictview)\n' + '\n' + ' Return an iterator over the keys, values or items ' + '(represented as\n' + ' tuples of "(key, value)") in the dictionary.\n' + '\n' + ' Keys and values are iterated over in an arbitrary order ' + 'which is\n' + ' non-random, varies across Python implementations, and ' + 'depends on\n' + " the dictionary's history of insertions and deletions. " + 'If keys,\n' + ' values and items views are iterated over with no ' + 'intervening\n' + ' modifications to the dictionary, the order of items ' + 'will directly\n' + ' correspond. This allows the creation of "(value, key)" ' + 'pairs using\n' + ' "zip()": "pairs = zip(d.values(), d.keys())". Another ' + 'way to\n' + ' create the same list is "pairs = [(v, k) for (k, v) in ' + 'd.items()]".\n' + '\n' + ' Iterating views while adding or deleting entries in the ' + 'dictionary\n' + ' may raise a "RuntimeError" or fail to iterate over all ' + 'entries.\n' + '\n' + 'x in dictview\n' + '\n' + ' Return "True" if *x* is in the underlying dictionary\'s ' + 'keys, values\n' + ' or items (in the latter case, *x* should be a "(key, ' + 'value)"\n' + ' tuple).\n' + '\n' + 'Keys views are set-like since their entries are unique and ' + 'hashable.\n' + 'If all values are hashable, so that "(key, value)" pairs ' + 'are unique\n' + 'and hashable, then the items view is also set-like. ' + '(Values views are\n' + 'not treated as set-like since the entries are generally ' + 'not unique.)\n' + 'For set-like views, all of the operations defined for the ' + 'abstract\n' + 'base class "collections.abc.Set" are available (for ' + 'example, "==",\n' + '"<", or "^").\n' + '\n' + 'An example of dictionary view usage:\n' + '\n' + " >>> dishes = {'eggs': 2, 'sausage': 1, 'bacon': 1, " + "'spam': 500}\n" + ' >>> keys = dishes.keys()\n' + ' >>> values = dishes.values()\n' + '\n' + ' >>> # iteration\n' + ' >>> n = 0\n' + ' >>> for val in values:\n' + ' ... n += val\n' + ' >>> print(n)\n' + ' 504\n' + '\n' + ' >>> # keys and values are iterated over in the same ' + 'order\n' + ' >>> list(keys)\n' + " ['eggs', 'bacon', 'sausage', 'spam']\n" + ' >>> list(values)\n' + ' [2, 1, 1, 500]\n' + '\n' + ' >>> # view objects are dynamic and reflect dict ' + 'changes\n' + " >>> del dishes['eggs']\n" + " >>> del dishes['sausage']\n" + ' >>> list(keys)\n' + " ['spam', 'bacon']\n" + '\n' + ' >>> # set operations\n' + " >>> keys & {'eggs', 'bacon', 'salad'}\n" + " {'bacon'}\n" + " >>> keys ^ {'sausage', 'juice'}\n" + " {'juice', 'sausage', 'bacon', 'spam'}\n", + 'typesmethods': '\n' + 'Methods\n' + '*******\n' + '\n' + 'Methods are functions that are called using the attribute ' + 'notation.\n' + 'There are two flavors: built-in methods (such as ' + '"append()" on lists)\n' + 'and class instance methods. Built-in methods are ' + 'described with the\n' + 'types that support them.\n' + '\n' + 'If you access a method (a function defined in a class ' + 'namespace)\n' + 'through an instance, you get a special object: a *bound ' + 'method* (also\n' + 'called *instance method*) object. When called, it will add ' + 'the "self"\n' + 'argument to the argument list. Bound methods have two ' + 'special read-\n' + 'only attributes: "m.__self__" is the object on which the ' + 'method\n' + 'operates, and "m.__func__" is the function implementing ' + 'the method.\n' + 'Calling "m(arg-1, arg-2, ..., arg-n)" is completely ' + 'equivalent to\n' + 'calling "m.__func__(m.__self__, arg-1, arg-2, ..., ' + 'arg-n)".\n' + '\n' + 'Like function objects, bound method objects support ' + 'getting arbitrary\n' + 'attributes. However, since method attributes are actually ' + 'stored on\n' + 'the underlying function object ("meth.__func__"), setting ' + 'method\n' + 'attributes on bound methods is disallowed. Attempting to ' + 'set an\n' + 'attribute on a method results in an "AttributeError" being ' + 'raised. In\n' + 'order to set a method attribute, you need to explicitly ' + 'set it on the\n' + 'underlying function object:\n' + '\n' + ' >>> class C:\n' + ' ... def method(self):\n' + ' ... pass\n' + ' ...\n' + ' >>> c = C()\n' + " >>> c.method.whoami = 'my name is method' # can't set " + 'on the method\n' + ' Traceback (most recent call last):\n' + ' File "", line 1, in \n' + " AttributeError: 'method' object has no attribute " + "'whoami'\n" + " >>> c.method.__func__.whoami = 'my name is method'\n" + ' >>> c.method.whoami\n' + " 'my name is method'\n" + '\n' + 'See *The standard type hierarchy* for more information.\n', + 'typesmodules': '\n' + 'Modules\n' + '*******\n' + '\n' + 'The only special operation on a module is attribute ' + 'access: "m.name",\n' + 'where *m* is a module and *name* accesses a name defined ' + "in *m*'s\n" + 'symbol table. Module attributes can be assigned to. (Note ' + 'that the\n' + '"import" statement is not, strictly speaking, an operation ' + 'on a module\n' + 'object; "import foo" does not require a module object ' + 'named *foo* to\n' + 'exist, rather it requires an (external) *definition* for a ' + 'module\n' + 'named *foo* somewhere.)\n' + '\n' + 'A special attribute of every module is "__dict__". This is ' + 'the\n' + "dictionary containing the module's symbol table. Modifying " + 'this\n' + "dictionary will actually change the module's symbol table, " + 'but direct\n' + 'assignment to the "__dict__" attribute is not possible ' + '(you can write\n' + '"m.__dict__[\'a\'] = 1", which defines "m.a" to be "1", ' + "but you can't\n" + 'write "m.__dict__ = {}"). Modifying "__dict__" directly ' + 'is not\n' + 'recommended.\n' + '\n' + 'Modules built into the interpreter are written like this: ' + '"". If loaded from a file, they are ' + 'written as\n' + '"".\n', + 'typesseq': '\n' + 'Sequence Types --- "list", "tuple", "range"\n' + '*******************************************\n' + '\n' + 'There are three basic sequence types: lists, tuples, and ' + 'range\n' + 'objects. Additional sequence types tailored for processing of ' + '*binary\n' + 'data* and *text strings* are described in dedicated sections.\n' + '\n' + '\n' + 'Common Sequence Operations\n' + '==========================\n' + '\n' + 'The operations in the following table are supported by most ' + 'sequence\n' + 'types, both mutable and immutable. The ' + '"collections.abc.Sequence" ABC\n' + 'is provided to make it easier to correctly implement these ' + 'operations\n' + 'on custom sequence types.\n' + '\n' + 'This table lists the sequence operations sorted in ascending ' + 'priority.\n' + 'In the table, *s* and *t* are sequences of the same type, *n*, ' + '*i*,\n' + '*j* and *k* are integers and *x* is an arbitrary object that ' + 'meets any\n' + 'type and value restrictions imposed by *s*.\n' + '\n' + 'The "in" and "not in" operations have the same priorities as ' + 'the\n' + 'comparison operations. The "+" (concatenation) and "*" ' + '(repetition)\n' + 'operations have the same priority as the corresponding ' + 'numeric\n' + 'operations.\n' + '\n' + '+----------------------------+----------------------------------+------------+\n' + '| Operation | ' + 'Result | Notes |\n' + '+============================+==================================+============+\n' + '| "x in s" | "True" if an item of *s* ' + 'is | (1) |\n' + '| | equal to *x*, else ' + '"False" | |\n' + '+----------------------------+----------------------------------+------------+\n' + '| "x not in s" | "False" if an item of *s* ' + 'is | (1) |\n' + '| | equal to *x*, else ' + '"True" | |\n' + '+----------------------------+----------------------------------+------------+\n' + '| "s + t" | the concatenation of *s* and ' + '*t* | (6)(7) |\n' + '+----------------------------+----------------------------------+------------+\n' + '| "s * n" or "n * s" | *n* shallow copies of ' + '*s* | (2)(7) |\n' + '| | ' + 'concatenated | |\n' + '+----------------------------+----------------------------------+------------+\n' + '| "s[i]" | *i*th item of *s*, origin ' + '0 | (3) |\n' + '+----------------------------+----------------------------------+------------+\n' + '| "s[i:j]" | slice of *s* from *i* to ' + '*j* | (3)(4) |\n' + '+----------------------------+----------------------------------+------------+\n' + '| "s[i:j:k]" | slice of *s* from *i* to ' + '*j* | (3)(5) |\n' + '| | with step ' + '*k* | |\n' + '+----------------------------+----------------------------------+------------+\n' + '| "len(s)" | length of ' + '*s* | |\n' + '+----------------------------+----------------------------------+------------+\n' + '| "min(s)" | smallest item of ' + '*s* | |\n' + '+----------------------------+----------------------------------+------------+\n' + '| "max(s)" | largest item of ' + '*s* | |\n' + '+----------------------------+----------------------------------+------------+\n' + '| "s.index(x[, i[, j]])" | index of the first occurrence ' + 'of | (8) |\n' + '| | *x* in *s* (at or after ' + 'index | |\n' + '| | *i* and before index ' + '*j*) | |\n' + '+----------------------------+----------------------------------+------------+\n' + '| "s.count(x)" | total number of occurrences ' + 'of | |\n' + '| | *x* in ' + '*s* | |\n' + '+----------------------------+----------------------------------+------------+\n' + '\n' + 'Sequences of the same type also support comparisons. In ' + 'particular,\n' + 'tuples and lists are compared lexicographically by comparing\n' + 'corresponding elements. This means that to compare equal, ' + 'every\n' + 'element must compare equal and the two sequences must be of ' + 'the same\n' + 'type and have the same length. (For full details see ' + '*Comparisons* in\n' + 'the language reference.)\n' + '\n' + 'Notes:\n' + '\n' + '1. While the "in" and "not in" operations are used only for ' + 'simple\n' + ' containment testing in the general case, some specialised ' + 'sequences\n' + ' (such as "str", "bytes" and "bytearray") also use them for\n' + ' subsequence testing:\n' + '\n' + ' >>> "gg" in "eggs"\n' + ' True\n' + '\n' + '2. Values of *n* less than "0" are treated as "0" (which ' + 'yields an\n' + ' empty sequence of the same type as *s*). Note also that ' + 'the copies\n' + ' are shallow; nested structures are not copied. This often ' + 'haunts\n' + ' new Python programmers; consider:\n' + '\n' + ' >>> lists = [[]] * 3\n' + ' >>> lists\n' + ' [[], [], []]\n' + ' >>> lists[0].append(3)\n' + ' >>> lists\n' + ' [[3], [3], [3]]\n' + '\n' + ' What has happened is that "[[]]" is a one-element list ' + 'containing\n' + ' an empty list, so all three elements of "[[]] * 3" are ' + '(pointers\n' + ' to) this single empty list. Modifying any of the elements ' + 'of\n' + ' "lists" modifies this single list. You can create a list ' + 'of\n' + ' different lists this way:\n' + '\n' + ' >>> lists = [[] for i in range(3)]\n' + ' >>> lists[0].append(3)\n' + ' >>> lists[1].append(5)\n' + ' >>> lists[2].append(7)\n' + ' >>> lists\n' + ' [[3], [5], [7]]\n' + '\n' + '3. If *i* or *j* is negative, the index is relative to the end ' + 'of\n' + ' the string: "len(s) + i" or "len(s) + j" is substituted. ' + 'But note\n' + ' that "-0" is still "0".\n' + '\n' + '4. The slice of *s* from *i* to *j* is defined as the sequence ' + 'of\n' + ' items with index *k* such that "i <= k < j". If *i* or *j* ' + 'is\n' + ' greater than "len(s)", use "len(s)". If *i* is omitted or ' + '"None",\n' + ' use "0". If *j* is omitted or "None", use "len(s)". If ' + '*i* is\n' + ' greater than or equal to *j*, the slice is empty.\n' + '\n' + '5. The slice of *s* from *i* to *j* with step *k* is defined ' + 'as the\n' + ' sequence of items with index "x = i + n*k" such that "0 <= ' + 'n <\n' + ' (j-i)/k". In other words, the indices are "i", "i+k", ' + '"i+2*k",\n' + ' "i+3*k" and so on, stopping when *j* is reached (but never\n' + ' including *j*). If *i* or *j* is greater than "len(s)", ' + 'use\n' + ' "len(s)". If *i* or *j* are omitted or "None", they become ' + '"end"\n' + ' values (which end depends on the sign of *k*). Note, *k* ' + 'cannot be\n' + ' zero. If *k* is "None", it is treated like "1".\n' + '\n' + '6. Concatenating immutable sequences always results in a new\n' + ' object. This means that building up a sequence by repeated\n' + ' concatenation will have a quadratic runtime cost in the ' + 'total\n' + ' sequence length. To get a linear runtime cost, you must ' + 'switch to\n' + ' one of the alternatives below:\n' + '\n' + ' * if concatenating "str" objects, you can build a list and ' + 'use\n' + ' "str.join()" at the end or else write to a "io.StringIO" ' + 'instance\n' + ' and retrieve its value when complete\n' + '\n' + ' * if concatenating "bytes" objects, you can similarly use\n' + ' "bytes.join()" or "io.BytesIO", or you can do in-place\n' + ' concatenation with a "bytearray" object. "bytearray" ' + 'objects are\n' + ' mutable and have an efficient overallocation mechanism\n' + '\n' + ' * if concatenating "tuple" objects, extend a "list" ' + 'instead\n' + '\n' + ' * for other types, investigate the relevant class ' + 'documentation\n' + '\n' + '7. Some sequence types (such as "range") only support item\n' + " sequences that follow specific patterns, and hence don't " + 'support\n' + ' sequence concatenation or repetition.\n' + '\n' + '8. "index" raises "ValueError" when *x* is not found in *s*. ' + 'When\n' + ' supported, the additional arguments to the index method ' + 'allow\n' + ' efficient searching of subsections of the sequence. Passing ' + 'the\n' + ' extra arguments is roughly equivalent to using ' + '"s[i:j].index(x)",\n' + ' only without copying any data and with the returned index ' + 'being\n' + ' relative to the start of the sequence rather than the start ' + 'of the\n' + ' slice.\n' + '\n' + '\n' + 'Immutable Sequence Types\n' + '========================\n' + '\n' + 'The only operation that immutable sequence types generally ' + 'implement\n' + 'that is not also implemented by mutable sequence types is ' + 'support for\n' + 'the "hash()" built-in.\n' + '\n' + 'This support allows immutable sequences, such as "tuple" ' + 'instances, to\n' + 'be used as "dict" keys and stored in "set" and "frozenset" ' + 'instances.\n' + '\n' + 'Attempting to hash an immutable sequence that contains ' + 'unhashable\n' + 'values will result in "TypeError".\n' + '\n' + '\n' + 'Mutable Sequence Types\n' + '======================\n' + '\n' + 'The operations in the following table are defined on mutable ' + 'sequence\n' + 'types. The "collections.abc.MutableSequence" ABC is provided ' + 'to make\n' + 'it easier to correctly implement these operations on custom ' + 'sequence\n' + 'types.\n' + '\n' + 'In the table *s* is an instance of a mutable sequence type, ' + '*t* is any\n' + 'iterable object and *x* is an arbitrary object that meets any ' + 'type and\n' + 'value restrictions imposed by *s* (for example, "bytearray" ' + 'only\n' + 'accepts integers that meet the value restriction "0 <= x <= ' + '255").\n' + '\n' + '+--------------------------------+----------------------------------+-----------------------+\n' + '| Operation | ' + 'Result | Notes |\n' + '+================================+==================================+=======================+\n' + '| "s[i] = x" | item *i* of *s* is replaced ' + 'by | |\n' + '| | ' + '*x* | |\n' + '+--------------------------------+----------------------------------+-----------------------+\n' + '| "s[i:j] = t" | slice of *s* from *i* to ' + '*j* is | |\n' + '| | replaced by the contents of ' + 'the | |\n' + '| | iterable ' + '*t* | |\n' + '+--------------------------------+----------------------------------+-----------------------+\n' + '| "del s[i:j]" | same as "s[i:j] = ' + '[]" | |\n' + '+--------------------------------+----------------------------------+-----------------------+\n' + '| "s[i:j:k] = t" | the elements of "s[i:j:k]" ' + 'are | (1) |\n' + '| | replaced by those of ' + '*t* | |\n' + '+--------------------------------+----------------------------------+-----------------------+\n' + '| "del s[i:j:k]" | removes the elements ' + 'of | |\n' + '| | "s[i:j:k]" from the ' + 'list | |\n' + '+--------------------------------+----------------------------------+-----------------------+\n' + '| "s.append(x)" | appends *x* to the end of ' + 'the | |\n' + '| | sequence (same ' + 'as | |\n' + '| | "s[len(s):len(s)] = ' + '[x]") | |\n' + '+--------------------------------+----------------------------------+-----------------------+\n' + '| "s.clear()" | removes all items from "s" ' + '(same | (5) |\n' + '| | as "del ' + 's[:]") | |\n' + '+--------------------------------+----------------------------------+-----------------------+\n' + '| "s.copy()" | creates a shallow copy of ' + '"s" | (5) |\n' + '| | (same as ' + '"s[:]") | |\n' + '+--------------------------------+----------------------------------+-----------------------+\n' + '| "s.extend(t)" | extends *s* with the ' + 'contents of | |\n' + '| | *t* (same as ' + '"s[len(s):len(s)] = | |\n' + '| | ' + 't") | |\n' + '+--------------------------------+----------------------------------+-----------------------+\n' + '| "s.insert(i, x)" | inserts *x* into *s* at ' + 'the | |\n' + '| | index given by *i* (same ' + 'as | |\n' + '| | "s[i:i] = ' + '[x]") | |\n' + '+--------------------------------+----------------------------------+-----------------------+\n' + '| "s.pop([i])" | retrieves the item at *i* ' + 'and | (2) |\n' + '| | also removes it from ' + '*s* | |\n' + '+--------------------------------+----------------------------------+-----------------------+\n' + '| "s.remove(x)" | remove the first item from ' + '*s* | (3) |\n' + '| | where "s[i] == ' + 'x" | |\n' + '+--------------------------------+----------------------------------+-----------------------+\n' + '| "s.reverse()" | reverses the items of *s* ' + 'in | (4) |\n' + '| | ' + 'place | |\n' + '+--------------------------------+----------------------------------+-----------------------+\n' + '\n' + 'Notes:\n' + '\n' + '1. *t* must have the same length as the slice it is ' + 'replacing.\n' + '\n' + '2. The optional argument *i* defaults to "-1", so that by ' + 'default\n' + ' the last item is removed and returned.\n' + '\n' + '3. "remove" raises "ValueError" when *x* is not found in *s*.\n' + '\n' + '4. The "reverse()" method modifies the sequence in place for\n' + ' economy of space when reversing a large sequence. To ' + 'remind users\n' + ' that it operates by side effect, it does not return the ' + 'reversed\n' + ' sequence.\n' + '\n' + '5. "clear()" and "copy()" are included for consistency with ' + 'the\n' + " interfaces of mutable containers that don't support " + 'slicing\n' + ' operations (such as "dict" and "set")\n' + '\n' + ' New in version 3.3: "clear()" and "copy()" methods.\n' + '\n' + '\n' + 'Lists\n' + '=====\n' + '\n' + 'Lists are mutable sequences, typically used to store ' + 'collections of\n' + 'homogeneous items (where the precise degree of similarity will ' + 'vary by\n' + 'application).\n' + '\n' + 'class class list([iterable])\n' + '\n' + ' Lists may be constructed in several ways:\n' + '\n' + ' * Using a pair of square brackets to denote the empty list: ' + '"[]"\n' + '\n' + ' * Using square brackets, separating items with commas: ' + '"[a]",\n' + ' "[a, b, c]"\n' + '\n' + ' * Using a list comprehension: "[x for x in iterable]"\n' + '\n' + ' * Using the type constructor: "list()" or "list(iterable)"\n' + '\n' + ' The constructor builds a list whose items are the same and ' + 'in the\n' + " same order as *iterable*'s items. *iterable* may be either " + 'a\n' + ' sequence, a container that supports iteration, or an ' + 'iterator\n' + ' object. If *iterable* is already a list, a copy is made ' + 'and\n' + ' returned, similar to "iterable[:]". For example, ' + '"list(\'abc\')"\n' + ' returns "[\'a\', \'b\', \'c\']" and "list( (1, 2, 3) )" ' + 'returns "[1, 2,\n' + ' 3]". If no argument is given, the constructor creates a new ' + 'empty\n' + ' list, "[]".\n' + '\n' + ' Many other operations also produce lists, including the ' + '"sorted()"\n' + ' built-in.\n' + '\n' + ' Lists implement all of the *common* and *mutable* sequence\n' + ' operations. Lists also provide the following additional ' + 'method:\n' + '\n' + ' sort(*, key=None, reverse=None)\n' + '\n' + ' This method sorts the list in place, using only "<" ' + 'comparisons\n' + ' between items. Exceptions are not suppressed - if any ' + 'comparison\n' + ' operations fail, the entire sort operation will fail ' + '(and the\n' + ' list will likely be left in a partially modified ' + 'state).\n' + '\n' + ' "sort()" accepts two arguments that can only be passed ' + 'by\n' + ' keyword (*keyword-only arguments*):\n' + '\n' + ' *key* specifies a function of one argument that is used ' + 'to\n' + ' extract a comparison key from each list element (for ' + 'example,\n' + ' "key=str.lower"). The key corresponding to each item in ' + 'the list\n' + ' is calculated once and then used for the entire sorting ' + 'process.\n' + ' The default value of "None" means that list items are ' + 'sorted\n' + ' directly without calculating a separate key value.\n' + '\n' + ' The "functools.cmp_to_key()" utility is available to ' + 'convert a\n' + ' 2.x style *cmp* function to a *key* function.\n' + '\n' + ' *reverse* is a boolean value. If set to "True", then ' + 'the list\n' + ' elements are sorted as if each comparison were ' + 'reversed.\n' + '\n' + ' This method modifies the sequence in place for economy ' + 'of space\n' + ' when sorting a large sequence. To remind users that it ' + 'operates\n' + ' by side effect, it does not return the sorted sequence ' + '(use\n' + ' "sorted()" to explicitly request a new sorted list ' + 'instance).\n' + '\n' + ' The "sort()" method is guaranteed to be stable. A sort ' + 'is\n' + ' stable if it guarantees not to change the relative order ' + 'of\n' + ' elements that compare equal --- this is helpful for ' + 'sorting in\n' + ' multiple passes (for example, sort by department, then ' + 'by salary\n' + ' grade).\n' + '\n' + ' **CPython implementation detail:** While a list is being ' + 'sorted,\n' + ' the effect of attempting to mutate, or even inspect, the ' + 'list is\n' + ' undefined. The C implementation of Python makes the ' + 'list appear\n' + ' empty for the duration, and raises "ValueError" if it ' + 'can detect\n' + ' that the list has been mutated during a sort.\n' + '\n' + '\n' + 'Tuples\n' + '======\n' + '\n' + 'Tuples are immutable sequences, typically used to store ' + 'collections of\n' + 'heterogeneous data (such as the 2-tuples produced by the ' + '"enumerate()"\n' + 'built-in). Tuples are also used for cases where an immutable ' + 'sequence\n' + 'of homogeneous data is needed (such as allowing storage in a ' + '"set" or\n' + '"dict" instance).\n' + '\n' + 'class class tuple([iterable])\n' + '\n' + ' Tuples may be constructed in a number of ways:\n' + '\n' + ' * Using a pair of parentheses to denote the empty tuple: ' + '"()"\n' + '\n' + ' * Using a trailing comma for a singleton tuple: "a," or ' + '"(a,)"\n' + '\n' + ' * Separating items with commas: "a, b, c" or "(a, b, c)"\n' + '\n' + ' * Using the "tuple()" built-in: "tuple()" or ' + '"tuple(iterable)"\n' + '\n' + ' The constructor builds a tuple whose items are the same and ' + 'in the\n' + " same order as *iterable*'s items. *iterable* may be either " + 'a\n' + ' sequence, a container that supports iteration, or an ' + 'iterator\n' + ' object. If *iterable* is already a tuple, it is returned\n' + ' unchanged. For example, "tuple(\'abc\')" returns "(\'a\', ' + '\'b\', \'c\')"\n' + ' and "tuple( [1, 2, 3] )" returns "(1, 2, 3)". If no ' + 'argument is\n' + ' given, the constructor creates a new empty tuple, "()".\n' + '\n' + ' Note that it is actually the comma which makes a tuple, not ' + 'the\n' + ' parentheses. The parentheses are optional, except in the ' + 'empty\n' + ' tuple case, or when they are needed to avoid syntactic ' + 'ambiguity.\n' + ' For example, "f(a, b, c)" is a function call with three ' + 'arguments,\n' + ' while "f((a, b, c))" is a function call with a 3-tuple as ' + 'the sole\n' + ' argument.\n' + '\n' + ' Tuples implement all of the *common* sequence operations.\n' + '\n' + 'For heterogeneous collections of data where access by name is ' + 'clearer\n' + 'than access by index, "collections.namedtuple()" may be a ' + 'more\n' + 'appropriate choice than a simple tuple object.\n' + '\n' + '\n' + 'Ranges\n' + '======\n' + '\n' + 'The "range" type represents an immutable sequence of numbers ' + 'and is\n' + 'commonly used for looping a specific number of times in "for" ' + 'loops.\n' + '\n' + 'class class range(stop)\n' + 'class class range(start, stop[, step])\n' + '\n' + ' The arguments to the range constructor must be integers ' + '(either\n' + ' built-in "int" or any object that implements the ' + '"__index__"\n' + ' special method). If the *step* argument is omitted, it ' + 'defaults to\n' + ' "1". If the *start* argument is omitted, it defaults to ' + '"0". If\n' + ' *step* is zero, "ValueError" is raised.\n' + '\n' + ' For a positive *step*, the contents of a range "r" are ' + 'determined\n' + ' by the formula "r[i] = start + step*i" where "i >= 0" and ' + '"r[i] <\n' + ' stop".\n' + '\n' + ' For a negative *step*, the contents of the range are still\n' + ' determined by the formula "r[i] = start + step*i", but the\n' + ' constraints are "i >= 0" and "r[i] > stop".\n' + '\n' + ' A range object will be empty if "r[0]" does not meet the ' + 'value\n' + ' constraint. Ranges do support negative indices, but these ' + 'are\n' + ' interpreted as indexing from the end of the sequence ' + 'determined by\n' + ' the positive indices.\n' + '\n' + ' Ranges containing absolute values larger than "sys.maxsize" ' + 'are\n' + ' permitted but some features (such as "len()") may raise\n' + ' "OverflowError".\n' + '\n' + ' Range examples:\n' + '\n' + ' >>> list(range(10))\n' + ' [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]\n' + ' >>> list(range(1, 11))\n' + ' [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]\n' + ' >>> list(range(0, 30, 5))\n' + ' [0, 5, 10, 15, 20, 25]\n' + ' >>> list(range(0, 10, 3))\n' + ' [0, 3, 6, 9]\n' + ' >>> list(range(0, -10, -1))\n' + ' [0, -1, -2, -3, -4, -5, -6, -7, -8, -9]\n' + ' >>> list(range(0))\n' + ' []\n' + ' >>> list(range(1, 0))\n' + ' []\n' + '\n' + ' Ranges implement all of the *common* sequence operations ' + 'except\n' + ' concatenation and repetition (due to the fact that range ' + 'objects\n' + ' can only represent sequences that follow a strict pattern ' + 'and\n' + ' repetition and concatenation will usually violate that ' + 'pattern).\n' + '\n' + 'The advantage of the "range" type over a regular "list" or ' + '"tuple" is\n' + 'that a "range" object will always take the same (small) amount ' + 'of\n' + 'memory, no matter the size of the range it represents (as it ' + 'only\n' + 'stores the "start", "stop" and "step" values, calculating ' + 'individual\n' + 'items and subranges as needed).\n' + '\n' + 'Range objects implement the "collections.abc.Sequence" ABC, ' + 'and\n' + 'provide features such as containment tests, element index ' + 'lookup,\n' + 'slicing and support for negative indices (see *Sequence Types ' + '---\n' + 'list, tuple, range*):\n' + '\n' + '>>> r = range(0, 20, 2)\n' + '>>> r\n' + 'range(0, 20, 2)\n' + '>>> 11 in r\n' + 'False\n' + '>>> 10 in r\n' + 'True\n' + '>>> r.index(10)\n' + '5\n' + '>>> r[5]\n' + '10\n' + '>>> r[:5]\n' + 'range(0, 10, 2)\n' + '>>> r[-1]\n' + '18\n' + '\n' + 'Testing range objects for equality with "==" and "!=" compares ' + 'them as\n' + 'sequences. That is, two range objects are considered equal if ' + 'they\n' + 'represent the same sequence of values. (Note that two range ' + 'objects\n' + 'that compare equal might have different "start", "stop" and ' + '"step"\n' + 'attributes, for example "range(0) == range(2, 1, 3)" or ' + '"range(0, 3,\n' + '2) == range(0, 4, 2)".)\n' + '\n' + 'Changed in version 3.2: Implement the Sequence ABC. Support ' + 'slicing\n' + 'and negative indices. Test "int" objects for membership in ' + 'constant\n' + 'time instead of iterating through all items.\n' + '\n' + "Changed in version 3.3: Define '==' and '!=' to compare range " + 'objects\n' + 'based on the sequence of values they define (instead of ' + 'comparing\n' + 'based on object identity).\n' + '\n' + 'New in version 3.3: The "start", "stop" and "step" ' + 'attributes.\n', + 'typesseq-mutable': '\n' + 'Mutable Sequence Types\n' + '**********************\n' + '\n' + 'The operations in the following table are defined on ' + 'mutable sequence\n' + 'types. The "collections.abc.MutableSequence" ABC is ' + 'provided to make\n' + 'it easier to correctly implement these operations on ' + 'custom sequence\n' + 'types.\n' + '\n' + 'In the table *s* is an instance of a mutable sequence ' + 'type, *t* is any\n' + 'iterable object and *x* is an arbitrary object that ' + 'meets any type and\n' + 'value restrictions imposed by *s* (for example, ' + '"bytearray" only\n' + 'accepts integers that meet the value restriction "0 <= ' + 'x <= 255").\n' + '\n' + '+--------------------------------+----------------------------------+-----------------------+\n' + '| Operation | ' + 'Result | ' + 'Notes |\n' + '+================================+==================================+=======================+\n' + '| "s[i] = x" | item *i* of *s* is ' + 'replaced by | |\n' + '| | ' + '*x* ' + '| |\n' + '+--------------------------------+----------------------------------+-----------------------+\n' + '| "s[i:j] = t" | slice of *s* from ' + '*i* to *j* is | |\n' + '| | replaced by the ' + 'contents of the | |\n' + '| | iterable ' + '*t* | |\n' + '+--------------------------------+----------------------------------+-----------------------+\n' + '| "del s[i:j]" | same as "s[i:j] = ' + '[]" | |\n' + '+--------------------------------+----------------------------------+-----------------------+\n' + '| "s[i:j:k] = t" | the elements of ' + '"s[i:j:k]" are | (1) |\n' + '| | replaced by those ' + 'of *t* | |\n' + '+--------------------------------+----------------------------------+-----------------------+\n' + '| "del s[i:j:k]" | removes the ' + 'elements of | |\n' + '| | "s[i:j:k]" from the ' + 'list | |\n' + '+--------------------------------+----------------------------------+-----------------------+\n' + '| "s.append(x)" | appends *x* to the ' + 'end of the | |\n' + '| | sequence (same ' + 'as | |\n' + '| | "s[len(s):len(s)] = ' + '[x]") | |\n' + '+--------------------------------+----------------------------------+-----------------------+\n' + '| "s.clear()" | removes all items ' + 'from "s" (same | (5) |\n' + '| | as "del ' + 's[:]") | |\n' + '+--------------------------------+----------------------------------+-----------------------+\n' + '| "s.copy()" | creates a shallow ' + 'copy of "s" | (5) |\n' + '| | (same as ' + '"s[:]") | |\n' + '+--------------------------------+----------------------------------+-----------------------+\n' + '| "s.extend(t)" | extends *s* with ' + 'the contents of | |\n' + '| | *t* (same as ' + '"s[len(s):len(s)] = | |\n' + '| | ' + 't") ' + '| |\n' + '+--------------------------------+----------------------------------+-----------------------+\n' + '| "s.insert(i, x)" | inserts *x* into ' + '*s* at the | |\n' + '| | index given by *i* ' + '(same as | |\n' + '| | "s[i:i] = ' + '[x]") | |\n' + '+--------------------------------+----------------------------------+-----------------------+\n' + '| "s.pop([i])" | retrieves the item ' + 'at *i* and | (2) |\n' + '| | also removes it ' + 'from *s* | |\n' + '+--------------------------------+----------------------------------+-----------------------+\n' + '| "s.remove(x)" | remove the first ' + 'item from *s* | (3) |\n' + '| | where "s[i] == ' + 'x" | |\n' + '+--------------------------------+----------------------------------+-----------------------+\n' + '| "s.reverse()" | reverses the items ' + 'of *s* in | (4) |\n' + '| | ' + 'place ' + '| |\n' + '+--------------------------------+----------------------------------+-----------------------+\n' + '\n' + 'Notes:\n' + '\n' + '1. *t* must have the same length as the slice it is ' + 'replacing.\n' + '\n' + '2. The optional argument *i* defaults to "-1", so that ' + 'by default\n' + ' the last item is removed and returned.\n' + '\n' + '3. "remove" raises "ValueError" when *x* is not found ' + 'in *s*.\n' + '\n' + '4. The "reverse()" method modifies the sequence in ' + 'place for\n' + ' economy of space when reversing a large sequence. ' + 'To remind users\n' + ' that it operates by side effect, it does not return ' + 'the reversed\n' + ' sequence.\n' + '\n' + '5. "clear()" and "copy()" are included for consistency ' + 'with the\n' + " interfaces of mutable containers that don't support " + 'slicing\n' + ' operations (such as "dict" and "set")\n' + '\n' + ' New in version 3.3: "clear()" and "copy()" ' + 'methods.\n', + 'unary': '\n' + 'Unary arithmetic and bitwise operations\n' + '***************************************\n' + '\n' + 'All unary arithmetic and bitwise operations have the same ' + 'priority:\n' + '\n' + ' u_expr ::= power | "-" u_expr | "+" u_expr | "~" u_expr\n' + '\n' + 'The unary "-" (minus) operator yields the negation of its ' + 'numeric\n' + 'argument.\n' + '\n' + 'The unary "+" (plus) operator yields its numeric argument ' + 'unchanged.\n' + '\n' + 'The unary "~" (invert) operator yields the bitwise inversion of ' + 'its\n' + 'integer argument. The bitwise inversion of "x" is defined as\n' + '"-(x+1)". It only applies to integral numbers.\n' + '\n' + 'In all three cases, if the argument does not have the proper ' + 'type, a\n' + '"TypeError" exception is raised.\n', + 'while': '\n' + 'The "while" statement\n' + '*********************\n' + '\n' + 'The "while" statement is used for repeated execution as long as ' + 'an\n' + 'expression is true:\n' + '\n' + ' while_stmt ::= "while" expression ":" suite\n' + ' ["else" ":" suite]\n' + '\n' + 'This repeatedly tests the expression and, if it is true, executes ' + 'the\n' + 'first suite; if the expression is false (which may be the first ' + 'time\n' + 'it is tested) the suite of the "else" clause, if present, is ' + 'executed\n' + 'and the loop terminates.\n' + '\n' + 'A "break" statement executed in the first suite terminates the ' + 'loop\n' + 'without executing the "else" clause\'s suite. A "continue" ' + 'statement\n' + 'executed in the first suite skips the rest of the suite and goes ' + 'back\n' + 'to testing the expression.\n', + 'with': '\n' + 'The "with" statement\n' + '********************\n' + '\n' + 'The "with" statement is used to wrap the execution of a block ' + 'with\n' + 'methods defined by a context manager (see section *With Statement\n' + 'Context Managers*). This allows common ' + '"try"..."except"..."finally"\n' + 'usage patterns to be encapsulated for convenient reuse.\n' + '\n' + ' with_stmt ::= "with" with_item ("," with_item)* ":" suite\n' + ' with_item ::= expression ["as" target]\n' + '\n' + 'The execution of the "with" statement with one "item" proceeds as\n' + 'follows:\n' + '\n' + '1. The context expression (the expression given in the ' + '"with_item")\n' + ' is evaluated to obtain a context manager.\n' + '\n' + '2. The context manager\'s "__exit__()" is loaded for later use.\n' + '\n' + '3. The context manager\'s "__enter__()" method is invoked.\n' + '\n' + '4. If a target was included in the "with" statement, the return\n' + ' value from "__enter__()" is assigned to it.\n' + '\n' + ' Note: The "with" statement guarantees that if the ' + '"__enter__()"\n' + ' method returns without an error, then "__exit__()" will ' + 'always be\n' + ' called. Thus, if an error occurs during the assignment to ' + 'the\n' + ' target list, it will be treated the same as an error ' + 'occurring\n' + ' within the suite would be. See step 6 below.\n' + '\n' + '5. The suite is executed.\n' + '\n' + '6. The context manager\'s "__exit__()" method is invoked. If an\n' + ' exception caused the suite to be exited, its type, value, and\n' + ' traceback are passed as arguments to "__exit__()". Otherwise, ' + 'three\n' + ' "None" arguments are supplied.\n' + '\n' + ' If the suite was exited due to an exception, and the return ' + 'value\n' + ' from the "__exit__()" method was false, the exception is ' + 'reraised.\n' + ' If the return value was true, the exception is suppressed, and\n' + ' execution continues with the statement following the "with"\n' + ' statement.\n' + '\n' + ' If the suite was exited for any reason other than an exception, ' + 'the\n' + ' return value from "__exit__()" is ignored, and execution ' + 'proceeds\n' + ' at the normal location for the kind of exit that was taken.\n' + '\n' + 'With more than one item, the context managers are processed as if\n' + 'multiple "with" statements were nested:\n' + '\n' + ' with A() as a, B() as b:\n' + ' suite\n' + '\n' + 'is equivalent to\n' + '\n' + ' with A() as a:\n' + ' with B() as b:\n' + ' suite\n' + '\n' + 'Changed in version 3.1: Support for multiple context expressions.\n' + '\n' + 'See also: **PEP 0343** - The "with" statement\n' + '\n' + ' The specification, background, and examples for the Python ' + '"with"\n' + ' statement.\n', + 'yield': '\n' + 'The "yield" statement\n' + '*********************\n' + '\n' + ' yield_stmt ::= yield_expression\n' + '\n' + 'A "yield" statement is semantically equivalent to a *yield\n' + 'expression*. The yield statement can be used to omit the ' + 'parentheses\n' + 'that would otherwise be required in the equivalent yield ' + 'expression\n' + 'statement. For example, the yield statements\n' + '\n' + ' yield \n' + ' yield from \n' + '\n' + 'are equivalent to the yield expression statements\n' + '\n' + ' (yield )\n' + ' (yield from )\n' + '\n' + 'Yield expressions and statements are only used when defining a\n' + '*generator* function, and are only used in the body of the ' + 'generator\n' + 'function. Using yield in a function definition is sufficient to ' + 'cause\n' + 'that definition to create a generator function instead of a ' + 'normal\n' + 'function.\n' + '\n' + 'For full details of "yield" semantics, refer to the *Yield\n' + 'expressions* section.\n'} diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -3,8 +3,8 @@ +++++++++++ -What's New in Python 3.5.1 -========================== +What's New in Python 3.5.1 release candidate 1? +=============================================== Release date: TBA @@ -113,7 +113,8 @@ ----- - Issue #25071: Windows installer should not require TargetDir - parameter when installing quietly + parameter when installing quietly. + What's New in Python 3.5.0 release candidate 4? =============================================== diff --git a/README b/README --- a/README +++ b/README @@ -1,13 +1,14 @@ -This is Python version 3.5.0 release candidate 4 -================================================ +This is Python version 3.5.0 +============================ Copyright (c) 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014, 2015 Python Software Foundation. All rights reserved. -Python 3.x is a new version of the language, which is incompatible with the 2.x -line of releases. The language is mostly the same, but many details, especially -how built-in objects like dictionaries and strings work, have changed -considerably, and a lot of deprecated features have finally been removed. +Python 3.x is a new version of the language, which is incompatible with the +2.x line of releases. The language is mostly the same, but many details, +especially how built-in objects like dictionaries and strings work, +have changed considerably, and a lot of deprecated features have finally +been removed. Build Instructions @@ -27,14 +28,14 @@ elsewhere it's just python. On Mac OS X, if you have configured Python with --enable-framework, you should -use "make frameworkinstall" to do the installation. Note that this installs the -Python executable in a place that is not normally on your PATH, you may want to -set up a symlink in /usr/local/bin. +use "make frameworkinstall" to do the installation. Note that this installs +the Python executable in a place that is not normally on your PATH, you may +want to set up a symlink in /usr/local/bin. On Windows, see PCbuild/readme.txt. -If you wish, you can create a subdirectory and invoke configure from there. For -example: +If you wish, you can create a subdirectory and invoke configure from there. +For example: mkdir debug cd debug @@ -42,21 +43,21 @@ make make test -(This will fail if you *also* built at the top-level directory. You should do a -"make clean" at the toplevel first.) +(This will fail if you *also* built at the top-level directory. +You should do a "make clean" at the toplevel first.) What's New ---------- -We try to have a comprehensive overview of the changes in the "What's New in +We have a comprehensive overview of the changes in the "What's New in Python 3.5" document, found at http://docs.python.org/3.5/whatsnew/3.5.html -For a more detailed change log, read Misc/NEWS (though this file, too, is -incomplete, and also doesn't list anything merged in from the 2.7 release under -development). +For a more detailed change log, read Misc/NEWS (though this file, too, +is incomplete, and also doesn't list anything merged in from the 2.7 +release under development). If you want to install multiple versions of Python see the section below entitled "Installing multiple versions". @@ -98,10 +99,11 @@ Testing ------- -To test the interpreter, type "make test" in the top-level directory. The test -set produces some output. You can generally ignore the messages about skipped -tests due to optional features which can't be imported. If a message is printed -about a failed test or a traceback or core dump is produced, something is wrong. +To test the interpreter, type "make test" in the top-level directory. +The test set produces some output. You can generally ignore the messages +about skipped tests due to optional features which can't be imported. +If a message is printed about a failed test or a traceback or core dump +is produced, something is wrong. By default, tests are prevented from overusing resources like disk space and memory. To enable these tests, run "make testall". @@ -121,14 +123,14 @@ On Unix and Mac systems if you intend to install multiple versions of Python using the same installation prefix (--prefix argument to the configure script) -you must take care that your primary python executable is not overwritten by the -installation of a different version. All files and directories installed using -"make altinstall" contain the major and minor version and can thus live -side-by-side. "make install" also creates ${prefix}/bin/python3 which refers to -${prefix}/bin/pythonX.Y. If you intend to install multiple versions using the -same prefix you must decide which version (if any) is your "primary" version. -Install that version using "make install". Install all other versions using -"make altinstall". +you must take care that your primary python executable is not overwritten by +the installation of a different version. All files and directories installed +using "make altinstall" contain the major and minor version and can thus live +side-by-side. "make install" also creates ${prefix}/bin/python3 which refers +to ${prefix}/bin/pythonX.Y. If you intend to install multiple versions using +the same prefix you must decide which version (if any) is your "primary" +version. Install that version using "make install". Install all other +versions using "make altinstall". For example, if you want to install Python 2.6, 2.7 and 3.5 with 2.7 being the primary version, you would execute "make install" in your 2.7 build directory @@ -139,7 +141,7 @@ ------------------------------ We're soliciting bug reports about all aspects of the language. Fixes are also -welcome, preferable in unified diff format. Please use the issue tracker: +welcome, preferably in unified diff format. Please use the issue tracker: http://bugs.python.org/ @@ -182,11 +184,11 @@ Copyright (c) 1991-1995 Stichting Mathematisch Centrum. All rights reserved. -See the file "LICENSE" for information on the history of this software, terms & -conditions for usage, and a DISCLAIMER OF ALL WARRANTIES. +See the file "LICENSE" for information on the history of this software, +terms & conditions for usage, and a DISCLAIMER OF ALL WARRANTIES. -This Python distribution contains *no* GNU General Public License (GPL) code, so -it may be used in proprietary projects. There are interfaces to some GNU code -but these are entirely optional. +This Python distribution contains *no* GNU General Public License (GPL) code, +so it may be used in proprietary projects. There are interfaces to some GNU +code but these are entirely optional. All trademarks referenced herein are property of their respective holders. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 13 17:24:19 2015 From: python-checkins at python.org (yury.selivanov) Date: Sun, 13 Sep 2015 15:24:19 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogd2hhdHNuZXcvMy41?= =?utf-8?q?=3A_Edits?= Message-ID: <20150913152415.114986.84011@psf.io> https://hg.python.org/cpython/rev/746add37495c changeset: 97986:746add37495c branch: 3.5 user: Yury Selivanov date: Sun Sep 13 11:21:25 2015 -0400 summary: whatsnew/3.5: Edits Patch by me and Elvis Pranskevichus files: Doc/whatsnew/3.5.rst | 388 ++++++++++++++++-------------- 1 files changed, 204 insertions(+), 184 deletions(-) diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -61,57 +61,56 @@ New syntax features: -* :pep:`492`, coroutines with async and await syntax. -* :pep:`465`, a new matrix multiplication operator: ``a @ b``. -* :pep:`448`, additional unpacking generalizations. +* :ref:`PEP 492 `, coroutines with async and await syntax. +* :ref:`PEP 465 `, a new matrix multiplication operator: ``a @ b``. +* :ref:`PEP 448 `, additional unpacking generalizations. New library modules: -* :mod:`typing`: :ref:`Type Hints ` (:pep:`484`). -* :mod:`zipapp`: :ref:`Improving Python ZIP Application Support - ` (:pep:`441`). +* :mod:`typing`: :ref:`PEP 484 -- Type Hints `. +* :mod:`zipapp`: :ref:`PEP 441 Improving Python ZIP Application Support + `. New built-in features: -* ``bytes % args``, ``bytearray % args``: :pep:`461` - Adding ``%`` formatting - to bytes and bytearray. +* ``bytes % args``, ``bytearray % args``: :ref:`PEP 461 ` -- + Adding ``%`` formatting to bytes and bytearray. * ``b'\xf0\x9f\x90\x8d'.hex()``, ``bytearray(b'\xf0\x9f\x90\x8d').hex()``, ``memoryview(b'\xf0\x9f\x90\x8d').hex()``: :issue:`9951` - A ``hex`` method has been added to bytes, bytearray, and memoryview. -* :class:`memoryview` (including multi-dimensional) now supports tuple indexing. +* :class:`memoryview` now supports tuple indexing (including multi-dimensional). (Contributed by Antoine Pitrou in :issue:`23632`.) -* Generators have new ``gi_yieldfrom`` attribute, which returns the +* Generators have a new ``gi_yieldfrom`` attribute, which returns the object being iterated by ``yield from`` expressions. (Contributed by Benno Leslie and Yury Selivanov in :issue:`24450`.) -* New :exc:`RecursionError` exception. (Contributed by Georg Brandl +* A new :exc:`RecursionError` exception is now raised when maximum + recursion depth is reached. (Contributed by Georg Brandl in :issue:`19235`.) -* New :exc:`StopAsyncIteration` exception. (Contributed by - Yury Selivanov in :issue:`24017`. See also :pep:`492`.) - CPython implementation improvements: * When the ``LC_TYPE`` locale is the POSIX locale (``C`` locale), - :py:data:`sys.stdin` and :py:data:`sys.stdout` are now using the + :py:data:`sys.stdin` and :py:data:`sys.stdout` now use the ``surrogateescape`` error handler, instead of the ``strict`` error handler. (Contributed by Victor Stinner in :issue:`19977`.) * ``.pyo`` files are no longer used and have been replaced by a more flexible - scheme that inclides the optimization level explicitly in ``.pyc`` name. - (:pep:`488`) + scheme that includes the optimization level explicitly in ``.pyc`` name. + (See :ref:`PEP 488 overview `.) * Builtin and extension modules are now initialized in a multi-phase process, - which is similar to how Python modules are loaded. (:pep:`489`). - - -Significant improvements in standard library: + which is similar to how Python modules are loaded. + (See :ref:`PEP 489 overview `.) + + +Significant improvements in the standard library: * :class:`collections.OrderedDict` is now :ref:`implemented in C `, which makes it @@ -125,14 +124,14 @@ :ref:`better and significantly faster way ` of directory traversal. -* :func:`functools.lru_cache` has been largely +* :func:`functools.lru_cache` has been mostly :ref:`reimplemented in C `, yielding much better performance. * The new :func:`subprocess.run` function provides a :ref:`streamlined way to run subprocesses `. -* :mod:`traceback` module has been significantly +* The :mod:`traceback` module has been significantly :ref:`enhanced ` for improved performance and developer convenience. @@ -190,7 +189,7 @@ PEP 492 also adds :keyword:`async for` statement for convenient iteration over asynchronous iterables. -An example of a simple HTTP client written using the new syntax:: +An example of a rudimentary HTTP client written using the new syntax:: import asyncio @@ -249,7 +248,7 @@ be used inside a coroutine function declared with :keyword:`async def`. Coroutine functions are intended to be run inside a compatible event loop, -such as :class:`asyncio.Loop`. +such as the :ref:`asyncio loop `. .. seealso:: @@ -355,7 +354,7 @@ and :class:`bytearray`. While interpolation is usually thought of as a string operation, there are -cases where interpolation on ``bytes`` or ``bytearrays`` make sense, and the +cases where interpolation on ``bytes`` or ``bytearrays`` makes sense, and the work needed to make up for this missing functionality detracts from the overall readability of the code. This issue is particularly important when dealing with wire format protocols, which are often a mixture of binary @@ -415,8 +414,9 @@ While these annotations are available at runtime through the usual :attr:`__annotations__` attribute, *no automatic type checking happens at -runtime*. Instead, it is assumed that a separate off-line type checker will -be used for on-demand source code analysis. +runtime*. Instead, it is assumed that a separate off-line type checker +(e.g. `mypy `_) will be used for on-demand +source code analysis. The type system supports unions, generic types, and a special type named :class:`~typing.Any` which is consistent with (i.e. assignable to @@ -450,7 +450,7 @@ The following example shows a simple use of :func:`os.scandir` to display all the files (excluding directories) in the given *path* that don't start with -``'.'``. The :meth:`entry.is_file ` call will generally +``'.'``. The :meth:`entry.is_file() ` call will generally not make an additional system call:: for entry in os.scandir(path): @@ -468,7 +468,7 @@ PEP 475: Retry system calls failing with EINTR ---------------------------------------------- -A :py:data:`~errno.EINTR` error code is returned whenever a system call, that +A :py:data:`errno.EINTR` error code is returned whenever a system call, that is waiting for I/O, is interrupted by a signal. Previously, Python would raise :exc:`InterruptedError` in such case. This meant that, when writing a Python application, the developer had two choices: @@ -502,16 +502,16 @@ Below is a list of functions which are now retried when interrupted by a signal: -* :func:`open`, :func:`os.open`, :func:`io.open`; +* :func:`open` and :func:`io.open`; * functions of the :mod:`faulthandler` module; * :mod:`os` functions: :func:`~os.fchdir`, :func:`~os.fchmod`, :func:`~os.fchown`, :func:`~os.fdatasync`, :func:`~os.fstat`, :func:`~os.fstatvfs`, :func:`~os.fsync`, :func:`~os.ftruncate`, - :func:`~os.mkfifo`, :func:`~os.mknod`, :func:`~os.posix_fadvise`, - :func:`~os.posix_fallocate`, :func:`~os.pread`, :func:`~os.pwrite`, - :func:`~os.read`, :func:`~os.readv`, :func:`~os.sendfile`, + :func:`~os.mkfifo`, :func:`~os.mknod`, :func:`~os.open`, + :func:`~os.posix_fadvise`, :func:`~os.posix_fallocate`, :func:`~os.pread`, + :func:`~os.pwrite`, :func:`~os.read`, :func:`~os.readv`, :func:`~os.sendfile`, :func:`~os.wait3`, :func:`~os.wait4`, :func:`~os.wait`, :func:`~os.waitid`, :func:`~os.waitpid`, :func:`~os.write`, :func:`~os.writev`; @@ -520,18 +520,19 @@ :py:data:`~errno.EINTR` error, the syscall is not retried (see the PEP for the rationale); -* :mod:`select` functions: :func:`~select.devpoll.poll`, - :func:`~select.epoll.poll`, :func:`~select.kqueue.control`, - :func:`~select.poll.poll`, :func:`~select.select`; - -* :func:`socket.socket` methods: :meth:`~socket.socket.accept`, +* :mod:`select` functions: :func:`devpoll.poll() `, + :func:`epoll.poll() `, + :func:`kqueue.control() `, + :func:`poll.poll() `, :func:`~select.select`; + +* methods of the :class:`~socket.socket` class: :meth:`~socket.socket.accept`, :meth:`~socket.socket.connect` (except for non-blocking sockets), :meth:`~socket.socket.recv`, :meth:`~socket.socket.recvfrom`, :meth:`~socket.socket.recvmsg`, :meth:`~socket.socket.send`, :meth:`~socket.socket.sendall`, :meth:`~socket.socket.sendmsg`, :meth:`~socket.socket.sendto`; -* :func:`signal.sigtimedwait`, :func:`signal.sigwaitinfo`; +* :func:`signal.sigtimedwait` and :func:`signal.sigwaitinfo`; * :func:`time.sleep`. @@ -548,7 +549,7 @@ -------------------------------------------------------- The interaction of generators and :exc:`StopIteration` in Python 3.4 and -earlier was somewhat surprising, and could conceal obscure bugs. Previously, +earlier was sometimes surprising, and could conceal obscure bugs. Previously, ``StopIteration`` raised accidentally inside a generator function was interpreted as the end of the iteration by the loop construct driving the generator. @@ -564,7 +565,22 @@ This is a backwards incompatible change, so to enable the new behavior, a :term:`__future__` import is necessary:: - from __future__ import generator_stop + >>> from __future__ import generator_stop + + >>> def gen(): + ... next(iter([])) + ... yield + ... + >>> next(gen()) + Traceback (most recent call last): + File "", line 2, in gen + StopIteration + + The above exception was the direct cause of the following exception: + + Traceback (most recent call last): + File "", line 1, in + RuntimeError: generator raised StopIteration Without a ``__future__`` import, a :exc:`PendingDeprecationWarning` will be raised whenever a ``StopIteration`` exception is raised inside a generator. @@ -576,6 +592,44 @@ Chris Angelico, Yury Selivanov and Nick Coghlan. +.. _whatsnew-pep-485: + +PEP 485: A function for testing approximate equality +---------------------------------------------------- + +:pep:`485` adds the :func:`math.isclose` and :func:`cmath.isclose` +functions which tell whether two values are approximately equal or +"close" to each other. Whether or not two values are considered +close is determined according to given absolute and relative tolerances. +Relative tolerance is the maximum allowed difference between ``isclose`` +arguments, relative to the larger absolute value:: + + >>> import math + >>> a = 5.0 + >>> b = 4.99998 + >>> math.isclose(a, b, rel_tol=1e-5) + True + >>> math.isclose(a, b, rel_tol=1e-6) + False + +It is also possible to compare two values using absolute tolerance, which +must be a non-negative value:: + + >>> import math + >>> a = 5.0 + >>> b = 4.99998 + >>> math.isclose(a, b, abs_tol=0.00003) + True + >>> math.isclose(a, b, abs_tol=0.00001) + False + +.. seealso:: + + :pep:`485` -- A function for testing approximate equality + PEP written by Christopher Barker; implemented by Chris Barker and + Tal Einat. + + .. _whatsnew-pep-486: PEP 486: Make the Python Launcher aware of virtual environments @@ -633,61 +687,20 @@ implemented by Petr Viktorin. -.. _whatsnew-pep-485: - -PEP 485: A function for testing approximate equality ----------------------------------------------------- - -:pep:`485` adds the :func:`math.isclose` and :func:`cmath.isclose` -functions which tell whether two values are approximately equal or -"close" to each other. Whether or not two values are considered -close is determined according to given absolute and relative tolerances. -Relative tolerance is the maximum allowed difference between ``isclose()`` -arguments, relative to the larger absolute value:: - - >>> import math - >>> a = 5.0 - >>> b = 4.99998 - >>> math.isclose(a, b, rel_tol=1e-5) - True - >>> math.isclose(a, b, rel_tol=1e-6) - False - -It is also possible to compare two values using absolute tolerance, which -must be a non-negative value:: - - >>> import math - >>> a = 5.0 - >>> b = 4.99998 - >>> math.isclose(a, b, abs_tol=0.00003) - True - >>> math.isclose(a, b, abs_tol=0.00001) - False - -.. seealso:: - - :pep:`485` -- A function for testing approximate equality - PEP written by Christopher Barker; implemented by Chris Barker and - Tal Einat. - - Other Language Changes ====================== Some smaller changes made to the core Python language are: * Added the ``"namereplace"`` error handlers. The ``"backslashreplace"`` - error handlers now works with decoding and translating. + error handlers now work with decoding and translating. (Contributed by Serhiy Storchaka in :issue:`19676` and :issue:`22286`.) * The :option:`-b` option now affects comparisons of :class:`bytes` with :class:`int`. (Contributed by Serhiy Storchaka in :issue:`23681`.) -* New Kazakh :ref:`codec ` ``kz1048``. (Contributed by - Serhiy Storchaka in :issue:`22682`.) - -* New Tajik :ref:`codec ` ``koi8_t``. (Contributed by - Serhiy Storchaka in :issue:`22681`.) +* New Kazakh ``kz1048`` and Tajik ``koi8_t`` :ref:`codecs `. + (Contributed by Serhiy Storchaka in :issue:`22682` and :issue:`22681`.) * Property docstrings are now writable. This is especially useful for :func:`collections.namedtuple` docstrings. @@ -750,9 +763,9 @@ Since :mod:`asyncio` module is :term:`provisional `, all changes introduced in Python 3.5 have also been backported to Python 3.4.x. -Notable changes in :mod:`asyncio` module since Python 3.4.0: - -* A new debugging APIs: :meth:`loop.set_debug() ` +Notable changes in the :mod:`asyncio` module since Python 3.4.0: + +* New debugging APIs: :meth:`loop.set_debug() ` and :meth:`loop.get_debug() ` methods. (Contributed by Victor Stinner.) @@ -766,11 +779,11 @@ * A new :meth:`loop.create_task() ` to conveniently create and schedule a new :class:`~asyncio.Task` for a coroutine. The ``create_task`` method is also used by all - asyncio functions that wrap coroutines into tasks: :func:`asyncio.wait`, - :func:`asyncio.gather`, etc. + asyncio functions that wrap coroutines into tasks, such as + :func:`asyncio.wait`, :func:`asyncio.gather`, etc. (Contributed by Victor Stinner.) -* A new :meth:`WriteTransport.get_write_buffer_limits ` +* A new :meth:`transport.get_write_buffer_limits() ` method to inquire for *high-* and *low-* water limits of the flow control. (Contributed by Victor Stinner.) @@ -813,13 +826,13 @@ ----- A new function :func:`~cmath.isclose` provides a way to test for approximate -equality. (Contributed by Chris Barker and Tal Einat in :issue:`24270`.) +equality. (Contributed by Chris Barker and Tal Einat in :issue:`24270`.) code ---- -The :func:`InteractiveInterpreter.showtraceback ` +The :func:`InteractiveInterpreter.showtraceback() ` method now prints the full chained traceback, just like the interactive interpreter. (Contributed by Claudiu Popa in :issue:`17442`.) @@ -832,9 +845,9 @@ The :class:`~collections.OrderedDict` class is now implemented in C, which makes it 4 to 100 times faster. (Contributed by Eric Snow in :issue:`16991`.) -:meth:`OrderedDict.items `, -:meth:`OrderedDict.keys `, -:meth:`OrderedDict.values ` views now support +:meth:`OrderedDict.items() `, +:meth:`OrderedDict.keys() `, +:meth:`OrderedDict.values() ` views now support :func:`reversed` iteration. (Contributed by Serhiy Storchaka in :issue:`19505`.) @@ -857,14 +870,14 @@ The :class:`~collections.UserString` class now implements :meth:`__getnewargs__`, :meth:`__rmod__`, :meth:`~str.casefold`, :meth:`~str.format_map`, :meth:`~str.isprintable`, and :meth:`~str.maketrans` -methods to match corresponding methods of :class:`str`. +methods to match the corresponding methods of :class:`str`. (Contributed by Joe Jevnik in :issue:`22189`.) collections.abc --------------- -The :meth:`Sequence.index ` method now +The :meth:`Sequence.index() ` method now accepts *start* and *stop* arguments to match the corresponding methods of :class:`tuple`, :class:`list`, etc. (Contributed by Devin Jeanpierre in :issue:`23086`.) @@ -877,6 +890,9 @@ :class:`~collections.abc.AsyncIterable` abstract base classes. (Contributed by Yury Selivanov in :issue:`24184`.) +For earlier Python versions, a backport of the new ABCs is available in an +external `PyPI package `_. + compileall ---------- @@ -900,7 +916,7 @@ concurrent.futures ------------------ -The :meth:`Executor.map ` method now accepts a +The :meth:`Executor.map() ` method now accepts a *chunksize* argument to allow batching of tasks to improve performance when :meth:`~concurrent.futures.ProcessPoolExecutor` is used. (Contributed by Dan O'Reilly in :issue:`11271`.) @@ -913,9 +929,11 @@ configparser ------------ -Config parsers can be customized by providing a dictionary of converters in the -constructor. All converters defined in config parser (either by subclassing or -by providing in a constructor) will be available on all section proxies. +:mod:`configparser` now provides a way to customize the conversion +of values by specifying a dictionary of converters in +:class:`~configparser.ConfigParser` constructor, or by defining them +as methods in ``ConfigParser`` subclasses. Converters defined in +parser instance are inherited by its section proxies. Example:: @@ -931,7 +949,9 @@ 'a b c d e f g' >>> cfg.getlist('s', 'list') ['a', 'b', 'c', 'd', 'e', 'f', 'g'] - + >>> section = cfg['s'] + >>> section.getlist('list') + ['a', 'b', 'c', 'd', 'e', 'f', 'g'] (Contributed by ?ukasz Langa in :issue:`18159`.) @@ -974,14 +994,14 @@ --- :func:`dumb.open ` always creates a new database when the flag -has the value ``'n'``. (Contributed by Claudiu Popa in :issue:`18039`.) +has the value ``"n"``. (Contributed by Claudiu Popa in :issue:`18039`.) difflib ------- The charset of HTML documents generated by -:meth:`HtmlDiff.make_file ` +:meth:`HtmlDiff.make_file() ` can now be customized by using a new *charset* keyword-only argument. The default charset of HTML document changed from ``"ISO-8859-1"`` to ``"utf-8"``. @@ -1022,7 +1042,7 @@ (Contributed by Milan Oberkirch in :issue:`20098`.) A new -:meth:`Message.get_content_disposition ` +:meth:`Message.get_content_disposition() ` method provides easy access to a canonical value for the :mailheader:`Content-Disposition` header. (Contributed by Abhilash Raj in :issue:`21083`.) @@ -1034,8 +1054,8 @@ ``SMTPUTF8`` extension. (Contributed by R. David Murray in :issue:`24211`.) -:class:`~email.mime.text.MIMEText` constructor now accepts a -:class:`~email.charset.Charset` instance. +The :class:`mime.text.MIMEText ` constructor now +accepts a :class:`charset.Charset ` instance. (Contributed by Claude Paroz and Berker Peksag in :issue:`16324`.) @@ -1158,7 +1178,7 @@ (Contributed by Tarek Ziad? and Serhiy Storchaka in :issue:`4972`.) The :mod:`imaplib` module now supports :rfc:`5161` (ENABLE Extension) -and :rfc:`6855` (UTF-8 Support) via the :meth:`IMAP4.enable ` +and :rfc:`6855` (UTF-8 Support) via the :meth:`IMAP4.enable() ` method. A new :attr:`IMAP4.utf8_enabled ` attribute, tracks whether or not :rfc:`6855` support is enabled. (Contributed by Milan Oberkirch, R. David Murray, and Maciej Szulik in @@ -1186,13 +1206,13 @@ lazy loading of modules in applications where startup time is important. (Contributed by Brett Cannon in :issue:`17621`.) -The :func:`abc.InspectLoader.source_to_code ` +The :func:`abc.InspectLoader.source_to_code() ` method is now a static method. This makes it easier to initialize a module object with code compiled from a string by running ``exec(code, module.__dict__)``. (Contributed by Brett Cannon in :issue:`21156`.) -The new :func:`util.module_from_spec ` +The new :func:`util.module_from_spec() ` function is now the preferred way to create a new module. As opposed to creating a :class:`types.ModuleType` instance directly, this new function will set the various import-controlled attributes based on the passed-in @@ -1207,7 +1227,7 @@ and :issue:`20334`.) A new -:meth:`BoundArguments.apply_defaults ` +:meth:`BoundArguments.apply_defaults() ` method provides a way to set default values for missing arguments:: >>> def foo(a, b='ham', *args): pass @@ -1219,7 +1239,7 @@ (Contributed by Yury Selivanov in :issue:`24190`.) A new class method -:meth:`Signature.from_callable ` makes +:meth:`Signature.from_callable() ` makes subclassing of :class:`~inspect.Signature` easier. (Contributed by Yury Selivanov and Eric Snow in :issue:`17373`.) @@ -1245,10 +1265,10 @@ io -- -A new :meth:`BufferedIOBase.readinto1 ` +A new :meth:`BufferedIOBase.readinto1() ` method, that uses at most one call to the underlying raw stream's -:meth:`RawIOBase.read ` (or -:meth:`RawIOBase.readinto `) method. +:meth:`RawIOBase.read() ` or +:meth:`RawIOBase.readinto() ` methods. (Contributed by Nikolaus Rath in :issue:`20578`.) @@ -1356,7 +1376,7 @@ lzma ---- -The :meth:`LZMADecompressor.decompress ` +The :meth:`LZMADecompressor.decompress() ` method now accepts an optional *max_length* argument to limit the maximum size of decompressed data. (Contributed by Martin Panter in :issue:`15955`.) @@ -1379,7 +1399,7 @@ multiprocessing --------------- -:func:`sharedctypes.synchronized ` +:func:`sharedctypes.synchronized() ` objects now support the :term:`context manager` protocol. (Contributed by Charles-Fran?ois Natali in :issue:`21565`.) @@ -1408,8 +1428,8 @@ On Windows, a new :attr:`stat_result.st_file_attributes ` -attribute is now available. It corresponds to ``dwFileAttributes`` member of -the ``BY_HANDLE_FILE_INFORMATION`` structure returned by +attribute is now available. It corresponds to the ``dwFileAttributes`` member +of the ``BY_HANDLE_FILE_INFORMATION`` structure returned by ``GetFileInformationByHandle()``. (Contributed by Ben Hoyt in :issue:`21719`.) The :func:`~os.urandom` function now uses ``getrandom()`` syscall on Linux 3.17 @@ -1441,7 +1461,7 @@ pathlib ------- -The new :meth:`Path.samefile ` method can be used +The new :meth:`Path.samefile() ` method can be used to check whether the path points to the same file as other path, which can be either an another :class:`~pathlib.Path` object, or a string:: @@ -1453,23 +1473,23 @@ (Contributed by Vajrasky Kok and Antoine Pitrou in :issue:`19775`.) -The :meth:`Path.mkdir ` method how accepts a new optional +The :meth:`Path.mkdir() ` method how accepts a new optional *exist_ok* argument to match ``mkdir -p`` and :func:`os.makrdirs` functionality. (Contributed by Berker Peksag in :issue:`21539`.) -There is a new :meth:`Path.expanduser ` method to +There is a new :meth:`Path.expanduser() ` method to expand ``~`` and ``~user`` prefixes. (Contributed by Serhiy Storchaka and Claudiu Popa in :issue:`19776`.) -A new :meth:`Path.home ` class method can be used to get +A new :meth:`Path.home() ` class method can be used to get an instance of :class:`~pathlib.Path` object representing the user?s home directory. (Contributed by Victor Salgado and Mayank Tripathi in :issue:`19777`.) -New :meth:`Path.write_text `, -:meth:`Path.read_text `, -:meth:`Path.write_bytes `, -:meth:`Path.read_bytes ` methods to simplify +New :meth:`Path.write_text() `, +:meth:`Path.read_text() `, +:meth:`Path.write_bytes() `, +:meth:`Path.read_bytes() ` methods to simplify read/write operations on files. The following code snippet will create or rewrite existing file @@ -1495,7 +1515,7 @@ poplib ------ -A new :meth:`POP3.utf8 ` command enables :rfc:`6856` +A new :meth:`POP3.utf8() ` command enables :rfc:`6856` (Internationalized Email) support, if a POP server supports it. (Contributed by Milan OberKirch in :issue:`21804`.) @@ -1587,27 +1607,27 @@ accept a *decode_data* keyword argument to determine if the ``DATA`` portion of the SMTP transaction is decoded using the ``"utf-8"`` codec or is instead provided to the -:meth:`SMTPServer.process_message ` +:meth:`SMTPServer.process_message() ` method as a byte string. The default is ``True`` for backward compatibility reasons, but will change to ``False`` in Python 3.6. If *decode_data* is set -to ``False``, the :meth:`~smtpd.SMTPServer.process_message` method must -be prepared to accept keyword arguments. +to ``False``, the ``process_message`` method must be prepared to accept keyword +arguments. (Contributed by Maciej Szulik in :issue:`19662`.) The :class:`~smtpd.SMTPServer` class now advertises the ``8BITMIME`` extension (:rfc:`6152`) if *decode_data* has been set ``True``. If the client specifies ``BODY=8BITMIME`` on the ``MAIL`` command, it is passed to -:meth:`SMTPServer.process_message ` +:meth:`SMTPServer.process_message() ` via the *mail_options* keyword. (Contributed by Milan Oberkirch and R. David Murray in :issue:`21795`.) The :class:`~smtpd.SMTPServer` class now also supports the ``SMTPUTF8`` extension (:rfc:`6531`: Internationalized Email). If the client specified ``SMTPUTF8 BODY=8BITMIME`` on the ``MAIL`` command, they are passed to -:meth:`SMTPServer.process_message ` +:meth:`SMTPServer.process_message() ` via the *mail_options* keyword. It is the responsibility of the -:meth:`~smtpd.SMTPServer.process_message` method to correctly handle the -``SMTPUTF8`` data. (Contributed by Milan Oberkirch in :issue:`21725`.) +``process_message`` method to correctly handle the ``SMTPUTF8`` data. +(Contributed by Milan Oberkirch in :issue:`21725`.) It is now possible to provide, directly or via name resolution, IPv6 addresses in the :class:`~smtpd.SMTPServer` constructor, and have it @@ -1617,16 +1637,16 @@ smtplib ------- -A new :meth:`SMTP.auth ` method provides a convenient way to +A new :meth:`SMTP.auth() ` method provides a convenient way to implement custom authentication mechanisms. (Contributed by Milan Oberkirch in :issue:`15014`.) -The :meth:`SMTP.set_debuglevel ` method now +The :meth:`SMTP.set_debuglevel() ` method now accepts an additional debuglevel (2), which enables timestamps in debug messages. (Contributed by Gavin Chappell and Maciej Szulik in :issue:`16914`.) -Both :meth:`SMTP.sendmail ` and -:meth:`SMTP.send_message ` methods now +Both :meth:`SMTP.sendmail() ` and +:meth:`SMTP.send_message() ` methods now support support :rfc:`6531` (SMTPUTF8). (Contributed by Milan Oberkirch and R. David Murray in :issue:`22027`.) @@ -1645,18 +1665,18 @@ Functions with timeouts now use a monotonic clock, instead of a system clock. (Contributed by Victor Stinner in :issue:`22043`.) -A new :meth:`socket.sendfile ` method allows to +A new :meth:`socket.sendfile() ` method allows to send a file over a socket by using the high-performance :func:`os.sendfile` function on UNIX resulting in uploads being from 2 to 3 times faster than when -using plain :meth:`socket.send `. +using plain :meth:`socket.send() `. (Contributed by Giampaolo Rodola' in :issue:`17552`.) -The :meth:`socket.sendall ` method no longer resets the +The :meth:`socket.sendall() ` method no longer resets the socket timeout every time bytes are received or sent. The socket timeout is now the maximum total duration to send all data. (Contributed by Victor Stinner in :issue:`23853`.) -The *backlog* argument of the :meth:`socket.listen ` +The *backlog* argument of the :meth:`socket.listen() ` method is now optional. By default it is set to :data:`SOMAXCONN ` or to ``128`` whichever is less. (Contributed by Charles-Fran?ois Natali in :issue:`21455`.) @@ -1674,7 +1694,7 @@ The new :class:`~ssl.SSLObject` class has been added to provide SSL protocol support for cases when the network I/O capabilities of :class:`~ssl.SSLSocket` -are not necessary or suboptimal. :class:`~ssl.SSLObject` represents +are not necessary or suboptimal. ``SSLObject`` represents an SSL protocol instance, but does not implement any network I/O methods, and instead provides a memory buffer interface. The new :class:`~ssl.MemoryBIO` class can be used to pass data between Python and an SSL protocol instance. @@ -1683,8 +1703,8 @@ implementing asynchronous I/O for which :class:`~ssl.SSLSocket`'s readiness model ("select/poll") is inefficient. -A new :meth:`SSLContext.wrap_bio ` method can be used -to create a new :class:`~ssl.SSLObject` instance. +A new :meth:`SSLContext.wrap_bio() ` method can be used +to create a new ``SSLObject`` instance. Application-Layer Protocol Negotiation Support @@ -1696,12 +1716,12 @@ *Application-Layer Protocol Negotiation* TLS extension as described in :rfc:`7301`. -The new :meth:`SSLContext.set_alpn_protocols ` +The new :meth:`SSLContext.set_alpn_protocols() ` can be used to specify which protocols a socket should advertise during the TLS handshake. The new -:meth:`SSLSocket.selected_alpn_protocol ` +:meth:`SSLSocket.selected_alpn_protocol() ` returns the protocol that was selected during the TLS handshake. :data:`~ssl.HAS_ALPN` flag indicates whether APLN support is present. @@ -1709,15 +1729,15 @@ Other Changes ~~~~~~~~~~~~~ -There is a new :meth:`SSLSocket.version ` method to query -the actual protocol version in use. +There is a new :meth:`SSLSocket.version() ` method to +query the actual protocol version in use. (Contributed by Antoine Pitrou in :issue:`20421`.) The :class:`~ssl.SSLSocket` class now implements -a :meth:`SSLSocket.sendfile ` method. +a :meth:`SSLSocket.sendfile() ` method. (Contributed by Giampaolo Rodola' in :issue:`17552`.) -The :meth:`SSLSocket.send ` method now raises either +The :meth:`SSLSocket.send() ` method now raises either :exc:`ssl.SSLWantReadError` or :exc:`ssl.SSLWantWriteError` exception on a non-blocking socket if the operation would block. Previously, it would return ``0``. (Contributed by Nikolaus Rath in :issue:`20951`.) @@ -1726,15 +1746,15 @@ as UTC and not as local time, per :rfc:`5280`. Additionally, the return value is always an :class:`int`. (Contributed by Akira Li in :issue:`19940`.) -New :meth:`SSLObject.shared_ciphers ` and -:meth:`SSLSocket.shared_ciphers ` methods return +New :meth:`SSLObject.shared_ciphers() ` and +:meth:`SSLSocket.shared_ciphers() ` methods return the list of ciphers sent by the client during the handshake. (Contributed by Benjamin Peterson in :issue:`23186`.) -The :meth:`SSLSocket.do_handshake `, -:meth:`SSLSocket.read `, -:meth:`SSLSocket.shutdown `, and -:meth:`SSLSocket.write ` methods of :class:`ssl.SSLSocket` +The :meth:`SSLSocket.do_handshake() `, +:meth:`SSLSocket.read() `, +:meth:`SSLSocket.shutdown() `, and +:meth:`SSLSocket.write() ` methods of :class:`~ssl.SSLSocket` class no longer reset the socket timeout every time bytes are received or sent. The socket timeout is now the maximum total duration of the method. (Contributed by Victor Stinner in :issue:`23853`.) @@ -1810,25 +1830,25 @@ The *mode* argument of the :func:`~tarfile.open` function now accepts ``"x"`` to request exclusive creation. (Contributed by Berker Peksag in :issue:`21717`.) -:meth:`TarFile.extractall ` and -:meth:`TarFile.extract ` methods now take a keyword +:meth:`TarFile.extractall() ` and +:meth:`TarFile.extract() ` methods now take a keyword argument *numeric_only*. If set to ``True``, the extracted files and directories will be owned by the numeric ``uid`` and ``gid`` from the tarfile. If set to ``False`` (the default, and the behavior in versions prior to 3.5), they will be owned by the named user and group in the tarfile. (Contributed by Michael Vogt and Eric Smith in :issue:`23193`.) -The :meth:`TarFile.list ` now accepts an optional +The :meth:`TarFile.list() ` now accepts an optional *members* keyword argument that can be set to a subset of the list returned -by :meth:`TarFile.getmembers `. +by :meth:`TarFile.getmembers() `. (Contributed by Serhiy Storchaka in :issue:`21549`.) threading --------- -Both :meth:`Lock.acquire ` and -:meth:`RLock.acquire ` methods +Both :meth:`Lock.acquire() ` and +:meth:`RLock.acquire() ` methods now use a monotonic clock for timeout management. (Contributed by Victor Stinner in :issue:`22043`.) @@ -1903,7 +1923,7 @@ unittest -------- -The :meth:`TestLoader.loadTestsFromModule ` +The :meth:`TestLoader.loadTestsFromModule() ` method now accepts a keyword-only argument *pattern* which is passed to ``load_tests`` as the third argument. Found packages are now checked for ``load_tests`` regardless of whether their path matches *pattern*, because it @@ -1929,7 +1949,7 @@ with ``"assert"``. (Contributed by Kushal Das in :issue:`21238`.) -* A new :meth:`Mock.assert_not_called ` +* A new :meth:`Mock.assert_not_called() ` method to check if the mock object was called. (Contributed by Kushal Das in :issue:`21262`.) @@ -1956,15 +1976,15 @@ :issue:`7159`.) A new *quote_via* argument for the -:func:`parse.urlencode ` +:func:`parse.urlencode() ` function provides a way to control the encoding of query parts if needed. (Contributed by Samwyse and Arnon Yaari in :issue:`13866`.) -The :func:`request.urlopen ` function accepts an +The :func:`request.urlopen() ` function accepts an :class:`ssl.SSLContext` object as a *context* argument, which will be used for the HTTPS connection. (Contributed by Alex Gaynor in :issue:`22366`.) -The :func:`parse.urljoin ` was updated to use the +The :func:`parse.urljoin() ` was updated to use the :rfc:`3986` semantics for the resolution of relative URLs, rather than :rfc:`1808` and :rfc:`2396`. (Contributed by Demian Brecht and Senthil Kumaran in :issue:`22118`.) @@ -2007,7 +2027,7 @@ ZIP output can now be written to unseekable streams. (Contributed by Serhiy Storchaka in :issue:`23252`.) -The *mode* argument of :meth:`ZipFile.open ` method now +The *mode* argument of :meth:`ZipFile.open() ` method now accepts ``"x"`` to request exclusive creation. (Contributed by Serhiy Storchaka in :issue:`21717`.) @@ -2120,8 +2140,8 @@ (Contributed by Georg Brandl in :issue:`19235`.) New :c:func:`PyModule_FromDefAndSpec`, :c:func:`PyModule_FromDefAndSpec2`, -and :c:func:`PyModule_ExecDef` introduced by :pep:`489` -- multi-phase -extension module initialization. +and :c:func:`PyModule_ExecDef` functions introduced by :pep:`489` -- +multi-phase extension module initialization. (Contributed by Petr Viktorin in :issue:`24268`.) New :c:func:`PyNumber_MatrixMultiply` and @@ -2193,7 +2213,7 @@ Deprecated Python Behavior -------------------------- -Raising :exc:`StopIteration` inside a generator will now generate a silent +Raising :exc:`StopIteration` exception inside a generator will now generate a silent :exc:`PendingDeprecationWarning`, which will become a non-silent deprecation warning in Python 3.6 and will trigger a :exc:`RuntimeError` in Python 3.7. See :ref:`PEP 479: Change StopIteration handling inside generators ` @@ -2224,10 +2244,10 @@ Directly assigning values to the :attr:`~http.cookies.Morsel.key`, :attr:`~http.cookies.Morsel.value` and -:attr:`~http.cookies.Morsel.coded_value` of :class:`~http.cookies.Morsel` -objects is deprecated. Use the :func:`~http.cookies.Morsel.set` method +:attr:`~http.cookies.Morsel.coded_value` of :class:`http.cookies.Morsel` +objects is deprecated. Use the :meth:`~http.cookies.Morsel.set` method instead. In addition, the undocumented *LegalChars* parameter of -:func:`~http.cookies.Morsel.set` is deprecated, and is now ignored. +:meth:`~http.cookies.Morsel.set` is deprecated, and is now ignored. Passing a format string as keyword argument *format_string* to the :meth:`~string.Formatter.format` method of the :class:`string.Formatter` @@ -2241,9 +2261,9 @@ (Contributed by Vajrasky Kok and Berker Peksag in :issue:`1322`.) The previously undocumented ``from_function`` and ``from_builtin`` methods of -:class:`inspect.Signature` are deprecated. Use new -:meth:`inspect.Signature.from_callable` instead. (Contributed by Yury -Selivanov in :issue:`24248`.) +:class:`inspect.Signature` are deprecated. Use the new +:meth:`Signature.from_callable() ` +method instead. (Contributed by Yury Selivanov in :issue:`24248`.) The :func:`inspect.getargspec` function is deprecated and scheduled to be removed in Python 3.6. (See :issue:`20438` for details.) @@ -2360,7 +2380,7 @@ :mod:`http.client` and :mod:`http.server` remain available for backwards compatibility. (Contributed by Demian Brecht in :issue:`21793`.) -* When an import loader defines :meth:`~importlib.machinery.Loader.exec_module` +* When an import loader defines :meth:`importlib.machinery.Loader.exec_module` it is now expected to also define :meth:`~importlib.machinery.Loader.create_module` (raises a :exc:`DeprecationWarning` now, will be an error in Python 3.6). If the loader @@ -2376,7 +2396,7 @@ an empty string (such as ``"\b"``) now raise an error. (Contributed by Serhiy Storchaka in :issue:`22818`.) -* The :class:`~http.cookies.Morsel` dict-like interface has been made self +* The :class:`http.cookies.Morsel` dict-like interface has been made self consistent: morsel comparison now takes the :attr:`~http.cookies.Morsel.key` and :attr:`~http.cookies.Morsel.value` into account, :meth:`~http.cookies.Morsel.copy` now results in a @@ -2400,7 +2420,7 @@ * The :mod:`socket` module now exports the :data:`~socket.CAN_RAW_FD_FRAMES` constant on linux 3.6 and greater. -* The :func:`~ssl.cert_time_to_seconds` function now interprets the input time +* The :func:`ssl.cert_time_to_seconds` function now interprets the input time as UTC and not as local time, per :rfc:`5280`. Additionally, the return value is always an :class:`int`. (Contributed by Akira Li in :issue:`19940`.) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 13 17:40:12 2015 From: python-checkins at python.org (yury.selivanov) Date: Sun, 13 Sep 2015 15:40:12 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogd2hhdHNuZXcvMy41?= =?utf-8?q?=3A_One_more_edit?= Message-ID: <20150913154011.66856.68736@psf.io> https://hg.python.org/cpython/rev/684f8908c1b2 changeset: 97987:684f8908c1b2 branch: 3.5 user: Yury Selivanov date: Sun Sep 13 11:40:00 2015 -0400 summary: whatsnew/3.5: One more edit files: Doc/whatsnew/3.5.rst | 8 +++----- 1 files changed, 3 insertions(+), 5 deletions(-) diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -44,12 +44,10 @@ This saves the maintainer the effort of going through the Mercurial log when researching a change. -Python 3.5 was released on September 13, 2015. - This article explains the new features in Python 3.5, compared to 3.4. -For full details, see the -`changelog `_. - +Python 3.4 was released on September 13, 2015. ?See the +`changelog `_ for a full +list of changes. .. seealso:: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 13 17:52:12 2015 From: python-checkins at python.org (yury.selivanov) Date: Sun, 13 Sep 2015 15:52:12 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogd2hhdHNuZXcvMy41?= =?utf-8?q?=3A_Fix_typo?= Message-ID: <20150913155212.12017.34381@psf.io> https://hg.python.org/cpython/rev/4661e0ad2278 changeset: 97988:4661e0ad2278 branch: 3.5 user: Yury Selivanov date: Sun Sep 13 11:52:07 2015 -0400 summary: whatsnew/3.5: Fix typo files: Doc/whatsnew/3.5.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -45,7 +45,7 @@ when researching a change. This article explains the new features in Python 3.5, compared to 3.4. -Python 3.4 was released on September 13, 2015. ?See the +Python 3.5 was released on September 13, 2015. ?See the `changelog `_ for a full list of changes. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 13 17:57:30 2015 From: python-checkins at python.org (larry.hastings) Date: Sun, 13 Sep 2015 15:57:30 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogTWVyZ2UgZnJvbSAzLjUu?= Message-ID: <20150913155730.27701.989@psf.io> https://hg.python.org/cpython/rev/b0302854eaa6 changeset: 97989:b0302854eaa6 parent: 97978:c31b1b63a0de parent: 97987:684f8908c1b2 user: Larry Hastings date: Sun Sep 13 16:57:16 2015 +0100 summary: Merge from 3.5. files: .hgtags | 1 + Doc/whatsnew/3.5.rst | 393 ++++++++++++++++-------------- Misc/NEWS | 8 +- README | 46 +- 4 files changed, 237 insertions(+), 211 deletions(-) diff --git a/.hgtags b/.hgtags --- a/.hgtags +++ b/.hgtags @@ -156,3 +156,4 @@ cc15d736d860303b9da90d43cd32db39bab048df v3.5.0rc2 66ed52375df802f9d0a34480daaa8ce79fc41313 v3.5.0rc3 2d033fedfa7f1e325fd14ccdaa9cb42155da206f v3.5.0rc4 +374f501f4567b7595f2ad7798aa09afa2456bb28 v3.5.0 diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -45,8 +45,9 @@ when researching a change. This article explains the new features in Python 3.5, compared to 3.4. -For full details, see the :source:`Misc/NEWS` file. - +Python 3.4 was released on September 13, 2015. ?See the +`changelog `_ for a full +list of changes. .. seealso:: @@ -58,57 +59,56 @@ New syntax features: -* :pep:`492`, coroutines with async and await syntax. -* :pep:`465`, a new matrix multiplication operator: ``a @ b``. -* :pep:`448`, additional unpacking generalizations. +* :ref:`PEP 492 `, coroutines with async and await syntax. +* :ref:`PEP 465 `, a new matrix multiplication operator: ``a @ b``. +* :ref:`PEP 448 `, additional unpacking generalizations. New library modules: -* :mod:`typing`: :ref:`Type Hints ` (:pep:`484`). -* :mod:`zipapp`: :ref:`Improving Python ZIP Application Support - ` (:pep:`441`). +* :mod:`typing`: :ref:`PEP 484 -- Type Hints `. +* :mod:`zipapp`: :ref:`PEP 441 Improving Python ZIP Application Support + `. New built-in features: -* ``bytes % args``, ``bytearray % args``: :pep:`461` - Adding ``%`` formatting - to bytes and bytearray. +* ``bytes % args``, ``bytearray % args``: :ref:`PEP 461 ` -- + Adding ``%`` formatting to bytes and bytearray. * ``b'\xf0\x9f\x90\x8d'.hex()``, ``bytearray(b'\xf0\x9f\x90\x8d').hex()``, ``memoryview(b'\xf0\x9f\x90\x8d').hex()``: :issue:`9951` - A ``hex`` method has been added to bytes, bytearray, and memoryview. -* :class:`memoryview` (including multi-dimensional) now supports tuple indexing. +* :class:`memoryview` now supports tuple indexing (including multi-dimensional). (Contributed by Antoine Pitrou in :issue:`23632`.) -* Generators have new ``gi_yieldfrom`` attribute, which returns the +* Generators have a new ``gi_yieldfrom`` attribute, which returns the object being iterated by ``yield from`` expressions. (Contributed by Benno Leslie and Yury Selivanov in :issue:`24450`.) -* New :exc:`RecursionError` exception. (Contributed by Georg Brandl +* A new :exc:`RecursionError` exception is now raised when maximum + recursion depth is reached. (Contributed by Georg Brandl in :issue:`19235`.) -* New :exc:`StopAsyncIteration` exception. (Contributed by - Yury Selivanov in :issue:`24017`. See also :pep:`492`.) - CPython implementation improvements: * When the ``LC_TYPE`` locale is the POSIX locale (``C`` locale), - :py:data:`sys.stdin` and :py:data:`sys.stdout` are now using the + :py:data:`sys.stdin` and :py:data:`sys.stdout` now use the ``surrogateescape`` error handler, instead of the ``strict`` error handler. (Contributed by Victor Stinner in :issue:`19977`.) * ``.pyo`` files are no longer used and have been replaced by a more flexible - scheme that inclides the optimization level explicitly in ``.pyc`` name. - (:pep:`488`) + scheme that includes the optimization level explicitly in ``.pyc`` name. + (See :ref:`PEP 488 overview `.) * Builtin and extension modules are now initialized in a multi-phase process, - which is similar to how Python modules are loaded. (:pep:`489`). - - -Significant improvements in standard library: + which is similar to how Python modules are loaded. + (See :ref:`PEP 489 overview `.) + + +Significant improvements in the standard library: * :class:`collections.OrderedDict` is now :ref:`implemented in C `, which makes it @@ -122,14 +122,14 @@ :ref:`better and significantly faster way ` of directory traversal. -* :func:`functools.lru_cache` has been largely +* :func:`functools.lru_cache` has been mostly :ref:`reimplemented in C `, yielding much better performance. * The new :func:`subprocess.run` function provides a :ref:`streamlined way to run subprocesses `. -* :mod:`traceback` module has been significantly +* The :mod:`traceback` module has been significantly :ref:`enhanced ` for improved performance and developer convenience. @@ -187,7 +187,7 @@ PEP 492 also adds :keyword:`async for` statement for convenient iteration over asynchronous iterables. -An example of a simple HTTP client written using the new syntax:: +An example of a rudimentary HTTP client written using the new syntax:: import asyncio @@ -246,7 +246,7 @@ be used inside a coroutine function declared with :keyword:`async def`. Coroutine functions are intended to be run inside a compatible event loop, -such as :class:`asyncio.Loop`. +such as the :ref:`asyncio loop `. .. seealso:: @@ -352,7 +352,7 @@ and :class:`bytearray`. While interpolation is usually thought of as a string operation, there are -cases where interpolation on ``bytes`` or ``bytearrays`` make sense, and the +cases where interpolation on ``bytes`` or ``bytearrays`` makes sense, and the work needed to make up for this missing functionality detracts from the overall readability of the code. This issue is particularly important when dealing with wire format protocols, which are often a mixture of binary @@ -412,8 +412,9 @@ While these annotations are available at runtime through the usual :attr:`__annotations__` attribute, *no automatic type checking happens at -runtime*. Instead, it is assumed that a separate off-line type checker will -be used for on-demand source code analysis. +runtime*. Instead, it is assumed that a separate off-line type checker +(e.g. `mypy `_) will be used for on-demand +source code analysis. The type system supports unions, generic types, and a special type named :class:`~typing.Any` which is consistent with (i.e. assignable to @@ -447,7 +448,7 @@ The following example shows a simple use of :func:`os.scandir` to display all the files (excluding directories) in the given *path* that don't start with -``'.'``. The :meth:`entry.is_file ` call will generally +``'.'``. The :meth:`entry.is_file() ` call will generally not make an additional system call:: for entry in os.scandir(path): @@ -465,7 +466,7 @@ PEP 475: Retry system calls failing with EINTR ---------------------------------------------- -A :py:data:`~errno.EINTR` error code is returned whenever a system call, that +A :py:data:`errno.EINTR` error code is returned whenever a system call, that is waiting for I/O, is interrupted by a signal. Previously, Python would raise :exc:`InterruptedError` in such case. This meant that, when writing a Python application, the developer had two choices: @@ -499,16 +500,16 @@ Below is a list of functions which are now retried when interrupted by a signal: -* :func:`open`, :func:`os.open`, :func:`io.open`; +* :func:`open` and :func:`io.open`; * functions of the :mod:`faulthandler` module; * :mod:`os` functions: :func:`~os.fchdir`, :func:`~os.fchmod`, :func:`~os.fchown`, :func:`~os.fdatasync`, :func:`~os.fstat`, :func:`~os.fstatvfs`, :func:`~os.fsync`, :func:`~os.ftruncate`, - :func:`~os.mkfifo`, :func:`~os.mknod`, :func:`~os.posix_fadvise`, - :func:`~os.posix_fallocate`, :func:`~os.pread`, :func:`~os.pwrite`, - :func:`~os.read`, :func:`~os.readv`, :func:`~os.sendfile`, + :func:`~os.mkfifo`, :func:`~os.mknod`, :func:`~os.open`, + :func:`~os.posix_fadvise`, :func:`~os.posix_fallocate`, :func:`~os.pread`, + :func:`~os.pwrite`, :func:`~os.read`, :func:`~os.readv`, :func:`~os.sendfile`, :func:`~os.wait3`, :func:`~os.wait4`, :func:`~os.wait`, :func:`~os.waitid`, :func:`~os.waitpid`, :func:`~os.write`, :func:`~os.writev`; @@ -517,18 +518,19 @@ :py:data:`~errno.EINTR` error, the syscall is not retried (see the PEP for the rationale); -* :mod:`select` functions: :func:`~select.devpoll.poll`, - :func:`~select.epoll.poll`, :func:`~select.kqueue.control`, - :func:`~select.poll.poll`, :func:`~select.select`; - -* :func:`socket.socket` methods: :meth:`~socket.socket.accept`, +* :mod:`select` functions: :func:`devpoll.poll() `, + :func:`epoll.poll() `, + :func:`kqueue.control() `, + :func:`poll.poll() `, :func:`~select.select`; + +* methods of the :class:`~socket.socket` class: :meth:`~socket.socket.accept`, :meth:`~socket.socket.connect` (except for non-blocking sockets), :meth:`~socket.socket.recv`, :meth:`~socket.socket.recvfrom`, :meth:`~socket.socket.recvmsg`, :meth:`~socket.socket.send`, :meth:`~socket.socket.sendall`, :meth:`~socket.socket.sendmsg`, :meth:`~socket.socket.sendto`; -* :func:`signal.sigtimedwait`, :func:`signal.sigwaitinfo`; +* :func:`signal.sigtimedwait` and :func:`signal.sigwaitinfo`; * :func:`time.sleep`. @@ -545,7 +547,7 @@ -------------------------------------------------------- The interaction of generators and :exc:`StopIteration` in Python 3.4 and -earlier was somewhat surprising, and could conceal obscure bugs. Previously, +earlier was sometimes surprising, and could conceal obscure bugs. Previously, ``StopIteration`` raised accidentally inside a generator function was interpreted as the end of the iteration by the loop construct driving the generator. @@ -561,7 +563,22 @@ This is a backwards incompatible change, so to enable the new behavior, a :term:`__future__` import is necessary:: - from __future__ import generator_stop + >>> from __future__ import generator_stop + + >>> def gen(): + ... next(iter([])) + ... yield + ... + >>> next(gen()) + Traceback (most recent call last): + File "", line 2, in gen + StopIteration + + The above exception was the direct cause of the following exception: + + Traceback (most recent call last): + File "", line 1, in + RuntimeError: generator raised StopIteration Without a ``__future__`` import, a :exc:`PendingDeprecationWarning` will be raised whenever a ``StopIteration`` exception is raised inside a generator. @@ -573,6 +590,44 @@ Chris Angelico, Yury Selivanov and Nick Coghlan. +.. _whatsnew-pep-485: + +PEP 485: A function for testing approximate equality +---------------------------------------------------- + +:pep:`485` adds the :func:`math.isclose` and :func:`cmath.isclose` +functions which tell whether two values are approximately equal or +"close" to each other. Whether or not two values are considered +close is determined according to given absolute and relative tolerances. +Relative tolerance is the maximum allowed difference between ``isclose`` +arguments, relative to the larger absolute value:: + + >>> import math + >>> a = 5.0 + >>> b = 4.99998 + >>> math.isclose(a, b, rel_tol=1e-5) + True + >>> math.isclose(a, b, rel_tol=1e-6) + False + +It is also possible to compare two values using absolute tolerance, which +must be a non-negative value:: + + >>> import math + >>> a = 5.0 + >>> b = 4.99998 + >>> math.isclose(a, b, abs_tol=0.00003) + True + >>> math.isclose(a, b, abs_tol=0.00001) + False + +.. seealso:: + + :pep:`485` -- A function for testing approximate equality + PEP written by Christopher Barker; implemented by Chris Barker and + Tal Einat. + + .. _whatsnew-pep-486: PEP 486: Make the Python Launcher aware of virtual environments @@ -630,61 +685,20 @@ implemented by Petr Viktorin. -.. _whatsnew-pep-485: - -PEP 485: A function for testing approximate equality ----------------------------------------------------- - -:pep:`485` adds the :func:`math.isclose` and :func:`cmath.isclose` -functions which tell whether two values are approximately equal or -"close" to each other. Whether or not two values are considered -close is determined according to given absolute and relative tolerances. -Relative tolerance is the maximum allowed difference between ``isclose()`` -arguments, relative to the larger absolute value:: - - >>> import math - >>> a = 5.0 - >>> b = 4.99998 - >>> math.isclose(a, b, rel_tol=1e-5) - True - >>> math.isclose(a, b, rel_tol=1e-6) - False - -It is also possible to compare two values using absolute tolerance, which -must be a non-negative value:: - - >>> import math - >>> a = 5.0 - >>> b = 4.99998 - >>> math.isclose(a, b, abs_tol=0.00003) - True - >>> math.isclose(a, b, abs_tol=0.00001) - False - -.. seealso:: - - :pep:`485` -- A function for testing approximate equality - PEP written by Christopher Barker; implemented by Chris Barker and - Tal Einat. - - Other Language Changes ====================== Some smaller changes made to the core Python language are: * Added the ``"namereplace"`` error handlers. The ``"backslashreplace"`` - error handlers now works with decoding and translating. + error handlers now work with decoding and translating. (Contributed by Serhiy Storchaka in :issue:`19676` and :issue:`22286`.) * The :option:`-b` option now affects comparisons of :class:`bytes` with :class:`int`. (Contributed by Serhiy Storchaka in :issue:`23681`.) -* New Kazakh :ref:`codec ` ``kz1048``. (Contributed by - Serhiy Storchaka in :issue:`22682`.) - -* New Tajik :ref:`codec ` ``koi8_t``. (Contributed by - Serhiy Storchaka in :issue:`22681`.) +* New Kazakh ``kz1048`` and Tajik ``koi8_t`` :ref:`codecs `. + (Contributed by Serhiy Storchaka in :issue:`22682` and :issue:`22681`.) * Property docstrings are now writable. This is especially useful for :func:`collections.namedtuple` docstrings. @@ -747,9 +761,9 @@ Since :mod:`asyncio` module is :term:`provisional `, all changes introduced in Python 3.5 have also been backported to Python 3.4.x. -Notable changes in :mod:`asyncio` module since Python 3.4.0: - -* A new debugging APIs: :meth:`loop.set_debug() ` +Notable changes in the :mod:`asyncio` module since Python 3.4.0: + +* New debugging APIs: :meth:`loop.set_debug() ` and :meth:`loop.get_debug() ` methods. (Contributed by Victor Stinner.) @@ -763,11 +777,11 @@ * A new :meth:`loop.create_task() ` to conveniently create and schedule a new :class:`~asyncio.Task` for a coroutine. The ``create_task`` method is also used by all - asyncio functions that wrap coroutines into tasks: :func:`asyncio.wait`, - :func:`asyncio.gather`, etc. + asyncio functions that wrap coroutines into tasks, such as + :func:`asyncio.wait`, :func:`asyncio.gather`, etc. (Contributed by Victor Stinner.) -* A new :meth:`WriteTransport.get_write_buffer_limits ` +* A new :meth:`transport.get_write_buffer_limits() ` method to inquire for *high-* and *low-* water limits of the flow control. (Contributed by Victor Stinner.) @@ -810,13 +824,13 @@ ----- A new function :func:`~cmath.isclose` provides a way to test for approximate -equality. (Contributed by Chris Barker and Tal Einat in :issue:`24270`.) +equality. (Contributed by Chris Barker and Tal Einat in :issue:`24270`.) code ---- -The :func:`InteractiveInterpreter.showtraceback ` +The :func:`InteractiveInterpreter.showtraceback() ` method now prints the full chained traceback, just like the interactive interpreter. (Contributed by Claudiu Popa in :issue:`17442`.) @@ -829,9 +843,9 @@ The :class:`~collections.OrderedDict` class is now implemented in C, which makes it 4 to 100 times faster. (Contributed by Eric Snow in :issue:`16991`.) -:meth:`OrderedDict.items `, -:meth:`OrderedDict.keys `, -:meth:`OrderedDict.values ` views now support +:meth:`OrderedDict.items() `, +:meth:`OrderedDict.keys() `, +:meth:`OrderedDict.values() ` views now support :func:`reversed` iteration. (Contributed by Serhiy Storchaka in :issue:`19505`.) @@ -854,14 +868,14 @@ The :class:`~collections.UserString` class now implements :meth:`__getnewargs__`, :meth:`__rmod__`, :meth:`~str.casefold`, :meth:`~str.format_map`, :meth:`~str.isprintable`, and :meth:`~str.maketrans` -methods to match corresponding methods of :class:`str`. +methods to match the corresponding methods of :class:`str`. (Contributed by Joe Jevnik in :issue:`22189`.) collections.abc --------------- -The :meth:`Sequence.index ` method now +The :meth:`Sequence.index() ` method now accepts *start* and *stop* arguments to match the corresponding methods of :class:`tuple`, :class:`list`, etc. (Contributed by Devin Jeanpierre in :issue:`23086`.) @@ -874,6 +888,9 @@ :class:`~collections.abc.AsyncIterable` abstract base classes. (Contributed by Yury Selivanov in :issue:`24184`.) +For earlier Python versions, a backport of the new ABCs is available in an +external `PyPI package `_. + compileall ---------- @@ -897,7 +914,7 @@ concurrent.futures ------------------ -The :meth:`Executor.map ` method now accepts a +The :meth:`Executor.map() ` method now accepts a *chunksize* argument to allow batching of tasks to improve performance when :meth:`~concurrent.futures.ProcessPoolExecutor` is used. (Contributed by Dan O'Reilly in :issue:`11271`.) @@ -910,9 +927,11 @@ configparser ------------ -Config parsers can be customized by providing a dictionary of converters in the -constructor. All converters defined in config parser (either by subclassing or -by providing in a constructor) will be available on all section proxies. +:mod:`configparser` now provides a way to customize the conversion +of values by specifying a dictionary of converters in +:class:`~configparser.ConfigParser` constructor, or by defining them +as methods in ``ConfigParser`` subclasses. Converters defined in +parser instance are inherited by its section proxies. Example:: @@ -928,7 +947,9 @@ 'a b c d e f g' >>> cfg.getlist('s', 'list') ['a', 'b', 'c', 'd', 'e', 'f', 'g'] - + >>> section = cfg['s'] + >>> section.getlist('list') + ['a', 'b', 'c', 'd', 'e', 'f', 'g'] (Contributed by ?ukasz Langa in :issue:`18159`.) @@ -971,14 +992,14 @@ --- :func:`dumb.open ` always creates a new database when the flag -has the value ``'n'``. (Contributed by Claudiu Popa in :issue:`18039`.) +has the value ``"n"``. (Contributed by Claudiu Popa in :issue:`18039`.) difflib ------- The charset of HTML documents generated by -:meth:`HtmlDiff.make_file ` +:meth:`HtmlDiff.make_file() ` can now be customized by using a new *charset* keyword-only argument. The default charset of HTML document changed from ``"ISO-8859-1"`` to ``"utf-8"``. @@ -1019,7 +1040,7 @@ (Contributed by Milan Oberkirch in :issue:`20098`.) A new -:meth:`Message.get_content_disposition ` +:meth:`Message.get_content_disposition() ` method provides easy access to a canonical value for the :mailheader:`Content-Disposition` header. (Contributed by Abhilash Raj in :issue:`21083`.) @@ -1031,8 +1052,8 @@ ``SMTPUTF8`` extension. (Contributed by R. David Murray in :issue:`24211`.) -:class:`~email.mime.text.MIMEText` constructor now accepts a -:class:`~email.charset.Charset` instance. +The :class:`mime.text.MIMEText ` constructor now +accepts a :class:`charset.Charset ` instance. (Contributed by Claude Paroz and Berker Peksag in :issue:`16324`.) @@ -1155,7 +1176,7 @@ (Contributed by Tarek Ziad? and Serhiy Storchaka in :issue:`4972`.) The :mod:`imaplib` module now supports :rfc:`5161` (ENABLE Extension) -and :rfc:`6855` (UTF-8 Support) via the :meth:`IMAP4.enable ` +and :rfc:`6855` (UTF-8 Support) via the :meth:`IMAP4.enable() ` method. A new :attr:`IMAP4.utf8_enabled ` attribute, tracks whether or not :rfc:`6855` support is enabled. (Contributed by Milan Oberkirch, R. David Murray, and Maciej Szulik in @@ -1183,13 +1204,13 @@ lazy loading of modules in applications where startup time is important. (Contributed by Brett Cannon in :issue:`17621`.) -The :func:`abc.InspectLoader.source_to_code ` +The :func:`abc.InspectLoader.source_to_code() ` method is now a static method. This makes it easier to initialize a module object with code compiled from a string by running ``exec(code, module.__dict__)``. (Contributed by Brett Cannon in :issue:`21156`.) -The new :func:`util.module_from_spec ` +The new :func:`util.module_from_spec() ` function is now the preferred way to create a new module. As opposed to creating a :class:`types.ModuleType` instance directly, this new function will set the various import-controlled attributes based on the passed-in @@ -1204,7 +1225,7 @@ and :issue:`20334`.) A new -:meth:`BoundArguments.apply_defaults ` +:meth:`BoundArguments.apply_defaults() ` method provides a way to set default values for missing arguments:: >>> def foo(a, b='ham', *args): pass @@ -1216,7 +1237,7 @@ (Contributed by Yury Selivanov in :issue:`24190`.) A new class method -:meth:`Signature.from_callable ` makes +:meth:`Signature.from_callable() ` makes subclassing of :class:`~inspect.Signature` easier. (Contributed by Yury Selivanov and Eric Snow in :issue:`17373`.) @@ -1242,10 +1263,10 @@ io -- -A new :meth:`BufferedIOBase.readinto1 ` +A new :meth:`BufferedIOBase.readinto1() ` method, that uses at most one call to the underlying raw stream's -:meth:`RawIOBase.read ` (or -:meth:`RawIOBase.readinto `) method. +:meth:`RawIOBase.read() ` or +:meth:`RawIOBase.readinto() ` methods. (Contributed by Nikolaus Rath in :issue:`20578`.) @@ -1353,7 +1374,7 @@ lzma ---- -The :meth:`LZMADecompressor.decompress ` +The :meth:`LZMADecompressor.decompress() ` method now accepts an optional *max_length* argument to limit the maximum size of decompressed data. (Contributed by Martin Panter in :issue:`15955`.) @@ -1376,7 +1397,7 @@ multiprocessing --------------- -:func:`sharedctypes.synchronized ` +:func:`sharedctypes.synchronized() ` objects now support the :term:`context manager` protocol. (Contributed by Charles-Fran?ois Natali in :issue:`21565`.) @@ -1405,8 +1426,8 @@ On Windows, a new :attr:`stat_result.st_file_attributes ` -attribute is now available. It corresponds to ``dwFileAttributes`` member of -the ``BY_HANDLE_FILE_INFORMATION`` structure returned by +attribute is now available. It corresponds to the ``dwFileAttributes`` member +of the ``BY_HANDLE_FILE_INFORMATION`` structure returned by ``GetFileInformationByHandle()``. (Contributed by Ben Hoyt in :issue:`21719`.) The :func:`~os.urandom` function now uses ``getrandom()`` syscall on Linux 3.17 @@ -1438,7 +1459,7 @@ pathlib ------- -The new :meth:`Path.samefile ` method can be used +The new :meth:`Path.samefile() ` method can be used to check whether the path points to the same file as other path, which can be either an another :class:`~pathlib.Path` object, or a string:: @@ -1450,23 +1471,23 @@ (Contributed by Vajrasky Kok and Antoine Pitrou in :issue:`19775`.) -The :meth:`Path.mkdir ` method how accepts a new optional +The :meth:`Path.mkdir() ` method how accepts a new optional *exist_ok* argument to match ``mkdir -p`` and :func:`os.makrdirs` functionality. (Contributed by Berker Peksag in :issue:`21539`.) -There is a new :meth:`Path.expanduser ` method to +There is a new :meth:`Path.expanduser() ` method to expand ``~`` and ``~user`` prefixes. (Contributed by Serhiy Storchaka and Claudiu Popa in :issue:`19776`.) -A new :meth:`Path.home ` class method can be used to get +A new :meth:`Path.home() ` class method can be used to get an instance of :class:`~pathlib.Path` object representing the user?s home directory. (Contributed by Victor Salgado and Mayank Tripathi in :issue:`19777`.) -New :meth:`Path.write_text `, -:meth:`Path.read_text `, -:meth:`Path.write_bytes `, -:meth:`Path.read_bytes ` methods to simplify +New :meth:`Path.write_text() `, +:meth:`Path.read_text() `, +:meth:`Path.write_bytes() `, +:meth:`Path.read_bytes() ` methods to simplify read/write operations on files. The following code snippet will create or rewrite existing file @@ -1492,7 +1513,7 @@ poplib ------ -A new :meth:`POP3.utf8 ` command enables :rfc:`6856` +A new :meth:`POP3.utf8() ` command enables :rfc:`6856` (Internationalized Email) support, if a POP server supports it. (Contributed by Milan OberKirch in :issue:`21804`.) @@ -1584,27 +1605,27 @@ accept a *decode_data* keyword argument to determine if the ``DATA`` portion of the SMTP transaction is decoded using the ``"utf-8"`` codec or is instead provided to the -:meth:`SMTPServer.process_message ` +:meth:`SMTPServer.process_message() ` method as a byte string. The default is ``True`` for backward compatibility reasons, but will change to ``False`` in Python 3.6. If *decode_data* is set -to ``False``, the :meth:`~smtpd.SMTPServer.process_message` method must -be prepared to accept keyword arguments. +to ``False``, the ``process_message`` method must be prepared to accept keyword +arguments. (Contributed by Maciej Szulik in :issue:`19662`.) The :class:`~smtpd.SMTPServer` class now advertises the ``8BITMIME`` extension (:rfc:`6152`) if *decode_data* has been set ``True``. If the client specifies ``BODY=8BITMIME`` on the ``MAIL`` command, it is passed to -:meth:`SMTPServer.process_message ` +:meth:`SMTPServer.process_message() ` via the *mail_options* keyword. (Contributed by Milan Oberkirch and R. David Murray in :issue:`21795`.) The :class:`~smtpd.SMTPServer` class now also supports the ``SMTPUTF8`` extension (:rfc:`6531`: Internationalized Email). If the client specified ``SMTPUTF8 BODY=8BITMIME`` on the ``MAIL`` command, they are passed to -:meth:`SMTPServer.process_message ` +:meth:`SMTPServer.process_message() ` via the *mail_options* keyword. It is the responsibility of the -:meth:`~smtpd.SMTPServer.process_message` method to correctly handle the -``SMTPUTF8`` data. (Contributed by Milan Oberkirch in :issue:`21725`.) +``process_message`` method to correctly handle the ``SMTPUTF8`` data. +(Contributed by Milan Oberkirch in :issue:`21725`.) It is now possible to provide, directly or via name resolution, IPv6 addresses in the :class:`~smtpd.SMTPServer` constructor, and have it @@ -1614,16 +1635,16 @@ smtplib ------- -A new :meth:`SMTP.auth ` method provides a convenient way to +A new :meth:`SMTP.auth() ` method provides a convenient way to implement custom authentication mechanisms. (Contributed by Milan Oberkirch in :issue:`15014`.) -The :meth:`SMTP.set_debuglevel ` method now +The :meth:`SMTP.set_debuglevel() ` method now accepts an additional debuglevel (2), which enables timestamps in debug messages. (Contributed by Gavin Chappell and Maciej Szulik in :issue:`16914`.) -Both :meth:`SMTP.sendmail ` and -:meth:`SMTP.send_message ` methods now +Both :meth:`SMTP.sendmail() ` and +:meth:`SMTP.send_message() ` methods now support support :rfc:`6531` (SMTPUTF8). (Contributed by Milan Oberkirch and R. David Murray in :issue:`22027`.) @@ -1642,18 +1663,18 @@ Functions with timeouts now use a monotonic clock, instead of a system clock. (Contributed by Victor Stinner in :issue:`22043`.) -A new :meth:`socket.sendfile ` method allows to +A new :meth:`socket.sendfile() ` method allows to send a file over a socket by using the high-performance :func:`os.sendfile` function on UNIX resulting in uploads being from 2 to 3 times faster than when -using plain :meth:`socket.send `. +using plain :meth:`socket.send() `. (Contributed by Giampaolo Rodola' in :issue:`17552`.) -The :meth:`socket.sendall ` method no longer resets the +The :meth:`socket.sendall() ` method no longer resets the socket timeout every time bytes are received or sent. The socket timeout is now the maximum total duration to send all data. (Contributed by Victor Stinner in :issue:`23853`.) -The *backlog* argument of the :meth:`socket.listen ` +The *backlog* argument of the :meth:`socket.listen() ` method is now optional. By default it is set to :data:`SOMAXCONN ` or to ``128`` whichever is less. (Contributed by Charles-Fran?ois Natali in :issue:`21455`.) @@ -1671,7 +1692,7 @@ The new :class:`~ssl.SSLObject` class has been added to provide SSL protocol support for cases when the network I/O capabilities of :class:`~ssl.SSLSocket` -are not necessary or suboptimal. :class:`~ssl.SSLObject` represents +are not necessary or suboptimal. ``SSLObject`` represents an SSL protocol instance, but does not implement any network I/O methods, and instead provides a memory buffer interface. The new :class:`~ssl.MemoryBIO` class can be used to pass data between Python and an SSL protocol instance. @@ -1680,8 +1701,8 @@ implementing asynchronous I/O for which :class:`~ssl.SSLSocket`'s readiness model ("select/poll") is inefficient. -A new :meth:`SSLContext.wrap_bio ` method can be used -to create a new :class:`~ssl.SSLObject` instance. +A new :meth:`SSLContext.wrap_bio() ` method can be used +to create a new ``SSLObject`` instance. Application-Layer Protocol Negotiation Support @@ -1693,12 +1714,12 @@ *Application-Layer Protocol Negotiation* TLS extension as described in :rfc:`7301`. -The new :meth:`SSLContext.set_alpn_protocols ` +The new :meth:`SSLContext.set_alpn_protocols() ` can be used to specify which protocols a socket should advertise during the TLS handshake. The new -:meth:`SSLSocket.selected_alpn_protocol ` +:meth:`SSLSocket.selected_alpn_protocol() ` returns the protocol that was selected during the TLS handshake. :data:`~ssl.HAS_ALPN` flag indicates whether APLN support is present. @@ -1706,15 +1727,15 @@ Other Changes ~~~~~~~~~~~~~ -There is a new :meth:`SSLSocket.version ` method to query -the actual protocol version in use. +There is a new :meth:`SSLSocket.version() ` method to +query the actual protocol version in use. (Contributed by Antoine Pitrou in :issue:`20421`.) The :class:`~ssl.SSLSocket` class now implements -a :meth:`SSLSocket.sendfile ` method. +a :meth:`SSLSocket.sendfile() ` method. (Contributed by Giampaolo Rodola' in :issue:`17552`.) -The :meth:`SSLSocket.send ` method now raises either +The :meth:`SSLSocket.send() ` method now raises either :exc:`ssl.SSLWantReadError` or :exc:`ssl.SSLWantWriteError` exception on a non-blocking socket if the operation would block. Previously, it would return ``0``. (Contributed by Nikolaus Rath in :issue:`20951`.) @@ -1723,15 +1744,15 @@ as UTC and not as local time, per :rfc:`5280`. Additionally, the return value is always an :class:`int`. (Contributed by Akira Li in :issue:`19940`.) -New :meth:`SSLObject.shared_ciphers ` and -:meth:`SSLSocket.shared_ciphers ` methods return +New :meth:`SSLObject.shared_ciphers() ` and +:meth:`SSLSocket.shared_ciphers() ` methods return the list of ciphers sent by the client during the handshake. (Contributed by Benjamin Peterson in :issue:`23186`.) -The :meth:`SSLSocket.do_handshake `, -:meth:`SSLSocket.read `, -:meth:`SSLSocket.shutdown `, and -:meth:`SSLSocket.write ` methods of :class:`ssl.SSLSocket` +The :meth:`SSLSocket.do_handshake() `, +:meth:`SSLSocket.read() `, +:meth:`SSLSocket.shutdown() `, and +:meth:`SSLSocket.write() ` methods of :class:`~ssl.SSLSocket` class no longer reset the socket timeout every time bytes are received or sent. The socket timeout is now the maximum total duration of the method. (Contributed by Victor Stinner in :issue:`23853`.) @@ -1807,25 +1828,25 @@ The *mode* argument of the :func:`~tarfile.open` function now accepts ``"x"`` to request exclusive creation. (Contributed by Berker Peksag in :issue:`21717`.) -:meth:`TarFile.extractall ` and -:meth:`TarFile.extract ` methods now take a keyword +:meth:`TarFile.extractall() ` and +:meth:`TarFile.extract() ` methods now take a keyword argument *numeric_only*. If set to ``True``, the extracted files and directories will be owned by the numeric ``uid`` and ``gid`` from the tarfile. If set to ``False`` (the default, and the behavior in versions prior to 3.5), they will be owned by the named user and group in the tarfile. (Contributed by Michael Vogt and Eric Smith in :issue:`23193`.) -The :meth:`TarFile.list ` now accepts an optional +The :meth:`TarFile.list() ` now accepts an optional *members* keyword argument that can be set to a subset of the list returned -by :meth:`TarFile.getmembers `. +by :meth:`TarFile.getmembers() `. (Contributed by Serhiy Storchaka in :issue:`21549`.) threading --------- -Both :meth:`Lock.acquire ` and -:meth:`RLock.acquire ` methods +Both :meth:`Lock.acquire() ` and +:meth:`RLock.acquire() ` methods now use a monotonic clock for timeout management. (Contributed by Victor Stinner in :issue:`22043`.) @@ -1900,7 +1921,7 @@ unittest -------- -The :meth:`TestLoader.loadTestsFromModule ` +The :meth:`TestLoader.loadTestsFromModule() ` method now accepts a keyword-only argument *pattern* which is passed to ``load_tests`` as the third argument. Found packages are now checked for ``load_tests`` regardless of whether their path matches *pattern*, because it @@ -1926,7 +1947,7 @@ with ``"assert"``. (Contributed by Kushal Das in :issue:`21238`.) -* A new :meth:`Mock.assert_not_called ` +* A new :meth:`Mock.assert_not_called() ` method to check if the mock object was called. (Contributed by Kushal Das in :issue:`21262`.) @@ -1953,15 +1974,15 @@ :issue:`7159`.) A new *quote_via* argument for the -:func:`parse.urlencode ` +:func:`parse.urlencode() ` function provides a way to control the encoding of query parts if needed. (Contributed by Samwyse and Arnon Yaari in :issue:`13866`.) -The :func:`request.urlopen ` function accepts an +The :func:`request.urlopen() ` function accepts an :class:`ssl.SSLContext` object as a *context* argument, which will be used for the HTTPS connection. (Contributed by Alex Gaynor in :issue:`22366`.) -The :func:`parse.urljoin ` was updated to use the +The :func:`parse.urljoin() ` was updated to use the :rfc:`3986` semantics for the resolution of relative URLs, rather than :rfc:`1808` and :rfc:`2396`. (Contributed by Demian Brecht and Senthil Kumaran in :issue:`22118`.) @@ -2004,7 +2025,7 @@ ZIP output can now be written to unseekable streams. (Contributed by Serhiy Storchaka in :issue:`23252`.) -The *mode* argument of :meth:`ZipFile.open ` method now +The *mode* argument of :meth:`ZipFile.open() ` method now accepts ``"x"`` to request exclusive creation. (Contributed by Serhiy Storchaka in :issue:`21717`.) @@ -2117,8 +2138,8 @@ (Contributed by Georg Brandl in :issue:`19235`.) New :c:func:`PyModule_FromDefAndSpec`, :c:func:`PyModule_FromDefAndSpec2`, -and :c:func:`PyModule_ExecDef` introduced by :pep:`489` -- multi-phase -extension module initialization. +and :c:func:`PyModule_ExecDef` functions introduced by :pep:`489` -- +multi-phase extension module initialization. (Contributed by Petr Viktorin in :issue:`24268`.) New :c:func:`PyNumber_MatrixMultiply` and @@ -2190,7 +2211,7 @@ Deprecated Python Behavior -------------------------- -Raising :exc:`StopIteration` inside a generator will now generate a silent +Raising :exc:`StopIteration` exception inside a generator will now generate a silent :exc:`PendingDeprecationWarning`, which will become a non-silent deprecation warning in Python 3.6 and will trigger a :exc:`RuntimeError` in Python 3.7. See :ref:`PEP 479: Change StopIteration handling inside generators ` @@ -2221,10 +2242,10 @@ Directly assigning values to the :attr:`~http.cookies.Morsel.key`, :attr:`~http.cookies.Morsel.value` and -:attr:`~http.cookies.Morsel.coded_value` of :class:`~http.cookies.Morsel` -objects is deprecated. Use the :func:`~http.cookies.Morsel.set` method +:attr:`~http.cookies.Morsel.coded_value` of :class:`http.cookies.Morsel` +objects is deprecated. Use the :meth:`~http.cookies.Morsel.set` method instead. In addition, the undocumented *LegalChars* parameter of -:func:`~http.cookies.Morsel.set` is deprecated, and is now ignored. +:meth:`~http.cookies.Morsel.set` is deprecated, and is now ignored. Passing a format string as keyword argument *format_string* to the :meth:`~string.Formatter.format` method of the :class:`string.Formatter` @@ -2238,9 +2259,9 @@ (Contributed by Vajrasky Kok and Berker Peksag in :issue:`1322`.) The previously undocumented ``from_function`` and ``from_builtin`` methods of -:class:`inspect.Signature` are deprecated. Use new -:meth:`inspect.Signature.from_callable` instead. (Contributed by Yury -Selivanov in :issue:`24248`.) +:class:`inspect.Signature` are deprecated. Use the new +:meth:`Signature.from_callable() ` +method instead. (Contributed by Yury Selivanov in :issue:`24248`.) The :func:`inspect.getargspec` function is deprecated and scheduled to be removed in Python 3.6. (See :issue:`20438` for details.) @@ -2357,7 +2378,7 @@ :mod:`http.client` and :mod:`http.server` remain available for backwards compatibility. (Contributed by Demian Brecht in :issue:`21793`.) -* When an import loader defines :meth:`~importlib.machinery.Loader.exec_module` +* When an import loader defines :meth:`importlib.machinery.Loader.exec_module` it is now expected to also define :meth:`~importlib.machinery.Loader.create_module` (raises a :exc:`DeprecationWarning` now, will be an error in Python 3.6). If the loader @@ -2373,7 +2394,7 @@ an empty string (such as ``"\b"``) now raise an error. (Contributed by Serhiy Storchaka in :issue:`22818`.) -* The :class:`~http.cookies.Morsel` dict-like interface has been made self +* The :class:`http.cookies.Morsel` dict-like interface has been made self consistent: morsel comparison now takes the :attr:`~http.cookies.Morsel.key` and :attr:`~http.cookies.Morsel.value` into account, :meth:`~http.cookies.Morsel.copy` now results in a @@ -2397,7 +2418,7 @@ * The :mod:`socket` module now exports the :data:`~socket.CAN_RAW_FD_FRAMES` constant on linux 3.6 and greater. -* The :func:`~ssl.cert_time_to_seconds` function now interprets the input time +* The :func:`ssl.cert_time_to_seconds` function now interprets the input time as UTC and not as local time, per :rfc:`5280`. Additionally, the return value is always an :class:`int`. (Contributed by Akira Li in :issue:`19940`.) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -92,8 +92,8 @@ - Issue #25022: Removed very outdated PC/example_nt/ directory. -What's New in Python 3.5.1 -========================== +What's New in Python 3.5.1 release candidate 1? +=============================================== Release date: TBA @@ -194,6 +194,7 @@ - Issue #25022: Removed very outdated PC/example_nt/ directory. + What's New in Python 3.5.0 final? ================================= @@ -203,7 +204,8 @@ ----- - Issue #25071: Windows installer should not require TargetDir - parameter when installing quietly + parameter when installing quietly. + What's New in Python 3.5.0 release candidate 4? =============================================== diff --git a/README b/README --- a/README +++ b/README @@ -4,10 +4,11 @@ Copyright (c) 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014, 2015 Python Software Foundation. All rights reserved. -Python 3.x is a new version of the language, which is incompatible with the 2.x -line of releases. The language is mostly the same, but many details, especially -how built-in objects like dictionaries and strings work, have changed -considerably, and a lot of deprecated features have finally been removed. +Python 3.x is a new version of the language, which is incompatible with the +2.x line of releases. The language is mostly the same, but many details, +especially how built-in objects like dictionaries and strings work, +have changed considerably, and a lot of deprecated features have finally +been removed. Build Instructions @@ -33,8 +34,8 @@ On Windows, see PCbuild/readme.txt. -If you wish, you can create a subdirectory and invoke configure from there. For -example: +If you wish, you can create a subdirectory and invoke configure from there. +For example: mkdir debug cd debug @@ -42,21 +43,21 @@ make make test -(This will fail if you *also* built at the top-level directory. You should do a -"make clean" at the toplevel first.) +(This will fail if you *also* built at the top-level directory. +You should do a "make clean" at the toplevel first.) What's New ---------- -We try to have a comprehensive overview of the changes in the "What's New in +We have a comprehensive overview of the changes in the "What's New in Python 3.6" document, found at http://docs.python.org/3.6/whatsnew/3.6.html -For a more detailed change log, read Misc/NEWS (though this file, too, is -incomplete, and also doesn't list anything merged in from the 2.7 release under -development). +For a more detailed change log, read Misc/NEWS (though this file, too, +is incomplete, and also doesn't list anything merged in from the 2.7 +release under development). If you want to install multiple versions of Python see the section below entitled "Installing multiple versions". @@ -98,10 +99,11 @@ Testing ------- -To test the interpreter, type "make test" in the top-level directory. The test -set produces some output. You can generally ignore the messages about skipped -tests due to optional features which can't be imported. If a message is printed -about a failed test or a traceback or core dump is produced, something is wrong. +To test the interpreter, type "make test" in the top-level directory. +The test set produces some output. You can generally ignore the messages +about skipped tests due to optional features which can't be imported. +If a message is printed about a failed test or a traceback or core dump +is produced, something is wrong. By default, tests are prevented from overusing resources like disk space and memory. To enable these tests, run "make testall". @@ -139,7 +141,7 @@ ------------------------------ We're soliciting bug reports about all aspects of the language. Fixes are also -welcome, preferable in unified diff format. Please use the issue tracker: +welcome, preferably in unified diff format. Please use the issue tracker: http://bugs.python.org/ @@ -182,11 +184,11 @@ Copyright (c) 1991-1995 Stichting Mathematisch Centrum. All rights reserved. -See the file "LICENSE" for information on the history of this software, terms & -conditions for usage, and a DISCLAIMER OF ALL WARRANTIES. +See the file "LICENSE" for information on the history of this software, +terms & conditions for usage, and a DISCLAIMER OF ALL WARRANTIES. -This Python distribution contains *no* GNU General Public License (GPL) code, so -it may be used in proprietary projects. There are interfaces to some GNU code -but these are entirely optional. +This Python distribution contains *no* GNU General Public License (GPL) code, +so it may be used in proprietary projects. There are interfaces to some GNU +code but these are entirely optional. All trademarks referenced herein are property of their respective holders. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 13 18:51:49 2015 From: python-checkins at python.org (yury.selivanov) Date: Sun, 13 Sep 2015 16:51:49 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogTWVyZ2UgMy41?= Message-ID: <20150913165149.15724.75072@psf.io> https://hg.python.org/cpython/rev/a830b179b662 changeset: 97990:a830b179b662 parent: 97989:b0302854eaa6 parent: 97988:4661e0ad2278 user: Yury Selivanov date: Sun Sep 13 12:51:46 2015 -0400 summary: Merge 3.5 files: Doc/whatsnew/3.5.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -45,7 +45,7 @@ when researching a change. This article explains the new features in Python 3.5, compared to 3.4. -Python 3.4 was released on September 13, 2015. ?See the +Python 3.5 was released on September 13, 2015. ?See the `changelog `_ for a full list of changes. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 13 20:10:21 2015 From: python-checkins at python.org (serhiy.storchaka) Date: Sun, 13 Sep 2015 18:10:21 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Fixed_a_typo_in_the_-b_option=2E?= Message-ID: <20150913181021.11254.3079@psf.io> https://hg.python.org/cpython/rev/8912efaa96a8 changeset: 97995:8912efaa96a8 parent: 97993:1a8b20b8f90d parent: 97994:16d23c358848 user: Serhiy Storchaka date: Sun Sep 13 21:09:36 2015 +0300 summary: Fixed a typo in the -b option. files: Doc/library/compileall.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/library/compileall.rst b/Doc/library/compileall.rst --- a/Doc/library/compileall.rst +++ b/Doc/library/compileall.rst @@ -89,7 +89,7 @@ .. versionchanged:: 3.5 Added the ``-j``, ``-r``, and ``-qq`` options. ``-q`` option - was changed to a multilevel value. ``b`` will always produce a + was changed to a multilevel value. ``-b`` will always produce a byte-code file ending in ``.pyc``, never ``.pyo``. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 13 20:10:21 2015 From: python-checkins at python.org (serhiy.storchaka) Date: Sun, 13 Sep 2015 18:10:21 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Use_=3Amenuselection=3A_in_whatsnew/3=2E4=2E?= Message-ID: <20150913181020.11274.64449@psf.io> https://hg.python.org/cpython/rev/3c6604b46b04 changeset: 97992:3c6604b46b04 branch: 3.5 parent: 97988:4661e0ad2278 parent: 97991:5e7f9c8098d3 user: Serhiy Storchaka date: Sun Sep 13 21:06:06 2015 +0300 summary: Use :menuselection: in whatsnew/3.4. files: Doc/whatsnew/3.4.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst --- a/Doc/whatsnew/3.4.rst +++ b/Doc/whatsnew/3.4.rst @@ -974,7 +974,7 @@ import by other programs, it gets improvements with every release. See :file:`Lib/idlelib/NEWS.txt` for a cumulative list of changes since 3.3.0, as well as changes made in future 3.4.x releases. This file is also available -from the IDLE Help -> About Idle dialog. +from the IDLE :menuselection:`Help --> About IDLE` dialog. importlib -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 13 20:10:21 2015 From: python-checkins at python.org (serhiy.storchaka) Date: Sun, 13 Sep 2015 18:10:21 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogVXNlIDptZW51c2Vs?= =?utf-8?q?ection=3A_in_whatsnew/3=2E4=2E?= Message-ID: <20150913181020.15730.54678@psf.io> https://hg.python.org/cpython/rev/5e7f9c8098d3 changeset: 97991:5e7f9c8098d3 branch: 3.4 parent: 97956:1208c85af6d5 user: Serhiy Storchaka date: Sun Sep 13 21:05:37 2015 +0300 summary: Use :menuselection: in whatsnew/3.4. files: Doc/whatsnew/3.4.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst --- a/Doc/whatsnew/3.4.rst +++ b/Doc/whatsnew/3.4.rst @@ -974,7 +974,7 @@ import by other programs, it gets improvements with every release. See :file:`Lib/idlelib/NEWS.txt` for a cumulative list of changes since 3.3.0, as well as changes made in future 3.4.x releases. This file is also available -from the IDLE Help -> About Idle dialog. +from the IDLE :menuselection:`Help --> About IDLE` dialog. importlib -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 13 20:11:15 2015 From: python-checkins at python.org (serhiy.storchaka) Date: Sun, 13 Sep 2015 18:11:15 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_Fixed_a_typo_i?= =?utf-8?q?n_the_-b_option=2E?= Message-ID: <20150913181021.68865.44735@psf.io> https://hg.python.org/cpython/rev/16d23c358848 changeset: 97994:16d23c358848 branch: 3.5 parent: 97992:3c6604b46b04 user: Serhiy Storchaka date: Sun Sep 13 21:09:17 2015 +0300 summary: Fixed a typo in the -b option. files: Doc/library/compileall.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/library/compileall.rst b/Doc/library/compileall.rst --- a/Doc/library/compileall.rst +++ b/Doc/library/compileall.rst @@ -89,7 +89,7 @@ .. versionchanged:: 3.5 Added the ``-j``, ``-r``, and ``-qq`` options. ``-q`` option - was changed to a multilevel value. ``b`` will always produce a + was changed to a multilevel value. ``-b`` will always produce a byte-code file ending in ``.pyc``, never ``.pyo``. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 13 20:11:15 2015 From: python-checkins at python.org (serhiy.storchaka) Date: Sun, 13 Sep 2015 18:11:15 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Use_=3Amenuselection=3A_in_whatsnew/3=2E4=2E?= Message-ID: <20150913181021.27687.62076@psf.io> https://hg.python.org/cpython/rev/1a8b20b8f90d changeset: 97993:1a8b20b8f90d parent: 97990:a830b179b662 parent: 97992:3c6604b46b04 user: Serhiy Storchaka date: Sun Sep 13 21:06:40 2015 +0300 summary: Use :menuselection: in whatsnew/3.4. files: Doc/whatsnew/3.4.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst --- a/Doc/whatsnew/3.4.rst +++ b/Doc/whatsnew/3.4.rst @@ -974,7 +974,7 @@ import by other programs, it gets improvements with every release. See :file:`Lib/idlelib/NEWS.txt` for a cumulative list of changes since 3.3.0, as well as changes made in future 3.4.x releases. This file is also available -from the IDLE Help -> About Idle dialog. +from the IDLE :menuselection:`Help --> About IDLE` dialog. importlib -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 13 23:42:08 2015 From: python-checkins at python.org (steve.dower) Date: Sun, 13 Sep 2015 21:42:08 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogQ2xvc2VzICMyNTA3?= =?utf-8?q?8=3A_Document_InstallAllUsers_installer_parameter_default_0?= Message-ID: <20150913214208.68881.65770@psf.io> https://hg.python.org/cpython/rev/399b7d5616f1 changeset: 97996:399b7d5616f1 branch: 3.5 parent: 97994:16d23c358848 user: Steve Dower date: Sun Sep 13 14:39:26 2015 -0700 summary: Closes #25078: Document InstallAllUsers installer parameter default 0 files: Doc/using/windows.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/using/windows.rst b/Doc/using/windows.rst --- a/Doc/using/windows.rst +++ b/Doc/using/windows.rst @@ -89,7 +89,7 @@ +---------------------------+--------------------------------------+--------------------------+ | Name | Description | Default | +===========================+======================================+==========================+ -| InstallAllUsers | Perform a system-wide installation. | 1 | +| InstallAllUsers | Perform a system-wide installation. | 0 | +---------------------------+--------------------------------------+--------------------------+ | TargetDir | The installation directory | Selected based on | | | | InstallAllUsers | -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 13 23:42:08 2015 From: python-checkins at python.org (steve.dower) Date: Sun, 13 Sep 2015 21:42:08 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge_with_3=2E5?= Message-ID: <20150913214208.17967.91069@psf.io> https://hg.python.org/cpython/rev/7e2a1d6bddc7 changeset: 97997:7e2a1d6bddc7 parent: 97995:8912efaa96a8 parent: 97996:399b7d5616f1 user: Steve Dower date: Sun Sep 13 14:41:24 2015 -0700 summary: Merge with 3.5 files: Doc/using/windows.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/using/windows.rst b/Doc/using/windows.rst --- a/Doc/using/windows.rst +++ b/Doc/using/windows.rst @@ -89,7 +89,7 @@ +---------------------------+--------------------------------------+--------------------------+ | Name | Description | Default | +===========================+======================================+==========================+ -| InstallAllUsers | Perform a system-wide installation. | 1 | +| InstallAllUsers | Perform a system-wide installation. | 0 | +---------------------------+--------------------------------------+--------------------------+ | TargetDir | The installation directory | Selected based on | | | | InstallAllUsers | -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 14 01:27:07 2015 From: python-checkins at python.org (raymond.hettinger) Date: Sun, 13 Sep 2015 23:27:07 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Add_an_exact_type_match_fa?= =?utf-8?q?st_path_for_deque=5Fcopy=28=29=2E?= Message-ID: <20150913232707.66866.10842@psf.io> https://hg.python.org/cpython/rev/9296469dcaed changeset: 97998:9296469dcaed user: Raymond Hettinger date: Sun Sep 13 19:27:01 2015 -0400 summary: Add an exact type match fast path for deque_copy(). files: Modules/_collectionsmodule.c | 16 ++++++++++++++++ 1 files changed, 16 insertions(+), 0 deletions(-) diff --git a/Modules/_collectionsmodule.c b/Modules/_collectionsmodule.c --- a/Modules/_collectionsmodule.c +++ b/Modules/_collectionsmodule.c @@ -1205,6 +1205,22 @@ static PyObject * deque_copy(PyObject *deque) { + if (Py_TYPE(deque) == &deque_type) { + dequeobject *new_deque; + PyObject *rv; + + new_deque = (dequeobject *)deque_new(&deque_type, (PyObject *)NULL, (PyObject *)NULL); + if (new_deque == NULL) + return NULL; + new_deque->maxlen = ((dequeobject *)deque)->maxlen; + rv = deque_extend(new_deque, deque); + if (rv != NULL) { + Py_DECREF(rv); + return (PyObject *)new_deque; + } + Py_DECREF(new_deque); + return NULL; + } if (((dequeobject *)deque)->maxlen == -1) return PyObject_CallFunction((PyObject *)(Py_TYPE(deque)), "O", deque, NULL); else -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 14 07:03:09 2015 From: python-checkins at python.org (raymond.hettinger) Date: Mon, 14 Sep 2015 05:03:09 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Tighten_inner-loop_for_deq?= =?utf-8?b?dWVfaW5wbGFjZV9yZXBlYXQoKS4=?= Message-ID: <20150914050309.17967.22841@psf.io> https://hg.python.org/cpython/rev/c2c4400bb12c changeset: 97999:c2c4400bb12c user: Raymond Hettinger date: Mon Sep 14 01:03:04 2015 -0400 summary: Tighten inner-loop for deque_inplace_repeat(). files: Modules/_collectionsmodule.c | 10 ++++++---- 1 files changed, 6 insertions(+), 4 deletions(-) diff --git a/Modules/_collectionsmodule.c b/Modules/_collectionsmodule.c --- a/Modules/_collectionsmodule.c +++ b/Modules/_collectionsmodule.c @@ -568,7 +568,7 @@ return PyErr_NoMemory(); deque->state++; - for (i = 0 ; i < n-1 ; i++) { + for (i = 0 ; i < n-1 ; ) { if (deque->rightindex == BLOCKLEN - 1) { block *b = newblock(Py_SIZE(deque) + i); if (b == NULL) { @@ -582,9 +582,11 @@ MARK_END(b->rightlink); deque->rightindex = -1; } - deque->rightindex++; - Py_INCREF(item); - deque->rightblock->data[deque->rightindex] = item; + for ( ; i < n-1 && deque->rightindex != BLOCKLEN - 1 ; i++) { + deque->rightindex++; + Py_INCREF(item); + deque->rightblock->data[deque->rightindex] = item; + } } Py_SIZE(deque) += i; Py_INCREF(deque); -- Repository URL: https://hg.python.org/cpython From lp_benchmark_robot at intel.com Mon Sep 14 09:50:48 2015 From: lp_benchmark_robot at intel.com (lp_benchmark_robot) Date: Mon, 14 Sep 2015 07:50:48 +0000 Subject: [Python-checkins] Benchmark Results for Python Default 2015-09-14 Message-ID: <078AA0FFE8C7034097F90205717F504611D7C06E@IRSMSX102.ger.corp.intel.com> Results for project python_default-nightly, build date 2015-09-14 03:02:22 commit: 9296469dcaed8302e585b477879168725e6eeb31 revision date: 2015-09-13 23:27:01 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v3.4.3, with hash b4cbecbc0781e89a309d03b60a1f75f8499250e6 from 2015-02-25 12:15:33+00:00 ------------------------------------------------------------------------------------------ benchmark relative change since change since current rev with std_dev* last run v3.4.3 regrtest PGO ------------------------------------------------------------------------------------------ :-) django_v2 0.35674% 0.40007% 9.18679% 15.12022% :-| pybench 0.11765% 0.14174% -1.91488% 8.57644% :-( regex_v8 2.80246% -0.44122% -3.88524% 6.98850% :-| nbody 0.10682% 0.05816% -1.72290% 8.79249% :-( json_dump_v2 0.30424% -3.52035% -4.86184% 13.28794% :-| normal_startup 0.73087% 0.19563% 0.08681% 5.32052% ------------------------------------------------------------------------------------------ Note: Benchmark results are measured in seconds. * Relative Standard Deviation (Standard Deviation/Average) Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. From yselivanov.ml at gmail.com Fri Sep 11 17:01:37 2015 From: yselivanov.ml at gmail.com (Yury Selivanov) Date: Fri, 11 Sep 2015 11:01:37 -0400 Subject: [Python-checkins] Daily reference leaks (1c94ab9c5ecf): sum=18942 In-Reply-To: <20150911084715.15718.84096@psf.io> References: <20150911084715.15718.84096@psf.io> Message-ID: <55F2ECD1.10904@gmail.com> On 2015-09-11 4:47 AM, solipsis at pitrou.net wrote: > results for 1c94ab9c5ecf on branch "default" > -------------------------------------------- > > test_capi leaked [1598, 1598, 1598] references, sum=4794 > test_capi leaked [387, 389, 389] memory blocks, sum=1165 > test_deque leaked [257, 257, 257] references, sum=771 > test_deque leaked [79, 80, 80] memory blocks, sum=239 > test_functools leaked [0, 2, 2] memory blocks, sum=4 > test_multiprocessing_forkserver leaked [0, 38, 0] references, sum=38 > test_multiprocessing_forkserver leaked [0, 17, 0] memory blocks, sum=17 > test_threading leaked [3196, 3196, 3196] references, sum=9588 > test_threading leaked [774, 776, 776] memory blocks, sum=2326 > > > Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/psf-users/antoine/refleaks/reflogxAVMGq', '--timeout', '7200'] > _______________________________________________ > Python-checkins mailing list > Python-checkins at python.org > https://mail.python.org/mailman/listinfo/python-checkins I couldn't reproduce the above. Is it because of some misconfiguration? Although there seems to be some problem with test_collections. Once in two/three times I have this: yury at ysmbp ~/dev/py/cpython $ ./python.exe -m test -R3:3 test_collections [1/1] test_collections beginning 6 repetitions 123456 ...... test_collections leaked [-6, 0, 0] references, sum=-6 test_collections leaked [-3, 1, 0] memory blocks, sum=-2 1 test failed: test_collections Yury From lp_benchmark_robot at intel.com Mon Sep 14 09:55:48 2015 From: lp_benchmark_robot at intel.com (lp_benchmark_robot) Date: Mon, 14 Sep 2015 07:55:48 +0000 Subject: [Python-checkins] Benchmark Results for Python Default 2015-09-14 Message-ID: <078AA0FFE8C7034097F90205717F504611D7C08C@IRSMSX102.ger.corp.intel.com> Results for project python_2.7-nightly, build date 2015-09-14 03:43:14 commit: 63bbe9f8090992c81a273aa32c386581e5bd09e1 revision date: 2015-09-13 00:20:47 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v2.7.10, with hash 15c95b7d81dcf821daade360741e00714667653f from 2015-05-23 16:02:14+00:00 ------------------------------------------------------------------------------------------ benchmark relative change since change since current rev with std_dev* last run v2.7.10 regrtest PGO ------------------------------------------------------------------------------------------ :-) django_v2 0.21024% -0.06153% 4.85016% 10.52656% :-) pybench 0.16547% 0.05631% 6.76898% 7.19202% :-| regex_v8 0.53916% -0.11740% -1.18217% 7.12544% :-) nbody 0.22570% -0.07386% 9.04252% 3.06672% :-) json_dump_v2 0.20477% -0.52035% 3.96086% 14.79653% :-| normal_startup 1.82231% 0.56996% -1.23032% 2.50067% :-| ssbench 1.12235% 0.31370% 1.54159% 3.05895% ------------------------------------------------------------------------------------------ Note: Benchmark results for ssbench are measured in requests/second while all other are measured in seconds. * Relative Standard Deviation (Standard Deviation/Average) Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. From lp_benchmark_robot at intel.com Mon Sep 14 10:18:40 2015 From: lp_benchmark_robot at intel.com (lp_benchmark_robot) Date: Mon, 14 Sep 2015 08:18:40 +0000 Subject: [Python-checkins] Benchmark Results for Python Default 2015-09-14 Message-ID: <078AA0FFE8C7034097F90205717F504611D7C0C4@IRSMSX102.ger.corp.intel.com> Hi, Please disregard this email as these are the results for the 2.7 branch, not the default one. -----Original Message----- From: lp_benchmark_robot Sent: Monday, September 14, 2015 10:56 AM To: 'python-checkins at python.org'; 'langperf at lists.01.org' Subject: Benchmark Results for Python Default 2015-09-14 Results for project python_2.7-nightly, build date 2015-09-14 03:43:14 commit: 63bbe9f8090992c81a273aa32c386581e5bd09e1 revision date: 2015-09-13 00:20:47 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v2.7.10, with hash 15c95b7d81dcf821daade360741e00714667653f from 2015-05-23 16:02:14+00:00 ------------------------------------------------------------------------------------------ benchmark relative change since change since current rev with std_dev* last run v2.7.10 regrtest PGO ------------------------------------------------------------------------------------------ :-) django_v2 0.21024% -0.06153% 4.85016% 10.52656% :-) pybench 0.16547% 0.05631% 6.76898% 7.19202% :-| regex_v8 0.53916% -0.11740% -1.18217% 7.12544% :-) nbody 0.22570% -0.07386% 9.04252% 3.06672% :-) json_dump_v2 0.20477% -0.52035% 3.96086% 14.79653% :-| normal_startup 1.82231% 0.56996% -1.23032% 2.50067% :-| ssbench 1.12235% 0.31370% 1.54159% 3.05895% ------------------------------------------------------------------------------------------ Note: Benchmark results for ssbench are measured in requests/second while all other are measured in seconds. * Relative Standard Deviation (Standard Deviation/Average) Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. From lp_benchmark_robot at intel.com Mon Sep 14 10:18:45 2015 From: lp_benchmark_robot at intel.com (lp_benchmark_robot) Date: Mon, 14 Sep 2015 08:18:45 +0000 Subject: [Python-checkins] Benchmark Results for Python 2.7 2015-09-14 Message-ID: <078AA0FFE8C7034097F90205717F504611D7C0D9@IRSMSX102.ger.corp.intel.com> Results for project python_2.7-nightly, build date 2015-09-14 03:43:14 commit: 63bbe9f8090992c81a273aa32c386581e5bd09e1 revision date: 2015-09-13 00:20:47 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v2.7.10, with hash 15c95b7d81dcf821daade360741e00714667653f from 2015-05-23 16:02:14+00:00 ------------------------------------------------------------------------------------------ benchmark relative change since change since current rev with std_dev* last run v2.7.10 regrtest PGO ------------------------------------------------------------------------------------------ :-) django_v2 0.21024% -0.06153% 4.85016% 10.52656% :-) pybench 0.16547% 0.05631% 6.76898% 7.19202% :-| regex_v8 0.53916% -0.11740% -1.18217% 7.12544% :-) nbody 0.22570% -0.07386% 9.04252% 3.06672% :-) json_dump_v2 0.20477% -0.52035% 3.96086% 14.79653% :-| normal_startup 1.82231% 0.56996% -1.23032% 2.50067% :-| ssbench 1.12235% 0.31370% 1.54159% 3.05895% ------------------------------------------------------------------------------------------ Note: Benchmark results for ssbench are measured in requests/second while all other are measured in seconds. * Relative Standard Deviation (Standard Deviation/Average) Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. From solipsis at pitrou.net Mon Sep 14 10:43:45 2015 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Mon, 14 Sep 2015 08:43:45 +0000 Subject: [Python-checkins] Daily reference leaks (c2c4400bb12c): sum=17877 Message-ID: <20150914084345.17969.41152@psf.io> results for c2c4400bb12c on branch "default" -------------------------------------------- test_capi leaked [1598, 1598, 1598] references, sum=4794 test_capi leaked [387, 389, 389] memory blocks, sum=1165 test_functools leaked [0, 2, 2] memory blocks, sum=4 test_threading leaked [3196, 3196, 3196] references, sum=9588 test_threading leaked [774, 776, 776] memory blocks, sum=2326 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/psf-users/antoine/refleaks/reflogRH3xYV', '--timeout', '7200'] From bcannon at gmail.com Mon Sep 14 18:49:15 2015 From: bcannon at gmail.com (Brett Cannon) Date: Mon, 14 Sep 2015 16:49:15 +0000 Subject: [Python-checkins] cpython: In-line the append operations inside deque_inplace_repeat(). In-Reply-To: <20150912150038.15706.90493@psf.io> References: <20150912150038.15706.90493@psf.io> Message-ID: Would it be worth adding a comment that the block of code is an inlined copy of deque_append()? Or maybe even turn the append() function into a macro so you minimize code duplication? On Sat, 12 Sep 2015 at 08:00 raymond.hettinger wrote: > https://hg.python.org/cpython/rev/cb96ffe6ff10 > changeset: 97943:cb96ffe6ff10 > parent: 97941:b8f3a01937be > user: Raymond Hettinger > date: Sat Sep 12 11:00:20 2015 -0400 > summary: > In-line the append operations inside deque_inplace_repeat(). > > files: > Modules/_collectionsmodule.c | 22 ++++++++++++++++++---- > 1 files changed, 18 insertions(+), 4 deletions(-) > > > diff --git a/Modules/_collectionsmodule.c b/Modules/_collectionsmodule.c > --- a/Modules/_collectionsmodule.c > +++ b/Modules/_collectionsmodule.c > @@ -567,12 +567,26 @@ > if (n > MAX_DEQUE_LEN) > return PyErr_NoMemory(); > > + deque->state++; > for (i = 0 ; i < n-1 ; i++) { > - rv = deque_append(deque, item); > - if (rv == NULL) > - return NULL; > - Py_DECREF(rv); > + if (deque->rightindex == BLOCKLEN - 1) { > + block *b = newblock(Py_SIZE(deque) + i); > + if (b == NULL) { > + Py_SIZE(deque) += i; > + return NULL; > + } > + b->leftlink = deque->rightblock; > + CHECK_END(deque->rightblock->rightlink); > + deque->rightblock->rightlink = b; > + deque->rightblock = b; > + MARK_END(b->rightlink); > + deque->rightindex = -1; > + } > + deque->rightindex++; > + Py_INCREF(item); > + deque->rightblock->data[deque->rightindex] = item; > } > + Py_SIZE(deque) += i; > Py_INCREF(deque); > return (PyObject *)deque; > } > > -- > Repository URL: https://hg.python.org/cpython > _______________________________________________ > Python-checkins mailing list > Python-checkins at python.org > https://mail.python.org/mailman/listinfo/python-checkins > -------------- next part -------------- An HTML attachment was scrubbed... URL: From python-checkins at python.org Tue Sep 15 00:12:45 2015 From: python-checkins at python.org (barry.warsaw) Date: Mon, 14 Sep 2015 22:12:45 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_PEP_13?= Message-ID: <20150914221245.11272.10281@psf.io> https://hg.python.org/peps/rev/07406baeafac changeset: 6058:07406baeafac user: Barry Warsaw date: Mon Sep 14 18:12:42 2015 -0400 summary: PEP 13 files: pep-0013.rst | 924 +++++++++++++++++++++++++++++++++++++++ 1 files changed, 924 insertions(+), 0 deletions(-) diff --git a/pep-0013.rst b/pep-0013.rst new file mode 100644 --- /dev/null +++ b/pep-0013.rst @@ -0,0 +1,924 @@ +PEP: 13 +Title: Collecting information about git +Version: $Revision$ +Last-Modified: $Date$ +Author: Oleg Broytman +Status: Draft +Type: Informational +Content-Type: text/x-rst +Created: 01-Jun-2015 +Post-History: 12-Sep-2015 + +Abstract +======== + +This Informational PEP collects information about git. There is, of +course, a lot of documentation for git, so the PEP concentrates on +more complex (and more related to Python development) issues, +scenarios and examples. + +The plan is to extend the PEP in the future collecting information +about equivalence of Mercurial and git scenarios to help migrating +Python development from Mercurial to git. + +The author of the PEP doesn't currently plan to write a Process PEP on +migration Python development from Mercurial to git. + + +Documentation +============= + +Git is accompanied with a lot of documentation, both online and +offline. + + +Documentation for starters +-------------------------- + +Git Tutorial: `part 1 +`_, +`part 2 +`_. + +`Git User's manual +`_. +`Everyday GIT With 20 Commands Or So +`_. +`Git workflows +`_. + + +Advanced documentation +---------------------- + +`Git Magic +`_, +with a number of translations. + +`Pro Git `_. The Book about git. Buy it at +Amazon or download in PDF, mobi, or ePub form. It has translations to +many different languages. Download Russian translation from `GArik +`_. + +`Git Wiki `_. + + +Offline documentation +--------------------- + +Git has builtin help: run ``git help $TOPIC``. For example, run +``git help git`` or ``git help help``. + + +Quick start +=========== + +Download and installation +------------------------- + +Unix users: `download and install using your package manager +`_. + +Microsoft Windows: download `git-for-windows +`_ or `msysGit +`_. + +MacOS X: use git installed with `XCode +`_ or download from +`MacPorts `_ or +`git-osx-installer +`_ or +install git with `Homebrew `_: ``brew install git``. + +`git-cola `_ is a Git GUI +written in Python and GPL licensed. Linux, Windows, MacOS X. + +`TortoiseGit `_ is a Windows Shell Interface +to Git based on TortoiseSVN; open source. + + +Initial configuration +--------------------- + +This simple code is often appears in documentation, but it is +important so let repeat it here. Git stores author and committer +names/emails in every commit, so configure your real name and +preferred email:: + + $ git config --global user.name "User Name" + $ git config --global user.email user.name at example.org + + +Examples in this PEP +==================== + +Examples of git commands in this PEP use the following approach. It is +supposed that you, the user, works with a local repository named +``python`` that has an upstream remote repo named ``origin``. Your +local repo has two branches ``v1`` and ``master``. For most examples +the currently checked out branch is ``master``. That is, it's assumed +you have done something like that:: + + $ git clone https://git.python.org/python.git + $ cd python + $ git branch v1 origin/v1 + +The first command clones remote repository into local directory +`python``, creates a new local branch master, sets +remotes/origin/master as its upstream remote-tracking branch and +checks it out into the working directory. + +The last command creates a new local branch v1 and sets +remotes/origin/v1 as its upstream remote-tracking branch. + +The same result can be achieved with commands:: + + $ git clone -b v1 https://git.python.org/python.git + $ cd python + $ git checkout --track origin/master + +The last command creates a new local branch master, sets +remotes/origin/master as its upstream remote-tracking branch and +checks it out into the working directory. + + +Branches and branches +===================== + +Git terminology can be a bit misleading. Take, for example, the term +"branch". In git it has two meanings. A branch is a directed line of +commits (possibly with merges). And a branch is a label or a pointer +assigned to a line of commits. It is important to distinguish when you +talk about commits and when about their labels. Lines of commits are +by itself unnamed and are usually only lengthening and merging. +Labels, on the other hand, can be created, moved, renamed and deleted +freely. + + +Remote repositories and remote branches +======================================= + +Remote-tracking branches are branches (pointers to commits) in your +local repository. They are there for git (and for you) to remember +what branches and commits have been pulled from and pushed to what +remote repos (you can pull from and push to many remotes). +Remote-tracking branches live under ``remotes/$REMOTE`` namespaces, +e.g. ``remotes/origin/master``. + +To see the status of remote-tracking branches run:: + + $ git branch -rv + +To see local and remote-tracking branches (and tags) pointing to +commits:: + + $ git log --decorate + +You never do your own development on remote-tracking branches. You +create a local branch that has a remote branch as upstream and do +development on that local branch. On push git pushes commits to the +remote repo and updates remote-tracking branches, on pull git fetches +commits from the remote repo, updates remote-tracking branches and +fast-forwards, merges or rebases local branches. + +When you do an initial clone like this:: + + $ git clone -b v1 https://git.python.org/python.git + +git clones remote repository ``https://git.python.org/python.git`` to +directory ``python``, creates a remote named ``origin``, creates +remote-tracking branches, creates a local branch ``v1``, configure it +to track upstream remotes/origin/v1 branch and checks out ``v1`` into +the working directory. + + +Updating local and remote-tracking branches +------------------------------------------- + +There is a major difference between + +:: + + $ git fetch $REMOTE $BRANCH + +and + +:: + + $ git fetch $REMOTE $BRANCH:$BRANCH + +The first command fetches commits from the named $BRANCH in the +$REMOTE repository that are not in your repository, updates +remote-tracking branch and leaves the id (the hash) of the head commit +in file .git/FETCH_HEAD. + +The second command fetches commits from the named $BRANCH in the +$REMOTE repository that are not in your repository and updates both +the local branch $BRANCH and its upstream remote-tracking branch. But +it refuses to update branches in case of non-fast-forward. And it +refuses to update the current branch (currently checked out branch, +where HEAD is pointing to). + +The first command is used internally by ``git pull``. + +:: + + $ git pull $REMOTE $BRANCH + +is equivalent to + +:: + + $ git fetch $REMOTE $BRANCH + $ git merge FETCH_HEAD + +Certainly, $BRANCH in that case should be your current branch. If you +want to merge a different branch into your current branch first update +that non-current branch and then merge:: + + $ git fetch origin v1:v1 # Update v1 + $ git pull --rebase origin master # Update the current branch master + # using rebase instead of merge + $ git merge v1 + +If you have not yet pushed commits on ``v1``, though, the scenario has +to become a bit more complex. Git refuses to update +non-fast-forwardable branch, and you don't want to do force-pull +because that would remove your non-pushed commits and you would need +to recover. So you want to rebase ``v1`` but you cannot rebase +non-current branch. Hence, checkout ``v1`` and rebase it before +merging:: + + $ git checkout v1 + $ git pull --rebase origin v1 + $ git checkout master + $ git pull --rebase origin master + $ git merge v1 + +It is possible to configure git to make it fetch/pull a few branches +or all branches at once, so you can simply run + +:: + + $ git pull origin + +or even + +:: + + $ git pull + +Default remote repository for fetching/pulling is ``origin``. Default +set of references to fetch is calculated using matching algorithm: git +fetches all branches having the same name on both ends. + + +Push +'''' + +Pushing is a bit simpler. There is only one command ``push``. When you +run + +:: + + $ git push origin v1 master + +git pushes local v1 to remote v1 and local master to remote master. +The same as:: + + $ git push origin v1:v1 master:master + +Git pushes commits to the remote repo and updates remote-tracking +branches. Git refuses to push commits that aren't fast-forwardable. +You can force-push anyway, but please remember - you can force-push to +your own repositories but don't force-push to public or shared repos. +If you find git refuses to push commits that aren't fast-forwardable, +better fetch and merge commits from the remote repo (or rebase your +commits on top of the fetched commits), then push. Only force-push if +you know what you do and why you do it. See the section `Commit +editing and caveats`_ below. + +It is possible to configure git to make it push a few branches or all +branches at once, so you can simply run + +:: + + $ git push origin + +or even + +:: + + $ git push + +Default remote repository for pushing is ``origin``. Default set of +references to push in git before 2.0 is calculated using matching +algorithm: git pushes all branches having the same name on both ends. +Default set of references to push in git 2.0+ is calculated using +simple algorithm: git pushes the current branch back to its +@{upstream}. + +To configure git before 2.0 to the new behaviour run:: + +$ git config push.default simple + +To configure git 2.0+ to the old behaviour run:: + +$ git config push.default matching + +Git doesn't allow to push a branch if it's the current branch in the +remote non-bare repository: git refuses to update remote working +directory. You really should push only to bare repositories. For +non-bare repositories git prefers pull-based workflow. + +When you want to deploy code on a remote host and can only use push +(because your workstation is behind a firewall and you cannot pull +from it) you do that in two steps using two repositories: you push +from the workstation to a bare repo on the remote host, ssh to the +remote host and pull from the bare repo to a non-bare deployment repo. + +That changed in git 2.3, but see `the blog post +`_ +for caveats; in 2.4 the push-to-deploy feature was `further improved +`_. + + +Tags +'''' + +Git automatically fetches tags that point to commits being fetched +during fetch/pull. To fetch all tags (and commits they point to) run +``git fetch --tags origin``. To fetch some specific tags fetch them +explicitly:: + + $ git fetch origin tag $TAG1 tag $TAG2... + +For example:: + + $ git fetch origin tag 1.4.2 + $ git fetch origin v1:v1 tag 2.1.7 + +Git doesn't automatically pushes tags. That allows you to have private +tags. To push tags list them explicitly:: + + $ git push origin tag 1.4.2 + $ git push origin v1 master tag 2.1.7 + +Or push all tags at once:: + + $ git push --tags origin + +Don't move tags with ``git tag -f`` or remove tags with ``git tag -d`` +after they have been published. + + +Private information +''''''''''''''''''' + +When cloning/fetching/pulling/pushing git copies only database objects +(commits, trees, files and tags) and symbolic references (branches and +lightweight tags). Everything else is private to the repository and +never cloned, updated or pushed. It's your config, your hooks, your +private exclude file. + +If you want to distribute hooks, copy them to the working tree, add, +commit, push and instruct the team to update and install the hooks +manually. + + +Commit editing and caveats +========================== + +A warning not to edit published (pushed) commits also appears in +documentation but it's repeated here anyway as it's very important. + +It is possible to recover from a forced push but it's PITA for the +entire team. Please avoid it. + +To see what commits have not been published yet compare the head of the +branch with its upstream remote-tracking branch:: + + $ git log origin/master.. # from origin/master to HEAD (of master) + $ git log origin/v1..v1 # from origin/v1 to the head of v1 + +For every branch that has an upstream remote-tracking branch git +maintains an alias @{upstream} (short version @{u}), so the commands +above can be given as:: + + $ git log @{u}.. + $ git log v1@{u}..v1 + +To see the status of all branches:: + + $ git branch -avv + +To compare the status of local branches with a remote repo:: + + $ git remote show origin + +Read `how to recover from upstream rebase +`_. +It is in ``git help rebase``. + +On the other hand don't be too afraid about commit editing. You can +safely edit, reorder, remove, combine and split commits that haven't +been pushed yet. You can even push commits to your own (backup) repo, +edit them later and force-push edited commits to replace what have +already been pushed. Not a problem until commits are in a public +or shared repository. + + +Undo +==== + +Whatever you do, don't panic. Almost anything in git can be undone. + + +git checkout: restore file's content +------------------------------------ + +``git checkout``, for example, can be used to restore the content of +file(s) to that one of a commit. Like this:: + + git checkout HEAD~ README + +The commands restores the contents of README file to the last but one +commit in the current branch. By default the commit ID is simply HEAD; +i.e. ``git checkout README`` restores README to the latest commit. + +(Do not use ``git checkout`` to view a content of a file in a commit, +use ``git cat-file -p``; e.g. ``git cat-file -p HEAD~:path/to/README``). + + +git reset: remove (non-pushed) commits +-------------------------------------- + +``git reset`` moves the head of the current branch. The head can be +moved to point to any commit but it's often used to remove a commit or +a few (preferably, non-pushed ones) from the top of the branch - that +is, to move the branch backward in order to undo a few (non-pushed) +commits. + +``git reset`` has three modes of operation - soft, hard and mixed. +Default is mixed. ProGit `explains +`_ the +difference very clearly. Bare repositories don't have indices or +working trees so in a bare repo only soft reset is possible. + + +Unstaging +''''''''' + +Mixed mode reset with a path or paths can be used to unstage changes - +that is, to remove from index changes added with ``git add`` for +committing. See `The Book +`_ for details +about unstaging and other undo tricks. + + +git reflog: reference log +------------------------- + +Removing commits with ``git reset`` or moving the head of a branch +sounds dangerous and it is. But there is a way to undo: another +reset back to the original commit. Git doesn't remove commits +immediately; unreferenced commits (in git terminology they are called +"dangling commits") stay in the database for some time (default is two +weeks) so you can reset back to it or create a new branch pointing to +the original commit. + +For every move of a branch's head - with ``git commit``, ``git +checkout``, ``git fetch``, ``git pull``, ``git rebase``, ``git reset`` +and so on - git stores a reference log (reflog for short). For every +move git stores where the head was. Command ``git reflog`` can be used +to view (and manipulate) the log. + +In addition to the moves of the head of every branch git stores the +moves of the HEAD - a symbolic reference that (usually) names the +current branch. HEAD is changed with ``git checkout $BRANCH``. + +By default ``git reflog`` shows the moves of the HEAD, i.e. the +command is equivalent to ``git reflog HEAD``. To show the moves of the +head of a branch use the command ``git reflog $BRANCH``. + +So to undo a ``git reset`` lookup the original commit in ``git +reflog``, verify it with ``git show`` or ``git log`` and run ``git +reset $COMMIT_ID``. Git stores the move of the branch's head in +reflog, so you can undo that undo later again. + +In a more complex situation you'd want to move some commits along with +resetting the head of the branch. Cherry-pick them to the new branch. +For example, if you want to reset the branch ``master`` back to the +original commit but preserve two commits created in the current branch +do something like:: + + $ git branch save-master # create a new branch saving master + $ git reflog # find the original place of master + $ git reset $COMMIT_ID + $ git cherry-pick save-master~ save-master + $ git branch -D save-master # remove temporary branch + + +git revert: revert a commit +--------------------------- + +``git revert`` reverts a commit or commits, that is, it creates a new +commit or commits that revert(s) the effects of the given commits. +It's the only way to undo published commits (``git commit --amend``, +``git rebase`` and ``git reset`` change the branch in +non-fast-forwardable ways so they should only be used for non-pushed +commits.) + +There is a problem with reverting a merge commit. ``git revert`` can +undo the code created by the merge commit but it cannot undo the fact +of merge. See the discussion `How to revert a faulty merge +`_. + + +One thing that cannot be undone +------------------------------- + +Whatever you undo, there is one thing that cannot be undone - +overwritten uncommitted changes. Uncommitted changes don't belong to +git so git cannot help preserving them. + +Most of the time git warns you when you're going to execute a command +that overwrites uncommitted changes. Git doesn't allow you to switch +branches with ``git checkout``. It stops you when you're going to +rebase with non-clean working tree. It refuses to pull new commits +over non-committed files. + +But there are commands that do exactly that - overwrite files in the +working tree. Commands like ``git checkout $PATHs`` or ``git reset +--hard`` silently overwrite files including your uncommitted changes. + +With that in mind you can understand the stance "commit early, commit +often". Commit as often as possible. Commit on every save in your +editor or IDE. You can edit your commits before pushing - edit commit +messages, change commits, reorder, combine, split, remove. But save +your changes in git database, either commit changes or at least stash +them with ``git stash``. + + +Merge or rebase? +================ + +Internet is full of heated discussions on the topic: "merge or +rebase?" Most of them are meaningless. When a DVCS is being used in a +big team with a big and complex project with many branches there is +simply no way to avoid merges. So the question's diminished to +"whether to use rebase, and if yes - when to use rebase?" Considering +that it is very much recommended not to rebase published commits the +question's diminished even further: "whether to use rebase on +non-pushed commits?" + +That small question is for the team to decide. The author of the PEP +recommends to use rebase when pulling, i.e. always do ``git pull +--rebase`` or even configure automatic setup of rebase for every new +branch:: + + $ git config branch.autosetuprebase always + +and configure rebase for existing branches:: + + $ git config branch.$NAME.rebase true + +For example:: + + $ git config branch.v1.rebase true + $ git config branch.master.rebase true + +After that ``git pull origin master`` becomes equivalent to ``git pull +--rebase origin master``. + +It is recommended to create new commits in a separate feature or topic +branch while using rebase to update the mainline branch. When the +topic branch is ready merge it into mainline. To avoid a tedious task +of resolving large number of conflicts at once you can merge the topic +branch to the mainline from time to time and switch back to the topic +branch to continue working on it. The entire workflow would be +something like:: + + $ git checkout -b issue-42 # create a new issue branch and switch to it + ...edit/test/commit... + $ git checkout master + $ git pull --rebase origin master # update master from the upstream + $ git merge issue-42 + $ git branch -d issue-42 # delete the topic branch + $ git push origin master + +When the topic branch is deleted only the label is removed, commits +are stayed in the database, they are now merged into master:: + + o--o--o--o--o--M--< master - the mainline branch + \ / + --*--*--* - the topic branch, now unnamed + +The topic branch is deleted to avoid cluttering branch namespace with +small topic branches. Information on what issue was fixed or what +feature was implemented should be in the commit messages. + + +Null-merges +=========== + +Git has a builtin merge strategy for what Python core developers call +"null-merge":: + + $ git merge -s ours v1 # null-merge v1 into master + + +Advanced configuration +====================== + +Line endings +------------ + +Git has builtin mechanisms to handle line endings between platforms +with different end-of-line styles. To allow git to do CRLF conversion +assign ``text`` attribute to files using `.gitattributes +`_. +For files that have to have specific line endings assign ``eol`` +attribute. For binary files the attribute is, naturally, ``binary``. + +For example:: + + $ cat .gitattributes + *.py text + *.txt text + *.png binary + /readme.txt eol=CRLF + +To check what attributes git uses for files use ``git check-attr`` +command. For example:: + +$ git check-attr -a -- \*.py + + +Advanced topics +=============== + +Staging area +------------ + +Staging area aka index aka cache is a distinguishing feature of git. +Staging area is where git collects patches before committing them. +Separation between collecting patches and commit phases provides a +very useful feature of git: you can review collected patches before +commit and even edit them - remove some hunks, add new hunks and +review again. + +To add files to the index use ``git add``. Collecting patches before +committing means you need to do that for every change, not only to add +new (untracked) files. To simplify committing in case you just want to +commit everything without reviewing run ``git commit --all`` (or just +``-a``) - the command adds every changed tracked file to the index and +then commit. To commit a file or files regardless of patches collected +in the index run ``git commit [--only|-o] -- $FILE...``. + +To add hunks of patches to the index use ``git add --patch`` (or just +``-p``). To remove collected files from the index use ``git reset HEAD +-- $FILE...`` To add/inspect/remove collected hunks use ``git add +--interactive`` (``-i``). + +To see the diff between the index and the last commit (i.e., collected +patches) use ``git diff --cached``. To see the diff between the +working tree and the index (i.e., uncollected patches) use just ``git +diff``. To see the diff between the working tree and the last commit +(i.e., both collected and uncollected patches) run ``git diff HEAD``. + +See `WhatIsTheIndex +`_ and +`IndexCommandQuickref +`_ in Git +Wiki. + + +ReReRe +====== + +Rerere is a mechanism that helps to resolve repeated merge conflicts. +The most frequent source of recurring merge conflicts are topic +branches that are merged into mainline and then the merge commits are +removed; that's often performed to test the topic branches and train +rerere; merge commits are removed to have clean linear history and +finish the topic branch with only one last merge commit. + +Rerere works by remembering the states of tree before and after a +successful commit. That way rerere can automatically resolve conflicts +if they appear in the same files. + +Rerere can be used manually with ``git rerere`` command but most often +it's used automatically. Enable rerere with these commands in a +working tree:: + + $ git config rerere.enabled true + $ git config rerere.autoupdate true + +You don't need to turn rerere on globally - you don't want rerere in +bare repositories or single-branche repositories; you only need rerere +in repos where you often perform merges and resolve merge conflicts. + +See `Rerere `_ in The +Book. + + +Database maintenance +==================== + +Git object database and other files/directories under ``.git`` require +periodic maintenance and cleanup. For example, commit editing left +unreferenced objects (dangling objects, in git terminology) and these +objects should be pruned to avoid collecting cruft in the DB. The +command ``git gc`` is used for maintenance. Git automatically runs +``git gc --auto`` as a part of some commands to do quick maintenance. +Users are recommended to run ``git gc --aggressive`` from time to +time; ``git help gc`` recommends to run it every few hundred +changesets; for more intensive projects it should be something like +once a week and less frequently (biweekly or monthly) for lesser +active projects. + +``git gc --aggressive`` not only removes dangling objects, it also +repacks object database into indexed and better optimized pack(s); it +also packs symbolic references (branches and tags). Another way to do +it is to run ``git repack``. + +There is a well-known `message +`_ from Linus +Torvalds regarding "stupidity" of ``git gc --aggressive``. The message +can safely be ignored now. It is old and outdated, ``git gc +--aggressive`` became much better since that time. + +For those who still prefer ``git repack`` over ``git gc --aggressive`` +the recommended parameters are ``git repack -a -d -f --depth=20 +--window=250``. See `this detailed experiment +`_ +for explanation of the effects of these parameters. + +From time to time run ``git fsck [--strict]`` to verify integrity of +the database. ``git fsck`` may produce a list of dangling objects; +that's not an error, just a reminder to perform regular maintenance. + + +Tips and tricks +=============== + +Command-line options and arguments +---------------------------------- + +`git help cli +`_ +recommends not to combine short options/flags. Most of the times +combining works: ``git commit -av`` works perfectly, but there are +situations when it doesn't. E.g., ``git log -p -5`` cannot be combined +as ``git log -p5``. + +Some options have arguments, some even have default arguments. In that +case the argument for such option must be spelled in a sticky way: +``-Oarg``, never ``-O arg`` because for an option that has a default +argument the latter means "use default value for option ``-O`` and +pass ``arg`` further to the option parser". For example, ``git grep`` +has an option ``-O`` that passes a list of names of the found files to +a program; default program for ``-O`` is a pager (usually ``less``), +but you can use your editor:: + + $ git grep -Ovim # but not -O vim + +BTW, if git is instructed to use ``less`` as the pager (i.e., if pager +is not configured in git at all it uses ``less`` by default, or if it +gets ``less`` from GIT_PAGER or PAGER environment variables, or if it +was configured with ``git config --global core.pager less``, or +``less`` is used in the command ``git grep -Oless``) ``git grep`` +passes ``+/$pattern`` option to ``less`` which is quite convenient. +Unfortunately, ``git grep`` doesn't pass the pattern if the pager is +not exactly ``less``, even if it's ``less`` with parameters (something +like ``git config --global core.pager less -FRSXgimq``); fortunately, +``git grep -Oless`` always passes the pattern. + + +bash/zsh completion +------------------- + +It's a bit hard to type ``git rebase --interactive --preserve-merges +HEAD~5`` manually even for those who are happy to use command-line, +and this is where shell completion is of great help. Bash/zsh come +with programmable completion, often automatically installed and +enabled, so if you have bash/zsh and git installed, chances are you +are already done - just go and use it at the command-line. + +If you don't have necessary bits installed, install and enable +bash_completion package. If you want to upgrade your git completion to +the latest and greatest download necessary file from `git contrib +`_. + +Git-for-windows comes with git-bash for which bash completion is +installed and enabled. + + +bash/zsh prompt +--------------- + +For command-line lovers shell prompt can carry a lot of useful +information. To include git information in the prompt use +`git-prompt.sh +`_. +Read the detailed instructions in the file. + +Search the Net for "git prompt" to find other prompt variants. + + +git on server +============= + +The simplest way to publish a repository or a group of repositories is +``git daemon``. The daemon provides anonymous access, by default it is +read-only. The repositories are accessible by git protocol (git:// +URLs). Write access can be enabled but the protocol lacks any +authentication means, so it should be enabled only within a trusted +LAN. See ``git help daemon`` for details. + +Git over ssh provides authentication and repo-level authorisation as +repositories can be made user- or group-writeable (see parameter +``core.sharedRepository`` in ``git help config``). If that's too +permissive or too restrictive for some project's needs there is a +wrapper `gitolite `_ that can +be configured to allow access with great granularity; gitolite is +written in Perl and has a lot of documentation. + +Web interface to browse repositories can be created using `gitweb +`_ or `cgit +`_. Both are CGI scripts (written in +Perl and C). In addition to web interface both provide read-only dumb +http access for git (http(s):// URLs). + +There are also more advanced web-based development environments that +include ability to manage users, groups and projects; private, +group-accessible and public repositories; they often include issue +trackers, wiki pages, pull requests and other tools for development +and communication. Among these environments are `Kallithea +`_ and `pagure `_, +both are written in Python; pagure was written by Fedora developers +and is being used to develop some Fedora projects. `Gogs +`_ is written in Go; there is a fork `Gitea +`_. + +And last but not least, `Gitlab `_. It's +perhaps the most advanced web-based development environment for git. +Written in Ruby, community edition is free and open source (MIT +license). + + +From Mercurial to git +===================== + +There are many tools to convert Mercurial repositories to git. The +most famous are, probably, `hg-git `_ and +`fast-export `_ (many years ago +it was known under the name ``hg2git``). + +But a better tool, perhaps the best, is `git-remote-hg +`_. It provides transparent +bidirectional (pull and push) access to Mercurial repositories from +git. Its author wrote a `comparison of alternatives +`_ +that seems to be mostly objective. + +To use git-remote-hg, install or clone it, add to your PATH (or copy +script ``git-remote-hg`` to a directory that's already in PATH) and +prepend ``hg::`` to Mercurial URLs. For example:: + + $ git clone https://github.com/felipec/git-remote-hg.git + $ PATH=$PATH:"`pwd`"/git-remote-hg + $ git clone hg::https://hg.python.org/peps/ PEPs + +To work with the repository just use regular git commands including +``git fetch/pull/push``. + +To start converting your Mercurial habits to git see the page +`Mercurial for Git users +`_ at Mercurial wiki. +At the second half of the page there is a table that lists +corresponding Mercurial and git commands. Should work perfectly in +both directions. + +Python Developer's Guide also has a chapter `Mercurial for git +developers `_ that +documents a few differences between git and hg. + + +Copyright +========= + +This document has been placed in the public domain. + + + +.. + Local Variables: + mode: indented-text + indent-tabs-mode: nil + sentence-end-double-space: t + fill-column: 70 + coding: utf-8 + End: + vim: set fenc=us-ascii tw=70 : -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Tue Sep 15 00:25:55 2015 From: python-checkins at python.org (victor.stinner) Date: Mon, 14 Sep 2015 22:25:55 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogTWVyZ2UgMy41ICh0ZXN0X2dkYik=?= Message-ID: <20150914222555.14871.18967@psf.io> https://hg.python.org/cpython/rev/4a431c65e57e changeset: 98003:4a431c65e57e parent: 97999:c2c4400bb12c parent: 98002:849bdcbff518 user: Victor Stinner date: Tue Sep 15 00:23:20 2015 +0200 summary: Merge 3.5 (test_gdb) files: Lib/test/test_gdb.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Lib/test/test_gdb.py b/Lib/test/test_gdb.py --- a/Lib/test/test_gdb.py +++ b/Lib/test/test_gdb.py @@ -38,7 +38,7 @@ # 'GNU gdb (GDB) Fedora 7.9.1-17.fc22\n' -> 7.9 # 'GNU gdb 6.1.1 [FreeBSD]\n' -> 6.1 # 'GNU gdb (GDB) Fedora (7.5.1-37.fc18)\n' -> 7.5 - match = re.search(r"^GNU gdb.*?\b(\d+)\.(\d)", version) + match = re.search(r"^GNU gdb.*?\b(\d+)\.(\d+)", version) if match is None: raise Exception("unable to parse GDB version: %r" % version) return (version, int(match.group(1)), int(match.group(2))) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 15 00:25:55 2015 From: python-checkins at python.org (victor.stinner) Date: Mon, 14 Sep 2015 22:25:55 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogdGVzdF9nZGI6IGZp?= =?utf-8?q?x_regex_to_parse_the_GDB_version?= Message-ID: <20150914222555.11254.24131@psf.io> https://hg.python.org/cpython/rev/21d6b2752fe8 changeset: 98000:21d6b2752fe8 branch: 2.7 parent: 97957:63bbe9f80909 user: Victor Stinner date: Tue Sep 15 00:19:47 2015 +0200 summary: test_gdb: fix regex to parse the GDB version Fix the regex to support the version 7.10: minor version with two digits files: Lib/test/test_gdb.py | 5 +++-- 1 files changed, 3 insertions(+), 2 deletions(-) diff --git a/Lib/test/test_gdb.py b/Lib/test/test_gdb.py --- a/Lib/test/test_gdb.py +++ b/Lib/test/test_gdb.py @@ -33,8 +33,9 @@ # Regex to parse: # 'GNU gdb (GDB; SUSE Linux Enterprise 12) 7.7\n' -> 7.7 # 'GNU gdb (GDB) Fedora 7.9.1-17.fc22\n' -> 7.9 - # 'GNU gdb 6.1.1 [FreeBSD]\n' - match = re.search("^GNU gdb.*? (\d+)\.(\d)", version) + # 'GNU gdb 6.1.1 [FreeBSD]\n' -> 6.1 + # 'GNU gdb (GDB) Fedora (7.5.1-37.fc18)\n' -> 7.5 + match = re.search(r"^GNU gdb.*?\b(\d+)\.(\d+)", version) if match is None: raise Exception("unable to parse GDB version: %r" % version) return (version, int(match.group(1)), int(match.group(2))) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 15 00:25:55 2015 From: python-checkins at python.org (victor.stinner) Date: Mon, 14 Sep 2015 22:25:55 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogdGVzdF9nZGI6IGZp?= =?utf-8?q?x_regex_to_parse_the_GDB_version?= Message-ID: <20150914222555.101496.40391@psf.io> https://hg.python.org/cpython/rev/6a8aa246485e changeset: 98001:6a8aa246485e branch: 3.4 parent: 97991:5e7f9c8098d3 user: Victor Stinner date: Tue Sep 15 00:22:55 2015 +0200 summary: test_gdb: fix regex to parse the GDB version Fix the regex to support the version 7.10: minor version with two digits files: Lib/test/test_gdb.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Lib/test/test_gdb.py b/Lib/test/test_gdb.py --- a/Lib/test/test_gdb.py +++ b/Lib/test/test_gdb.py @@ -38,7 +38,7 @@ # 'GNU gdb (GDB) Fedora 7.9.1-17.fc22\n' -> 7.9 # 'GNU gdb 6.1.1 [FreeBSD]\n' -> 6.1 # 'GNU gdb (GDB) Fedora (7.5.1-37.fc18)\n' -> 7.5 - match = re.search(r"^GNU gdb.*?\b(\d+)\.(\d)", version) + match = re.search(r"^GNU gdb.*?\b(\d+)\.(\d+)", version) if match is None: raise Exception("unable to parse GDB version: %r" % version) return (version, int(match.group(1)), int(match.group(2))) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 15 00:25:55 2015 From: python-checkins at python.org (victor.stinner) Date: Mon, 14 Sep 2015 22:25:55 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?b?IE1lcmdlIDMuNCAodGVzdF9nZGIp?= Message-ID: <20150914222555.11993.18122@psf.io> https://hg.python.org/cpython/rev/849bdcbff518 changeset: 98002:849bdcbff518 branch: 3.5 parent: 97996:399b7d5616f1 parent: 98001:6a8aa246485e user: Victor Stinner date: Tue Sep 15 00:23:08 2015 +0200 summary: Merge 3.4 (test_gdb) files: Lib/test/test_gdb.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Lib/test/test_gdb.py b/Lib/test/test_gdb.py --- a/Lib/test/test_gdb.py +++ b/Lib/test/test_gdb.py @@ -38,7 +38,7 @@ # 'GNU gdb (GDB) Fedora 7.9.1-17.fc22\n' -> 7.9 # 'GNU gdb 6.1.1 [FreeBSD]\n' -> 6.1 # 'GNU gdb (GDB) Fedora (7.5.1-37.fc18)\n' -> 7.5 - match = re.search(r"^GNU gdb.*?\b(\d+)\.(\d)", version) + match = re.search(r"^GNU gdb.*?\b(\d+)\.(\d+)", version) if match is None: raise Exception("unable to parse GDB version: %r" % version) return (version, int(match.group(1)), int(match.group(2))) -- Repository URL: https://hg.python.org/cpython From raymond.hettinger at gmail.com Tue Sep 15 00:37:15 2015 From: raymond.hettinger at gmail.com (Raymond Hettinger) Date: Mon, 14 Sep 2015 18:37:15 -0400 Subject: [Python-checkins] [Python-Dev] cpython: In-line the append operations inside deque_inplace_repeat(). In-Reply-To: References: <20150912150038.15706.90493@psf.io> Message-ID: <7097FAA2-5F18-4FAE-9876-606414AEFF8F@gmail.com> > On Sep 14, 2015, at 12:49 PM, Brett Cannon wrote: > > Would it be worth adding a comment that the block of code is an inlined copy of deque_append()? > Or maybe even turn the append() function into a macro so you minimize code duplication? I don't think either would be helpful. The point of the inlining was to let the code evolve independently from deque_append(). Once separated from the mother ship, the code in deque_inline_repeat() could now shed the unnecessary work. The state variable is updated once. The updates within a single block are now in the own inner loop. The deque size is updated outside of that loop, etc. In other words, they are no longer the same code. The original append-in-a-loop version was already being in-lined by the compiler but was doing way too much work. For each item written in the original, there were 7 memory reads, 5 writes, 6 predictable compare-and-branches, and 5 add/sub operations. In the current form, there are 0 reads, 1 writes, 2 predictable compare-and-branches, and 3 add/sub operations. FWIW, my work flow is that periodically I expand the code with new features (the upcoming work is to add slicing support http://bugs.python.org/issue17394), then once it is correct and tested, I make a series optimization passes (such as the work I just described above). After that, I come along and factor-out common code, usually with clean, in-lineable functions rather than macros (such as the recent check-in replacing redundant code in deque_repeat with a call to the common code in deque_inplace_repeat). My schedule lately hasn't given me any big blocks of time to work with, so I do the steps piecemeal as I get snippets of development time. Raymond P.S. For those who are interested, here is the before and after: ---- before --------------------------------- L1152: movq __Py_NoneStruct at GOTPCREL(%rip), %rdi cmpq $0, (%rdi) < je L1257 L1159: addq $1, %r13 cmpq %r14, %r13 je L1141 movq 16(%rbx), %rsi < L1142: movq 48(%rbx), %rdx < addq $1, 56(%rbx) <> cmpq $63, %rdx je L1143 movq 32(%rbx), %rax < addq $1, %rdx L1144: addq $1, 0(%rbp) <> leaq 1(%rsi), %rcx movq %rdx, 48(%rbx) > movq %rcx, 16(%rbx) > movq %rbp, 8(%rax,%rdx,8) > movq 64(%rbx), %rax < cmpq %rax, %rcx jle L1152 cmpq $-1, %rax je L1152 ---- after ------------------------------------ L777: cmpq $63, %rdx je L816 L779: addq $1, %rdx movq %rbp, 16(%rsi,%rbx,8) < addq $1, %rbx leaq (%rdx,%r9), %rcx subq %r8, %rcx cmpq %r12, %rbx jl L777 # outside the inner-loop movq %rdx, 48(%r13) movq %rcx, 0(%rbp) cmpq %r12, %rbx jl L780 From bcannon at gmail.com Tue Sep 15 01:36:09 2015 From: bcannon at gmail.com (Brett Cannon) Date: Mon, 14 Sep 2015 23:36:09 +0000 Subject: [Python-checkins] [Python-Dev] cpython: In-line the append operations inside deque_inplace_repeat(). In-Reply-To: <7097FAA2-5F18-4FAE-9876-606414AEFF8F@gmail.com> References: <20150912150038.15706.90493@psf.io> <7097FAA2-5F18-4FAE-9876-606414AEFF8F@gmail.com> Message-ID: On Mon, 14 Sep 2015 at 15:37 Raymond Hettinger wrote: > > > On Sep 14, 2015, at 12:49 PM, Brett Cannon wrote: > > > > Would it be worth adding a comment that the block of code is an inlined > copy of deque_append()? > > Or maybe even turn the append() function into a macro so you minimize > code duplication? > > I don't think either would be helpful. The point of the inlining was to > let the code evolve independently from deque_append(). > OK, commit message just didn't point that out as the reason for the inlining (I guess in the future call it a fork of the code to know it is meant to evolve independently?). -Brett > > Once separated from the mother ship, the code in deque_inline_repeat() > could now shed the unnecessary work. The state variable is updated once. > The updates within a single block are now in the own inner loop. The deque > size is updated outside of that loop, etc. In other words, they are no > longer the same code. > > The original append-in-a-loop version was already being in-lined by the > compiler but was doing way too much work. For each item written in the > original, there were 7 memory reads, 5 writes, 6 predictable > compare-and-branches, and 5 add/sub operations. In the current form, there > are 0 reads, 1 writes, 2 predictable compare-and-branches, and 3 add/sub > operations. > > FWIW, my work flow is that periodically I expand the code with new > features (the upcoming work is to add slicing support > http://bugs.python.org/issue17394), then once it is correct and tested, I > make a series optimization passes (such as the work I just described > above). After that, I come along and factor-out common code, usually with > clean, in-lineable functions rather than macros (such as the recent > check-in replacing redundant code in deque_repeat with a call to the common > code in deque_inplace_repeat). > > My schedule lately hasn't given me any big blocks of time to work with, so > I do the steps piecemeal as I get snippets of development time. > > > Raymond > > > P.S. For those who are interested, here is the before and after: > > ---- before --------------------------------- > L1152: > movq __Py_NoneStruct at GOTPCREL(%rip), %rdi > cmpq $0, (%rdi) < > je L1257 > L1159: > addq $1, %r13 > cmpq %r14, %r13 > je L1141 > movq 16(%rbx), %rsi < > L1142: > movq 48(%rbx), %rdx < > addq $1, 56(%rbx) <> > cmpq $63, %rdx > je L1143 > movq 32(%rbx), %rax < > addq $1, %rdx > L1144: > addq $1, 0(%rbp) <> > leaq 1(%rsi), %rcx > movq %rdx, 48(%rbx) > > movq %rcx, 16(%rbx) > > movq %rbp, 8(%rax,%rdx,8) > > movq 64(%rbx), %rax < > cmpq %rax, %rcx > jle L1152 > cmpq $-1, %rax > je L1152 > > > ---- after ------------------------------------ > L777: > cmpq $63, %rdx > je L816 > L779: > addq $1, %rdx > movq %rbp, 16(%rsi,%rbx,8) < > addq $1, %rbx > leaq (%rdx,%r9), %rcx > subq %r8, %rcx > cmpq %r12, %rbx > jl L777 > > # outside the inner-loop > movq %rdx, 48(%r13) > movq %rcx, 0(%rbp) > cmpq %r12, %rbx > jl L780 -------------- next part -------------- An HTML attachment was scrubbed... URL: From python-checkins at python.org Tue Sep 15 03:21:43 2015 From: python-checkins at python.org (barry.warsaw) Date: Tue, 15 Sep 2015 01:21:43 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_Move_13_-=3E_103?= Message-ID: <20150915012142.27709.21417@psf.io> https://hg.python.org/peps/rev/e275c4cd4b44 changeset: 6059:e275c4cd4b44 user: Barry Warsaw date: Mon Sep 14 21:21:40 2015 -0400 summary: Move 13 -> 103 files: pep-0103.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/pep-0013.rst b/pep-0103.rst rename from pep-0013.rst rename to pep-0103.rst --- a/pep-0013.rst +++ b/pep-0103.rst @@ -1,4 +1,4 @@ -PEP: 13 +PEP: 103 Title: Collecting information about git Version: $Revision$ Last-Modified: $Date$ -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Tue Sep 15 03:37:33 2015 From: python-checkins at python.org (chris.angelico) Date: Tue, 15 Sep 2015 01:37:33 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_Apply_PEP_502_changes_from_Mi?= =?utf-8?q?ke_Miller?= Message-ID: <20150915013733.17991.44116@psf.io> https://hg.python.org/peps/rev/1bee90a93fa2 changeset: 6060:1bee90a93fa2 user: Chris Angelico date: Tue Sep 15 11:37:22 2015 +1000 summary: Apply PEP 502 changes from Mike Miller files: pep-0502.txt | 453 ++++++++++++++------------------------ 1 files changed, 163 insertions(+), 290 deletions(-) diff --git a/pep-0502.txt b/pep-0502.txt --- a/pep-0502.txt +++ b/pep-0502.txt @@ -1,44 +1,46 @@ PEP: 502 -Title: String Interpolation Redux +Title: String Interpolation - Extended Discussion Version: $Revision$ Last-Modified: $Date$ Author: Mike G. Miller Status: Draft -Type: Standards Track +Type: Informational Content-Type: text/x-rst Created: 10-Aug-2015 Python-Version: 3.6 -Note: Open issues below are stated with a question mark (?), -and are therefore searchable. - Abstract ======== -This proposal describes a new string interpolation feature for Python, -called an *expression-string*, -that is both concise and powerful, -improves readability in most cases, -yet does not conflict with existing code. +PEP 498: *Literal String Interpolation*, which proposed "formatted strings" was +accepted September 9th, 2015. +Additional background and rationale given during its design phase is detailed +below. -To achieve this end, -a new string prefix is introduced, -which expands at compile-time into an equivalent expression-string object, -with requested variables from its context passed as keyword arguments. +To recap that PEP, +a string prefix was introduced that marks the string as a template to be +rendered. +These formatted strings may contain one or more expressions +built on `the existing syntax`_ of ``str.format()``. +The formatted string expands at compile-time into a conventional string format +operation, +with the given expressions from its text extracted and passed instead as +positional arguments. + At runtime, -the new object uses these passed values to render a string to given -specifications, building on `the existing syntax`_ of ``str.format()``:: +the resulting expressions are evaluated to render a string to given +specifications:: >>> location = 'World' - >>> e'Hello, {location} !' # new prefix: e'' - 'Hello, World !' # interpolated result + >>> f'Hello, {location} !' # new prefix: f'' + 'Hello, World !' # interpolated result + +Format-strings may be thought of as merely syntactic sugar to simplify traditional +calls to ``str.format()``. .. _the existing syntax: https://docs.python.org/3/library/string.html#format-string-syntax -This PEP does not recommend to remove or deprecate any of the existing string -formatting mechanisms. - Motivation ========== @@ -50,12 +52,16 @@ with similar use cases, the amount of code necessary to build similar strings is substantially higher, while at times offering lower readability due to verbosity, dense syntax, -or identifier duplication. [1]_ +or identifier duplication. + +These difficulties are described at moderate length in the original +`post to python-ideas`_ +that started the snowball (that became PEP 498) rolling. [1]_ Furthermore, replacement of the print statement with the more consistent print function of Python 3 (PEP 3105) has added one additional minor burden, an additional set of parentheses to type and read. -Combined with the verbosity of current formatting solutions, +Combined with the verbosity of current string formatting solutions, this puts an otherwise simple language at an unfortunate disadvantage to its peers:: @@ -66,7 +72,7 @@ # Python 3, str.format with named parameters print('Hello, user: {user}, id: {id}, on host: {hostname}'.format(**locals())) - # Python 3, variation B, worst case + # Python 3, worst case print('Hello, user: {user}, id: {id}, on host: {hostname}'.format(user=user, id=id, hostname= @@ -74,7 +80,7 @@ In Python, the formatting and printing of a string with multiple variables in a single line of code of standard width is noticeably harder and more verbose, -indentation often exacerbating the issue. +with indentation exacerbating the issue. For use cases such as smaller projects, systems programming, shell script replacements, and even one-liners, @@ -82,36 +88,17 @@ this verbosity has likely lead a significant number of developers and administrators to choose other languages over the years. +.. _post to python-ideas: https://mail.python.org/pipermail/python-ideas/2015-July/034659.html + Rationale ========= -Naming ------- - -The term expression-string was chosen because other applicable terms, -such as format-string and template are already well used in the Python standard -library. - -The string prefix itself, ``e''`` was chosen to demonstrate that the -specification enables expressions, -is not limited to ``str.format()`` syntax, -and also does not lend itself to `the shorthand term`_ "f-string". -It is also slightly easier to type than other choices such as ``_''`` and -``i''``, -while perhaps `less odd-looking`_ to C-developers. -``printf('')`` vs. ``print(f'')``. - -.. _the shorthand term: reference_needed -.. _less odd-looking: https://mail.python.org/pipermail/python-dev/2015-August/141147.html - - - Goals ------------- -The design goals of expression-strings are as follows: +The design goals of format strings are as follows: #. Eliminate need to pass variables manually. #. Eliminate repetition of identifiers and redundant parentheses. @@ -133,40 +120,44 @@ characters to enclose strings. It is not reasonable to choose one of them now to enable interpolation, while leaving the other for uninterpolated strings. -"Backtick" characters (`````) are also `constrained by history`_ as a shortcut -for ``repr()``. +Other characters, +such as the "Backtick" (or grave accent `````) are also +`constrained by history`_ +as a shortcut for ``repr()``. This leaves a few remaining options for the design of such a feature: * An operator, as in printf-style string formatting via ``%``. * A class, such as ``string.Template()``. -* A function, such as ``str.format()``. -* New syntax +* A method or function, such as ``str.format()``. +* New syntax, or * A new string prefix marker, such as the well-known ``r''`` or ``u''``. -The first three options above currently work well. +The first three options above are mature. Each has specific use cases and drawbacks, yet also suffer from the verbosity and visual noise mentioned previously. -All are discussed in the next section. +All options are discussed in the next sections. .. _constrained by history: https://mail.python.org/pipermail/python-ideas/2007-January/000054.html + Background ------------- -This proposal builds on several existing techniques and proposals and what +Formatted strings build on several existing techniques and proposals and what we've collectively learned from them. +In keeping with the design goals of readability and error-prevention, +the following examples therefore use named, +not positional arguments. -The following examples focus on the design goals of readability and -error-prevention using named parameters. Let's assume we have the following dictionary, and would like to print out its items as an informative string for end users:: >>> params = {'user': 'nobody', 'id': 9, 'hostname': 'darkstar'} -Printf-style formatting -''''''''''''''''''''''' +Printf-style formatting, via operator +''''''''''''''''''''''''''''''''''''' This `venerable technique`_ continues to have its uses, such as with byte-based protocols, @@ -178,7 +169,7 @@ In this form, considering the prerequisite dictionary creation, the technique is verbose, a tad noisy, -and relatively readable. +yet relatively readable. Additional issues are that an operator can only take one argument besides the original string, meaning multiple parameters must be passed in a tuple or dictionary. @@ -190,8 +181,8 @@ .. _venerable technique: https://docs.python.org/3/library/stdtypes.html#printf-style-string-formatting -string.Template -''''''''''''''' +string.Template Class +''''''''''''''''''''' The ``string.Template`` `class from`_ PEP 292 (Simpler String Substitutions) @@ -202,7 +193,7 @@ Template('Hello, user: $user, id: ${id}, on host: $hostname').substitute(params) -Also verbose, however the string itself is readable. +While also verbose, the string itself is readable. Though functionality is limited, it meets its requirements well. It isn't powerful enough for many cases, @@ -232,8 +223,8 @@ It was superseded by the following proposal. -str.format() -'''''''''''' +str.format() Method +''''''''''''''''''' The ``str.format()`` `syntax of`_ PEP 3101 is the most recent and modern of the existing options. @@ -253,36 +244,32 @@ host=hostname) 'Hello, user: nobody, id: 9, on host: darkstar' +The verbosity of the method-based approach is illustrated here. + .. _syntax of: https://docs.python.org/3/library/string.html#format-string-syntax PEP 498 -- Literal String Formatting '''''''''''''''''''''''''''''''''''' -PEP 498 discusses and delves partially into implementation details of -expression-strings, -which it calls f-strings, -the idea and syntax -(with exception of the prefix letter) -of which is identical to that discussed here. -The resulting compile-time transformation however -returns a string joined from parts at runtime, -rather than an object. +PEP 498 defines and discusses format strings, +as also described in the `Abstract`_ above. -It also, somewhat controversially to those first exposed to it, -introduces the idea that these strings shall be augmented with support for -arbitrary expressions, -which is discussed further in the following sections. - +It also, somewhat controversially to those first exposed, +introduces the idea that format-strings shall be augmented with support for +arbitrary expressions. +This is discussed further in the +Restricting Syntax section under +`Rejected Ideas`_. PEP 501 -- Translation ready string interpolation ''''''''''''''''''''''''''''''''''''''''''''''''' The complimentary PEP 501 brings internationalization into the discussion as a -first-class concern, with its proposal of i-strings, +first-class concern, with its proposal of the i-prefix, ``string.Template`` syntax integration compatible with ES6 (Javascript), deferred rendering, -and a similar object return value. +and an object return value. Implementations in Other Languages @@ -374,7 +361,8 @@ Designers of `Template strings`_ faced the same issue as Python where single and double quotes were taken. Unlike Python however, "backticks" were not. -They were chosen as part of the ECMAScript 2015 (ES6) standard:: +Despite `their issues`_, +they were chosen as part of the ECMAScript 2015 (ES6) standard:: console.log(`Fifteen is ${a + b} and\nnot ${2 * a + b}.`); @@ -391,8 +379,10 @@ * User implemented prefixes supported. * Arbitrary expressions are supported. +.. _their issues: https://mail.python.org/pipermail/python-ideas/2007-January/000054.html .. _Template strings: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/template_strings + C#, Version 6 ''''''''''''' @@ -428,13 +418,14 @@ Additional examples ''''''''''''''''''' -A number of additional examples may be `found at Wikipedia`_. +A number of additional examples of string interpolation may be +`found at Wikipedia`_. + +Now that background and history have been covered, +let's continue on for a solution. .. _found at Wikipedia: https://en.wikipedia.org/wiki/String_interpolation#Examples -Now that background and imlementation history have been covered, -let's continue on for a solution. - New Syntax ---------- @@ -442,178 +433,47 @@ This should be an option of last resort, as every new syntax feature has a cost in terms of real-estate in a brain it inhabits. -There is one alternative left on our list of possibilities, +There is however one alternative left on our list of possibilities, which follows. New String Prefix ----------------- -Given the history of string formatting in Python, -backwards-compatibility, +Given the history of string formatting in Python and backwards-compatibility, implementations in other languages, -and the avoidance of new syntax unless necessary, +avoidance of new syntax unless necessary, an acceptable design is reached through elimination rather than unique insight. -Therefore, we choose to explicitly mark interpolated string literals with a -string prefix. +Therefore, marking interpolated string literals with a string prefix is chosen. -We also choose an expression syntax that reuses and builds on the strongest of +We also choose an expression syntax that reuses and builds on the strongest of the existing choices, -``str.format()`` to avoid further duplication. - - -Specification -============= - -String literals with the prefix of ``e`` shall be converted at compile-time to -the construction of an ``estr`` (perhaps ``types.ExpressionString``?) object. -Strings and values are parsed from the literal and passed as tuples to the -constructor:: +``str.format()`` to avoid further duplication of functionality:: >>> location = 'World' - >>> e'Hello, {location} !' + >>> f'Hello, {location} !' # new prefix: f'' + 'Hello, World !' # interpolated result - # becomes - # estr('Hello, {location} !', # template - ('Hello, ', ' !'), # string fragments - ('location',), # expressions - ('World',), # values - ) +PEP 498 -- Literal String Formatting, delves into the mechanics and +implementation of this design. -The object interpolates its result immediately at run-time:: - 'Hello, World !' - - -ExpressionString Objects ------------------------- - -The ExpressionString object supports both immediate and deferred rendering of -its given template and parameters. -It does this by immediately rendering its inputs to its internal string and -``.rendered`` string member (still necessary?), -useful in the majority of use cases. -To allow for deferred rendering and caller-specified escaping, -all inputs are saved for later inspection, -with convenience methods available. - -Notes: - -* Inputs are saved to the object as ``.template`` and ``.context`` members - for later use. -* No explicit ``str(estr)`` call is necessary to render the result, - though doing so might be desired to free resources if significant. -* Additional or deferred rendering is available through the ``.render()`` - method, which allows template and context to be overriden for flexibility. -* Manual escaping of potentially dangerous input is available through the - ``.escape(escape_function)`` method, - the rules of which may therefore be specified by the caller. - The given function should both accept and return a single modified string. - -* A sample Python implementation can `found at Bitbucket`_: - -.. _found at Bitbucket: https://bitbucket.org/mixmastamyk/docs/src/default/pep/estring_demo.py - - -Inherits From ``str`` Type -''''''''''''''''''''''''''' - -Inheriting from the ``str`` class is one of the techniques available to improve -compatibility with code expecting a string object, -as it will pass an ``isinstance(obj, str)`` test. -ExpressionString implements this and also renders its result into the "raw" -string of its string superclass, -providing compatibility with a majority of code. - - -Interpolation Syntax --------------------- - -The strongest of the existing string formatting syntaxes is chosen, -``str.format()`` as a base to build on. [10]_ [11]_ - -.. - -* Additionally, single arbitrary expressions shall also be supported inside - braces as an extension:: - - >>> e'My age is {age + 1} years.' - - See below for section on safety. - -* Triple quoted strings with multiple lines shall be supported:: - - >>> e'''Hello, - {location} !''' - 'Hello,\n World !' - -* Adjacent implicit concatenation shall be supported; - interpolation does not `not bleed into`_ other strings:: - - >>> 'Hello {1, 2, 3} ' e'{location} !' - 'Hello {1, 2, 3} World !' - -* Additional implementation details, - for example expression and error-handling, - are specified in the compatible PEP 498. - -.. _not bleed into: https://mail.python.org/pipermail/python-ideas/2015-July/034763.html - - -Composition with Other Prefixes -------------------------------- - -* Expression-strings apply to unicode objects only, - therefore ``u''`` is never needed. - Should it be prevented? - -* Bytes objects are not included here and do not compose with e'' as they - do not support ``__format__()``. - -* Complimentary to raw strings, - backslash codes shall not be converted in the expression-string, - when combined with ``r''`` as ``re''``. - - -Examples --------- - -A more complicated example follows:: - - n = 5; # t0, t1 = ? TODO - a = e"Sliced {n} onions in {t1-t0:.3f} seconds." - # returns the equvalent of - estr("Sliced {n} onions in {t1-t0:.3f} seconds", # template - ('Sliced ', ' onions in ', ' seconds'), # strings - ('n', 't1-t0:.3f'), # expressions - (5, 0.555555) # values - ) - -With expressions only:: - - b = e"Three random numbers: {rand()}, {rand()}, {rand()}." - # returns the equvalent of - estr("Three random numbers: {rand():f}, {rand():f}, {rand():}.", # template - ('Three random numbers: ', ', ', ', ', '.'), # strings - ('rand():f', 'rand():f', 'rand():f'), # expressions - (rand(), rand(), rand()) # values - ) +Additional Topics +================= Safety ----------- In this section we will describe the safety situation and precautions taken -in support of expression-strings. +in support of format-strings. -#. Only string literals shall be considered here, +#. Only string literals have been considered for format-strings, not variables to be taken as input or passed around, making external attacks difficult to accomplish. - * ``str.format()`` `already handles`_ this use-case. - * Direct instantiation of the ExpressionString object with non-literal input - shall not be allowed. (Practicality?) + ``str.format()`` and alternatives `already handle`_ this use-case. #. Neither ``locals()`` nor ``globals()`` are necessary nor used during the transformation, @@ -622,37 +482,72 @@ #. To eliminate complexity as well as ``RuntimeError`` (s) due to recursion depth, recursive interpolation is not supported. -#. Restricted characters or expression classes?, such as ``=`` for assignment. - However, mistakes or malicious code could be missed inside string literals. Though that can be said of code in general, that these expressions are inside strings means they are a bit more likely to be obscured. -.. _already handles: https://mail.python.org/pipermail/python-ideas/2015-July/034729.html +.. _already handle: https://mail.python.org/pipermail/python-ideas/2015-July/034729.html -Mitigation via tools +Mitigation via Tools '''''''''''''''''''' The idea is that tools or linters such as pyflakes, pylint, or Pycharm, -could check inside strings for constructs that exceed project policy. -As this is a common task with languages these days, -tools won't have to implement this feature solely for Python, +may check inside strings with expressions and mark them up appropriately. +As this is a common task with programming languages today, +multi-language tools won't have to implement this feature solely for Python, significantly shortening time to implementation. -Additionally the Python interpreter could check(?) and warn with appropriate -command-line parameters passed. +Farther in the future, +strings might also be checked for constructs that exceed the safety policy of +a project. + + +Style Guide/Precautions +----------------------- + +As arbitrary expressions may accomplish anything a Python expression is +able to, +it is highly recommended to avoid constructs inside format-strings that could +cause side effects. + +Further guidelines may be written once usage patterns and true problems are +known. + + +Reference Implementation(s) +--------------------------- + +The `say module on PyPI`_ implements string interpolation as described here +with the small burden of a callable interface:: + + ? pip install say + + from say import say + nums = list(range(4)) + say("Nums has {len(nums)} items: {nums}") + +A Python implementation of Ruby interpolation `is also available`_. +It uses the codecs module to do its work:: + + ? pip install interpy + + # coding: interpy + location = 'World' + print("Hello #{location}.") + +.. _say module on PyPI: https://pypi.python.org/pypi/say/ +.. _is also available: https://github.com/syrusakbary/interpy Backwards Compatibility ----------------------- -By using existing syntax and avoiding use of current or historical features, -expression-strings (and any associated sub-features), -were designed so as to not interfere with existing code and is not expected -to cause any issues. +By using existing syntax and avoiding current or historical features, +format strings were designed so as to not interfere with existing code and are +not expected to cause any issues. Postponed Ideas @@ -666,20 +561,12 @@ the finer details diverge at almost every point, making a common solution unlikely: [15]_ -* Use-cases -* Compile and run-time tasks -* Interpolation Syntax +* Use-cases differ +* Compile vs. run-time tasks +* Interpolation syntax needs * Intended audience * Security policy -Rather than try to fit a "square peg in a round hole," -this PEP attempts to allow internationalization to be supported in the future -by not preventing it. -In this proposal, -expression-string inputs are saved for inspection and re-rendering at a later -time, -allowing for their use by an external library of any sort. - Rejected Ideas -------------- @@ -687,17 +574,24 @@ Restricting Syntax to ``str.format()`` Only ''''''''''''''''''''''''''''''''''''''''''' -This was deemed not enough of a solution to the problem. +The common `arguments against`_ support of arbitrary expresssions were: + +#. `YAGNI`_, "You aren't gonna need it." +#. The feature is not congruent with historical Python conservatism. +#. Postpone - can implement in a future version if need is demonstrated. + +.. _YAGNI: https://en.wikipedia.org/wiki/You_aren't_gonna_need_it +.. _arguments against: https://mail.python.org/pipermail/python-ideas/2015-August/034913.html + +Support of only ``str.format()`` syntax however, +was deemed not enough of a solution to the problem. +Often a simple length or increment of an object, for example, +is desired before printing. + It can be seen in the `Implementations in Other Languages`_ section that the developer community at large tends to agree. - -The common `arguments against`_ arbitrary expresssions were: - -#. YAGNI, "You ain't gonna need it." -#. The change is not congruent with historical Python conservatism. -#. Postpone - can implement in a future version if need is demonstrated. - -.. _arguments against: https://mail.python.org/pipermail/python-ideas/2015-August/034913.html +String interpolation with arbitrary expresssions is becoming an industry +standard in modern languages due to its utility. Additional/Custom String-Prefixes @@ -720,7 +614,7 @@ expressions could be used safely or not. The concept was also difficult to describe to others. [12]_ -Always consider expression-string variables to be unescaped, +Always consider format string variables to be unescaped, unless the developer has explicitly escaped them. @@ -735,33 +629,13 @@ which could encourage bad habits. [13]_ -Reference Implementation(s) -=========================== - -An expression-string implementation is currently attached to PEP 498, -under the ``f''`` prefix, -and may be available in nightly builds. - -A Python implementation of Ruby interpolation `is also available`_, -which is similar to this proposal. -It uses the codecs module to do its work:: - - ? pip install interpy - - # coding: interpy - location = 'World' - print("Hello #{location}.") - -.. _is also available: https://github.com/syrusakbary/interpy - - Acknowledgements ================ -* Eric V. Smith for providing invaluable implementation work and design - opinions, helping to focus this PEP. -* Others on the python-ideas mailing list for rejecting the craziest of ideas, - also helping to achieve focus. +* Eric V. Smith for the authoring and implementation of PEP 498. +* Everyone on the python-ideas mailing list for rejecting the various crazy + ideas that came up, + helping to keep the final design in focus. References @@ -771,7 +645,6 @@ (https://mail.python.org/pipermail/python-ideas/2015-July/034659.html) - .. [2] Briefer String Format (https://mail.python.org/pipermail/python-ideas/2015-July/034669.html) -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Tue Sep 15 10:10:54 2015 From: python-checkins at python.org (larry.hastings) Date: Tue, 15 Sep 2015 08:10:54 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_Touchup_for_PEP_101=3A_instal?= =?utf-8?q?led_docs_should_be_group-writeable=2E?= Message-ID: <20150915081054.11260.60695@psf.io> https://hg.python.org/peps/rev/d951ef0556b9 changeset: 6061:d951ef0556b9 user: Larry Hastings date: Tue Sep 15 09:10:50 2015 +0100 summary: Touchup for PEP 101: installed docs should be group-writeable. files: pep-0101.txt | 11 ++++++----- 1 files changed, 6 insertions(+), 5 deletions(-) diff --git a/pep-0101.txt b/pep-0101.txt --- a/pep-0101.txt +++ b/pep-0101.txt @@ -424,11 +424,12 @@ that directory. Note though that if you're releasing a maintenance release for an older version, don't change the current link. - ___ If this is a final release (even a maintenance release), also unpack - the HTML docs to /srv/docs.python.org/release/X.Y.Z on - docs.iad1.psf.io. Make sure the files are in group "docs". If it is a - release of a security-fix-only version, tell the DE to build a version - with the "version switcher" and put it there. + ___ If this is a final release (even a maintenance release), also + unpack the HTML docs to /srv/docs.python.org/release/X.Y.Z on + docs.iad1.psf.io. Make sure the files are in group "docs" and are + group-writeable. If it is a release of a security-fix-only version, + tell the DE to build a version with the "version switcher" + and put it there. ___ Let the DE check if the docs are built and work all right. -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Tue Sep 15 10:26:28 2015 From: python-checkins at python.org (victor.stinner) Date: Tue, 15 Sep 2015 08:26:28 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI1MTE4?= =?utf-8?q?=3A_Fix_a_regression_of_Python_3=2E5=2E0_in_os=2Ewaitpid=28=29_?= =?utf-8?q?on_Windows=2E?= Message-ID: <20150915082628.11270.29178@psf.io> https://hg.python.org/cpython/rev/9a80c687c28d changeset: 98004:9a80c687c28d branch: 3.5 parent: 98002:849bdcbff518 user: Victor Stinner date: Tue Sep 15 10:11:03 2015 +0200 summary: Issue #25118: Fix a regression of Python 3.5.0 in os.waitpid() on Windows. Add an unit test on os.waitpid() files: Lib/test/test_os.py | 6 ++++++ Misc/NEWS | 2 ++ Modules/posixmodule.c | 4 ++-- 3 files changed, 10 insertions(+), 2 deletions(-) diff --git a/Lib/test/test_os.py b/Lib/test/test_os.py --- a/Lib/test/test_os.py +++ b/Lib/test/test_os.py @@ -2078,6 +2078,12 @@ # We are the parent of our subprocess self.assertEqual(int(stdout), os.getpid()) + def test_waitpid(self): + args = [sys.executable, '-c', 'pass'] + pid = os.spawnv(os.P_NOWAIT, args[0], args) + status = os.waitpid(pid, 0) + self.assertEqual(status, (pid, 0)) + # The introduction of this TestCase caused at least two different errors on # *nix buildbots. Temporarily skip this to let the buildbots move along. diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -14,6 +14,8 @@ Library ------- +- Issue #25118: Fix a regression of Python 3.5.0 in os.waitpid() on Windows. + - Issue #24684: socket.socket.getaddrinfo() now calls PyUnicode_AsEncodedString() instead of calling the encode() method of the host, to handle correctly custom string with an encode() method which doesn't diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c --- a/Modules/posixmodule.c +++ b/Modules/posixmodule.c @@ -7021,7 +7021,7 @@ res = _cwait(&status, pid, options); Py_END_ALLOW_THREADS } while (res < 0 && errno == EINTR && !(async_err = PyErr_CheckSignals())); - if (res != 0) + if (res < 0) return (!async_err) ? posix_error() : NULL; /* shift the status left a byte so this is more like the POSIX waitpid */ @@ -7731,7 +7731,7 @@ } while (fd < 0 && errno == EINTR && !(async_err = PyErr_CheckSignals())); _Py_END_SUPPRESS_IPH - if (fd == -1) { + if (fd < 0) { if (!async_err) PyErr_SetFromErrnoWithFilenameObject(PyExc_OSError, path->object); return -1; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 15 10:26:29 2015 From: python-checkins at python.org (victor.stinner) Date: Tue, 15 Sep 2015 08:26:29 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogTWVyZ2UgMy41IChvcy53YWl0cGlkKQ==?= Message-ID: <20150915082628.114691.82199@psf.io> https://hg.python.org/cpython/rev/1b366e231387 changeset: 98005:1b366e231387 parent: 98003:4a431c65e57e parent: 98004:9a80c687c28d user: Victor Stinner date: Tue Sep 15 10:24:27 2015 +0200 summary: Merge 3.5 (os.waitpid) files: Lib/test/test_os.py | 6 ++++++ Misc/NEWS | 2 ++ Modules/posixmodule.c | 4 ++-- 3 files changed, 10 insertions(+), 2 deletions(-) diff --git a/Lib/test/test_os.py b/Lib/test/test_os.py --- a/Lib/test/test_os.py +++ b/Lib/test/test_os.py @@ -2077,6 +2077,12 @@ # We are the parent of our subprocess self.assertEqual(int(stdout), os.getpid()) + def test_waitpid(self): + args = [sys.executable, '-c', 'pass'] + pid = os.spawnv(os.P_NOWAIT, args[0], args) + status = os.waitpid(pid, 0) + self.assertEqual(status, (pid, 0)) + # The introduction of this TestCase caused at least two different errors on # *nix buildbots. Temporarily skip this to let the buildbots move along. diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -103,6 +103,8 @@ Library ------- +- Issue #25118: Fix a regression of Python 3.5.0 in os.waitpid() on Windows. + - Issue #24684: socket.socket.getaddrinfo() now calls PyUnicode_AsEncodedString() instead of calling the encode() method of the host, to handle correctly custom string with an encode() method which doesn't diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c --- a/Modules/posixmodule.c +++ b/Modules/posixmodule.c @@ -7021,7 +7021,7 @@ res = _cwait(&status, pid, options); Py_END_ALLOW_THREADS } while (res < 0 && errno == EINTR && !(async_err = PyErr_CheckSignals())); - if (res != 0) + if (res < 0) return (!async_err) ? posix_error() : NULL; /* shift the status left a byte so this is more like the POSIX waitpid */ @@ -7731,7 +7731,7 @@ } while (fd < 0 && errno == EINTR && !(async_err = PyErr_CheckSignals())); _Py_END_SUPPRESS_IPH - if (fd == -1) { + if (fd < 0) { if (!async_err) PyErr_SetFromErrnoWithFilenameObject(PyExc_OSError, path->object); return -1; -- Repository URL: https://hg.python.org/cpython From solipsis at pitrou.net Tue Sep 15 10:46:42 2015 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Tue, 15 Sep 2015 08:46:42 +0000 Subject: [Python-checkins] Daily reference leaks (4a431c65e57e): sum=61923 Message-ID: <20150915084641.11272.45314@psf.io> results for 4a431c65e57e on branch "default" -------------------------------------------- test_capi leaked [5446, 5446, 5446] references, sum=16338 test_capi leaked [1433, 1435, 1435] memory blocks, sum=4303 test_functools leaked [0, 2, 2] memory blocks, sum=4 test_threading leaked [10892, 10892, 10892] references, sum=32676 test_threading leaked [2866, 2868, 2868] memory blocks, sum=8602 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/psf-users/antoine/refleaks/reflog732tSo', '--timeout', '7200'] From python-checkins at python.org Tue Sep 15 12:19:35 2015 From: python-checkins at python.org (victor.stinner) Date: Tue, 15 Sep 2015 10:19:35 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2325122=3A_try_to_d?= =?utf-8?q?ebug_test=5Feintr_hang_on_FreeBSD?= Message-ID: <20150915101934.11258.70912@psf.io> https://hg.python.org/cpython/rev/3d9164aecc6f changeset: 98006:3d9164aecc6f user: Victor Stinner date: Tue Sep 15 12:15:59 2015 +0200 summary: Issue #25122: try to debug test_eintr hang on FreeBSD * Add verbose mode to test_eintr * Always enable verbose mode in test_eintr * Use faulthandler.dump_traceback_later() with a timeout of 15 minutes in eintr_tester.py files: Lib/test/eintrdata/eintr_tester.py | 8 ++++++++ Lib/test/test_eintr.py | 12 +++++++++++- 2 files changed, 19 insertions(+), 1 deletions(-) diff --git a/Lib/test/eintrdata/eintr_tester.py b/Lib/test/eintrdata/eintr_tester.py --- a/Lib/test/eintrdata/eintr_tester.py +++ b/Lib/test/eintrdata/eintr_tester.py @@ -8,6 +8,7 @@ sub-second periodicity (contrarily to signal()). """ +import faulthandler import io import os import select @@ -36,12 +37,19 @@ @classmethod def setUpClass(cls): cls.orig_handler = signal.signal(signal.SIGALRM, lambda *args: None) + if hasattr(faulthandler, 'dump_traceback_later'): + # Most tests take less than 30 seconds, so 15 minutes should be + # enough. dump_traceback_later() is implemented with a thread, but + # pthread_sigmask() is used to mask all signaled on this thread. + faulthandler.dump_traceback_later(5 * 60, exit=True) signal.setitimer(signal.ITIMER_REAL, cls.signal_delay, cls.signal_period) @classmethod def stop_alarm(cls): signal.setitimer(signal.ITIMER_REAL, 0, 0) + if hasattr(faulthandler, 'cancel_dump_traceback_later'): + faulthandler.cancel_dump_traceback_later() @classmethod def tearDownClass(cls): diff --git a/Lib/test/test_eintr.py b/Lib/test/test_eintr.py --- a/Lib/test/test_eintr.py +++ b/Lib/test/test_eintr.py @@ -1,5 +1,7 @@ import os import signal +import subprocess +import sys import unittest from test import support @@ -14,7 +16,15 @@ # Run the tester in a sub-process, to make sure there is only one # thread (for reliable signal delivery). tester = support.findfile("eintr_tester.py", subdir="eintrdata") - script_helper.assert_python_ok(tester) + + # FIXME: Issue #25122, always run in verbose mode to debug hang on FreeBSD + if True: #support.verbose: + args = [sys.executable, tester] + with subprocess.Popen(args, stdout=sys.stderr) as proc: + exitcode = proc.wait() + self.assertEqual(exitcode, 0) + else: + script_helper.assert_python_ok(tester) if __name__ == "__main__": -- Repository URL: https://hg.python.org/cpython From lp_benchmark_robot at intel.com Tue Sep 15 13:40:37 2015 From: lp_benchmark_robot at intel.com (lp_benchmark_robot) Date: Tue, 15 Sep 2015 11:40:37 +0000 Subject: [Python-checkins] Benchmark Results for Python 2.7 2015-09-15 Message-ID: Results for project python_2.7-nightly, build date 2015-09-15 03:43:43 commit: 21d6b2752fe8db0428c92d928678b15d76a9a27b revision date: 2015-09-14 22:19:47 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v2.7.10, with hash 15c95b7d81dcf821daade360741e00714667653f from 2015-05-23 16:02:14+00:00 ------------------------------------------------------------------------------------------ benchmark relative change since change since current rev with std_dev* last run v2.7.10 regrtest PGO ------------------------------------------------------------------------------------------ :-) django_v2 0.29878% -1.76221% 3.17342% 11.27247% :-) pybench 0.16246% -0.03220% 6.73896% 6.53390% :-| regex_v8 0.59862% 0.02829% -1.15354% 7.50447% :-) nbody 0.28427% 0.00747% 9.04932% 2.89917% :-) json_dump_v2 0.25198% 0.48636% 4.42795% 13.96206% :-| normal_startup 1.83289% -0.42891% -1.66451% 3.47175% :-) ssbench 0.15324% 0.85916% 2.41400% 2.04298% ------------------------------------------------------------------------------------------ Note: Benchmark results for ssbench are measured in requests/second while all other are measured in seconds. * Relative Standard Deviation (Standard Deviation/Average) Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. From lp_benchmark_robot at intel.com Tue Sep 15 13:40:33 2015 From: lp_benchmark_robot at intel.com (lp_benchmark_robot) Date: Tue, 15 Sep 2015 11:40:33 +0000 Subject: [Python-checkins] Benchmark Results for Python Default 2015-09-15 Message-ID: Results for project python_default-nightly, build date 2015-09-15 03:02:33 commit: 4a431c65e57ee97bb7d9a279246f25176c42cd56 revision date: 2015-09-14 22:23:20 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v3.4.3, with hash b4cbecbc0781e89a309d03b60a1f75f8499250e6 from 2015-02-25 12:15:33+00:00 ------------------------------------------------------------------------------------------ benchmark relative change since change since current rev with std_dev* last run v3.4.3 regrtest PGO ------------------------------------------------------------------------------------------ :-) django_v2 0.19308% -0.72808% 8.52559% 13.65444% :-( pybench 0.11644% -0.42210% -2.34506% 9.00908% :-( regex_v8 2.77002% 0.41101% -3.45827% 4.65906% :-| nbody 0.10981% -0.08802% -1.81244% 7.94668% :-) json_dump_v2 0.38178% 2.54161% -2.19667% 12.87668% :-| normal_startup 0.64812% -0.04470% 0.24311% 4.72316% ------------------------------------------------------------------------------------------ Note: Benchmark results are measured in seconds. * Relative Standard Deviation (Standard Deviation/Average) Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. From python-checkins at python.org Tue Sep 15 14:32:19 2015 From: python-checkins at python.org (barry.warsaw) Date: Tue, 15 Sep 2015 12:32:19 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_pep-0103=2Erst_-=3E_pep-0103?= =?utf-8?q?=2Etxt?= Message-ID: <20150915123218.11274.38155@psf.io> https://hg.python.org/peps/rev/e221cd71bf88 changeset: 6062:e221cd71bf88 user: Barry Warsaw date: Tue Sep 15 08:32:16 2015 -0400 summary: pep-0103.rst -> pep-0103.txt Update Oleg's email address. files: pep-0103.txt | 0 pep-3140.txt | 2 +- 2 files changed, 1 insertions(+), 1 deletions(-) diff --git a/pep-0103.rst b/pep-0103.txt rename from pep-0103.rst rename to pep-0103.txt diff --git a/pep-3140.txt b/pep-3140.txt --- a/pep-3140.txt +++ b/pep-3140.txt @@ -2,7 +2,7 @@ Title: str(container) should call str(item), not repr(item) Version: $Revision$ Last-Modified: $Date$ -Author: Oleg Broytmann , +Author: Oleg Broytman , Jim J. Jewett Discussions-To: python-3000 at python.org Status: Rejected -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Tue Sep 15 14:33:41 2015 From: python-checkins at python.org (barry.warsaw) Date: Tue, 15 Sep 2015 12:33:41 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_Another_update_to_103?= Message-ID: <20150915123340.11270.56389@psf.io> https://hg.python.org/peps/rev/7442df41c689 changeset: 6063:7442df41c689 user: Barry Warsaw date: Tue Sep 15 08:33:38 2015 -0400 summary: Another update to 103 files: pep-0103.txt | 27 +++++++++++++++++++++++++++ 1 files changed, 27 insertions(+), 0 deletions(-) diff --git a/pep-0103.txt b/pep-0103.txt --- a/pep-0103.txt +++ b/pep-0103.txt @@ -628,6 +628,33 @@ $ git merge -s ours v1 # null-merge v1 into master +Branching models +================ + +Git doesn't assume any particular development model regarding +branching and merging. Some projects prefer to graduate patches from +the oldest branch to the newest, some prefer to cherry-pick commits +backwards, some use squashing (combining a number of commits into +one). Anything is possible. + +There are a few examples to start with. `git help workflows +`_ +describes how the very git authors develop git. + +ProGit book has a few chapters devoted to branch management in +different projects: `Git Branching - Branching Workflows +`_ and +`Distributed Git - Contributing to a Project +`_. + +There is also a well-known article `A successful Git branching model +`_ by Vincent +Driessen. It recommends a set of very detailed rules on creating and +managing mainline, topic and bugfix branches. To support the model the +author implemented `git flow `_ +extension. + + Advanced configuration ====================== -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Tue Sep 15 16:29:13 2015 From: python-checkins at python.org (nick.coghlan) Date: Tue, 15 Sep 2015 14:29:13 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_PEP_504=3A_Using_the_System_R?= =?utf-8?q?NG_by_default?= Message-ID: <20150915142912.11252.31343@psf.io> https://hg.python.org/peps/rev/61d05f14aa37 changeset: 6064:61d05f14aa37 user: Nick Coghlan date: Wed Sep 16 00:29:04 2015 +1000 summary: PEP 504: Using the System RNG by default files: pep-0504.txt | 337 +++++++++++++++++++++++++++++++++++++++ 1 files changed, 337 insertions(+), 0 deletions(-) diff --git a/pep-0504.txt b/pep-0504.txt new file mode 100644 --- /dev/null +++ b/pep-0504.txt @@ -0,0 +1,337 @@ +PEP: 504 +Title: Using the System RNG by default +Version: $Revision$ +Last-Modified: $Date$ +Author: Nick Coghlan +Status: Draft +Type: Standards Track +Content-Type: text/x-rst +Created: 15-Sep-2015 +Python-Version: 3.6 +Post-History: 15-Sep-2015 + +Abstract +======== + +Python currently defaults to using the deterministic Mersenne Twister random +number generator for the module level APIs in the ``random`` module, requiring +users to know that when they're performing "security sensitive" work, they +should instead switch to using the cryptographically secure ``os.urandom`` or +``random.SystemRandom`` interfaces or a third party library like +``cryptography``. + +Unfortunately, this approach has resulted in a situation where developers that +aren't aware that they're doing security sensitive work use the default module +level APIs, and thus expose their users to unnecessary risks. + +This isn't an acute problem, but it is a chronic one, and if documentation and +developer education were going to solve it, they would have done so by now. + +In order to provide an eventually pervasive solution to the problem, this PEP +proposes that Python switch to using the system random number generator by +default in Python 3.6, and require developers to opt-in to using the +deterministic random number generator. + +To minimise the compatibility break, calling any of the following module level +functions will count as opting in to using the deterministic random number +generator for all future calls to module level functions in the random +module in the same process: + +* ``random.seed`` +* ``random.getstate`` +* ``random.setstate`` + +Proposal +======== + +Currently, it is never correct to use the module level functions in the +``random`` module for security sensitive applications. This PEP proposes to +change that admonition in Python 3.6+ to instead be that it is not correct to +use the module level functions in the ``random`` module for security sensitive +applications if ``random.seed``, ``random.getstate``, or ``random.setstate`` +are ever called in that process. + +This PEP further proposes to make it easier to explicitly opt in to using +either the system random number generator or Python's deterministic PRNG by +converting the random module to a package that exposes the same top-level API, +and offering two new subpackages: + +* ``random.system`` +* ``random.seedable`` + +The ``random.system`` submodule would provide the following bound methods of a +module global ``random.SystemRandom`` instance as module attributes: +``betavariate``, ``choice``, ``expovariate``, ``gammavariate``, ``gauss``, ``getrandbits``, ``lognormvariate``, ``normalvariate``, ``paretovariate``, +``randint``, ``random``, ``randrange``, ``sample``, ``shuffle``, +``triangular``, ``uniform``, ``vonmisesvariate``, ``weibullvariate`` + +The ``random.seedable`` submodule would provide the same operations, but as +methods of a ``random.Random`` instance. In addition, it would provide the +following additional methods which are only meaningful when using a +deterministic random number generator: ``seed``, ``getstate``, ``setstate``. + +Rather than being bound methods of a ``random.Random`` instance as they are +today, the module level callables in ``random`` itself would change to be +functions that, by default, delegated to the ``random.SystemRandom`` instance +in ``random.system``. + +Calling any one of ``random.seed``, ``random.getstate``, or ``random.setstate`` +would change the delegation to instead refer to the ``random.Random`` instance +in ``random.seedable``. + +Warning on implicit opt-in +-------------------------- + +In Python 3.6, implicitly opting in to the use of the seedable PRNG will emit a +deprecation warning. This warning will suggest explicitly opting in to either +the system RNG or the seedable PRNG. Possible wording: + + "DeprecationWarning: Implicitly switching to the seedable PRNG. Consider + importing from random.system or random.seedable as appropriate" + +Whatever precise wording is chosen should have an answer added to Stack +Overflow as was done for the custom error message that was added for missing +parentheses in a call to print [#print]_. + +In the first Python 3 release after Python 2.7 switches to security fix only +mode, the deprecation warning will be upgraded to a RuntimeWarning so it is +visible by default. + +This PEP does *not* propose removing the ability to seed the default RNG used +process wide - it's not a good idea relative to the alternative of explicitly +importing from the appropriate submodule (hence the eventually +visible-by-default warning), but it's also a concern that can be more +readily addressed on a project-by-project basis. + +Documentation changes +--------------------- + +The ``random`` module documentation would be updated to move the documentation +of the ``seed``, ``getstate`` and ``setstate`` interfaces later in the module, +along with the associated security warning. + +The docs would gain a discussion of the respective use cases for the seedable +PRNG (games, modelling & simulation, software testing) and the system RNG +(cryptography, security token generation). + +Rationale +========= + +Writing secure software under deadline and budget pressures is a hard problem. +This is reflected in ongoing problems with data breaches involving personally +identifiable information [#breaches]_, as well as with failures to take +security considerations into account when new systems, like motor vehicles +[#uconnect]_, are connected to the internet. Compounding the issue is the fact +that a lot of the programming advice readily available on the internet [#search] +simply doesn't take the mathemetical arcana of computer security into account, +and the fact that defenders have to cover *all* of their potential +vulnerabilites, as a single mistake can make it possible to subvert other +defences [#bcrypt]_. + +One of the factors that contributes to making this last aspect particularly +difficult is APIs where using them inappropriately creates a *silent* security +failure - one where the only way to find out that what you're doing is +incorrect is for someone reviewing your code to say "that's a potential +security problem", or for a system you're responsible for to be compromised +through such an oversight (and your intrusion detection and auditing mechanisms +are good enough for you to be able to figure out after the event how the +compromise took place). + +This kind of situation is a significant contributor to "security fatigue", +where developers (often rightly [#owasptopten]_) feel that security engineers +spend all their time saying "don't do that the easy way, it creates a +security vulnerability". + +As the designers of one of the world's most popular languages [#ieeetopten]_, +we can help reduce that problem by making the easy way the right way (or at +least the "not wrong" way) in more circumstances, so developers and security +engineers can spend more time worrying about mitigating actually interesting +threats, and less time fighting with default language behaviours. + +Discussion +========== + +Why "seedable" over "deterministic"? +------------------------------------ + +This is a case where the meaning of a word as specialist jargon conflicts with +the typical meaning of the word, even though it's *technically* the same. + +From a technical perspective, a "deterministic RNG" means that given knowledge +of the algorithm and the current state, you can reliably compute arbitrary +future states. + +The problem is that "deterministic" on its own doesn't convey those qualifiers, +so it's likely to instead be interpreted as "predictable" or "not random" by +folks that aren't familiar with the technical meaning. + +The other problem with "deterministic" as a description for the traditional RNG +is that it doesn't tell you what you can *do* with the traditional RNG that you +can't do with the system one. + +"seedable" aims to address both those problems, as it doesn't have a misleading +common meaning, and it's a word form that means "you can seed this", which then +leads naturally into an exploration of what it means to "seed" a random number +generator. + +Only changing the default for Python 3.6+ +----------------------------------------- + +Some other recent security changes, such as upgrading the capabilities of the +``ssl`` module and switching to properly verifying HTTPS certificates by +default, have been considered critical enough to justify backporting the +change to all currently supported versions of Python. + +The difference in this case is one of degree - the additional benefits from +rolling out this particular change a couple of years earlier than will +otherwise be the case aren't sufficient to justify the additional effort and +stability risks involved in making such an intrusive change in a maintenance +release. + +Keeping the module level functions +---------------------------------- + +In additional to general backwards compatibility considerations, Python is +widely used for educational purposes, and we specifically don't want to +invalidate the wide array of educational material that assumes the availabilty +of the current ``random`` module API. Accordingly, this proposal ensures that +most of the public API can continue to be used not only without modification, +but without generating any new warnings. + +Implicitly opting in to the deterministic RNG +--------------------------------------------- + +Python is widely used for modelling and simulation purposes, and in many cases, +these software models won't have a dedicated maintenance team tasked with +ensuing they keep working on the latest versions of Python. + +Using first DeprecationWarning, and then eventually a RuntimeWarning, to +advise against implicitly switching to the deterministic PRNG, preserves +compatibility with this existing software, while still nudging future users +that need a deterministic generator towards importing ``random.seedable`` +explicitly. + +Avoiding the introduction of a userspace CSPRNG +----------------------------------------------- + +The original discussion of this proposal on python-ideas[#csprng]_ suggested +introducing a cryptographically secure pseudo-random number generator and using +that by default, rather than defaulting to the relatively slow system random +number generator. + +The problem [#nocsprng]_ with this approach is that it introduces an additional +point of failure in security sensitive situations, for the sake of applications +where the random number generation may not even be on a critical performance +path. + +What about the performance impact? +---------------------------------- + +Rather than introducing a userspace CSPRNG, this PEP instead proposes that we +accept the performance regression in cases where: + +* an application is using the module level random API +* cryptographic quality randomness isn't needed +* the application doesn't already implicitly opt back in to the deterministic + PRNG by calling ``random.seed``, ``random.getstate``, or ``random.setstate`` +* the application isn't updated to explicitly import from ``random.seedable`` + rather than ``random`` + +Applications that need cryptographic quality randomness should be using the +system random number generator regardless of speed considerations, while other +applications where speed is a more important consideration are better off with +the current PRNG implementation than they would be with a new CSPRNG. + +Isn't the deterministic PRNG "secure enough"? +--------------------------------------------- + +In a word, "No" - that's why there's a warning in the module documentation +that says not to use it for security sensitive purposes. While we're not +currently aware of any studies of Python's random number generator specifically, +studies of PHP's random number generator [#php]_ have demonstrated the ability +to use weaknesses in that subsystem to facilitate a practical attack on +password recovery tokens in popular PHP web applications. + +Security fatigue in the Python ecosystem +---------------------------------------- + +Over the past few years, the computing industry as a whole has been +making a concerted effort to upgrade the shared network infrastructure we all +depend on to a "secure by default" stance. As one of the most widely used +programming languages for network service development (including the OpenStack +Infrastructure-as-a-Service platform) and for systems administration +on Linux systems in general, a fair share of that burden has fallen on the +Python ecosystem, which is understandably frustrating for Pythonistas using +Python in other contexts where these issues aren't of as great a concern. + +This consideration is one of the primary factors driving the backwards +compatibility improvements in this proposal relative to the initial draft +concept posted to python-ideas [#draft]_. + +Acknowledgements +================ + +* Theo de Raadt, for making the suggestion to Guido van Rossum that we + seriously consider defaulting to a cryptographically secure random number + generator +* Serhiy Storchaka, Terry Reedy, Petr Viktorin, and anyone else in the + python-ideas threads that suggested the approach of transparently switching + to the ``random.Random`` implementation when any of the functions that only + make sense for a deterministic RNG are called +* Nathaniel Smith for providing the reference on practical attacks against + PHP's random number generator when used to generate password reset tokens +* Donald Stufft for pursuing additional discussions with network security + experts that suggested the introduction of a userspace CSPRNG would mean + additional complexity for insufficient gain relative to just using the + system RNG directly + +References +========== + +.. [#breaches] Visualization of data breaches involving more than 30k records (each) + (http://www.informationisbeautiful.net/visualizations/worlds-biggest-data-breaches-hacks/) + +.. [#uconnect] Remote UConnect hack for Jeep Cherokee + (http://www.wired.com/2015/07/hackers-remotely-kill-jeep-highway/) + +.. [#php] PRNG based attack against password reset tokens in PHP applications + (https://media.blackhat.com/bh-us-12/Briefings/Argyros/BH_US_12_Argyros_PRNG_WP.pdf) + +.. [#search] Search link for "python password generator" + (https://www.google.com.au/search?q=python+password+generator) + +.. [#csprng] python-ideas thread discussing using a userspace CSPRNG + (https://mail.python.org/pipermail/python-ideas/2015-September/035886.html) + +.. [#draft] Initial draft concept that eventually became this PEP + (https://mail.python.org/pipermail/python-ideas/2015-September/036095.html) + +.. [#nocsprng] Safely generating random numbers + (http://sockpuppet.org/blog/2014/02/25/safely-generate-random-numbers/) + +.. [#ieeetopten] IEEE Spectrum 2015 Top Ten Programming Languages + (http://spectrum.ieee.org/computing/software/the-2015-top-ten-programming-languages) + +.. [#owasptopten] OWASP Top Ten Web Security Issues for 2013 + (https://www.owasp.org/index.php/OWASP_Top_Ten_Project#tab=OWASP_Top_10_for_2013) + +.. [#print] Stack Overflow answer for missing parentheses in call to print + (http://stackoverflow.com/questions/25445439/what-does-syntaxerror-missing-parentheses-in-call-to-print-mean-in-python/25445440#25445440) + +.. [#bcrypt] Bypassing bcrypt through an insecure data cache + (http://arstechnica.com/security/2015/09/once-seen-as-bulletproof-11-million-ashley-madison-passwords-already-cracked/) + +Copyright +========= + +This document has been placed in the public domain. + + +.. + Local Variables: + mode: indented-text + indent-tabs-mode: nil + sentence-end-double-space: t + fill-column: 70 + coding: utf-8 + End: -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Tue Sep 15 18:53:48 2015 From: python-checkins at python.org (berker.peksag) Date: Tue, 15 Sep 2015 16:53:48 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2325105=3A_Update_susp-ignored=2Ecsv_to_avoid_fal?= =?utf-8?q?se_positives?= Message-ID: <20150915164338.114907.50636@psf.io> https://hg.python.org/cpython/rev/efdbe17a9208 changeset: 98008:efdbe17a9208 parent: 98006:3d9164aecc6f parent: 98007:953a14984aec user: Berker Peksag date: Tue Sep 15 19:43:26 2015 +0300 summary: Issue #25105: Update susp-ignored.csv to avoid false positives files: Doc/tools/susp-ignored.csv | 5 +++++ 1 files changed, 5 insertions(+), 0 deletions(-) diff --git a/Doc/tools/susp-ignored.csv b/Doc/tools/susp-ignored.csv --- a/Doc/tools/susp-ignored.csv +++ b/Doc/tools/susp-ignored.csv @@ -290,3 +290,8 @@ library/stdtypes,,::,>>> m[::2].tolist() library/sys,1115,`,# ``wrapper`` creates a ``wrap(coro)`` coroutine: tutorial/venv,77,:c7b9645a6f35,"Python 3.4.3+ (3.4:c7b9645a6f35+, May 22 2015, 09:31:25)" +whatsnew/3.5,965,:root,'WARNING:root:warning\n' +whatsnew/3.5,965,:warning,'WARNING:root:warning\n' +whatsnew/3.5,1292,::,>>> addr6 = ipaddress.IPv6Address('::1') +whatsnew/3.5,1354,:root,ERROR:root:exception +whatsnew/3.5,1354,:exception,ERROR:root:exception -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 15 18:53:48 2015 From: python-checkins at python.org (berker.peksag) Date: Tue, 15 Sep 2015 16:53:48 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI1MTA1?= =?utf-8?q?=3A_Update_susp-ignored=2Ecsv_to_avoid_false_positives?= Message-ID: <20150915164338.66876.88172@psf.io> https://hg.python.org/cpython/rev/953a14984aec changeset: 98007:953a14984aec branch: 3.5 parent: 98004:9a80c687c28d user: Berker Peksag date: Tue Sep 15 19:43:04 2015 +0300 summary: Issue #25105: Update susp-ignored.csv to avoid false positives files: Doc/tools/susp-ignored.csv | 5 +++++ 1 files changed, 5 insertions(+), 0 deletions(-) diff --git a/Doc/tools/susp-ignored.csv b/Doc/tools/susp-ignored.csv --- a/Doc/tools/susp-ignored.csv +++ b/Doc/tools/susp-ignored.csv @@ -290,3 +290,8 @@ library/stdtypes,,::,>>> m[::2].tolist() library/sys,1115,`,# ``wrapper`` creates a ``wrap(coro)`` coroutine: tutorial/venv,77,:c7b9645a6f35,"Python 3.4.3+ (3.4:c7b9645a6f35+, May 22 2015, 09:31:25)" +whatsnew/3.5,965,:root,'WARNING:root:warning\n' +whatsnew/3.5,965,:warning,'WARNING:root:warning\n' +whatsnew/3.5,1292,::,>>> addr6 = ipaddress.IPv6Address('::1') +whatsnew/3.5,1354,:root,ERROR:root:exception +whatsnew/3.5,1354,:exception,ERROR:root:exception -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 15 19:00:07 2015 From: python-checkins at python.org (berker.peksag) Date: Tue, 15 Sep 2015 17:00:07 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzI1MTI3?= =?utf-8?q?=3A_Fix_typo_in_concurrent=2Efutures=2Erst?= Message-ID: <20150915170005.68873.20800@psf.io> https://hg.python.org/cpython/rev/8f94e695f56b changeset: 98009:8f94e695f56b branch: 3.4 parent: 98001:6a8aa246485e user: Berker Peksag date: Tue Sep 15 19:59:03 2015 +0300 summary: Issue #25127: Fix typo in concurrent.futures.rst Reported by Jakub Wilk. files: Doc/library/concurrent.futures.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/library/concurrent.futures.rst b/Doc/library/concurrent.futures.rst --- a/Doc/library/concurrent.futures.rst +++ b/Doc/library/concurrent.futures.rst @@ -75,7 +75,7 @@ e.submit(shutil.copy, 'src1.txt', 'dest1.txt') e.submit(shutil.copy, 'src2.txt', 'dest2.txt') e.submit(shutil.copy, 'src3.txt', 'dest3.txt') - e.submit(shutil.copy, 'src3.txt', 'dest4.txt') + e.submit(shutil.copy, 'src4.txt', 'dest4.txt') ThreadPoolExecutor -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 15 19:00:10 2015 From: python-checkins at python.org (berker.peksag) Date: Tue, 15 Sep 2015 17:00:10 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Issue_=2325127=3A_Fix_typo_in_concurrent=2Efutures=2Erst?= Message-ID: <20150915170005.2357.48618@psf.io> https://hg.python.org/cpython/rev/afe82e6f00df changeset: 98010:afe82e6f00df branch: 3.5 parent: 98007:953a14984aec parent: 98009:8f94e695f56b user: Berker Peksag date: Tue Sep 15 19:59:26 2015 +0300 summary: Issue #25127: Fix typo in concurrent.futures.rst Reported by Jakub Wilk. files: Doc/library/concurrent.futures.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/library/concurrent.futures.rst b/Doc/library/concurrent.futures.rst --- a/Doc/library/concurrent.futures.rst +++ b/Doc/library/concurrent.futures.rst @@ -84,7 +84,7 @@ e.submit(shutil.copy, 'src1.txt', 'dest1.txt') e.submit(shutil.copy, 'src2.txt', 'dest2.txt') e.submit(shutil.copy, 'src3.txt', 'dest3.txt') - e.submit(shutil.copy, 'src3.txt', 'dest4.txt') + e.submit(shutil.copy, 'src4.txt', 'dest4.txt') ThreadPoolExecutor -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 15 19:00:10 2015 From: python-checkins at python.org (berker.peksag) Date: Tue, 15 Sep 2015 17:00:10 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2325127=3A_Fix_typo_in_concurrent=2Efutures=2Erst?= Message-ID: <20150915170005.16638.74107@psf.io> https://hg.python.org/cpython/rev/23ad070a6a2d changeset: 98011:23ad070a6a2d parent: 98008:efdbe17a9208 parent: 98010:afe82e6f00df user: Berker Peksag date: Tue Sep 15 19:59:46 2015 +0300 summary: Issue #25127: Fix typo in concurrent.futures.rst Reported by Jakub Wilk. files: Doc/library/concurrent.futures.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/library/concurrent.futures.rst b/Doc/library/concurrent.futures.rst --- a/Doc/library/concurrent.futures.rst +++ b/Doc/library/concurrent.futures.rst @@ -84,7 +84,7 @@ e.submit(shutil.copy, 'src1.txt', 'dest1.txt') e.submit(shutil.copy, 'src2.txt', 'dest2.txt') e.submit(shutil.copy, 'src3.txt', 'dest3.txt') - e.submit(shutil.copy, 'src3.txt', 'dest4.txt') + e.submit(shutil.copy, 'src4.txt', 'dest4.txt') ThreadPoolExecutor -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 15 19:06:58 2015 From: python-checkins at python.org (berker.peksag) Date: Tue, 15 Sep 2015 17:06:58 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogd2hhdHNuZXcvMy41?= =?utf-8?q?=3A_Add_missing_word_=22class=22?= Message-ID: <20150915170657.16640.21115@psf.io> https://hg.python.org/cpython/rev/3af3cf5d7646 changeset: 98012:3af3cf5d7646 branch: 3.5 parent: 98010:afe82e6f00df user: Berker Peksag date: Tue Sep 15 20:06:28 2015 +0300 summary: whatsnew/3.5: Add missing word "class" files: Doc/whatsnew/3.5.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -1940,7 +1940,7 @@ unittest.mock ------------- -The :class:`~unittest.mock.Mock` has the following improvements: +The :class:`~unittest.mock.Mock` class has the following improvements: * Class constructor has a new *unsafe* parameter, which causes mock objects to raise :exc:`AttributeError` on attribute names starting -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 15 19:07:03 2015 From: python-checkins at python.org (berker.peksag) Date: Tue, 15 Sep 2015 17:07:03 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_whatsnew/3=2E5=3A_Add_missing_word_=22class=22?= Message-ID: <20150915170658.68879.6794@psf.io> https://hg.python.org/cpython/rev/dab3e7a7a863 changeset: 98013:dab3e7a7a863 parent: 98011:23ad070a6a2d parent: 98012:3af3cf5d7646 user: Berker Peksag date: Tue Sep 15 20:06:48 2015 +0300 summary: whatsnew/3.5: Add missing word "class" files: Doc/whatsnew/3.5.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -1940,7 +1940,7 @@ unittest.mock ------------- -The :class:`~unittest.mock.Mock` has the following improvements: +The :class:`~unittest.mock.Mock` class has the following improvements: * Class constructor has a new *unsafe* parameter, which causes mock objects to raise :exc:`AttributeError` on attribute names starting -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 15 22:11:44 2015 From: python-checkins at python.org (alexander.belopolsky) Date: Tue, 15 Sep 2015 20:11:44 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_PEP_495=3A_Added_a_fitting_ep?= =?utf-8?q?igtaph_to_the_=22Temporal_Arithmetic=22_section=2E?= Message-ID: <20150915201143.5937.87358@psf.io> https://hg.python.org/peps/rev/614eb50a8a18 changeset: 6065:614eb50a8a18 user: Alexander Belopolsky date: Tue Sep 15 16:11:37 2015 -0400 summary: PEP 495: Added a fitting epigtaph to the "Temporal Arithmetic" section. files: pep-0495.txt | 13 +++++++++++++ 1 files changed, 13 insertions(+), 0 deletions(-) diff --git a/pep-0495.txt b/pep-0495.txt --- a/pep-0495.txt +++ b/pep-0495.txt @@ -400,6 +400,19 @@ Temporal Arithmetic and Comparison Operators ============================================ +.. epigraph:: + + | In *mathematicks* he was greater + | Than Tycho Brahe, or Erra Pater: + | For he, by geometric scale, + | Could take the size of pots of ale; + | Resolve, by sines and tangents straight, + | If bread or butter wanted weight, + | And wisely tell what hour o' th' day + | The clock does strike by algebra. + + -- "Hudibras" by Samuel Butler + The value of the ``fold`` attribute will be ignored in all operations with naive datetime instances. As a consequence, naive ``datetime.datetime`` or ``datetime.time`` instances that differ only -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Tue Sep 15 22:40:13 2015 From: python-checkins at python.org (victor.stinner) Date: Tue, 15 Sep 2015 20:40:13 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2325122=3A_Fix_test?= =?utf-8?q?=5Feintr=2C_kill_child_process_on_error?= Message-ID: <20150915204012.68861.28921@psf.io> https://hg.python.org/cpython/rev/edbc35d8babb changeset: 98014:edbc35d8babb user: Victor Stinner date: Tue Sep 15 22:38:09 2015 +0200 summary: Issue #25122: Fix test_eintr, kill child process on error Some test_eintr hangs on waiting for the child process completion if an error occurred on the parent. Kill the child process on error (in the parent) to avoid the hang. files: Lib/test/eintrdata/eintr_tester.py | 32 +++++++++++++---- 1 files changed, 24 insertions(+), 8 deletions(-) diff --git a/Lib/test/eintrdata/eintr_tester.py b/Lib/test/eintrdata/eintr_tester.py --- a/Lib/test/eintrdata/eintr_tester.py +++ b/Lib/test/eintrdata/eintr_tester.py @@ -8,6 +8,7 @@ sub-second periodicity (contrarily to signal()). """ +import contextlib import faulthandler import io import os @@ -21,6 +22,16 @@ from test import support + at contextlib.contextmanager +def kill_on_error(proc): + """Context manager killing the subprocess if a Python exception is raised.""" + with proc: + try: + yield proc + except: + proc.kill() + raise + @unittest.skipUnless(hasattr(signal, "setitimer"), "requires setitimer()") class EINTRBaseTest(unittest.TestCase): @@ -38,7 +49,7 @@ def setUpClass(cls): cls.orig_handler = signal.signal(signal.SIGALRM, lambda *args: None) if hasattr(faulthandler, 'dump_traceback_later'): - # Most tests take less than 30 seconds, so 15 minutes should be + # Most tests take less than 30 seconds, so 5 minutes should be # enough. dump_traceback_later() is implemented with a thread, but # pthread_sigmask() is used to mask all signaled on this thread. faulthandler.dump_traceback_later(5 * 60, exit=True) @@ -120,7 +131,8 @@ ' os.write(wr, data)', )) - with self.subprocess(code, str(wr), pass_fds=[wr]) as proc: + proc = self.subprocess(code, str(wr), pass_fds=[wr]) + with kill_on_error(proc): os.close(wr) for data in datas: self.assertEqual(data, os.read(rd, len(data))) @@ -156,7 +168,8 @@ ' % (len(value), data_len))', )) - with self.subprocess(code, str(rd), pass_fds=[rd]) as proc: + proc = self.subprocess(code, str(rd), pass_fds=[rd]) + with kill_on_error(proc): os.close(rd) written = 0 while written < len(data): @@ -198,7 +211,7 @@ fd = wr.fileno() proc = self.subprocess(code, str(fd), pass_fds=[fd]) - with proc: + with kill_on_error(proc): wr.close() for data in datas: self.assertEqual(data, recv_func(rd, len(data))) @@ -248,7 +261,7 @@ fd = rd.fileno() proc = self.subprocess(code, str(fd), pass_fds=[fd]) - with proc: + with kill_on_error(proc): rd.close() written = 0 while written < len(data): @@ -288,7 +301,8 @@ ' time.sleep(sleep_time)', )) - with self.subprocess(code) as proc: + proc = self.subprocess(code) + with kill_on_error(proc): client_sock, _ = sock.accept() client_sock.close() self.assertEqual(proc.wait(), 0) @@ -315,7 +329,8 @@ do_open_close_reader, )) - with self.subprocess(code) as proc: + proc = self.subprocess(code) + with kill_on_error(proc): do_open_close_writer(filename) self.assertEqual(proc.wait(), 0) @@ -372,7 +387,8 @@ )) t0 = time.monotonic() - with self.subprocess(code) as proc: + proc = self.subprocess(code) + with kill_on_error(proc): # parent signal.sigwaitinfo([signum]) dt = time.monotonic() - t0 -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 15 22:43:34 2015 From: python-checkins at python.org (victor.stinner) Date: Tue, 15 Sep 2015 20:43:34 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge_3=2E5_=28asyncio_doc=29?= Message-ID: <20150915204333.5931.9829@psf.io> https://hg.python.org/cpython/rev/73f8fca329cb changeset: 98016:73f8fca329cb parent: 98014:edbc35d8babb parent: 98015:d371ce905395 user: Victor Stinner date: Tue Sep 15 22:43:05 2015 +0200 summary: Merge 3.5 (asyncio doc) files: Doc/library/asyncio-eventloop.rst | 8 ++++++-- 1 files changed, 6 insertions(+), 2 deletions(-) diff --git a/Doc/library/asyncio-eventloop.rst b/Doc/library/asyncio-eventloop.rst --- a/Doc/library/asyncio-eventloop.rst +++ b/Doc/library/asyncio-eventloop.rst @@ -275,7 +275,9 @@ to bind the socket to locally. The *local_host* and *local_port* are looked up using getaddrinfo(), similarly to *host* and *port*. - On Windows with :class:`ProactorEventLoop`, SSL/TLS is not supported. + .. versionchanged:: 3.5 + + On Windows with :class:`ProactorEventLoop`, SSL/TLS is now supported. .. seealso:: @@ -358,7 +360,9 @@ This method is a :ref:`coroutine `. - On Windows with :class:`ProactorEventLoop`, SSL/TLS is not supported. + .. versionchanged:: 3.5 + + On Windows with :class:`ProactorEventLoop`, SSL/TLS is now supported. .. seealso:: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 15 22:43:36 2015 From: python-checkins at python.org (victor.stinner) Date: Tue, 15 Sep 2015 20:43:36 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI1MTM0?= =?utf-8?q?=3A_Update_asyncio_doc_for_SSL_on_Windows?= Message-ID: <20150915204333.115020.40721@psf.io> https://hg.python.org/cpython/rev/d371ce905395 changeset: 98015:d371ce905395 branch: 3.5 parent: 98012:3af3cf5d7646 user: Victor Stinner date: Tue Sep 15 22:41:52 2015 +0200 summary: Issue #25134: Update asyncio doc for SSL on Windows ProactorEventLoop now supports SSL. files: Doc/library/asyncio-eventloop.rst | 8 ++++++-- 1 files changed, 6 insertions(+), 2 deletions(-) diff --git a/Doc/library/asyncio-eventloop.rst b/Doc/library/asyncio-eventloop.rst --- a/Doc/library/asyncio-eventloop.rst +++ b/Doc/library/asyncio-eventloop.rst @@ -275,7 +275,9 @@ to bind the socket to locally. The *local_host* and *local_port* are looked up using getaddrinfo(), similarly to *host* and *port*. - On Windows with :class:`ProactorEventLoop`, SSL/TLS is not supported. + .. versionchanged:: 3.5 + + On Windows with :class:`ProactorEventLoop`, SSL/TLS is now supported. .. seealso:: @@ -358,7 +360,9 @@ This method is a :ref:`coroutine `. - On Windows with :class:`ProactorEventLoop`, SSL/TLS is not supported. + .. versionchanged:: 3.5 + + On Windows with :class:`ProactorEventLoop`, SSL/TLS is now supported. .. seealso:: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 15 23:00:17 2015 From: python-checkins at python.org (victor.stinner) Date: Tue, 15 Sep 2015 21:00:17 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2325122=3A_test=5Fe?= =?utf-8?q?intr=3A_don=27t_redirect_stdout_to_stderr?= Message-ID: <20150915210013.27689.19271@psf.io> https://hg.python.org/cpython/rev/24dbca4e746c changeset: 98017:24dbca4e746c user: Victor Stinner date: Tue Sep 15 22:55:52 2015 +0200 summary: Issue #25122: test_eintr: don't redirect stdout to stderr sys.stderr is sometimes a StringIO. The redirection was just a hack to see eintr_tester.py output in red in the buildbot output. files: Lib/test/test_eintr.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Lib/test/test_eintr.py b/Lib/test/test_eintr.py --- a/Lib/test/test_eintr.py +++ b/Lib/test/test_eintr.py @@ -20,7 +20,7 @@ # FIXME: Issue #25122, always run in verbose mode to debug hang on FreeBSD if True: #support.verbose: args = [sys.executable, tester] - with subprocess.Popen(args, stdout=sys.stderr) as proc: + with subprocess.Popen(args) as proc: exitcode = proc.wait() self.assertEqual(exitcode, 0) else: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 16 00:01:56 2015 From: python-checkins at python.org (victor.stinner) Date: Tue, 15 Sep 2015 22:01:56 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2325122=3A_optimize?= =?utf-8?q?_test=5Feintr?= Message-ID: <20150915220155.68861.64474@psf.io> https://hg.python.org/cpython/rev/5388fa98f7a3 changeset: 98018:5388fa98f7a3 user: Victor Stinner date: Tue Sep 15 23:59:00 2015 +0200 summary: Issue #25122: optimize test_eintr Fix test_write(): copy support.PIPE_MAX_SIZE bytes, not support.PIPE_MAX_SIZE*3 bytes. files: Lib/test/eintrdata/eintr_tester.py | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Lib/test/eintrdata/eintr_tester.py b/Lib/test/eintrdata/eintr_tester.py --- a/Lib/test/eintrdata/eintr_tester.py +++ b/Lib/test/eintrdata/eintr_tester.py @@ -144,14 +144,14 @@ # rd closed explicitly by parent # we must write enough data for the write() to block - data = b"xyz" * support.PIPE_MAX_SIZE + data = b"x" * support.PIPE_MAX_SIZE code = '\n'.join(( 'import io, os, sys, time', '', 'rd = int(sys.argv[1])', 'sleep_time = %r' % self.sleep_time, - 'data = b"xyz" * %s' % support.PIPE_MAX_SIZE, + 'data = b"x" * %s' % support.PIPE_MAX_SIZE, 'data_len = len(data)', '', '# let the parent block on write()', -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 16 09:23:44 2015 From: python-checkins at python.org (victor.stinner) Date: Wed, 16 Sep 2015 07:23:44 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2325122=3A_add_debu?= =?utf-8?q?g_traces_to_test=5Feintr=2Etest=5Fopen=28=29?= Message-ID: <20150916072343.2345.71233@psf.io> https://hg.python.org/cpython/rev/30bc256f2346 changeset: 98019:30bc256f2346 user: Victor Stinner date: Wed Sep 16 09:23:28 2015 +0200 summary: Issue #25122: add debug traces to test_eintr.test_open() files: Lib/test/eintrdata/eintr_tester.py | 14 ++++++++++++++ 1 files changed, 14 insertions(+), 0 deletions(-) diff --git a/Lib/test/eintrdata/eintr_tester.py b/Lib/test/eintrdata/eintr_tester.py --- a/Lib/test/eintrdata/eintr_tester.py +++ b/Lib/test/eintrdata/eintr_tester.py @@ -326,12 +326,26 @@ '# let the parent block', 'time.sleep(sleep_time)', '', + 'print("try to open %a fifo for reading, pid %s, ppid %s"' + ' % (path, os.getpid(), os.getppid()),' + ' flush=True)', + '', do_open_close_reader, + '', + 'print("%a fifo opened for reading and closed, pid %s, ppid %s"' + ' % (path, os.getpid(), os.getppid()),' + ' flush=True)', )) proc = self.subprocess(code) with kill_on_error(proc): + print("try to open %a fifo for writing, pid %s" + % (filename, os.getpid()), + flush=True) do_open_close_writer(filename) + print("%a fifo opened for writing and closed, pid %s" + % (filename, os.getpid()), + flush=True) self.assertEqual(proc.wait(), 0) -- Repository URL: https://hg.python.org/cpython From solipsis at pitrou.net Wed Sep 16 10:50:43 2015 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Wed, 16 Sep 2015 08:50:43 +0000 Subject: [Python-checkins] Daily reference leaks (5388fa98f7a3): sum=17869 Message-ID: <20150916085043.30610.51061@psf.io> results for 5388fa98f7a3 on branch "default" -------------------------------------------- test_capi leaked [1598, 1598, 1598] references, sum=4794 test_capi leaked [387, 389, 389] memory blocks, sum=1165 test_collections leaked [-6, 0, 0] references, sum=-6 test_collections leaked [-3, 1, 0] memory blocks, sum=-2 test_functools leaked [0, 2, 2] memory blocks, sum=4 test_threading leaked [3196, 3196, 3196] references, sum=9588 test_threading leaked [774, 776, 776] memory blocks, sum=2326 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/psf-users/antoine/refleaks/reflogQl4sbm', '--timeout', '7200'] From lp_benchmark_robot at intel.com Wed Sep 16 14:58:18 2015 From: lp_benchmark_robot at intel.com (lp_benchmark_robot) Date: Wed, 16 Sep 2015 12:58:18 +0000 Subject: [Python-checkins] Benchmark Results for Python 2.7 2015-09-16 Message-ID: No new revisions. Here are the previous results: Results for project python_2.7-nightly, build date 2015-09-16 03:42:44 commit: 21d6b2752fe8db0428c92d928678b15d76a9a27b revision date: 2015-09-14 22:19:47 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v2.7.10, with hash 15c95b7d81dcf821daade360741e00714667653f from 2015-05-23 16:02:14+00:00 ------------------------------------------------------------------------------------------ benchmark relative change since change since current rev with std_dev* last run v2.7.10 regrtest PGO ------------------------------------------------------------------------------------------ :-) django_v2 0.29878% -1.76221% 3.17342% 11.27247% :-) pybench 0.16246% -0.03220% 6.73896% 6.53390% :-| regex_v8 0.59862% 0.02829% -1.15354% 7.50447% :-) nbody 0.28427% 0.00747% 9.04932% 2.89917% :-) json_dump_v2 0.25198% 0.48636% 4.42795% 13.96206% :-| normal_startup 1.83289% -0.42891% -1.66451% 3.47175% :-) ssbench 0.15324% 0.85916% 2.41400% 2.04298% ------------------------------------------------------------------------------------------ Note: Benchmark results for ssbench are measured in requests/second while all other are measured in seconds. * Relative Standard Deviation (Standard Deviation/Average) Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. From lp_benchmark_robot at intel.com Wed Sep 16 14:58:20 2015 From: lp_benchmark_robot at intel.com (lp_benchmark_robot) Date: Wed, 16 Sep 2015 12:58:20 +0000 Subject: [Python-checkins] Benchmark Results for Python Default 2015-09-16 Message-ID: Results for project python_default-nightly, build date 2015-09-16 03:02:04 commit: 5388fa98f7a34b9137a4466f4bb21419354ba6f6 revision date: 2015-09-15 21:59:00 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v3.4.3, with hash b4cbecbc0781e89a309d03b60a1f75f8499250e6 from 2015-02-25 12:15:33+00:00 ------------------------------------------------------------------------------------------ benchmark relative change since change since current rev with std_dev* last run v3.4.3 regrtest PGO ------------------------------------------------------------------------------------------ :-) django_v2 0.19770% 1.04264% 9.47935% 16.42263% :-( pybench 0.17945% -0.13763% -2.48591% 9.32358% :-( regex_v8 2.80756% -0.06924% -3.52991% 4.94887% :-| nbody 0.14407% 0.85896% -0.93791% 9.24179% :-| json_dump_v2 0.24302% 1.02559% -1.14855% 11.91771% :-| normal_startup 0.84331% -0.22650% -0.04007% 5.63909% ------------------------------------------------------------------------------------------ Note: Benchmark results are measured in seconds. * Relative Standard Deviation (Standard Deviation/Average) Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. From python-checkins at python.org Wed Sep 16 16:31:49 2015 From: python-checkins at python.org (nick.coghlan) Date: Wed, 16 Sep 2015 14:31:49 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_PEP_504=3A_expand_ensure=5Fre?= =?utf-8?q?peatable_docstring?= Message-ID: <20150916143148.16626.20187@psf.io> https://hg.python.org/peps/rev/8912fd9e3ece changeset: 6067:8912fd9e3ece user: Nick Coghlan date: Thu Sep 17 00:31:39 2015 +1000 summary: PEP 504: expand ensure_repeatable docstring files: pep-0504.txt | 13 ++++++++++++- 1 files changed, 12 insertions(+), 1 deletions(-) diff --git a/pep-0504.txt b/pep-0504.txt --- a/pep-0504.txt +++ b/pep-0504.txt @@ -61,7 +61,18 @@ additional level of indirection):: def ensure_repeatable(): - """Switch to using random.Random() for the module level APIs""" + """Switch to using random.Random() for the module level APIs + + This switches the default RNG instance from the crytographically + secure random.SystemRandom() to the deterministic random.Random(), + enabling the seed(), getstate() and setstate() operations. This means + a particular random scenario can be replayed later by providing the + same seed value or restoring a previously saved state. + + NOTE: Libraries implementing security sensitive operations should + always explicitly use random.SystemRandom() or os.urandom in order to + correctly handle applications that call this function. + """ if not isinstance(_inst, Random): _inst = random.Random() -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Wed Sep 16 16:31:49 2015 From: python-checkins at python.org (nick.coghlan) Date: Wed, 16 Sep 2015 14:31:49 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_Major_simplification_of_PEP_5?= =?utf-8?q?04?= Message-ID: <20150916143147.2359.35766@psf.io> https://hg.python.org/peps/rev/ccbc2225361f changeset: 6066:ccbc2225361f user: Nick Coghlan date: Thu Sep 17 00:21:45 2015 +1000 summary: Major simplification of PEP 504 - drop the submodule idea - call random.ensure_repeatable() to opt in to the PRNG - seed(), getstate(), setstate() all call ensure_repeatable() files: pep-0504.txt | 241 ++++++++++++++++++++++---------------- 1 files changed, 137 insertions(+), 104 deletions(-) diff --git a/pep-0504.txt b/pep-0504.txt --- a/pep-0504.txt +++ b/pep-0504.txt @@ -24,22 +24,19 @@ aren't aware that they're doing security sensitive work use the default module level APIs, and thus expose their users to unnecessary risks. -This isn't an acute problem, but it is a chronic one, and if documentation and -developer education were going to solve it, they would have done so by now. +This isn't an acute problem, but it is a chronic one, and the often long +delays between the introduction of security flaws and their exploitation means +that it is difficult for developers to naturally learn from experience. In order to provide an eventually pervasive solution to the problem, this PEP proposes that Python switch to using the system random number generator by default in Python 3.6, and require developers to opt-in to using the -deterministic random number generator. +deterministic random number generator process wide either by using a new +``random.ensure_repeatable()`` API, or by explicitly creating their own +``random.Random()`` instance. -To minimise the compatibility break, calling any of the following module level -functions will count as opting in to using the deterministic random number -generator for all future calls to module level functions in the random -module in the same process: - -* ``random.seed`` -* ``random.getstate`` -* ``random.setstate`` +To minimise the impact on existing code, module level APIs that require +determinism will implicitly switch to the deterministic PRNG. Proposal ======== @@ -48,94 +45,130 @@ ``random`` module for security sensitive applications. This PEP proposes to change that admonition in Python 3.6+ to instead be that it is not correct to use the module level functions in the ``random`` module for security sensitive -applications if ``random.seed``, ``random.getstate``, or ``random.setstate`` -are ever called in that process. +applications if ``random.ensure_repeatable()`` is ever called (directly or +indirectly) in that process. -This PEP further proposes to make it easier to explicitly opt in to using -either the system random number generator or Python's deterministic PRNG by -converting the random module to a package that exposes the same top-level API, -and offering two new subpackages: +To achieve this, rather than being bound methods of a ``random.Random`` +instance as they are today, the module level callables in ``random`` would +change to be functions that delegate to the corresponding method of the +existing ``random._inst`` module attribute. -* ``random.system`` -* ``random.seedable`` +By default, this attribute will be bound to a ``random.SystemRandom`` instance. -The ``random.system`` submodule would provide the following bound methods of a -module global ``random.SystemRandom`` instance as module attributes: -``betavariate``, ``choice``, ``expovariate``, ``gammavariate``, ``gauss``, ``getrandbits``, ``lognormvariate``, ``normalvariate``, ``paretovariate``, -``randint``, ``random``, ``randrange``, ``sample``, ``shuffle``, -``triangular``, ``uniform``, ``vonmisesvariate``, ``weibullvariate`` +A new ``random.ensure_repeatable()`` API will then rebind the ``random._inst`` +attribute to a ``system.Random`` instance, restoring the same module level +API behaviour as existed in previous Python versions (aside from the +additional level of indirection):: -The ``random.seedable`` submodule would provide the same operations, but as -methods of a ``random.Random`` instance. In addition, it would provide the -following additional methods which are only meaningful when using a -deterministic random number generator: ``seed``, ``getstate``, ``setstate``. + def ensure_repeatable(): + """Switch to using random.Random() for the module level APIs""" + if not isinstance(_inst, Random): + _inst = random.Random() -Rather than being bound methods of a ``random.Random`` instance as they are -today, the module level callables in ``random`` itself would change to be -functions that, by default, delegated to the ``random.SystemRandom`` instance -in ``random.system``. +To minimise the impact on existing code, calling any of the following module +level functions will implicitly call ``random.ensure_repeatable()``: -Calling any one of ``random.seed``, ``random.getstate``, or ``random.setstate`` -would change the delegation to instead refer to the ``random.Random`` instance -in ``random.seedable``. +* ``random.seed`` +* ``random.getstate`` +* ``random.setstate`` + +There are no changes proposed to the ``random.Random`` or +``random.SystemRandom`` class APIs - applications that explicitly instantiate +their own random number generators will be entirely unaffected by this +proposal. Warning on implicit opt-in -------------------------- -In Python 3.6, implicitly opting in to the use of the seedable PRNG will emit a -deprecation warning. This warning will suggest explicitly opting in to either -the system RNG or the seedable PRNG. Possible wording: +In Python 3.6, implicitly opting in to the use of the deterministic PRNG will +emit a deprecation warning using the following check:: - "DeprecationWarning: Implicitly switching to the seedable PRNG. Consider - importing from random.system or random.seedable as appropriate" + if not isinstance(_inst, Random): + warnings.warn(DeprecationWarning, + "Implicitly ensuring repeatability. " + "See help(random.ensure_repeatable) for details") + ensure_repeatable() -Whatever precise wording is chosen should have an answer added to Stack -Overflow as was done for the custom error message that was added for missing -parentheses in a call to print [#print]_. +The specific wording of the warning should have a suitable answer added to +Stack Overflow as was done for the custom error message that was added for +missing parentheses in a call to print [#print]_. In the first Python 3 release after Python 2.7 switches to security fix only mode, the deprecation warning will be upgraded to a RuntimeWarning so it is visible by default. -This PEP does *not* propose removing the ability to seed the default RNG used -process wide - it's not a good idea relative to the alternative of explicitly -importing from the appropriate submodule (hence the eventually -visible-by-default warning), but it's also a concern that can be more -readily addressed on a project-by-project basis. +This PEP does *not* propose ever removing the ability to ensure the default RNG +used process wide is a deterministic PRNG that will produce the same series of +outputs given a specific seed. That capability is widely used in modelling +and simulation scenarios, and requiring that ``ensure_repeatable()`` be called +either directly or indirectly is a sufficient enhancement to address the cases +where the module level random API is used for security sensitive tasks in web +applications without due consideration for the potential security implications +of using a deterministic PRNG. + +Performance impact +------------------ + +Due to the large performance difference between ``random.Random`` and +``random.SystemRandom``, applications ported to Python 3.6 will encounter a +significant performance regression in cases where: + +* the application is using the module level random API +* cryptographic quality randomness isn't needed +* the application doesn't already implicitly opt back in to the deterministic + PRNG by calling ``random.seed``, ``random.getstate``, or ``random.setstate`` +* the application isn't updated to explicitly call ``random.ensure_repeatable`` + +This would be noted in the Porting section of the Python 3.6 What's New guide, +with the recommendation to include the following code in the ``__main__`` +module of affected applications:: + + if hasattr(random, "ensure_repeatable"): + random.ensure_repeatable() + +Applications that do need cryptographic quality randomness should be using the +system random number generator regardless of speed considerations, so in those +cases the change proposed in this PEP will fix a previously latent security +defect. Documentation changes --------------------- The ``random`` module documentation would be updated to move the documentation of the ``seed``, ``getstate`` and ``setstate`` interfaces later in the module, -along with the associated security warning. +along with the documentation of the new ``ensure_repeatable`` function and the +associated security warning. -The docs would gain a discussion of the respective use cases for the seedable -PRNG (games, modelling & simulation, software testing) and the system RNG -(cryptography, security token generation). +That section of the module documentation would also gain a discussion of the +respective use cases for the deterministic PRNG enabled by +``ensure_repeatable`` (games, modelling & simulation, software testing) and the +system RNG that is used by default (cryptography, security token generation). +This discussion will also recommend the use of third party security libraries +for the latter task. Rationale ========= Writing secure software under deadline and budget pressures is a hard problem. -This is reflected in ongoing problems with data breaches involving personally +This is reflected in regular notifications of data breaches involving personally identifiable information [#breaches]_, as well as with failures to take security considerations into account when new systems, like motor vehicles -[#uconnect]_, are connected to the internet. Compounding the issue is the fact -that a lot of the programming advice readily available on the internet [#search] -simply doesn't take the mathemetical arcana of computer security into account, -and the fact that defenders have to cover *all* of their potential -vulnerabilites, as a single mistake can make it possible to subvert other -defences [#bcrypt]_. +[#uconnect]_, are connected to the internet. It's also the case that a lot of +the programming advice readily available on the internet [#search] simply +doesn't take the mathemetical arcana of computer security into account. +Compounding these issues is the fact that defenders have to cover *all* of +their potential vulnerabilites, as a single mistake can make it possible to +subvert other defences [#bcrypt]_. One of the factors that contributes to making this last aspect particularly difficult is APIs where using them inappropriately creates a *silent* security failure - one where the only way to find out that what you're doing is incorrect is for someone reviewing your code to say "that's a potential security problem", or for a system you're responsible for to be compromised -through such an oversight (and your intrusion detection and auditing mechanisms -are good enough for you to be able to figure out after the event how the -compromise took place). +through such an oversight (and you're not only still responsible for that +system when it is compromised, but your intrusion detection and auditing +mechanisms are good enough for you to be able to figure out after the event +how the compromise took place). This kind of situation is a significant contributor to "security fatigue", where developers (often rightly [#owasptopten]_) feel that security engineers @@ -151,8 +184,8 @@ Discussion ========== -Why "seedable" over "deterministic"? ------------------------------------- +Why "ensure_repeatable" over "ensure_deterministic"? +---------------------------------------------------- This is a case where the meaning of a word as specialist jargon conflicts with the typical meaning of the word, even though it's *technically* the same. @@ -163,16 +196,17 @@ The problem is that "deterministic" on its own doesn't convey those qualifiers, so it's likely to instead be interpreted as "predictable" or "not random" by -folks that aren't familiar with the technical meaning. +folks that are familiar with the conventional meaning, but aren't familiar with +the additional qualifiers on the technical meaning. -The other problem with "deterministic" as a description for the traditional RNG -is that it doesn't tell you what you can *do* with the traditional RNG that you -can't do with the system one. +A second problem with "deterministic" as a description for the traditional RNG +is that it doesn't really tell you what you can *do* with the traditional RNG +that you can't do with the system one. -"seedable" aims to address both those problems, as it doesn't have a misleading -common meaning, and it's a word form that means "you can seed this", which then -leads naturally into an exploration of what it means to "seed" a random number -generator. +"ensure_repeatable" aims to address both of those problems, as its common +meaning accurately describes the main reason for preferring the deterministic +PRNG over the system RNG: ensuring you can repeat the same series of outputs +by providing the same seed value, or by restoring a previously saved PRNG state. Only changing the default for Python 3.6+ ----------------------------------------- @@ -184,9 +218,9 @@ The difference in this case is one of degree - the additional benefits from rolling out this particular change a couple of years earlier than will -otherwise be the case aren't sufficient to justify the additional effort and -stability risks involved in making such an intrusive change in a maintenance -release. +otherwise be the case aren't sufficient to justify either the additional effort +or the stability risks involved in making such an intrusive change in a +maintenance release. Keeping the module level functions ---------------------------------- @@ -198,18 +232,24 @@ most of the public API can continue to be used not only without modification, but without generating any new warnings. -Implicitly opting in to the deterministic RNG ---------------------------------------------- +Warning when implicitly opting in to the deterministic RNG +---------------------------------------------------------- -Python is widely used for modelling and simulation purposes, and in many cases, -these software models won't have a dedicated maintenance team tasked with -ensuing they keep working on the latest versions of Python. +It's necessary to implicitly opt in to the deterministic PRNG as Python is +widely used for modelling and simulation purposes where this is the right +thing to do, and in many cases, these software models won't have a dedicated +maintenance team tasked with ensuring they keep working on the latest versions +of Python. + +Unfortunately, explicitly calling ``random.seed`` with data from ``os.urandom`` +is also a mistake that appears in a number of the flawed "how to generate a +security token in Python" guides readily available online. Using first DeprecationWarning, and then eventually a RuntimeWarning, to -advise against implicitly switching to the deterministic PRNG, preserves -compatibility with this existing software, while still nudging future users -that need a deterministic generator towards importing ``random.seedable`` -explicitly. +advise against implicitly switching to the deterministic PRNG aims to +nudge future users that need a cryptographically secure RNG away from +calling ``random.seed()`` and those that genuinely need a deterministic +generator towards explicitily calling ``random.ensure_repeatable()``. Avoiding the introduction of a userspace CSPRNG ----------------------------------------------- @@ -224,23 +264,9 @@ where the random number generation may not even be on a critical performance path. -What about the performance impact? ----------------------------------- - -Rather than introducing a userspace CSPRNG, this PEP instead proposes that we -accept the performance regression in cases where: - -* an application is using the module level random API -* cryptographic quality randomness isn't needed -* the application doesn't already implicitly opt back in to the deterministic - PRNG by calling ``random.seed``, ``random.getstate``, or ``random.setstate`` -* the application isn't updated to explicitly import from ``random.seedable`` - rather than ``random`` - -Applications that need cryptographic quality randomness should be using the -system random number generator regardless of speed considerations, while other -applications where speed is a more important consideration are better off with -the current PRNG implementation than they would be with a new CSPRNG. +Applications that do need cryptographic quality randomness should be using the +system random number generator regardless of speed considerations, so in those +cases. Isn't the deterministic PRNG "secure enough"? --------------------------------------------- @@ -252,6 +278,11 @@ to use weaknesses in that subsystem to facilitate a practical attack on password recovery tokens in popular PHP web applications. +However, one of the rules of secure software development is that "attacks only +get better, never worse", so it may be that by the time Python 3.6 is released +we will actually see a practical attack on Python's deterministic PRNG publicly +documented. + Security fatigue in the Python ecosystem ---------------------------------------- @@ -264,9 +295,9 @@ Python ecosystem, which is understandably frustrating for Pythonistas using Python in other contexts where these issues aren't of as great a concern. -This consideration is one of the primary factors driving the backwards -compatibility improvements in this proposal relative to the initial draft -concept posted to python-ideas [#draft]_. +This consideration is one of the primary factors driving the substantial +backwards compatibility improvements in this proposal relative to the initial +draft concept posted to python-ideas [#draft]_. Acknowledgements ================ @@ -284,6 +315,8 @@ experts that suggested the introduction of a userspace CSPRNG would mean additional complexity for insufficient gain relative to just using the system RNG directly +* Paul Moore for eloquently making the case for the current level of security + fatigue in the Python ecosystem References ========== -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Wed Sep 16 18:18:56 2015 From: python-checkins at python.org (yury.selivanov) Date: Wed, 16 Sep 2015 16:18:56 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogd2hhdHNuZXcvMy41?= =?utf-8?q?=3A_Reword_bytes*=2Ehex_message?= Message-ID: <20150916161856.2353.41794@psf.io> https://hg.python.org/cpython/rev/ea16232f0d50 changeset: 98020:ea16232f0d50 branch: 3.5 parent: 98015:d371ce905395 user: Yury Selivanov date: Wed Sep 16 12:18:29 2015 -0400 summary: whatsnew/3.5: Reword bytes*.hex message files: Doc/whatsnew/3.5.rst | 5 ++--- 1 files changed, 2 insertions(+), 3 deletions(-) diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -76,9 +76,8 @@ * ``bytes % args``, ``bytearray % args``: :ref:`PEP 461 ` -- Adding ``%`` formatting to bytes and bytearray. -* ``b'\xf0\x9f\x90\x8d'.hex()``, ``bytearray(b'\xf0\x9f\x90\x8d').hex()``, - ``memoryview(b'\xf0\x9f\x90\x8d').hex()``: :issue:`9951` - A ``hex`` method - has been added to bytes, bytearray, and memoryview. +* New :meth:`bytes.hex`, :meth:`bytearray.hex` and :meth:`memoryview.hex` + methods. (Contributed by Arnon Yaari in :issue:`9951`.) * :class:`memoryview` now supports tuple indexing (including multi-dimensional). (Contributed by Antoine Pitrou in :issue:`23632`.) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 16 18:18:56 2015 From: python-checkins at python.org (yury.selivanov) Date: Wed, 16 Sep 2015 16:18:56 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogTWVyZ2UgMy41?= Message-ID: <20150916161856.5939.14833@psf.io> https://hg.python.org/cpython/rev/e33b4c18af59 changeset: 98021:e33b4c18af59 parent: 98019:30bc256f2346 parent: 98020:ea16232f0d50 user: Yury Selivanov date: Wed Sep 16 12:18:55 2015 -0400 summary: Merge 3.5 files: Doc/whatsnew/3.5.rst | 5 ++--- 1 files changed, 2 insertions(+), 3 deletions(-) diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -76,9 +76,8 @@ * ``bytes % args``, ``bytearray % args``: :ref:`PEP 461 ` -- Adding ``%`` formatting to bytes and bytearray. -* ``b'\xf0\x9f\x90\x8d'.hex()``, ``bytearray(b'\xf0\x9f\x90\x8d').hex()``, - ``memoryview(b'\xf0\x9f\x90\x8d').hex()``: :issue:`9951` - A ``hex`` method - has been added to bytes, bytearray, and memoryview. +* New :meth:`bytes.hex`, :meth:`bytearray.hex` and :meth:`memoryview.hex` + methods. (Contributed by Arnon Yaari in :issue:`9951`.) * :class:`memoryview` now supports tuple indexing (including multi-dimensional). (Contributed by Antoine Pitrou in :issue:`23632`.) -- Repository URL: https://hg.python.org/cpython From solipsis at pitrou.net Thu Sep 17 10:41:47 2015 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Thu, 17 Sep 2015 08:41:47 +0000 Subject: [Python-checkins] Daily reference leaks (e33b4c18af59): sum=17877 Message-ID: <20150917084147.66854.85842@psf.io> results for e33b4c18af59 on branch "default" -------------------------------------------- test_capi leaked [1598, 1598, 1598] references, sum=4794 test_capi leaked [387, 389, 389] memory blocks, sum=1165 test_functools leaked [0, 2, 2] memory blocks, sum=4 test_threading leaked [3196, 3196, 3196] references, sum=9588 test_threading leaked [774, 776, 776] memory blocks, sum=2326 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/psf-users/antoine/refleaks/reflogUQFS9m', '--timeout', '7200'] From lp_benchmark_robot at intel.com Thu Sep 17 15:58:10 2015 From: lp_benchmark_robot at intel.com (lp_benchmark_robot) Date: Thu, 17 Sep 2015 13:58:10 +0000 Subject: [Python-checkins] Benchmark Results for Python 2.7 2015-09-17 Message-ID: No new revisions. Here are the previous results: Results for project python_2.7-nightly, build date 2015-09-17 03:43:17 commit: 21d6b2752fe8db0428c92d928678b15d76a9a27b revision date: 2015-09-14 22:19:47 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v2.7.10, with hash 15c95b7d81dcf821daade360741e00714667653f from 2015-05-23 16:02:14+00:00 ------------------------------------------------------------------------------------------ benchmark relative change since change since current rev with std_dev* last run v2.7.10 regrtest PGO ------------------------------------------------------------------------------------------ :-) django_v2 0.29878% -1.76221% 3.17342% 11.27247% :-) pybench 0.16246% -0.03220% 6.73896% 6.53390% :-| regex_v8 0.59862% 0.02829% -1.15354% 7.50447% :-) nbody 0.28427% 0.00747% 9.04932% 2.89917% :-) json_dump_v2 0.25198% 0.48636% 4.42795% 13.96206% :-| normal_startup 1.83289% -0.42891% -1.66451% 3.47175% :-) ssbench 0.15324% 0.85916% 2.41400% 2.04298% ------------------------------------------------------------------------------------------ Note: Benchmark results for ssbench are measured in requests/second while all other are measured in seconds. * Relative Standard Deviation (Standard Deviation/Average) Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. From lp_benchmark_robot at intel.com Thu Sep 17 15:58:11 2015 From: lp_benchmark_robot at intel.com (lp_benchmark_robot) Date: Thu, 17 Sep 2015 13:58:11 +0000 Subject: [Python-checkins] Benchmark Results for Python Default 2015-09-17 Message-ID: Results for project python_default-nightly, build date 2015-09-17 03:02:37 commit: e33b4c18af59f938225141c1515602c0a644e4f2 revision date: 2015-09-16 16:18:55 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v3.4.3, with hash b4cbecbc0781e89a309d03b60a1f75f8499250e6 from 2015-02-25 12:15:33+00:00 ------------------------------------------------------------------------------------------ benchmark relative change since change since current rev with std_dev* last run v3.4.3 regrtest PGO ------------------------------------------------------------------------------------------ :-) django_v2 0.20081% -0.61311% 8.92436% 16.28674% :-( pybench 0.13539% 0.02229% -2.46307% 9.43340% :-( regex_v8 2.87636% -0.30130% -3.84184% 7.62587% :-| nbody 0.09229% -0.02505% -0.96319% 10.31006% :-| json_dump_v2 0.20164% 0.02256% -1.12573% 11.84351% :-| normal_startup 0.63661% 0.39540% 0.45931% 5.22018% ------------------------------------------------------------------------------------------ Note: Benchmark results are measured in seconds. * Relative Standard Deviation (Standard Deviation/Average) Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. From python-checkins at python.org Fri Sep 18 06:49:33 2015 From: python-checkins at python.org (ethan.furman) Date: Fri, 18 Sep 2015 04:49:33 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Close_issue24840=3A_Enum?= =?utf-8?q?=2E=5Fvalue=5F_is_queried_for_bool=28=29=3B_original_patch_by_M?= =?utf-8?q?ike?= Message-ID: <20150918044932.11702.57615@psf.io> https://hg.python.org/cpython/rev/87a9dff5106c changeset: 98022:87a9dff5106c user: Ethan Furman date: Thu Sep 17 21:49:12 2015 -0700 summary: Close issue24840: Enum._value_ is queried for bool(); original patch by Mike Lundy files: Lib/enum.py | 3 +++ Lib/test/test_enum.py | 7 +++++++ Misc/ACKS | 1 + 3 files changed, 11 insertions(+), 0 deletions(-) diff --git a/Lib/enum.py b/Lib/enum.py --- a/Lib/enum.py +++ b/Lib/enum.py @@ -476,6 +476,9 @@ def __str__(self): return "%s.%s" % (self.__class__.__name__, self._name_) + def __bool__(self): + return bool(self._value_) + def __dir__(self): added_behavior = [ m diff --git a/Lib/test/test_enum.py b/Lib/test/test_enum.py --- a/Lib/test/test_enum.py +++ b/Lib/test/test_enum.py @@ -270,6 +270,13 @@ class Wrong(Enum): _any_name_ = 9 + def test_bool(self): + class Logic(Enum): + true = True + false = False + self.assertTrue(Logic.true) + self.assertFalse(Logic.false) + def test_contains(self): Season = self.Season self.assertIn(Season.AUTUMN, Season) diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -877,6 +877,7 @@ Lukas Lueg Loren Luke Fredrik Lundh +Mike Lundy Zhongyue Luo Mark Lutz Taras Lyapun -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 18 07:04:18 2015 From: python-checkins at python.org (ethan.furman) Date: Fri, 18 Sep 2015 05:04:18 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Close_issue25147=3A_use_C_?= =?utf-8?q?implementation_of_OrderedDict?= Message-ID: <20150918050418.31189.37185@psf.io> https://hg.python.org/cpython/rev/b77916d2d7cc changeset: 98023:b77916d2d7cc user: Ethan Furman date: Thu Sep 17 22:03:52 2015 -0700 summary: Close issue25147: use C implementation of OrderedDict files: Lib/enum.py | 7 ++++++- 1 files changed, 6 insertions(+), 1 deletions(-) diff --git a/Lib/enum.py b/Lib/enum.py --- a/Lib/enum.py +++ b/Lib/enum.py @@ -1,7 +1,12 @@ import sys -from collections import OrderedDict from types import MappingProxyType, DynamicClassAttribute +try: + from _collections import OrderedDict +except ImportError: + from collections import OrderedDict + + __all__ = ['Enum', 'IntEnum', 'unique'] -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 18 07:22:50 2015 From: python-checkins at python.org (ethan.furman) Date: Fri, 18 Sep 2015 05:22:50 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Issue24756=3A_clarify_usage_of_run=5Fdocstring=5Fexamples=28?= =?utf-8?q?=29?= Message-ID: <20150918052249.82656.69736@psf.io> https://hg.python.org/cpython/rev/767cc99020d2 changeset: 98026:767cc99020d2 branch: 3.5 parent: 98020:ea16232f0d50 parent: 98025:ce595a047bd9 user: Ethan Furman date: Thu Sep 17 22:21:36 2015 -0700 summary: Issue24756: clarify usage of run_docstring_examples() files: Doc/library/doctest.rst | 30 ++++++++++++++++++++++------ 1 files changed, 23 insertions(+), 7 deletions(-) diff --git a/Doc/library/doctest.rst b/Doc/library/doctest.rst --- a/Doc/library/doctest.rst +++ b/Doc/library/doctest.rst @@ -914,15 +914,10 @@ above, except that *globs* defaults to ``m.__dict__``. -There's also a function to run the doctests associated with a single object. -This function is provided for backward compatibility. There are no plans to -deprecate it, but it's rarely useful: - - .. function:: run_docstring_examples(f, globs, verbose=False, name="NoName", compileflags=None, optionflags=0) - Test examples associated with object *f*; for example, *f* may be a module, - function, or class object. + Test examples associated with object *f*; for example, *f* may be a string, + a module, a function, or a class object. A shallow copy of dictionary argument *globs* is used for the execution context. @@ -1815,6 +1810,27 @@ * Define a ``__test__`` dictionary mapping from regression test topics to docstrings containing test cases. +When you have placed your tests in a module, the module can itself be the test +runner. When a test fails, you can arrange for your test runner to re-run only +the failing doctest while you debug the problem. Here is a minimal example of +such a test runner:: + + if __name__ == '__main__': + import doctest + flags = doctest.REPORT_NDIFF|doctest.FAIL_FAST + if len(sys.argv) > 1: + name = sys.argv[1] + if name in globals(): + obj = globals()[name] + else: + obj = __test__[name] + doctest.run_docstring_examples(obj, globals(), name=name, + optionflags=flags) + else: + fail, total = doctest.testmod(optionflags=flags) + print("{} failures out of {} tests".format(fail, total)) + + .. rubric:: Footnotes .. [#] Examples containing both expected output and an exception are not supported. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 18 07:22:49 2015 From: python-checkins at python.org (ethan.furman) Date: Fri, 18 Sep 2015 05:22:49 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUyNDc1Njog?= =?utf-8?q?clarify_usage_of_run=5Fdocstring=5Fexamples=28=29?= Message-ID: <20150918052249.31205.70895@psf.io> https://hg.python.org/cpython/rev/ce595a047bd9 changeset: 98025:ce595a047bd9 branch: 3.4 parent: 98009:8f94e695f56b user: Ethan Furman date: Thu Sep 17 22:20:41 2015 -0700 summary: Issue24756: clarify usage of run_docstring_examples() files: Doc/library/doctest.rst | 30 ++++++++++++++++++++++------ 1 files changed, 23 insertions(+), 7 deletions(-) diff --git a/Doc/library/doctest.rst b/Doc/library/doctest.rst --- a/Doc/library/doctest.rst +++ b/Doc/library/doctest.rst @@ -914,15 +914,10 @@ above, except that *globs* defaults to ``m.__dict__``. -There's also a function to run the doctests associated with a single object. -This function is provided for backward compatibility. There are no plans to -deprecate it, but it's rarely useful: - - .. function:: run_docstring_examples(f, globs, verbose=False, name="NoName", compileflags=None, optionflags=0) - Test examples associated with object *f*; for example, *f* may be a module, - function, or class object. + Test examples associated with object *f*; for example, *f* may be a string, + a module, a function, or a class object. A shallow copy of dictionary argument *globs* is used for the execution context. @@ -1821,6 +1816,27 @@ * Define a ``__test__`` dictionary mapping from regression test topics to docstrings containing test cases. +When you have placed your tests in a module, the module can itself be the test +runner. When a test fails, you can arrange for your test runner to re-run only +the failing doctest while you debug the problem. Here is a minimal example of +such a test runner:: + + if __name__ == '__main__': + import doctest + flags = doctest.REPORT_NDIFF|doctest.FAIL_FAST + if len(sys.argv) > 1: + name = sys.argv[1] + if name in globals(): + obj = globals()[name] + else: + obj = __test__[name] + doctest.run_docstring_examples(obj, globals(), name=name, + optionflags=flags) + else: + fail, total = doctest.testmod(optionflags=flags) + print("{} failures out of {} tests".format(fail, total)) + + .. rubric:: Footnotes .. [#] Examples containing both expected output and an exception are not supported. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 18 07:22:49 2015 From: python-checkins at python.org (ethan.furman) Date: Fri, 18 Sep 2015 05:22:49 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUyNDc1Njog?= =?utf-8?q?clarify_usage_of_run=5Fdocstring=5Fexamples=28=29?= Message-ID: <20150918052249.98368.39795@psf.io> https://hg.python.org/cpython/rev/64d48786d6b0 changeset: 98024:64d48786d6b0 branch: 2.7 parent: 98000:21d6b2752fe8 user: Ethan Furman date: Thu Sep 17 22:19:48 2015 -0700 summary: Issue24756: clarify usage of run_docstring_examples() files: Doc/library/doctest.rst | 29 +++++++++++++++++++++++------ 1 files changed, 23 insertions(+), 6 deletions(-) diff --git a/Doc/library/doctest.rst b/Doc/library/doctest.rst --- a/Doc/library/doctest.rst +++ b/Doc/library/doctest.rst @@ -950,15 +950,11 @@ .. versionchanged:: 2.5 The optional argument *isprivate*, deprecated in 2.4, was removed. -There's also a function to run the doctests associated with a single object. -This function is provided for backward compatibility. There are no plans to -deprecate it, but it's rarely useful: - .. function:: run_docstring_examples(f, globs[, verbose][, name][, compileflags][, optionflags]) - Test examples associated with object *f*; for example, *f* may be a module, - function, or class object. + Test examples associated with object *f*; for example, *f* may be a string, + a module, a function, or a class object. A shallow copy of dictionary argument *globs* is used for the execution context. @@ -1899,6 +1895,27 @@ * Define a ``__test__`` dictionary mapping from regression test topics to docstrings containing test cases. +When you have placed your tests in a module, the module can itself be the test +runner. When a test fails, you can arrange for your test runner to re-run only +the failing doctest while you debug the problem. Here is a minimal example of +such a test runner:: + + if __name__ == '__main__': + import doctest + flags = doctest.REPORT_NDIFF|doctest.REPORT_ONLY_FIRST_FAILURE + if len(sys.argv) > 1: + name = sys.argv[1] + if name in globals(): + obj = globals()[name] + else: + obj = __test__[name] + doctest.run_docstring_examples(obj, globals(), name=name, + optionflags=flags) + else: + fail, total = doctest.testmod(optionflags=flags) + print("{} failures out of {} tests".format(fail, total)) + + .. rubric:: Footnotes .. [#] Examples containing both expected output and an exception are not supported. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 18 07:22:50 2015 From: python-checkins at python.org (ethan.furman) Date: Fri, 18 Sep 2015 05:22:50 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Close_issue24756=3A_clarify_usage_of_run=5Fdocstring=5Fe?= =?utf-8?b?eGFtcGxlcygp?= Message-ID: <20150918052250.98372.66033@psf.io> https://hg.python.org/cpython/rev/c8689d3df68a changeset: 98027:c8689d3df68a parent: 98023:b77916d2d7cc parent: 98026:767cc99020d2 user: Ethan Furman date: Thu Sep 17 22:22:20 2015 -0700 summary: Close issue24756: clarify usage of run_docstring_examples() files: Doc/library/doctest.rst | 30 ++++++++++++++++++++++------ 1 files changed, 23 insertions(+), 7 deletions(-) diff --git a/Doc/library/doctest.rst b/Doc/library/doctest.rst --- a/Doc/library/doctest.rst +++ b/Doc/library/doctest.rst @@ -914,15 +914,10 @@ above, except that *globs* defaults to ``m.__dict__``. -There's also a function to run the doctests associated with a single object. -This function is provided for backward compatibility. There are no plans to -deprecate it, but it's rarely useful: - - .. function:: run_docstring_examples(f, globs, verbose=False, name="NoName", compileflags=None, optionflags=0) - Test examples associated with object *f*; for example, *f* may be a module, - function, or class object. + Test examples associated with object *f*; for example, *f* may be a string, + a module, a function, or a class object. A shallow copy of dictionary argument *globs* is used for the execution context. @@ -1815,6 +1810,27 @@ * Define a ``__test__`` dictionary mapping from regression test topics to docstrings containing test cases. +When you have placed your tests in a module, the module can itself be the test +runner. When a test fails, you can arrange for your test runner to re-run only +the failing doctest while you debug the problem. Here is a minimal example of +such a test runner:: + + if __name__ == '__main__': + import doctest + flags = doctest.REPORT_NDIFF|doctest.FAIL_FAST + if len(sys.argv) > 1: + name = sys.argv[1] + if name in globals(): + obj = globals()[name] + else: + obj = __test__[name] + doctest.run_docstring_examples(obj, globals(), name=name, + optionflags=flags) + else: + fail, total = doctest.testmod(optionflags=flags) + print("{} failures out of {} tests".format(fail, total)) + + .. rubric:: Footnotes .. [#] Examples containing both expected output and an exception are not supported. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 18 07:56:11 2015 From: python-checkins at python.org (ethan.furman) Date: Fri, 18 Sep 2015 05:56:11 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_25147=3A_add_reason_?= =?utf-8?q?for_using_=5Fcollections?= Message-ID: <20150918055610.81625.56129@psf.io> https://hg.python.org/cpython/rev/c0363f849624 changeset: 98028:c0363f849624 user: Ethan Furman date: Thu Sep 17 22:55:40 2015 -0700 summary: Issue 25147: add reason for using _collections files: Lib/enum.py | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diff --git a/Lib/enum.py b/Lib/enum.py --- a/Lib/enum.py +++ b/Lib/enum.py @@ -1,6 +1,7 @@ import sys from types import MappingProxyType, DynamicClassAttribute +# try _collections first to reduce startup cost try: from _collections import OrderedDict except ImportError: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 18 09:10:17 2015 From: python-checkins at python.org (serhiy.storchaka) Date: Fri, 18 Sep 2015 07:10:17 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Null_merge?= Message-ID: <20150918071017.31191.94674@psf.io> https://hg.python.org/cpython/rev/4b11f20ea549 changeset: 98032:4b11f20ea549 branch: 3.5 parent: 98029:c9fb4362fb9f parent: 98031:9f57c937958f user: Serhiy Storchaka date: Fri Sep 18 10:09:06 2015 +0300 summary: Null merge files: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 18 09:10:17 2015 From: python-checkins at python.org (serhiy.storchaka) Date: Fri, 18 Sep 2015 07:10:17 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzI1MTA4?= =?utf-8?q?=3A_Backported_tests_for_traceback_functions_print=5Fstack=28?= =?utf-8?b?KSw=?= Message-ID: <20150918071017.31203.93144@psf.io> https://hg.python.org/cpython/rev/9f57c937958f changeset: 98031:9f57c937958f branch: 3.4 parent: 98025:ce595a047bd9 user: Serhiy Storchaka date: Fri Sep 18 10:07:18 2015 +0300 summary: Issue #25108: Backported tests for traceback functions print_stack(), format_stack(), and extract_stack() called without arguments. files: Lib/test/test_traceback.py | 35 ++++++++++++++++++++++++++ 1 files changed, 35 insertions(+), 0 deletions(-) diff --git a/Lib/test/test_traceback.py b/Lib/test/test_traceback.py --- a/Lib/test/test_traceback.py +++ b/Lib/test/test_traceback.py @@ -242,6 +242,31 @@ self.assertEqual(ststderr.getvalue(), "".join(stfmt)) + def test_print_stack(self): + def prn(): + traceback.print_stack() + with captured_output("stderr") as stderr: + prn() + lineno = prn.__code__.co_firstlineno + self.assertEqual(stderr.getvalue().splitlines()[-4:], [ + ' File "%s", line %d, in test_print_stack' % (__file__, lineno+3), + ' prn()', + ' File "%s", line %d, in prn' % (__file__, lineno+1), + ' traceback.print_stack()', + ]) + + def test_format_stack(self): + def fmt(): + return traceback.format_stack() + result = fmt() + lineno = fmt.__code__.co_firstlineno + self.assertEqual(result[-2:], [ + ' File "%s", line %d, in test_format_stack\n' + ' result = fmt()\n' % (__file__, lineno+2), + ' File "%s", line %d, in fmt\n' + ' return traceback.format_stack()\n' % (__file__, lineno+1), + ]) + cause_message = ( "\nThe above exception was the direct cause " @@ -443,6 +468,16 @@ # Local variable dict should now be empty. self.assertEqual(len(inner_frame.f_locals), 0) + def test_extract_stack(self): + def extract(): + return traceback.extract_stack() + result = extract() + lineno = extract.__code__.co_firstlineno + self.assertEqual(result[-2:], [ + (__file__, lineno+2, 'test_extract_stack', 'result = extract()'), + (__file__, lineno+1, 'extract', 'return traceback.extract_stack()'), + ]) + def test_main(): run_unittest(__name__) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 18 09:10:17 2015 From: python-checkins at python.org (serhiy.storchaka) Date: Fri, 18 Sep 2015 07:10:17 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Null_merge?= Message-ID: <20150918071017.16573.18549@psf.io> https://hg.python.org/cpython/rev/4ecf82ef7983 changeset: 98033:4ecf82ef7983 parent: 98030:4e617566bcb6 parent: 98032:4b11f20ea549 user: Serhiy Storchaka date: Fri Sep 18 10:09:18 2015 +0300 summary: Null merge files: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 18 09:10:17 2015 From: python-checkins at python.org (serhiy.storchaka) Date: Fri, 18 Sep 2015 07:10:17 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI1MTA4?= =?utf-8?q?=3A_Omitted_internal_frames_in_traceback_functions_print=5Fstac?= =?utf-8?b?aygpLA==?= Message-ID: <20150918071016.94137.40568@psf.io> https://hg.python.org/cpython/rev/c9fb4362fb9f changeset: 98029:c9fb4362fb9f branch: 3.5 parent: 98026:767cc99020d2 user: Serhiy Storchaka date: Fri Sep 18 10:04:47 2015 +0300 summary: Issue #25108: Omitted internal frames in traceback functions print_stack(), format_stack(), and extract_stack() called without arguments. files: Lib/test/test_traceback.py | 35 ++++++++++++++++++++++++++ Lib/traceback.py | 6 ++++ Misc/NEWS | 3 ++ 3 files changed, 44 insertions(+), 0 deletions(-) diff --git a/Lib/test/test_traceback.py b/Lib/test/test_traceback.py --- a/Lib/test/test_traceback.py +++ b/Lib/test/test_traceback.py @@ -289,6 +289,31 @@ self.assertEqual(ststderr.getvalue(), "".join(stfmt)) + def test_print_stack(self): + def prn(): + traceback.print_stack() + with captured_output("stderr") as stderr: + prn() + lineno = prn.__code__.co_firstlineno + self.assertEqual(stderr.getvalue().splitlines()[-4:], [ + ' File "%s", line %d, in test_print_stack' % (__file__, lineno+3), + ' prn()', + ' File "%s", line %d, in prn' % (__file__, lineno+1), + ' traceback.print_stack()', + ]) + + def test_format_stack(self): + def fmt(): + return traceback.format_stack() + result = fmt() + lineno = fmt.__code__.co_firstlineno + self.assertEqual(result[-2:], [ + ' File "%s", line %d, in test_format_stack\n' + ' result = fmt()\n' % (__file__, lineno+2), + ' File "%s", line %d, in fmt\n' + ' return traceback.format_stack()\n' % (__file__, lineno+1), + ]) + cause_message = ( "\nThe above exception was the direct cause " @@ -610,6 +635,16 @@ # Local variable dict should now be empty. self.assertEqual(len(inner_frame.f_locals), 0) + def test_extract_stack(self): + def extract(): + return traceback.extract_stack() + result = extract() + lineno = extract.__code__.co_firstlineno + self.assertEqual([tuple(x) for x in result[-2:]], [ + (__file__, lineno+2, 'test_extract_stack', 'result = extract()'), + (__file__, lineno+1, 'extract', 'return traceback.extract_stack()'), + ]) + class TestFrame(unittest.TestCase): diff --git a/Lib/traceback.py b/Lib/traceback.py --- a/Lib/traceback.py +++ b/Lib/traceback.py @@ -181,11 +181,15 @@ stack frame at which to start. The optional 'limit' and 'file' arguments have the same meaning as for print_exception(). """ + if f is None: + f = sys._getframe().f_back print_list(extract_stack(f, limit=limit), file=file) def format_stack(f=None, limit=None): """Shorthand for 'format_list(extract_stack(f, limit))'.""" + if f is None: + f = sys._getframe().f_back return format_list(extract_stack(f, limit=limit)) @@ -198,6 +202,8 @@ line number, function name, text), and the entries are in order from oldest to newest stack frame. """ + if f is None: + f = sys._getframe().f_back stack = StackSummary.extract(walk_stack(f), limit=limit) stack.reverse() return stack diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -14,6 +14,9 @@ Library ------- +- Issue #25108: Omitted internal frames in traceback functions print_stack(), + format_stack(), and extract_stack() called without arguments. + - Issue #25118: Fix a regression of Python 3.5.0 in os.waitpid() on Windows. - Issue #24684: socket.socket.getaddrinfo() now calls -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 18 09:10:17 2015 From: python-checkins at python.org (serhiy.storchaka) Date: Fri, 18 Sep 2015 07:10:17 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2325108=3A_Omitted_internal_frames_in_traceback_f?= =?utf-8?q?unctions_print=5Fstack=28=29=2C?= Message-ID: <20150918071017.94123.19716@psf.io> https://hg.python.org/cpython/rev/4e617566bcb6 changeset: 98030:4e617566bcb6 parent: 98028:c0363f849624 parent: 98029:c9fb4362fb9f user: Serhiy Storchaka date: Fri Sep 18 10:06:23 2015 +0300 summary: Issue #25108: Omitted internal frames in traceback functions print_stack(), format_stack(), and extract_stack() called without arguments. files: Lib/test/test_traceback.py | 35 ++++++++++++++++++++++++++ Lib/traceback.py | 6 ++++ Misc/NEWS | 3 ++ 3 files changed, 44 insertions(+), 0 deletions(-) diff --git a/Lib/test/test_traceback.py b/Lib/test/test_traceback.py --- a/Lib/test/test_traceback.py +++ b/Lib/test/test_traceback.py @@ -289,6 +289,31 @@ self.assertEqual(ststderr.getvalue(), "".join(stfmt)) + def test_print_stack(self): + def prn(): + traceback.print_stack() + with captured_output("stderr") as stderr: + prn() + lineno = prn.__code__.co_firstlineno + self.assertEqual(stderr.getvalue().splitlines()[-4:], [ + ' File "%s", line %d, in test_print_stack' % (__file__, lineno+3), + ' prn()', + ' File "%s", line %d, in prn' % (__file__, lineno+1), + ' traceback.print_stack()', + ]) + + def test_format_stack(self): + def fmt(): + return traceback.format_stack() + result = fmt() + lineno = fmt.__code__.co_firstlineno + self.assertEqual(result[-2:], [ + ' File "%s", line %d, in test_format_stack\n' + ' result = fmt()\n' % (__file__, lineno+2), + ' File "%s", line %d, in fmt\n' + ' return traceback.format_stack()\n' % (__file__, lineno+1), + ]) + cause_message = ( "\nThe above exception was the direct cause " @@ -610,6 +635,16 @@ # Local variable dict should now be empty. self.assertEqual(len(inner_frame.f_locals), 0) + def test_extract_stack(self): + def extract(): + return traceback.extract_stack() + result = extract() + lineno = extract.__code__.co_firstlineno + self.assertEqual([tuple(x) for x in result[-2:]], [ + (__file__, lineno+2, 'test_extract_stack', 'result = extract()'), + (__file__, lineno+1, 'extract', 'return traceback.extract_stack()'), + ]) + class TestFrame(unittest.TestCase): diff --git a/Lib/traceback.py b/Lib/traceback.py --- a/Lib/traceback.py +++ b/Lib/traceback.py @@ -181,11 +181,15 @@ stack frame at which to start. The optional 'limit' and 'file' arguments have the same meaning as for print_exception(). """ + if f is None: + f = sys._getframe().f_back print_list(extract_stack(f, limit=limit), file=file) def format_stack(f=None, limit=None): """Shorthand for 'format_list(extract_stack(f, limit))'.""" + if f is None: + f = sys._getframe().f_back return format_list(extract_stack(f, limit=limit)) @@ -198,6 +202,8 @@ line number, function name, text), and the entries are in order from oldest to newest stack frame. """ + if f is None: + f = sys._getframe().f_back stack = StackSummary.extract(walk_stack(f), limit=limit) stack.reverse() return stack diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -103,6 +103,9 @@ Library ------- +- Issue #25108: Omitted internal frames in traceback functions print_stack(), + format_stack(), and extract_stack() called without arguments. + - Issue #25118: Fix a regression of Python 3.5.0 in os.waitpid() on Windows. - Issue #24684: socket.socket.getaddrinfo() now calls -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 18 09:10:18 2015 From: python-checkins at python.org (serhiy.storchaka) Date: Fri, 18 Sep 2015 07:10:18 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzI1MTA4?= =?utf-8?q?=3A_Backported_tests_for_traceback_functions_print=5Fstack=28?= =?utf-8?b?KSw=?= Message-ID: <20150918071017.81625.96606@psf.io> https://hg.python.org/cpython/rev/f6125114b55f changeset: 98034:f6125114b55f branch: 2.7 parent: 98024:64d48786d6b0 user: Serhiy Storchaka date: Fri Sep 18 10:09:29 2015 +0300 summary: Issue #25108: Backported tests for traceback functions print_stack(), format_stack(), and extract_stack() called without arguments. files: Lib/test/test_traceback.py | 46 ++++++++++++++++++++++++- 1 files changed, 44 insertions(+), 2 deletions(-) diff --git a/Lib/test/test_traceback.py b/Lib/test/test_traceback.py --- a/Lib/test/test_traceback.py +++ b/Lib/test/test_traceback.py @@ -4,7 +4,8 @@ import sys import unittest from imp import reload -from test.test_support import run_unittest, is_jython, Error, cpython_only +from test.test_support import (run_unittest, is_jython, Error, cpython_only, + captured_output) import traceback @@ -206,9 +207,50 @@ self.assertTrue(location.startswith(' File')) self.assertTrue(source_line.startswith(' raise')) + def test_print_stack(self): + def prn(): + traceback.print_stack() + with captured_output("stderr") as stderr: + prn() + lineno = prn.__code__.co_firstlineno + self.assertEqual(stderr.getvalue().splitlines()[-4:], [ + ' File "%s", line %d, in test_print_stack' % (__file__, lineno+3), + ' prn()', + ' File "%s", line %d, in prn' % (__file__, lineno+1), + ' traceback.print_stack()', + ]) + + def test_format_stack(self): + def fmt(): + return traceback.format_stack() + result = fmt() + lineno = fmt.__code__.co_firstlineno + self.assertEqual(result[-2:], [ + ' File "%s", line %d, in test_format_stack\n' + ' result = fmt()\n' % (__file__, lineno+2), + ' File "%s", line %d, in fmt\n' + ' return traceback.format_stack()\n' % (__file__, lineno+1), + ]) + + +class MiscTracebackCases(unittest.TestCase): + # + # Check non-printing functions in traceback module + # + + def test_extract_stack(self): + def extract(): + return traceback.extract_stack() + result = extract() + lineno = extract.__code__.co_firstlineno + self.assertEqual(result[-2:], [ + (__file__, lineno+2, 'test_extract_stack', 'result = extract()'), + (__file__, lineno+1, 'extract', 'return traceback.extract_stack()'), + ]) + def test_main(): - run_unittest(TracebackCases, TracebackFormatTests) + run_unittest(TracebackCases, TracebackFormatTests, MiscTracebackCases) if __name__ == "__main__": -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 18 09:12:34 2015 From: python-checkins at python.org (victor.stinner) Date: Fri, 18 Sep 2015 07:12:34 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI1MTYw?= =?utf-8?q?=3A_Fix_import=5Finit=28=29_comments_and_messages?= Message-ID: <20150918071234.82660.32448@psf.io> https://hg.python.org/cpython/rev/03cd8340e0ce changeset: 98035:03cd8340e0ce branch: 3.5 parent: 98032:4b11f20ea549 user: Victor Stinner date: Fri Sep 18 09:11:57 2015 +0200 summary: Issue #25160: Fix import_init() comments and messages import_init() imports the "_imp" module, not the "imp" module. files: Python/pylifecycle.c | 7 ++++--- 1 files changed, 4 insertions(+), 3 deletions(-) diff --git a/Python/pylifecycle.c b/Python/pylifecycle.c --- a/Python/pylifecycle.c +++ b/Python/pylifecycle.c @@ -252,13 +252,13 @@ interp->importlib = importlib; Py_INCREF(interp->importlib); - /* Install _importlib as __import__ */ + /* Import the _imp module */ impmod = PyInit_imp(); if (impmod == NULL) { - Py_FatalError("Py_Initialize: can't import imp"); + Py_FatalError("Py_Initialize: can't import _imp"); } else if (Py_VerboseFlag) { - PySys_FormatStderr("import imp # builtin\n"); + PySys_FormatStderr("import _imp # builtin\n"); } sys_modules = PyImport_GetModuleDict(); if (Py_VerboseFlag) { @@ -268,6 +268,7 @@ Py_FatalError("Py_Initialize: can't save _imp to sys.modules"); } + /* Install importlib as the implementation of import */ value = PyObject_CallMethod(importlib, "_install", "OO", sysmod, impmod); if (value == NULL) { PyErr_Print(); -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 18 09:12:34 2015 From: python-checkins at python.org (victor.stinner) Date: Fri, 18 Sep 2015 07:12:34 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogTWVyZ2UgMy41IChpbXAvX2ltcCk=?= Message-ID: <20150918071234.81645.47297@psf.io> https://hg.python.org/cpython/rev/ca37b31f71ac changeset: 98036:ca37b31f71ac parent: 98033:4ecf82ef7983 parent: 98035:03cd8340e0ce user: Victor Stinner date: Fri Sep 18 09:12:08 2015 +0200 summary: Merge 3.5 (imp/_imp) files: Python/pylifecycle.c | 7 ++++--- 1 files changed, 4 insertions(+), 3 deletions(-) diff --git a/Python/pylifecycle.c b/Python/pylifecycle.c --- a/Python/pylifecycle.c +++ b/Python/pylifecycle.c @@ -252,13 +252,13 @@ interp->importlib = importlib; Py_INCREF(interp->importlib); - /* Install _importlib as __import__ */ + /* Import the _imp module */ impmod = PyInit_imp(); if (impmod == NULL) { - Py_FatalError("Py_Initialize: can't import imp"); + Py_FatalError("Py_Initialize: can't import _imp"); } else if (Py_VerboseFlag) { - PySys_FormatStderr("import imp # builtin\n"); + PySys_FormatStderr("import _imp # builtin\n"); } sys_modules = PyImport_GetModuleDict(); if (Py_VerboseFlag) { @@ -268,6 +268,7 @@ Py_FatalError("Py_Initialize: can't save _imp to sys.modules"); } + /* Install importlib as the implementation of import */ value = PyObject_CallMethod(importlib, "_install", "OO", sysmod, impmod); if (value == NULL) { PyErr_Print(); -- Repository URL: https://hg.python.org/cpython From solipsis at pitrou.net Fri Sep 18 10:46:08 2015 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Fri, 18 Sep 2015 08:46:08 +0000 Subject: [Python-checkins] Daily reference leaks (c8689d3df68a): sum=17877 Message-ID: <20150918084608.98372.89503@psf.io> results for c8689d3df68a on branch "default" -------------------------------------------- test_capi leaked [1598, 1598, 1598] references, sum=4794 test_capi leaked [387, 389, 389] memory blocks, sum=1165 test_functools leaked [0, 2, 2] memory blocks, sum=4 test_threading leaked [3196, 3196, 3196] references, sum=9588 test_threading leaked [774, 776, 776] memory blocks, sum=2326 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/psf-users/antoine/refleaks/reflogvs05Tp', '--timeout', '7200'] From lp_benchmark_robot at intel.com Fri Sep 18 10:48:52 2015 From: lp_benchmark_robot at intel.com (lp_benchmark_robot) Date: Fri, 18 Sep 2015 08:48:52 +0000 Subject: [Python-checkins] Benchmark Results for Python Default 2015-09-18 Message-ID: No new revisions. Here are the previous results: Results for project python_default-nightly, build date 2015-09-18 03:02:31 commit: e33b4c18af59f938225141c1515602c0a644e4f2 revision date: 2015-09-16 16:18:55 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v3.4.3, with hash b4cbecbc0781e89a309d03b60a1f75f8499250e6 from 2015-02-25 12:15:33+00:00 ------------------------------------------------------------------------------------------ benchmark relative change since change since current rev with std_dev* last run v3.4.3 regrtest PGO ------------------------------------------------------------------------------------------ :-) django_v2 0.20081% -0.61311% 8.92436% 16.28674% :-( pybench 0.13539% 0.02229% -2.46307% 9.43340% :-( regex_v8 2.87636% -0.30130% -3.84184% 7.62587% :-| nbody 0.09229% -0.02505% -0.96319% 10.31006% :-| json_dump_v2 0.20164% 0.02256% -1.12573% 11.84351% :-| normal_startup 0.63661% 0.39540% 0.45931% 5.22018% ------------------------------------------------------------------------------------------ Note: Benchmark results are measured in seconds. * Relative Standard Deviation (Standard Deviation/Average) Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. From lp_benchmark_robot at intel.com Fri Sep 18 10:49:01 2015 From: lp_benchmark_robot at intel.com (lp_benchmark_robot) Date: Fri, 18 Sep 2015 08:49:01 +0000 Subject: [Python-checkins] Benchmark Results for Python 2.7 2015-09-18 Message-ID: No new revisions. Here are the previous results: Results for project python_2.7-nightly, build date 2015-09-18 03:04:18 commit: 21d6b2752fe8db0428c92d928678b15d76a9a27b revision date: 2015-09-14 22:19:47 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v2.7.10, with hash 15c95b7d81dcf821daade360741e00714667653f from 2015-05-23 16:02:14+00:00 ------------------------------------------------------------------------------------------ benchmark relative change since change since current rev with std_dev* last run v2.7.10 regrtest PGO ------------------------------------------------------------------------------------------ :-) django_v2 0.29878% -1.76221% 3.17342% 11.27247% :-) pybench 0.16246% -0.03220% 6.73896% 6.53390% :-| regex_v8 0.59862% 0.02829% -1.15354% 7.50447% :-) nbody 0.28427% 0.00747% 9.04932% 2.89917% :-) json_dump_v2 0.25198% 0.48636% 4.42795% 13.96206% :-| normal_startup 1.83289% -0.42891% -1.66451% 3.47175% :-) ssbench 0.15324% 0.85916% 2.41400% 2.04298% ------------------------------------------------------------------------------------------ Note: Benchmark results for ssbench are measured in requests/second while all other are measured in seconds. * Relative Standard Deviation (Standard Deviation/Average) Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. From python-checkins at python.org Fri Sep 18 11:31:30 2015 From: python-checkins at python.org (victor.stinner) Date: Fri, 18 Sep 2015 09:31:30 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Null_merge_3=2E5?= Message-ID: <20150918093130.3642.63819@psf.io> https://hg.python.org/cpython/rev/70cd7895499d changeset: 98039:70cd7895499d parent: 98037:90722634f211 parent: 98038:f347ea4391f3 user: Victor Stinner date: Fri Sep 18 11:30:42 2015 +0200 summary: Null merge 3.5 files: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 18 11:31:30 2015 From: python-checkins at python.org (victor.stinner) Date: Fri, 18 Sep 2015 09:31:30 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2325122=3A_Fix_test?= =?utf-8?q?=5Feintr=2Etest=5Fopen=28=29_on_FreeBSD?= Message-ID: <20150918093130.9931.31478@psf.io> https://hg.python.org/cpython/rev/90722634f211 changeset: 98037:90722634f211 user: Victor Stinner date: Fri Sep 18 11:23:42 2015 +0200 summary: Issue #25122: Fix test_eintr.test_open() on FreeBSD Skip test_open() and test_os_open(): both tests uses a FIFO and signals, but there is a bug in the FreeBSD kernel which blocks the test. Skip the tests until the bug is fixed in FreeBSD kernel. Remove also debug traces from test_eintr: * stop using faulthandler to have a timeout * remove print() Write also open and close on two lines in test_open() and test_os_open() tests. If these tests block again, we can know if the test is stuck at open or close. test_eintr: don't always run the test in debug mode. files: Lib/test/eintrdata/eintr_tester.py | 44 ++++++----------- Lib/test/test_eintr.py | 3 +- 2 files changed, 18 insertions(+), 29 deletions(-) diff --git a/Lib/test/eintrdata/eintr_tester.py b/Lib/test/eintrdata/eintr_tester.py --- a/Lib/test/eintrdata/eintr_tester.py +++ b/Lib/test/eintrdata/eintr_tester.py @@ -9,7 +9,6 @@ """ import contextlib -import faulthandler import io import os import select @@ -48,19 +47,12 @@ @classmethod def setUpClass(cls): cls.orig_handler = signal.signal(signal.SIGALRM, lambda *args: None) - if hasattr(faulthandler, 'dump_traceback_later'): - # Most tests take less than 30 seconds, so 5 minutes should be - # enough. dump_traceback_later() is implemented with a thread, but - # pthread_sigmask() is used to mask all signaled on this thread. - faulthandler.dump_traceback_later(5 * 60, exit=True) signal.setitimer(signal.ITIMER_REAL, cls.signal_delay, cls.signal_period) @classmethod def stop_alarm(cls): signal.setitimer(signal.ITIMER_REAL, 0, 0) - if hasattr(faulthandler, 'cancel_dump_traceback_later'): - faulthandler.cancel_dump_traceback_later() @classmethod def tearDownClass(cls): @@ -307,6 +299,11 @@ client_sock.close() self.assertEqual(proc.wait(), 0) + # Issue #25122: There is a race condition in the FreeBSD kernel on + # handling signals in the FIFO device. Skip the test until the bug is + # fixed in the kernel. Bet that the bug will be fixed in FreeBSD 11. + # https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=203162 + @support.requires_freebsd_version(11) @unittest.skipUnless(hasattr(os, 'mkfifo'), 'needs mkfifo()') def _test_open(self, do_open_close_reader, do_open_close_writer): filename = support.TESTFN @@ -326,36 +323,29 @@ '# let the parent block', 'time.sleep(sleep_time)', '', - 'print("try to open %a fifo for reading, pid %s, ppid %s"' - ' % (path, os.getpid(), os.getppid()),' - ' flush=True)', - '', do_open_close_reader, - '', - 'print("%a fifo opened for reading and closed, pid %s, ppid %s"' - ' % (path, os.getpid(), os.getppid()),' - ' flush=True)', )) proc = self.subprocess(code) with kill_on_error(proc): - print("try to open %a fifo for writing, pid %s" - % (filename, os.getpid()), - flush=True) do_open_close_writer(filename) - print("%a fifo opened for writing and closed, pid %s" - % (filename, os.getpid()), - flush=True) - self.assertEqual(proc.wait(), 0) + def python_open(self, path): + fp = open(path, 'w') + fp.close() + def test_open(self): - self._test_open("open(path, 'r').close()", - lambda path: open(path, 'w').close()) + self._test_open("fp = open(path, 'r')\nfp.close()", + self.python_open) + + def os_open(self, path): + fd = os.open(path, os.O_WRONLY) + os.close(fd) def test_os_open(self): - self._test_open("os.close(os.open(path, os.O_RDONLY))", - lambda path: os.close(os.open(path, os.O_WRONLY))) + self._test_open("fd = os.open(path, os.O_RDONLY)\nos.close(fd)", + self.os_open) @unittest.skipUnless(hasattr(signal, "setitimer"), "requires setitimer()") diff --git a/Lib/test/test_eintr.py b/Lib/test/test_eintr.py --- a/Lib/test/test_eintr.py +++ b/Lib/test/test_eintr.py @@ -17,8 +17,7 @@ # thread (for reliable signal delivery). tester = support.findfile("eintr_tester.py", subdir="eintrdata") - # FIXME: Issue #25122, always run in verbose mode to debug hang on FreeBSD - if True: #support.verbose: + if support.verbose: args = [sys.executable, tester] with subprocess.Popen(args) as proc: exitcode = proc.wait() -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 18 11:31:31 2015 From: python-checkins at python.org (victor.stinner) Date: Fri, 18 Sep 2015 09:31:31 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI1MTIy?= =?utf-8?q?=3A_sync_test=5Feintr_with_Python_3=2E6?= Message-ID: <20150918093130.98358.94187@psf.io> https://hg.python.org/cpython/rev/f347ea4391f3 changeset: 98038:f347ea4391f3 branch: 3.5 parent: 98035:03cd8340e0ce user: Victor Stinner date: Fri Sep 18 11:29:16 2015 +0200 summary: Issue #25122: sync test_eintr with Python 3.6 * test_eintr: support verbose mode, don't redirect eintr_tester output into a pipe * eintr_tester: replace os.fork() with subprocess to have a cleaner child process (ex: don't inherit setitimer()) * eintr_tester: kill the process if the unit test fails * test_open/test_os_open(): write support.PIPE_MAX_SIZE bytes instead of support.PIPE_MAX_SIZE*3 bytes files: Lib/test/eintrdata/eintr_tester.py | 283 +++++++++++----- Lib/test/test_eintr.py | 11 +- 2 files changed, 198 insertions(+), 96 deletions(-) diff --git a/Lib/test/eintrdata/eintr_tester.py b/Lib/test/eintrdata/eintr_tester.py --- a/Lib/test/eintrdata/eintr_tester.py +++ b/Lib/test/eintrdata/eintr_tester.py @@ -8,16 +8,29 @@ sub-second periodicity (contrarily to signal()). """ +import contextlib import io import os import select import signal import socket +import subprocess +import sys import time import unittest from test import support + at contextlib.contextmanager +def kill_on_error(proc): + """Context manager killing the subprocess if a Python exception is raised.""" + with proc: + try: + yield proc + except: + proc.kill() + raise + @unittest.skipUnless(hasattr(signal, "setitimer"), "requires setitimer()") class EINTRBaseTest(unittest.TestCase): @@ -28,7 +41,7 @@ # signal delivery periodicity signal_period = 0.1 # default sleep time for tests - should obviously have: - #?sleep_time > signal_period + # sleep_time > signal_period sleep_time = 0.2 @classmethod @@ -51,18 +64,22 @@ # default sleep time time.sleep(cls.sleep_time) + def subprocess(self, *args, **kw): + cmd_args = (sys.executable, '-c') + args + return subprocess.Popen(cmd_args, **kw) + @unittest.skipUnless(hasattr(signal, "setitimer"), "requires setitimer()") class OSEINTRTest(EINTRBaseTest): """ EINTR tests for the os module. """ + def new_sleep_process(self): + code = 'import time; time.sleep(%r)' % self.sleep_time + return self.subprocess(code) + def _test_wait_multiple(self, wait_func): num = 3 - for _ in range(num): - pid = os.fork() - if pid == 0: - self._sleep() - os._exit(0) + processes = [self.new_sleep_process() for _ in range(num)] for _ in range(num): wait_func() @@ -74,12 +91,8 @@ self._test_wait_multiple(lambda: os.wait3(0)) def _test_wait_single(self, wait_func): - pid = os.fork() - if pid == 0: - self._sleep() - os._exit(0) - else: - wait_func(pid) + proc = self.new_sleep_process() + wait_func(proc.pid) def test_waitpid(self): self._test_wait_single(lambda pid: os.waitpid(pid, 0)) @@ -97,19 +110,25 @@ # atomic datas = [b"hello", b"world", b"spam"] - pid = os.fork() - if pid == 0: - os.close(rd) - for data in datas: - # let the parent block on read() - self._sleep() - os.write(wr, data) - os._exit(0) - else: - self.addCleanup(os.waitpid, pid, 0) + code = '\n'.join(( + 'import os, sys, time', + '', + 'wr = int(sys.argv[1])', + 'datas = %r' % datas, + 'sleep_time = %r' % self.sleep_time, + '', + 'for data in datas:', + ' # let the parent block on read()', + ' time.sleep(sleep_time)', + ' os.write(wr, data)', + )) + + proc = self.subprocess(code, str(wr), pass_fds=[wr]) + with kill_on_error(proc): os.close(wr) for data in datas: self.assertEqual(data, os.read(rd, len(data))) + self.assertEqual(proc.wait(), 0) def test_write(self): rd, wr = os.pipe() @@ -117,25 +136,37 @@ # rd closed explicitly by parent # we must write enough data for the write() to block - data = b"xyz" * support.PIPE_MAX_SIZE + data = b"x" * support.PIPE_MAX_SIZE - pid = os.fork() - if pid == 0: - os.close(wr) - read_data = io.BytesIO() - # let the parent block on write() - self._sleep() - while len(read_data.getvalue()) < len(data): - chunk = os.read(rd, 2 * len(data)) - read_data.write(chunk) - self.assertEqual(read_data.getvalue(), data) - os._exit(0) - else: + code = '\n'.join(( + 'import io, os, sys, time', + '', + 'rd = int(sys.argv[1])', + 'sleep_time = %r' % self.sleep_time, + 'data = b"x" * %s' % support.PIPE_MAX_SIZE, + 'data_len = len(data)', + '', + '# let the parent block on write()', + 'time.sleep(sleep_time)', + '', + 'read_data = io.BytesIO()', + 'while len(read_data.getvalue()) < data_len:', + ' chunk = os.read(rd, 2 * data_len)', + ' read_data.write(chunk)', + '', + 'value = read_data.getvalue()', + 'if value != data:', + ' raise Exception("read error: %s vs %s bytes"', + ' % (len(value), data_len))', + )) + + proc = self.subprocess(code, str(rd), pass_fds=[rd]) + with kill_on_error(proc): os.close(rd) written = 0 while written < len(data): written += os.write(wr, memoryview(data)[written:]) - self.assertEqual(0, os.waitpid(pid, 0)[1]) + self.assertEqual(proc.wait(), 0) @unittest.skipUnless(hasattr(signal, "setitimer"), "requires setitimer()") @@ -151,19 +182,32 @@ # single-byte payload guard us against partial recv datas = [b"x", b"y", b"z"] - pid = os.fork() - if pid == 0: - rd.close() - for data in datas: - # let the parent block on recv() - self._sleep() - wr.sendall(data) - os._exit(0) - else: - self.addCleanup(os.waitpid, pid, 0) + code = '\n'.join(( + 'import os, socket, sys, time', + '', + 'fd = int(sys.argv[1])', + 'family = %s' % int(wr.family), + 'sock_type = %s' % int(wr.type), + 'datas = %r' % datas, + 'sleep_time = %r' % self.sleep_time, + '', + 'wr = socket.fromfd(fd, family, sock_type)', + 'os.close(fd)', + '', + 'with wr:', + ' for data in datas:', + ' # let the parent block on recv()', + ' time.sleep(sleep_time)', + ' wr.sendall(data)', + )) + + fd = wr.fileno() + proc = self.subprocess(code, str(fd), pass_fds=[fd]) + with kill_on_error(proc): wr.close() for data in datas: self.assertEqual(data, recv_func(rd, len(data))) + self.assertEqual(proc.wait(), 0) def test_recv(self): self._test_recv(socket.socket.recv) @@ -180,25 +224,43 @@ # we must send enough data for the send() to block data = b"xyz" * (support.SOCK_MAX_SIZE // 3) - pid = os.fork() - if pid == 0: - wr.close() - # let the parent block on send() - self._sleep() - received_data = bytearray(len(data)) - n = 0 - while n < len(data): - n += rd.recv_into(memoryview(received_data)[n:]) - self.assertEqual(received_data, data) - os._exit(0) - else: + code = '\n'.join(( + 'import os, socket, sys, time', + '', + 'fd = int(sys.argv[1])', + 'family = %s' % int(rd.family), + 'sock_type = %s' % int(rd.type), + 'sleep_time = %r' % self.sleep_time, + 'data = b"xyz" * %s' % (support.SOCK_MAX_SIZE // 3), + 'data_len = len(data)', + '', + 'rd = socket.fromfd(fd, family, sock_type)', + 'os.close(fd)', + '', + 'with rd:', + ' # let the parent block on send()', + ' time.sleep(sleep_time)', + '', + ' received_data = bytearray(data_len)', + ' n = 0', + ' while n < data_len:', + ' n += rd.recv_into(memoryview(received_data)[n:])', + '', + 'if received_data != data:', + ' raise Exception("recv error: %s vs %s bytes"', + ' % (len(received_data), data_len))', + )) + + fd = rd.fileno() + proc = self.subprocess(code, str(fd), pass_fds=[fd]) + with kill_on_error(proc): rd.close() written = 0 while written < len(data): sent = send_func(wr, memoryview(data)[written:]) # sendall() returns None written += len(data) if sent is None else sent - self.assertEqual(0, os.waitpid(pid, 0)[1]) + self.assertEqual(proc.wait(), 0) def test_send(self): self._test_send(socket.socket.send) @@ -215,46 +277,75 @@ self.addCleanup(sock.close) sock.bind((support.HOST, 0)) - _, port = sock.getsockname() + port = sock.getsockname()[1] sock.listen() - pid = os.fork() - if pid == 0: - # let parent block on accept() - self._sleep() - with socket.create_connection((support.HOST, port)): - self._sleep() - os._exit(0) - else: - self.addCleanup(os.waitpid, pid, 0) + code = '\n'.join(( + 'import socket, time', + '', + 'host = %r' % support.HOST, + 'port = %s' % port, + 'sleep_time = %r' % self.sleep_time, + '', + '# let parent block on accept()', + 'time.sleep(sleep_time)', + 'with socket.create_connection((host, port)):', + ' time.sleep(sleep_time)', + )) + + proc = self.subprocess(code) + with kill_on_error(proc): client_sock, _ = sock.accept() client_sock.close() + self.assertEqual(proc.wait(), 0) + # Issue #25122: There is a race condition in the FreeBSD kernel on + # handling signals in the FIFO device. Skip the test until the bug is + # fixed in the kernel. Bet that the bug will be fixed in FreeBSD 11. + # https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=203162 + @support.requires_freebsd_version(11) @unittest.skipUnless(hasattr(os, 'mkfifo'), 'needs mkfifo()') def _test_open(self, do_open_close_reader, do_open_close_writer): + filename = support.TESTFN + # Use a fifo: until the child opens it for reading, the parent will # block when trying to open it for writing. - support.unlink(support.TESTFN) - os.mkfifo(support.TESTFN) - self.addCleanup(support.unlink, support.TESTFN) + support.unlink(filename) + os.mkfifo(filename) + self.addCleanup(support.unlink, filename) - pid = os.fork() - if pid == 0: - # let the parent block - self._sleep() - do_open_close_reader(support.TESTFN) - os._exit(0) - else: - self.addCleanup(os.waitpid, pid, 0) - do_open_close_writer(support.TESTFN) + code = '\n'.join(( + 'import os, time', + '', + 'path = %a' % filename, + 'sleep_time = %r' % self.sleep_time, + '', + '# let the parent block', + 'time.sleep(sleep_time)', + '', + do_open_close_reader, + )) + + proc = self.subprocess(code) + with kill_on_error(proc): + do_open_close_writer(filename) + self.assertEqual(proc.wait(), 0) + + def python_open(self, path): + fp = open(path, 'w') + fp.close() def test_open(self): - self._test_open(lambda path: open(path, 'r').close(), - lambda path: open(path, 'w').close()) + self._test_open("fp = open(path, 'r')\nfp.close()", + self.python_open) + + def os_open(self, path): + fd = os.open(path, os.O_WRONLY) + os.close(fd) def test_os_open(self): - self._test_open(lambda path: os.close(os.open(path, os.O_RDONLY)), - lambda path: os.close(os.open(path, os.O_WRONLY))) + self._test_open("fd = os.open(path, os.O_RDONLY)\nos.close(fd)", + self.os_open) @unittest.skipUnless(hasattr(signal, "setitimer"), "requires setitimer()") @@ -290,20 +381,22 @@ old_handler = signal.signal(signum, lambda *args: None) self.addCleanup(signal.signal, signum, old_handler) + code = '\n'.join(( + 'import os, time', + 'pid = %s' % os.getpid(), + 'signum = %s' % int(signum), + 'sleep_time = %r' % self.sleep_time, + 'time.sleep(sleep_time)', + 'os.kill(pid, signum)', + )) + t0 = time.monotonic() - child_pid = os.fork() - if child_pid == 0: - # child - try: - self._sleep() - os.kill(pid, signum) - finally: - os._exit(0) - else: + proc = self.subprocess(code) + with kill_on_error(proc): # parent signal.sigwaitinfo([signum]) dt = time.monotonic() - t0 - os.waitpid(child_pid, 0) + self.assertEqual(proc.wait(), 0) self.assertGreaterEqual(dt, self.sleep_time) diff --git a/Lib/test/test_eintr.py b/Lib/test/test_eintr.py --- a/Lib/test/test_eintr.py +++ b/Lib/test/test_eintr.py @@ -1,5 +1,7 @@ import os import signal +import subprocess +import sys import unittest from test import support @@ -14,7 +16,14 @@ # Run the tester in a sub-process, to make sure there is only one # thread (for reliable signal delivery). tester = support.findfile("eintr_tester.py", subdir="eintrdata") - script_helper.assert_python_ok(tester) + + if support.verbose: + args = [sys.executable, tester] + with subprocess.Popen(args) as proc: + exitcode = proc.wait() + self.assertEqual(exitcode, 0) + else: + script_helper.assert_python_ok(tester) if __name__ == "__main__": -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 18 13:27:01 2015 From: python-checkins at python.org (victor.stinner) Date: Fri, 18 Sep 2015 11:27:01 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2325155=3A_Add_=5FP?= =?utf-8?q?yTime=5FAsTimevalTime=5Ft=28=29_function?= Message-ID: <20150918112700.9957.31833@psf.io> https://hg.python.org/cpython/rev/203134592edf changeset: 98040:203134592edf user: Victor Stinner date: Fri Sep 18 13:23:02 2015 +0200 summary: Issue #25155: Add _PyTime_AsTimevalTime_t() function On Windows, the tv_sec field of the timeval structure has the type C long, whereas it has the type C time_t on all other platforms. A C long has a size of 32 bits (signed inter, 1 bit for the sign, 31 bits for the value) which is not enough to store an Epoch timestamp after the year 2038. Add the _PyTime_AsTimevalTime_t() function written for datetime.datetime.now(): convert a _PyTime_t timestamp to a (secs, us) tuple where secs type is time_t. It allows to support dates after the year 2038 on Windows. Enhance also _PyTime_AsTimeval_impl() to detect overflow on the number of seconds when rounding the number of microseconds. files: Include/pytime.h | 12 +++ Modules/_datetimemodule.c | 13 ++- Python/pytime.c | 85 ++++++++++++++++++++------ 3 files changed, 82 insertions(+), 28 deletions(-) diff --git a/Include/pytime.h b/Include/pytime.h --- a/Include/pytime.h +++ b/Include/pytime.h @@ -120,6 +120,18 @@ struct timeval *tv, _PyTime_round_t round); +/* Convert a timestamp to a number of seconds (secs) and microseconds (us). + us is always positive. This function is similar to _PyTime_AsTimeval() + except that secs is always a time_t type, whereas the timeval structure + uses a C long for tv_sec on Windows. + Raise an exception and return -1 if the conversion overflowed, + return 0 on success. */ +PyAPI_FUNC(int) _PyTime_AsTimevalTime_t( + _PyTime_t t, + time_t *secs, + int *us, + _PyTime_round_t round); + #if defined(HAVE_CLOCK_GETTIME) || defined(HAVE_KQUEUE) /* Convert a timestamp to a timespec structure (nanosecond resolution). tv_nsec is always positive. diff --git a/Modules/_datetimemodule.c b/Modules/_datetimemodule.c --- a/Modules/_datetimemodule.c +++ b/Modules/_datetimemodule.c @@ -4117,13 +4117,14 @@ datetime_best_possible(PyObject *cls, TM_FUNC f, PyObject *tzinfo) { _PyTime_t ts = _PyTime_GetSystemClock(); - struct timeval tv; - - if (_PyTime_AsTimeval(ts, &tv, _PyTime_ROUND_FLOOR) < 0) + time_t secs; + int us; + + if (_PyTime_AsTimevalTime_t(ts, &secs, &us, _PyTime_ROUND_FLOOR) < 0) return NULL; - assert(0 <= tv.tv_usec && tv.tv_usec <= 999999); - - return datetime_from_timet_and_us(cls, f, tv.tv_sec, tv.tv_usec, tzinfo); + assert(0 <= us && us <= 999999); + + return datetime_from_timet_and_us(cls, f, secs, us, tzinfo); } /*[clinic input] diff --git a/Python/pytime.c b/Python/pytime.c --- a/Python/pytime.c +++ b/Python/pytime.c @@ -417,54 +417,95 @@ } static int -_PyTime_AsTimeval_impl(_PyTime_t t, struct timeval *tv, _PyTime_round_t round, - int raise) +_PyTime_AsTimeval_impl(_PyTime_t t, _PyTime_t *p_secs, int *p_us, + _PyTime_round_t round) { _PyTime_t secs, ns; + int usec; int res = 0; - int usec; secs = t / SEC_TO_NS; ns = t % SEC_TO_NS; + usec = (int)_PyTime_Divide(ns, US_TO_NS, round); + if (usec < 0) { + usec += SEC_TO_US; + if (secs != _PyTime_MIN) + secs -= 1; + else + res = -1; + } + else if (usec >= SEC_TO_US) { + usec -= SEC_TO_US; + if (secs != _PyTime_MAX) + secs += 1; + else + res = -1; + } + assert(0 <= usec && usec < SEC_TO_US); + + *p_secs = secs; + *p_us = usec; + + return res; +} + +static int +_PyTime_AsTimevalStruct_impl(_PyTime_t t, struct timeval *tv, + _PyTime_round_t round, int raise) +{ + _PyTime_t secs; + int us; + int res; + + res = _PyTime_AsTimeval_impl(t, &secs, &us, round); + #ifdef MS_WINDOWS tv->tv_sec = (long)secs; #else tv->tv_sec = secs; #endif - if ((_PyTime_t)tv->tv_sec != secs) - res = -1; + tv->tv_usec = us; - usec = (int)_PyTime_Divide(ns, US_TO_NS, round); - if (usec < 0) { - usec += SEC_TO_US; - tv->tv_sec -= 1; + if (res < 0 || (_PyTime_t)tv->tv_sec != secs) { + if (raise) + error_time_t_overflow(); + return -1; } - else if (usec >= SEC_TO_US) { - usec -= SEC_TO_US; - tv->tv_sec += 1; - } - - assert(0 <= usec && usec < SEC_TO_US); - tv->tv_usec = usec; - - if (res && raise) - error_time_t_overflow(); - return res; + return 0; } int _PyTime_AsTimeval(_PyTime_t t, struct timeval *tv, _PyTime_round_t round) { - return _PyTime_AsTimeval_impl(t, tv, round, 1); + return _PyTime_AsTimevalStruct_impl(t, tv, round, 1); } int _PyTime_AsTimeval_noraise(_PyTime_t t, struct timeval *tv, _PyTime_round_t round) { - return _PyTime_AsTimeval_impl(t, tv, round, 0); + return _PyTime_AsTimevalStruct_impl(t, tv, round, 0); } +int +_PyTime_AsTimevalTime_t(_PyTime_t t, time_t *p_secs, int *us, + _PyTime_round_t round) +{ + _PyTime_t secs; + int res; + + res = _PyTime_AsTimeval_impl(t, &secs, us, round); + + *p_secs = secs; + + if (res < 0 || (_PyTime_t)*p_secs != secs) { + error_time_t_overflow(); + return -1; + } + return 0; +} + + #if defined(HAVE_CLOCK_GETTIME) || defined(HAVE_KQUEUE) int _PyTime_AsTimespec(_PyTime_t t, struct timespec *ts) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 18 13:56:07 2015 From: python-checkins at python.org (victor.stinner) Date: Fri, 18 Sep 2015 11:56:07 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogb2RpY3RvYmplY3Qu?= =?utf-8?q?c=3A_fix_compiler_warning?= Message-ID: <20150918115606.3660.71209@psf.io> https://hg.python.org/cpython/rev/e8f5502f7724 changeset: 98042:e8f5502f7724 branch: 3.5 user: Victor Stinner date: Fri Sep 18 13:44:11 2015 +0200 summary: odictobject.c: fix compiler warning PyObject_Length() returns a P_ssize_t, not an int. Use a Py_ssize_t to avoid overflow. files: Objects/odictobject.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Objects/odictobject.c b/Objects/odictobject.c --- a/Objects/odictobject.c +++ b/Objects/odictobject.c @@ -998,7 +998,7 @@ goto Done; else { PyObject *empty, *od_vars, *iterator, *key; - int ns_len; + Py_ssize_t ns_len; /* od.__dict__ isn't necessarily a dict... */ ns = PyObject_CallMethod((PyObject *)vars, "copy", NULL); -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 18 13:56:07 2015 From: python-checkins at python.org (victor.stinner) Date: Fri, 18 Sep 2015 11:56:07 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge_3=2E5_=28pytime=2C_odict=29?= Message-ID: <20150918115606.31187.51002@psf.io> https://hg.python.org/cpython/rev/a745a1318807 changeset: 98043:a745a1318807 parent: 98040:203134592edf parent: 98042:e8f5502f7724 user: Victor Stinner date: Fri Sep 18 13:55:15 2015 +0200 summary: Merge 3.5 (pytime, odict) files: Objects/odictobject.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Objects/odictobject.c b/Objects/odictobject.c --- a/Objects/odictobject.c +++ b/Objects/odictobject.c @@ -998,7 +998,7 @@ goto Done; else { PyObject *empty, *od_vars, *iterator, *key; - int ns_len; + Py_ssize_t ns_len; /* od.__dict__ isn't necessarily a dict... */ ns = PyObject_CallMethod((PyObject *)vars, "copy", NULL); -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 18 13:56:08 2015 From: python-checkins at python.org (victor.stinner) Date: Fri, 18 Sep 2015 11:56:08 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI1MTU1?= =?utf-8?q?=3A_Add_=5FPyTime=5FAsTimevalTime=5Ft=28=29_function?= Message-ID: <20150918115606.98362.34447@psf.io> https://hg.python.org/cpython/rev/4ca99a0a18e4 changeset: 98041:4ca99a0a18e4 branch: 3.5 parent: 98038:f347ea4391f3 user: Victor Stinner date: Fri Sep 18 13:36:17 2015 +0200 summary: Issue #25155: Add _PyTime_AsTimevalTime_t() function On Windows, the tv_sec field of the timeval structure has the type C long, whereas it has the type C time_t on all other platforms. A C long has a size of 32 bits (signed inter, 1 bit for the sign, 31 bits for the value) which is not enough to store an Epoch timestamp after the year 2038. Add the _PyTime_AsTimevalTime_t() function written for datetime.datetime.now(): convert a _PyTime_t timestamp to a (secs, us) tuple where secs type is time_t. It allows to support dates after the year 2038 on Windows. Enhance also _PyTime_AsTimeval_impl() to detect overflow on the number of seconds when rounding the number of microseconds. files: Include/pytime.h | 12 +++ Modules/_datetimemodule.c | 13 +- Python/pytime.c | 99 ++++++++++++++++---------- 3 files changed, 80 insertions(+), 44 deletions(-) diff --git a/Include/pytime.h b/Include/pytime.h --- a/Include/pytime.h +++ b/Include/pytime.h @@ -117,6 +117,18 @@ struct timeval *tv, _PyTime_round_t round); +/* Convert a timestamp to a number of seconds (secs) and microseconds (us). + us is always positive. This function is similar to _PyTime_AsTimeval() + except that secs is always a time_t type, whereas the timeval structure + uses a C long for tv_sec on Windows. + Raise an exception and return -1 if the conversion overflowed, + return 0 on success. */ +PyAPI_FUNC(int) _PyTime_AsTimevalTime_t( + _PyTime_t t, + time_t *secs, + int *us, + _PyTime_round_t round); + #if defined(HAVE_CLOCK_GETTIME) || defined(HAVE_KQUEUE) /* Convert a timestamp to a timespec structure (nanosecond resolution). tv_nsec is always positive. diff --git a/Modules/_datetimemodule.c b/Modules/_datetimemodule.c --- a/Modules/_datetimemodule.c +++ b/Modules/_datetimemodule.c @@ -4113,13 +4113,14 @@ datetime_best_possible(PyObject *cls, TM_FUNC f, PyObject *tzinfo) { _PyTime_t ts = _PyTime_GetSystemClock(); - struct timeval tv; - - if (_PyTime_AsTimeval(ts, &tv, _PyTime_ROUND_FLOOR) < 0) + time_t secs; + int us; + + if (_PyTime_AsTimevalTime_t(ts, &secs, &us, _PyTime_ROUND_FLOOR) < 0) return NULL; - assert(0 <= tv.tv_usec && tv.tv_usec <= 999999); - - return datetime_from_timet_and_us(cls, f, tv.tv_sec, tv.tv_usec, tzinfo); + assert(0 <= us && us <= 999999); + + return datetime_from_timet_and_us(cls, f, secs, us, tzinfo); } /*[clinic input] diff --git a/Python/pytime.c b/Python/pytime.c --- a/Python/pytime.c +++ b/Python/pytime.c @@ -331,69 +331,92 @@ } static int -_PyTime_AsTimeval_impl(_PyTime_t t, struct timeval *tv, _PyTime_round_t round, - int raise) +_PyTime_AsTimeval_impl(_PyTime_t t, _PyTime_t *p_secs, int *p_us, + _PyTime_round_t round) { _PyTime_t secs, ns; + int usec; int res = 0; secs = t / SEC_TO_NS; ns = t % SEC_TO_NS; - if (ns < 0) { - ns += SEC_TO_NS; - secs -= 1; + + usec = (int)_PyTime_Divide(ns, US_TO_NS, round); + if (usec < 0) { + usec += SEC_TO_US; + if (secs != _PyTime_MIN) + secs -= 1; + else + res = -1; } + else if (usec >= SEC_TO_US) { + usec -= SEC_TO_US; + if (secs != _PyTime_MAX) + secs += 1; + else + res = -1; + } + assert(0 <= usec && usec < SEC_TO_US); + + *p_secs = secs; + *p_us = usec; + + return res; +} + +static int +_PyTime_AsTimevalStruct_impl(_PyTime_t t, struct timeval *tv, + _PyTime_round_t round, int raise) +{ + _PyTime_t secs; + int us; + int res; + + res = _PyTime_AsTimeval_impl(t, &secs, &us, round); #ifdef MS_WINDOWS - /* On Windows, timeval.tv_sec is a long (32 bit), - whereas time_t can be 64-bit. */ - assert(sizeof(tv->tv_sec) == sizeof(long)); -#if SIZEOF_TIME_T > SIZEOF_LONG - if (secs > LONG_MAX) { - secs = LONG_MAX; - res = -1; - } - else if (secs < LONG_MIN) { - secs = LONG_MIN; - res = -1; - } -#endif tv->tv_sec = (long)secs; #else - /* On OpenBSD 5.4, timeval.tv_sec is a long. - Example: long is 64-bit, whereas time_t is 32-bit. */ tv->tv_sec = secs; - if ((_PyTime_t)tv->tv_sec != secs) - res = -1; #endif + tv->tv_usec = us; - if (round == _PyTime_ROUND_CEILING) - tv->tv_usec = (int)((ns + US_TO_NS - 1) / US_TO_NS); - else - tv->tv_usec = (int)(ns / US_TO_NS); - - if (tv->tv_usec >= SEC_TO_US) { - tv->tv_usec -= SEC_TO_US; - tv->tv_sec += 1; + if (res < 0 || (_PyTime_t)tv->tv_sec != secs) { + if (raise) + error_time_t_overflow(); + return -1; } - - if (res && raise) - _PyTime_overflow(); - - assert(0 <= tv->tv_usec && tv->tv_usec <= 999999); - return res; + return 0; } int _PyTime_AsTimeval(_PyTime_t t, struct timeval *tv, _PyTime_round_t round) { - return _PyTime_AsTimeval_impl(t, tv, round, 1); + return _PyTime_AsTimevalStruct_impl(t, tv, round, 1); } int _PyTime_AsTimeval_noraise(_PyTime_t t, struct timeval *tv, _PyTime_round_t round) { - return _PyTime_AsTimeval_impl(t, tv, round, 0); + return _PyTime_AsTimevalStruct_impl(t, tv, round, 0); +} + +int +_PyTime_AsTimevalTime_t(_PyTime_t t, time_t *p_secs, int *us, + _PyTime_round_t round) +{ + _PyTime_t secs; + int res; + + res = _PyTime_AsTimeval_impl(t, &secs, us, round); + + *p_secs = secs; + + if (res < 0 || (_PyTime_t)*p_secs != secs) { + error_time_t_overflow(); + return -1; + } + return 0; } #if defined(HAVE_CLOCK_GETTIME) || defined(HAVE_KQUEUE) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 18 13:59:37 2015 From: python-checkins at python.org (victor.stinner) Date: Fri, 18 Sep 2015 11:59:37 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogTWVyZ2UgMy41IChORVdTKQ==?= Message-ID: <20150918115937.9935.82133@psf.io> https://hg.python.org/cpython/rev/c6fe4cb70ffa changeset: 98045:c6fe4cb70ffa parent: 98043:a745a1318807 parent: 98044:5bfcccf229c4 user: Victor Stinner date: Fri Sep 18 13:59:30 2015 +0200 summary: Merge 3.5 (NEWS) files: Misc/NEWS | 4 ++++ 1 files changed, 4 insertions(+), 0 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -103,6 +103,10 @@ Library ------- +- Issue #25155: Fix datetime.datetime.now() and datetime.datetime.utcnow() on + Windows to support date after year 2038. It was a regression introduced in + Python 3.5.0. + - Issue #25108: Omitted internal frames in traceback functions print_stack(), format_stack(), and extract_stack() called without arguments. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 18 13:59:55 2015 From: python-checkins at python.org (victor.stinner) Date: Fri, 18 Sep 2015 11:59:55 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI1MTU1?= =?utf-8?q?=3A_document_the_bugfix_in_Misc/NEWS?= Message-ID: <20150918115936.82660.69728@psf.io> https://hg.python.org/cpython/rev/5bfcccf229c4 changeset: 98044:5bfcccf229c4 branch: 3.5 parent: 98042:e8f5502f7724 user: Victor Stinner date: Fri Sep 18 13:59:09 2015 +0200 summary: Issue #25155: document the bugfix in Misc/NEWS Oops, I forgot to document my change. files: Misc/NEWS | 4 ++++ 1 files changed, 4 insertions(+), 0 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -14,6 +14,10 @@ Library ------- +- Issue #25155: Fix datetime.datetime.now() and datetime.datetime.utcnow() on + Windows to support date after year 2038. It was a regression introduced in + Python 3.5.0. + - Issue #25108: Omitted internal frames in traceback functions print_stack(), format_stack(), and extract_stack() called without arguments. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 18 14:22:30 2015 From: python-checkins at python.org (victor.stinner) Date: Fri, 18 Sep 2015 12:22:30 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI1MTU1?= =?utf-8?q?=3A_Fix_=5FPyTime=5FDivide=28=29_rounding?= Message-ID: <20150918122230.16575.28827@psf.io> https://hg.python.org/cpython/rev/f1cc1f379b00 changeset: 98046:f1cc1f379b00 branch: 3.5 parent: 98044:5bfcccf229c4 user: Victor Stinner date: Fri Sep 18 14:21:14 2015 +0200 summary: Issue #25155: Fix _PyTime_Divide() rounding _PyTime_Divide() rounding was wrong: copy code from Python default which has now much better unit tests. files: Lib/test/test_time.py | 16 ++++++++-------- Python/pytime.c | 11 ++++++++--- 2 files changed, 16 insertions(+), 11 deletions(-) diff --git a/Lib/test/test_time.py b/Lib/test/test_time.py --- a/Lib/test/test_time.py +++ b/Lib/test/test_time.py @@ -946,14 +946,14 @@ # nanoseconds (1, 0, FLOOR), (1, 1, CEILING), - (-1, 0, FLOOR), - (-1, -1, CEILING), + (-1, -1, FLOOR), + (-1, 0, CEILING), # seconds + nanoseconds (1234 * MS_TO_NS + 1, 1234, FLOOR), (1234 * MS_TO_NS + 1, 1235, CEILING), - (-1234 * MS_TO_NS - 1, -1234, FLOOR), - (-1234 * MS_TO_NS - 1, -1235, CEILING), + (-1234 * MS_TO_NS - 1, -1235, FLOOR), + (-1234 * MS_TO_NS - 1, -1234, CEILING), ): with self.subTest(nanoseconds=ns, milliseconds=ms, round=rnd): self.assertEqual(PyTime_AsMilliseconds(ns, rnd), ms) @@ -983,14 +983,14 @@ # nanoseconds (1, 0, FLOOR), (1, 1, CEILING), - (-1, 0, FLOOR), - (-1, -1, CEILING), + (-1, -1, FLOOR), + (-1, 0, CEILING), # seconds + nanoseconds (1234 * US_TO_NS + 1, 1234, FLOOR), (1234 * US_TO_NS + 1, 1235, CEILING), - (-1234 * US_TO_NS - 1, -1234, FLOOR), - (-1234 * US_TO_NS - 1, -1235, CEILING), + (-1234 * US_TO_NS - 1, -1235, FLOOR), + (-1234 * US_TO_NS - 1, -1234, CEILING), ): with self.subTest(nanoseconds=ns, milliseconds=ms, round=rnd): self.assertEqual(PyTime_AsMicroseconds(ns, rnd), ms) diff --git a/Python/pytime.c b/Python/pytime.c --- a/Python/pytime.c +++ b/Python/pytime.c @@ -305,17 +305,22 @@ } static _PyTime_t -_PyTime_Divide(_PyTime_t t, _PyTime_t k, _PyTime_round_t round) +_PyTime_Divide(const _PyTime_t t, const _PyTime_t k, + const _PyTime_round_t round) { assert(k > 1); if (round == _PyTime_ROUND_CEILING) { if (t >= 0) return (t + k - 1) / k; else + return t / k; + } + else { + if (t >= 0) + return t / k; + else return (t - (k - 1)) / k; } - else - return t / k; } _PyTime_t -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 18 14:22:36 2015 From: python-checkins at python.org (victor.stinner) Date: Fri, 18 Sep 2015 12:22:36 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogTWVyZ2UgMy41IChweXRpbWUp?= Message-ID: <20150918122230.81637.52627@psf.io> https://hg.python.org/cpython/rev/820ae8082688 changeset: 98047:820ae8082688 parent: 98045:c6fe4cb70ffa parent: 98046:f1cc1f379b00 user: Victor Stinner date: Fri Sep 18 14:21:55 2015 +0200 summary: Merge 3.5 (pytime) files: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 18 14:58:29 2015 From: python-checkins at python.org (victor.stinner) Date: Fri, 18 Sep 2015 12:58:29 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Null_merge_3=2E5=3A_datetime_was_already_fixed=2C_but_wi?= =?utf-8?q?th_a_very_different?= Message-ID: <20150918125829.11712.67523@psf.io> https://hg.python.org/cpython/rev/70e18474e0ee changeset: 98051:70e18474e0ee parent: 98050:c828b730abca parent: 98049:d04a0954e142 user: Victor Stinner date: Fri Sep 18 14:58:09 2015 +0200 summary: Null merge 3.5: datetime was already fixed, but with a very different implementation files: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 18 14:58:29 2015 From: python-checkins at python.org (victor.stinner) Date: Fri, 18 Sep 2015 12:58:29 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzIzNTE3?= =?utf-8?q?=3A_Fix_rounding_in_fromtimestamp=28=29_and_utcfromtimestamp=28?= =?utf-8?q?=29_methods?= Message-ID: <20150918125828.3652.95433@psf.io> https://hg.python.org/cpython/rev/ee1cf1b188d2 changeset: 98048:ee1cf1b188d2 branch: 3.4 parent: 98031:9f57c937958f user: Victor Stinner date: Fri Sep 18 14:42:05 2015 +0200 summary: Issue #23517: Fix rounding in fromtimestamp() and utcfromtimestamp() methods of datetime.datetime: microseconds are now rounded to nearest with ties going to nearest even integer (ROUND_HALF_EVEN), instead of being rounding towards zero (ROUND_DOWN). It's important that these methods use the same rounding mode than datetime.timedelta to keep the property: (datetime(1970,1,1) + timedelta(seconds=t)) == datetime.utcfromtimestamp(t) It also the rounding mode used by round(float) for example. Add more unit tests on the rounding mode in test_datetime. files: Lib/datetime.py | 53 ++++++++---------- Lib/test/datetimetester.py | 24 +++++++- Misc/NEWS | 8 ++ Modules/_datetimemodule.c | 73 ++++++++++++++++++++++--- 4 files changed, 115 insertions(+), 43 deletions(-) diff --git a/Lib/datetime.py b/Lib/datetime.py --- a/Lib/datetime.py +++ b/Lib/datetime.py @@ -1362,49 +1362,42 @@ return self._tzinfo @classmethod + def _fromtimestamp(cls, t, utc, tz): + """Construct a datetime from a POSIX timestamp (like time.time()). + + A timezone info object may be passed in as well. + """ + frac, t = _math.modf(t) + us = round(frac * 1e6) + if us >= 1000000: + t += 1 + us -= 1000000 + elif us < 0: + t -= 1 + us += 1000000 + + converter = _time.gmtime if utc else _time.localtime + y, m, d, hh, mm, ss, weekday, jday, dst = converter(t) + ss = min(ss, 59) # clamp out leap seconds if the platform has them + return cls(y, m, d, hh, mm, ss, us, tz) + + @classmethod def fromtimestamp(cls, t, tz=None): """Construct a datetime from a POSIX timestamp (like time.time()). A timezone info object may be passed in as well. """ - _check_tzinfo_arg(tz) - converter = _time.localtime if tz is None else _time.gmtime - - t, frac = divmod(t, 1.0) - us = int(frac * 1e6) - - # If timestamp is less than one microsecond smaller than a - # full second, us can be rounded up to 1000000. In this case, - # roll over to seconds, otherwise, ValueError is raised - # by the constructor. - if us == 1000000: - t += 1 - us = 0 - y, m, d, hh, mm, ss, weekday, jday, dst = converter(t) - ss = min(ss, 59) # clamp out leap seconds if the platform has them - result = cls(y, m, d, hh, mm, ss, us, tz) + result = cls._fromtimestamp(t, tz is not None, tz) if tz is not None: result = tz.fromutc(result) return result @classmethod def utcfromtimestamp(cls, t): - "Construct a UTC datetime from a POSIX timestamp (like time.time())." - t, frac = divmod(t, 1.0) - us = int(frac * 1e6) - - # If timestamp is less than one microsecond smaller than a - # full second, us can be rounded up to 1000000. In this case, - # roll over to seconds, otherwise, ValueError is raised - # by the constructor. - if us == 1000000: - t += 1 - us = 0 - y, m, d, hh, mm, ss, weekday, jday, dst = _time.gmtime(t) - ss = min(ss, 59) # clamp out leap seconds if the platform has them - return cls(y, m, d, hh, mm, ss, us) + """Construct a naive UTC datetime from a POSIX timestamp.""" + return cls._fromtimestamp(t, True, None) # XXX This is supposed to do better than we *can* do by using time.time(), # XXX if the platform supports a more accurate way. The C implementation diff --git a/Lib/test/datetimetester.py b/Lib/test/datetimetester.py --- a/Lib/test/datetimetester.py +++ b/Lib/test/datetimetester.py @@ -650,8 +650,16 @@ # Single-field rounding. eq(td(milliseconds=0.4/1000), td(0)) # rounds to 0 eq(td(milliseconds=-0.4/1000), td(0)) # rounds to 0 + eq(td(milliseconds=0.5/1000), td(microseconds=0)) + eq(td(milliseconds=-0.5/1000), td(microseconds=-0)) eq(td(milliseconds=0.6/1000), td(microseconds=1)) eq(td(milliseconds=-0.6/1000), td(microseconds=-1)) + eq(td(milliseconds=1.5/1000), td(microseconds=2)) + eq(td(milliseconds=-1.5/1000), td(microseconds=-2)) + eq(td(seconds=0.5/10**6), td(microseconds=0)) + eq(td(seconds=-0.5/10**6), td(microseconds=-0)) + eq(td(seconds=1/2**7), td(microseconds=7812)) + eq(td(seconds=-1/2**7), td(microseconds=-7812)) # Rounding due to contributions from more than one field. us_per_hour = 3600e6 @@ -1824,12 +1832,14 @@ tzinfo=timezone(timedelta(hours=-5), 'EST')) self.assertEqual(t.timestamp(), 18000 + 3600 + 2*60 + 3 + 4*1e-6) + def test_microsecond_rounding(self): for fts in [self.theclass.fromtimestamp, self.theclass.utcfromtimestamp]: zero = fts(0) self.assertEqual(zero.second, 0) self.assertEqual(zero.microsecond, 0) + one = fts(1e-6) try: minus_one = fts(-1e-6) except OSError: @@ -1840,22 +1850,28 @@ self.assertEqual(minus_one.microsecond, 999999) t = fts(-1e-8) - self.assertEqual(t, minus_one) + self.assertEqual(t, zero) t = fts(-9e-7) self.assertEqual(t, minus_one) t = fts(-1e-7) - self.assertEqual(t, minus_one) + self.assertEqual(t, zero) + t = fts(-1/2**7) + self.assertEqual(t.second, 59) + self.assertEqual(t.microsecond, 992188) t = fts(1e-7) self.assertEqual(t, zero) t = fts(9e-7) - self.assertEqual(t, zero) + self.assertEqual(t, one) t = fts(0.99999949) self.assertEqual(t.second, 0) self.assertEqual(t.microsecond, 999999) t = fts(0.9999999) + self.assertEqual(t.second, 1) + self.assertEqual(t.microsecond, 0) + t = fts(1/2**7) self.assertEqual(t.second, 0) - self.assertEqual(t.microsecond, 999999) + self.assertEqual(t.microsecond, 7812) def test_insane_fromtimestamp(self): # It's possible that some platform maps time_t to double, diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -81,6 +81,14 @@ Library ------- +- Issue #23517: Fix rounding in fromtimestamp() and utcfromtimestamp() methods + of datetime.datetime: microseconds are now rounded to nearest with ties + going to nearest even integer (ROUND_HALF_EVEN), instead of being rounding + towards zero (ROUND_DOWN). It's important that these methods use the same + rounding mode than datetime.timedelta to keep the property: + (datetime(1970,1,1) + timedelta(seconds=t)) == datetime.utcfromtimestamp(t). + It also the rounding mode used by round(float) for example. + - Issue #24684: socket.socket.getaddrinfo() now calls PyUnicode_AsEncodedString() instead of calling the encode() method of the host, to handle correctly custom string with an encode() method which doesn't diff --git a/Modules/_datetimemodule.c b/Modules/_datetimemodule.c --- a/Modules/_datetimemodule.c +++ b/Modules/_datetimemodule.c @@ -4113,6 +4113,44 @@ tzinfo); } +static time_t +_PyTime_DoubleToTimet(double x) +{ + time_t result; + double diff; + + result = (time_t)x; + /* How much info did we lose? time_t may be an integral or + * floating type, and we don't know which. If it's integral, + * we don't know whether C truncates, rounds, returns the floor, + * etc. If we lost a second or more, the C rounding is + * unreasonable, or the input just doesn't fit in a time_t; + * call it an error regardless. Note that the original cast to + * time_t can cause a C error too, but nothing we can do to + * worm around that. + */ + diff = x - (double)result; + if (diff <= -1.0 || diff >= 1.0) { + PyErr_SetString(PyExc_OverflowError, + "timestamp out of range for platform time_t"); + result = (time_t)-1; + } + return result; +} + +/* Round a double to the nearest long. |x| must be small enough to fit + * in a C long; this is not checked. + */ +static double +_PyTime_RoundHalfEven(double x) +{ + double rounded = round(x); + if (fabs(x-rounded) == 0.5) + /* halfway case: round to even */ + rounded = 2.0*round(x/2.0); + return rounded; +} + /* Internal helper. * Build datetime from a Python timestamp. Pass localtime or gmtime for f, * to control the interpretation of the timestamp. Since a double doesn't @@ -4121,15 +4159,32 @@ * to get that much precision (e.g., C time() isn't good enough). */ static PyObject * -datetime_from_timestamp(PyObject *cls, TM_FUNC f, PyObject *timestamp, +datetime_from_timestamp(PyObject *cls, TM_FUNC f, double timestamp, PyObject *tzinfo) { time_t timet; - long us; - - if (_PyTime_ObjectToTimeval(timestamp, &timet, &us, _PyTime_ROUND_DOWN) == -1) + double fraction; + int us; + + timet = _PyTime_DoubleToTimet(timestamp); + if (timet == (time_t)-1 && PyErr_Occurred()) return NULL; - return datetime_from_timet_and_us(cls, f, timet, (int)us, tzinfo); + fraction = timestamp - (double)timet; + us = (int)_PyTime_RoundHalfEven(fraction * 1e6); + if (us < 0) { + /* Truncation towards zero is not what we wanted + for negative numbers (Python's mod semantics) */ + timet -= 1; + us += 1000000; + } + /* If timestamp is less than one microsecond smaller than a + * full second, round up. Otherwise, ValueErrors are raised + * for some floats. */ + if (us == 1000000) { + timet += 1; + us = 0; + } + return datetime_from_timet_and_us(cls, f, timet, us, tzinfo); } /* Internal helper. @@ -4231,11 +4286,11 @@ datetime_fromtimestamp(PyObject *cls, PyObject *args, PyObject *kw) { PyObject *self; - PyObject *timestamp; + double timestamp; PyObject *tzinfo = Py_None; static char *keywords[] = {"timestamp", "tz", NULL}; - if (! PyArg_ParseTupleAndKeywords(args, kw, "O|O:fromtimestamp", + if (! PyArg_ParseTupleAndKeywords(args, kw, "d|O:fromtimestamp", keywords, ×tamp, &tzinfo)) return NULL; if (check_tzinfo_subclass(tzinfo) < 0) @@ -4259,10 +4314,10 @@ static PyObject * datetime_utcfromtimestamp(PyObject *cls, PyObject *args) { - PyObject *timestamp; + double timestamp; PyObject *result = NULL; - if (PyArg_ParseTuple(args, "O:utcfromtimestamp", ×tamp)) + if (PyArg_ParseTuple(args, "d:utcfromtimestamp", ×tamp)) result = datetime_from_timestamp(cls, gmtime, timestamp, Py_None); return result; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 18 14:58:35 2015 From: python-checkins at python.org (victor.stinner) Date: Fri, 18 Sep 2015 12:58:35 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Oops=2C_fix_test=5Fmicrose?= =?utf-8?b?Y29uZF9yb3VuZGluZygp?= Message-ID: <20150918125829.82650.82828@psf.io> https://hg.python.org/cpython/rev/c828b730abca changeset: 98050:c828b730abca parent: 98047:820ae8082688 user: Victor Stinner date: Fri Sep 18 14:52:15 2015 +0200 summary: Oops, fix test_microsecond_rounding() Test self.theclass, not datetime. Regression introduced by manual tests. files: Lib/test/datetimetester.py | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Lib/test/datetimetester.py b/Lib/test/datetimetester.py --- a/Lib/test/datetimetester.py +++ b/Lib/test/datetimetester.py @@ -1851,8 +1851,8 @@ 18000 + 3600 + 2*60 + 3 + 4*1e-6) def test_microsecond_rounding(self): - for fts in (datetime.fromtimestamp, - self.theclass.utcfromtimestamp): + for fts in [self.theclass.fromtimestamp, + self.theclass.utcfromtimestamp]: zero = fts(0) self.assertEqual(zero.second, 0) self.assertEqual(zero.microsecond, 0) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 18 14:58:35 2015 From: python-checkins at python.org (victor.stinner) Date: Fri, 18 Sep 2015 12:58:35 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Merge_3=2E4_=28datetime_rounding=29?= Message-ID: <20150918125828.94119.17721@psf.io> https://hg.python.org/cpython/rev/d04a0954e142 changeset: 98049:d04a0954e142 branch: 3.5 parent: 98046:f1cc1f379b00 parent: 98048:ee1cf1b188d2 user: Victor Stinner date: Fri Sep 18 14:50:18 2015 +0200 summary: Merge 3.4 (datetime rounding) files: Lib/datetime.py | 50 +++++++--------- Lib/test/datetimetester.py | 23 +++++-- Misc/NEWS | 8 ++ Modules/_datetimemodule.c | 76 +++++++++++++++++++++---- 4 files changed, 111 insertions(+), 46 deletions(-) diff --git a/Lib/datetime.py b/Lib/datetime.py --- a/Lib/datetime.py +++ b/Lib/datetime.py @@ -1366,6 +1366,26 @@ return self._tzinfo @classmethod + def _fromtimestamp(cls, t, utc, tz): + """Construct a datetime from a POSIX timestamp (like time.time()). + + A timezone info object may be passed in as well. + """ + frac, t = _math.modf(t) + us = round(frac * 1e6) + if us >= 1000000: + t += 1 + us -= 1000000 + elif us < 0: + t -= 1 + us += 1000000 + + converter = _time.gmtime if utc else _time.localtime + y, m, d, hh, mm, ss, weekday, jday, dst = converter(t) + ss = min(ss, 59) # clamp out leap seconds if the platform has them + return cls(y, m, d, hh, mm, ss, us, tz) + + @classmethod def fromtimestamp(cls, t, tz=None): """Construct a datetime from a POSIX timestamp (like time.time()). @@ -1373,21 +1393,7 @@ """ _check_tzinfo_arg(tz) - converter = _time.localtime if tz is None else _time.gmtime - - t, frac = divmod(t, 1.0) - us = int(frac * 1e6) - - # If timestamp is less than one microsecond smaller than a - # full second, us can be rounded up to 1000000. In this case, - # roll over to seconds, otherwise, ValueError is raised - # by the constructor. - if us == 1000000: - t += 1 - us = 0 - y, m, d, hh, mm, ss, weekday, jday, dst = converter(t) - ss = min(ss, 59) # clamp out leap seconds if the platform has them - result = cls(y, m, d, hh, mm, ss, us, tz) + result = cls._fromtimestamp(t, tz is not None, tz) if tz is not None: result = tz.fromutc(result) return result @@ -1395,19 +1401,7 @@ @classmethod def utcfromtimestamp(cls, t): """Construct a naive UTC datetime from a POSIX timestamp.""" - t, frac = divmod(t, 1.0) - us = int(frac * 1e6) - - # If timestamp is less than one microsecond smaller than a - # full second, us can be rounded up to 1000000. In this case, - # roll over to seconds, otherwise, ValueError is raised - # by the constructor. - if us == 1000000: - t += 1 - us = 0 - y, m, d, hh, mm, ss, weekday, jday, dst = _time.gmtime(t) - ss = min(ss, 59) # clamp out leap seconds if the platform has them - return cls(y, m, d, hh, mm, ss, us) + return cls._fromtimestamp(t, True, None) @classmethod def now(cls, tz=None): diff --git a/Lib/test/datetimetester.py b/Lib/test/datetimetester.py --- a/Lib/test/datetimetester.py +++ b/Lib/test/datetimetester.py @@ -663,11 +663,15 @@ eq(td(milliseconds=0.4/1000), td(0)) # rounds to 0 eq(td(milliseconds=-0.4/1000), td(0)) # rounds to 0 eq(td(milliseconds=0.5/1000), td(microseconds=0)) - eq(td(milliseconds=-0.5/1000), td(microseconds=0)) + eq(td(milliseconds=-0.5/1000), td(microseconds=-0)) eq(td(milliseconds=0.6/1000), td(microseconds=1)) eq(td(milliseconds=-0.6/1000), td(microseconds=-1)) + eq(td(milliseconds=1.5/1000), td(microseconds=2)) + eq(td(milliseconds=-1.5/1000), td(microseconds=-2)) eq(td(seconds=0.5/10**6), td(microseconds=0)) - eq(td(seconds=-0.5/10**6), td(microseconds=0)) + eq(td(seconds=-0.5/10**6), td(microseconds=-0)) + eq(td(seconds=1/2**7), td(microseconds=7812)) + eq(td(seconds=-1/2**7), td(microseconds=-7812)) # Rounding due to contributions from more than one field. us_per_hour = 3600e6 @@ -1851,6 +1855,7 @@ zero = fts(0) self.assertEqual(zero.second, 0) self.assertEqual(zero.microsecond, 0) + one = fts(1e-6) try: minus_one = fts(-1e-6) except OSError: @@ -1861,22 +1866,28 @@ self.assertEqual(minus_one.microsecond, 999999) t = fts(-1e-8) - self.assertEqual(t, minus_one) + self.assertEqual(t, zero) t = fts(-9e-7) self.assertEqual(t, minus_one) t = fts(-1e-7) - self.assertEqual(t, minus_one) + self.assertEqual(t, zero) + t = fts(-1/2**7) + self.assertEqual(t.second, 59) + self.assertEqual(t.microsecond, 992188) t = fts(1e-7) self.assertEqual(t, zero) t = fts(9e-7) - self.assertEqual(t, zero) + self.assertEqual(t, one) t = fts(0.99999949) self.assertEqual(t.second, 0) self.assertEqual(t.microsecond, 999999) t = fts(0.9999999) + self.assertEqual(t.second, 1) + self.assertEqual(t.microsecond, 0) + t = fts(1/2**7) self.assertEqual(t.second, 0) - self.assertEqual(t.microsecond, 999999) + self.assertEqual(t.microsecond, 7812) def test_insane_fromtimestamp(self): # It's possible that some platform maps time_t to double, diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -14,6 +14,14 @@ Library ------- +- Issue #23517: Fix rounding in fromtimestamp() and utcfromtimestamp() methods + of datetime.datetime: microseconds are now rounded to nearest with ties + going to nearest even integer (ROUND_HALF_EVEN), instead of being rounding + towards minus infinity (ROUND_FLOOR). It's important that these methods use + the same rounding mode than datetime.timedelta to keep the property: + (datetime(1970,1,1) + timedelta(seconds=t)) == datetime.utcfromtimestamp(t). + It also the rounding mode used by round(float) for example. + - Issue #25155: Fix datetime.datetime.now() and datetime.datetime.utcnow() on Windows to support date after year 2038. It was a regression introduced in Python 3.5.0. diff --git a/Modules/_datetimemodule.c b/Modules/_datetimemodule.c --- a/Modules/_datetimemodule.c +++ b/Modules/_datetimemodule.c @@ -4083,6 +4083,44 @@ tzinfo); } +static time_t +_PyTime_DoubleToTimet(double x) +{ + time_t result; + double diff; + + result = (time_t)x; + /* How much info did we lose? time_t may be an integral or + * floating type, and we don't know which. If it's integral, + * we don't know whether C truncates, rounds, returns the floor, + * etc. If we lost a second or more, the C rounding is + * unreasonable, or the input just doesn't fit in a time_t; + * call it an error regardless. Note that the original cast to + * time_t can cause a C error too, but nothing we can do to + * worm around that. + */ + diff = x - (double)result; + if (diff <= -1.0 || diff >= 1.0) { + PyErr_SetString(PyExc_OverflowError, + "timestamp out of range for platform time_t"); + result = (time_t)-1; + } + return result; +} + +/* Round a double to the nearest long. |x| must be small enough to fit + * in a C long; this is not checked. + */ +static double +_PyTime_RoundHalfEven(double x) +{ + double rounded = round(x); + if (fabs(x-rounded) == 0.5) + /* halfway case: round to even */ + rounded = 2.0*round(x/2.0); + return rounded; +} + /* Internal helper. * Build datetime from a Python timestamp. Pass localtime or gmtime for f, * to control the interpretation of the timestamp. Since a double doesn't @@ -4091,18 +4129,32 @@ * to get that much precision (e.g., C time() isn't good enough). */ static PyObject * -datetime_from_timestamp(PyObject *cls, TM_FUNC f, PyObject *timestamp, +datetime_from_timestamp(PyObject *cls, TM_FUNC f, double timestamp, PyObject *tzinfo) { time_t timet; - long us; - - if (_PyTime_ObjectToTimeval(timestamp, - &timet, &us, _PyTime_ROUND_FLOOR) == -1) + double fraction; + int us; + + timet = _PyTime_DoubleToTimet(timestamp); + if (timet == (time_t)-1 && PyErr_Occurred()) return NULL; - assert(0 <= us && us <= 999999); - - return datetime_from_timet_and_us(cls, f, timet, (int)us, tzinfo); + fraction = timestamp - (double)timet; + us = (int)_PyTime_RoundHalfEven(fraction * 1e6); + if (us < 0) { + /* Truncation towards zero is not what we wanted + for negative numbers (Python's mod semantics) */ + timet -= 1; + us += 1000000; + } + /* If timestamp is less than one microsecond smaller than a + * full second, round up. Otherwise, ValueErrors are raised + * for some floats. */ + if (us == 1000000) { + timet += 1; + us = 0; + } + return datetime_from_timet_and_us(cls, f, timet, us, tzinfo); } /* Internal helper. @@ -4175,11 +4227,11 @@ datetime_fromtimestamp(PyObject *cls, PyObject *args, PyObject *kw) { PyObject *self; - PyObject *timestamp; + double timestamp; PyObject *tzinfo = Py_None; static char *keywords[] = {"timestamp", "tz", NULL}; - if (! PyArg_ParseTupleAndKeywords(args, kw, "O|O:fromtimestamp", + if (! PyArg_ParseTupleAndKeywords(args, kw, "d|O:fromtimestamp", keywords, ×tamp, &tzinfo)) return NULL; if (check_tzinfo_subclass(tzinfo) < 0) @@ -4203,10 +4255,10 @@ static PyObject * datetime_utcfromtimestamp(PyObject *cls, PyObject *args) { - PyObject *timestamp; + double timestamp; PyObject *result = NULL; - if (PyArg_ParseTuple(args, "O:utcfromtimestamp", ×tamp)) + if (PyArg_ParseTuple(args, "d:utcfromtimestamp", ×tamp)) result = datetime_from_timestamp(cls, gmtime, timestamp, Py_None); return result; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 18 15:09:06 2015 From: python-checkins at python.org (victor.stinner) Date: Fri, 18 Sep 2015 13:09:06 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI1MTUw?= =?utf-8?q?=3A_Hide_the_private_=5FPy=5Fatomic=5Fxxx_symbols_from_the_publ?= =?utf-8?q?ic?= Message-ID: <20150918130905.16585.33757@psf.io> https://hg.python.org/cpython/rev/d4fcb362f7c6 changeset: 98052:d4fcb362f7c6 branch: 3.5 parent: 98049:d04a0954e142 user: Victor Stinner date: Fri Sep 18 15:06:34 2015 +0200 summary: Issue #25150: Hide the private _Py_atomic_xxx symbols from the public Python.h header to fix a compilation error with OpenMP. PyThreadState_GET() becomes an alias to PyThreadState_Get() to avoid ABI incompatibilies. It is important that the _PyThreadState_Current variable is always accessed with the same implementation of pyatomic.h. Use the PyThreadState_Get() function so extension modules will all reuse the same implementation. files: Include/pyatomic.h | 6 ++---- Include/pystate.h | 17 +++++------------ Misc/NEWS | 4 ++++ 3 files changed, 11 insertions(+), 16 deletions(-) diff --git a/Include/pyatomic.h b/Include/pyatomic.h --- a/Include/pyatomic.h +++ b/Include/pyatomic.h @@ -1,8 +1,6 @@ -/* Issue #23644: is incompatible with C++, see: - https://gcc.gnu.org/bugzilla/show_bug.cgi?id=60932 */ -#if !defined(Py_LIMITED_API) && !defined(__cplusplus) #ifndef Py_ATOMIC_H #define Py_ATOMIC_H +#ifdef Py_BUILD_CORE #include "dynamic_annotations.h" @@ -248,5 +246,5 @@ #define _Py_atomic_load_relaxed(ATOMIC_VAL) \ _Py_atomic_load_explicit(ATOMIC_VAL, _Py_memory_order_relaxed) +#endif /* Py_BUILD_CORE */ #endif /* Py_ATOMIC_H */ -#endif /* Py_LIMITED_API */ diff --git a/Include/pystate.h b/Include/pystate.h --- a/Include/pystate.h +++ b/Include/pystate.h @@ -177,20 +177,13 @@ /* Variable and macro for in-line access to current thread state */ /* Assuming the current thread holds the GIL, this is the - PyThreadState for the current thread. - - Issue #23644: pyatomic.h is incompatible with C++ (yet). Disable - PyThreadState_GET() optimization: declare it as an alias to - PyThreadState_Get(), as done for limited API. */ -#if !defined(Py_LIMITED_API) && !defined(__cplusplus) + PyThreadState for the current thread. */ +#ifdef Py_BUILD_CORE PyAPI_DATA(_Py_atomic_address) _PyThreadState_Current; -#endif - -#if defined(Py_DEBUG) || defined(Py_LIMITED_API) || defined(__cplusplus) -#define PyThreadState_GET() PyThreadState_Get() +# define PyThreadState_GET() \ + ((PyThreadState*)_Py_atomic_load_relaxed(&_PyThreadState_Current)) #else -#define PyThreadState_GET() \ - ((PyThreadState*)_Py_atomic_load_relaxed(&_PyThreadState_Current)) +# define PyThreadState_GET() PyThreadState_Get() #endif typedef diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -11,6 +11,10 @@ Core and Builtins ----------------- +- Issue #25150: Hide the private _Py_atomic_xxx symbols from the public + Python.h header to fix a compilation error with OpenMP. PyThreadState_GET() + becomes an alias to PyThreadState_Get() to avoid ABI incompatibilies. + Library ------- -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 18 15:09:21 2015 From: python-checkins at python.org (victor.stinner) Date: Fri, 18 Sep 2015 13:09:21 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogTWVyZ2UgMy41?= Message-ID: <20150918130905.81641.76449@psf.io> https://hg.python.org/cpython/rev/8d3ae2fafab1 changeset: 98053:8d3ae2fafab1 parent: 98051:70e18474e0ee parent: 98052:d4fcb362f7c6 user: Victor Stinner date: Fri Sep 18 15:08:14 2015 +0200 summary: Merge 3.5 files: Include/pyatomic.h | 6 ++---- Include/pystate.h | 17 +++++------------ Misc/NEWS | 4 ++++ 3 files changed, 11 insertions(+), 16 deletions(-) diff --git a/Include/pyatomic.h b/Include/pyatomic.h --- a/Include/pyatomic.h +++ b/Include/pyatomic.h @@ -1,8 +1,6 @@ -/* Issue #23644: is incompatible with C++, see: - https://gcc.gnu.org/bugzilla/show_bug.cgi?id=60932 */ -#if !defined(Py_LIMITED_API) && !defined(__cplusplus) #ifndef Py_ATOMIC_H #define Py_ATOMIC_H +#ifdef Py_BUILD_CORE #include "dynamic_annotations.h" @@ -248,5 +246,5 @@ #define _Py_atomic_load_relaxed(ATOMIC_VAL) \ _Py_atomic_load_explicit(ATOMIC_VAL, _Py_memory_order_relaxed) +#endif /* Py_BUILD_CORE */ #endif /* Py_ATOMIC_H */ -#endif /* Py_LIMITED_API */ diff --git a/Include/pystate.h b/Include/pystate.h --- a/Include/pystate.h +++ b/Include/pystate.h @@ -177,20 +177,13 @@ /* Variable and macro for in-line access to current thread state */ /* Assuming the current thread holds the GIL, this is the - PyThreadState for the current thread. - - Issue #23644: pyatomic.h is incompatible with C++ (yet). Disable - PyThreadState_GET() optimization: declare it as an alias to - PyThreadState_Get(), as done for limited API. */ -#if !defined(Py_LIMITED_API) && !defined(__cplusplus) + PyThreadState for the current thread. */ +#ifdef Py_BUILD_CORE PyAPI_DATA(_Py_atomic_address) _PyThreadState_Current; -#endif - -#if defined(Py_DEBUG) || defined(Py_LIMITED_API) || defined(__cplusplus) -#define PyThreadState_GET() PyThreadState_Get() +# define PyThreadState_GET() \ + ((PyThreadState*)_Py_atomic_load_relaxed(&_PyThreadState_Current)) #else -#define PyThreadState_GET() \ - ((PyThreadState*)_Py_atomic_load_relaxed(&_PyThreadState_Current)) +# define PyThreadState_GET() PyThreadState_Get() #endif typedef diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -100,6 +100,10 @@ Core and Builtins ----------------- +- Issue #25150: Hide the private _Py_atomic_xxx symbols from the public + Python.h header to fix a compilation error with OpenMP. PyThreadState_GET() + becomes an alias to PyThreadState_Get() to avoid ABI incompatibilies. + Library ------- -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 18 15:38:59 2015 From: python-checkins at python.org (victor.stinner) Date: Fri, 18 Sep 2015 13:38:59 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2325003=3A_On_Solar?= =?utf-8?q?is_11=2E3_or_newer=2C_os=2Eurandom=28=29_now_uses_the_getrandom?= =?utf-8?b?KCk=?= Message-ID: <20150918133847.81643.44866@psf.io> https://hg.python.org/cpython/rev/8b0c2c7cb4a7 changeset: 98054:8b0c2c7cb4a7 user: Victor Stinner date: Fri Sep 18 15:38:37 2015 +0200 summary: Issue #25003: On Solaris 11.3 or newer, os.urandom() now uses the getrandom() function instead of the getentropy() function. The getentropy() function is blocking to generate very good quality entropy, os.urandom() doesn't need such high-quality entropy. files: Misc/NEWS | 5 +++ Python/random.c | 49 +++++++++++++++++++++++++----------- configure | 43 ++++++++++++++++++++++++++++++-- configure.ac | 31 +++++++++++++++++++++-- pyconfig.h.in | 3 ++ 5 files changed, 110 insertions(+), 21 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -10,6 +10,11 @@ Core and Builtins ----------------- +- Issue #25003: On Solaris 11.3 or newer, os.urandom() now uses the + getrandom() function instead of the getentropy() function. The getentropy() + function is blocking to generate very good quality entropy, os.urandom() + doesn't need such high-quality entropy. + - Issue #9232: Modify Python's grammar to allow trailing commas in the argument list of a function declaration. For example, "def f(*, a = 3,): pass" is now legal. Patch from Mark Dickinson. diff --git a/Python/random.c b/Python/random.c --- a/Python/random.c +++ b/Python/random.c @@ -6,7 +6,9 @@ # ifdef HAVE_SYS_STAT_H # include # endif -# ifdef HAVE_GETRANDOM_SYSCALL +# ifdef HAVE_GETRANDOM +# include +# elif defined(HAVE_GETRANDOM_SYSCALL) # include # endif #endif @@ -70,7 +72,9 @@ return 0; } -#elif HAVE_GETENTROPY +#elif defined(HAVE_GETENTROPY) && !defined(sun) +#define PY_GETENTROPY + /* Fill buffer with size pseudo-random bytes generated by getentropy(). Return 0 on success, or raise an exception and return -1 on error. @@ -105,16 +109,19 @@ return 0; } -#else /* !HAVE_GETENTROPY */ +#else -#ifdef HAVE_GETRANDOM_SYSCALL +#if defined(HAVE_GETRANDOM) || defined(HAVE_GETRANDOM_SYSCALL) +#define PY_GETRANDOM + static int py_getrandom(void *buffer, Py_ssize_t size, int raise) { - /* is getrandom() supported by the running kernel? - * need Linux kernel 3.17 or later */ + /* Is getrandom() supported by the running kernel? + * Need Linux kernel 3.17 or newer, or Solaris 11.3 or newer */ static int getrandom_works = 1; - /* Use /dev/urandom, block if the kernel has no entropy */ + /* Use non-blocking /dev/urandom device. On Linux at boot, the getrandom() + * syscall blocks until /dev/urandom is initialized with enough entropy. */ const int flags = 0; int n; @@ -124,7 +131,18 @@ while (0 < size) { errno = 0; - /* Use syscall() because the libc doesn't expose getrandom() yet, see: +#ifdef HAVE_GETRANDOM + if (raise) { + Py_BEGIN_ALLOW_THREADS + n = getrandom(buffer, size, flags); + Py_END_ALLOW_THREADS + } + else { + n = getrandom(buffer, size, flags); + } +#else + /* On Linux, use the syscall() function because the GNU libc doesn't + * expose the Linux getrandom() syscall yet. See: * https://sourceware.org/bugzilla/show_bug.cgi?id=17252 */ if (raise) { Py_BEGIN_ALLOW_THREADS @@ -134,6 +152,7 @@ else { n = syscall(SYS_getrandom, buffer, size, flags); } +#endif if (n < 0) { if (errno == ENOSYS) { @@ -182,7 +201,7 @@ assert (0 < size); -#ifdef HAVE_GETRANDOM_SYSCALL +#ifdef PY_GETRANDOM if (py_getrandom(buffer, size, 0) == 1) return; /* getrandom() is not supported by the running kernel, fall back @@ -218,14 +237,14 @@ int fd; Py_ssize_t n; struct _Py_stat_struct st; -#ifdef HAVE_GETRANDOM_SYSCALL +#ifdef PY_GETRANDOM int res; #endif if (size <= 0) return 0; -#ifdef HAVE_GETRANDOM_SYSCALL +#ifdef PY_GETRANDOM res = py_getrandom(buffer, size, 1); if (res < 0) return -1; @@ -304,7 +323,7 @@ } } -#endif /* HAVE_GETENTROPY */ +#endif /* Fill buffer with pseudo-random bytes generated by a linear congruent generator (LCG): @@ -345,7 +364,7 @@ #ifdef MS_WINDOWS return win32_urandom((unsigned char *)buffer, size, 1); -#elif HAVE_GETENTROPY +#elif PY_GETENTROPY return py_getentropy(buffer, size, 0); #else return dev_urandom_python((char*)buffer, size); @@ -392,7 +411,7 @@ else { #ifdef MS_WINDOWS (void)win32_urandom(secret, secret_size, 0); -#elif HAVE_GETENTROPY +#elif PY_GETENTROPY (void)py_getentropy(secret, secret_size, 1); #else dev_urandom_noraise(secret, secret_size); @@ -408,7 +427,7 @@ CryptReleaseContext(hCryptProv, 0); hCryptProv = 0; } -#elif HAVE_GETENTROPY +#elif PY_GETENTROPY /* nothing to clean */ #else dev_urandom_close(); diff --git a/configure b/configure --- a/configure +++ b/configure @@ -16005,11 +16005,11 @@ #include int main() { + char buffer[1]; + const size_t buflen = sizeof(buffer); const int flags = 0; - char buffer[1]; - int n; /* ignore the result, Python checks for ENOSYS at runtime */ - (void)syscall(SYS_getrandom, buffer, sizeof(buffer), flags); + (void)syscall(SYS_getrandom, buffer, buflen, flags); return 0; } @@ -16031,6 +16031,43 @@ fi +# check if the getrandom() function is available +# the test was written for the Solaris function of +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for the getrandom() function" >&5 +$as_echo_n "checking for the getrandom() function... " >&6; } +cat confdefs.h - <<_ACEOF >conftest.$ac_ext +/* end confdefs.h. */ + + + #include + + int main() { + char buffer[1]; + const size_t buflen = sizeof(buffer); + const int flags = 0; + /* ignore the result, Python checks for ENOSYS at runtime */ + (void)getrandom(buffer, buflen, flags); + return 0; + } + + +_ACEOF +if ac_fn_c_try_link "$LINENO"; then : + have_getrandom=yes +else + have_getrandom=no +fi +rm -f core conftest.err conftest.$ac_objext \ + conftest$ac_exeext conftest.$ac_ext +{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $have_getrandom" >&5 +$as_echo "$have_getrandom" >&6; } + +if test "$have_getrandom" = yes; then + +$as_echo "#define HAVE_GETRANDOM 1" >>confdefs.h + +fi + # generate output files ac_config_files="$ac_config_files Makefile.pre Modules/Setup.config Misc/python.pc Misc/python-config.sh" diff --git a/configure.ac b/configure.ac --- a/configure.ac +++ b/configure.ac @@ -5111,11 +5111,11 @@ #include int main() { + char buffer[1]; + const size_t buflen = sizeof(buffer); const int flags = 0; - char buffer[1]; - int n; /* ignore the result, Python checks for ENOSYS at runtime */ - (void)syscall(SYS_getrandom, buffer, sizeof(buffer), flags); + (void)syscall(SYS_getrandom, buffer, buflen, flags); return 0; } ]]) @@ -5127,6 +5127,31 @@ [Define to 1 if the Linux getrandom() syscall is available]) fi +# check if the getrandom() function is available +# the test was written for the Solaris function of +AC_MSG_CHECKING(for the getrandom() function) +AC_LINK_IFELSE( +[ + AC_LANG_SOURCE([[ + #include + + int main() { + char buffer[1]; + const size_t buflen = sizeof(buffer); + const int flags = 0; + /* ignore the result, Python checks for ENOSYS at runtime */ + (void)getrandom(buffer, buflen, flags); + return 0; + } + ]]) +],[have_getrandom=yes],[have_getrandom=no]) +AC_MSG_RESULT($have_getrandom) + +if test "$have_getrandom" = yes; then + AC_DEFINE(HAVE_GETRANDOM, 1, + [Define to 1 if the getrandom() function is available]) +fi + # generate output files AC_CONFIG_FILES(Makefile.pre Modules/Setup.config Misc/python.pc Misc/python-config.sh) AC_CONFIG_FILES([Modules/ld_so_aix], [chmod +x Modules/ld_so_aix]) diff --git a/pyconfig.h.in b/pyconfig.h.in --- a/pyconfig.h.in +++ b/pyconfig.h.in @@ -395,6 +395,9 @@ /* Define to 1 if you have the `getpwent' function. */ #undef HAVE_GETPWENT +/* Define to 1 if the getrandom() function is available */ +#undef HAVE_GETRANDOM + /* Define to 1 if the Linux getrandom() syscall is available */ #undef HAVE_GETRANDOM_SYSCALL -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 18 16:26:37 2015 From: python-checkins at python.org (victor.stinner) Date: Fri, 18 Sep 2015 14:26:37 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2325003=3A_Skip_tes?= =?utf-8?q?t=5Fos=2EURandomFDTests_on_Solaris_11=2E3_and_newer?= Message-ID: <20150918142636.94115.94829@psf.io> https://hg.python.org/cpython/rev/221e09b89186 changeset: 98055:221e09b89186 user: Victor Stinner date: Fri Sep 18 16:24:31 2015 +0200 summary: Issue #25003: Skip test_os.URandomFDTests on Solaris 11.3 and newer When os.urandom() is implemented with the getrandom() function, it doesn't use a file descriptor. files: Lib/test/test_os.py | 16 +++++++++------- 1 files changed, 9 insertions(+), 7 deletions(-) diff --git a/Lib/test/test_os.py b/Lib/test/test_os.py --- a/Lib/test/test_os.py +++ b/Lib/test/test_os.py @@ -1225,13 +1225,15 @@ self.assertNotEqual(data1, data2) -HAVE_GETENTROPY = (sysconfig.get_config_var('HAVE_GETENTROPY') == 1) -HAVE_GETRANDOM = (sysconfig.get_config_var('HAVE_GETRANDOM_SYSCALL') == 1) - - at unittest.skipIf(HAVE_GETENTROPY, - "getentropy() does not use a file descriptor") - at unittest.skipIf(HAVE_GETRANDOM, - "getrandom() does not use a file descriptor") +# os.urandom() doesn't use a file descriptor when it is implemented with the +# getentropy() function, the getrandom() function or the getrandom() syscall +OS_URANDOM_DONT_USE_FD = ( + sysconfig.get_config_var('HAVE_GETENTROPY') == 1 + or sysconfig.get_config_var('HAVE_GETRANDOM') == 1 + or sysconfig.get_config_var('HAVE_GETRANDOM_SYSCALL') == 1) + + at unittest.skipIf(OS_URANDOM_DONT_USE_FD , + "os.random() does not use a file descriptor") class URandomFDTests(unittest.TestCase): @unittest.skipUnless(resource, "test requires the resource module") def test_urandom_failure(self): -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 18 16:34:03 2015 From: python-checkins at python.org (victor.stinner) Date: Fri, 18 Sep 2015 14:34:03 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Merge_3=2E4_=28test=5Femail=29?= Message-ID: <20150918143403.115250.11885@psf.io> https://hg.python.org/cpython/rev/165d44ac7894 changeset: 98057:165d44ac7894 branch: 3.5 parent: 98052:d4fcb362f7c6 parent: 98056:9fb47dcd02fd user: Victor Stinner date: Fri Sep 18 16:32:51 2015 +0200 summary: Merge 3.4 (test_email) files: Lib/test/test_email/test_utils.py | 3 +++ 1 files changed, 3 insertions(+), 0 deletions(-) diff --git a/Lib/test/test_email/test_utils.py b/Lib/test/test_email/test_utils.py --- a/Lib/test/test_email/test_utils.py +++ b/Lib/test/test_email/test_utils.py @@ -136,6 +136,9 @@ t1 = utils.localtime(t0) self.assertEqual(t1.tzname(), 'EET') +# Issue #24836: The timezone files are out of date (pre 2011k) +# on Mac OS X Snow Leopard. + at test.support.requires_mac_ver(10, 7) class FormatDateTests(unittest.TestCase): @test.support.run_with_tz('Europe/Minsk') -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 18 16:34:03 2015 From: python-checkins at python.org (victor.stinner) Date: Fri, 18 Sep 2015 14:34:03 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogTWVyZ2UgMy41ICh0ZXN0X2VtYWlsKQ==?= Message-ID: <20150918143403.115327.18880@psf.io> https://hg.python.org/cpython/rev/97f853542dd6 changeset: 98058:97f853542dd6 parent: 98055:221e09b89186 parent: 98057:165d44ac7894 user: Victor Stinner date: Fri Sep 18 16:33:04 2015 +0200 summary: Merge 3.5 (test_email) files: Lib/test/test_email/test_utils.py | 3 +++ 1 files changed, 3 insertions(+), 0 deletions(-) diff --git a/Lib/test/test_email/test_utils.py b/Lib/test/test_email/test_utils.py --- a/Lib/test/test_email/test_utils.py +++ b/Lib/test/test_email/test_utils.py @@ -136,6 +136,9 @@ t1 = utils.localtime(t0) self.assertEqual(t1.tzname(), 'EET') +# Issue #24836: The timezone files are out of date (pre 2011k) +# on Mac OS X Snow Leopard. + at test.support.requires_mac_ver(10, 7) class FormatDateTests(unittest.TestCase): @test.support.run_with_tz('Europe/Minsk') -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 18 16:34:03 2015 From: python-checkins at python.org (victor.stinner) Date: Fri, 18 Sep 2015 14:34:03 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzI0ODM2?= =?utf-8?q?=3A_Skip_FormatDateTests_of_test=5Femail=2Etest=5Futils_on_Mac_?= =?utf-8?q?OS_X_Snow?= Message-ID: <20150918143402.82650.66900@psf.io> https://hg.python.org/cpython/rev/9fb47dcd02fd changeset: 98056:9fb47dcd02fd branch: 3.4 parent: 98048:ee1cf1b188d2 user: Victor Stinner date: Fri Sep 18 16:32:23 2015 +0200 summary: Issue #24836: Skip FormatDateTests of test_email.test_utils on Mac OS X Snow Leopard because this OS uses out of date (pre 2011k) timezone files. files: Lib/test/test_email/test_utils.py | 3 +++ 1 files changed, 3 insertions(+), 0 deletions(-) diff --git a/Lib/test/test_email/test_utils.py b/Lib/test/test_email/test_utils.py --- a/Lib/test/test_email/test_utils.py +++ b/Lib/test/test_email/test_utils.py @@ -136,6 +136,9 @@ t1 = utils.localtime(t0) self.assertEqual(t1.tzname(), 'EET') +# Issue #24836: The timezone files are out of date (pre 2011k) +# on Mac OS X Snow Leopard. + at test.support.requires_mac_ver(10, 7) class FormatDateTests(unittest.TestCase): @test.support.run_with_tz('Europe/Minsk') -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 19 00:11:34 2015 From: python-checkins at python.org (brett.cannon) Date: Fri, 18 Sep 2015 22:11:34 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_Give_proper_cr?= =?utf-8?q?edit_for_issue_=2324915?= Message-ID: <20150918221132.98364.6799@psf.io> https://hg.python.org/cpython/rev/f211c8f554f9 changeset: 98060:f211c8f554f9 branch: 2.7 user: Brett Cannon date: Fri Sep 18 15:11:26 2015 -0700 summary: Give proper credit for issue #24915 files: Misc/ACKS | 1 + Misc/NEWS | 3 ++- 2 files changed, 3 insertions(+), 1 deletions(-) diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -1034,6 +1034,7 @@ Harri Pasanen Ga?l Pasgrimaud Ashish Nitin Patil +Alecsandru Patrascu Randy Pausch Samuele Pedroni Justin Peel diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -151,7 +151,8 @@ ----- - Issue #24915: When doing a PGO build, the test suite is now used instead of - pybench; Clang support was also added as part off this work. + pybench; Clang support was also added as part off this work. Initial patch by + Alecsandru Patrascu of Intel. - Issue #24986: It is now possible to build Python on Windows without errors when external libraries are not available. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 19 00:11:34 2015 From: python-checkins at python.org (brett.cannon) Date: Fri, 18 Sep 2015 22:11:34 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzI0OTE1?= =?utf-8?q?=3A_Make_PGO_builds_support_Clang_and_use_the_test_suite_for?= Message-ID: <20150918221132.81623.31333@psf.io> https://hg.python.org/cpython/rev/0f4e6c303531 changeset: 98059:0f4e6c303531 branch: 2.7 parent: 98034:f6125114b55f user: Brett Cannon date: Fri Sep 18 15:09:42 2015 -0700 summary: Issue #24915: Make PGO builds support Clang and use the test suite for profile data. Thanks to Alecsandru Patrascu of Intel for the initial patch. files: .gitignore | 3 + .hgignore | 3 + Makefile.pre.in | 31 ++++++++++--- Misc/NEWS | 3 + README | 29 +++++++++++++ configure | 82 +++++++++++++++++++++++++++++++++++++ configure.ac | 45 ++++++++++++++++++++ 7 files changed, 189 insertions(+), 7 deletions(-) diff --git a/.gitignore b/.gitignore --- a/.gitignore +++ b/.gitignore @@ -6,6 +6,9 @@ *.pyo *.rej *~ +*.gc?? +*.profclang? +*.profraw Doc/build/ Doc/tools/docutils/ Doc/tools/jinja2/ diff --git a/.hgignore b/.hgignore --- a/.hgignore +++ b/.hgignore @@ -45,6 +45,9 @@ *.pyd *.cover *~ +*.gc?? +*.profclang? +*.profraw Lib/distutils/command/*.pdb Lib/lib2to3/*.pickle Lib/test/data/* diff --git a/Makefile.pre.in b/Makefile.pre.in --- a/Makefile.pre.in +++ b/Makefile.pre.in @@ -42,6 +42,11 @@ HGVERSION= @HGVERSION@ HGTAG= @HGTAG@ HGBRANCH= @HGBRANCH@ +PGO_PROF_GEN_FLAG=@PGO_PROF_GEN_FLAG@ +PGO_PROF_USE_FLAG=@PGO_PROF_USE_FLAG@ +LLVM_PROF_MERGER=@LLVM_PROF_MERGER@ +LLVM_PROF_FILE=@LLVM_PROF_FILE@ +LLVM_PROF_ERR=@LLVM_PROF_ERR@ GNULD= @GNULD@ @@ -204,8 +209,7 @@ TCLTK_LIBS= @TCLTK_LIBS@ # The task to run while instrument when building the profile-opt target -PROFILE_TASK= $(srcdir)/Tools/pybench/pybench.py -n 2 --with-gc --with-syscheck -#PROFILE_TASK= $(srcdir)/Lib/test/regrtest.py +PROFILE_TASK=-m test.regrtest >/dev/null 2>&1 # === Definitions added by makesetup === @@ -420,27 +424,38 @@ all: build_all build_all: $(BUILDPYTHON) oldsharedmods sharedmods gdbhooks -# Compile a binary with gcc profile guided optimization. +# Compile a binary with profile guided optimization. profile-opt: + @if [ $(LLVM_PROF_ERR) == yes ]; then \ + echo "Error: Cannot perform PGO build because llvm-profdata was not found in PATH" ;\ + echo "Please add it to PATH and run ./configure again" ;\ + exit 1;\ + fi @echo "Building with support for profile generation:" $(MAKE) clean + $(MAKE) profile-removal $(MAKE) build_all_generate_profile - @echo "Running benchmark to generate profile data:" $(MAKE) profile-removal + @echo "Running code to generate profile data (this can take a while):" $(MAKE) run_profile_task + $(MAKE) build_all_merge_profile @echo "Rebuilding with profile guided optimizations:" $(MAKE) clean $(MAKE) build_all_use_profile + $(MAKE) profile-removal build_all_generate_profile: - $(MAKE) all CFLAGS="$(CFLAGS) -fprofile-generate" LIBS="$(LIBS) -lgcov" + $(MAKE) all CFLAGS="$(CFLAGS) $(PGO_PROF_GEN_FLAG)" LDFLAGS="$(LDFLAGS) $(PGO_PROF_GEN_FLAG)" LIBS="$(LIBS)" run_profile_task: : # FIXME: can't run for a cross build - ./$(BUILDPYTHON) $(PROFILE_TASK) + $(LLVM_PROF_FILE) ./$(BUILDPYTHON) $(PROFILE_TASK) || true + +build_all_merge_profile: + $(LLVM_PROF_MERGER) build_all_use_profile: - $(MAKE) all CFLAGS="$(CFLAGS) -fprofile-use" + $(MAKE) all CFLAGS="$(CFLAGS) $(PGO_PROF_USE_FLAG)" coverage: @echo "Building with support for coverage checking:" @@ -1330,9 +1345,11 @@ find build -name 'fficonfig.h' -exec rm -f {} ';' || true find build -name 'fficonfig.py' -exec rm -f {} ';' || true -rm -f Lib/lib2to3/*Grammar*.pickle + -rm -rf build profile-removal: find . -name '*.gc??' -exec rm -f {} ';' + find . -name '*.profclang?' -exec rm -f {} ';' clobber: clean profile-removal -rm -f $(BUILDPYTHON) $(PGEN) $(LIBRARY) $(LDLIBRARY) $(DLLLIBRARY) \ diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -150,6 +150,9 @@ Build ----- +- Issue #24915: When doing a PGO build, the test suite is now used instead of + pybench; Clang support was also added as part off this work. + - Issue #24986: It is now possible to build Python on Windows without errors when external libraries are not available. diff --git a/README b/README --- a/README +++ b/README @@ -173,6 +173,11 @@ build your desired target. The interpreter executable is built in the top level directory. +If you need an optimized version of Python, you type "make profile-opt" +in the top level directory. This will rebuild the interpreter executable +using Profile Guided Optimization (PGO). For more details, see the +section below. + Once you have built a Python interpreter, see the subsections below on testing and installation. If you run into trouble, see the next section. @@ -185,6 +190,30 @@ interpreter has been built. +Profile Guided Optimization +--------------------------- + +PGO takes advantage of recent versions of the GCC or Clang compilers. +If ran, the "profile-opt" rule will do several steps. + +First, the entire Python directory is cleaned of temporary files that +may resulted in a previous compilation. + +Then, an instrumented version of the interpreter is built, using suitable +compiler flags for each flavour. Note that this is just an intermediary +step and the binary resulted after this step is not good for real life +workloads, as it has profiling instructions embedded inside. + +After this instrumented version of the interpreter is built, the Makefile +will automatically run a training workload. This is necessary in order to +profile the interpreter execution. Note also that any output, both stdout +and stderr, that may appear at this step is supressed. + +Finally, the last step is to rebuild the interpreter, using the information +collected in the previous one. The end result will be a the Python binary +that is optimized and suitable for distribution or production installation. + + Troubleshooting --------------- diff --git a/configure b/configure --- a/configure +++ b/configure @@ -661,6 +661,12 @@ SO LIBTOOL_CRUFT OTHER_LIBTOOL_OPT +LLVM_PROF_FOUND +LLVM_PROF_ERR +LLVM_PROF_FILE +LLVM_PROF_MERGER +PGO_PROF_USE_FLAG +PGO_PROF_GEN_FLAG UNIVERSAL_ARCH_FLAGS BASECFLAGS OPT @@ -6326,6 +6332,82 @@ CFLAGS=$save_CFLAGS fi + +# Enable PGO flags. +# Extract the first word of "llvm-profdata", so it can be a program name with args. +set dummy llvm-profdata; ac_word=$2 +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 +$as_echo_n "checking for $ac_word... " >&6; } +if ${ac_cv_prog_LLVM_PROF_FOUND+:} false; then : + $as_echo_n "(cached) " >&6 +else + if test -n "$LLVM_PROF_FOUND"; then + ac_cv_prog_LLVM_PROF_FOUND="$LLVM_PROF_FOUND" # Let the user override the test. +else +as_save_IFS=$IFS; IFS=$PATH_SEPARATOR +for as_dir in $PATH +do + IFS=$as_save_IFS + test -z "$as_dir" && as_dir=. + for ac_exec_ext in '' $ac_executable_extensions; do + if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then + ac_cv_prog_LLVM_PROF_FOUND="found" + $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 + break 2 + fi +done + done +IFS=$as_save_IFS + + test -z "$ac_cv_prog_LLVM_PROF_FOUND" && ac_cv_prog_LLVM_PROF_FOUND="not-found" +fi +fi +LLVM_PROF_FOUND=$ac_cv_prog_LLVM_PROF_FOUND +if test -n "$LLVM_PROF_FOUND"; then + { $as_echo "$as_me:${as_lineno-$LINENO}: result: $LLVM_PROF_FOUND" >&5 +$as_echo "$LLVM_PROF_FOUND" >&6; } +else + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 +$as_echo "no" >&6; } +fi + + +LLVM_PROF_ERR=no +case $CC in + *clang*) + # Any changes made here should be reflected in the GCC+Darwin case below + PGO_PROF_GEN_FLAG="-fprofile-instr-generate" + PGO_PROF_USE_FLAG="-fprofile-instr-use=code.profclangd" + LLVM_PROF_MERGER="llvm-profdata merge -output=code.profclangd *.profclangr" + LLVM_PROF_FILE="LLVM_PROFILE_FILE=\"code-%p.profclangr\"" + if test $LLVM_PROF_FOUND = not-found + then + LLVM_PROF_ERR=yes + fi + ;; + *gcc*) + case $ac_sys_system in + Darwin*) + PGO_PROF_GEN_FLAG="-fprofile-instr-generate" + PGO_PROF_USE_FLAG="-fprofile-instr-use=code.profclangd" + LLVM_PROF_MERGER="llvm-profdata merge -output=code.profclangd *.profclangr" + LLVM_PROF_FILE="LLVM_PROFILE_FILE=\"code-%p.profclangr\"" + if test $LLVM_PROF_FOUND = not-found + then + LLVM_PROF_ERR=yes + fi + ;; + *) + PGO_PROF_GEN_FLAG="-fprofile-generate" + PGO_PROF_USE_FLAG="-fprofile-use -fprofile-correction" + LLVM_PROF_MERGER="true" + LLVM_PROF_FILE="" + ;; + esac + ;; +esac + + # On some compilers, pthreads are available without further options # (e.g. MacOS X). On some of these systems, the compiler will not # complain if unaccepted options are passed (e.g. gcc on Mac OS X). diff --git a/configure.ac b/configure.ac --- a/configure.ac +++ b/configure.ac @@ -1352,6 +1352,51 @@ CFLAGS=$save_CFLAGS fi + +# Enable PGO flags. +AC_SUBST(PGO_PROF_GEN_FLAG) +AC_SUBST(PGO_PROF_USE_FLAG) +AC_SUBST(LLVM_PROF_MERGER) +AC_SUBST(LLVM_PROF_FILE) +AC_SUBST(LLVM_PROF_ERR) +AC_SUBST(LLVM_PROF_FOUND) +AC_CHECK_PROG(LLVM_PROF_FOUND, llvm-profdata, found, not-found) +LLVM_PROF_ERR=no +case $CC in + *clang*) + # Any changes made here should be reflected in the GCC+Darwin case below + PGO_PROF_GEN_FLAG="-fprofile-instr-generate" + PGO_PROF_USE_FLAG="-fprofile-instr-use=code.profclangd" + LLVM_PROF_MERGER="llvm-profdata merge -output=code.profclangd *.profclangr" + LLVM_PROF_FILE="LLVM_PROFILE_FILE=\"code-%p.profclangr\"" + if test $LLVM_PROF_FOUND = not-found + then + LLVM_PROF_ERR=yes + fi + ;; + *gcc*) + case $ac_sys_system in + Darwin*) + PGO_PROF_GEN_FLAG="-fprofile-instr-generate" + PGO_PROF_USE_FLAG="-fprofile-instr-use=code.profclangd" + LLVM_PROF_MERGER="llvm-profdata merge -output=code.profclangd *.profclangr" + LLVM_PROF_FILE="LLVM_PROFILE_FILE=\"code-%p.profclangr\"" + if test $LLVM_PROF_FOUND = not-found + then + LLVM_PROF_ERR=yes + fi + ;; + *) + PGO_PROF_GEN_FLAG="-fprofile-generate" + PGO_PROF_USE_FLAG="-fprofile-use -fprofile-correction" + LLVM_PROF_MERGER="true" + LLVM_PROF_FILE="" + ;; + esac + ;; +esac + + # On some compilers, pthreads are available without further options # (e.g. MacOS X). On some of these systems, the compiler will not # complain if unaccepted options are passed (e.g. gcc on Mac OS X). -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 19 00:17:42 2015 From: python-checkins at python.org (brett.cannon) Date: Fri, 18 Sep 2015 22:17:42 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI0OTE1?= =?utf-8?q?=3A_Add_Clang_support_to_PGO_builds_and_use_the_test_suite?= Message-ID: <20150918221742.11692.31482@psf.io> https://hg.python.org/cpython/rev/7fcff838d09e changeset: 98061:7fcff838d09e branch: 3.5 parent: 98057:165d44ac7894 user: Brett Cannon date: Fri Sep 18 15:13:44 2015 -0700 summary: Issue #24915: Add Clang support to PGO builds and use the test suite for profile data. Thanks to Alecsandru Patrascu of Intel for the initial patch. files: .gitignore | 3 + .hgignore | 3 + Makefile.pre.in | 31 +++++++++++--- Misc/ACKS | 1 + Misc/NEWS | 3 + README | 28 ++++++++++++ configure | 80 +++++++++++++++++++++++++++++++++++++ configure.ac | 43 +++++++++++++++++++ 8 files changed, 185 insertions(+), 7 deletions(-) diff --git a/.gitignore b/.gitignore --- a/.gitignore +++ b/.gitignore @@ -11,6 +11,9 @@ *.rej *.swp *~ +*.gc?? +*.profclang? +*.profraw .gdb_history Doc/build/ Doc/venv/ diff --git a/.hgignore b/.hgignore --- a/.hgignore +++ b/.hgignore @@ -50,6 +50,9 @@ *.pyd *.cover *~ +*.gc?? +*.profclang? +*.profraw Lib/distutils/command/*.pdb Lib/lib2to3/*.pickle Lib/test/data/* diff --git a/Makefile.pre.in b/Makefile.pre.in --- a/Makefile.pre.in +++ b/Makefile.pre.in @@ -43,6 +43,11 @@ HGVERSION= @HGVERSION@ HGTAG= @HGTAG@ HGBRANCH= @HGBRANCH@ +PGO_PROF_GEN_FLAG=@PGO_PROF_GEN_FLAG@ +PGO_PROF_USE_FLAG=@PGO_PROF_USE_FLAG@ +LLVM_PROF_MERGER=@LLVM_PROF_MERGER@ +LLVM_PROF_FILE=@LLVM_PROF_FILE@ +LLVM_PROF_ERR=@LLVM_PROF_ERR@ GNULD= @GNULD@ @@ -226,8 +231,7 @@ TCLTK_LIBS= @TCLTK_LIBS@ # The task to run while instrument when building the profile-opt target -PROFILE_TASK= $(srcdir)/Tools/pybench/pybench.py -n 2 --with-gc --with-syscheck -#PROFILE_TASK= $(srcdir)/Lib/test/regrtest.py +PROFILE_TASK=-m test.regrtest >/dev/null 2>&1 # report files for gcov / lcov coverage report COVERAGE_INFO= $(abs_builddir)/coverage.info @@ -477,27 +481,38 @@ all: build_all build_all: $(BUILDPYTHON) oldsharedmods sharedmods gdbhooks Programs/_testembed python-config -# Compile a binary with gcc profile guided optimization. +# Compile a binary with profile guided optimization. profile-opt: + @if [ $(LLVM_PROF_ERR) == yes ]; then \ + echo "Error: Cannot perform PGO build because llvm-profdata was not found in PATH" ;\ + echo "Please add it to PATH and run ./configure again" ;\ + exit 1;\ + fi @echo "Building with support for profile generation:" $(MAKE) clean + $(MAKE) profile-removal $(MAKE) build_all_generate_profile - @echo "Running benchmark to generate profile data:" $(MAKE) profile-removal + @echo "Running code to generate profile data (this can take a while):" $(MAKE) run_profile_task + $(MAKE) build_all_merge_profile @echo "Rebuilding with profile guided optimizations:" $(MAKE) clean $(MAKE) build_all_use_profile + $(MAKE) profile-removal build_all_generate_profile: - $(MAKE) all CFLAGS_NODIST="$(CFLAGS) -fprofile-generate" LDFLAGS="-fprofile-generate" LIBS="$(LIBS) -lgcov" + $(MAKE) all CFLAGS_NODIST="$(CFLAGS) $(PGO_PROF_GEN_FLAG)" LDFLAGS="$(LDFLAGS) $(PGO_PROF_GEN_FLAG)" LIBS="$(LIBS)" run_profile_task: : # FIXME: can't run for a cross build - $(RUNSHARED) ./$(BUILDPYTHON) $(PROFILE_TASK) + $(LLVM_PROF_FILE) $(RUNSHARED) ./$(BUILDPYTHON) $(PROFILE_TASK) || true + +build_all_merge_profile: + $(LLVM_PROF_MERGER) build_all_use_profile: - $(MAKE) all CFLAGS_NODIST="$(CFLAGS) -fprofile-use -fprofile-correction" + $(MAKE) all CFLAGS_NODIST="$(CFLAGS) $(PGO_PROF_USE_FLAG)" # Compile and run with gcov .PHONY=coverage coverage-lcov coverage-report @@ -1568,9 +1583,11 @@ -rm -f pybuilddir.txt -rm -f Lib/lib2to3/*Grammar*.pickle -rm -f Programs/_testembed Programs/_freeze_importlib + -rm -rf build profile-removal: find . -name '*.gc??' -exec rm -f {} ';' + find . -name '*.profclang?' -exec rm -f {} ';' rm -f $(COVERAGE_INFO) rm -rf $(COVERAGE_REPORT) diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -1079,6 +1079,7 @@ Harri Pasanen Ga?l Pasgrimaud Ashish Nitin Patil +Alecsandru Patrascu Randy Pausch Samuele Pedroni Justin Peel diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -115,6 +115,9 @@ Build ----- +- Issue #24915: Add LLVM support for PGO builds and use the test suite to + generate the profile data. Initiial patch by Alecsandru Patrascu of Intel. + - Issue #24910: Windows MSIs now have unique display names. - Issue #24986: It is now possible to build Python on Windows without errors diff --git a/README b/README --- a/README +++ b/README @@ -46,6 +46,34 @@ (This will fail if you *also* built at the top-level directory. You should do a "make clean" at the toplevel first.) +If you need an optimized version of Python, you type "make profile-opt" in the +top level directory. This will rebuild the interpreter executable using Profile +Guided Optimization (PGO). For more details, see the section bellow. + + +Profile Guided Optimization +--------------------------- + +PGO takes advantage of recent versions of the GCC or Clang compilers. +If ran, the "profile-opt" rule will do several steps. + +First, the entire Python directory is cleaned of temporary files that +may resulted in a previous compilation. + +Then, an instrumented version of the interpreter is built, using suitable +compiler flags for each flavour. Note that this is just an intermediary +step and the binary resulted after this step is not good for real life +workloads, as it has profiling instructions embedded inside. + +After this instrumented version of the interpreter is built, the Makefile +will automatically run a training workload. This is necessary in order to +profile the interpreter execution. Note also that any output, both stdout +and stderr, that may appear at this step is supressed. + +Finally, the last step is to rebuild the interpreter, using the information +collected in the previous one. The end result will be a the Python binary +that is optimized and suitable for distribution or production installation. + What's New ---------- diff --git a/configure b/configure --- a/configure +++ b/configure @@ -667,6 +667,12 @@ CFLAGS_NODIST BASECFLAGS OPT +LLVM_PROF_FOUND +LLVM_PROF_ERR +LLVM_PROF_FILE +LLVM_PROF_MERGER +PGO_PROF_USE_FLAG +PGO_PROF_GEN_FLAG ABIFLAGS LN MKDIR_P @@ -6431,6 +6437,80 @@ fi +# Enable PGO flags. +# Extract the first word of "llvm-profdata", so it can be a program name with args. +set dummy llvm-profdata; ac_word=$2 +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 +$as_echo_n "checking for $ac_word... " >&6; } +if ${ac_cv_prog_LLVM_PROF_FOUND+:} false; then : + $as_echo_n "(cached) " >&6 +else + if test -n "$LLVM_PROF_FOUND"; then + ac_cv_prog_LLVM_PROF_FOUND="$LLVM_PROF_FOUND" # Let the user override the test. +else +as_save_IFS=$IFS; IFS=$PATH_SEPARATOR +for as_dir in $PATH +do + IFS=$as_save_IFS + test -z "$as_dir" && as_dir=. + for ac_exec_ext in '' $ac_executable_extensions; do + if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then + ac_cv_prog_LLVM_PROF_FOUND="found" + $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 + break 2 + fi +done + done +IFS=$as_save_IFS + + test -z "$ac_cv_prog_LLVM_PROF_FOUND" && ac_cv_prog_LLVM_PROF_FOUND="not-found" +fi +fi +LLVM_PROF_FOUND=$ac_cv_prog_LLVM_PROF_FOUND +if test -n "$LLVM_PROF_FOUND"; then + { $as_echo "$as_me:${as_lineno-$LINENO}: result: $LLVM_PROF_FOUND" >&5 +$as_echo "$LLVM_PROF_FOUND" >&6; } +else + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 +$as_echo "no" >&6; } +fi + + +LLVM_PROF_ERR=no +case $CC in + *clang*) + # Any changes made here should be reflected in the GCC+Darwin case below + PGO_PROF_GEN_FLAG="-fprofile-instr-generate" + PGO_PROF_USE_FLAG="-fprofile-instr-use=code.profclangd" + LLVM_PROF_MERGER="llvm-profdata merge -output=code.profclangd *.profclangr" + LLVM_PROF_FILE="LLVM_PROFILE_FILE=\"code-%p.profclangr\"" + if test $LLVM_PROF_FOUND = not-found + then + LLVM_PROF_ERR=yes + fi + ;; + *gcc*) + case $ac_sys_system in + Darwin*) + PGO_PROF_GEN_FLAG="-fprofile-instr-generate" + PGO_PROF_USE_FLAG="-fprofile-instr-use=code.profclangd" + LLVM_PROF_MERGER="llvm-profdata merge -output=code.profclangd *.profclangr" + LLVM_PROF_FILE="LLVM_PROFILE_FILE=\"code-%p.profclangr\"" + if test $LLVM_PROF_FOUND = not-found + then + LLVM_PROF_ERR=yes + fi + ;; + *) + PGO_PROF_GEN_FLAG="-fprofile-generate" + PGO_PROF_USE_FLAG="-fprofile-use -fprofile-correction" + LLVM_PROF_MERGER="true" + LLVM_PROF_FILE="" + ;; + esac + ;; +esac + # XXX Shouldn't the code above that fiddles with BASECFLAGS and OPT be # merged with this chunk of code? diff --git a/configure.ac b/configure.ac --- a/configure.ac +++ b/configure.ac @@ -1218,6 +1218,49 @@ fi], [AC_MSG_RESULT(no)]) +# Enable PGO flags. +AC_SUBST(PGO_PROF_GEN_FLAG) +AC_SUBST(PGO_PROF_USE_FLAG) +AC_SUBST(LLVM_PROF_MERGER) +AC_SUBST(LLVM_PROF_FILE) +AC_SUBST(LLVM_PROF_ERR) +AC_SUBST(LLVM_PROF_FOUND) +AC_CHECK_PROG(LLVM_PROF_FOUND, llvm-profdata, found, not-found) +LLVM_PROF_ERR=no +case $CC in + *clang*) + # Any changes made here should be reflected in the GCC+Darwin case below + PGO_PROF_GEN_FLAG="-fprofile-instr-generate" + PGO_PROF_USE_FLAG="-fprofile-instr-use=code.profclangd" + LLVM_PROF_MERGER="llvm-profdata merge -output=code.profclangd *.profclangr" + LLVM_PROF_FILE="LLVM_PROFILE_FILE=\"code-%p.profclangr\"" + if test $LLVM_PROF_FOUND = not-found + then + LLVM_PROF_ERR=yes + fi + ;; + *gcc*) + case $ac_sys_system in + Darwin*) + PGO_PROF_GEN_FLAG="-fprofile-instr-generate" + PGO_PROF_USE_FLAG="-fprofile-instr-use=code.profclangd" + LLVM_PROF_MERGER="llvm-profdata merge -output=code.profclangd *.profclangr" + LLVM_PROF_FILE="LLVM_PROFILE_FILE=\"code-%p.profclangr\"" + if test $LLVM_PROF_FOUND = not-found + then + LLVM_PROF_ERR=yes + fi + ;; + *) + PGO_PROF_GEN_FLAG="-fprofile-generate" + PGO_PROF_USE_FLAG="-fprofile-use -fprofile-correction" + LLVM_PROF_MERGER="true" + LLVM_PROF_FILE="" + ;; + esac + ;; +esac + # XXX Shouldn't the code above that fiddles with BASECFLAGS and OPT be # merged with this chunk of code? -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 19 00:17:44 2015 From: python-checkins at python.org (brett.cannon) Date: Fri, 18 Sep 2015 22:17:44 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge_for_issue_=2324915?= Message-ID: <20150918221742.9939.91089@psf.io> https://hg.python.org/cpython/rev/7749fc0a5ea6 changeset: 98062:7749fc0a5ea6 parent: 98058:97f853542dd6 parent: 98061:7fcff838d09e user: Brett Cannon date: Fri Sep 18 15:17:37 2015 -0700 summary: Merge for issue #24915 files: .gitignore | 3 + .hgignore | 3 + Makefile.pre.in | 31 +++++++++++--- Misc/ACKS | 1 + Misc/NEWS | 3 + README | 28 ++++++++++++ configure | 80 +++++++++++++++++++++++++++++++++++++ configure.ac | 43 +++++++++++++++++++ 8 files changed, 185 insertions(+), 7 deletions(-) diff --git a/.gitignore b/.gitignore --- a/.gitignore +++ b/.gitignore @@ -11,6 +11,9 @@ *.rej *.swp *~ +*.gc?? +*.profclang? +*.profraw .gdb_history Doc/build/ Doc/venv/ diff --git a/.hgignore b/.hgignore --- a/.hgignore +++ b/.hgignore @@ -50,6 +50,9 @@ *.pyd *.cover *~ +*.gc?? +*.profclang? +*.profraw Lib/distutils/command/*.pdb Lib/lib2to3/*.pickle Lib/test/data/* diff --git a/Makefile.pre.in b/Makefile.pre.in --- a/Makefile.pre.in +++ b/Makefile.pre.in @@ -43,6 +43,11 @@ HGVERSION= @HGVERSION@ HGTAG= @HGTAG@ HGBRANCH= @HGBRANCH@ +PGO_PROF_GEN_FLAG=@PGO_PROF_GEN_FLAG@ +PGO_PROF_USE_FLAG=@PGO_PROF_USE_FLAG@ +LLVM_PROF_MERGER=@LLVM_PROF_MERGER@ +LLVM_PROF_FILE=@LLVM_PROF_FILE@ +LLVM_PROF_ERR=@LLVM_PROF_ERR@ GNULD= @GNULD@ @@ -226,8 +231,7 @@ TCLTK_LIBS= @TCLTK_LIBS@ # The task to run while instrument when building the profile-opt target -PROFILE_TASK= $(srcdir)/Tools/pybench/pybench.py -n 2 --with-gc --with-syscheck -#PROFILE_TASK= $(srcdir)/Lib/test/regrtest.py +PROFILE_TASK=-m test.regrtest >/dev/null 2>&1 # report files for gcov / lcov coverage report COVERAGE_INFO= $(abs_builddir)/coverage.info @@ -477,27 +481,38 @@ all: build_all build_all: $(BUILDPYTHON) oldsharedmods sharedmods gdbhooks Programs/_testembed python-config -# Compile a binary with gcc profile guided optimization. +# Compile a binary with profile guided optimization. profile-opt: + @if [ $(LLVM_PROF_ERR) == yes ]; then \ + echo "Error: Cannot perform PGO build because llvm-profdata was not found in PATH" ;\ + echo "Please add it to PATH and run ./configure again" ;\ + exit 1;\ + fi @echo "Building with support for profile generation:" $(MAKE) clean + $(MAKE) profile-removal $(MAKE) build_all_generate_profile - @echo "Running benchmark to generate profile data:" $(MAKE) profile-removal + @echo "Running code to generate profile data (this can take a while):" $(MAKE) run_profile_task + $(MAKE) build_all_merge_profile @echo "Rebuilding with profile guided optimizations:" $(MAKE) clean $(MAKE) build_all_use_profile + $(MAKE) profile-removal build_all_generate_profile: - $(MAKE) all CFLAGS_NODIST="$(CFLAGS) -fprofile-generate" LDFLAGS="-fprofile-generate" LIBS="$(LIBS) -lgcov" + $(MAKE) all CFLAGS_NODIST="$(CFLAGS) $(PGO_PROF_GEN_FLAG)" LDFLAGS="$(LDFLAGS) $(PGO_PROF_GEN_FLAG)" LIBS="$(LIBS)" run_profile_task: : # FIXME: can't run for a cross build - $(RUNSHARED) ./$(BUILDPYTHON) $(PROFILE_TASK) + $(LLVM_PROF_FILE) $(RUNSHARED) ./$(BUILDPYTHON) $(PROFILE_TASK) || true + +build_all_merge_profile: + $(LLVM_PROF_MERGER) build_all_use_profile: - $(MAKE) all CFLAGS_NODIST="$(CFLAGS) -fprofile-use -fprofile-correction" + $(MAKE) all CFLAGS_NODIST="$(CFLAGS) $(PGO_PROF_USE_FLAG)" # Compile and run with gcov .PHONY=coverage coverage-lcov coverage-report @@ -1568,9 +1583,11 @@ -rm -f pybuilddir.txt -rm -f Lib/lib2to3/*Grammar*.pickle -rm -f Programs/_testembed Programs/_freeze_importlib + -rm -rf build profile-removal: find . -name '*.gc??' -exec rm -f {} ';' + find . -name '*.profclang?' -exec rm -f {} ';' rm -f $(COVERAGE_INFO) rm -rf $(COVERAGE_REPORT) diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -1080,6 +1080,7 @@ Harri Pasanen Ga?l Pasgrimaud Ashish Nitin Patil +Alecsandru Patrascu Randy Pausch Samuele Pedroni Justin Peel diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -201,6 +201,9 @@ Build ----- +- Issue #24915: Add LLVM support for PGO builds and use the test suite to + generate the profile data. Initiial patch by Alecsandru Patrascu of Intel. + - Issue #24910: Windows MSIs now have unique display names. - Issue #24986: It is now possible to build Python on Windows without errors diff --git a/README b/README --- a/README +++ b/README @@ -46,6 +46,34 @@ (This will fail if you *also* built at the top-level directory. You should do a "make clean" at the toplevel first.) +If you need an optimized version of Python, you type "make profile-opt" in the +top level directory. This will rebuild the interpreter executable using Profile +Guided Optimization (PGO). For more details, see the section bellow. + + +Profile Guided Optimization +--------------------------- + +PGO takes advantage of recent versions of the GCC or Clang compilers. +If ran, the "profile-opt" rule will do several steps. + +First, the entire Python directory is cleaned of temporary files that +may resulted in a previous compilation. + +Then, an instrumented version of the interpreter is built, using suitable +compiler flags for each flavour. Note that this is just an intermediary +step and the binary resulted after this step is not good for real life +workloads, as it has profiling instructions embedded inside. + +After this instrumented version of the interpreter is built, the Makefile +will automatically run a training workload. This is necessary in order to +profile the interpreter execution. Note also that any output, both stdout +and stderr, that may appear at this step is supressed. + +Finally, the last step is to rebuild the interpreter, using the information +collected in the previous one. The end result will be a the Python binary +that is optimized and suitable for distribution or production installation. + What's New ---------- diff --git a/configure b/configure --- a/configure +++ b/configure @@ -667,6 +667,12 @@ CFLAGS_NODIST BASECFLAGS OPT +LLVM_PROF_FOUND +LLVM_PROF_ERR +LLVM_PROF_FILE +LLVM_PROF_MERGER +PGO_PROF_USE_FLAG +PGO_PROF_GEN_FLAG ABIFLAGS LN MKDIR_P @@ -6431,6 +6437,80 @@ fi +# Enable PGO flags. +# Extract the first word of "llvm-profdata", so it can be a program name with args. +set dummy llvm-profdata; ac_word=$2 +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 +$as_echo_n "checking for $ac_word... " >&6; } +if ${ac_cv_prog_LLVM_PROF_FOUND+:} false; then : + $as_echo_n "(cached) " >&6 +else + if test -n "$LLVM_PROF_FOUND"; then + ac_cv_prog_LLVM_PROF_FOUND="$LLVM_PROF_FOUND" # Let the user override the test. +else +as_save_IFS=$IFS; IFS=$PATH_SEPARATOR +for as_dir in $PATH +do + IFS=$as_save_IFS + test -z "$as_dir" && as_dir=. + for ac_exec_ext in '' $ac_executable_extensions; do + if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then + ac_cv_prog_LLVM_PROF_FOUND="found" + $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 + break 2 + fi +done + done +IFS=$as_save_IFS + + test -z "$ac_cv_prog_LLVM_PROF_FOUND" && ac_cv_prog_LLVM_PROF_FOUND="not-found" +fi +fi +LLVM_PROF_FOUND=$ac_cv_prog_LLVM_PROF_FOUND +if test -n "$LLVM_PROF_FOUND"; then + { $as_echo "$as_me:${as_lineno-$LINENO}: result: $LLVM_PROF_FOUND" >&5 +$as_echo "$LLVM_PROF_FOUND" >&6; } +else + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 +$as_echo "no" >&6; } +fi + + +LLVM_PROF_ERR=no +case $CC in + *clang*) + # Any changes made here should be reflected in the GCC+Darwin case below + PGO_PROF_GEN_FLAG="-fprofile-instr-generate" + PGO_PROF_USE_FLAG="-fprofile-instr-use=code.profclangd" + LLVM_PROF_MERGER="llvm-profdata merge -output=code.profclangd *.profclangr" + LLVM_PROF_FILE="LLVM_PROFILE_FILE=\"code-%p.profclangr\"" + if test $LLVM_PROF_FOUND = not-found + then + LLVM_PROF_ERR=yes + fi + ;; + *gcc*) + case $ac_sys_system in + Darwin*) + PGO_PROF_GEN_FLAG="-fprofile-instr-generate" + PGO_PROF_USE_FLAG="-fprofile-instr-use=code.profclangd" + LLVM_PROF_MERGER="llvm-profdata merge -output=code.profclangd *.profclangr" + LLVM_PROF_FILE="LLVM_PROFILE_FILE=\"code-%p.profclangr\"" + if test $LLVM_PROF_FOUND = not-found + then + LLVM_PROF_ERR=yes + fi + ;; + *) + PGO_PROF_GEN_FLAG="-fprofile-generate" + PGO_PROF_USE_FLAG="-fprofile-use -fprofile-correction" + LLVM_PROF_MERGER="true" + LLVM_PROF_FILE="" + ;; + esac + ;; +esac + # XXX Shouldn't the code above that fiddles with BASECFLAGS and OPT be # merged with this chunk of code? diff --git a/configure.ac b/configure.ac --- a/configure.ac +++ b/configure.ac @@ -1218,6 +1218,49 @@ fi], [AC_MSG_RESULT(no)]) +# Enable PGO flags. +AC_SUBST(PGO_PROF_GEN_FLAG) +AC_SUBST(PGO_PROF_USE_FLAG) +AC_SUBST(LLVM_PROF_MERGER) +AC_SUBST(LLVM_PROF_FILE) +AC_SUBST(LLVM_PROF_ERR) +AC_SUBST(LLVM_PROF_FOUND) +AC_CHECK_PROG(LLVM_PROF_FOUND, llvm-profdata, found, not-found) +LLVM_PROF_ERR=no +case $CC in + *clang*) + # Any changes made here should be reflected in the GCC+Darwin case below + PGO_PROF_GEN_FLAG="-fprofile-instr-generate" + PGO_PROF_USE_FLAG="-fprofile-instr-use=code.profclangd" + LLVM_PROF_MERGER="llvm-profdata merge -output=code.profclangd *.profclangr" + LLVM_PROF_FILE="LLVM_PROFILE_FILE=\"code-%p.profclangr\"" + if test $LLVM_PROF_FOUND = not-found + then + LLVM_PROF_ERR=yes + fi + ;; + *gcc*) + case $ac_sys_system in + Darwin*) + PGO_PROF_GEN_FLAG="-fprofile-instr-generate" + PGO_PROF_USE_FLAG="-fprofile-instr-use=code.profclangd" + LLVM_PROF_MERGER="llvm-profdata merge -output=code.profclangd *.profclangr" + LLVM_PROF_FILE="LLVM_PROFILE_FILE=\"code-%p.profclangr\"" + if test $LLVM_PROF_FOUND = not-found + then + LLVM_PROF_ERR=yes + fi + ;; + *) + PGO_PROF_GEN_FLAG="-fprofile-generate" + PGO_PROF_USE_FLAG="-fprofile-use -fprofile-correction" + LLVM_PROF_MERGER="true" + LLVM_PROF_FILE="" + ;; + esac + ;; +esac + # XXX Shouldn't the code above that fiddles with BASECFLAGS and OPT be # merged with this chunk of code? -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 19 00:21:28 2015 From: python-checkins at python.org (brett.cannon) Date: Fri, 18 Sep 2015 22:21:28 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge_for_issue_=2325133?= Message-ID: <20150918222128.9929.51860@psf.io> https://hg.python.org/cpython/rev/f03c074b6242 changeset: 98064:f03c074b6242 parent: 98062:7749fc0a5ea6 parent: 98063:8054a9f788e7 user: Brett Cannon date: Fri Sep 18 15:21:22 2015 -0700 summary: Merge for issue #25133 files: Doc/library/selectors.rst | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Doc/library/selectors.rst b/Doc/library/selectors.rst --- a/Doc/library/selectors.rst +++ b/Doc/library/selectors.rst @@ -50,8 +50,8 @@ In the following, *events* is a bitwise mask indicating which I/O events should -be waited for on a given file object. It can be a combination of the constants -below: +be waited for on a given file object. It can be a combination of the modules +constants below: +-----------------------+-----------------------------------------------+ | Constant | Meaning | -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 19 00:21:28 2015 From: python-checkins at python.org (brett.cannon) Date: Fri, 18 Sep 2015 22:21:28 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_Make_it_cleare?= =?utf-8?q?r_that_the_constants_in_the_selectors_docs_are_module-level?= Message-ID: <20150918222128.9931.38196@psf.io> https://hg.python.org/cpython/rev/8054a9f788e7 changeset: 98063:8054a9f788e7 branch: 3.5 parent: 98061:7fcff838d09e user: Brett Cannon date: Fri Sep 18 15:21:02 2015 -0700 summary: Make it clearer that the constants in the selectors docs are module-level files: Doc/library/selectors.rst | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Doc/library/selectors.rst b/Doc/library/selectors.rst --- a/Doc/library/selectors.rst +++ b/Doc/library/selectors.rst @@ -50,8 +50,8 @@ In the following, *events* is a bitwise mask indicating which I/O events should -be waited for on a given file object. It can be a combination of the constants -below: +be waited for on a given file object. It can be a combination of the modules +constants below: +-----------------------+-----------------------------------------------+ | Constant | Meaning | -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 19 04:11:08 2015 From: python-checkins at python.org (chris.angelico) Date: Sat, 19 Sep 2015 02:11:08 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_Create_PEP_505_for_None-aware?= =?utf-8?q?_operators?= Message-ID: <20150919021108.94131.31326@psf.io> https://hg.python.org/peps/rev/bdd5fc534c29 changeset: 6068:bdd5fc534c29 user: Chris Angelico date: Sat Sep 19 12:10:53 2015 +1000 summary: Create PEP 505 for None-aware operators files: pep-0505.txt | 199 +++++++++++++++++++++++++++++++++++++++ 1 files changed, 199 insertions(+), 0 deletions(-) diff --git a/pep-0505.txt b/pep-0505.txt new file mode 100644 --- /dev/null +++ b/pep-0505.txt @@ -0,0 +1,199 @@ +PEP: 505 +Title: None coalescing operators +Version: $Revision$ +Last-Modified: $Date$ +Author: Mark E. Haase +Status: Draft +Type: Standards Track +Content-Type: text/x-rst +Created: 18-Sep-2015 +Python-Version: 3.6 + +Abstract +======== + +Several modern programming languages have so-called "null coalescing" or +"null aware" operators, including C#, Dart, Perl, Swift, and PHP (starting in +version 7). These operators provide syntactic sugar for common patterns +involving null references. [1]_ [2]_ + +* The "null coalescing" operator is a binary operator that returns its first + first non-null operand. +* The "null aware member access" operator is a binary operator that accesses + an instance member only if that instance is non-null. It returns null + otherwise. +* The "null aware index access" operator is a binary operator that accesses a + member of a collection only if that collection is non-null. It returns null + otherwise. + +Python does not have any directly equivalent syntax. The ``or`` operator can +be used to similar effect but checks for a truthy value, not ``None`` +specifically. The ternary operator ``... if ... else ...`` can be used for +explicit null checks but is more verbose and typically duplicates part of the +expression in between ``if`` and ``else``. The proposed ``None`` coalescing +and ``None`` aware operators ofter an alternative syntax that is more +intuitive and concise. + + +Rationale +========= + +Null Coalescing Operator +------------------------ + +The following code illustrates how the ``None`` coalescing operators would +work in Python:: + + >>> title = 'My Title' + >>> title ?? 'Default Title' + 'My Title' + >>> title = None + >>> title ?? 'Bar' + 'Default Title' + +Similar behavior can be achieved with the ``or`` operator, but ``or`` checks +whether its left operand is false-y, not specifically ``None``. This can lead +to surprising behavior. Consider the scenario of computing the price of some +products a customer has in his/her shopping cart:: + + >>> price = 100 + >>> requested_quantity = 5 + >>> default_quantity = 1 + >>> (requested_quantity or default_quantity) * price + 500 + >>> requested_quantity = None + >>> (requested_quantity or default_quantity) * price + 100 + >>> requested_quantity = 0 + >>> (requested_quantity or default_quantity) * price # oops! + 100 + +This type of bug is not possible with the ``None`` coalescing operator, +because there is no implicit type coersion to ``bool``:: + + >>> price = 100 + >>> requested_quantity = 0 + >>> default_quantity = 1 + >>> (requested_quantity ?? default_quantity) * price + 0 + +The same correct behavior can be achieved with the ternary operator. Here is +an excerpt from the popular Requests package:: + + data = [] if data is None else data + files = [] if files is None else files + headers = {} if headers is None else headers + params = {} if params is None else params + hooks = {} if hooks is None else hooks + +This particular formulation has the undesirable effect of putting the operands +in an unintuitive order: the brain thinks, "use ``data`` if possible and use +``[]`` as a fallback," but the code puts the fallback _before_ the preferred +value. + +The author of this package could have written it like this instead:: + + data = data if data is not None else [] + files = files if files is not None else [] + headers = headers if headers is not None else {} + params = params if params is not None else {} + hooks = hooks if hooks is not None else {} + +This ordering of the operands is more intuitive, but it requires 4 extra +characters (for "not "). It also highlights the repetition of identifiers: +``data if data``, ``files if files``, etc. The ``None`` coalescing operator +improves readability:: + + data = data ?? [] + files = files ?? [] + headers = headers ?? {} + params = params ?? {} + hooks = hooks ?? {} + +The ``None`` coalescing operator also has a corresponding assignment shortcut. + +:: + + data ??= [] + files ??= [] + headers ??= {} + params ??= {} + hooks ??= {} + +The ``None`` coalescing operator is left-associative, which allows for easy +chaining:: + + >>> user_title = None + >>> local_default_title = None + >>> global_default_title = 'Global Default Title' + >>> title = user_title ?? local_default_title ?? global_default_title + 'My Title' + +The direction of associativity is important because the ``None`` coalesing +operator short circuits: if it's left operand is non-null, then the right +operand is not evaluated. + + >>> def get_default(): raise Exception() + >>> 'My Title' ?? get_default() + 'My Title' + + +Null-Aware Member Access Operator +--------------------------------- + + >>> title = 'My Title' + >>> title.upper() + 'MY TITLE' + >>> title = None + >>> title.upper() + Traceback (most recent call last): + File "", line 1, in + AttributeError: 'NoneType' object has no attribute 'upper' + >>> title?.upper() + None + + +Null-Aware Index Access Operator +--------------------------------- + + >>> person = {'name': 'Mark', 'age': 32} + >>> person['name'] + 'Mark' + >>> person = None + >>> person['name'] + Traceback (most recent call last): + File "", line 1, in + TypeError: 'NoneType' object is not subscriptable + >>> person?['name'] + None + + +Specification +============= + + +References +========== + +.. [1] Wikipedia: Null coalescing operator + (https://en.wikipedia.org/wiki/Null_coalescing_operator) + +.. [2] Seth Ladd's Blog: Null-aware operators in Dart + (http://blog.sethladd.com/2015/07/null-aware-operators-in-dart.html) + + +Copyright +========= + +This document has been placed in the public domain. + + + +.. + Local Variables: + mode: indented-text + indent-tabs-mode: nil + sentence-end-double-space: t + fill-column: 70 + coding: utf-8 + End: -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Sat Sep 19 04:20:39 2015 From: python-checkins at python.org (chris.angelico) Date: Sat, 19 Sep 2015 02:20:39 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_Copyedit_PEP_505_=28one_bit_I?= =?utf-8?q?_missed_in_the_first_pass=29?= Message-ID: <20150919022039.3672.87359@psf.io> https://hg.python.org/peps/rev/65e0bb82aed7 changeset: 6069:65e0bb82aed7 user: Chris Angelico date: Sat Sep 19 12:20:26 2015 +1000 summary: Copyedit PEP 505 (one bit I missed in the first pass) files: pep-0505.txt | 4 +++- 1 files changed, 3 insertions(+), 1 deletions(-) diff --git a/pep-0505.txt b/pep-0505.txt --- a/pep-0505.txt +++ b/pep-0505.txt @@ -130,9 +130,11 @@ 'My Title' The direction of associativity is important because the ``None`` coalesing -operator short circuits: if it's left operand is non-null, then the right +operator short circuits: if its left operand is non-null, then the right operand is not evaluated. +:: + >>> def get_default(): raise Exception() >>> 'My Title' ?? get_default() 'My Title' -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Sat Sep 19 08:21:39 2015 From: python-checkins at python.org (raymond.hettinger) Date: Sat, 19 Sep 2015 06:21:39 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Hoist_constant_expression_?= =?utf-8?q?out_of_an_inner_loop?= Message-ID: <20150919062139.115327.42801@psf.io> https://hg.python.org/cpython/rev/fb9d0fffb494 changeset: 98065:fb9d0fffb494 user: Raymond Hettinger date: Sat Sep 19 00:21:33 2015 -0600 summary: Hoist constant expression out of an inner loop files: Modules/_collectionsmodule.c | 8 ++++++-- 1 files changed, 6 insertions(+), 2 deletions(-) diff --git a/Modules/_collectionsmodule.c b/Modules/_collectionsmodule.c --- a/Modules/_collectionsmodule.c +++ b/Modules/_collectionsmodule.c @@ -371,6 +371,7 @@ deque_extend(dequeobject *deque, PyObject *iterable) { PyObject *it, *item; + int trim = (deque->maxlen != -1); /* Handle case where id(deque) == id(iterable) */ if ((PyObject *)deque == iterable) { @@ -417,7 +418,8 @@ Py_SIZE(deque)++; deque->rightindex++; deque->rightblock->data[deque->rightindex] = item; - deque_trim_left(deque); + if (trim) + deque_trim_left(deque); } if (PyErr_Occurred()) { Py_DECREF(it); @@ -434,6 +436,7 @@ deque_extendleft(dequeobject *deque, PyObject *iterable) { PyObject *it, *item; + int trim = (deque->maxlen != -1); /* Handle case where id(deque) == id(iterable) */ if ((PyObject *)deque == iterable) { @@ -480,7 +483,8 @@ Py_SIZE(deque)++; deque->leftindex--; deque->leftblock->data[deque->leftindex] = item; - deque_trim_right(deque); + if (trim) + deque_trim_right(deque); } if (PyErr_Occurred()) { Py_DECREF(it); -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 19 10:00:52 2015 From: python-checkins at python.org (serhiy.storchaka) Date: Sat, 19 Sep 2015 08:00:52 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzI1MTAx?= =?utf-8?q?=3A_Try_to_create_a_file_to_test_write_access_in_test=5Fzipfile?= =?utf-8?q?=2E?= Message-ID: <20150919080052.9951.45134@psf.io> https://hg.python.org/cpython/rev/80d002edc9c1 changeset: 98066:80d002edc9c1 branch: 3.4 parent: 98056:9fb47dcd02fd user: Serhiy Storchaka date: Sat Sep 19 10:55:20 2015 +0300 summary: Issue #25101: Try to create a file to test write access in test_zipfile. files: Lib/test/test_zipfile.py | 7 +++++++ 1 files changed, 7 insertions(+), 0 deletions(-) diff --git a/Lib/test/test_zipfile.py b/Lib/test/test_zipfile.py --- a/Lib/test/test_zipfile.py +++ b/Lib/test/test_zipfile.py @@ -653,6 +653,13 @@ if not os.access(path, os.W_OK, effective_ids=os.access in os.supports_effective_ids): self.skipTest('requires write access to the installed location') + filename = os.path.join(path, 'test_zipfile.try') + try: + fd = os.open(filename, os.O_WRONLY | os.O_CREAT) + os.close(fd) + except Exception: + self.skipTest('requires write access to the installed location') + unlink(filename) def test_write_pyfile(self): self.requiresWriteAccess(os.path.dirname(__file__)) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 19 10:00:53 2015 From: python-checkins at python.org (serhiy.storchaka) Date: Sat, 19 Sep 2015 08:00:53 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2325101=3A_Try_to_create_a_file_to_test_write_acc?= =?utf-8?q?ess_in_test=5Fzipfile=2E?= Message-ID: <20150919080053.9937.97782@psf.io> https://hg.python.org/cpython/rev/399746fde4f8 changeset: 98069:399746fde4f8 parent: 98065:fb9d0fffb494 parent: 98068:6899bf8d21c3 user: Serhiy Storchaka date: Sat Sep 19 11:00:11 2015 +0300 summary: Issue #25101: Try to create a file to test write access in test_zipfile. files: Lib/test/test_zipfile.py | 7 +++++++ 1 files changed, 7 insertions(+), 0 deletions(-) diff --git a/Lib/test/test_zipfile.py b/Lib/test/test_zipfile.py --- a/Lib/test/test_zipfile.py +++ b/Lib/test/test_zipfile.py @@ -684,6 +684,13 @@ if not os.access(path, os.W_OK, effective_ids=os.access in os.supports_effective_ids): self.skipTest('requires write access to the installed location') + filename = os.path.join(path, 'test_zipfile.try') + try: + fd = os.open(filename, os.O_WRONLY | os.O_CREAT) + os.close(fd) + except Exception: + self.skipTest('requires write access to the installed location') + unlink(filename) def test_write_pyfile(self): self.requiresWriteAccess(os.path.dirname(__file__)) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 19 10:00:54 2015 From: python-checkins at python.org (serhiy.storchaka) Date: Sat, 19 Sep 2015 08:00:54 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Issue_=2325101=3A_Try_to_create_a_file_to_test_write_access_in?= =?utf-8?q?_test=5Fzipfile=2E?= Message-ID: <20150919080053.98376.73014@psf.io> https://hg.python.org/cpython/rev/6899bf8d21c3 changeset: 98068:6899bf8d21c3 branch: 3.5 parent: 98063:8054a9f788e7 parent: 98066:80d002edc9c1 user: Serhiy Storchaka date: Sat Sep 19 10:59:48 2015 +0300 summary: Issue #25101: Try to create a file to test write access in test_zipfile. files: Lib/test/test_zipfile.py | 7 +++++++ 1 files changed, 7 insertions(+), 0 deletions(-) diff --git a/Lib/test/test_zipfile.py b/Lib/test/test_zipfile.py --- a/Lib/test/test_zipfile.py +++ b/Lib/test/test_zipfile.py @@ -684,6 +684,13 @@ if not os.access(path, os.W_OK, effective_ids=os.access in os.supports_effective_ids): self.skipTest('requires write access to the installed location') + filename = os.path.join(path, 'test_zipfile.try') + try: + fd = os.open(filename, os.O_WRONLY | os.O_CREAT) + os.close(fd) + except Exception: + self.skipTest('requires write access to the installed location') + unlink(filename) def test_write_pyfile(self): self.requiresWriteAccess(os.path.dirname(__file__)) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 19 10:00:54 2015 From: python-checkins at python.org (serhiy.storchaka) Date: Sat, 19 Sep 2015 08:00:54 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzI1MTAx?= =?utf-8?q?=3A_Try_to_create_a_file_to_test_write_access_in_test=5Fzipfile?= =?utf-8?q?=2E?= Message-ID: <20150919080053.81625.27963@psf.io> https://hg.python.org/cpython/rev/fe84898fbe98 changeset: 98067:fe84898fbe98 branch: 2.7 parent: 98060:f211c8f554f9 user: Serhiy Storchaka date: Sat Sep 19 10:55:20 2015 +0300 summary: Issue #25101: Try to create a file to test write access in test_zipfile. files: Lib/test/test_zipfile.py | 7 +++++++ 1 files changed, 7 insertions(+), 0 deletions(-) diff --git a/Lib/test/test_zipfile.py b/Lib/test/test_zipfile.py --- a/Lib/test/test_zipfile.py +++ b/Lib/test/test_zipfile.py @@ -778,6 +778,13 @@ def requiresWriteAccess(self, path): if not os.access(path, os.W_OK): self.skipTest('requires write access to the installed location') + filename = os.path.join(path, 'test_zipfile.try') + try: + fd = os.open(filename, os.O_WRONLY | os.O_CREAT) + os.close(fd) + except Exception: + self.skipTest('requires write access to the installed location') + unlink(filename) def test_write_pyfile(self): self.requiresWriteAccess(os.path.dirname(__file__)) -- Repository URL: https://hg.python.org/cpython From solipsis at pitrou.net Sat Sep 19 10:45:43 2015 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Sat, 19 Sep 2015 08:45:43 +0000 Subject: [Python-checkins] Daily reference leaks (f03c074b6242): sum=17877 Message-ID: <20150919084543.81625.28980@psf.io> results for f03c074b6242 on branch "default" -------------------------------------------- test_capi leaked [1598, 1598, 1598] references, sum=4794 test_capi leaked [387, 389, 389] memory blocks, sum=1165 test_functools leaked [0, 2, 2] memory blocks, sum=4 test_threading leaked [3196, 3196, 3196] references, sum=9588 test_threading leaked [774, 776, 776] memory blocks, sum=2326 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/psf-users/antoine/refleaks/reflogDRAvgu', '--timeout', '7200'] From python-checkins at python.org Sat Sep 19 13:41:09 2015 From: python-checkins at python.org (victor.stinner) Date: Sat, 19 Sep 2015 11:41:09 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogTWVyZ2UgMy41?= Message-ID: <20150919114108.94123.48242@psf.io> https://hg.python.org/cpython/rev/3704cea9fd8e changeset: 98071:3704cea9fd8e parent: 98069:399746fde4f8 parent: 98070:21076e24cd8a user: Victor Stinner date: Sat Sep 19 13:39:16 2015 +0200 summary: Merge 3.5 files: Objects/longobject.c | 10 ++++++---- 1 files changed, 6 insertions(+), 4 deletions(-) diff --git a/Objects/longobject.c b/Objects/longobject.c --- a/Objects/longobject.c +++ b/Objects/longobject.c @@ -4495,11 +4495,13 @@ simple: assert(Py_REFCNT(a) > 0); assert(Py_REFCNT(b) > 0); -#if LONG_MAX >> 2*PyLong_SHIFT +/* Issue #24999: use two shifts instead of ">> 2*PyLong_SHIFT" to avoid + undefined behaviour when LONG_MAX type is smaller than 60 bits */ +#if LONG_MAX >> PyLong_SHIFT >> PyLong_SHIFT /* a fits into a long, so b must too */ x = PyLong_AsLong((PyObject *)a); y = PyLong_AsLong((PyObject *)b); -#elif defined(PY_LONG_LONG) && PY_LLONG_MAX >> 2*PyLong_SHIFT +#elif defined(PY_LONG_LONG) && PY_LLONG_MAX >> PyLong_SHIFT >> PyLong_SHIFT x = PyLong_AsLongLong((PyObject *)a); y = PyLong_AsLongLong((PyObject *)b); #else @@ -4516,9 +4518,9 @@ y = x % y; x = t; } -#if LONG_MAX >> 2*PyLong_SHIFT +#if LONG_MAX >> PyLong_SHIFT >> PyLong_SHIFT return PyLong_FromLong(x); -#elif defined(PY_LONG_LONG) && PY_LLONG_MAX >> 2*PyLong_SHIFT +#elif defined(PY_LONG_LONG) && PY_LLONG_MAX >> PyLong_SHIFT >> PyLong_SHIFT return PyLong_FromLongLong(x); #else # error "_PyLong_GCD" -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 19 13:41:08 2015 From: python-checkins at python.org (victor.stinner) Date: Sat, 19 Sep 2015 11:41:08 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI0OTk5?= =?utf-8?q?=3A_In_longobject=2Ec=2C_use_two_shifts_instead_of_=22=3E=3E_2*?= =?utf-8?q?PyLong=5FSHIFT=22_to?= Message-ID: <20150919114108.115507.47361@psf.io> https://hg.python.org/cpython/rev/21076e24cd8a changeset: 98070:21076e24cd8a branch: 3.5 parent: 98068:6899bf8d21c3 user: Victor Stinner date: Sat Sep 19 13:39:03 2015 +0200 summary: Issue #24999: In longobject.c, use two shifts instead of ">> 2*PyLong_SHIFT" to avoid undefined behaviour when LONG_MAX type is smaller than 60 bits. This change should fix a warning with the ICC compiler. files: Objects/longobject.c | 10 ++++++---- 1 files changed, 6 insertions(+), 4 deletions(-) diff --git a/Objects/longobject.c b/Objects/longobject.c --- a/Objects/longobject.c +++ b/Objects/longobject.c @@ -4495,11 +4495,13 @@ simple: assert(Py_REFCNT(a) > 0); assert(Py_REFCNT(b) > 0); -#if LONG_MAX >> 2*PyLong_SHIFT +/* Issue #24999: use two shifts instead of ">> 2*PyLong_SHIFT" to avoid + undefined behaviour when LONG_MAX type is smaller than 60 bits */ +#if LONG_MAX >> PyLong_SHIFT >> PyLong_SHIFT /* a fits into a long, so b must too */ x = PyLong_AsLong((PyObject *)a); y = PyLong_AsLong((PyObject *)b); -#elif defined(PY_LONG_LONG) && PY_LLONG_MAX >> 2*PyLong_SHIFT +#elif defined(PY_LONG_LONG) && PY_LLONG_MAX >> PyLong_SHIFT >> PyLong_SHIFT x = PyLong_AsLongLong((PyObject *)a); y = PyLong_AsLongLong((PyObject *)b); #else @@ -4516,9 +4518,9 @@ y = x % y; x = t; } -#if LONG_MAX >> 2*PyLong_SHIFT +#if LONG_MAX >> PyLong_SHIFT >> PyLong_SHIFT return PyLong_FromLong(x); -#elif defined(PY_LONG_LONG) && PY_LLONG_MAX >> 2*PyLong_SHIFT +#elif defined(PY_LONG_LONG) && PY_LLONG_MAX >> PyLong_SHIFT >> PyLong_SHIFT return PyLong_FromLongLong(x); #else # error "_PyLong_GCD" -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 19 17:24:13 2015 From: python-checkins at python.org (eric.smith) Date: Sat, 19 Sep 2015 15:24:13 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_Make_it_clear_that_=5F=5Fform?= =?utf-8?q?at=5F=5F_isn=27t_called_on_the_value_of_each_expression=2C_but?= Message-ID: <20150919152413.82644.81299@psf.io> https://hg.python.org/peps/rev/0060ddc240a7 changeset: 6070:0060ddc240a7 user: Eric V. Smith date: Sat Sep 19 11:24:19 2015 -0400 summary: Make it clear that __format__ isn't called on the value of each expression, but rather on the type of the value. This is how the builtin format() function works, too. files: pep-0498.txt | 19 ++++++++++++------- 1 files changed, 12 insertions(+), 7 deletions(-) diff --git a/pep-0498.txt b/pep-0498.txt --- a/pep-0498.txt +++ b/pep-0498.txt @@ -8,7 +8,7 @@ Content-Type: text/x-rst Created: 01-Aug-2015 Python-Version: 3.6 -Post-History: 07-Aug-2015, 30-Aug-2015, 04-Sep-2015 +Post-History: 07-Aug-2015, 30-Aug-2015, 04-Sep-2015, 19-Sep-2015 Resolution: https://mail.python.org/pipermail/python-dev/2015-September/141526.html Abstract @@ -201,6 +201,11 @@ replaced by the corresponding single brace. Doubled opening braces do not signify the start of an expression. +Note that ``__format__()`` is not called directly on each value. The +actual code uses the equivalent of ``type(value).__format__(value, +format_spec)``, or ``format(value, format_spec)``. See the +documentation of the builtin ``format()`` function for more details. + Comments, using the ``'#'`` character, are not allowed inside an expression. @@ -209,7 +214,7 @@ ``'!a'``. These are treated the same as in ``str.format()``: ``'!s'`` calls ``str()`` on the expression, ``'!r'`` calls ``repr()`` on the expression, and ``'!a'`` calls ``ascii()`` on the expression. These -conversions are applied before the call to ``__format__``. The only +conversions are applied before the call to ``format()``. The only reason to use ``'!s'`` is if you want to specify a format specifier that applies to ``str``, not to the type of the expression. @@ -222,9 +227,9 @@ f ' { } ... ' -The resulting expression's ``__format__`` method is called with the -format specifier as an argument. The resulting value is used when -building the value of the f-string. +The expression is then formatted using the ``__format__`` protocol, +using the format specifier as an argument. The resulting value is +used when building the value of the f-string. Expressions cannot contain ``':'`` or ``'!'`` outside of strings or parentheses, brackets, or braces. The exception is that the ``'!='`` @@ -293,7 +298,7 @@ Might be be evaluated as:: - 'abc' + expr1.__format__(spec1) + repr(expr2).__format__(spec2) + 'def' + str(expr3).__format__('') + 'ghi' + 'abc' + format(expr1, spec1) + format(repr(expr2)) + 'def' + format(str(expr3)) + 'ghi' Expression evaluation --------------------- @@ -371,7 +376,7 @@ While the exact method of this run time concatenation is unspecified, the above code might evaluate to:: - 'ab' + x.__format__('') + '{c}' + 'str<' + y.__format__('^4') + '>de' + 'ab' + format(x) + '{c}' + 'str<' + format(y, '^4') + '>de' Each f-string is entirely evaluated before being concatenated to adjacent f-strings. That means that this:: -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Sat Sep 19 18:05:49 2015 From: python-checkins at python.org (raymond.hettinger) Date: Sat, 19 Sep 2015 16:05:49 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Add_a_fast_path_=28no_iter?= =?utf-8?q?ator_creation=29_for_a_common_case_for_repeating_deques?= Message-ID: <20150919160549.94125.38195@psf.io> https://hg.python.org/cpython/rev/4c99c6eab05f changeset: 98072:4c99c6eab05f user: Raymond Hettinger date: Sat Sep 19 09:05:42 2015 -0700 summary: Add a fast path (no iterator creation) for a common case for repeating deques of size 1 files: Lib/test/test_deque.py | 9 +++++++++ Modules/_collectionsmodule.c | 15 +++++++++++---- 2 files changed, 20 insertions(+), 4 deletions(-) diff --git a/Lib/test/test_deque.py b/Lib/test/test_deque.py --- a/Lib/test/test_deque.py +++ b/Lib/test/test_deque.py @@ -654,6 +654,15 @@ self.assertNotEqual(id(d), id(e)) self.assertEqual(list(d), list(e)) + for i in range(5): + for maxlen in range(-1, 6): + s = [random.random() for j in range(i)] + d = deque(s) if maxlen == -1 else deque(s, maxlen) + e = d.copy() + self.assertEqual(d, e) + self.assertEqual(d.maxlen, e.maxlen) + self.assertTrue(all(x is y for x, y in zip(d, e))) + def test_copy_method(self): mut = [10] d = deque([mut]) diff --git a/Modules/_collectionsmodule.c b/Modules/_collectionsmodule.c --- a/Modules/_collectionsmodule.c +++ b/Modules/_collectionsmodule.c @@ -1211,6 +1211,7 @@ static PyObject * deque_copy(PyObject *deque) { + dequeobject *old_deque = (dequeobject *)deque; if (Py_TYPE(deque) == &deque_type) { dequeobject *new_deque; PyObject *rv; @@ -1218,8 +1219,14 @@ new_deque = (dequeobject *)deque_new(&deque_type, (PyObject *)NULL, (PyObject *)NULL); if (new_deque == NULL) return NULL; - new_deque->maxlen = ((dequeobject *)deque)->maxlen; - rv = deque_extend(new_deque, deque); + new_deque->maxlen = old_deque->maxlen; + /* Fast path for the deque_repeat() common case where len(deque) == 1 */ + if (Py_SIZE(deque) == 1 && new_deque->maxlen != 0) { + PyObject *item = old_deque->leftblock->data[old_deque->leftindex]; + rv = deque_append(new_deque, item); + } else { + rv = deque_extend(new_deque, deque); + } if (rv != NULL) { Py_DECREF(rv); return (PyObject *)new_deque; @@ -1227,11 +1234,11 @@ Py_DECREF(new_deque); return NULL; } - if (((dequeobject *)deque)->maxlen == -1) + if (old_deque->maxlen == -1) return PyObject_CallFunction((PyObject *)(Py_TYPE(deque)), "O", deque, NULL); else return PyObject_CallFunction((PyObject *)(Py_TYPE(deque)), "Oi", - deque, ((dequeobject *)deque)->maxlen, NULL); + deque, old_deque->maxlen, NULL); } PyDoc_STRVAR(copy_doc, "Return a shallow copy of a deque."); -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 19 20:52:00 2015 From: python-checkins at python.org (eric.smith) Date: Sat, 19 Sep 2015 18:52:00 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2324965=3A_Implemen?= =?utf-8?q?t_PEP_498_=22Literal_String_Interpolation=22=2E_Documentation?= Message-ID: <20150919185200.81647.31118@psf.io> https://hg.python.org/cpython/rev/a10d37f04569 changeset: 98073:a10d37f04569 user: Eric V. Smith date: Sat Sep 19 14:51:32 2015 -0400 summary: Issue #24965: Implement PEP 498 "Literal String Interpolation". Documentation is still needed, I'll open an issue for that. files: Include/Python-ast.h | 23 +- Lib/test/test_fstring.py | 715 +++++++++++++++++++ Misc/NEWS | 5 + Parser/Python.asdl | 2 + Parser/tokenizer.c | 8 +- Python/Python-ast.c | 165 ++++ Python/ast.c | 987 +++++++++++++++++++++++++- Python/compile.c | 117 +++- Python/symtable.c | 8 + 9 files changed, 1966 insertions(+), 64 deletions(-) diff --git a/Include/Python-ast.h b/Include/Python-ast.h --- a/Include/Python-ast.h +++ b/Include/Python-ast.h @@ -201,9 +201,10 @@ SetComp_kind=9, DictComp_kind=10, GeneratorExp_kind=11, Await_kind=12, Yield_kind=13, YieldFrom_kind=14, Compare_kind=15, Call_kind=16, Num_kind=17, Str_kind=18, - Bytes_kind=19, NameConstant_kind=20, Ellipsis_kind=21, - Attribute_kind=22, Subscript_kind=23, Starred_kind=24, - Name_kind=25, List_kind=26, Tuple_kind=27}; + FormattedValue_kind=19, JoinedStr_kind=20, Bytes_kind=21, + NameConstant_kind=22, Ellipsis_kind=23, Attribute_kind=24, + Subscript_kind=25, Starred_kind=26, Name_kind=27, + List_kind=28, Tuple_kind=29}; struct _expr { enum _expr_kind kind; union { @@ -297,6 +298,16 @@ } Str; struct { + expr_ty value; + int conversion; + expr_ty format_spec; + } FormattedValue; + + struct { + asdl_seq *values; + } JoinedStr; + + struct { bytes s; } Bytes; @@ -543,6 +554,12 @@ expr_ty _Py_Num(object n, int lineno, int col_offset, PyArena *arena); #define Str(a0, a1, a2, a3) _Py_Str(a0, a1, a2, a3) expr_ty _Py_Str(string s, int lineno, int col_offset, PyArena *arena); +#define FormattedValue(a0, a1, a2, a3, a4, a5) _Py_FormattedValue(a0, a1, a2, a3, a4, a5) +expr_ty _Py_FormattedValue(expr_ty value, int conversion, expr_ty format_spec, + int lineno, int col_offset, PyArena *arena); +#define JoinedStr(a0, a1, a2, a3) _Py_JoinedStr(a0, a1, a2, a3) +expr_ty _Py_JoinedStr(asdl_seq * values, int lineno, int col_offset, PyArena + *arena); #define Bytes(a0, a1, a2, a3) _Py_Bytes(a0, a1, a2, a3) expr_ty _Py_Bytes(bytes s, int lineno, int col_offset, PyArena *arena); #define NameConstant(a0, a1, a2, a3) _Py_NameConstant(a0, a1, a2, a3) diff --git a/Lib/test/test_fstring.py b/Lib/test/test_fstring.py new file mode 100644 --- /dev/null +++ b/Lib/test/test_fstring.py @@ -0,0 +1,715 @@ +import ast +import types +import decimal +import unittest + +a_global = 'global variable' + +# You could argue that I'm too strict in looking for specific error +# values with assertRaisesRegex, but without it it's way too easy to +# make a syntax error in the test strings. Especially with all of the +# triple quotes, raw strings, backslashes, etc. I think it's a +# worthwhile tradeoff. When I switched to this method, I found many +# examples where I wasn't testing what I thought I was. + +class TestCase(unittest.TestCase): + def assertAllRaise(self, exception_type, regex, error_strings): + for str in error_strings: + with self.subTest(str=str): + with self.assertRaisesRegex(exception_type, regex): + eval(str) + + def test__format__lookup(self): + # Make sure __format__ is looked up on the type, not the instance. + class X: + def __format__(self, spec): + return 'class' + + x = X() + + # Add a bound __format__ method to the 'y' instance, but not + # the 'x' instance. + y = X() + y.__format__ = types.MethodType(lambda self, spec: 'instance', y) + + self.assertEqual(f'{y}', format(y)) + self.assertEqual(f'{y}', 'class') + self.assertEqual(format(x), format(y)) + + # __format__ is not called this way, but still make sure it + # returns what we expect (so we can make sure we're bypassing + # it). + self.assertEqual(x.__format__(''), 'class') + self.assertEqual(y.__format__(''), 'instance') + + # This is how __format__ is actually called. + self.assertEqual(type(x).__format__(x, ''), 'class') + self.assertEqual(type(y).__format__(y, ''), 'class') + + def test_ast(self): + # Inspired by http://bugs.python.org/issue24975 + class X: + def __init__(self): + self.called = False + def __call__(self): + self.called = True + return 4 + x = X() + expr = """ +a = 10 +f'{a * x()}'""" + t = ast.parse(expr) + c = compile(t, '', 'exec') + + # Make sure x was not called. + self.assertFalse(x.called) + + # Actually run the code. + exec(c) + + # Make sure x was called. + self.assertTrue(x.called) + + def test_literal_eval(self): + # With no expressions, an f-string is okay. + self.assertEqual(ast.literal_eval("f'x'"), 'x') + self.assertEqual(ast.literal_eval("f'x' 'y'"), 'xy') + + # But this should raise an error. + with self.assertRaisesRegex(ValueError, 'malformed node or string'): + ast.literal_eval("f'x{3}'") + + # As should this, which uses a different ast node + with self.assertRaisesRegex(ValueError, 'malformed node or string'): + ast.literal_eval("f'{3}'") + + def test_ast_compile_time_concat(self): + x = [''] + + expr = """x[0] = 'foo' f'{3}'""" + t = ast.parse(expr) + c = compile(t, '', 'exec') + exec(c) + self.assertEqual(x[0], 'foo3') + + def test_literal(self): + self.assertEqual(f'', '') + self.assertEqual(f'a', 'a') + self.assertEqual(f' ', ' ') + self.assertEqual(f'\N{GREEK CAPITAL LETTER DELTA}', + '\N{GREEK CAPITAL LETTER DELTA}') + self.assertEqual(f'\N{GREEK CAPITAL LETTER DELTA}', + '\u0394') + self.assertEqual(f'\N{True}', '\u22a8') + self.assertEqual(rf'\N{True}', r'\NTrue') + + def test_escape_order(self): + # note that hex(ord('{')) == 0x7b, so this + # string becomes f'a{4*10}b' + self.assertEqual(f'a\u007b4*10}b', 'a40b') + self.assertEqual(f'a\x7b4*10}b', 'a40b') + self.assertEqual(f'a\x7b4*10\N{RIGHT CURLY BRACKET}b', 'a40b') + self.assertEqual(f'{"a"!\N{LATIN SMALL LETTER R}}', "'a'") + self.assertEqual(f'{10\x3a02X}', '0A') + self.assertEqual(f'{10:02\N{LATIN CAPITAL LETTER X}}', '0A') + + self.assertAllRaise(SyntaxError, "f-string: single '}' is not allowed", + [r"""f'a{\u007b4*10}b'""", # mis-matched brackets + ]) + self.assertAllRaise(SyntaxError, 'unexpected character after line continuation character', + [r"""f'{"a"\!r}'""", + r"""f'{a\!r}'""", + ]) + + def test_unterminated_string(self): + self.assertAllRaise(SyntaxError, 'f-string: unterminated string', + [r"""f'{"x'""", + r"""f'{"x}'""", + r"""f'{("x'""", + r"""f'{("x}'""", + ]) + + def test_mismatched_parens(self): + self.assertAllRaise(SyntaxError, 'f-string: mismatched', + ["f'{((}'", + ]) + + def test_double_braces(self): + self.assertEqual(f'{{', '{') + self.assertEqual(f'a{{', 'a{') + self.assertEqual(f'{{b', '{b') + self.assertEqual(f'a{{b', 'a{b') + self.assertEqual(f'}}', '}') + self.assertEqual(f'a}}', 'a}') + self.assertEqual(f'}}b', '}b') + self.assertEqual(f'a}}b', 'a}b') + + self.assertEqual(f'{{{10}', '{10') + self.assertEqual(f'}}{10}', '}10') + self.assertEqual(f'}}{{{10}', '}{10') + self.assertEqual(f'}}a{{{10}', '}a{10') + + self.assertEqual(f'{10}{{', '10{') + self.assertEqual(f'{10}}}', '10}') + self.assertEqual(f'{10}}}{{', '10}{') + self.assertEqual(f'{10}}}a{{' '}', '10}a{}') + + # Inside of strings, don't interpret doubled brackets. + self.assertEqual(f'{"{{}}"}', '{{}}') + + self.assertAllRaise(TypeError, 'unhashable type', + ["f'{ {{}} }'", # dict in a set + ]) + + def test_compile_time_concat(self): + x = 'def' + self.assertEqual('abc' f'## {x}ghi', 'abc## defghi') + self.assertEqual('abc' f'{x}' 'ghi', 'abcdefghi') + self.assertEqual('abc' f'{x}' 'gh' f'i{x:4}', 'abcdefghidef ') + self.assertEqual('{x}' f'{x}', '{x}def') + self.assertEqual('{x' f'{x}', '{xdef') + self.assertEqual('{x}' f'{x}', '{x}def') + self.assertEqual('{{x}}' f'{x}', '{{x}}def') + self.assertEqual('{{x' f'{x}', '{{xdef') + self.assertEqual('x}}' f'{x}', 'x}}def') + self.assertEqual(f'{x}' 'x}}', 'defx}}') + self.assertEqual(f'{x}' '', 'def') + self.assertEqual('' f'{x}' '', 'def') + self.assertEqual('' f'{x}', 'def') + self.assertEqual(f'{x}' '2', 'def2') + self.assertEqual('1' f'{x}' '2', '1def2') + self.assertEqual('1' f'{x}', '1def') + self.assertEqual(f'{x}' f'-{x}', 'def-def') + self.assertEqual('' f'', '') + self.assertEqual('' f'' '', '') + self.assertEqual('' f'' '' f'', '') + self.assertEqual(f'', '') + self.assertEqual(f'' '', '') + self.assertEqual(f'' '' f'', '') + self.assertEqual(f'' '' f'' '', '') + + self.assertAllRaise(SyntaxError, "f-string: expecting '}'", + ["f'{3' f'}'", # can't concat to get a valid f-string + ]) + + def test_comments(self): + # These aren't comments, since they're in strings. + d = {'#': 'hash'} + self.assertEqual(f'{"#"}', '#') + self.assertEqual(f'{d["#"]}', 'hash') + + self.assertAllRaise(SyntaxError, "f-string cannot include '#'", + ["f'{1#}'", # error because the expression becomes "(1#)" + "f'{3(#)}'", + ]) + + def test_many_expressions(self): + # Create a string with many expressions in it. Note that + # because we have a space in here as a literal, we're actually + # going to use twice as many ast nodes: one for each literal + # plus one for each expression. + def build_fstr(n, extra=''): + return "f'" + ('{x} ' * n) + extra + "'" + + x = 'X' + width = 1 + + # Test around 256. + for i in range(250, 260): + self.assertEqual(eval(build_fstr(i)), (x+' ')*i) + + # Test concatenating 2 largs fstrings. + self.assertEqual(eval(build_fstr(255)*256), (x+' ')*(255*256)) + + s = build_fstr(253, '{x:{width}} ') + self.assertEqual(eval(s), (x+' ')*254) + + # Test lots of expressions and constants, concatenated. + s = "f'{1}' 'x' 'y'" * 1024 + self.assertEqual(eval(s), '1xy' * 1024) + + def test_format_specifier_expressions(self): + width = 10 + precision = 4 + value = decimal.Decimal('12.34567') + self.assertEqual(f'result: {value:{width}.{precision}}', 'result: 12.35') + self.assertEqual(f'result: {value:{width!r}.{precision}}', 'result: 12.35') + self.assertEqual(f'result: {value:{width:0}.{precision:1}}', 'result: 12.35') + self.assertEqual(f'result: {value:{1}{0:0}.{precision:1}}', 'result: 12.35') + self.assertEqual(f'result: {value:{ 1}{ 0:0}.{ precision:1}}', 'result: 12.35') + self.assertEqual(f'{10:#{1}0x}', ' 0xa') + self.assertEqual(f'{10:{"#"}1{0}{"x"}}', ' 0xa') + self.assertEqual(f'{-10:-{"#"}1{0}x}', ' -0xa') + self.assertEqual(f'{-10:{"-"}#{1}0{"x"}}', ' -0xa') + self.assertEqual(f'{10:#{3 != {4:5} and width}x}', ' 0xa') + + self.assertAllRaise(SyntaxError, "f-string: expecting '}'", + ["""f'{"s"!r{":10"}}'""", + + # This looks like a nested format spec. + ]) + + self.assertAllRaise(SyntaxError, "invalid syntax", + [# Invalid sytax inside a nested spec. + "f'{4:{/5}}'", + ]) + + self.assertAllRaise(SyntaxError, "f-string: expressions nested too deeply", + [# Can't nest format specifiers. + "f'result: {value:{width:{0}}.{precision:1}}'", + ]) + + self.assertAllRaise(SyntaxError, 'f-string: invalid conversion character', + [# No expansion inside conversion or for + # the : or ! itself. + """f'{"s"!{"r"}}'""", + ]) + + def test_side_effect_order(self): + class X: + def __init__(self): + self.i = 0 + def __format__(self, spec): + self.i += 1 + return str(self.i) + + x = X() + self.assertEqual(f'{x} {x}', '1 2') + + def test_missing_expression(self): + self.assertAllRaise(SyntaxError, 'f-string: empty expression not allowed', + ["f'{}'", + "f'{ }'" + "f' {} '", + "f'{!r}'", + "f'{ !r}'", + "f'{10:{ }}'", + "f' { } '", + r"f'{\n}'", + r"f'{\n \n}'", + ]) + + def test_parens_in_expressions(self): + self.assertEqual(f'{3,}', '(3,)') + + # Add these because when an expression is evaluated, parens + # are added around it. But we shouldn't go from an invalid + # expression to a valid one. The added parens are just + # supposed to allow whitespace (including newlines). + self.assertAllRaise(SyntaxError, 'invalid syntax', + ["f'{,}'", + "f'{,}'", # this is (,), which is an error + ]) + + self.assertAllRaise(SyntaxError, "f-string: expecting '}'", + ["f'{3)+(4}'", + ]) + + self.assertAllRaise(SyntaxError, 'EOL while scanning string literal', + ["f'{\n}'", + ]) + + def test_newlines_in_expressions(self): + self.assertEqual(f'{0}', '0') + self.assertEqual(f'{0\n}', '0') + self.assertEqual(f'{0\r}', '0') + self.assertEqual(f'{\n0\n}', '0') + self.assertEqual(f'{\r0\r}', '0') + self.assertEqual(f'{\n0\r}', '0') + self.assertEqual(f'{\n0}', '0') + self.assertEqual(f'{3+\n4}', '7') + self.assertEqual(f'{3+\\\n4}', '7') + self.assertEqual(rf'''{3+ +4}''', '7') + self.assertEqual(f'''{3+\ +4}''', '7') + + self.assertAllRaise(SyntaxError, 'f-string: empty expression not allowed', + [r"f'{\n}'", + ]) + + def test_lambda(self): + x = 5 + self.assertEqual(f'{(lambda y:x*y)("8")!r}', "'88888'") + self.assertEqual(f'{(lambda y:x*y)("8")!r:10}', "'88888' ") + self.assertEqual(f'{(lambda y:x*y)("8"):10}', "88888 ") + + # lambda doesn't work without parens, because the colon + # makes the parser think it's a format_spec + self.assertAllRaise(SyntaxError, 'unexpected EOF while parsing', + ["f'{lambda x:x}'", + ]) + + def test_yield(self): + # Not terribly useful, but make sure the yield turns + # a function into a generator + def fn(y): + f'y:{yield y*2}' + + g = fn(4) + self.assertEqual(next(g), 8) + + def test_yield_send(self): + def fn(x): + yield f'x:{yield (lambda i: x * i)}' + + g = fn(10) + the_lambda = next(g) + self.assertEqual(the_lambda(4), 40) + self.assertEqual(g.send('string'), 'x:string') + + def test_expressions_with_triple_quoted_strings(self): + self.assertEqual(f"{'''x'''}", 'x') + self.assertEqual(f"{'''eric's'''}", "eric's") + self.assertEqual(f'{"""eric\'s"""}', "eric's") + self.assertEqual(f"{'''eric\"s'''}", 'eric"s') + self.assertEqual(f'{"""eric"s"""}', 'eric"s') + + # Test concatenation within an expression + self.assertEqual(f'{"x" """eric"s""" "y"}', 'xeric"sy') + self.assertEqual(f'{"x" """eric"s"""}', 'xeric"s') + self.assertEqual(f'{"""eric"s""" "y"}', 'eric"sy') + self.assertEqual(f'{"""x""" """eric"s""" "y"}', 'xeric"sy') + self.assertEqual(f'{"""x""" """eric"s""" """y"""}', 'xeric"sy') + self.assertEqual(f'{r"""x""" """eric"s""" """y"""}', 'xeric"sy') + + def test_multiple_vars(self): + x = 98 + y = 'abc' + self.assertEqual(f'{x}{y}', '98abc') + + self.assertEqual(f'X{x}{y}', 'X98abc') + self.assertEqual(f'{x}X{y}', '98Xabc') + self.assertEqual(f'{x}{y}X', '98abcX') + + self.assertEqual(f'X{x}Y{y}', 'X98Yabc') + self.assertEqual(f'X{x}{y}Y', 'X98abcY') + self.assertEqual(f'{x}X{y}Y', '98XabcY') + + self.assertEqual(f'X{x}Y{y}Z', 'X98YabcZ') + + def test_closure(self): + def outer(x): + def inner(): + return f'x:{x}' + return inner + + self.assertEqual(outer('987')(), 'x:987') + self.assertEqual(outer(7)(), 'x:7') + + def test_arguments(self): + y = 2 + def f(x, width): + return f'x={x*y:{width}}' + + self.assertEqual(f('foo', 10), 'x=foofoo ') + x = 'bar' + self.assertEqual(f(10, 10), 'x= 20') + + def test_locals(self): + value = 123 + self.assertEqual(f'v:{value}', 'v:123') + + def test_missing_variable(self): + with self.assertRaises(NameError): + f'v:{value}' + + def test_missing_format_spec(self): + class O: + def __format__(self, spec): + if not spec: + return '*' + return spec + + self.assertEqual(f'{O():x}', 'x') + self.assertEqual(f'{O()}', '*') + self.assertEqual(f'{O():}', '*') + + self.assertEqual(f'{3:}', '3') + self.assertEqual(f'{3!s:}', '3') + + def test_global(self): + self.assertEqual(f'g:{a_global}', 'g:global variable') + self.assertEqual(f'g:{a_global!r}', "g:'global variable'") + + a_local = 'local variable' + self.assertEqual(f'g:{a_global} l:{a_local}', + 'g:global variable l:local variable') + self.assertEqual(f'g:{a_global!r}', + "g:'global variable'") + self.assertEqual(f'g:{a_global} l:{a_local!r}', + "g:global variable l:'local variable'") + + self.assertIn("module 'unittest' from", f'{unittest}') + + def test_shadowed_global(self): + a_global = 'really a local' + self.assertEqual(f'g:{a_global}', 'g:really a local') + self.assertEqual(f'g:{a_global!r}', "g:'really a local'") + + a_local = 'local variable' + self.assertEqual(f'g:{a_global} l:{a_local}', + 'g:really a local l:local variable') + self.assertEqual(f'g:{a_global!r}', + "g:'really a local'") + self.assertEqual(f'g:{a_global} l:{a_local!r}', + "g:really a local l:'local variable'") + + def test_call(self): + def foo(x): + return 'x=' + str(x) + + self.assertEqual(f'{foo(10)}', 'x=10') + + def test_nested_fstrings(self): + y = 5 + self.assertEqual(f'{f"{0}"*3}', '000') + self.assertEqual(f'{f"{y}"*3}', '555') + self.assertEqual(f'{f"{\'x\'}"*3}', 'xxx') + + self.assertEqual(f"{r'x' f'{\"s\"}'}", 'xs') + self.assertEqual(f"{r'x'rf'{\"s\"}'}", 'xs') + + def test_invalid_string_prefixes(self): + self.assertAllRaise(SyntaxError, 'unexpected EOF while parsing', + ["fu''", + "uf''", + "Fu''", + "fU''", + "Uf''", + "uF''", + "ufr''", + "urf''", + "fur''", + "fru''", + "rfu''", + "ruf''", + "FUR''", + "Fur''", + ]) + + def test_leading_trailing_spaces(self): + self.assertEqual(f'{ 3}', '3') + self.assertEqual(f'{ 3}', '3') + self.assertEqual(f'{\t3}', '3') + self.assertEqual(f'{\t\t3}', '3') + self.assertEqual(f'{3 }', '3') + self.assertEqual(f'{3 }', '3') + self.assertEqual(f'{3\t}', '3') + self.assertEqual(f'{3\t\t}', '3') + + self.assertEqual(f'expr={ {x: y for x, y in [(1, 2), ]}}', + 'expr={1: 2}') + self.assertEqual(f'expr={ {x: y for x, y in [(1, 2), ]} }', + 'expr={1: 2}') + + def test_character_name(self): + self.assertEqual(f'{4}\N{GREEK CAPITAL LETTER DELTA}{3}', + '4\N{GREEK CAPITAL LETTER DELTA}3') + self.assertEqual(f'{{}}\N{GREEK CAPITAL LETTER DELTA}{3}', + '{}\N{GREEK CAPITAL LETTER DELTA}3') + + def test_not_equal(self): + # There's a special test for this because there's a special + # case in the f-string parser to look for != as not ending an + # expression. Normally it would, while looking for !s or !r. + + self.assertEqual(f'{3!=4}', 'True') + self.assertEqual(f'{3!=4:}', 'True') + self.assertEqual(f'{3!=4!s}', 'True') + self.assertEqual(f'{3!=4!s:.3}', 'Tru') + + def test_conversions(self): + self.assertEqual(f'{3.14:10.10}', ' 3.14') + self.assertEqual(f'{3.14!s:10.10}', '3.14 ') + self.assertEqual(f'{3.14!r:10.10}', '3.14 ') + self.assertEqual(f'{3.14!a:10.10}', '3.14 ') + + self.assertEqual(f'{"a"}', 'a') + self.assertEqual(f'{"a"!r}', "'a'") + self.assertEqual(f'{"a"!a}', "'a'") + + # Not a conversion. + self.assertEqual(f'{"a!r"}', "a!r") + + # Not a conversion, but show that ! is allowed in a format spec. + self.assertEqual(f'{3.14:!<10.10}', '3.14!!!!!!') + + self.assertEqual(f'{"\N{GREEK CAPITAL LETTER DELTA}"}', '\u0394') + self.assertEqual(f'{"\N{GREEK CAPITAL LETTER DELTA}"!r}', "'\u0394'") + self.assertEqual(f'{"\N{GREEK CAPITAL LETTER DELTA}"!a}', "'\\u0394'") + + self.assertAllRaise(SyntaxError, 'f-string: invalid conversion character', + ["f'{3!g}'", + "f'{3!A}'", + "f'{3!A}'", + "f'{3!A}'", + "f'{3!!}'", + "f'{3!:}'", + "f'{3!\N{GREEK CAPITAL LETTER DELTA}}'", + "f'{3! s}'", # no space before conversion char + "f'{x!\\x00:.<10}'", + ]) + + self.assertAllRaise(SyntaxError, "f-string: expecting '}'", + ["f'{x!s{y}}'", + "f'{3!ss}'", + "f'{3!ss:}'", + "f'{3!ss:s}'", + ]) + + def test_assignment(self): + self.assertAllRaise(SyntaxError, 'invalid syntax', + ["f'' = 3", + "f'{0}' = x", + "f'{x}' = x", + ]) + + def test_del(self): + self.assertAllRaise(SyntaxError, 'invalid syntax', + ["del f''", + "del '' f''", + ]) + + def test_mismatched_braces(self): + self.assertAllRaise(SyntaxError, "f-string: single '}' is not allowed", + ["f'{{}'", + "f'{{}}}'", + "f'}'", + "f'x}'", + "f'x}x'", + + # Can't have { or } in a format spec. + "f'{3:}>10}'", + r"f'{3:\\}>10}'", + "f'{3:}}>10}'", + ]) + + self.assertAllRaise(SyntaxError, "f-string: expecting '}'", + ["f'{3:{{>10}'", + "f'{3'", + "f'{3!'", + "f'{3:'", + "f'{3!s'", + "f'{3!s:'", + "f'{3!s:3'", + "f'x{'", + "f'x{x'", + "f'{3:s'", + "f'{{{'", + "f'{{}}{'", + "f'{'", + ]) + + self.assertAllRaise(SyntaxError, 'invalid syntax', + [r"f'{3:\\{>10}'", + ]) + + # But these are just normal strings. + self.assertEqual(f'{"{"}', '{') + self.assertEqual(f'{"}"}', '}') + self.assertEqual(f'{3:{"}"}>10}', '}}}}}}}}}3') + self.assertEqual(f'{2:{"{"}>10}', '{{{{{{{{{2') + + def test_if_conditional(self): + # There's special logic in compile.c to test if the + # conditional for an if (and while) are constants. Exercise + # that code. + + def test_fstring(x, expected): + flag = 0 + if f'{x}': + flag = 1 + else: + flag = 2 + self.assertEqual(flag, expected) + + def test_concat_empty(x, expected): + flag = 0 + if '' f'{x}': + flag = 1 + else: + flag = 2 + self.assertEqual(flag, expected) + + def test_concat_non_empty(x, expected): + flag = 0 + if ' ' f'{x}': + flag = 1 + else: + flag = 2 + self.assertEqual(flag, expected) + + test_fstring('', 2) + test_fstring(' ', 1) + + test_concat_empty('', 2) + test_concat_empty(' ', 1) + + test_concat_non_empty('', 1) + test_concat_non_empty(' ', 1) + + def test_empty_format_specifier(self): + x = 'test' + self.assertEqual(f'{x}', 'test') + self.assertEqual(f'{x:}', 'test') + self.assertEqual(f'{x!s:}', 'test') + self.assertEqual(f'{x!r:}', "'test'") + + def test_str_format_differences(self): + d = {'a': 'string', + 0: 'integer', + } + a = 0 + self.assertEqual(f'{d[0]}', 'integer') + self.assertEqual(f'{d["a"]}', 'string') + self.assertEqual(f'{d[a]}', 'integer') + self.assertEqual('{d[a]}'.format(d=d), 'string') + self.assertEqual('{d[0]}'.format(d=d), 'integer') + + def test_invalid_expressions(self): + self.assertAllRaise(SyntaxError, 'invalid syntax', + [r"f'{a[4)}'", + r"f'{a(4]}'", + ]) + + def test_loop(self): + for i in range(1000): + self.assertEqual(f'i:{i}', 'i:' + str(i)) + + def test_dict(self): + d = {'"': 'dquote', + "'": 'squote', + 'foo': 'bar', + } + self.assertEqual(f'{d["\'"]}', 'squote') + self.assertEqual(f"{d['\"']}", 'dquote') + + self.assertEqual(f'''{d["'"]}''', 'squote') + self.assertEqual(f"""{d['"']}""", 'dquote') + + self.assertEqual(f'{d["foo"]}', 'bar') + self.assertEqual(f"{d['foo']}", 'bar') + self.assertEqual(f'{d[\'foo\']}', 'bar') + self.assertEqual(f"{d[\"foo\"]}", 'bar') + + def test_escaped_quotes(self): + d = {'"': 'a', + "'": 'b'} + + self.assertEqual(fr"{d['\"']}", 'a') + self.assertEqual(fr'{d["\'"]}', 'b') + self.assertEqual(fr"{'\"'}", '"') + self.assertEqual(fr'{"\'"}', "'") + self.assertEqual(f'{"\\"3"}', '"3') + + self.assertAllRaise(SyntaxError, 'f-string: unterminated string', + [r'''f'{"""\\}' ''', # Backslash at end of expression + ]) + self.assertAllRaise(SyntaxError, 'unexpected character after line continuation', + [r"rf'{3\}'", + ]) + + +if __name__ == '__main__': + unittest.main() diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -19,6 +19,11 @@ argument list of a function declaration. For example, "def f(*, a = 3,): pass" is now legal. Patch from Mark Dickinson. +- Issue #24965: Implement PEP 498 "Literal String Interpolation". This + allows you to embed expressions inside f-strings, which are + converted to normal strings at run time. Given x=3, then + f'value={x}' == 'value=3'. Patch by Eric V. Smith. + Library ------- diff --git a/Parser/Python.asdl b/Parser/Python.asdl --- a/Parser/Python.asdl +++ b/Parser/Python.asdl @@ -71,6 +71,8 @@ | Call(expr func, expr* args, keyword* keywords) | Num(object n) -- a number as a PyObject. | Str(string s) -- need to specify raw, unicode, etc? + | FormattedValue(expr value, int? conversion, expr? format_spec) + | JoinedStr(expr* values) | Bytes(bytes s) | NameConstant(singleton value) | Ellipsis diff --git a/Parser/tokenizer.c b/Parser/tokenizer.c --- a/Parser/tokenizer.c +++ b/Parser/tokenizer.c @@ -1477,17 +1477,19 @@ nonascii = 0; if (is_potential_identifier_start(c)) { /* Process b"", r"", u"", br"" and rb"" */ - int saw_b = 0, saw_r = 0, saw_u = 0; + int saw_b = 0, saw_r = 0, saw_u = 0, saw_f = 0; while (1) { - if (!(saw_b || saw_u) && (c == 'b' || c == 'B')) + if (!(saw_b || saw_u || saw_f) && (c == 'b' || c == 'B')) saw_b = 1; /* Since this is a backwards compatibility support literal we don't want to support it in arbitrary order like byte literals. */ - else if (!(saw_b || saw_u || saw_r) && (c == 'u' || c == 'U')) + else if (!(saw_b || saw_u || saw_r || saw_f) && (c == 'u' || c == 'U')) saw_u = 1; /* ur"" and ru"" are not supported */ else if (!(saw_r || saw_u) && (c == 'r' || c == 'R')) saw_r = 1; + else if (!(saw_f || saw_b || saw_u) && (c == 'f' || c == 'F')) + saw_f = 1; else break; c = tok_nextc(tok); diff --git a/Python/Python-ast.c b/Python/Python-ast.c --- a/Python/Python-ast.c +++ b/Python/Python-ast.c @@ -285,6 +285,18 @@ static char *Str_fields[]={ "s", }; +static PyTypeObject *FormattedValue_type; +_Py_IDENTIFIER(conversion); +_Py_IDENTIFIER(format_spec); +static char *FormattedValue_fields[]={ + "value", + "conversion", + "format_spec", +}; +static PyTypeObject *JoinedStr_type; +static char *JoinedStr_fields[]={ + "values", +}; static PyTypeObject *Bytes_type; static char *Bytes_fields[]={ "s", @@ -917,6 +929,11 @@ if (!Num_type) return 0; Str_type = make_type("Str", expr_type, Str_fields, 1); if (!Str_type) return 0; + FormattedValue_type = make_type("FormattedValue", expr_type, + FormattedValue_fields, 3); + if (!FormattedValue_type) return 0; + JoinedStr_type = make_type("JoinedStr", expr_type, JoinedStr_fields, 1); + if (!JoinedStr_type) return 0; Bytes_type = make_type("Bytes", expr_type, Bytes_fields, 1); if (!Bytes_type) return 0; NameConstant_type = make_type("NameConstant", expr_type, @@ -2063,6 +2080,42 @@ } expr_ty +FormattedValue(expr_ty value, int conversion, expr_ty format_spec, int lineno, + int col_offset, PyArena *arena) +{ + expr_ty p; + if (!value) { + PyErr_SetString(PyExc_ValueError, + "field value is required for FormattedValue"); + return NULL; + } + p = (expr_ty)PyArena_Malloc(arena, sizeof(*p)); + if (!p) + return NULL; + p->kind = FormattedValue_kind; + p->v.FormattedValue.value = value; + p->v.FormattedValue.conversion = conversion; + p->v.FormattedValue.format_spec = format_spec; + p->lineno = lineno; + p->col_offset = col_offset; + return p; +} + +expr_ty +JoinedStr(asdl_seq * values, int lineno, int col_offset, PyArena *arena) +{ + expr_ty p; + p = (expr_ty)PyArena_Malloc(arena, sizeof(*p)); + if (!p) + return NULL; + p->kind = JoinedStr_kind; + p->v.JoinedStr.values = values; + p->lineno = lineno; + p->col_offset = col_offset; + return p; +} + +expr_ty Bytes(bytes s, int lineno, int col_offset, PyArena *arena) { expr_ty p; @@ -3161,6 +3214,34 @@ goto failed; Py_DECREF(value); break; + case FormattedValue_kind: + result = PyType_GenericNew(FormattedValue_type, NULL, NULL); + if (!result) goto failed; + value = ast2obj_expr(o->v.FormattedValue.value); + if (!value) goto failed; + if (_PyObject_SetAttrId(result, &PyId_value, value) == -1) + goto failed; + Py_DECREF(value); + value = ast2obj_int(o->v.FormattedValue.conversion); + if (!value) goto failed; + if (_PyObject_SetAttrId(result, &PyId_conversion, value) == -1) + goto failed; + Py_DECREF(value); + value = ast2obj_expr(o->v.FormattedValue.format_spec); + if (!value) goto failed; + if (_PyObject_SetAttrId(result, &PyId_format_spec, value) == -1) + goto failed; + Py_DECREF(value); + break; + case JoinedStr_kind: + result = PyType_GenericNew(JoinedStr_type, NULL, NULL); + if (!result) goto failed; + value = ast2obj_list(o->v.JoinedStr.values, ast2obj_expr); + if (!value) goto failed; + if (_PyObject_SetAttrId(result, &PyId_values, value) == -1) + goto failed; + Py_DECREF(value); + break; case Bytes_kind: result = PyType_GenericNew(Bytes_type, NULL, NULL); if (!result) goto failed; @@ -6022,6 +6103,86 @@ if (*out == NULL) goto failed; return 0; } + isinstance = PyObject_IsInstance(obj, (PyObject*)FormattedValue_type); + if (isinstance == -1) { + return 1; + } + if (isinstance) { + expr_ty value; + int conversion; + expr_ty format_spec; + + if (_PyObject_HasAttrId(obj, &PyId_value)) { + int res; + tmp = _PyObject_GetAttrId(obj, &PyId_value); + if (tmp == NULL) goto failed; + res = obj2ast_expr(tmp, &value, arena); + if (res != 0) goto failed; + Py_CLEAR(tmp); + } else { + PyErr_SetString(PyExc_TypeError, "required field \"value\" missing from FormattedValue"); + return 1; + } + if (exists_not_none(obj, &PyId_conversion)) { + int res; + tmp = _PyObject_GetAttrId(obj, &PyId_conversion); + if (tmp == NULL) goto failed; + res = obj2ast_int(tmp, &conversion, arena); + if (res != 0) goto failed; + Py_CLEAR(tmp); + } else { + conversion = 0; + } + if (exists_not_none(obj, &PyId_format_spec)) { + int res; + tmp = _PyObject_GetAttrId(obj, &PyId_format_spec); + if (tmp == NULL) goto failed; + res = obj2ast_expr(tmp, &format_spec, arena); + if (res != 0) goto failed; + Py_CLEAR(tmp); + } else { + format_spec = NULL; + } + *out = FormattedValue(value, conversion, format_spec, lineno, + col_offset, arena); + if (*out == NULL) goto failed; + return 0; + } + isinstance = PyObject_IsInstance(obj, (PyObject*)JoinedStr_type); + if (isinstance == -1) { + return 1; + } + if (isinstance) { + asdl_seq* values; + + if (_PyObject_HasAttrId(obj, &PyId_values)) { + int res; + Py_ssize_t len; + Py_ssize_t i; + tmp = _PyObject_GetAttrId(obj, &PyId_values); + if (tmp == NULL) goto failed; + if (!PyList_Check(tmp)) { + PyErr_Format(PyExc_TypeError, "JoinedStr field \"values\" must be a list, not a %.200s", tmp->ob_type->tp_name); + goto failed; + } + len = PyList_GET_SIZE(tmp); + values = _Py_asdl_seq_new(len, arena); + if (values == NULL) goto failed; + for (i = 0; i < len; i++) { + expr_ty value; + res = obj2ast_expr(PyList_GET_ITEM(tmp, i), &value, arena); + if (res != 0) goto failed; + asdl_seq_SET(values, i, value); + } + Py_CLEAR(tmp); + } else { + PyErr_SetString(PyExc_TypeError, "required field \"values\" missing from JoinedStr"); + return 1; + } + *out = JoinedStr(values, lineno, col_offset, arena); + if (*out == NULL) goto failed; + return 0; + } isinstance = PyObject_IsInstance(obj, (PyObject*)Bytes_type); if (isinstance == -1) { return 1; @@ -7319,6 +7480,10 @@ if (PyDict_SetItemString(d, "Call", (PyObject*)Call_type) < 0) return NULL; if (PyDict_SetItemString(d, "Num", (PyObject*)Num_type) < 0) return NULL; if (PyDict_SetItemString(d, "Str", (PyObject*)Str_type) < 0) return NULL; + if (PyDict_SetItemString(d, "FormattedValue", + (PyObject*)FormattedValue_type) < 0) return NULL; + if (PyDict_SetItemString(d, "JoinedStr", (PyObject*)JoinedStr_type) < 0) + return NULL; if (PyDict_SetItemString(d, "Bytes", (PyObject*)Bytes_type) < 0) return NULL; if (PyDict_SetItemString(d, "NameConstant", (PyObject*)NameConstant_type) < diff --git a/Python/ast.c b/Python/ast.c --- a/Python/ast.c +++ b/Python/ast.c @@ -257,6 +257,14 @@ } return 1; } + case JoinedStr_kind: + return validate_exprs(exp->v.JoinedStr.values, Load, 0); + case FormattedValue_kind: + if (validate_expr(exp->v.FormattedValue.value, Load) == 0) + return 0; + if (exp->v.FormattedValue.format_spec) + return validate_expr(exp->v.FormattedValue.format_spec, Load); + return 1; case Bytes_kind: { PyObject *b = exp->v.Bytes.s; if (!PyBytes_CheckExact(b)) { @@ -535,9 +543,7 @@ static expr_ty ast_for_call(struct compiling *, const node *, expr_ty); static PyObject *parsenumber(struct compiling *, const char *); -static PyObject *parsestr(struct compiling *, const node *n, int *bytesmode); -static PyObject *parsestrplus(struct compiling *, const node *n, - int *bytesmode); +static expr_ty parsestrplus(struct compiling *, const node *n); #define COMP_GENEXP 0 #define COMP_LISTCOMP 1 @@ -986,6 +992,8 @@ case Num_kind: case Str_kind: case Bytes_kind: + case JoinedStr_kind: + case FormattedValue_kind: expr_name = "literal"; break; case NameConstant_kind: @@ -2001,7 +2009,6 @@ | '...' | 'None' | 'True' | 'False' */ node *ch = CHILD(n, 0); - int bytesmode = 0; switch (TYPE(ch)) { case NAME: { @@ -2023,7 +2030,7 @@ return Name(name, Load, LINENO(n), n->n_col_offset, c->c_arena); } case STRING: { - PyObject *str = parsestrplus(c, n, &bytesmode); + expr_ty str = parsestrplus(c, n); if (!str) { const char *errtype = NULL; if (PyErr_ExceptionMatches(PyExc_UnicodeError)) @@ -2050,14 +2057,7 @@ } return NULL; } - if (PyArena_AddPyObject(c->c_arena, str) < 0) { - Py_DECREF(str); - return NULL; - } - if (bytesmode) - return Bytes(str, LINENO(n), n->n_col_offset, c->c_arena); - else - return Str(str, LINENO(n), n->n_col_offset, c->c_arena); + return str; } case NUMBER: { PyObject *pynum = parsenumber(c, STR(ch)); @@ -4002,12 +4002,838 @@ return v; } -/* s is a Python string literal, including the bracketing quote characters, - * and r &/or b prefixes (if any), and embedded escape sequences (if any). - * parsestr parses it, and returns the decoded Python string object. - */ +/* Compile this expression in to an expr_ty. We know that we can + temporarily modify the character before the start of this string + (it's '{'), and we know we can temporarily modify the character + after this string (it is a '}'). Leverage this to create a + sub-string with enough room for us to add parens around the + expression. This is to allow strings with embedded newlines, for + example. */ +static expr_ty +fstring_expression_compile(PyObject *str, Py_ssize_t expr_start, + Py_ssize_t expr_end, PyArena *arena) +{ + PyCompilerFlags cf; + mod_ty mod; + char *utf_expr; + Py_ssize_t i; + int all_whitespace; + PyObject *sub = NULL; + + /* We only decref sub if we allocated it with a PyUnicode_Substring. + decref_sub records that. */ + int decref_sub = 0; + + assert(str); + + /* If the substring is all whitespace, it's an error. We need to + catch this here, and not when we call PyParser_ASTFromString, + because turning the expression '' in to '()' would go from + being invalid to valid. */ + /* Note that this code says an empty string is all + whitespace. That's important. There's a test for it: f'{}'. */ + all_whitespace = 1; + for (i = expr_start; i < expr_end; i++) { + if (!Py_UNICODE_ISSPACE(PyUnicode_READ_CHAR(str, i))) { + all_whitespace = 0; + break; + } + } + if (all_whitespace) { + PyErr_SetString(PyExc_SyntaxError, "f-string: empty expression " + "not allowed"); + goto error; + } + + /* If the substring will be the entire source string, we can't use + PyUnicode_Substring, since it will return another reference to + our original string. Because we're modifying the string in + place, that's a no-no. So, detect that case and just use our + string directly. */ + + if (expr_start-1 == 0 && expr_end+1 == PyUnicode_GET_LENGTH(str)) { + /* No need to actually remember these characters, because we + know they must be braces. */ + assert(PyUnicode_ReadChar(str, 0) == '{'); + assert(PyUnicode_ReadChar(str, expr_end-expr_start+1) == '}'); + sub = str; + } else { + /* Create a substring object. It must be a new object, with + refcount==1, so that we can modify it. */ + sub = PyUnicode_Substring(str, expr_start-1, expr_end+1); + if (!sub) + goto error; + assert(sub != str); /* Make sure it's a new string. */ + decref_sub = 1; /* Remember to deallocate it on error. */ + } + + if (PyUnicode_WriteChar(sub, 0, '(') < 0 || + PyUnicode_WriteChar(sub, expr_end-expr_start+1, ')') < 0) + goto error; + + cf.cf_flags = PyCF_ONLY_AST; + + /* No need to free the memory returned here: it's managed by the + string. */ + utf_expr = PyUnicode_AsUTF8(sub); + if (!utf_expr) + goto error; + mod = PyParser_ASTFromString(utf_expr, "", + Py_eval_input, &cf, arena); + if (!mod) + goto error; + if (sub != str) + /* Clear instead of decref in case we ever modify this code to change + the error handling: this is safest because the XDECREF won't try + and decref it when it's NULL. */ + /* No need to restore the chars in sub, since we know it's getting + ready to get deleted (refcount must be 1, since we got a new string + in PyUnicode_Substring). */ + Py_CLEAR(sub); + else { + assert(!decref_sub); + /* Restore str, which we earlier modified directly. */ + if (PyUnicode_WriteChar(str, 0, '{') < 0 || + PyUnicode_WriteChar(str, expr_end-expr_start+1, '}') < 0) + goto error; + } + return mod->v.Expression.body; + +error: + /* Only decref sub if it was the result of a call to SubString. */ + if (decref_sub) + Py_XDECREF(sub); + return NULL; +} + +/* Return -1 on error. + + Return 0 if we reached the end of the literal. + + Return 1 if we haven't reached the end of the literal, but we want + the caller to process the literal up to this point. Used for + doubled braces. +*/ +static int +fstring_find_literal(PyObject *str, Py_ssize_t *ofs, PyObject **literal, + int recurse_lvl, struct compiling *c, const node *n) +{ + /* Get any literal string. It ends when we hit an un-doubled brace, or the + end of the string. */ + + Py_ssize_t literal_start, literal_end; + int result = 0; + + enum PyUnicode_Kind kind = PyUnicode_KIND(str); + void *data = PyUnicode_DATA(str); + + assert(*literal == NULL); + + literal_start = *ofs; + for (; *ofs < PyUnicode_GET_LENGTH(str); *ofs += 1) { + Py_UCS4 ch = PyUnicode_READ(kind, data, *ofs); + if (ch == '{' || ch == '}') { + /* Check for doubled braces, but only at the top level. If + we checked at every level, then f'{0:{3}}' would fail + with the two closing braces. */ + if (recurse_lvl == 0) { + if (*ofs + 1 < PyUnicode_GET_LENGTH(str) && + PyUnicode_READ(kind, data, *ofs + 1) == ch) { + /* We're going to tell the caller that the literal ends + here, but that they should continue scanning. But also + skip over the second brace when we resume scanning. */ + literal_end = *ofs + 1; + *ofs += 2; + result = 1; + goto done; + } + + /* Where a single '{' is the start of a new expression, a + single '}' is not allowed. */ + if (ch == '}') { + ast_error(c, n, "f-string: single '}' is not allowed"); + return -1; + } + } + + /* We're either at a '{', which means we're starting another + expression; or a '}', which means we're at the end of this + f-string (for a nested format_spec). */ + break; + } + } + literal_end = *ofs; + + assert(*ofs == PyUnicode_GET_LENGTH(str) || + PyUnicode_READ(kind, data, *ofs) == '{' || + PyUnicode_READ(kind, data, *ofs) == '}'); +done: + if (literal_start != literal_end) { + *literal = PyUnicode_Substring(str, literal_start, literal_end); + if (!*literal) + return -1; + } + + return result; +} + +/* Forward declaration because parsing is recursive. */ +static expr_ty +fstring_parse(PyObject *str, Py_ssize_t *ofs, int recurse_lvl, + struct compiling *c, const node *n); + +/* Parse the f-string str, starting at ofs. We know *ofs starts an + expression (so it must be a '{'). Returns the FormattedValue node, + which includes the expression, conversion character, and + format_spec expression. + + Note that I don't do a perfect job here: I don't make sure that a + closing brace doesn't match an opening paren, for example. It + doesn't need to error on all invalid expressions, just correctly + find the end of all valid ones. Any errors inside the expression + will be caught when we parse it later. */ +static int +fstring_find_expr(PyObject *str, Py_ssize_t *ofs, int recurse_lvl, + expr_ty *expression, struct compiling *c, const node *n) +{ + /* Return -1 on error, else 0. */ + + Py_ssize_t expr_start; + Py_ssize_t expr_end; + expr_ty simple_expression; + expr_ty format_spec = NULL; /* Optional format specifier. */ + Py_UCS4 conversion = -1; /* The conversion char. -1 if not specified. */ + + enum PyUnicode_Kind kind = PyUnicode_KIND(str); + void *data = PyUnicode_DATA(str); + + /* 0 if we're not in a string, else the quote char we're trying to + match (single or double quote). */ + Py_UCS4 quote_char = 0; + + /* If we're inside a string, 1=normal, 3=triple-quoted. */ + int string_type = 0; + + /* Keep track of nesting level for braces/parens/brackets in + expressions. */ + Py_ssize_t nested_depth = 0; + + /* Can only nest one level deep. */ + if (recurse_lvl >= 2) { + ast_error(c, n, "f-string: expressions nested too deeply"); + return -1; + } + + /* The first char must be a left brace, or we wouldn't have gotten + here. Skip over it. */ + assert(PyUnicode_READ(kind, data, *ofs) == '{'); + *ofs += 1; + + expr_start = *ofs; + for (; *ofs < PyUnicode_GET_LENGTH(str); *ofs += 1) { + Py_UCS4 ch; + + /* Loop invariants. */ + assert(nested_depth >= 0); + assert(*ofs >= expr_start); + if (quote_char) + assert(string_type == 1 || string_type == 3); + else + assert(string_type == 0); + + ch = PyUnicode_READ(kind, data, *ofs); + if (quote_char) { + /* We're inside a string. See if we're at the end. */ + /* This code needs to implement the same non-error logic + as tok_get from tokenizer.c, at the letter_quote + label. To actually share that code would be a + nightmare. But, it's unlikely to change and is small, + so duplicate it here. Note we don't need to catch all + of the errors, since they'll be caught when parsing the + expression. We just need to match the non-error + cases. Thus we can ignore \n in single-quoted strings, + for example. Or non-terminated strings. */ + if (ch == quote_char) { + /* Does this match the string_type (single or triple + quoted)? */ + if (string_type == 3) { + if (*ofs+2 < PyUnicode_GET_LENGTH(str) && + PyUnicode_READ(kind, data, *ofs+1) == ch && + PyUnicode_READ(kind, data, *ofs+2) == ch) { + /* We're at the end of a triple quoted string. */ + *ofs += 2; + string_type = 0; + quote_char = 0; + continue; + } + } else { + /* We're at the end of a normal string. */ + quote_char = 0; + string_type = 0; + continue; + } + } + /* We're inside a string, and not finished with the + string. If this is a backslash, skip the next char (it + might be an end quote that needs skipping). Otherwise, + just consume this character normally. */ + if (ch == '\\' && *ofs+1 < PyUnicode_GET_LENGTH(str)) { + /* Just skip the next char, whatever it is. */ + *ofs += 1; + } + } else if (ch == '\'' || ch == '"') { + /* Is this a triple quoted string? */ + if (*ofs+2 < PyUnicode_GET_LENGTH(str) && + PyUnicode_READ(kind, data, *ofs+1) == ch && + PyUnicode_READ(kind, data, *ofs+2) == ch) { + string_type = 3; + *ofs += 2; + } else { + /* Start of a normal string. */ + string_type = 1; + } + /* Start looking for the end of the string. */ + quote_char = ch; + } else if (ch == '[' || ch == '{' || ch == '(') { + nested_depth++; + } else if (nested_depth != 0 && + (ch == ']' || ch == '}' || ch == ')')) { + nested_depth--; + } else if (ch == '#') { + /* Error: can't include a comment character, inside parens + or not. */ + ast_error(c, n, "f-string cannot include '#'"); + return -1; + } else if (nested_depth == 0 && + (ch == '!' || ch == ':' || ch == '}')) { + /* First, test for the special case of "!=". Since '=' is + not an allowed conversion character, nothing is lost in + this test. */ + if (ch == '!' && *ofs+1 < PyUnicode_GET_LENGTH(str) && + PyUnicode_READ(kind, data, *ofs+1) == '=') + /* This isn't a conversion character, just continue. */ + continue; + + /* Normal way out of this loop. */ + break; + } else { + /* Just consume this char and loop around. */ + } + } + expr_end = *ofs; + /* If we leave this loop in a string or with mismatched parens, we + don't care. We'll get a syntax error when compiling the + expression. But, we can produce a better error message, so + let's just do that.*/ + if (quote_char) { + ast_error(c, n, "f-string: unterminated string"); + return -1; + } + if (nested_depth) { + ast_error(c, n, "f-string: mismatched '(', '{', or '['"); + return -1; + } + + /* Check for a conversion char, if present. */ + if (*ofs >= PyUnicode_GET_LENGTH(str)) + goto unexpected_end_of_string; + if (PyUnicode_READ(kind, data, *ofs) == '!') { + *ofs += 1; + if (*ofs >= PyUnicode_GET_LENGTH(str)) + goto unexpected_end_of_string; + + conversion = PyUnicode_READ(kind, data, *ofs); + *ofs += 1; + + /* Validate the conversion. */ + if (!(conversion == 's' || conversion == 'r' + || conversion == 'a')) { + ast_error(c, n, "f-string: invalid conversion character: " + "expected 's', 'r', or 'a'"); + return -1; + } + } + + /* Check for the format spec, if present. */ + if (*ofs >= PyUnicode_GET_LENGTH(str)) + goto unexpected_end_of_string; + if (PyUnicode_READ(kind, data, *ofs) == ':') { + *ofs += 1; + if (*ofs >= PyUnicode_GET_LENGTH(str)) + goto unexpected_end_of_string; + + /* Parse the format spec. */ + format_spec = fstring_parse(str, ofs, recurse_lvl+1, c, n); + if (!format_spec) + return -1; + } + + if (*ofs >= PyUnicode_GET_LENGTH(str) || + PyUnicode_READ(kind, data, *ofs) != '}') + goto unexpected_end_of_string; + + /* We're at a right brace. Consume it. */ + assert(*ofs < PyUnicode_GET_LENGTH(str)); + assert(PyUnicode_READ(kind, data, *ofs) == '}'); + *ofs += 1; + + /* Compile the expression. */ + simple_expression = fstring_expression_compile(str, expr_start, expr_end, + c->c_arena); + if (!simple_expression) + return -1; + + /* And now create the FormattedValue node that represents this entire + expression with the conversion and format spec. */ + *expression = FormattedValue(simple_expression, (int)conversion, + format_spec, LINENO(n), n->n_col_offset, + c->c_arena); + if (!*expression) + return -1; + + return 0; + +unexpected_end_of_string: + ast_error(c, n, "f-string: expecting '}'"); + return -1; +} + +/* Return -1 on error. + + Return 0 if we have a literal (possible zero length) and an + expression (zero length if at the end of the string. + + Return 1 if we have a literal, but no expression, and we want the + caller to call us again. This is used to deal with doubled + braces. + + When called multiple times on the string 'a{{b{0}c', this function + will return: + + 1. the literal 'a{' with no expression, and a return value + of 1. Despite the fact that there's no expression, the return + value of 1 means we're not finished yet. + + 2. the literal 'b' and the expression '0', with a return value of + 0. The fact that there's an expression means we're not finished. + + 3. literal 'c' with no expression and a return value of 0. The + combination of the return value of 0 with no expression means + we're finished. +*/ +static int +fstring_find_literal_and_expr(PyObject *str, Py_ssize_t *ofs, int recurse_lvl, + PyObject **literal, expr_ty *expression, + struct compiling *c, const node *n) +{ + int result; + + assert(*literal == NULL && *expression == NULL); + + /* Get any literal string. */ + result = fstring_find_literal(str, ofs, literal, recurse_lvl, c, n); + if (result < 0) + goto error; + + assert(result == 0 || result == 1); + + if (result == 1) + /* We have a literal, but don't look at the expression. */ + return 1; + + assert(*ofs <= PyUnicode_GET_LENGTH(str)); + + if (*ofs >= PyUnicode_GET_LENGTH(str) || + PyUnicode_READ_CHAR(str, *ofs) == '}') + /* We're at the end of the string or the end of a nested + f-string: no expression. The top-level error case where we + expect to be at the end of the string but we're at a '}' is + handled later. */ + return 0; + + /* We must now be the start of an expression, on a '{'. */ + assert(*ofs < PyUnicode_GET_LENGTH(str) && + PyUnicode_READ_CHAR(str, *ofs) == '{'); + + if (fstring_find_expr(str, ofs, recurse_lvl, expression, c, n) < 0) + goto error; + + return 0; + +error: + Py_XDECREF(*literal); + *literal = NULL; + return -1; +} + +#define EXPRLIST_N_CACHED 64 + +typedef struct { + /* Incrementally build an array of expr_ty, so be used in an + asdl_seq. Cache some small but reasonably sized number of + expr_ty's, and then after that start dynamically allocating, + doubling the number allocated each time. Note that the f-string + f'{0}a{1}' contains 3 expr_ty's: 2 FormattedValue's, and one + Str for the literal 'a'. So you add expr_ty's about twice as + fast as you add exressions in an f-string. */ + + Py_ssize_t allocated; /* Number we've allocated. */ + Py_ssize_t size; /* Number we've used. */ + expr_ty *p; /* Pointer to the memory we're actually + using. Will point to 'data' until we + start dynamically allocating. */ + expr_ty data[EXPRLIST_N_CACHED]; +} ExprList; + +#ifdef NDEBUG +#define ExprList_check_invariants(l) +#else +static void +ExprList_check_invariants(ExprList *l) +{ + /* Check our invariants. Make sure this object is "live", and + hasn't been deallocated. */ + assert(l->size >= 0); + assert(l->p != NULL); + if (l->size <= EXPRLIST_N_CACHED) + assert(l->data == l->p); +} +#endif + +static void +ExprList_Init(ExprList *l) +{ + l->allocated = EXPRLIST_N_CACHED; + l->size = 0; + + /* Until we start allocating dynamically, p points to data. */ + l->p = l->data; + + ExprList_check_invariants(l); +} + +static int +ExprList_Append(ExprList *l, expr_ty exp) +{ + ExprList_check_invariants(l); + if (l->size >= l->allocated) { + /* We need to alloc (or realloc) the memory. */ + Py_ssize_t new_size = l->allocated * 2; + + /* See if we've ever allocated anything dynamically. */ + if (l->p == l->data) { + Py_ssize_t i; + /* We're still using the cached data. Switch to + alloc-ing. */ + l->p = PyMem_RawMalloc(sizeof(expr_ty) * new_size); + if (!l->p) + return -1; + /* Copy the cached data into the new buffer. */ + for (i = 0; i < l->size; i++) + l->p[i] = l->data[i]; + } else { + /* Just realloc. */ + expr_ty *tmp = PyMem_RawRealloc(l->p, sizeof(expr_ty) * new_size); + if (!tmp) { + PyMem_RawFree(l->p); + l->p = NULL; + return -1; + } + l->p = tmp; + } + + l->allocated = new_size; + assert(l->allocated == 2 * l->size); + } + + l->p[l->size++] = exp; + + ExprList_check_invariants(l); + return 0; +} + +static void +ExprList_Dealloc(ExprList *l) +{ + ExprList_check_invariants(l); + + /* If there's been an error, or we've never dynamically allocated, + do nothing. */ + if (!l->p || l->p == l->data) { + /* Do nothing. */ + } else { + /* We have dynamically allocated. Free the memory. */ + PyMem_RawFree(l->p); + } + l->p = NULL; + l->size = -1; +} + +static asdl_seq * +ExprList_Finish(ExprList *l, PyArena *arena) +{ + asdl_seq *seq; + + ExprList_check_invariants(l); + + /* Allocate the asdl_seq and copy the expressions in to it. */ + seq = _Py_asdl_seq_new(l->size, arena); + if (seq) { + Py_ssize_t i; + for (i = 0; i < l->size; i++) + asdl_seq_SET(seq, i, l->p[i]); + } + ExprList_Dealloc(l); + return seq; +} + +/* The FstringParser is designed to add a mix of strings and + f-strings, and concat them together as needed. Ultimately, it + generates an expr_ty. */ +typedef struct { + PyObject *last_str; + ExprList expr_list; +} FstringParser; + +#ifdef NDEBUG +#define FstringParser_check_invariants(state) +#else +static void +FstringParser_check_invariants(FstringParser *state) +{ + if (state->last_str) + assert(PyUnicode_CheckExact(state->last_str)); + ExprList_check_invariants(&state->expr_list); +} +#endif + +static void +FstringParser_Init(FstringParser *state) +{ + state->last_str = NULL; + ExprList_Init(&state->expr_list); + FstringParser_check_invariants(state); +} + +static void +FstringParser_Dealloc(FstringParser *state) +{ + FstringParser_check_invariants(state); + + Py_XDECREF(state->last_str); + ExprList_Dealloc(&state->expr_list); +} + +/* Make a Str node, but decref the PyUnicode object being added. */ +static expr_ty +make_str_node_and_del(PyObject **str, struct compiling *c, const node* n) +{ + PyObject *s = *str; + *str = NULL; + assert(PyUnicode_CheckExact(s)); + if (PyArena_AddPyObject(c->c_arena, s) < 0) { + Py_DECREF(s); + return NULL; + } + return Str(s, LINENO(n), n->n_col_offset, c->c_arena); +} + +/* Add a non-f-string (that is, a regular literal string). str is + decref'd. */ +static int +FstringParser_ConcatAndDel(FstringParser *state, PyObject *str) +{ + FstringParser_check_invariants(state); + + assert(PyUnicode_CheckExact(str)); + + if (PyUnicode_GET_LENGTH(str) == 0) { + Py_DECREF(str); + return 0; + } + + if (!state->last_str) { + /* We didn't have a string before, so just remember this one. */ + state->last_str = str; + } else { + /* Concatenate this with the previous string. */ + PyObject *temp = PyUnicode_Concat(state->last_str, str); + Py_DECREF(state->last_str); + Py_DECREF(str); + state->last_str = temp; + if (!temp) + return -1; + } + FstringParser_check_invariants(state); + return 0; +} + +/* Parse an f-string. The f-string is in str, starting at ofs, with no 'f' + or quotes. str is not decref'd, since we don't know if it's used elsewhere. + And if we're only looking at a part of a string, then decref'ing is + definitely not the right thing to do! */ +static int +FstringParser_ConcatFstring(FstringParser *state, PyObject *str, + Py_ssize_t *ofs, int recurse_lvl, + struct compiling *c, const node *n) +{ + FstringParser_check_invariants(state); + + /* Parse the f-string. */ + while (1) { + PyObject *literal = NULL; + expr_ty expression = NULL; + + /* If there's a zero length literal in front of the + expression, literal will be NULL. If we're at the end of + the f-string, expression will be NULL (unless result == 1, + see below). */ + int result = fstring_find_literal_and_expr(str, ofs, recurse_lvl, + &literal, &expression, + c, n); + if (result < 0) + return -1; + + /* Add the literal, if any. */ + if (!literal) { + /* Do nothing. Just leave last_str alone (and possibly + NULL). */ + } else if (!state->last_str) { + state->last_str = literal; + literal = NULL; + } else { + /* We have a literal, concatenate it. */ + assert(PyUnicode_GET_LENGTH(literal) != 0); + if (FstringParser_ConcatAndDel(state, literal) < 0) + return -1; + literal = NULL; + } + assert(!state->last_str || + PyUnicode_GET_LENGTH(state->last_str) != 0); + + /* We've dealt with the literal now. It can't be leaked on further + errors. */ + assert(literal == NULL); + + /* See if we should just loop around to get the next literal + and expression, while ignoring the expression this + time. This is used for un-doubling braces, as an + optimization. */ + if (result == 1) + continue; + + if (!expression) + /* We're done with this f-string. */ + break; + + /* We know we have an expression. Convert any existing string + to a Str node. */ + if (!state->last_str) { + /* Do nothing. No previous literal. */ + } else { + /* Convert the existing last_str literal to a Str node. */ + expr_ty str = make_str_node_and_del(&state->last_str, c, n); + if (!str || ExprList_Append(&state->expr_list, str) < 0) + return -1; + } + + if (ExprList_Append(&state->expr_list, expression) < 0) + return -1; + } + + assert(*ofs <= PyUnicode_GET_LENGTH(str)); + + /* If recurse_lvl is zero, then we must be at the end of the + string. Otherwise, we must be at a right brace. */ + + if (recurse_lvl == 0 && *ofs < PyUnicode_GET_LENGTH(str)) { + ast_error(c, n, "f-string: unexpected end of string"); + return -1; + } + if (recurse_lvl != 0 && PyUnicode_READ_CHAR(str, *ofs) != '}') { + ast_error(c, n, "f-string: expecting '}'"); + return -1; + } + + FstringParser_check_invariants(state); + return 0; +} + +/* Convert the partial state reflected in last_str and expr_list to an + expr_ty. The expr_ty can be a Str, or a JoinedStr. */ +static expr_ty +FstringParser_Finish(FstringParser *state, struct compiling *c, + const node *n) +{ + asdl_seq *seq; + + FstringParser_check_invariants(state); + + /* If we're just a constant string with no expressions, return + that. */ + if(state->expr_list.size == 0) { + if (!state->last_str) { + /* Create a zero length string. */ + state->last_str = PyUnicode_FromStringAndSize(NULL, 0); + if (!state->last_str) + goto error; + } + return make_str_node_and_del(&state->last_str, c, n); + } + + /* Create a Str node out of last_str, if needed. It will be the + last node in our expression list. */ + if (state->last_str) { + expr_ty str = make_str_node_and_del(&state->last_str, c, n); + if (!str || ExprList_Append(&state->expr_list, str) < 0) + goto error; + } + /* This has already been freed. */ + assert(state->last_str == NULL); + + seq = ExprList_Finish(&state->expr_list, c->c_arena); + if (!seq) + goto error; + + /* If there's only one expression, return it. Otherwise, we need + to join them together. */ + if (seq->size == 1) + return seq->elements[0]; + + return JoinedStr(seq, LINENO(n), n->n_col_offset, c->c_arena); + +error: + FstringParser_Dealloc(state); + return NULL; +} + +/* Given an f-string (with no 'f' or quotes) that's in str starting at + ofs, parse it into an expr_ty. Return NULL on error. Does not + decref str. */ +static expr_ty +fstring_parse(PyObject *str, Py_ssize_t *ofs, int recurse_lvl, + struct compiling *c, const node *n) +{ + FstringParser state; + + FstringParser_Init(&state); + if (FstringParser_ConcatFstring(&state, str, ofs, recurse_lvl, + c, n) < 0) { + FstringParser_Dealloc(&state); + return NULL; + } + + return FstringParser_Finish(&state, c, n); +} + +/* n is a Python string literal, including the bracketing quote + characters, and r, b, u, &/or f prefixes (if any), and embedded + escape sequences (if any). parsestr parses it, and returns the + decoded Python string object. If the string is an f-string, set + *fmode and return the unparsed string object. +*/ static PyObject * -parsestr(struct compiling *c, const node *n, int *bytesmode) +parsestr(struct compiling *c, const node *n, int *bytesmode, int *fmode) { size_t len; const char *s = STR(n); @@ -4027,15 +4853,24 @@ quote = *++s; rawmode = 1; } + else if (quote == 'f' || quote == 'F') { + quote = *++s; + *fmode = 1; + } else { break; } } } + if (*fmode && *bytesmode) { + PyErr_BadInternalCall(); + return NULL; + } if (quote != '\'' && quote != '\"') { PyErr_BadInternalCall(); return NULL; } + /* Skip the leading quote char. */ s++; len = strlen(s); if (len > INT_MAX) { @@ -4044,12 +4879,17 @@ return NULL; } if (s[--len] != quote) { + /* Last quote char must match the first. */ PyErr_BadInternalCall(); return NULL; } if (len >= 4 && s[0] == quote && s[1] == quote) { + /* A triple quoted string. We've already skipped one quote at + the start and one at the end of the string. Now skip the + two at the start. */ s += 2; len -= 2; + /* And check that the last two match. */ if (s[--len] != quote || s[--len] != quote) { PyErr_BadInternalCall(); return NULL; @@ -4088,51 +4928,84 @@ } } return PyBytes_DecodeEscape(s, len, NULL, 1, - need_encoding ? c->c_encoding : NULL); + need_encoding ? c->c_encoding : NULL); } -/* Build a Python string object out of a STRING+ atom. This takes care of - * compile-time literal catenation, calling parsestr() on each piece, and - * pasting the intermediate results together. - */ -static PyObject * -parsestrplus(struct compiling *c, const node *n, int *bytesmode) +/* Accepts a STRING+ atom, and produces an expr_ty node. Run through + each STRING atom, and process it as needed. For bytes, just + concatenate them together, and the result will be a Bytes node. For + normal strings and f-strings, concatenate them together. The result + will be a Str node if there were no f-strings; a FormattedValue + node if there's just an f-string (with no leading or trailing + literals), or a JoinedStr node if there are multiple f-strings or + any literals involved. */ +static expr_ty +parsestrplus(struct compiling *c, const node *n) { - PyObject *v; + int bytesmode = 0; + PyObject *bytes_str = NULL; int i; - REQ(CHILD(n, 0), STRING); - v = parsestr(c, CHILD(n, 0), bytesmode); - if (v != NULL) { - /* String literal concatenation */ - for (i = 1; i < NCH(n); i++) { - PyObject *s; - int subbm = 0; - s = parsestr(c, CHILD(n, i), &subbm); - if (s == NULL) - goto onError; - if (*bytesmode != subbm) { - ast_error(c, n, "cannot mix bytes and nonbytes literals"); - Py_DECREF(s); - goto onError; + + FstringParser state; + FstringParser_Init(&state); + + for (i = 0; i < NCH(n); i++) { + int this_bytesmode = 0; + int this_fmode = 0; + PyObject *s; + + REQ(CHILD(n, i), STRING); + s = parsestr(c, CHILD(n, i), &this_bytesmode, &this_fmode); + if (!s) + goto error; + + /* Check that we're not mixing bytes with unicode. */ + if (i != 0 && bytesmode != this_bytesmode) { + ast_error(c, n, "cannot mix bytes and nonbytes literals"); + Py_DECREF(s); + goto error; + } + bytesmode = this_bytesmode; + + assert(bytesmode ? PyBytes_CheckExact(s) : PyUnicode_CheckExact(s)); + + if (bytesmode) { + /* For bytes, concat as we go. */ + if (i == 0) { + /* First time, just remember this value. */ + bytes_str = s; + } else { + PyBytes_ConcatAndDel(&bytes_str, s); + if (!bytes_str) + goto error; } - if (PyBytes_Check(v) && PyBytes_Check(s)) { - PyBytes_ConcatAndDel(&v, s); - if (v == NULL) - goto onError; - } - else { - PyObject *temp = PyUnicode_Concat(v, s); - Py_DECREF(s); - Py_DECREF(v); - v = temp; - if (v == NULL) - goto onError; - } + } else if (this_fmode) { + /* This is an f-string. Concatenate and decref it. */ + Py_ssize_t ofs = 0; + int result = FstringParser_ConcatFstring(&state, s, &ofs, 0, c, n); + Py_DECREF(s); + if (result < 0) + goto error; + } else { + /* This is a regular string. Concatenate it. */ + if (FstringParser_ConcatAndDel(&state, s) < 0) + goto error; } } - return v; - - onError: - Py_XDECREF(v); + if (bytesmode) { + /* Just return the bytes object and we're done. */ + if (PyArena_AddPyObject(c->c_arena, bytes_str) < 0) + goto error; + return Bytes(bytes_str, LINENO(n), n->n_col_offset, c->c_arena); + } + + /* We're not a bytes string, bytes_str should never have been set. */ + assert(bytes_str == NULL); + + return FstringParser_Finish(&state, c, n); + +error: + Py_XDECREF(bytes_str); + FstringParser_Dealloc(&state); return NULL; } diff --git a/Python/compile.c b/Python/compile.c --- a/Python/compile.c +++ b/Python/compile.c @@ -731,6 +731,7 @@ return 1; } + /* Allocate a new block and return a pointer to it. Returns NULL on error. */ @@ -3209,6 +3210,117 @@ e->v.Call.keywords); } +static int +compiler_joined_str(struct compiler *c, expr_ty e) +{ + /* Concatenate parts of a string using ''.join(parts). There are + probably better ways of doing this. + + This is used for constructs like "'x=' f'{42}'", which have to + be evaluated at compile time. */ + + static PyObject *empty_string; + static PyObject *join_string; + + if (!empty_string) { + empty_string = PyUnicode_FromString(""); + if (!empty_string) + return 0; + } + if (!join_string) { + join_string = PyUnicode_FromString("join"); + if (!join_string) + return 0; + } + + ADDOP_O(c, LOAD_CONST, empty_string, consts); + ADDOP_NAME(c, LOAD_ATTR, join_string, names); + VISIT_SEQ(c, expr, e->v.JoinedStr.values); + ADDOP_I(c, BUILD_LIST, asdl_seq_LEN(e->v.JoinedStr.values)); + ADDOP_I(c, CALL_FUNCTION, 1); + return 1; +} + +/* Note that this code uses the builtin functions format(), str(), + repr(), and ascii(). You can break this code, or make it do odd + things, by redefining those functions. */ +static int +compiler_formatted_value(struct compiler *c, expr_ty e) +{ + PyObject *conversion_name = NULL; + + static PyObject *format_string; + static PyObject *str_string; + static PyObject *repr_string; + static PyObject *ascii_string; + + if (!format_string) { + format_string = PyUnicode_InternFromString("format"); + if (!format_string) + return 0; + } + + if (!str_string) { + str_string = PyUnicode_InternFromString("str"); + if (!str_string) + return 0; + } + + if (!repr_string) { + repr_string = PyUnicode_InternFromString("repr"); + if (!repr_string) + return 0; + } + if (!ascii_string) { + ascii_string = PyUnicode_InternFromString("ascii"); + if (!ascii_string) + return 0; + } + + ADDOP_NAME(c, LOAD_GLOBAL, format_string, names); + + /* If needed, convert via str, repr, or ascii. */ + if (e->v.FormattedValue.conversion != -1) { + switch (e->v.FormattedValue.conversion) { + case 's': + conversion_name = str_string; + break; + case 'r': + conversion_name = repr_string; + break; + case 'a': + conversion_name = ascii_string; + break; + default: + PyErr_SetString(PyExc_SystemError, + "Unrecognized conversion character"); + return 0; + } + ADDOP_NAME(c, LOAD_GLOBAL, conversion_name, names); + } + + /* Evaluate the value. */ + VISIT(c, expr, e->v.FormattedValue.value); + + /* If needed, convert via str, repr, or ascii. */ + if (conversion_name) { + /* Call the function we previously pushed. */ + ADDOP_I(c, CALL_FUNCTION, 1); + } + + /* If we have a format spec, use format(value, format_spec). Otherwise, + use the single argument form. */ + if (e->v.FormattedValue.format_spec) { + VISIT(c, expr, e->v.FormattedValue.format_spec); + ADDOP_I(c, CALL_FUNCTION, 2); + } else { + /* No format spec specified, call format(value). */ + ADDOP_I(c, CALL_FUNCTION, 1); + } + + return 1; +} + /* shared code between compiler_call and compiler_class */ static int compiler_call_helper(struct compiler *c, @@ -3878,6 +3990,10 @@ case Str_kind: ADDOP_O(c, LOAD_CONST, e->v.Str.s, consts); break; + case JoinedStr_kind: + return compiler_joined_str(c, e); + case FormattedValue_kind: + return compiler_formatted_value(c, e); case Bytes_kind: ADDOP_O(c, LOAD_CONST, e->v.Bytes.s, consts); break; @@ -4784,4 +4900,3 @@ { return PyAST_CompileEx(mod, filename, flags, -1, arena); } - diff --git a/Python/symtable.c b/Python/symtable.c --- a/Python/symtable.c +++ b/Python/symtable.c @@ -1439,6 +1439,14 @@ VISIT_SEQ(st, expr, e->v.Call.args); VISIT_SEQ_WITH_NULL(st, keyword, e->v.Call.keywords); break; + case FormattedValue_kind: + VISIT(st, expr, e->v.FormattedValue.value); + if (e->v.FormattedValue.format_spec) + VISIT(st, expr, e->v.FormattedValue.format_spec); + break; + case JoinedStr_kind: + VISIT_SEQ(st, expr, e->v.JoinedStr.values); + break; case Num_kind: case Str_kind: case Bytes_kind: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 19 21:10:10 2015 From: python-checkins at python.org (georg.brandl) Date: Sat, 19 Sep 2015 19:10:10 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_Some_PEP_505_fixes_from_Mark?= =?utf-8?q?=2E?= Message-ID: <20150919191009.9947.45843@psf.io> https://hg.python.org/peps/rev/8b188b2d6944 changeset: 6071:8b188b2d6944 user: Georg Brandl date: Sat Sep 19 21:10:05 2015 +0200 summary: Some PEP 505 fixes from Mark. files: pep-0505.txt | 14 +++++++------- 1 files changed, 7 insertions(+), 7 deletions(-) diff --git a/pep-0505.txt b/pep-0505.txt --- a/pep-0505.txt +++ b/pep-0505.txt @@ -48,7 +48,7 @@ >>> title ?? 'Default Title' 'My Title' >>> title = None - >>> title ?? 'Bar' + >>> title ?? 'Default Title' 'Default Title' Similar behavior can be achieved with the ``or`` operator, but ``or`` checks @@ -114,11 +114,11 @@ :: - data ??= [] - files ??= [] - headers ??= {} - params ??= {} - hooks ??= {} + data ?= [] + files ?= [] + headers ?= {} + params ?= {} + hooks ?= {} The ``None`` coalescing operator is left-associative, which allows for easy chaining:: @@ -127,7 +127,7 @@ >>> local_default_title = None >>> global_default_title = 'Global Default Title' >>> title = user_title ?? local_default_title ?? global_default_title - 'My Title' + 'Global Default Title' The direction of associativity is important because the ``None`` coalesing operator short circuits: if its left operand is non-null, then the right -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Sat Sep 19 21:50:08 2015 From: python-checkins at python.org (eric.smith) Date: Sat, 19 Sep 2015 19:50:08 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Temporary_hack_for_issue_?= =?utf-8?q?=2325180=3A_exclude_test=5Ffstring=2Epy_from_the_unparse?= Message-ID: <20150919195008.3668.3202@psf.io> https://hg.python.org/cpython/rev/37d3b95a289b changeset: 98074:37d3b95a289b user: Eric V. Smith date: Sat Sep 19 15:49:57 2015 -0400 summary: Temporary hack for issue #25180: exclude test_fstring.py from the unparse round-tripping, while I figure out how to properly fix it. files: Lib/test/test_tools/test_unparse.py | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) diff --git a/Lib/test/test_tools/test_unparse.py b/Lib/test/test_tools/test_unparse.py --- a/Lib/test/test_tools/test_unparse.py +++ b/Lib/test/test_tools/test_unparse.py @@ -264,6 +264,8 @@ for d in self.test_directories: test_dir = os.path.join(basepath, d) for n in os.listdir(test_dir): + if n == 'test_fstring.py': + continue if n.endswith('.py') and not n.startswith('bad'): names.append(os.path.join(test_dir, n)) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 20 00:59:00 2015 From: python-checkins at python.org (chris.angelico) Date: Sat, 19 Sep 2015 22:59:00 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_Create_PEP_506_by_RSTifying_S?= =?utf-8?q?teven=27s_content?= Message-ID: <20150919225900.3650.96476@psf.io> https://hg.python.org/peps/rev/a48965a64fd2 changeset: 6072:a48965a64fd2 user: Chris Angelico date: Sun Sep 20 08:58:49 2015 +1000 summary: Create PEP 506 by RSTifying Steven's content files: pep-0506.txt | 347 +++++++++++++++++++++++++++++++++++++++ 1 files changed, 347 insertions(+), 0 deletions(-) diff --git a/pep-0506.txt b/pep-0506.txt new file mode 100644 --- /dev/null +++ b/pep-0506.txt @@ -0,0 +1,347 @@ +PEP: 506 +Title: Adding A Secrets Module To The Standard Library +Version: $Revision$ +Last-Modified: $Date$ +Author: Steven D'Aprano +Status: Draft +Type: Standards Track +Content-Type: text/x-rst +Created: 19-Sep-2015 +Python-Version: 3.6 +Post-History: + + +Abstract +======== + +This PEP proposes the addition of a module for common security-related +functions such as generating tokens to the Python standard library. + + +Definitions +=========== + +Some common abbreviations used in this proposal: + +* PRNG: + + Pseudo Random Number Generator. A deterministic algorithm used + to produce random-looking numbers with certain desirable + statistical properties. + +* CSPRNG: + + Cryptographically Strong Pseudo Random Number Generator. An + algorithm used to produce random-looking numbers which are + resistant to prediction. + +* MT: + + Mersenne Twister. An extensively studied PRNG which is currently + used by the ``random`` module as the default. + + +Rationale +========= + +This proposal is motivated by concerns that Python's standard library +makes it too easy for developers to inadvertently make serious security +errors. Theo de Raadt, the founder of OpenBSD, contacted Guido van Rossum +and expressed some concern[1] about the use of MT for generating sensitive +information such as passwords, secure tokens, session keys and similar. + +Although the documentation for the random module explicitly states that +the default is not suitable for security purposes[2], it is strongly +believed that this warning may be missed, ignored or misunderstood by +many Python developers. In particular: + +* developers may not have read the documentation and consequently + not seen the warning; + +* they may not realise that their specific use of it has security + implications; or + +* not realising that there could be a problem, they have copied code + (or learned techniques) from websites which don't offer best + practises. + +The first[3] hit when searching for "python how to generate passwords" on +Google is a tutorial that uses the default functions from the ``random`` +module[4]. Although it is not intended for use in web applications, it is +likely that similar techniques find themselves used in that situation. +The second hit is to a StackOverflow question about generating +passwords[5]. Most of the answers given, including the accepted one, use +the default functions. When one user warned that the default could be +easily compromised, they were told "I think you worry too much."[6] + +This strongly suggests that the existing ``random`` module is an attractive +nuisance when it comes to generating (for example) passwords or secure +tokens. + +Additional motivation (of a more philosophical bent) can be found in the +post which first proposed this idea[7]. + + +Proposal +======== + +Alternative proposals have focused on the default PRNG in the ``random`` +module, with the aim of providing "secure by default" cryptographically +strong primitives that developers can build upon without thinking about +security. (See Alternatives below.) This proposes a different approach: + +* The standard library already provides cryptographically strong + primitives, but many users don't know they exist or when to use them. + +* Instead of requiring crypto-naive users to write secure code, the + standard library should include a set of ready-to-use "batteries" for + the most common needs, such as generating secure tokens. This code + will both directly satisfy a need ("How do I generate a password reset + token?"), and act as an example of acceptable practises which + developers can learn from[8]. + +To do this, this PEP proposes that we add a new module to the standard +library, with the suggested name ``secrets``. This module will contain a +set of ready-to-use functions for common activities with security +implications, together with some lower-level primitives. + +The suggestion is that ``secrets`` becomes the go-to module for dealing +with anything which should remain secret (passwords, tokens, etc.) +while the ``random`` module remains backward-compatible. + + +API and Implementation +====================== + +The contents of the ``secrets`` module is expected to evolve over time, and +likely will evolve between the time of writing this PEP and actual release +in the standard library[9]. At the time of writing, the following functions +have been suggested: + +* A high-level function for generating secure tokens suitable for use + in (e.g.) password recovery, as session keys, etc. + +* A limited interface to the system CSPRNG, using either ``os.urandom`` + directly or ``random.SystemRandom``. Unlike the ``random`` module, this + does not need to provide methods for seeding, getting or setting the + state, or any non-uniform distributions. It should provide the + following: + + - A function for choosing items from a sequence, ``secrets.choice``. + - A function for generating an integer within some range, such as + ``secrets.randrange`` or ``secrets.randint``. + - A function for generating a given number of random bits and/or bytes + as an integer. + - A similar function which returns the value as a hex digit string. + +* ``hmac.compare_digest`` under the name ``equal``. + +The consensus appears to be that there is no need to add a new CSPRNG to +the ``random`` module to support these uses, ``SystemRandom`` will be +sufficient. + +Some illustrative implementations have been given by Nick Coghlan[10]. +This idea has also been discussed on the issue tracker for the +"cryptography" module[11]. + +The ``secrets`` module itself will be pure Python, and other Python +implementations can easily make use of it unchanged, or adapt it as +necessary. + + +Alternatives +============ + +One alternative is to change the default PRNG provided by the ``random`` +module[12]. This received considerable scepticism and outright opposition: + +* There is fear that a CSPRNG may be slower than the current PRNG (which + in the case of MT is already quite slow). + +* Some applications (such as scientific simulations, and replaying + gameplay) require the ability to seed the PRNG into a known state, + which a CSPRNG lacks by design. + +* Another major use of the ``random`` module is for simple "guess a number" + games written by beginners, and many people are loath to make any + change to the ``random`` module which may make that harder. + +* Although there is no proposal to remove MT from the ``random`` module, + there was considerable hostility to the idea of having to opt-in to + a non-CSPRNG or any backwards-incompatible changes. + +* Demonstrated attacks against MT are typically against PHP applications. + It is believed that PHP's version of MT is a significantly softer target + than Python's version, due to a poor seeding technique[13]. Consequently, + without a proven attack against Python applications, many people object + to a backwards-incompatible change. + +Nick Coghlan made an earlier suggestion for a globally configurable PRNG +which uses the system CSPRNG by default[14], but has since hinted that he +may withdraw it in favour of this proposal[15]. + + +Comparison To Other Languages +============================= + +* PHP + + PHP includes a function ``uniqid`` [16] which by default returns a + thirteen character string based on the current time in microseconds. + Translated into Python syntax, it has the following signature:: + + def uniqid(prefix='', more_entropy=False)->str + + The PHP documentation warns that this function is not suitable for + security purposes. Nevertheless, various mature, well-known PHP + applications use it for that purpose (citation needed). + + PHP 5.3 and better also includes a function ``openssl_random_pseudo_bytes`` [17]. + Translated into Python syntax, it has roughly the following signature:: + + def openssl_random_pseudo_bytes(length:int)->Tuple[str, bool] + + This function returns a pseudo-random string of bytes of the given + length, and an boolean flag giving whether the string is considered + cryptographically strong. The PHP manual suggests that returning + anything but True should be rare except for old or broken platforms. + +* Javascript + + Based on a rather cursory search[18], there doesn't appear to be any + well-known standard functions for producing strong random values in + Javascript, although there may be good quality third-party libraries. + Standard Javascript doesn't seem to include an interface to the + system CSPRNG either, and people have extensively written about the + weaknesses of Javascript's Math.random[19]. + +* Ruby + + The Ruby standard library includes a module ``SecureRandom`` [20] + which includes the following methods: + + * base64 - returns a Base64 encoded random string. + + * hex - returns a random hexadecimal string. + + * random_bytes - returns a random byte string. + + * random_number - depending on the argument, returns either a random + integer in the range(0, n), or a random float between 0.0 and 1.0. + + * urlsafe_base64 - returns a random URL-safe Base64 encoded string. + + * uuid - return a version 4 random Universally Unique IDentifier. + + +What Should Be The Name Of The Module? +====================================== + +There was a proposal to add a "random.safe" submodule, quoting the Zen +of Python "Namespaces are one honking great idea" koan. However, the +author of the Zen, Tim Peters, has come out against this idea[21], and +recommends a top-level module. + +In discussion on the python-ideas mailing list so far, the name "secrets" +has received some approval, and no strong opposition. + + +Frequently Asked Questions +========================== + + * Q: Is this a real problem? Surely MT is random enough that nobody can + predict its output. + + A: The consensus among security professionals is that MT is not safe + in security contexts. It is not difficult to reconstruct the internal + state of MT[22][23] and so predict all past and future values. There + are a number of known, practical attacks on systems using MT for + randomness[24]. + + While there are currently no known direct attacks on applications + written in Python due to the use of MT, there is widespread agreement + that such usage is unsafe. + + * Q: Is this an alternative to specialise cryptographic software such as SSL? + + A: No. This is a "batteries included" solution, not a full-featured + "nuclear reactor". It is intended to mitigate against some basic + security errors, not be a solution to all security-related issues. To + quote Nick Coghlan referring to his earlier proposal:: + + "...folks really are better off learning to use things like + cryptography.io for security sensitive software, so this change + is just about harm mitigation given that it's inevitable that a + non-trivial proportion of the millions of current and future + Python developers won't do that."[25] + + +References +========== + +[1] https://mail.python.org/pipermail/python-ideas/2015-September/035820.html + +[2] https://docs.python.org/3/library/random.html + +[3] As of the date of writing. Also, as Google search terms may be automatically customised for the user without their knowledge, some readers may see different results. + +[4] http://interactivepython.org/runestone/static/everyday/2013/01/3_password.html + +[5] http://stackoverflow.com/questions/3854692/generate-password-in-python + +[6] http://stackoverflow.com/questions/3854692/generate-password-in-python/3854766#3854766 + +[7] https://mail.python.org/pipermail/python-ideas/2015-September/036238.html + +[8] At least those who are motivated to read the source code and documentation. + +[9] Tim Peters suggests that bike-shedding the contents of the module will be 10000 times more time consuming than actually implementing the module. Words do not begin to express how much I am looking forward to this. + +[10] https://mail.python.org/pipermail/python-ideas/2015-September/036271.html + +[11] https://github.com/pyca/cryptography/issues/2347 + +[12] Link needed. + +[13] By default PHP seeds the MT PRNG with the time (citation needed), which is exploitable by attackers, while Python seeds the PRNG with output from the system CSPRNG, which is believed to be much harder to exploit. + +[14] http://legacy.python.org/dev/peps/pep-0504/ + +[15] https://mail.python.org/pipermail/python-ideas/2015-September/036243.html + +[16] http://php.net/manual/en/function.uniqid.php + +[17] http://php.net/manual/en/function.openssl-random-pseudo-bytes.php + +[18] Volunteers and patches are welcome. + +[19] http://ifsec.blogspot.fr/2012/05/cross-domain-mathrandom-prediction.html + +[20] http://ruby-doc.org/stdlib-2.1.2/libdoc/securerandom/rdoc/SecureRandom.html + +[21] https://mail.python.org/pipermail/python-ideas/2015-September/036254.html + +[22] https://jazzy.id.au/2010/09/22/cracking_random_number_generators_part_3.html + +[23] https://mail.python.org/pipermail/python-ideas/2015-September/036077.html + +[24] https://media.blackhat.com/bh-us-12/Briefings/Argyros/BH_US_12_Argyros_PRNG_WP.pdf + +[25] https://mail.python.org/pipermail/python-ideas/2015-September/036157.html + + +Copyright +========= + +This document has been placed in the public domain. + + + +.. + Local Variables: + mode: indented-text + indent-tabs-mode: nil + sentence-end-double-space: t + fill-column: 70 + coding: utf-8 + End: -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Sun Sep 20 02:05:19 2015 From: python-checkins at python.org (berker.peksag) Date: Sun, 20 Sep 2015 00:05:19 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_PEP_505=3A_Fix_typo_and_marku?= =?utf-8?q?p?= Message-ID: <20150920000519.31193.16108@psf.io> https://hg.python.org/peps/rev/f8566aa434b1 changeset: 6073:f8566aa434b1 user: Berker Peksag date: Sun Sep 20 03:05:19 2015 +0300 summary: PEP 505: Fix typo and markup files: pep-0505.txt | 8 ++++++-- 1 files changed, 6 insertions(+), 2 deletions(-) diff --git a/pep-0505.txt b/pep-0505.txt --- a/pep-0505.txt +++ b/pep-0505.txt @@ -88,7 +88,7 @@ This particular formulation has the undesirable effect of putting the operands in an unintuitive order: the brain thinks, "use ``data`` if possible and use -``[]`` as a fallback," but the code puts the fallback _before_ the preferred +``[]`` as a fallback," but the code puts the fallback *before* the preferred value. The author of this package could have written it like this instead:: @@ -129,7 +129,7 @@ >>> title = user_title ?? local_default_title ?? global_default_title 'Global Default Title' -The direction of associativity is important because the ``None`` coalesing +The direction of associativity is important because the ``None`` coalescing operator short circuits: if its left operand is non-null, then the right operand is not evaluated. @@ -143,6 +143,8 @@ Null-Aware Member Access Operator --------------------------------- +:: + >>> title = 'My Title' >>> title.upper() 'MY TITLE' @@ -158,6 +160,8 @@ Null-Aware Index Access Operator --------------------------------- +:: + >>> person = {'name': 'Mark', 'age': 32} >>> person['name'] 'Mark' -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Sun Sep 20 02:22:00 2015 From: python-checkins at python.org (berker.peksag) Date: Sun, 20 Sep 2015 00:22:00 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_PEP_506=3A_Improve_markup?= Message-ID: <20150920002200.16571.35017@psf.io> https://hg.python.org/peps/rev/68a2455c09aa changeset: 6074:68a2455c09aa user: Berker Peksag date: Sun Sep 20 03:21:55 2015 +0300 summary: PEP 506: Improve markup files: pep-0506.txt | 143 ++++++++++++++++++++------------------ 1 files changed, 76 insertions(+), 67 deletions(-) diff --git a/pep-0506.txt b/pep-0506.txt --- a/pep-0506.txt +++ b/pep-0506.txt @@ -47,11 +47,11 @@ This proposal is motivated by concerns that Python's standard library makes it too easy for developers to inadvertently make serious security errors. Theo de Raadt, the founder of OpenBSD, contacted Guido van Rossum -and expressed some concern[1] about the use of MT for generating sensitive +and expressed some concern [1]_ about the use of MT for generating sensitive information such as passwords, secure tokens, session keys and similar. Although the documentation for the random module explicitly states that -the default is not suitable for security purposes[2], it is strongly +the default is not suitable for security purposes [2]_, it is strongly believed that this warning may be missed, ignored or misunderstood by many Python developers. In particular: @@ -65,21 +65,21 @@ (or learned techniques) from websites which don't offer best practises. -The first[3] hit when searching for "python how to generate passwords" on +The first [3]_ hit when searching for "python how to generate passwords" on Google is a tutorial that uses the default functions from the ``random`` -module[4]. Although it is not intended for use in web applications, it is +module [4]_. Although it is not intended for use in web applications, it is likely that similar techniques find themselves used in that situation. The second hit is to a StackOverflow question about generating -passwords[5]. Most of the answers given, including the accepted one, use +passwords [5]_. Most of the answers given, including the accepted one, use the default functions. When one user warned that the default could be -easily compromised, they were told "I think you worry too much."[6] +easily compromised, they were told "I think you worry too much." [6]_ This strongly suggests that the existing ``random`` module is an attractive nuisance when it comes to generating (for example) passwords or secure tokens. Additional motivation (of a more philosophical bent) can be found in the -post which first proposed this idea[7]. +post which first proposed this idea [7]_. Proposal @@ -98,7 +98,7 @@ the most common needs, such as generating secure tokens. This code will both directly satisfy a need ("How do I generate a password reset token?"), and act as an example of acceptable practises which - developers can learn from[8]. + developers can learn from [8]_. To do this, this PEP proposes that we add a new module to the standard library, with the suggested name ``secrets``. This module will contain a @@ -115,7 +115,7 @@ The contents of the ``secrets`` module is expected to evolve over time, and likely will evolve between the time of writing this PEP and actual release -in the standard library[9]. At the time of writing, the following functions +in the standard library [9]_. At the time of writing, the following functions have been suggested: * A high-level function for generating secure tokens suitable for use @@ -140,9 +140,9 @@ the ``random`` module to support these uses, ``SystemRandom`` will be sufficient. -Some illustrative implementations have been given by Nick Coghlan[10]. +Some illustrative implementations have been given by Nick Coghlan [10]_. This idea has also been discussed on the issue tracker for the -"cryptography" module[11]. +"cryptography" module [11]_. The ``secrets`` module itself will be pure Python, and other Python implementations can easily make use of it unchanged, or adapt it as @@ -153,7 +153,7 @@ ============ One alternative is to change the default PRNG provided by the ``random`` -module[12]. This received considerable scepticism and outright opposition: +module [12]_. This received considerable scepticism and outright opposition: * There is fear that a CSPRNG may be slower than the current PRNG (which in the case of MT is already quite slow). @@ -172,13 +172,13 @@ * Demonstrated attacks against MT are typically against PHP applications. It is believed that PHP's version of MT is a significantly softer target - than Python's version, due to a poor seeding technique[13]. Consequently, + than Python's version, due to a poor seeding technique [13]_. Consequently, without a proven attack against Python applications, many people object to a backwards-incompatible change. Nick Coghlan made an earlier suggestion for a globally configurable PRNG -which uses the system CSPRNG by default[14], but has since hinted that he -may withdraw it in favour of this proposal[15]. +which uses the system CSPRNG by default [14]_, but has since hinted that he +may withdraw it in favour of this proposal [15]_. Comparison To Other Languages @@ -186,7 +186,7 @@ * PHP - PHP includes a function ``uniqid`` [16] which by default returns a + PHP includes a function ``uniqid`` [16]_ which by default returns a thirteen character string based on the current time in microseconds. Translated into Python syntax, it has the following signature:: @@ -196,8 +196,9 @@ security purposes. Nevertheless, various mature, well-known PHP applications use it for that purpose (citation needed). - PHP 5.3 and better also includes a function ``openssl_random_pseudo_bytes`` [17]. - Translated into Python syntax, it has roughly the following signature:: + PHP 5.3 and better also includes a function ``openssl_random_pseudo_bytes`` + [17]_. Translated into Python syntax, it has roughly the following + signature:: def openssl_random_pseudo_bytes(length:int)->Tuple[str, bool] @@ -208,16 +209,16 @@ * Javascript - Based on a rather cursory search[18], there doesn't appear to be any + Based on a rather cursory search [18]_, there doesn't appear to be any well-known standard functions for producing strong random values in Javascript, although there may be good quality third-party libraries. Standard Javascript doesn't seem to include an interface to the system CSPRNG either, and people have extensively written about the - weaknesses of Javascript's Math.random[19]. + weaknesses of Javascript's ``Math.random`` [19]_. * Ruby - The Ruby standard library includes a module ``SecureRandom`` [20] + The Ruby standard library includes a module ``SecureRandom`` [20]_ which includes the following methods: * base64 - returns a Base64 encoded random string. @@ -239,7 +240,7 @@ There was a proposal to add a "random.safe" submodule, quoting the Zen of Python "Namespaces are one honking great idea" koan. However, the -author of the Zen, Tim Peters, has come out against this idea[21], and +author of the Zen, Tim Peters, has come out against this idea [21]_, and recommends a top-level module. In discussion on the python-ideas mailing list so far, the name "secrets" @@ -249,85 +250,93 @@ Frequently Asked Questions ========================== - * Q: Is this a real problem? Surely MT is random enough that nobody can - predict its output. +* Q: Is this a real problem? Surely MT is random enough that nobody can + predict its output. - A: The consensus among security professionals is that MT is not safe - in security contexts. It is not difficult to reconstruct the internal - state of MT[22][23] and so predict all past and future values. There - are a number of known, practical attacks on systems using MT for - randomness[24]. + A: The consensus among security professionals is that MT is not safe + in security contexts. It is not difficult to reconstruct the internal + state of MT [22]_ [23]_ and so predict all past and future values. There + are a number of known, practical attacks on systems using MT for + randomness [24]_. - While there are currently no known direct attacks on applications - written in Python due to the use of MT, there is widespread agreement - that such usage is unsafe. + While there are currently no known direct attacks on applications + written in Python due to the use of MT, there is widespread agreement + that such usage is unsafe. - * Q: Is this an alternative to specialise cryptographic software such as SSL? +* Q: Is this an alternative to specialise cryptographic software such as SSL? - A: No. This is a "batteries included" solution, not a full-featured - "nuclear reactor". It is intended to mitigate against some basic - security errors, not be a solution to all security-related issues. To - quote Nick Coghlan referring to his earlier proposal:: + A: No. This is a "batteries included" solution, not a full-featured + "nuclear reactor". It is intended to mitigate against some basic + security errors, not be a solution to all security-related issues. To + quote Nick Coghlan referring to his earlier proposal [25]_:: - "...folks really are better off learning to use things like - cryptography.io for security sensitive software, so this change - is just about harm mitigation given that it's inevitable that a - non-trivial proportion of the millions of current and future - Python developers won't do that."[25] + "...folks really are better off learning to use things like + cryptography.io for security sensitive software, so this change + is just about harm mitigation given that it's inevitable that a + non-trivial proportion of the millions of current and future + Python developers won't do that." References ========== -[1] https://mail.python.org/pipermail/python-ideas/2015-September/035820.html +.. [1] https://mail.python.org/pipermail/python-ideas/2015-September/035820.html -[2] https://docs.python.org/3/library/random.html +.. [2] https://docs.python.org/3/library/random.html -[3] As of the date of writing. Also, as Google search terms may be automatically customised for the user without their knowledge, some readers may see different results. +.. [3] As of the date of writing. Also, as Google search terms may be + automatically customised for the user without their knowledge, some + readers may see different results. -[4] http://interactivepython.org/runestone/static/everyday/2013/01/3_password.html +.. [4] http://interactivepython.org/runestone/static/everyday/2013/01/3_password.html -[5] http://stackoverflow.com/questions/3854692/generate-password-in-python +.. [5] http://stackoverflow.com/questions/3854692/generate-password-in-python -[6] http://stackoverflow.com/questions/3854692/generate-password-in-python/3854766#3854766 +.. [6] http://stackoverflow.com/questions/3854692/generate-password-in-python/3854766#3854766 -[7] https://mail.python.org/pipermail/python-ideas/2015-September/036238.html +.. [7] https://mail.python.org/pipermail/python-ideas/2015-September/036238.html -[8] At least those who are motivated to read the source code and documentation. +.. [8] At least those who are motivated to read the source code and documentation. -[9] Tim Peters suggests that bike-shedding the contents of the module will be 10000 times more time consuming than actually implementing the module. Words do not begin to express how much I am looking forward to this. +.. [9] Tim Peters suggests that bike-shedding the contents of the module will + be 10000 times more time consuming than actually implementing the + module. Words do not begin to express how much I am looking forward to + this. -[10] https://mail.python.org/pipermail/python-ideas/2015-September/036271.html +.. [10] https://mail.python.org/pipermail/python-ideas/2015-September/036271.html -[11] https://github.com/pyca/cryptography/issues/2347 +.. [11] https://github.com/pyca/cryptography/issues/2347 -[12] Link needed. +.. [12] Link needed. -[13] By default PHP seeds the MT PRNG with the time (citation needed), which is exploitable by attackers, while Python seeds the PRNG with output from the system CSPRNG, which is believed to be much harder to exploit. +.. [13] By default PHP seeds the MT PRNG with the time (citation needed), + which is exploitable by attackers, while Python seeds the PRNG with + output from the system CSPRNG, which is believed to be much harder to + exploit. -[14] http://legacy.python.org/dev/peps/pep-0504/ +.. [14] http://legacy.python.org/dev/peps/pep-0504/ -[15] https://mail.python.org/pipermail/python-ideas/2015-September/036243.html +.. [15] https://mail.python.org/pipermail/python-ideas/2015-September/036243.html -[16] http://php.net/manual/en/function.uniqid.php +.. [16] http://php.net/manual/en/function.uniqid.php -[17] http://php.net/manual/en/function.openssl-random-pseudo-bytes.php +.. [17] http://php.net/manual/en/function.openssl-random-pseudo-bytes.php -[18] Volunteers and patches are welcome. +.. [18] Volunteers and patches are welcome. -[19] http://ifsec.blogspot.fr/2012/05/cross-domain-mathrandom-prediction.html +.. [19] http://ifsec.blogspot.fr/2012/05/cross-domain-mathrandom-prediction.html -[20] http://ruby-doc.org/stdlib-2.1.2/libdoc/securerandom/rdoc/SecureRandom.html +.. [20] http://ruby-doc.org/stdlib-2.1.2/libdoc/securerandom/rdoc/SecureRandom.html -[21] https://mail.python.org/pipermail/python-ideas/2015-September/036254.html +.. [21] https://mail.python.org/pipermail/python-ideas/2015-September/036254.html -[22] https://jazzy.id.au/2010/09/22/cracking_random_number_generators_part_3.html +.. [22] https://jazzy.id.au/2010/09/22/cracking_random_number_generators_part_3.html -[23] https://mail.python.org/pipermail/python-ideas/2015-September/036077.html +.. [23] https://mail.python.org/pipermail/python-ideas/2015-September/036077.html -[24] https://media.blackhat.com/bh-us-12/Briefings/Argyros/BH_US_12_Argyros_PRNG_WP.pdf +.. [24] https://media.blackhat.com/bh-us-12/Briefings/Argyros/BH_US_12_Argyros_PRNG_WP.pdf -[25] https://mail.python.org/pipermail/python-ideas/2015-September/036157.html +.. [25] https://mail.python.org/pipermail/python-ideas/2015-September/036157.html Copyright -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Sun Sep 20 03:19:32 2015 From: python-checkins at python.org (martin.panter) Date: Sun, 20 Sep 2015 01:19:32 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2325176=3A_Merge_cgi=2Eparse=5Fqsl_link_from_3=2E?= =?utf-8?q?5?= Message-ID: <20150920011932.94107.44054@psf.io> https://hg.python.org/cpython/rev/f08d56387982 changeset: 98077:f08d56387982 parent: 98074:37d3b95a289b parent: 98076:138bbb7cf612 user: Martin Panter date: Sun Sep 20 01:11:50 2015 +0000 summary: Issue #25176: Merge cgi.parse_qsl link from 3.5 files: Doc/library/cgi.rst | 2 +- Misc/ACKS | 1 + 2 files changed, 2 insertions(+), 1 deletions(-) diff --git a/Doc/library/cgi.rst b/Doc/library/cgi.rst --- a/Doc/library/cgi.rst +++ b/Doc/library/cgi.rst @@ -292,7 +292,7 @@ .. function:: parse_qsl(qs, keep_blank_values=False, strict_parsing=False) - This function is deprecated in this module. Use :func:`urllib.parse.parse_qs` + This function is deprecated in this module. Use :func:`urllib.parse.parse_qsl` instead. It is maintained here only for backward compatibility. .. function:: parse_multipart(fp, pdict) diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -1335,6 +1335,7 @@ George Sipe J. Sipprell Kragen Sitaker +Ville Skytt? Michael Sloan Nick Sloan V?clav ?milauer -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 20 03:19:32 2015 From: python-checkins at python.org (martin.panter) Date: Sun, 20 Sep 2015 01:19:32 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Issue_=2325176=3A_Merge_cgi=2Eparse=5Fqsl_link_from_3=2E4_into?= =?utf-8?q?_3=2E5?= Message-ID: <20150920011931.98350.1947@psf.io> https://hg.python.org/cpython/rev/138bbb7cf612 changeset: 98076:138bbb7cf612 branch: 3.5 parent: 98070:21076e24cd8a parent: 98075:3ed2427758cf user: Martin Panter date: Sun Sep 20 01:07:41 2015 +0000 summary: Issue #25176: Merge cgi.parse_qsl link from 3.4 into 3.5 files: Doc/library/cgi.rst | 2 +- Misc/ACKS | 1 + 2 files changed, 2 insertions(+), 1 deletions(-) diff --git a/Doc/library/cgi.rst b/Doc/library/cgi.rst --- a/Doc/library/cgi.rst +++ b/Doc/library/cgi.rst @@ -292,7 +292,7 @@ .. function:: parse_qsl(qs, keep_blank_values=False, strict_parsing=False) - This function is deprecated in this module. Use :func:`urllib.parse.parse_qs` + This function is deprecated in this module. Use :func:`urllib.parse.parse_qsl` instead. It is maintained here only for backward compatibility. .. function:: parse_multipart(fp, pdict) diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -1334,6 +1334,7 @@ George Sipe J. Sipprell Kragen Sitaker +Ville Skytt? Michael Sloan Nick Sloan V?clav ?milauer -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 20 03:19:32 2015 From: python-checkins at python.org (martin.panter) Date: Sun, 20 Sep 2015 01:19:32 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzI1MTc2?= =?utf-8?q?=3A_Correct_link_for_cgi=2Eparse=5Fqsl=3B_patch_from_Ville_Skyt?= =?utf-8?b?dMOk?= Message-ID: <20150920011931.16597.33087@psf.io> https://hg.python.org/cpython/rev/3ed2427758cf changeset: 98075:3ed2427758cf branch: 3.4 parent: 98066:80d002edc9c1 user: Martin Panter date: Sun Sep 20 00:28:50 2015 +0000 summary: Issue #25176: Correct link for cgi.parse_qsl; patch from Ville Skytt? files: Doc/library/cgi.rst | 2 +- Misc/ACKS | 1 + 2 files changed, 2 insertions(+), 1 deletions(-) diff --git a/Doc/library/cgi.rst b/Doc/library/cgi.rst --- a/Doc/library/cgi.rst +++ b/Doc/library/cgi.rst @@ -285,7 +285,7 @@ .. function:: parse_qsl(qs, keep_blank_values=False, strict_parsing=False) - This function is deprecated in this module. Use :func:`urllib.parse.parse_qs` + This function is deprecated in this module. Use :func:`urllib.parse.parse_qsl` instead. It is maintained here only for backward compatibility. .. function:: parse_multipart(fp, pdict) diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -1297,6 +1297,7 @@ George Sipe J. Sipprell Kragen Sitaker +Ville Skytt? Michael Sloan Nick Sloan V?clav ?milauer -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 20 03:45:56 2015 From: python-checkins at python.org (martin.panter) Date: Sun, 20 Sep 2015 01:45:56 +0000 Subject: [Python-checkins] =?utf-8?q?test=3A_Trying_out_obvuscated_email_a?= =?utf-8?q?ddress?= Message-ID: <20150920014556.11690.60533@psf.io> https://hg.python.org/test/rev/7c43ca3de40e changeset: 225:7c43ca3de40e user: Martin Panter date: Sun Sep 20 01:43:59 2015 +0000 summary: Trying out obvuscated email address files: v | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/v b/v --- a/v +++ b/v @@ -1,1 +1,1 @@ -v +Vee -- Repository URL: https://hg.python.org/test From python-checkins at python.org Sun Sep 20 07:39:25 2015 From: python-checkins at python.org (serhiy.storchaka) Date: Sun, 20 Sep 2015 05:39:25 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzI1MTA4?= =?utf-8?q?=3A_Fixed_test=5Ftraceback_in_the_case_when_this_test_is_run_tw?= =?utf-8?q?ice=2E?= Message-ID: <20150920053925.9935.12233@psf.io> https://hg.python.org/cpython/rev/7e718bbf5152 changeset: 98078:7e718bbf5152 branch: 2.7 parent: 98067:fe84898fbe98 user: Serhiy Storchaka date: Sun Sep 20 08:38:40 2015 +0300 summary: Issue #25108: Fixed test_traceback in the case when this test is run twice. In this case __file__ is the name of precompiled file (*.py[co]). files: Lib/test/test_traceback.py | 15 +++++++++------ 1 files changed, 9 insertions(+), 6 deletions(-) diff --git a/Lib/test/test_traceback.py b/Lib/test/test_traceback.py --- a/Lib/test/test_traceback.py +++ b/Lib/test/test_traceback.py @@ -213,10 +213,11 @@ with captured_output("stderr") as stderr: prn() lineno = prn.__code__.co_firstlineno + file = prn.__code__.co_filename self.assertEqual(stderr.getvalue().splitlines()[-4:], [ - ' File "%s", line %d, in test_print_stack' % (__file__, lineno+3), + ' File "%s", line %d, in test_print_stack' % (file, lineno+3), ' prn()', - ' File "%s", line %d, in prn' % (__file__, lineno+1), + ' File "%s", line %d, in prn' % (file, lineno+1), ' traceback.print_stack()', ]) @@ -225,11 +226,12 @@ return traceback.format_stack() result = fmt() lineno = fmt.__code__.co_firstlineno + file = fmt.__code__.co_filename self.assertEqual(result[-2:], [ ' File "%s", line %d, in test_format_stack\n' - ' result = fmt()\n' % (__file__, lineno+2), + ' result = fmt()\n' % (file, lineno+2), ' File "%s", line %d, in fmt\n' - ' return traceback.format_stack()\n' % (__file__, lineno+1), + ' return traceback.format_stack()\n' % (file, lineno+1), ]) @@ -243,9 +245,10 @@ return traceback.extract_stack() result = extract() lineno = extract.__code__.co_firstlineno + file = extract.__code__.co_filename self.assertEqual(result[-2:], [ - (__file__, lineno+2, 'test_extract_stack', 'result = extract()'), - (__file__, lineno+1, 'extract', 'return traceback.extract_stack()'), + (file, lineno+2, 'test_extract_stack', 'result = extract()'), + (file, lineno+1, 'extract', 'return traceback.extract_stack()'), ]) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 20 08:35:07 2015 From: python-checkins at python.org (terry.reedy) Date: Sun, 20 Sep 2015 06:35:07 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzI0MTk5?= =?utf-8?q?=3A_Add_stacklevel_to_deprecation_warning_call=2E?= Message-ID: <20150920063507.9957.4838@psf.io> https://hg.python.org/cpython/rev/3c39413d277f changeset: 98079:3c39413d277f branch: 2.7 user: Terry Jan Reedy date: Sun Sep 20 02:33:57 2015 -0400 summary: Issue #24199: Add stacklevel to deprecation warning call. files: Lib/idlelib/idlever.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Lib/idlelib/idlever.py b/Lib/idlelib/idlever.py --- a/Lib/idlelib/idlever.py +++ b/Lib/idlelib/idlever.py @@ -7,6 +7,6 @@ """ # Kept for now only for possible existing extension use import warnings as w -w.warn(__doc__, DeprecationWarning) +w.warn(__doc__, DeprecationWarning, stacklevel=2) from sys import version IDLE_VERSION = version[:version.index(' ')] -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 20 08:35:08 2015 From: python-checkins at python.org (terry.reedy) Date: Sun, 20 Sep 2015 06:35:08 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Merge_with_3=2E4?= Message-ID: <20150920063507.81649.2213@psf.io> https://hg.python.org/cpython/rev/594d3db80d04 changeset: 98081:594d3db80d04 branch: 3.5 parent: 98076:138bbb7cf612 parent: 98080:048fce602bcd user: Terry Jan Reedy date: Sun Sep 20 02:34:20 2015 -0400 summary: Merge with 3.4 files: Lib/idlelib/idlever.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Lib/idlelib/idlever.py b/Lib/idlelib/idlever.py --- a/Lib/idlelib/idlever.py +++ b/Lib/idlelib/idlever.py @@ -7,6 +7,6 @@ """ # Kept for now only for possible existing extension use import warnings as w -w.warn(__doc__, DeprecationWarning) +w.warn(__doc__, DeprecationWarning, stacklevel=2) from sys import version IDLE_VERSION = version[:version.index(' ')] -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 20 08:35:07 2015 From: python-checkins at python.org (terry.reedy) Date: Sun, 20 Sep 2015 06:35:07 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzI0MTk5?= =?utf-8?q?=3A_Add_stacklevel_to_deprecation_warning_call=2E?= Message-ID: <20150920063507.3656.40355@psf.io> https://hg.python.org/cpython/rev/048fce602bcd changeset: 98080:048fce602bcd branch: 3.4 parent: 98075:3ed2427758cf user: Terry Jan Reedy date: Sun Sep 20 02:34:03 2015 -0400 summary: Issue #24199: Add stacklevel to deprecation warning call. files: Lib/idlelib/idlever.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Lib/idlelib/idlever.py b/Lib/idlelib/idlever.py --- a/Lib/idlelib/idlever.py +++ b/Lib/idlelib/idlever.py @@ -7,6 +7,6 @@ """ # Kept for now only for possible existing extension use import warnings as w -w.warn(__doc__, DeprecationWarning) +w.warn(__doc__, DeprecationWarning, stacklevel=2) from sys import version IDLE_VERSION = version[:version.index(' ')] -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 20 08:35:08 2015 From: python-checkins at python.org (terry.reedy) Date: Sun, 20 Sep 2015 06:35:08 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge_with_3=2E5?= Message-ID: <20150920063508.115050.97980@psf.io> https://hg.python.org/cpython/rev/5e341ec3248f changeset: 98082:5e341ec3248f parent: 98077:f08d56387982 parent: 98081:594d3db80d04 user: Terry Jan Reedy date: Sun Sep 20 02:34:51 2015 -0400 summary: Merge with 3.5 files: Lib/idlelib/idlever.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Lib/idlelib/idlever.py b/Lib/idlelib/idlever.py --- a/Lib/idlelib/idlever.py +++ b/Lib/idlelib/idlever.py @@ -7,6 +7,6 @@ """ # Kept for now only for possible existing extension use import warnings as w -w.warn(__doc__, DeprecationWarning) +w.warn(__doc__, DeprecationWarning, stacklevel=2) from sys import version IDLE_VERSION = version[:version.index(' ')] -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 20 09:44:34 2015 From: python-checkins at python.org (nick.coghlan) Date: Sun, 20 Sep 2015 07:44:34 +0000 Subject: [Python-checkins] =?utf-8?q?devguide=3A_Realign_section_label?= Message-ID: <20150920074434.94115.45759@psf.io> https://hg.python.org/devguide/rev/0dabe7c5f15e changeset: 762:0dabe7c5f15e user: Nick Coghlan date: Sun Sep 20 17:44:24 2015 +1000 summary: Realign section label files: experts.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/experts.rst b/experts.rst --- a/experts.rst +++ b/experts.rst @@ -1,4 +1,4 @@ - .. _experts: +.. _experts: Experts Index ================= -- Repository URL: https://hg.python.org/devguide From python-checkins at python.org Sun Sep 20 09:50:26 2015 From: python-checkins at python.org (nick.coghlan) Date: Sun, 20 Sep 2015 07:50:26 +0000 Subject: [Python-checkins] =?utf-8?q?devguide=3A_Add_maintainer_list_to_ma?= =?utf-8?q?in_ToC?= Message-ID: <20150920075025.16585.28989@psf.io> https://hg.python.org/devguide/rev/d3b485c8fd08 changeset: 763:d3b485c8fd08 user: Nick Coghlan date: Sun Sep 20 17:50:17 2015 +1000 summary: Add maintainer list to main ToC files: index.rst | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diff --git a/index.rst b/index.rst --- a/index.rst +++ b/index.rst @@ -85,6 +85,7 @@ * :doc:`fixingissues` * :ref:`tracker` and :ref:`helptriage` * :doc:`triaging` + * :doc:`experts` * :doc:`communication` * :doc:`coredev` * :doc:`committing` -- Repository URL: https://hg.python.org/devguide From solipsis at pitrou.net Sun Sep 20 10:44:10 2015 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Sun, 20 Sep 2015 08:44:10 +0000 Subject: [Python-checkins] Daily reference leaks (f08d56387982): sum=17877 Message-ID: <20150920084410.3662.94637@psf.io> results for f08d56387982 on branch "default" -------------------------------------------- test_capi leaked [1598, 1598, 1598] references, sum=4794 test_capi leaked [387, 389, 389] memory blocks, sum=1165 test_functools leaked [0, 2, 2] memory blocks, sum=4 test_threading leaked [3196, 3196, 3196] references, sum=9588 test_threading leaked [774, 776, 776] memory blocks, sum=2326 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/psf-users/antoine/refleaks/reflogg0FhFG', '--timeout', '7200'] From python-checkins at python.org Sun Sep 20 11:06:54 2015 From: python-checkins at python.org (nick.coghlan) Date: Sun, 20 Sep 2015 09:06:54 +0000 Subject: [Python-checkins] =?utf-8?q?devguide=3A_Fix_heading_marker?= Message-ID: <20150920090654.81631.72745@psf.io> https://hg.python.org/devguide/rev/2eb55cc7d601 changeset: 764:2eb55cc7d601 user: Nick Coghlan date: Sun Sep 20 19:06:40 2015 +1000 summary: Fix heading marker files: experts.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/experts.rst b/experts.rst --- a/experts.rst +++ b/experts.rst @@ -1,7 +1,7 @@ .. _experts: Experts Index -================= +============= This document has tables that list Python Modules, Tools, Platforms and Interest Areas and names for each item that indicate a maintainer or an -- Repository URL: https://hg.python.org/devguide From python-checkins at python.org Sun Sep 20 13:54:01 2015 From: python-checkins at python.org (nick.coghlan) Date: Sun, 20 Sep 2015 11:54:01 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_Withdraw_PEP_504_in_favour_of?= =?utf-8?q?_506?= Message-ID: <20150920115401.82642.1540@psf.io> https://hg.python.org/peps/rev/e3dabf1d9fc1 changeset: 6075:e3dabf1d9fc1 user: Nick Coghlan date: Sun Sep 20 21:53:54 2015 +1000 summary: Withdraw PEP 504 in favour of 506 files: pep-0504.txt | 17 ++++++++++++++++- 1 files changed, 16 insertions(+), 1 deletions(-) diff --git a/pep-0504.txt b/pep-0504.txt --- a/pep-0504.txt +++ b/pep-0504.txt @@ -3,7 +3,7 @@ Version: $Revision$ Last-Modified: $Date$ Author: Nick Coghlan -Status: Draft +Status: Withdrawn Type: Standards Track Content-Type: text/x-rst Created: 15-Sep-2015 @@ -38,6 +38,21 @@ To minimise the impact on existing code, module level APIs that require determinism will implicitly switch to the deterministic PRNG. +PEP Withdrawal +============== + +During discussion of this PEP, Steven D'Aprano proposed the simpler alternative +of offering a standardised ``secrets`` module that provides "one obvious way" +to handle security sensitive tasks like generating default passwords and other +tokens. + +Steven's proposal has the desired effect of aligning the easy way to generate +such tokens and the right way to generate them, without introducing any +compatibility risks for the existing ``random`` module API, so this PEP has +been deferred in favour of further work on refining Steven's proposal as +PEP 506. + + Proposal ======== -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Sun Sep 20 15:02:40 2015 From: python-checkins at python.org (nick.coghlan) Date: Sun, 20 Sep 2015 13:02:40 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_PEP_504_is_withdrawn=2C_not_d?= =?utf-8?q?eferred?= Message-ID: <20150920130240.9943.24713@psf.io> https://hg.python.org/peps/rev/3042d4b36030 changeset: 6076:3042d4b36030 user: Nick Coghlan date: Sun Sep 20 23:02:31 2015 +1000 summary: PEP 504 is withdrawn, not deferred files: pep-0504.txt | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/pep-0504.txt b/pep-0504.txt --- a/pep-0504.txt +++ b/pep-0504.txt @@ -49,7 +49,7 @@ Steven's proposal has the desired effect of aligning the easy way to generate such tokens and the right way to generate them, without introducing any compatibility risks for the existing ``random`` module API, so this PEP has -been deferred in favour of further work on refining Steven's proposal as +been withdrawn in favour of further work on refining Steven's proposal as PEP 506. -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Sun Sep 20 20:19:17 2015 From: python-checkins at python.org (benjamin.peterson) Date: Sun, 20 Sep 2015 18:19:17 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_use_a_more_mod?= =?utf-8?b?ZXJuIFVBICgjMjUxNDUp?= Message-ID: <20150920181914.94107.76483@psf.io> https://hg.python.org/cpython/rev/1fcdca298be9 changeset: 98083:1fcdca298be9 branch: 3.4 parent: 98080:048fce602bcd user: Benjamin Peterson date: Sun Sep 20 23:16:45 2015 +0500 summary: use a more modern UA (#25145) files: Doc/howto/urllib2.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/howto/urllib2.rst b/Doc/howto/urllib2.rst --- a/Doc/howto/urllib2.rst +++ b/Doc/howto/urllib2.rst @@ -174,7 +174,7 @@ import urllib.request url = 'http://www.someserver.com/cgi-bin/register.cgi' - user_agent = 'Mozilla/4.0 (compatible; MSIE 5.5; Windows NT)' + user_agent = 'Mozilla/5.0 (Windows NT 6.1; Win64; x64)' values = {'name' : 'Michael Foord', 'location' : 'Northampton', 'language' : 'Python' } -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 20 20:19:17 2015 From: python-checkins at python.org (benjamin.peterson) Date: Sun, 20 Sep 2015 18:19:17 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_remove_referen?= =?utf-8?q?ce_to_PyGoogle_=28=2325145=29?= Message-ID: <20150920181914.115289.54118@psf.io> https://hg.python.org/cpython/rev/2efd269b3eb8 changeset: 98084:2efd269b3eb8 branch: 3.4 user: Benjamin Peterson date: Sun Sep 20 23:17:41 2015 +0500 summary: remove reference to PyGoogle (#25145) Patch by Bar Harel. files: Doc/howto/urllib2.rst | 3 +-- 1 files changed, 1 insertions(+), 2 deletions(-) diff --git a/Doc/howto/urllib2.rst b/Doc/howto/urllib2.rst --- a/Doc/howto/urllib2.rst +++ b/Doc/howto/urllib2.rst @@ -572,8 +572,7 @@ This document was reviewed and revised by John Lee. -.. [#] Like Google for example. The *proper* way to use google from a program - is to use `PyGoogle `_ of course. +.. [#] Google for example. .. [#] Browser sniffing is a very bad practise for website design - building sites using web standards is much more sensible. Unfortunately a lot of sites still send different versions to different browsers. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 20 20:19:20 2015 From: python-checkins at python.org (benjamin.peterson) Date: Sun, 20 Sep 2015 18:19:20 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_merge_3=2E4_=28=2325145=29?= Message-ID: <20150920181920.11710.17648@psf.io> https://hg.python.org/cpython/rev/7f76c7d853be changeset: 98087:7f76c7d853be branch: 3.5 parent: 98081:594d3db80d04 parent: 98084:2efd269b3eb8 user: Benjamin Peterson date: Sun Sep 20 23:18:51 2015 +0500 summary: merge 3.4 (#25145) files: Doc/howto/urllib2.rst | 5 ++--- 1 files changed, 2 insertions(+), 3 deletions(-) diff --git a/Doc/howto/urllib2.rst b/Doc/howto/urllib2.rst --- a/Doc/howto/urllib2.rst +++ b/Doc/howto/urllib2.rst @@ -174,7 +174,7 @@ import urllib.request url = 'http://www.someserver.com/cgi-bin/register.cgi' - user_agent = 'Mozilla/4.0 (compatible; MSIE 5.5; Windows NT)' + user_agent = 'Mozilla/5.0 (Windows NT 6.1; Win64; x64)' values = {'name' : 'Michael Foord', 'location' : 'Northampton', 'language' : 'Python' } @@ -572,8 +572,7 @@ This document was reviewed and revised by John Lee. -.. [#] Like Google for example. The *proper* way to use google from a program - is to use `PyGoogle `_ of course. +.. [#] Google for example. .. [#] Browser sniffing is a very bad practise for website design - building sites using web standards is much more sensible. Unfortunately a lot of sites still send different versions to different browsers. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 20 20:19:22 2015 From: python-checkins at python.org (benjamin.peterson) Date: Sun, 20 Sep 2015 18:19:22 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_remove_referen?= =?utf-8?q?ce_to_PyGoogle_=28=2325145=29?= Message-ID: <20150920181920.94117.24744@psf.io> https://hg.python.org/cpython/rev/9b7bc13aed4e changeset: 98085:9b7bc13aed4e branch: 2.7 parent: 98079:3c39413d277f user: Benjamin Peterson date: Sun Sep 20 23:17:41 2015 +0500 summary: remove reference to PyGoogle (#25145) Patch by Bar Harel. files: Doc/howto/urllib2.rst | 3 +-- 1 files changed, 1 insertions(+), 2 deletions(-) diff --git a/Doc/howto/urllib2.rst b/Doc/howto/urllib2.rst --- a/Doc/howto/urllib2.rst +++ b/Doc/howto/urllib2.rst @@ -559,8 +559,7 @@ .. [#] For an introduction to the CGI protocol see `Writing Web Applications in Python `_. -.. [#] Like Google for example. The *proper* way to use google from a program - is to use `PyGoogle `_ of course. +.. [#] Google for example. .. [#] Browser sniffing is a very bad practise for website design - building sites using web standards is much more sensible. Unfortunately a lot of sites still send different versions to different browsers. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 20 20:19:22 2015 From: python-checkins at python.org (benjamin.peterson) Date: Sun, 20 Sep 2015 18:19:22 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogbWVyZ2UgMy41ICgjMjUxNDUp?= Message-ID: <20150920181920.98366.75022@psf.io> https://hg.python.org/cpython/rev/54ea36ce6eab changeset: 98088:54ea36ce6eab parent: 98082:5e341ec3248f parent: 98087:7f76c7d853be user: Benjamin Peterson date: Sun Sep 20 23:18:58 2015 +0500 summary: merge 3.5 (#25145) files: Doc/howto/urllib2.rst | 5 ++--- 1 files changed, 2 insertions(+), 3 deletions(-) diff --git a/Doc/howto/urllib2.rst b/Doc/howto/urllib2.rst --- a/Doc/howto/urllib2.rst +++ b/Doc/howto/urllib2.rst @@ -174,7 +174,7 @@ import urllib.request url = 'http://www.someserver.com/cgi-bin/register.cgi' - user_agent = 'Mozilla/4.0 (compatible; MSIE 5.5; Windows NT)' + user_agent = 'Mozilla/5.0 (Windows NT 6.1; Win64; x64)' values = {'name' : 'Michael Foord', 'location' : 'Northampton', 'language' : 'Python' } @@ -572,8 +572,7 @@ This document was reviewed and revised by John Lee. -.. [#] Like Google for example. The *proper* way to use google from a program - is to use `PyGoogle `_ of course. +.. [#] Google for example. .. [#] Browser sniffing is a very bad practise for website design - building sites using web standards is much more sensible. Unfortunately a lot of sites still send different versions to different browsers. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 20 20:19:22 2015 From: python-checkins at python.org (benjamin.peterson) Date: Sun, 20 Sep 2015 18:19:22 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_use_a_more_mod?= =?utf-8?b?ZXJuIFVBICgjMjUxNDUp?= Message-ID: <20150920181920.94119.27587@psf.io> https://hg.python.org/cpython/rev/96eff21fc47e changeset: 98086:96eff21fc47e branch: 2.7 user: Benjamin Peterson date: Sun Sep 20 23:16:45 2015 +0500 summary: use a more modern UA (#25145) files: Doc/howto/urllib2.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/howto/urllib2.rst b/Doc/howto/urllib2.rst --- a/Doc/howto/urllib2.rst +++ b/Doc/howto/urllib2.rst @@ -164,7 +164,7 @@ import urllib2 url = 'http://www.someserver.com/cgi-bin/register.cgi' - user_agent = 'Mozilla/4.0 (compatible; MSIE 5.5; Windows NT)' + user_agent = 'Mozilla/5.0 (Windows NT 6.1; Win64; x64)' values = {'name' : 'Michael Foord', 'location' : 'Northampton', 'language' : 'Python' } -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 20 21:09:30 2015 From: python-checkins at python.org (eric.smith) Date: Sun, 20 Sep 2015 19:09:30 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_25180=3A_Fix_Tools/p?= =?utf-8?q?arser/unparse=2Epy_for_f-strings=2E_Patch_by_Martin_Panter=2E?= Message-ID: <20150920190930.9933.85286@psf.io> https://hg.python.org/cpython/rev/2fcd2540ab97 changeset: 98089:2fcd2540ab97 user: Eric V. Smith date: Sun Sep 20 15:09:15 2015 -0400 summary: Issue 25180: Fix Tools/parser/unparse.py for f-strings. Patch by Martin Panter. files: Lib/test/test_tools/test_unparse.py | 11 +++- Tools/parser/unparse.py | 39 +++++++++++++++++ 2 files changed, 48 insertions(+), 2 deletions(-) diff --git a/Lib/test/test_tools/test_unparse.py b/Lib/test/test_tools/test_unparse.py --- a/Lib/test/test_tools/test_unparse.py +++ b/Lib/test/test_tools/test_unparse.py @@ -134,6 +134,15 @@ class UnparseTestCase(ASTTestCase): # Tests for specific bugs found in earlier versions of unparse + def test_fstrings(self): + # See issue 25180 + self.check_roundtrip(r"""f'{f"{0}"*3}'""") + self.check_roundtrip(r"""f'{f"{y}"*3}'""") + self.check_roundtrip(r"""f'{f"{\'x\'}"*3}'""") + + self.check_roundtrip(r'''f"{r'x' f'{\"s\"}'}"''') + self.check_roundtrip(r'''f"{r'x'rf'{\"s\"}'}"''') + def test_del_statement(self): self.check_roundtrip("del x, y, z") @@ -264,8 +273,6 @@ for d in self.test_directories: test_dir = os.path.join(basepath, d) for n in os.listdir(test_dir): - if n == 'test_fstring.py': - continue if n.endswith('.py') and not n.startswith('bad'): names.append(os.path.join(test_dir, n)) diff --git a/Tools/parser/unparse.py b/Tools/parser/unparse.py --- a/Tools/parser/unparse.py +++ b/Tools/parser/unparse.py @@ -322,6 +322,45 @@ def _Str(self, tree): self.write(repr(tree.s)) + def _JoinedStr(self, t): + self.write("f") + string = io.StringIO() + self._fstring_JoinedStr(t, string.write) + self.write(repr(string.getvalue())) + + def _FormattedValue(self, t): + self.write("f") + string = io.StringIO() + self._fstring_FormattedValue(t, string.write) + self.write(repr(string.getvalue())) + + def _fstring_JoinedStr(self, t, write): + for value in t.values: + meth = getattr(self, "_fstring_" + type(value).__name__) + meth(value, write) + + def _fstring_Str(self, t, write): + value = t.s.replace("{", "{{").replace("}", "}}") + write(value) + + def _fstring_FormattedValue(self, t, write): + write("{") + expr = io.StringIO() + Unparser(t.value, expr) + expr = expr.getvalue().rstrip("\n") + if expr.startswith("{"): + write(" ") # Separate pair of opening brackets as "{ {" + write(expr) + if t.conversion != -1: + conversion = chr(t.conversion) + assert conversion in "sra" + write(f"!{conversion}") + if t.format_spec: + write(":") + meth = getattr(self, "_fstring_" + type(t.format_spec).__name__) + meth(t.format_spec, write) + write("}") + def _Name(self, t): self.write(t.id) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 21 02:06:14 2015 From: python-checkins at python.org (terry.reedy) Date: Mon, 21 Sep 2015 00:06:14 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzE2ODkz?= =?utf-8?q?=3A_whitespace_in_idle=2Ehtml=2E?= Message-ID: <20150921000614.81649.29625@psf.io> https://hg.python.org/cpython/rev/e66fbfa282c6 changeset: 98095:e66fbfa282c6 branch: 2.7 parent: 98091:52510bc368f8 user: Terry Jan Reedy date: Sun Sep 20 20:02:23 2015 -0400 summary: Issue #16893: whitespace in idle.html. files: Lib/idlelib/idle.html | 28 ++++++++++++++-------------- 1 files changed, 14 insertions(+), 14 deletions(-) diff --git a/Lib/idlelib/idle.html b/Lib/idlelib/idle.html --- a/Lib/idlelib/idle.html +++ b/Lib/idlelib/idle.html @@ -5,12 +5,12 @@ - + 24.6. IDLE — Python 2.7.10 documentation - + - + - - - + + + - + +
- +

24.6. IDLE?

IDLE is the Python IDE built with the tkinter GUI toolkit.

@@ -628,7 +628,7 @@
-
+
+
- \ No newline at end of file + -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 21 02:06:13 2015 From: python-checkins at python.org (terry.reedy) Date: Mon, 21 Sep 2015 00:06:13 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzE2ODkz?= =?utf-8?q?=3A_Replace_help=2Etxt_with_idle=2Ehtml_for_Idle_doc_display=2E?= Message-ID: <20150921000613.82642.4649@psf.io> https://hg.python.org/cpython/rev/4038508240a1 changeset: 98090:4038508240a1 branch: 2.7 parent: 98086:96eff21fc47e user: Terry Jan Reedy date: Sun Sep 20 19:55:44 2015 -0400 summary: Issue #16893: Replace help.txt with idle.html for Idle doc display. The new idlelib/idle.html is copied from Doc/build/html/idle.html. It looks better than help.txt and will better document Idle as released. The tkinter html viewer that works for this file was written by Rose Roseman. The new code is in idlelib/help.py, a new file for help menu classes. The now unused EditorWindow.HelpDialog class and helt.txt file are deprecated. files: Lib/idlelib/EditorWindow.py | 12 ++++++++++-- Lib/idlelib/help.txt | 4 ++++ Lib/idlelib/idle_test/htest.py | 14 +++++++------- Lib/idlelib/macosxSupport.py | 5 ++--- 4 files changed, 23 insertions(+), 12 deletions(-) diff --git a/Lib/idlelib/EditorWindow.py b/Lib/idlelib/EditorWindow.py --- a/Lib/idlelib/EditorWindow.py +++ b/Lib/idlelib/EditorWindow.py @@ -17,6 +17,7 @@ from idlelib.configHandler import idleConf from idlelib import aboutDialog, textView, configDialog from idlelib import macosxSupport +from idlelib import help # The default tab setting for a Text widget, in average-width characters. TK_TABWIDTH_DEFAULT = 8 @@ -71,6 +72,11 @@ class HelpDialog(object): def __init__(self): + import warnings as w + w.warn("EditorWindow.HelpDialog is no longer used by Idle.\n" + "It will be removed in 3.6 or later.\n" + "It has been replaced by private help.HelpWindow\n", + DeprecationWarning, stacklevel=2) self.parent = None # parent of help window self.dlg = None # the help window iteself @@ -566,11 +572,13 @@ configDialog.ConfigExtensionsDialog(self.top) def help_dialog(self, event=None): + "Handle help doc event." + # edit maxosxSupport.overrideRootMenu.help_dialog to match if self.root: parent = self.root else: parent = self.top - helpDialog.display(parent, near=self.top) + help.show_idlehelp(parent) def python_docs(self, event=None): if sys.platform[:3] == 'win': @@ -1717,4 +1725,4 @@ if __name__ == '__main__': from idlelib.idle_test.htest import run - run(_help_dialog, _editor_window) + run(_editor_window) diff --git a/Lib/idlelib/help.txt b/Lib/idlelib/help.txt --- a/Lib/idlelib/help.txt +++ b/Lib/idlelib/help.txt @@ -1,3 +1,7 @@ +This file, idlelib/help.txt is out-of-date and no longer used by Idle. +It is deprecated and will be removed in the future, possibly in 3.6 +---------------------------------------------------------------------- + [See the end of this file for ** TIPS ** on using IDLE !!] File Menu: diff --git a/Lib/idlelib/idle_test/htest.py b/Lib/idlelib/idle_test/htest.py --- a/Lib/idlelib/idle_test/htest.py +++ b/Lib/idlelib/idle_test/htest.py @@ -194,13 +194,6 @@ "should open that file \nin a new EditorWindow." } -_help_dialog_spec = { - 'file': 'EditorWindow', - 'kwds': {}, - 'msg': "If the help text displays, this works.\n" - "Text is selectable. Window is scrollable." - } - _io_binding_spec = { 'file': 'IOBinding', 'kwds': {}, @@ -279,6 +272,13 @@ "Right clicking an item will display a popup." } +show_idlehelp_spec = { + 'file': 'help', + 'kwds': {}, + 'msg': "If the help text displays, this works.\n" + "Text is selectable. Window is scrollable." + } + _stack_viewer_spec = { 'file': 'StackViewer', 'kwds': {}, diff --git a/Lib/idlelib/macosxSupport.py b/Lib/idlelib/macosxSupport.py --- a/Lib/idlelib/macosxSupport.py +++ b/Lib/idlelib/macosxSupport.py @@ -170,9 +170,8 @@ configDialog.ConfigDialog(root, 'Settings') def help_dialog(event=None): - from idlelib import textView - fn = path.join(path.abspath(path.dirname(__file__)), 'help.txt') - textView.view_file(root, 'Help', fn) + from idlelib import help + help.show_idlehelp(root) root.bind('<>', about_dialog) root.bind('<>', config_dialog) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 21 02:06:14 2015 From: python-checkins at python.org (terry.reedy) Date: Mon, 21 Sep 2015 00:06:14 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzE2ODkz?= =?utf-8?q?=3A_include_new_files?= Message-ID: <20150921000613.98352.10206@psf.io> https://hg.python.org/cpython/rev/52510bc368f8 changeset: 98091:52510bc368f8 branch: 2.7 user: Terry Jan Reedy date: Sun Sep 20 19:56:54 2015 -0400 summary: Issue #16893: include new files files: Lib/idlelib/help.py | 236 ++++++++++ Lib/idlelib/idle.html | 671 ++++++++++++++++++++++++++++++ 2 files changed, 907 insertions(+), 0 deletions(-) diff --git a/Lib/idlelib/help.py b/Lib/idlelib/help.py new file mode 100644 --- /dev/null +++ b/Lib/idlelib/help.py @@ -0,0 +1,236 @@ +""" +help.py implements the Idle help menu and is subject to change. + +The contents are subject to revision at any time, without notice. + +Help => About IDLE: diplay About Idle dialog + + + +Help => IDLE Help: display idle.html with proper formatting + +HelpParser - Parses idle.html generated from idle.rst by Sphinx +and renders to tk Text. + +HelpText - Displays formatted idle.html. + +HelpFrame - Contains text, scrollbar, and table-of-contents. +(This will be needed for display in a future tabbed window.) + +HelpWindow - Display idleframe in a standalone window. + +show_idlehelp - Create HelpWindow. Called in EditorWindow.help_dialog. +""" +from HTMLParser import HTMLParser +from os.path import abspath, dirname, isdir, isfile, join +from Tkinter import Tk, Toplevel, Frame, Text, Scrollbar, Menu, Menubutton +import tkFont as tkfont + +use_ttk = False # until available to import +if use_ttk: + from tkinter.ttk import Menubutton + +## About IDLE ## + + +## IDLE Help ## + +class HelpParser(HTMLParser): + """Render idle.html generated by Sphinx from idle.rst. + + The overridden handle_xyz methods handle a subset of html tags. + The supplied text should have the needed tag configurations. + The behavior for unsupported tags, such as table, is undefined. + """ + def __init__(self, text): + HTMLParser.__init__(self) + self.text = text # text widget we're rendering into + self.tags = '' # current text tags to apply + self.show = False # used so we exclude page navigation + self.hdrlink = False # used so we don't show header links + self.level = 0 # indentation level + self.pre = False # displaying preformatted text + self.hprefix = '' # strip e.g. '25.5' from headings + self.nested_dl = False # if we're in a nested
+ self.simplelist = False # simple list (no double spacing) + self.tocid = 1 # id for table of contents entries + self.contents = [] # map toc ids to section titles + self.data = '' # to record data within header tags for toc + + def indent(self, amt=1): + self.level += amt + self.tags = '' if self.level == 0 else 'l'+str(self.level) + + def handle_starttag(self, tag, attrs): + "Handle starttags in idle.html." + class_ = '' + for a, v in attrs: + if a == 'class': + class_ = v + s = '' + if tag == 'div' and class_ == 'section': + self.show = True # start of main content + elif tag == 'div' and class_ == 'sphinxsidebar': + self.show = False # end of main content + elif tag == 'p' and class_ != 'first': + s = '\n\n' + elif tag == 'span' and class_ == 'pre': + self.tags = 'pre' + elif tag == 'span' and class_ == 'versionmodified': + self.tags = 'em' + elif tag == 'em': + self.tags = 'em' + elif tag in ['ul', 'ol']: + if class_.find('simple') != -1: + s = '\n' + self.simplelist = True + else: + self.simplelist = False + self.indent() + elif tag == 'dl': + if self.level > 0: + self.nested_dl = True + elif tag == 'li': + s = '\n* ' if self.simplelist else '\n\n* ' + elif tag == 'dt': + s = '\n\n' if not self.nested_dl else '\n' # avoid extra line + self.nested_dl = False + elif tag == 'dd': + self.indent() + s = '\n' + elif tag == 'pre': + self.pre = True + if self.show: + self.text.insert('end', '\n\n') + self.tags = 'preblock' + elif tag == 'a' and class_ == 'headerlink': + self.hdrlink = True + elif tag == 'h1': + self.text.mark_set('toc'+str(self.tocid), + self.text.index('end-1line')) + self.tags = tag + elif tag in ['h2', 'h3']: + if self.show: + self.data = '' + self.text.mark_set('toc'+str(self.tocid), + self.text.index('end-1line')) + self.text.insert('end', '\n\n') + self.tags = tag + if self.show: + self.text.insert('end', s, self.tags) + + def handle_endtag(self, tag): + "Handle endtags in idle.html." + if tag in ['h1', 'h2', 'h3', 'span', 'em']: + self.indent(0) # clear tag, reset indent + if self.show and tag in ['h1', 'h2', 'h3']: + title = self.data + self.contents.append(('toc'+str(self.tocid), title)) + self.tocid += 1 + elif tag == 'a': + self.hdrlink = False + elif tag == 'pre': + self.pre = False + self.tags = '' + elif tag in ['ul', 'dd', 'ol']: + self.indent(amt=-1) + + def handle_data(self, data): + "Handle date segments in idle.html." + if self.show and not self.hdrlink: + d = data if self.pre else data.replace('\n', ' ') + if self.tags == 'h1': + self.hprefix = d[0:d.index(' ')] + if self.tags in ['h1', 'h2', 'h3'] and self.hprefix != '': + if d[0:len(self.hprefix)] == self.hprefix: + d = d[len(self.hprefix):].strip() + self.data += d + self.text.insert('end', d, self.tags) + + def handle_charref(self, name): + self.text.insert('end', unichr(int(name))) + + +class HelpText(Text): + "Display idle.html." + def __init__(self, parent, filename): + "Configure tags and feed file to parser." + Text.__init__(self, parent, wrap='word', highlightthickness=0, + padx=5, borderwidth=0) + + normalfont = self.findfont(['TkDefaultFont', 'arial', 'helvetica']) + fixedfont = self.findfont(['TkFixedFont', 'monaco', 'courier']) + self['font'] = (normalfont, 12) + self.tag_configure('em', font=(normalfont, 12, 'italic')) + self.tag_configure('h1', font=(normalfont, 20, 'bold')) + self.tag_configure('h2', font=(normalfont, 18, 'bold')) + self.tag_configure('h3', font=(normalfont, 15, 'bold')) + self.tag_configure('pre', font=(fixedfont, 12)) + self.tag_configure('preblock', font=(fixedfont, 10), lmargin1=25, + borderwidth=1, relief='solid', background='#eeffcc') + self.tag_configure('l1', lmargin1=25, lmargin2=25) + self.tag_configure('l2', lmargin1=50, lmargin2=50) + self.tag_configure('l3', lmargin1=75, lmargin2=75) + self.tag_configure('l4', lmargin1=100, lmargin2=100) + + self.parser = HelpParser(self) + with open(filename) as f: + contents = f.read().decode(encoding='utf-8') + self.parser.feed(contents) + self['state'] = 'disabled' + + def findfont(self, names): + "Return name of first font family derived from names." + for name in names: + if name.lower() in (x.lower() for x in tkfont.names(root=self)): + font = tkfont.Font(name=name, exists=True, root=self) + return font.actual()['family'] + elif name.lower() in (x.lower() + for x in tkfont.families(root=self)): + return name + + +class HelpFrame(Frame): + def __init__(self, parent, filename): + Frame.__init__(self, parent) + text = HelpText(self, filename) + self['background'] = text['background'] + scroll = Scrollbar(self, command=text.yview) + text['yscrollcommand'] = scroll.set + text.grid(column=1, row=0, sticky='nsew') + scroll.grid(column=2, row=0, sticky='ns') + self.grid_columnconfigure(1, weight=1) + self.grid_rowconfigure(0, weight=1) + toc = self.contents_widget(text) + toc.grid(column=0, row=0, sticky='nw') + + def contents_widget(self, text): + toc = Menubutton(self, text='TOC') + drop = Menu(toc, tearoff=False) + for tag, lbl in text.parser.contents: + drop.add_command(label=lbl, command=lambda mark=tag:text.see(mark)) + toc['menu'] = drop + return toc + + +class HelpWindow(Toplevel): + + def __init__(self, parent, filename, title): + Toplevel.__init__(self, parent) + self.wm_title(title) + self.protocol("WM_DELETE_WINDOW", self.destroy) + HelpFrame(self, filename).grid(column=0, row=0, sticky='nsew') + self.grid_columnconfigure(0, weight=1) + self.grid_rowconfigure(0, weight=1) + + +def show_idlehelp(parent): + filename = join(abspath(dirname(__file__)), 'idle.html') + if not isfile(filename): + dirpath = join(abspath(dirname(dirname(dirname(__file__)))), + 'Doc', 'build', 'html', 'library') + HelpWindow(parent, filename, 'IDLE Help') + +if __name__ == '__main__': + from idlelib.idle_test.htest import run + run(show_idlehelp) diff --git a/Lib/idlelib/idle.html b/Lib/idlelib/idle.html new file mode 100644 --- /dev/null +++ b/Lib/idlelib/idle.html @@ -0,0 +1,671 @@ + + + + + + + + 24.6. IDLE — Python 2.7.10 documentation + + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+
+
+ +
+

24.6. IDLE?

+

IDLE is the Python IDE built with the tkinter GUI toolkit.

+

IDLE has the following features:

+
    +
  • coded in 100% pure Python, using the tkinter GUI toolkit
  • +
  • cross-platform: works on Windows, Unix, and Mac OS X
  • +
  • multi-window text editor with multiple undo, Python colorizing, +smart indent, call tips, and many other features
  • +
  • Python shell window (a.k.a. interactive interpreter)
  • +
  • debugger (not complete, but you can set breakpoints, view and step)
  • +
+ +
+

24.6.2. Editing and navigation?

+

In this section, ‘C’ refers to the Control key on Windows and Unix and +the Command key on Mac OSX.

+
    +
  • Backspace deletes to the left; Del deletes to the right

    +
  • +
  • C-Backspace delete word left; C-Del delete word to the right

    +
  • +
  • Arrow keys and Page Up/Page Down to move around

    +
  • +
  • C-LeftArrow and C-RightArrow moves by words

    +
  • +
  • Home/End go to begin/end of line

    +
  • +
  • C-Home/C-End go to begin/end of file

    +
  • +
  • Some useful Emacs bindings are inherited from Tcl/Tk:

    +
    +
      +
    • C-a beginning of line
    • +
    • C-e end of line
    • +
    • C-k kill line (but doesn’t put it in clipboard)
    • +
    • C-l center window around the insertion point
    • +
    • C-b go backwards one character without deleting (usually you can +also use the cursor key for this)
    • +
    • C-f go forward one character without deleting (usually you can +also use the cursor key for this)
    • +
    • C-p go up one line (usually you can also use the cursor key for +this)
    • +
    • C-d delete next character
    • +
    +
    +
  • +
+

Standard keybindings (like C-c to copy and C-v to paste) +may work. Keybindings are selected in the Configure IDLE dialog.

+
+

24.6.2.1. Automatic indentation?

+

After a block-opening statement, the next line is indented by 4 spaces (in the +Python Shell window by one tab). After certain keywords (break, return etc.) +the next line is dedented. In leading indentation, Backspace deletes up +to 4 spaces if they are there. Tab inserts spaces (in the Python +Shell window one tab), number depends on Indent width. Currently tabs +are restricted to four spaces due to Tcl/Tk limitations.

+

See also the indent/dedent region commands in the edit menu.

+
+
+

24.6.2.2. Completions?

+

Completions are supplied for functions, classes, and attributes of classes, +both built-in and user-defined. Completions are also provided for +filenames.

+

The AutoCompleteWindow (ACW) will open after a predefined delay (default is +two seconds) after a ‘.’ or (in a string) an os.sep is typed. If after one +of those characters (plus zero or more other characters) a tab is typed +the ACW will open immediately if a possible continuation is found.

+

If there is only one possible completion for the characters entered, a +Tab will supply that completion without opening the ACW.

+

‘Show Completions’ will force open a completions window, by default the +C-space will open a completions window. In an empty +string, this will contain the files in the current directory. On a +blank line, it will contain the built-in and user-defined functions and +classes in the current name spaces, plus any modules imported. If some +characters have been entered, the ACW will attempt to be more specific.

+

If a string of characters is typed, the ACW selection will jump to the +entry most closely matching those characters. Entering a tab will +cause the longest non-ambiguous match to be entered in the Editor window or +Shell. Two tab in a row will supply the current ACW selection, as +will return or a double click. Cursor keys, Page Up/Down, mouse selection, +and the scroll wheel all operate on the ACW.

+

“Hidden” attributes can be accessed by typing the beginning of hidden +name after a ‘.’, e.g. ‘_’. This allows access to modules with +__all__ set, or to class-private attributes.

+

Completions and the ‘Expand Word’ facility can save a lot of typing!

+

Completions are currently limited to those in the namespaces. Names in +an Editor window which are not via __main__ and sys.modules will +not be found. Run the module once with your imports to correct this situation. +Note that IDLE itself places quite a few modules in sys.modules, so +much can be found by default, e.g. the re module.

+

If you don’t like the ACW popping up unbidden, simply make the delay +longer or disable the extension. Or another option is the delay could +be set to zero. Another alternative to preventing ACW popups is to +disable the call tips extension.

+
+
+

24.6.2.3. Python Shell window?

+
    +
  • C-c interrupts executing command

    +
  • +
  • C-d sends end-of-file; closes window if typed at a >>> prompt

    +
  • +
  • Alt-/ (Expand word) is also useful to reduce typing

    +

    Command history

    +
      +
    • Alt-p retrieves previous command matching what you have typed. On +OS X use C-p.
    • +
    • Alt-n retrieves next. On OS X use C-n.
    • +
    • Return while on any previous command retrieves that command
    • +
    +
  • +
+
+
+
+

24.6.3. Syntax colors?

+

The coloring is applied in a background “thread,” so you may occasionally see +uncolorized text. To change the color scheme, edit the [Colors] section in +config.txt.

+
+
Python syntax colors:
+
+
Keywords
+
orange
+
Strings
+
green
+
Comments
+
red
+
Definitions
+
blue
+
+
+
Shell colors:
+
+
Console output
+
brown
+
stdout
+
blue
+
stderr
+
dark green
+
stdin
+
black
+
+
+
+
+
+

24.6.4. Startup?

+

Upon startup with the -s option, IDLE will execute the file referenced by +the environment variables IDLESTARTUP or PYTHONSTARTUP. +IDLE first checks for IDLESTARTUP; if IDLESTARTUP is present the file +referenced is run. If IDLESTARTUP is not present, IDLE checks for +PYTHONSTARTUP. Files referenced by these environment variables are +convenient places to store functions that are used frequently from the IDLE +shell, or for executing import statements to import common modules.

+

In addition, Tk also loads a startup file if it is present. Note that the +Tk file is loaded unconditionally. This additional file is .Idle.py and is +looked for in the user’s home directory. Statements in this file will be +executed in the Tk namespace, so this file is not useful for importing +functions to be used from IDLE’s Python shell.

+
+

24.6.4.1. Command line usage?

+
idle.py [-c command] [-d] [-e] [-s] [-t title] [arg] ...
+
+-c command  run this command
+-d          enable debugger
+-e          edit mode; arguments are files to be edited
+-s          run $IDLESTARTUP or $PYTHONSTARTUP first
+-t title    set title of shell window
+
+
+

If there are arguments:

+
    +
  1. If -e is used, arguments are files opened for editing and +sys.argv reflects the arguments passed to IDLE itself.
  2. +
  3. Otherwise, if -c is used, all arguments are placed in +sys.argv[1:...], with sys.argv[0] set to '-c'.
  4. +
  5. Otherwise, if neither -e nor -c is used, the first +argument is a script which is executed with the remaining arguments in +sys.argv[1:...] and sys.argv[0] set to the script name. If the +script name is ‘-‘, no script is executed but an interactive Python session +is started; the arguments are still available in sys.argv.
  6. +
+
+
+

24.6.4.2. Running without a subprocess?

+

If IDLE is started with the -n command line switch it will run in a +single process and will not create the subprocess which runs the RPC +Python execution server. This can be useful if Python cannot create +the subprocess or the RPC socket interface on your platform. However, +in this mode user code is not isolated from IDLE itself. Also, the +environment is not restarted when Run/Run Module (F5) is selected. If +your code has been modified, you must reload() the affected modules and +re-import any specific items (e.g. from foo import baz) if the changes +are to take effect. For these reasons, it is preferable to run IDLE +with the default subprocess if at all possible.

+
+

Deprecated since version 3.4.

+
+
+
+
+

24.6.5. Help and preferences?

+
+

24.6.5.1. Additional help sources?

+

IDLE includes a help menu entry called “Python Docs” that will open the +extensive sources of help, including tutorials, available at docs.python.org. +Selected URLs can be added or removed from the help menu at any time using the +Configure IDLE dialog. See the IDLE help option in the help menu of IDLE for +more information.

+
+
+

24.6.5.2. Setting preferences?

+

The font preferences, highlighting, keys, and general preferences can be +changed via Configure IDLE on the Option menu. Keys can be user defined; +IDLE ships with four built in key sets. In addition a user can create a +custom key set in the Configure IDLE dialog under the keys tab.

+
+
+

24.6.5.3. Extensions?

+

IDLE contains an extension facility. Peferences for extensions can be +changed with Configure Extensions. See the beginning of config-extensions.def +in the idlelib directory for further information. The default extensions +are currently:

+
    +
  • FormatParagraph
  • +
  • AutoExpand
  • +
  • ZoomHeight
  • +
  • ScriptBinding
  • +
  • CallTips
  • +
  • ParenMatch
  • +
  • AutoComplete
  • +
  • CodeContext
  • +
  • RstripExtension
  • +
+
+
+
+ + +
+
+
+ +
+
+ + + + + \ No newline at end of file -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 21 02:06:14 2015 From: python-checkins at python.org (terry.reedy) Date: Mon, 21 Sep 2015 00:06:14 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Merge_with_3=2E4?= Message-ID: <20150921000613.82652.78917@psf.io> https://hg.python.org/cpython/rev/605dffe0972c changeset: 98093:605dffe0972c branch: 3.5 parent: 98087:7f76c7d853be parent: 98092:2d808b72996d user: Terry Jan Reedy date: Sun Sep 20 19:57:37 2015 -0400 summary: Merge with 3.4 files: Lib/idlelib/EditorWindow.py | 12 +- Lib/idlelib/help.py | 233 +++++++ Lib/idlelib/help.txt | 4 + Lib/idlelib/idle.html | 671 +++++++++++++++++++++ Lib/idlelib/idle_test/htest.py | 14 +- Lib/idlelib/macosxSupport.py | 5 +- 6 files changed, 927 insertions(+), 12 deletions(-) diff --git a/Lib/idlelib/EditorWindow.py b/Lib/idlelib/EditorWindow.py --- a/Lib/idlelib/EditorWindow.py +++ b/Lib/idlelib/EditorWindow.py @@ -21,6 +21,7 @@ from idlelib.configHandler import idleConf from idlelib import aboutDialog, textView, configDialog from idlelib import macosxSupport +from idlelib import help # The default tab setting for a Text widget, in average-width characters. TK_TABWIDTH_DEFAULT = 8 @@ -42,6 +43,11 @@ class HelpDialog(object): def __init__(self): + import warnings as w + w.warn("EditorWindow.HelpDialog is no longer used by Idle.\n" + "It will be removed in 3.6 or later.\n" + "It has been replaced by private help.HelpWindow\n", + DeprecationWarning, stacklevel=2) self.parent = None # parent of help window self.dlg = None # the help window iteself @@ -539,11 +545,13 @@ configDialog.ConfigExtensionsDialog(self.top) def help_dialog(self, event=None): + "Handle help doc event." + # edit maxosxSupport.overrideRootMenu.help_dialog to match if self.root: parent = self.root else: parent = self.top - helpDialog.display(parent, near=self.top) + help.show_idlehelp(parent) def python_docs(self, event=None): if sys.platform[:3] == 'win': @@ -1716,4 +1724,4 @@ if __name__ == '__main__': from idlelib.idle_test.htest import run - run(_help_dialog, _editor_window) + run(_editor_window) diff --git a/Lib/idlelib/help.py b/Lib/idlelib/help.py new file mode 100644 --- /dev/null +++ b/Lib/idlelib/help.py @@ -0,0 +1,233 @@ +""" +help.py implements the Idle help menu and is subject to change. + +The contents are subject to revision at any time, without notice. + +Help => About IDLE: diplay About Idle dialog + + + +Help => IDLE Help: display idle.html with proper formatting + +HelpParser - Parses idle.html generated from idle.rst by Sphinx +and renders to tk Text. + +HelpText - Displays formatted idle.html. + +HelpFrame - Contains text, scrollbar, and table-of-contents. +(This will be needed for display in a future tabbed window.) + +HelpWindow - Display idleframe in a standalone window. + +show_idlehelp - Create HelpWindow. Called in EditorWindow.help_dialog. +""" +from html.parser import HTMLParser +from os.path import abspath, dirname, isdir, isfile, join +from tkinter import Tk, Toplevel, Frame, Text, Scrollbar, Menu, Menubutton +from tkinter import font as tkfont + +use_ttk = False # until available to import +if use_ttk: + from tkinter.ttk import Menubutton + +## About IDLE ## + + +## IDLE Help ## + +class HelpParser(HTMLParser): + """Render idle.html generated by Sphinx from idle.rst. + + The overridden handle_xyz methods handle a subset of html tags. + The supplied text should have the needed tag configurations. + The behavior for unsupported tags, such as table, is undefined. + """ + def __init__(self, text): + HTMLParser.__init__(self, convert_charrefs=True) + self.text = text # text widget we're rendering into + self.tags = '' # current text tags to apply + self.show = False # used so we exclude page navigation + self.hdrlink = False # used so we don't show header links + self.level = 0 # indentation level + self.pre = False # displaying preformatted text + self.hprefix = '' # strip e.g. '25.5' from headings + self.nested_dl = False # if we're in a nested
+ self.simplelist = False # simple list (no double spacing) + self.tocid = 1 # id for table of contents entries + self.contents = [] # map toc ids to section titles + self.data = '' # to record data within header tags for toc + + def indent(self, amt=1): + self.level += amt + self.tags = '' if self.level == 0 else 'l'+str(self.level) + + def handle_starttag(self, tag, attrs): + "Handle starttags in idle.html." + class_ = '' + for a, v in attrs: + if a == 'class': + class_ = v + s = '' + if tag == 'div' and class_ == 'section': + self.show = True # start of main content + elif tag == 'div' and class_ == 'sphinxsidebar': + self.show = False # end of main content + elif tag == 'p' and class_ != 'first': + s = '\n\n' + elif tag == 'span' and class_ == 'pre': + self.tags = 'pre' + elif tag == 'span' and class_ == 'versionmodified': + self.tags = 'em' + elif tag == 'em': + self.tags = 'em' + elif tag in ['ul', 'ol']: + if class_.find('simple') != -1: + s = '\n' + self.simplelist = True + else: + self.simplelist = False + self.indent() + elif tag == 'dl': + if self.level > 0: + self.nested_dl = True + elif tag == 'li': + s = '\n* ' if self.simplelist else '\n\n* ' + elif tag == 'dt': + s = '\n\n' if not self.nested_dl else '\n' # avoid extra line + self.nested_dl = False + elif tag == 'dd': + self.indent() + s = '\n' + elif tag == 'pre': + self.pre = True + if self.show: + self.text.insert('end', '\n\n') + self.tags = 'preblock' + elif tag == 'a' and class_ == 'headerlink': + self.hdrlink = True + elif tag == 'h1': + self.text.mark_set('toc'+str(self.tocid), + self.text.index('end-1line')) + self.tags = tag + elif tag in ['h2', 'h3']: + if self.show: + self.data = '' + self.text.mark_set('toc'+str(self.tocid), + self.text.index('end-1line')) + self.text.insert('end', '\n\n') + self.tags = tag + if self.show: + self.text.insert('end', s, self.tags) + + def handle_endtag(self, tag): + "Handle endtags in idle.html." + if tag in ['h1', 'h2', 'h3', 'span', 'em']: + self.indent(0) # clear tag, reset indent + if self.show and tag in ['h1', 'h2', 'h3']: + title = self.data + self.contents.append(('toc'+str(self.tocid), title)) + self.tocid += 1 + elif tag == 'a': + self.hdrlink = False + elif tag == 'pre': + self.pre = False + self.tags = '' + elif tag in ['ul', 'dd', 'ol']: + self.indent(amt=-1) + + def handle_data(self, data): + "Handle date segments in idle.html." + if self.show and not self.hdrlink: + d = data if self.pre else data.replace('\n', ' ') + if self.tags == 'h1': + self.hprefix = d[0:d.index(' ')] + if self.tags in ['h1', 'h2', 'h3'] and self.hprefix != '': + if d[0:len(self.hprefix)] == self.hprefix: + d = d[len(self.hprefix):].strip() + self.data += d + self.text.insert('end', d, self.tags) + + +class HelpText(Text): + "Display idle.html." + def __init__(self, parent, filename): + "Configure tags and feed file to parser." + Text.__init__(self, parent, wrap='word', highlightthickness=0, + padx=5, borderwidth=0) + + normalfont = self.findfont(['TkDefaultFont', 'arial', 'helvetica']) + fixedfont = self.findfont(['TkFixedFont', 'monaco', 'courier']) + self['font'] = (normalfont, 12) + self.tag_configure('em', font=(normalfont, 12, 'italic')) + self.tag_configure('h1', font=(normalfont, 20, 'bold')) + self.tag_configure('h2', font=(normalfont, 18, 'bold')) + self.tag_configure('h3', font=(normalfont, 15, 'bold')) + self.tag_configure('pre', font=(fixedfont, 12)) + self.tag_configure('preblock', font=(fixedfont, 10), lmargin1=25, + borderwidth=1, relief='solid', background='#eeffcc') + self.tag_configure('l1', lmargin1=25, lmargin2=25) + self.tag_configure('l2', lmargin1=50, lmargin2=50) + self.tag_configure('l3', lmargin1=75, lmargin2=75) + self.tag_configure('l4', lmargin1=100, lmargin2=100) + + self.parser = HelpParser(self) + with open(filename, encoding='utf-8') as f: + contents = f.read() + self.parser.feed(contents) + self['state'] = 'disabled' + + def findfont(self, names): + "Return name of first font family derived from names." + for name in names: + if name.lower() in (x.lower() for x in tkfont.names(root=self)): + font = tkfont.Font(name=name, exists=True, root=self) + return font.actual()['family'] + elif name.lower() in (x.lower() + for x in tkfont.families(root=self)): + return name + + +class HelpFrame(Frame): + def __init__(self, parent, filename): + Frame.__init__(self, parent) + text = HelpText(self, filename) + self['background'] = text['background'] + scroll = Scrollbar(self, command=text.yview) + text['yscrollcommand'] = scroll.set + text.grid(column=1, row=0, sticky='nsew') + scroll.grid(column=2, row=0, sticky='ns') + self.grid_columnconfigure(1, weight=1) + self.grid_rowconfigure(0, weight=1) + toc = self.contents_widget(text) + toc.grid(column=0, row=0, sticky='nw') + + def contents_widget(self, text): + toc = Menubutton(self, text='TOC') + drop = Menu(toc, tearoff=False) + for tag, lbl in text.parser.contents: + drop.add_command(label=lbl, command=lambda mark=tag:text.see(mark)) + toc['menu'] = drop + return toc + + +class HelpWindow(Toplevel): + + def __init__(self, parent, filename, title): + Toplevel.__init__(self, parent) + self.wm_title(title) + self.protocol("WM_DELETE_WINDOW", self.destroy) + HelpFrame(self, filename).grid(column=0, row=0, sticky='nsew') + self.grid_columnconfigure(0, weight=1) + self.grid_rowconfigure(0, weight=1) + + +def show_idlehelp(parent): + filename = join(abspath(dirname(__file__)), 'idle.html') + if not isfile(filename): + dirpath = join(abspath(dirname(dirname(dirname(__file__)))), + 'Doc', 'build', 'html', 'library') + HelpWindow(parent, filename, 'IDLE Help') + +if __name__ == '__main__': + from idlelib.idle_test.htest import run + run(show_idlehelp) diff --git a/Lib/idlelib/help.txt b/Lib/idlelib/help.txt --- a/Lib/idlelib/help.txt +++ b/Lib/idlelib/help.txt @@ -1,3 +1,7 @@ +This file, idlelib/help.txt is out-of-date and no longer used by Idle. +It is deprecated and will be removed in the future, possibly in 3.6 +---------------------------------------------------------------------- + [See the end of this file for ** TIPS ** on using IDLE !!] IDLE is the Python IDE built with the tkinter GUI toolkit. diff --git a/Lib/idlelib/idle.html b/Lib/idlelib/idle.html new file mode 100644 --- /dev/null +++ b/Lib/idlelib/idle.html @@ -0,0 +1,671 @@ + + + + + + + + 25.5. IDLE — Python 3.4.3 documentation + + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+
+
+ +
+

25.5. IDLE?

+

IDLE is the Python IDE built with the tkinter GUI toolkit.

+

IDLE has the following features:

+
    +
  • coded in 100% pure Python, using the tkinter GUI toolkit
  • +
  • cross-platform: works on Windows, Unix, and Mac OS X
  • +
  • multi-window text editor with multiple undo, Python colorizing, +smart indent, call tips, and many other features
  • +
  • Python shell window (a.k.a. interactive interpreter)
  • +
  • debugger (not complete, but you can set breakpoints, view and step)
  • +
+ +
+

25.5.2. Editing and navigation?

+

In this section, ‘C’ refers to the Control key on Windows and Unix and +the Command key on Mac OSX.

+
    +
  • Backspace deletes to the left; Del deletes to the right

    +
  • +
  • C-Backspace delete word left; C-Del delete word to the right

    +
  • +
  • Arrow keys and Page Up/Page Down to move around

    +
  • +
  • C-LeftArrow and C-RightArrow moves by words

    +
  • +
  • Home/End go to begin/end of line

    +
  • +
  • C-Home/C-End go to begin/end of file

    +
  • +
  • Some useful Emacs bindings are inherited from Tcl/Tk:

    +
    +
      +
    • C-a beginning of line
    • +
    • C-e end of line
    • +
    • C-k kill line (but doesn’t put it in clipboard)
    • +
    • C-l center window around the insertion point
    • +
    • C-b go backwards one character without deleting (usually you can +also use the cursor key for this)
    • +
    • C-f go forward one character without deleting (usually you can +also use the cursor key for this)
    • +
    • C-p go up one line (usually you can also use the cursor key for +this)
    • +
    • C-d delete next character
    • +
    +
    +
  • +
+

Standard keybindings (like C-c to copy and C-v to paste) +may work. Keybindings are selected in the Configure IDLE dialog.

+
+

25.5.2.1. Automatic indentation?

+

After a block-opening statement, the next line is indented by 4 spaces (in the +Python Shell window by one tab). After certain keywords (break, return etc.) +the next line is dedented. In leading indentation, Backspace deletes up +to 4 spaces if they are there. Tab inserts spaces (in the Python +Shell window one tab), number depends on Indent width. Currently tabs +are restricted to four spaces due to Tcl/Tk limitations.

+

See also the indent/dedent region commands in the edit menu.

+
+
+

25.5.2.2. Completions?

+

Completions are supplied for functions, classes, and attributes of classes, +both built-in and user-defined. Completions are also provided for +filenames.

+

The AutoCompleteWindow (ACW) will open after a predefined delay (default is +two seconds) after a ‘.’ or (in a string) an os.sep is typed. If after one +of those characters (plus zero or more other characters) a tab is typed +the ACW will open immediately if a possible continuation is found.

+

If there is only one possible completion for the characters entered, a +Tab will supply that completion without opening the ACW.

+

‘Show Completions’ will force open a completions window, by default the +C-space will open a completions window. In an empty +string, this will contain the files in the current directory. On a +blank line, it will contain the built-in and user-defined functions and +classes in the current name spaces, plus any modules imported. If some +characters have been entered, the ACW will attempt to be more specific.

+

If a string of characters is typed, the ACW selection will jump to the +entry most closely matching those characters. Entering a tab will +cause the longest non-ambiguous match to be entered in the Editor window or +Shell. Two tab in a row will supply the current ACW selection, as +will return or a double click. Cursor keys, Page Up/Down, mouse selection, +and the scroll wheel all operate on the ACW.

+

“Hidden” attributes can be accessed by typing the beginning of hidden +name after a ‘.’, e.g. ‘_’. This allows access to modules with +__all__ set, or to class-private attributes.

+

Completions and the ‘Expand Word’ facility can save a lot of typing!

+

Completions are currently limited to those in the namespaces. Names in +an Editor window which are not via __main__ and sys.modules will +not be found. Run the module once with your imports to correct this situation. +Note that IDLE itself places quite a few modules in sys.modules, so +much can be found by default, e.g. the re module.

+

If you don’t like the ACW popping up unbidden, simply make the delay +longer or disable the extension. Or another option is the delay could +be set to zero. Another alternative to preventing ACW popups is to +disable the call tips extension.

+
+
+

25.5.2.3. Python Shell window?

+
    +
  • C-c interrupts executing command

    +
  • +
  • C-d sends end-of-file; closes window if typed at a >>> prompt

    +
  • +
  • Alt-/ (Expand word) is also useful to reduce typing

    +

    Command history

    +
      +
    • Alt-p retrieves previous command matching what you have typed. On +OS X use C-p.
    • +
    • Alt-n retrieves next. On OS X use C-n.
    • +
    • Return while on any previous command retrieves that command
    • +
    +
  • +
+
+
+
+

25.5.3. Syntax colors?

+

The coloring is applied in a background “thread,” so you may occasionally see +uncolorized text. To change the color scheme, edit the [Colors] section in +config.txt.

+
+
Python syntax colors:
+
+
Keywords
+
orange
+
Strings
+
green
+
Comments
+
red
+
Definitions
+
blue
+
+
+
Shell colors:
+
+
Console output
+
brown
+
stdout
+
blue
+
stderr
+
dark green
+
stdin
+
black
+
+
+
+
+
+

25.5.4. Startup?

+

Upon startup with the -s option, IDLE will execute the file referenced by +the environment variables IDLESTARTUP or PYTHONSTARTUP. +IDLE first checks for IDLESTARTUP; if IDLESTARTUP is present the file +referenced is run. If IDLESTARTUP is not present, IDLE checks for +PYTHONSTARTUP. Files referenced by these environment variables are +convenient places to store functions that are used frequently from the IDLE +shell, or for executing import statements to import common modules.

+

In addition, Tk also loads a startup file if it is present. Note that the +Tk file is loaded unconditionally. This additional file is .Idle.py and is +looked for in the user’s home directory. Statements in this file will be +executed in the Tk namespace, so this file is not useful for importing +functions to be used from IDLE’s Python shell.

+
+

25.5.4.1. Command line usage?

+
idle.py [-c command] [-d] [-e] [-s] [-t title] [arg] ...
+
+-c command  run this command
+-d          enable debugger
+-e          edit mode; arguments are files to be edited
+-s          run $IDLESTARTUP or $PYTHONSTARTUP first
+-t title    set title of shell window
+
+
+

If there are arguments:

+
    +
  1. If -e is used, arguments are files opened for editing and +sys.argv reflects the arguments passed to IDLE itself.
  2. +
  3. Otherwise, if -c is used, all arguments are placed in +sys.argv[1:...], with sys.argv[0] set to '-c'.
  4. +
  5. Otherwise, if neither -e nor -c is used, the first +argument is a script which is executed with the remaining arguments in +sys.argv[1:...] and sys.argv[0] set to the script name. If the +script name is ‘-‘, no script is executed but an interactive Python session +is started; the arguments are still available in sys.argv.
  6. +
+
+
+

25.5.4.2. Running without a subprocess?

+

If IDLE is started with the -n command line switch it will run in a +single process and will not create the subprocess which runs the RPC +Python execution server. This can be useful if Python cannot create +the subprocess or the RPC socket interface on your platform. However, +in this mode user code is not isolated from IDLE itself. Also, the +environment is not restarted when Run/Run Module (F5) is selected. If +your code has been modified, you must reload() the affected modules and +re-import any specific items (e.g. from foo import baz) if the changes +are to take effect. For these reasons, it is preferable to run IDLE +with the default subprocess if at all possible.

+
+

Deprecated since version 3.4.

+
+
+
+
+

25.5.5. Help and preferences?

+
+

25.5.5.1. Additional help sources?

+

IDLE includes a help menu entry called “Python Docs” that will open the +extensive sources of help, including tutorials, available at docs.python.org. +Selected URLs can be added or removed from the help menu at any time using the +Configure IDLE dialog. See the IDLE help option in the help menu of IDLE for +more information.

+
+
+

25.5.5.2. Setting preferences?

+

The font preferences, highlighting, keys, and general preferences can be +changed via Configure IDLE on the Option menu. Keys can be user defined; +IDLE ships with four built in key sets. In addition a user can create a +custom key set in the Configure IDLE dialog under the keys tab.

+
+
+

25.5.5.3. Extensions?

+

IDLE contains an extension facility. Peferences for extensions can be +changed with Configure Extensions. See the beginning of config-extensions.def +in the idlelib directory for further information. The default extensions +are currently:

+
    +
  • FormatParagraph
  • +
  • AutoExpand
  • +
  • ZoomHeight
  • +
  • ScriptBinding
  • +
  • CallTips
  • +
  • ParenMatch
  • +
  • AutoComplete
  • +
  • CodeContext
  • +
  • RstripExtension
  • +
+
+
+
+ + +
+
+
+ +
+
+ + + + + \ No newline at end of file diff --git a/Lib/idlelib/idle_test/htest.py b/Lib/idlelib/idle_test/htest.py --- a/Lib/idlelib/idle_test/htest.py +++ b/Lib/idlelib/idle_test/htest.py @@ -194,13 +194,6 @@ "should open that file \nin a new EditorWindow." } -_help_dialog_spec = { - 'file': 'EditorWindow', - 'kwds': {}, - 'msg': "If the help text displays, this works.\n" - "Text is selectable. Window is scrollable." - } - _io_binding_spec = { 'file': 'IOBinding', 'kwds': {}, @@ -279,6 +272,13 @@ "Right clicking an item will display a popup." } +show_idlehelp_spec = { + 'file': 'help', + 'kwds': {}, + 'msg': "If the help text displays, this works.\n" + "Text is selectable. Window is scrollable." + } + _stack_viewer_spec = { 'file': 'StackViewer', 'kwds': {}, diff --git a/Lib/idlelib/macosxSupport.py b/Lib/idlelib/macosxSupport.py --- a/Lib/idlelib/macosxSupport.py +++ b/Lib/idlelib/macosxSupport.py @@ -174,9 +174,8 @@ configDialog.ConfigDialog(root, 'Settings') def help_dialog(event=None): - from idlelib import textView - fn = path.join(path.abspath(path.dirname(__file__)), 'help.txt') - textView.view_file(root, 'Help', fn) + from idlelib import help + help.show_idlehelp(root) root.bind('<>', about_dialog) root.bind('<>', config_dialog) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 21 02:06:14 2015 From: python-checkins at python.org (terry.reedy) Date: Mon, 21 Sep 2015 00:06:14 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge_with_3=2E5?= Message-ID: <20150921000614.98378.31825@psf.io> https://hg.python.org/cpython/rev/9e8c3be5b5db changeset: 98094:9e8c3be5b5db parent: 98089:2fcd2540ab97 parent: 98093:605dffe0972c user: Terry Jan Reedy date: Sun Sep 20 19:57:58 2015 -0400 summary: Merge with 3.5 files: Lib/idlelib/EditorWindow.py | 12 +- Lib/idlelib/help.py | 233 +++++++ Lib/idlelib/help.txt | 4 + Lib/idlelib/idle.html | 671 +++++++++++++++++++++ Lib/idlelib/idle_test/htest.py | 14 +- Lib/idlelib/macosxSupport.py | 5 +- 6 files changed, 927 insertions(+), 12 deletions(-) diff --git a/Lib/idlelib/EditorWindow.py b/Lib/idlelib/EditorWindow.py --- a/Lib/idlelib/EditorWindow.py +++ b/Lib/idlelib/EditorWindow.py @@ -21,6 +21,7 @@ from idlelib.configHandler import idleConf from idlelib import aboutDialog, textView, configDialog from idlelib import macosxSupport +from idlelib import help # The default tab setting for a Text widget, in average-width characters. TK_TABWIDTH_DEFAULT = 8 @@ -42,6 +43,11 @@ class HelpDialog(object): def __init__(self): + import warnings as w + w.warn("EditorWindow.HelpDialog is no longer used by Idle.\n" + "It will be removed in 3.6 or later.\n" + "It has been replaced by private help.HelpWindow\n", + DeprecationWarning, stacklevel=2) self.parent = None # parent of help window self.dlg = None # the help window iteself @@ -539,11 +545,13 @@ configDialog.ConfigExtensionsDialog(self.top) def help_dialog(self, event=None): + "Handle help doc event." + # edit maxosxSupport.overrideRootMenu.help_dialog to match if self.root: parent = self.root else: parent = self.top - helpDialog.display(parent, near=self.top) + help.show_idlehelp(parent) def python_docs(self, event=None): if sys.platform[:3] == 'win': @@ -1716,4 +1724,4 @@ if __name__ == '__main__': from idlelib.idle_test.htest import run - run(_help_dialog, _editor_window) + run(_editor_window) diff --git a/Lib/idlelib/help.py b/Lib/idlelib/help.py new file mode 100644 --- /dev/null +++ b/Lib/idlelib/help.py @@ -0,0 +1,233 @@ +""" +help.py implements the Idle help menu and is subject to change. + +The contents are subject to revision at any time, without notice. + +Help => About IDLE: diplay About Idle dialog + + + +Help => IDLE Help: display idle.html with proper formatting + +HelpParser - Parses idle.html generated from idle.rst by Sphinx +and renders to tk Text. + +HelpText - Displays formatted idle.html. + +HelpFrame - Contains text, scrollbar, and table-of-contents. +(This will be needed for display in a future tabbed window.) + +HelpWindow - Display idleframe in a standalone window. + +show_idlehelp - Create HelpWindow. Called in EditorWindow.help_dialog. +""" +from html.parser import HTMLParser +from os.path import abspath, dirname, isdir, isfile, join +from tkinter import Tk, Toplevel, Frame, Text, Scrollbar, Menu, Menubutton +from tkinter import font as tkfont + +use_ttk = False # until available to import +if use_ttk: + from tkinter.ttk import Menubutton + +## About IDLE ## + + +## IDLE Help ## + +class HelpParser(HTMLParser): + """Render idle.html generated by Sphinx from idle.rst. + + The overridden handle_xyz methods handle a subset of html tags. + The supplied text should have the needed tag configurations. + The behavior for unsupported tags, such as table, is undefined. + """ + def __init__(self, text): + HTMLParser.__init__(self, convert_charrefs=True) + self.text = text # text widget we're rendering into + self.tags = '' # current text tags to apply + self.show = False # used so we exclude page navigation + self.hdrlink = False # used so we don't show header links + self.level = 0 # indentation level + self.pre = False # displaying preformatted text + self.hprefix = '' # strip e.g. '25.5' from headings + self.nested_dl = False # if we're in a nested
+ self.simplelist = False # simple list (no double spacing) + self.tocid = 1 # id for table of contents entries + self.contents = [] # map toc ids to section titles + self.data = '' # to record data within header tags for toc + + def indent(self, amt=1): + self.level += amt + self.tags = '' if self.level == 0 else 'l'+str(self.level) + + def handle_starttag(self, tag, attrs): + "Handle starttags in idle.html." + class_ = '' + for a, v in attrs: + if a == 'class': + class_ = v + s = '' + if tag == 'div' and class_ == 'section': + self.show = True # start of main content + elif tag == 'div' and class_ == 'sphinxsidebar': + self.show = False # end of main content + elif tag == 'p' and class_ != 'first': + s = '\n\n' + elif tag == 'span' and class_ == 'pre': + self.tags = 'pre' + elif tag == 'span' and class_ == 'versionmodified': + self.tags = 'em' + elif tag == 'em': + self.tags = 'em' + elif tag in ['ul', 'ol']: + if class_.find('simple') != -1: + s = '\n' + self.simplelist = True + else: + self.simplelist = False + self.indent() + elif tag == 'dl': + if self.level > 0: + self.nested_dl = True + elif tag == 'li': + s = '\n* ' if self.simplelist else '\n\n* ' + elif tag == 'dt': + s = '\n\n' if not self.nested_dl else '\n' # avoid extra line + self.nested_dl = False + elif tag == 'dd': + self.indent() + s = '\n' + elif tag == 'pre': + self.pre = True + if self.show: + self.text.insert('end', '\n\n') + self.tags = 'preblock' + elif tag == 'a' and class_ == 'headerlink': + self.hdrlink = True + elif tag == 'h1': + self.text.mark_set('toc'+str(self.tocid), + self.text.index('end-1line')) + self.tags = tag + elif tag in ['h2', 'h3']: + if self.show: + self.data = '' + self.text.mark_set('toc'+str(self.tocid), + self.text.index('end-1line')) + self.text.insert('end', '\n\n') + self.tags = tag + if self.show: + self.text.insert('end', s, self.tags) + + def handle_endtag(self, tag): + "Handle endtags in idle.html." + if tag in ['h1', 'h2', 'h3', 'span', 'em']: + self.indent(0) # clear tag, reset indent + if self.show and tag in ['h1', 'h2', 'h3']: + title = self.data + self.contents.append(('toc'+str(self.tocid), title)) + self.tocid += 1 + elif tag == 'a': + self.hdrlink = False + elif tag == 'pre': + self.pre = False + self.tags = '' + elif tag in ['ul', 'dd', 'ol']: + self.indent(amt=-1) + + def handle_data(self, data): + "Handle date segments in idle.html." + if self.show and not self.hdrlink: + d = data if self.pre else data.replace('\n', ' ') + if self.tags == 'h1': + self.hprefix = d[0:d.index(' ')] + if self.tags in ['h1', 'h2', 'h3'] and self.hprefix != '': + if d[0:len(self.hprefix)] == self.hprefix: + d = d[len(self.hprefix):].strip() + self.data += d + self.text.insert('end', d, self.tags) + + +class HelpText(Text): + "Display idle.html." + def __init__(self, parent, filename): + "Configure tags and feed file to parser." + Text.__init__(self, parent, wrap='word', highlightthickness=0, + padx=5, borderwidth=0) + + normalfont = self.findfont(['TkDefaultFont', 'arial', 'helvetica']) + fixedfont = self.findfont(['TkFixedFont', 'monaco', 'courier']) + self['font'] = (normalfont, 12) + self.tag_configure('em', font=(normalfont, 12, 'italic')) + self.tag_configure('h1', font=(normalfont, 20, 'bold')) + self.tag_configure('h2', font=(normalfont, 18, 'bold')) + self.tag_configure('h3', font=(normalfont, 15, 'bold')) + self.tag_configure('pre', font=(fixedfont, 12)) + self.tag_configure('preblock', font=(fixedfont, 10), lmargin1=25, + borderwidth=1, relief='solid', background='#eeffcc') + self.tag_configure('l1', lmargin1=25, lmargin2=25) + self.tag_configure('l2', lmargin1=50, lmargin2=50) + self.tag_configure('l3', lmargin1=75, lmargin2=75) + self.tag_configure('l4', lmargin1=100, lmargin2=100) + + self.parser = HelpParser(self) + with open(filename, encoding='utf-8') as f: + contents = f.read() + self.parser.feed(contents) + self['state'] = 'disabled' + + def findfont(self, names): + "Return name of first font family derived from names." + for name in names: + if name.lower() in (x.lower() for x in tkfont.names(root=self)): + font = tkfont.Font(name=name, exists=True, root=self) + return font.actual()['family'] + elif name.lower() in (x.lower() + for x in tkfont.families(root=self)): + return name + + +class HelpFrame(Frame): + def __init__(self, parent, filename): + Frame.__init__(self, parent) + text = HelpText(self, filename) + self['background'] = text['background'] + scroll = Scrollbar(self, command=text.yview) + text['yscrollcommand'] = scroll.set + text.grid(column=1, row=0, sticky='nsew') + scroll.grid(column=2, row=0, sticky='ns') + self.grid_columnconfigure(1, weight=1) + self.grid_rowconfigure(0, weight=1) + toc = self.contents_widget(text) + toc.grid(column=0, row=0, sticky='nw') + + def contents_widget(self, text): + toc = Menubutton(self, text='TOC') + drop = Menu(toc, tearoff=False) + for tag, lbl in text.parser.contents: + drop.add_command(label=lbl, command=lambda mark=tag:text.see(mark)) + toc['menu'] = drop + return toc + + +class HelpWindow(Toplevel): + + def __init__(self, parent, filename, title): + Toplevel.__init__(self, parent) + self.wm_title(title) + self.protocol("WM_DELETE_WINDOW", self.destroy) + HelpFrame(self, filename).grid(column=0, row=0, sticky='nsew') + self.grid_columnconfigure(0, weight=1) + self.grid_rowconfigure(0, weight=1) + + +def show_idlehelp(parent): + filename = join(abspath(dirname(__file__)), 'idle.html') + if not isfile(filename): + dirpath = join(abspath(dirname(dirname(dirname(__file__)))), + 'Doc', 'build', 'html', 'library') + HelpWindow(parent, filename, 'IDLE Help') + +if __name__ == '__main__': + from idlelib.idle_test.htest import run + run(show_idlehelp) diff --git a/Lib/idlelib/help.txt b/Lib/idlelib/help.txt --- a/Lib/idlelib/help.txt +++ b/Lib/idlelib/help.txt @@ -1,3 +1,7 @@ +This file, idlelib/help.txt is out-of-date and no longer used by Idle. +It is deprecated and will be removed in the future, possibly in 3.6 +---------------------------------------------------------------------- + [See the end of this file for ** TIPS ** on using IDLE !!] IDLE is the Python IDE built with the tkinter GUI toolkit. diff --git a/Lib/idlelib/idle.html b/Lib/idlelib/idle.html new file mode 100644 --- /dev/null +++ b/Lib/idlelib/idle.html @@ -0,0 +1,671 @@ + + + + + + + + 25.5. IDLE — Python 3.4.3 documentation + + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+
+
+ +
+

25.5. IDLE?

+

IDLE is the Python IDE built with the tkinter GUI toolkit.

+

IDLE has the following features:

+
    +
  • coded in 100% pure Python, using the tkinter GUI toolkit
  • +
  • cross-platform: works on Windows, Unix, and Mac OS X
  • +
  • multi-window text editor with multiple undo, Python colorizing, +smart indent, call tips, and many other features
  • +
  • Python shell window (a.k.a. interactive interpreter)
  • +
  • debugger (not complete, but you can set breakpoints, view and step)
  • +
+ +
+

25.5.2. Editing and navigation?

+

In this section, ‘C’ refers to the Control key on Windows and Unix and +the Command key on Mac OSX.

+
    +
  • Backspace deletes to the left; Del deletes to the right

    +
  • +
  • C-Backspace delete word left; C-Del delete word to the right

    +
  • +
  • Arrow keys and Page Up/Page Down to move around

    +
  • +
  • C-LeftArrow and C-RightArrow moves by words

    +
  • +
  • Home/End go to begin/end of line

    +
  • +
  • C-Home/C-End go to begin/end of file

    +
  • +
  • Some useful Emacs bindings are inherited from Tcl/Tk:

    +
    +
      +
    • C-a beginning of line
    • +
    • C-e end of line
    • +
    • C-k kill line (but doesn’t put it in clipboard)
    • +
    • C-l center window around the insertion point
    • +
    • C-b go backwards one character without deleting (usually you can +also use the cursor key for this)
    • +
    • C-f go forward one character without deleting (usually you can +also use the cursor key for this)
    • +
    • C-p go up one line (usually you can also use the cursor key for +this)
    • +
    • C-d delete next character
    • +
    +
    +
  • +
+

Standard keybindings (like C-c to copy and C-v to paste) +may work. Keybindings are selected in the Configure IDLE dialog.

+
+

25.5.2.1. Automatic indentation?

+

After a block-opening statement, the next line is indented by 4 spaces (in the +Python Shell window by one tab). After certain keywords (break, return etc.) +the next line is dedented. In leading indentation, Backspace deletes up +to 4 spaces if they are there. Tab inserts spaces (in the Python +Shell window one tab), number depends on Indent width. Currently tabs +are restricted to four spaces due to Tcl/Tk limitations.

+

See also the indent/dedent region commands in the edit menu.

+
+
+

25.5.2.2. Completions?

+

Completions are supplied for functions, classes, and attributes of classes, +both built-in and user-defined. Completions are also provided for +filenames.

+

The AutoCompleteWindow (ACW) will open after a predefined delay (default is +two seconds) after a ‘.’ or (in a string) an os.sep is typed. If after one +of those characters (plus zero or more other characters) a tab is typed +the ACW will open immediately if a possible continuation is found.

+

If there is only one possible completion for the characters entered, a +Tab will supply that completion without opening the ACW.

+

‘Show Completions’ will force open a completions window, by default the +C-space will open a completions window. In an empty +string, this will contain the files in the current directory. On a +blank line, it will contain the built-in and user-defined functions and +classes in the current name spaces, plus any modules imported. If some +characters have been entered, the ACW will attempt to be more specific.

+

If a string of characters is typed, the ACW selection will jump to the +entry most closely matching those characters. Entering a tab will +cause the longest non-ambiguous match to be entered in the Editor window or +Shell. Two tab in a row will supply the current ACW selection, as +will return or a double click. Cursor keys, Page Up/Down, mouse selection, +and the scroll wheel all operate on the ACW.

+

“Hidden” attributes can be accessed by typing the beginning of hidden +name after a ‘.’, e.g. ‘_’. This allows access to modules with +__all__ set, or to class-private attributes.

+

Completions and the ‘Expand Word’ facility can save a lot of typing!

+

Completions are currently limited to those in the namespaces. Names in +an Editor window which are not via __main__ and sys.modules will +not be found. Run the module once with your imports to correct this situation. +Note that IDLE itself places quite a few modules in sys.modules, so +much can be found by default, e.g. the re module.

+

If you don’t like the ACW popping up unbidden, simply make the delay +longer or disable the extension. Or another option is the delay could +be set to zero. Another alternative to preventing ACW popups is to +disable the call tips extension.

+
+
+

25.5.2.3. Python Shell window?

+
    +
  • C-c interrupts executing command

    +
  • +
  • C-d sends end-of-file; closes window if typed at a >>> prompt

    +
  • +
  • Alt-/ (Expand word) is also useful to reduce typing

    +

    Command history

    +
      +
    • Alt-p retrieves previous command matching what you have typed. On +OS X use C-p.
    • +
    • Alt-n retrieves next. On OS X use C-n.
    • +
    • Return while on any previous command retrieves that command
    • +
    +
  • +
+
+
+
+

25.5.3. Syntax colors?

+

The coloring is applied in a background “thread,” so you may occasionally see +uncolorized text. To change the color scheme, edit the [Colors] section in +config.txt.

+
+
Python syntax colors:
+
+
Keywords
+
orange
+
Strings
+
green
+
Comments
+
red
+
Definitions
+
blue
+
+
+
Shell colors:
+
+
Console output
+
brown
+
stdout
+
blue
+
stderr
+
dark green
+
stdin
+
black
+
+
+
+
+
+

25.5.4. Startup?

+

Upon startup with the -s option, IDLE will execute the file referenced by +the environment variables IDLESTARTUP or PYTHONSTARTUP. +IDLE first checks for IDLESTARTUP; if IDLESTARTUP is present the file +referenced is run. If IDLESTARTUP is not present, IDLE checks for +PYTHONSTARTUP. Files referenced by these environment variables are +convenient places to store functions that are used frequently from the IDLE +shell, or for executing import statements to import common modules.

+

In addition, Tk also loads a startup file if it is present. Note that the +Tk file is loaded unconditionally. This additional file is .Idle.py and is +looked for in the user’s home directory. Statements in this file will be +executed in the Tk namespace, so this file is not useful for importing +functions to be used from IDLE’s Python shell.

+
+

25.5.4.1. Command line usage?

+
idle.py [-c command] [-d] [-e] [-s] [-t title] [arg] ...
+
+-c command  run this command
+-d          enable debugger
+-e          edit mode; arguments are files to be edited
+-s          run $IDLESTARTUP or $PYTHONSTARTUP first
+-t title    set title of shell window
+
+
+

If there are arguments:

+
    +
  1. If -e is used, arguments are files opened for editing and +sys.argv reflects the arguments passed to IDLE itself.
  2. +
  3. Otherwise, if -c is used, all arguments are placed in +sys.argv[1:...], with sys.argv[0] set to '-c'.
  4. +
  5. Otherwise, if neither -e nor -c is used, the first +argument is a script which is executed with the remaining arguments in +sys.argv[1:...] and sys.argv[0] set to the script name. If the +script name is ‘-‘, no script is executed but an interactive Python session +is started; the arguments are still available in sys.argv.
  6. +
+
+
+

25.5.4.2. Running without a subprocess?

+

If IDLE is started with the -n command line switch it will run in a +single process and will not create the subprocess which runs the RPC +Python execution server. This can be useful if Python cannot create +the subprocess or the RPC socket interface on your platform. However, +in this mode user code is not isolated from IDLE itself. Also, the +environment is not restarted when Run/Run Module (F5) is selected. If +your code has been modified, you must reload() the affected modules and +re-import any specific items (e.g. from foo import baz) if the changes +are to take effect. For these reasons, it is preferable to run IDLE +with the default subprocess if at all possible.

+
+

Deprecated since version 3.4.

+
+
+
+
+

25.5.5. Help and preferences?

+
+

25.5.5.1. Additional help sources?

+

IDLE includes a help menu entry called “Python Docs” that will open the +extensive sources of help, including tutorials, available at docs.python.org. +Selected URLs can be added or removed from the help menu at any time using the +Configure IDLE dialog. See the IDLE help option in the help menu of IDLE for +more information.

+
+
+

25.5.5.2. Setting preferences?

+

The font preferences, highlighting, keys, and general preferences can be +changed via Configure IDLE on the Option menu. Keys can be user defined; +IDLE ships with four built in key sets. In addition a user can create a +custom key set in the Configure IDLE dialog under the keys tab.

+
+
+

25.5.5.3. Extensions?

+

IDLE contains an extension facility. Peferences for extensions can be +changed with Configure Extensions. See the beginning of config-extensions.def +in the idlelib directory for further information. The default extensions +are currently:

+
    +
  • FormatParagraph
  • +
  • AutoExpand
  • +
  • ZoomHeight
  • +
  • ScriptBinding
  • +
  • CallTips
  • +
  • ParenMatch
  • +
  • AutoComplete
  • +
  • CodeContext
  • +
  • RstripExtension
  • +
+
+
+
+ + +
+
+
+ +
+
+ + + + + \ No newline at end of file diff --git a/Lib/idlelib/idle_test/htest.py b/Lib/idlelib/idle_test/htest.py --- a/Lib/idlelib/idle_test/htest.py +++ b/Lib/idlelib/idle_test/htest.py @@ -194,13 +194,6 @@ "should open that file \nin a new EditorWindow." } -_help_dialog_spec = { - 'file': 'EditorWindow', - 'kwds': {}, - 'msg': "If the help text displays, this works.\n" - "Text is selectable. Window is scrollable." - } - _io_binding_spec = { 'file': 'IOBinding', 'kwds': {}, @@ -279,6 +272,13 @@ "Right clicking an item will display a popup." } +show_idlehelp_spec = { + 'file': 'help', + 'kwds': {}, + 'msg': "If the help text displays, this works.\n" + "Text is selectable. Window is scrollable." + } + _stack_viewer_spec = { 'file': 'StackViewer', 'kwds': {}, diff --git a/Lib/idlelib/macosxSupport.py b/Lib/idlelib/macosxSupport.py --- a/Lib/idlelib/macosxSupport.py +++ b/Lib/idlelib/macosxSupport.py @@ -174,9 +174,8 @@ configDialog.ConfigDialog(root, 'Settings') def help_dialog(event=None): - from idlelib import textView - fn = path.join(path.abspath(path.dirname(__file__)), 'help.txt') - textView.view_file(root, 'Help', fn) + from idlelib import help + help.show_idlehelp(root) root.bind('<>', about_dialog) root.bind('<>', config_dialog) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 21 02:06:14 2015 From: python-checkins at python.org (terry.reedy) Date: Mon, 21 Sep 2015 00:06:14 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzE2ODkz?= =?utf-8?q?=3A_Replace_help=2Etxt_with_idle=2Ehtml_for_Idle_doc_display=2E?= Message-ID: <20150921000613.16591.7434@psf.io> https://hg.python.org/cpython/rev/2d808b72996d changeset: 98092:2d808b72996d branch: 3.4 parent: 98084:2efd269b3eb8 user: Terry Jan Reedy date: Sun Sep 20 19:57:13 2015 -0400 summary: Issue #16893: Replace help.txt with idle.html for Idle doc display. The new idlelib/idle.html is copied from Doc/build/html/idle.html. It looks better than help.txt and will better document Idle as released. The tkinter html viewer that works for this file was written by Rose Roseman. The new code is in idlelib/help.py, a new file for help menu classes. The now unused EditorWindow.HelpDialog class and helt.txt file are deprecated. files: Lib/idlelib/EditorWindow.py | 12 +- Lib/idlelib/help.py | 233 +++++++ Lib/idlelib/help.txt | 4 + Lib/idlelib/idle.html | 671 +++++++++++++++++++++ Lib/idlelib/idle_test/htest.py | 14 +- Lib/idlelib/macosxSupport.py | 5 +- 6 files changed, 927 insertions(+), 12 deletions(-) diff --git a/Lib/idlelib/EditorWindow.py b/Lib/idlelib/EditorWindow.py --- a/Lib/idlelib/EditorWindow.py +++ b/Lib/idlelib/EditorWindow.py @@ -21,6 +21,7 @@ from idlelib.configHandler import idleConf from idlelib import aboutDialog, textView, configDialog from idlelib import macosxSupport +from idlelib import help # The default tab setting for a Text widget, in average-width characters. TK_TABWIDTH_DEFAULT = 8 @@ -42,6 +43,11 @@ class HelpDialog(object): def __init__(self): + import warnings as w + w.warn("EditorWindow.HelpDialog is no longer used by Idle.\n" + "It will be removed in 3.6 or later.\n" + "It has been replaced by private help.HelpWindow\n", + DeprecationWarning, stacklevel=2) self.parent = None # parent of help window self.dlg = None # the help window iteself @@ -539,11 +545,13 @@ configDialog.ConfigExtensionsDialog(self.top) def help_dialog(self, event=None): + "Handle help doc event." + # edit maxosxSupport.overrideRootMenu.help_dialog to match if self.root: parent = self.root else: parent = self.top - helpDialog.display(parent, near=self.top) + help.show_idlehelp(parent) def python_docs(self, event=None): if sys.platform[:3] == 'win': @@ -1716,4 +1724,4 @@ if __name__ == '__main__': from idlelib.idle_test.htest import run - run(_help_dialog, _editor_window) + run(_editor_window) diff --git a/Lib/idlelib/help.py b/Lib/idlelib/help.py new file mode 100644 --- /dev/null +++ b/Lib/idlelib/help.py @@ -0,0 +1,233 @@ +""" +help.py implements the Idle help menu and is subject to change. + +The contents are subject to revision at any time, without notice. + +Help => About IDLE: diplay About Idle dialog + + + +Help => IDLE Help: display idle.html with proper formatting + +HelpParser - Parses idle.html generated from idle.rst by Sphinx +and renders to tk Text. + +HelpText - Displays formatted idle.html. + +HelpFrame - Contains text, scrollbar, and table-of-contents. +(This will be needed for display in a future tabbed window.) + +HelpWindow - Display idleframe in a standalone window. + +show_idlehelp - Create HelpWindow. Called in EditorWindow.help_dialog. +""" +from html.parser import HTMLParser +from os.path import abspath, dirname, isdir, isfile, join +from tkinter import Tk, Toplevel, Frame, Text, Scrollbar, Menu, Menubutton +from tkinter import font as tkfont + +use_ttk = False # until available to import +if use_ttk: + from tkinter.ttk import Menubutton + +## About IDLE ## + + +## IDLE Help ## + +class HelpParser(HTMLParser): + """Render idle.html generated by Sphinx from idle.rst. + + The overridden handle_xyz methods handle a subset of html tags. + The supplied text should have the needed tag configurations. + The behavior for unsupported tags, such as table, is undefined. + """ + def __init__(self, text): + HTMLParser.__init__(self, convert_charrefs=True) + self.text = text # text widget we're rendering into + self.tags = '' # current text tags to apply + self.show = False # used so we exclude page navigation + self.hdrlink = False # used so we don't show header links + self.level = 0 # indentation level + self.pre = False # displaying preformatted text + self.hprefix = '' # strip e.g. '25.5' from headings + self.nested_dl = False # if we're in a nested
+ self.simplelist = False # simple list (no double spacing) + self.tocid = 1 # id for table of contents entries + self.contents = [] # map toc ids to section titles + self.data = '' # to record data within header tags for toc + + def indent(self, amt=1): + self.level += amt + self.tags = '' if self.level == 0 else 'l'+str(self.level) + + def handle_starttag(self, tag, attrs): + "Handle starttags in idle.html." + class_ = '' + for a, v in attrs: + if a == 'class': + class_ = v + s = '' + if tag == 'div' and class_ == 'section': + self.show = True # start of main content + elif tag == 'div' and class_ == 'sphinxsidebar': + self.show = False # end of main content + elif tag == 'p' and class_ != 'first': + s = '\n\n' + elif tag == 'span' and class_ == 'pre': + self.tags = 'pre' + elif tag == 'span' and class_ == 'versionmodified': + self.tags = 'em' + elif tag == 'em': + self.tags = 'em' + elif tag in ['ul', 'ol']: + if class_.find('simple') != -1: + s = '\n' + self.simplelist = True + else: + self.simplelist = False + self.indent() + elif tag == 'dl': + if self.level > 0: + self.nested_dl = True + elif tag == 'li': + s = '\n* ' if self.simplelist else '\n\n* ' + elif tag == 'dt': + s = '\n\n' if not self.nested_dl else '\n' # avoid extra line + self.nested_dl = False + elif tag == 'dd': + self.indent() + s = '\n' + elif tag == 'pre': + self.pre = True + if self.show: + self.text.insert('end', '\n\n') + self.tags = 'preblock' + elif tag == 'a' and class_ == 'headerlink': + self.hdrlink = True + elif tag == 'h1': + self.text.mark_set('toc'+str(self.tocid), + self.text.index('end-1line')) + self.tags = tag + elif tag in ['h2', 'h3']: + if self.show: + self.data = '' + self.text.mark_set('toc'+str(self.tocid), + self.text.index('end-1line')) + self.text.insert('end', '\n\n') + self.tags = tag + if self.show: + self.text.insert('end', s, self.tags) + + def handle_endtag(self, tag): + "Handle endtags in idle.html." + if tag in ['h1', 'h2', 'h3', 'span', 'em']: + self.indent(0) # clear tag, reset indent + if self.show and tag in ['h1', 'h2', 'h3']: + title = self.data + self.contents.append(('toc'+str(self.tocid), title)) + self.tocid += 1 + elif tag == 'a': + self.hdrlink = False + elif tag == 'pre': + self.pre = False + self.tags = '' + elif tag in ['ul', 'dd', 'ol']: + self.indent(amt=-1) + + def handle_data(self, data): + "Handle date segments in idle.html." + if self.show and not self.hdrlink: + d = data if self.pre else data.replace('\n', ' ') + if self.tags == 'h1': + self.hprefix = d[0:d.index(' ')] + if self.tags in ['h1', 'h2', 'h3'] and self.hprefix != '': + if d[0:len(self.hprefix)] == self.hprefix: + d = d[len(self.hprefix):].strip() + self.data += d + self.text.insert('end', d, self.tags) + + +class HelpText(Text): + "Display idle.html." + def __init__(self, parent, filename): + "Configure tags and feed file to parser." + Text.__init__(self, parent, wrap='word', highlightthickness=0, + padx=5, borderwidth=0) + + normalfont = self.findfont(['TkDefaultFont', 'arial', 'helvetica']) + fixedfont = self.findfont(['TkFixedFont', 'monaco', 'courier']) + self['font'] = (normalfont, 12) + self.tag_configure('em', font=(normalfont, 12, 'italic')) + self.tag_configure('h1', font=(normalfont, 20, 'bold')) + self.tag_configure('h2', font=(normalfont, 18, 'bold')) + self.tag_configure('h3', font=(normalfont, 15, 'bold')) + self.tag_configure('pre', font=(fixedfont, 12)) + self.tag_configure('preblock', font=(fixedfont, 10), lmargin1=25, + borderwidth=1, relief='solid', background='#eeffcc') + self.tag_configure('l1', lmargin1=25, lmargin2=25) + self.tag_configure('l2', lmargin1=50, lmargin2=50) + self.tag_configure('l3', lmargin1=75, lmargin2=75) + self.tag_configure('l4', lmargin1=100, lmargin2=100) + + self.parser = HelpParser(self) + with open(filename, encoding='utf-8') as f: + contents = f.read() + self.parser.feed(contents) + self['state'] = 'disabled' + + def findfont(self, names): + "Return name of first font family derived from names." + for name in names: + if name.lower() in (x.lower() for x in tkfont.names(root=self)): + font = tkfont.Font(name=name, exists=True, root=self) + return font.actual()['family'] + elif name.lower() in (x.lower() + for x in tkfont.families(root=self)): + return name + + +class HelpFrame(Frame): + def __init__(self, parent, filename): + Frame.__init__(self, parent) + text = HelpText(self, filename) + self['background'] = text['background'] + scroll = Scrollbar(self, command=text.yview) + text['yscrollcommand'] = scroll.set + text.grid(column=1, row=0, sticky='nsew') + scroll.grid(column=2, row=0, sticky='ns') + self.grid_columnconfigure(1, weight=1) + self.grid_rowconfigure(0, weight=1) + toc = self.contents_widget(text) + toc.grid(column=0, row=0, sticky='nw') + + def contents_widget(self, text): + toc = Menubutton(self, text='TOC') + drop = Menu(toc, tearoff=False) + for tag, lbl in text.parser.contents: + drop.add_command(label=lbl, command=lambda mark=tag:text.see(mark)) + toc['menu'] = drop + return toc + + +class HelpWindow(Toplevel): + + def __init__(self, parent, filename, title): + Toplevel.__init__(self, parent) + self.wm_title(title) + self.protocol("WM_DELETE_WINDOW", self.destroy) + HelpFrame(self, filename).grid(column=0, row=0, sticky='nsew') + self.grid_columnconfigure(0, weight=1) + self.grid_rowconfigure(0, weight=1) + + +def show_idlehelp(parent): + filename = join(abspath(dirname(__file__)), 'idle.html') + if not isfile(filename): + dirpath = join(abspath(dirname(dirname(dirname(__file__)))), + 'Doc', 'build', 'html', 'library') + HelpWindow(parent, filename, 'IDLE Help') + +if __name__ == '__main__': + from idlelib.idle_test.htest import run + run(show_idlehelp) diff --git a/Lib/idlelib/help.txt b/Lib/idlelib/help.txt --- a/Lib/idlelib/help.txt +++ b/Lib/idlelib/help.txt @@ -1,3 +1,7 @@ +This file, idlelib/help.txt is out-of-date and no longer used by Idle. +It is deprecated and will be removed in the future, possibly in 3.6 +---------------------------------------------------------------------- + [See the end of this file for ** TIPS ** on using IDLE !!] IDLE is the Python IDE built with the tkinter GUI toolkit. diff --git a/Lib/idlelib/idle.html b/Lib/idlelib/idle.html new file mode 100644 --- /dev/null +++ b/Lib/idlelib/idle.html @@ -0,0 +1,671 @@ + + + + + + + + 25.5. IDLE — Python 3.4.3 documentation + + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+
+
+ +
+

25.5. IDLE?

+

IDLE is the Python IDE built with the tkinter GUI toolkit.

+

IDLE has the following features:

+
    +
  • coded in 100% pure Python, using the tkinter GUI toolkit
  • +
  • cross-platform: works on Windows, Unix, and Mac OS X
  • +
  • multi-window text editor with multiple undo, Python colorizing, +smart indent, call tips, and many other features
  • +
  • Python shell window (a.k.a. interactive interpreter)
  • +
  • debugger (not complete, but you can set breakpoints, view and step)
  • +
+ +
+

25.5.2. Editing and navigation?

+

In this section, ‘C’ refers to the Control key on Windows and Unix and +the Command key on Mac OSX.

+
    +
  • Backspace deletes to the left; Del deletes to the right

    +
  • +
  • C-Backspace delete word left; C-Del delete word to the right

    +
  • +
  • Arrow keys and Page Up/Page Down to move around

    +
  • +
  • C-LeftArrow and C-RightArrow moves by words

    +
  • +
  • Home/End go to begin/end of line

    +
  • +
  • C-Home/C-End go to begin/end of file

    +
  • +
  • Some useful Emacs bindings are inherited from Tcl/Tk:

    +
    +
      +
    • C-a beginning of line
    • +
    • C-e end of line
    • +
    • C-k kill line (but doesn’t put it in clipboard)
    • +
    • C-l center window around the insertion point
    • +
    • C-b go backwards one character without deleting (usually you can +also use the cursor key for this)
    • +
    • C-f go forward one character without deleting (usually you can +also use the cursor key for this)
    • +
    • C-p go up one line (usually you can also use the cursor key for +this)
    • +
    • C-d delete next character
    • +
    +
    +
  • +
+

Standard keybindings (like C-c to copy and C-v to paste) +may work. Keybindings are selected in the Configure IDLE dialog.

+
+

25.5.2.1. Automatic indentation?

+

After a block-opening statement, the next line is indented by 4 spaces (in the +Python Shell window by one tab). After certain keywords (break, return etc.) +the next line is dedented. In leading indentation, Backspace deletes up +to 4 spaces if they are there. Tab inserts spaces (in the Python +Shell window one tab), number depends on Indent width. Currently tabs +are restricted to four spaces due to Tcl/Tk limitations.

+

See also the indent/dedent region commands in the edit menu.

+
+
+

25.5.2.2. Completions?

+

Completions are supplied for functions, classes, and attributes of classes, +both built-in and user-defined. Completions are also provided for +filenames.

+

The AutoCompleteWindow (ACW) will open after a predefined delay (default is +two seconds) after a ‘.’ or (in a string) an os.sep is typed. If after one +of those characters (plus zero or more other characters) a tab is typed +the ACW will open immediately if a possible continuation is found.

+

If there is only one possible completion for the characters entered, a +Tab will supply that completion without opening the ACW.

+

‘Show Completions’ will force open a completions window, by default the +C-space will open a completions window. In an empty +string, this will contain the files in the current directory. On a +blank line, it will contain the built-in and user-defined functions and +classes in the current name spaces, plus any modules imported. If some +characters have been entered, the ACW will attempt to be more specific.

+

If a string of characters is typed, the ACW selection will jump to the +entry most closely matching those characters. Entering a tab will +cause the longest non-ambiguous match to be entered in the Editor window or +Shell. Two tab in a row will supply the current ACW selection, as +will return or a double click. Cursor keys, Page Up/Down, mouse selection, +and the scroll wheel all operate on the ACW.

+

“Hidden” attributes can be accessed by typing the beginning of hidden +name after a ‘.’, e.g. ‘_’. This allows access to modules with +__all__ set, or to class-private attributes.

+

Completions and the ‘Expand Word’ facility can save a lot of typing!

+

Completions are currently limited to those in the namespaces. Names in +an Editor window which are not via __main__ and sys.modules will +not be found. Run the module once with your imports to correct this situation. +Note that IDLE itself places quite a few modules in sys.modules, so +much can be found by default, e.g. the re module.

+

If you don’t like the ACW popping up unbidden, simply make the delay +longer or disable the extension. Or another option is the delay could +be set to zero. Another alternative to preventing ACW popups is to +disable the call tips extension.

+
+
+

25.5.2.3. Python Shell window?

+
    +
  • C-c interrupts executing command

    +
  • +
  • C-d sends end-of-file; closes window if typed at a >>> prompt

    +
  • +
  • Alt-/ (Expand word) is also useful to reduce typing

    +

    Command history

    +
      +
    • Alt-p retrieves previous command matching what you have typed. On +OS X use C-p.
    • +
    • Alt-n retrieves next. On OS X use C-n.
    • +
    • Return while on any previous command retrieves that command
    • +
    +
  • +
+
+
+
+

25.5.3. Syntax colors?

+

The coloring is applied in a background “thread,” so you may occasionally see +uncolorized text. To change the color scheme, edit the [Colors] section in +config.txt.

+
+
Python syntax colors:
+
+
Keywords
+
orange
+
Strings
+
green
+
Comments
+
red
+
Definitions
+
blue
+
+
+
Shell colors:
+
+
Console output
+
brown
+
stdout
+
blue
+
stderr
+
dark green
+
stdin
+
black
+
+
+
+
+
+

25.5.4. Startup?

+

Upon startup with the -s option, IDLE will execute the file referenced by +the environment variables IDLESTARTUP or PYTHONSTARTUP. +IDLE first checks for IDLESTARTUP; if IDLESTARTUP is present the file +referenced is run. If IDLESTARTUP is not present, IDLE checks for +PYTHONSTARTUP. Files referenced by these environment variables are +convenient places to store functions that are used frequently from the IDLE +shell, or for executing import statements to import common modules.

+

In addition, Tk also loads a startup file if it is present. Note that the +Tk file is loaded unconditionally. This additional file is .Idle.py and is +looked for in the user’s home directory. Statements in this file will be +executed in the Tk namespace, so this file is not useful for importing +functions to be used from IDLE’s Python shell.

+
+

25.5.4.1. Command line usage?

+
idle.py [-c command] [-d] [-e] [-s] [-t title] [arg] ...
+
+-c command  run this command
+-d          enable debugger
+-e          edit mode; arguments are files to be edited
+-s          run $IDLESTARTUP or $PYTHONSTARTUP first
+-t title    set title of shell window
+
+
+

If there are arguments:

+
    +
  1. If -e is used, arguments are files opened for editing and +sys.argv reflects the arguments passed to IDLE itself.
  2. +
  3. Otherwise, if -c is used, all arguments are placed in +sys.argv[1:...], with sys.argv[0] set to '-c'.
  4. +
  5. Otherwise, if neither -e nor -c is used, the first +argument is a script which is executed with the remaining arguments in +sys.argv[1:...] and sys.argv[0] set to the script name. If the +script name is ‘-‘, no script is executed but an interactive Python session +is started; the arguments are still available in sys.argv.
  6. +
+
+
+

25.5.4.2. Running without a subprocess?

+

If IDLE is started with the -n command line switch it will run in a +single process and will not create the subprocess which runs the RPC +Python execution server. This can be useful if Python cannot create +the subprocess or the RPC socket interface on your platform. However, +in this mode user code is not isolated from IDLE itself. Also, the +environment is not restarted when Run/Run Module (F5) is selected. If +your code has been modified, you must reload() the affected modules and +re-import any specific items (e.g. from foo import baz) if the changes +are to take effect. For these reasons, it is preferable to run IDLE +with the default subprocess if at all possible.

+
+

Deprecated since version 3.4.

+
+
+
+
+

25.5.5. Help and preferences?

+
+

25.5.5.1. Additional help sources?

+

IDLE includes a help menu entry called “Python Docs” that will open the +extensive sources of help, including tutorials, available at docs.python.org. +Selected URLs can be added or removed from the help menu at any time using the +Configure IDLE dialog. See the IDLE help option in the help menu of IDLE for +more information.

+
+
+

25.5.5.2. Setting preferences?

+

The font preferences, highlighting, keys, and general preferences can be +changed via Configure IDLE on the Option menu. Keys can be user defined; +IDLE ships with four built in key sets. In addition a user can create a +custom key set in the Configure IDLE dialog under the keys tab.

+
+
+

25.5.5.3. Extensions?

+

IDLE contains an extension facility. Peferences for extensions can be +changed with Configure Extensions. See the beginning of config-extensions.def +in the idlelib directory for further information. The default extensions +are currently:

+
    +
  • FormatParagraph
  • +
  • AutoExpand
  • +
  • ZoomHeight
  • +
  • ScriptBinding
  • +
  • CallTips
  • +
  • ParenMatch
  • +
  • AutoComplete
  • +
  • CodeContext
  • +
  • RstripExtension
  • +
+
+
+
+ + +
+
+
+ +
+
+ + + + + \ No newline at end of file diff --git a/Lib/idlelib/idle_test/htest.py b/Lib/idlelib/idle_test/htest.py --- a/Lib/idlelib/idle_test/htest.py +++ b/Lib/idlelib/idle_test/htest.py @@ -194,13 +194,6 @@ "should open that file \nin a new EditorWindow." } -_help_dialog_spec = { - 'file': 'EditorWindow', - 'kwds': {}, - 'msg': "If the help text displays, this works.\n" - "Text is selectable. Window is scrollable." - } - _io_binding_spec = { 'file': 'IOBinding', 'kwds': {}, @@ -279,6 +272,13 @@ "Right clicking an item will display a popup." } +show_idlehelp_spec = { + 'file': 'help', + 'kwds': {}, + 'msg': "If the help text displays, this works.\n" + "Text is selectable. Window is scrollable." + } + _stack_viewer_spec = { 'file': 'StackViewer', 'kwds': {}, diff --git a/Lib/idlelib/macosxSupport.py b/Lib/idlelib/macosxSupport.py --- a/Lib/idlelib/macosxSupport.py +++ b/Lib/idlelib/macosxSupport.py @@ -174,9 +174,8 @@ configDialog.ConfigDialog(root, 'Settings') def help_dialog(event=None): - from idlelib import textView - fn = path.join(path.abspath(path.dirname(__file__)), 'help.txt') - textView.view_file(root, 'Help', fn) + from idlelib import help + help.show_idlehelp(root) root.bind('<>', about_dialog) root.bind('<>', config_dialog) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 21 02:06:15 2015 From: python-checkins at python.org (terry.reedy) Date: Mon, 21 Sep 2015 00:06:15 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzE2ODkz?= =?utf-8?q?=3A_whitespace_in_help=2Epy=2E?= Message-ID: <20150921000615.9941.39650@psf.io> https://hg.python.org/cpython/rev/1107e3ee6103 changeset: 98099:1107e3ee6103 branch: 2.7 parent: 98095:e66fbfa282c6 user: Terry Jan Reedy date: Sun Sep 20 20:05:51 2015 -0400 summary: Issue #16893: whitespace in help.py. files: Lib/idlelib/help.py | 6 +++--- 1 files changed, 3 insertions(+), 3 deletions(-) diff --git a/Lib/idlelib/help.py b/Lib/idlelib/help.py --- a/Lib/idlelib/help.py +++ b/Lib/idlelib/help.py @@ -157,7 +157,7 @@ "Configure tags and feed file to parser." Text.__init__(self, parent, wrap='word', highlightthickness=0, padx=5, borderwidth=0) - + normalfont = self.findfont(['TkDefaultFont', 'arial', 'helvetica']) fixedfont = self.findfont(['TkFixedFont', 'monaco', 'courier']) self['font'] = (normalfont, 12) @@ -173,7 +173,7 @@ self.tag_configure('l3', lmargin1=75, lmargin2=75) self.tag_configure('l4', lmargin1=100, lmargin2=100) - self.parser = HelpParser(self) + self.parser = HelpParser(self) with open(filename) as f: contents = f.read().decode(encoding='utf-8') self.parser.feed(contents) @@ -203,7 +203,7 @@ self.grid_rowconfigure(0, weight=1) toc = self.contents_widget(text) toc.grid(column=0, row=0, sticky='nw') - + def contents_widget(self, text): toc = Menubutton(self, text='TOC') drop = Menu(toc, tearoff=False) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 21 02:06:15 2015 From: python-checkins at python.org (terry.reedy) Date: Mon, 21 Sep 2015 00:06:15 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzE2ODkz?= =?utf-8?q?=3A_whitespace_in_idle=2Ehtml=2E?= Message-ID: <20150921000614.94129.23018@psf.io> https://hg.python.org/cpython/rev/9b79a4901069 changeset: 98096:9b79a4901069 branch: 3.4 parent: 98092:2d808b72996d user: Terry Jan Reedy date: Sun Sep 20 20:03:01 2015 -0400 summary: Issue #16893: whitespace in idle.html. files: Lib/idlelib/idle.html | 28 ++++++++++++++-------------- 1 files changed, 14 insertions(+), 14 deletions(-) diff --git a/Lib/idlelib/idle.html b/Lib/idlelib/idle.html --- a/Lib/idlelib/idle.html +++ b/Lib/idlelib/idle.html @@ -5,12 +5,12 @@ - + 25.5. IDLE — Python 3.4.3 documentation - + - + - - - + + + - + +
- +

25.5. IDLE?

IDLE is the Python IDE built with the tkinter GUI toolkit.

@@ -628,7 +628,7 @@
-
+
+
- \ No newline at end of file + -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 21 02:06:16 2015 From: python-checkins at python.org (terry.reedy) Date: Mon, 21 Sep 2015 00:06:16 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge_with_3=2E5?= Message-ID: <20150921000615.11698.45112@psf.io> https://hg.python.org/cpython/rev/c0e9ca4254ef changeset: 98098:c0e9ca4254ef parent: 98094:9e8c3be5b5db parent: 98097:f4504ff121cc user: Terry Jan Reedy date: Sun Sep 20 20:03:37 2015 -0400 summary: Merge with 3.5 files: Lib/idlelib/idle.html | 28 ++++++++++++++-------------- 1 files changed, 14 insertions(+), 14 deletions(-) diff --git a/Lib/idlelib/idle.html b/Lib/idlelib/idle.html --- a/Lib/idlelib/idle.html +++ b/Lib/idlelib/idle.html @@ -5,12 +5,12 @@ - + 25.5. IDLE — Python 3.4.3 documentation - + - + - - - + + + - + +
- +

25.5. IDLE?

IDLE is the Python IDE built with the tkinter GUI toolkit.

@@ -628,7 +628,7 @@
-
+
+
- \ No newline at end of file + -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 21 02:06:17 2015 From: python-checkins at python.org (terry.reedy) Date: Mon, 21 Sep 2015 00:06:17 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Merge_with_3=2E4?= Message-ID: <20150921000614.82660.60689@psf.io> https://hg.python.org/cpython/rev/f4504ff121cc changeset: 98097:f4504ff121cc branch: 3.5 parent: 98093:605dffe0972c parent: 98096:9b79a4901069 user: Terry Jan Reedy date: Sun Sep 20 20:03:22 2015 -0400 summary: Merge with 3.4 files: Lib/idlelib/idle.html | 28 ++++++++++++++-------------- 1 files changed, 14 insertions(+), 14 deletions(-) diff --git a/Lib/idlelib/idle.html b/Lib/idlelib/idle.html --- a/Lib/idlelib/idle.html +++ b/Lib/idlelib/idle.html @@ -5,12 +5,12 @@ - + 25.5. IDLE — Python 3.4.3 documentation - + - + - - - + + + - + +
- +

25.5. IDLE?

IDLE is the Python IDE built with the tkinter GUI toolkit.

@@ -628,7 +628,7 @@
-
+
+
- \ No newline at end of file + -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 21 02:25:55 2015 From: python-checkins at python.org (alexander.belopolsky) Date: Mon, 21 Sep 2015 00:25:55 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_PEP_495=3A_Added_a_gap_sketch?= =?utf-8?q?=2E?= Message-ID: <20150921002555.98376.5355@psf.io> https://hg.python.org/peps/rev/e6c82824b10f changeset: 6077:e6c82824b10f parent: 6055:5967673690e8 user: Alexander Belopolsky date: Sun Sep 20 20:25:12 2015 -0400 summary: PEP 495: Added a gap sketch. files: pep-0495-fold.png | Bin pep-0495-gap.png | Bin pep-0495-gap.svg | 437 ++++++++++++++++++++++++++++++++++ pep-0495.txt | 12 +- 4 files changed, 445 insertions(+), 4 deletions(-) diff --git a/pep-0495-fold.png b/pep-0495-fold.png index d9fe8b6eeb680cf113a3097a6c93342a418ec087..d09eb41f721827d2e8fd1f748707fd66a8c13273 GIT binary patch [stripped] diff --git a/pep-0495-gap.png b/pep-0495-gap.png new file mode 100644 index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..e3ba3cb77e128bc593a501267500246b6ed77a54 GIT binary patch [stripped] diff --git a/pep-0495-gap.svg b/pep-0495-gap.svg new file mode 100644 --- /dev/null +++ b/pep-0495-gap.svg @@ -0,0 +1,437 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + image/svg+xml + + + + + + + + UTC local + + + + t + u0 + u1 + + + + Fold + + + + + + + + + + + diff --git a/pep-0495.txt b/pep-0495.txt --- a/pep-0495.txt +++ b/pep-0495.txt @@ -93,10 +93,6 @@ this PEP specifies how various functions should behave when given an invalid instance. -.. image:: pep-0495-fold.png - :align: center - :width: 60% - Affected APIs ------------- @@ -347,6 +343,10 @@ ``fromutc(u2)`` will return an instance with ``fold=1``. In all other cases the returned instance should have ``fold=0``. +.. image:: pep-0495-fold.png + :align: center + :width: 60% + On an ambiguous time introduced at the end of DST, the values returned by ``utcoffset()`` and ``dst()`` methods should be as follows @@ -366,6 +366,10 @@ Mind the DST Gap ---------------- +.. image:: pep-0495-gap.png + :align: center + :width: 60% + On a missing time introduced at the start of DST, the values returned by ``utcoffset()`` and ``dst()`` methods should be as follows -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Mon Sep 21 02:25:56 2015 From: python-checkins at python.org (alexander.belopolsky) Date: Mon, 21 Sep 2015 00:25:56 +0000 Subject: [Python-checkins] =?utf-8?q?peps_=28merge_default_-=3E_default=29?= =?utf-8?q?=3A_merge?= Message-ID: <20150921002555.81633.87894@psf.io> https://hg.python.org/peps/rev/bf2d4bda51bc changeset: 6078:bf2d4bda51bc parent: 6077:e6c82824b10f parent: 6076:3042d4b36030 user: Alexander Belopolsky date: Sun Sep 20 20:25:44 2015 -0400 summary: merge files: pep-0101.txt | 15 +- pep-0103.txt | 951 +++++++++++++++++++++++++++++++++++++++ pep-0478.txt | 2 +- pep-0495.txt | 13 + pep-0498.txt | 19 +- pep-0502.txt | 453 ++++++----------- pep-0504.txt | 396 ++++++++++++++++ pep-0505.txt | 205 ++++++++ pep-0506.txt | 356 ++++++++++++++ pep-3140.txt | 2 +- 10 files changed, 2108 insertions(+), 304 deletions(-) diff --git a/pep-0101.txt b/pep-0101.txt --- a/pep-0101.txt +++ b/pep-0101.txt @@ -424,11 +424,12 @@ that directory. Note though that if you're releasing a maintenance release for an older version, don't change the current link. - ___ If this is a final release (even a maintenance release), also unpack - the HTML docs to /srv/docs.python.org/release/X.Y.Z on - docs.iad1.psf.io. Make sure the files are in group "docs". If it is a - release of a security-fix-only version, tell the DE to build a version - with the "version switcher" and put it there. + ___ If this is a final release (even a maintenance release), also + unpack the HTML docs to /srv/docs.python.org/release/X.Y.Z on + docs.iad1.psf.io. Make sure the files are in group "docs" and are + group-writeable. If it is a release of a security-fix-only version, + tell the DE to build a version with the "version switcher" + and put it there. ___ Let the DE check if the docs are built and work all right. @@ -484,6 +485,10 @@ Note that the easiest thing is probably to copy fields from an existing Python release "page", editing as you go. + There should only be one "page" for a release (e.g. 3.5.0, 3.5.1). + Reuse the same page for all pre-releases, changing the version + number and the documentation as you go. + ___ If this isn't the first release for a version, open the existing "page" for editing and update it to the new release. Don't save yet! diff --git a/pep-0103.txt b/pep-0103.txt new file mode 100644 --- /dev/null +++ b/pep-0103.txt @@ -0,0 +1,951 @@ +PEP: 103 +Title: Collecting information about git +Version: $Revision$ +Last-Modified: $Date$ +Author: Oleg Broytman +Status: Draft +Type: Informational +Content-Type: text/x-rst +Created: 01-Jun-2015 +Post-History: 12-Sep-2015 + +Abstract +======== + +This Informational PEP collects information about git. There is, of +course, a lot of documentation for git, so the PEP concentrates on +more complex (and more related to Python development) issues, +scenarios and examples. + +The plan is to extend the PEP in the future collecting information +about equivalence of Mercurial and git scenarios to help migrating +Python development from Mercurial to git. + +The author of the PEP doesn't currently plan to write a Process PEP on +migration Python development from Mercurial to git. + + +Documentation +============= + +Git is accompanied with a lot of documentation, both online and +offline. + + +Documentation for starters +-------------------------- + +Git Tutorial: `part 1 +`_, +`part 2 +`_. + +`Git User's manual +`_. +`Everyday GIT With 20 Commands Or So +`_. +`Git workflows +`_. + + +Advanced documentation +---------------------- + +`Git Magic +`_, +with a number of translations. + +`Pro Git `_. The Book about git. Buy it at +Amazon or download in PDF, mobi, or ePub form. It has translations to +many different languages. Download Russian translation from `GArik +`_. + +`Git Wiki `_. + + +Offline documentation +--------------------- + +Git has builtin help: run ``git help $TOPIC``. For example, run +``git help git`` or ``git help help``. + + +Quick start +=========== + +Download and installation +------------------------- + +Unix users: `download and install using your package manager +`_. + +Microsoft Windows: download `git-for-windows +`_ or `msysGit +`_. + +MacOS X: use git installed with `XCode +`_ or download from +`MacPorts `_ or +`git-osx-installer +`_ or +install git with `Homebrew `_: ``brew install git``. + +`git-cola `_ is a Git GUI +written in Python and GPL licensed. Linux, Windows, MacOS X. + +`TortoiseGit `_ is a Windows Shell Interface +to Git based on TortoiseSVN; open source. + + +Initial configuration +--------------------- + +This simple code is often appears in documentation, but it is +important so let repeat it here. Git stores author and committer +names/emails in every commit, so configure your real name and +preferred email:: + + $ git config --global user.name "User Name" + $ git config --global user.email user.name at example.org + + +Examples in this PEP +==================== + +Examples of git commands in this PEP use the following approach. It is +supposed that you, the user, works with a local repository named +``python`` that has an upstream remote repo named ``origin``. Your +local repo has two branches ``v1`` and ``master``. For most examples +the currently checked out branch is ``master``. That is, it's assumed +you have done something like that:: + + $ git clone https://git.python.org/python.git + $ cd python + $ git branch v1 origin/v1 + +The first command clones remote repository into local directory +`python``, creates a new local branch master, sets +remotes/origin/master as its upstream remote-tracking branch and +checks it out into the working directory. + +The last command creates a new local branch v1 and sets +remotes/origin/v1 as its upstream remote-tracking branch. + +The same result can be achieved with commands:: + + $ git clone -b v1 https://git.python.org/python.git + $ cd python + $ git checkout --track origin/master + +The last command creates a new local branch master, sets +remotes/origin/master as its upstream remote-tracking branch and +checks it out into the working directory. + + +Branches and branches +===================== + +Git terminology can be a bit misleading. Take, for example, the term +"branch". In git it has two meanings. A branch is a directed line of +commits (possibly with merges). And a branch is a label or a pointer +assigned to a line of commits. It is important to distinguish when you +talk about commits and when about their labels. Lines of commits are +by itself unnamed and are usually only lengthening and merging. +Labels, on the other hand, can be created, moved, renamed and deleted +freely. + + +Remote repositories and remote branches +======================================= + +Remote-tracking branches are branches (pointers to commits) in your +local repository. They are there for git (and for you) to remember +what branches and commits have been pulled from and pushed to what +remote repos (you can pull from and push to many remotes). +Remote-tracking branches live under ``remotes/$REMOTE`` namespaces, +e.g. ``remotes/origin/master``. + +To see the status of remote-tracking branches run:: + + $ git branch -rv + +To see local and remote-tracking branches (and tags) pointing to +commits:: + + $ git log --decorate + +You never do your own development on remote-tracking branches. You +create a local branch that has a remote branch as upstream and do +development on that local branch. On push git pushes commits to the +remote repo and updates remote-tracking branches, on pull git fetches +commits from the remote repo, updates remote-tracking branches and +fast-forwards, merges or rebases local branches. + +When you do an initial clone like this:: + + $ git clone -b v1 https://git.python.org/python.git + +git clones remote repository ``https://git.python.org/python.git`` to +directory ``python``, creates a remote named ``origin``, creates +remote-tracking branches, creates a local branch ``v1``, configure it +to track upstream remotes/origin/v1 branch and checks out ``v1`` into +the working directory. + + +Updating local and remote-tracking branches +------------------------------------------- + +There is a major difference between + +:: + + $ git fetch $REMOTE $BRANCH + +and + +:: + + $ git fetch $REMOTE $BRANCH:$BRANCH + +The first command fetches commits from the named $BRANCH in the +$REMOTE repository that are not in your repository, updates +remote-tracking branch and leaves the id (the hash) of the head commit +in file .git/FETCH_HEAD. + +The second command fetches commits from the named $BRANCH in the +$REMOTE repository that are not in your repository and updates both +the local branch $BRANCH and its upstream remote-tracking branch. But +it refuses to update branches in case of non-fast-forward. And it +refuses to update the current branch (currently checked out branch, +where HEAD is pointing to). + +The first command is used internally by ``git pull``. + +:: + + $ git pull $REMOTE $BRANCH + +is equivalent to + +:: + + $ git fetch $REMOTE $BRANCH + $ git merge FETCH_HEAD + +Certainly, $BRANCH in that case should be your current branch. If you +want to merge a different branch into your current branch first update +that non-current branch and then merge:: + + $ git fetch origin v1:v1 # Update v1 + $ git pull --rebase origin master # Update the current branch master + # using rebase instead of merge + $ git merge v1 + +If you have not yet pushed commits on ``v1``, though, the scenario has +to become a bit more complex. Git refuses to update +non-fast-forwardable branch, and you don't want to do force-pull +because that would remove your non-pushed commits and you would need +to recover. So you want to rebase ``v1`` but you cannot rebase +non-current branch. Hence, checkout ``v1`` and rebase it before +merging:: + + $ git checkout v1 + $ git pull --rebase origin v1 + $ git checkout master + $ git pull --rebase origin master + $ git merge v1 + +It is possible to configure git to make it fetch/pull a few branches +or all branches at once, so you can simply run + +:: + + $ git pull origin + +or even + +:: + + $ git pull + +Default remote repository for fetching/pulling is ``origin``. Default +set of references to fetch is calculated using matching algorithm: git +fetches all branches having the same name on both ends. + + +Push +'''' + +Pushing is a bit simpler. There is only one command ``push``. When you +run + +:: + + $ git push origin v1 master + +git pushes local v1 to remote v1 and local master to remote master. +The same as:: + + $ git push origin v1:v1 master:master + +Git pushes commits to the remote repo and updates remote-tracking +branches. Git refuses to push commits that aren't fast-forwardable. +You can force-push anyway, but please remember - you can force-push to +your own repositories but don't force-push to public or shared repos. +If you find git refuses to push commits that aren't fast-forwardable, +better fetch and merge commits from the remote repo (or rebase your +commits on top of the fetched commits), then push. Only force-push if +you know what you do and why you do it. See the section `Commit +editing and caveats`_ below. + +It is possible to configure git to make it push a few branches or all +branches at once, so you can simply run + +:: + + $ git push origin + +or even + +:: + + $ git push + +Default remote repository for pushing is ``origin``. Default set of +references to push in git before 2.0 is calculated using matching +algorithm: git pushes all branches having the same name on both ends. +Default set of references to push in git 2.0+ is calculated using +simple algorithm: git pushes the current branch back to its +@{upstream}. + +To configure git before 2.0 to the new behaviour run:: + +$ git config push.default simple + +To configure git 2.0+ to the old behaviour run:: + +$ git config push.default matching + +Git doesn't allow to push a branch if it's the current branch in the +remote non-bare repository: git refuses to update remote working +directory. You really should push only to bare repositories. For +non-bare repositories git prefers pull-based workflow. + +When you want to deploy code on a remote host and can only use push +(because your workstation is behind a firewall and you cannot pull +from it) you do that in two steps using two repositories: you push +from the workstation to a bare repo on the remote host, ssh to the +remote host and pull from the bare repo to a non-bare deployment repo. + +That changed in git 2.3, but see `the blog post +`_ +for caveats; in 2.4 the push-to-deploy feature was `further improved +`_. + + +Tags +'''' + +Git automatically fetches tags that point to commits being fetched +during fetch/pull. To fetch all tags (and commits they point to) run +``git fetch --tags origin``. To fetch some specific tags fetch them +explicitly:: + + $ git fetch origin tag $TAG1 tag $TAG2... + +For example:: + + $ git fetch origin tag 1.4.2 + $ git fetch origin v1:v1 tag 2.1.7 + +Git doesn't automatically pushes tags. That allows you to have private +tags. To push tags list them explicitly:: + + $ git push origin tag 1.4.2 + $ git push origin v1 master tag 2.1.7 + +Or push all tags at once:: + + $ git push --tags origin + +Don't move tags with ``git tag -f`` or remove tags with ``git tag -d`` +after they have been published. + + +Private information +''''''''''''''''''' + +When cloning/fetching/pulling/pushing git copies only database objects +(commits, trees, files and tags) and symbolic references (branches and +lightweight tags). Everything else is private to the repository and +never cloned, updated or pushed. It's your config, your hooks, your +private exclude file. + +If you want to distribute hooks, copy them to the working tree, add, +commit, push and instruct the team to update and install the hooks +manually. + + +Commit editing and caveats +========================== + +A warning not to edit published (pushed) commits also appears in +documentation but it's repeated here anyway as it's very important. + +It is possible to recover from a forced push but it's PITA for the +entire team. Please avoid it. + +To see what commits have not been published yet compare the head of the +branch with its upstream remote-tracking branch:: + + $ git log origin/master.. # from origin/master to HEAD (of master) + $ git log origin/v1..v1 # from origin/v1 to the head of v1 + +For every branch that has an upstream remote-tracking branch git +maintains an alias @{upstream} (short version @{u}), so the commands +above can be given as:: + + $ git log @{u}.. + $ git log v1@{u}..v1 + +To see the status of all branches:: + + $ git branch -avv + +To compare the status of local branches with a remote repo:: + + $ git remote show origin + +Read `how to recover from upstream rebase +`_. +It is in ``git help rebase``. + +On the other hand don't be too afraid about commit editing. You can +safely edit, reorder, remove, combine and split commits that haven't +been pushed yet. You can even push commits to your own (backup) repo, +edit them later and force-push edited commits to replace what have +already been pushed. Not a problem until commits are in a public +or shared repository. + + +Undo +==== + +Whatever you do, don't panic. Almost anything in git can be undone. + + +git checkout: restore file's content +------------------------------------ + +``git checkout``, for example, can be used to restore the content of +file(s) to that one of a commit. Like this:: + + git checkout HEAD~ README + +The commands restores the contents of README file to the last but one +commit in the current branch. By default the commit ID is simply HEAD; +i.e. ``git checkout README`` restores README to the latest commit. + +(Do not use ``git checkout`` to view a content of a file in a commit, +use ``git cat-file -p``; e.g. ``git cat-file -p HEAD~:path/to/README``). + + +git reset: remove (non-pushed) commits +-------------------------------------- + +``git reset`` moves the head of the current branch. The head can be +moved to point to any commit but it's often used to remove a commit or +a few (preferably, non-pushed ones) from the top of the branch - that +is, to move the branch backward in order to undo a few (non-pushed) +commits. + +``git reset`` has three modes of operation - soft, hard and mixed. +Default is mixed. ProGit `explains +`_ the +difference very clearly. Bare repositories don't have indices or +working trees so in a bare repo only soft reset is possible. + + +Unstaging +''''''''' + +Mixed mode reset with a path or paths can be used to unstage changes - +that is, to remove from index changes added with ``git add`` for +committing. See `The Book +`_ for details +about unstaging and other undo tricks. + + +git reflog: reference log +------------------------- + +Removing commits with ``git reset`` or moving the head of a branch +sounds dangerous and it is. But there is a way to undo: another +reset back to the original commit. Git doesn't remove commits +immediately; unreferenced commits (in git terminology they are called +"dangling commits") stay in the database for some time (default is two +weeks) so you can reset back to it or create a new branch pointing to +the original commit. + +For every move of a branch's head - with ``git commit``, ``git +checkout``, ``git fetch``, ``git pull``, ``git rebase``, ``git reset`` +and so on - git stores a reference log (reflog for short). For every +move git stores where the head was. Command ``git reflog`` can be used +to view (and manipulate) the log. + +In addition to the moves of the head of every branch git stores the +moves of the HEAD - a symbolic reference that (usually) names the +current branch. HEAD is changed with ``git checkout $BRANCH``. + +By default ``git reflog`` shows the moves of the HEAD, i.e. the +command is equivalent to ``git reflog HEAD``. To show the moves of the +head of a branch use the command ``git reflog $BRANCH``. + +So to undo a ``git reset`` lookup the original commit in ``git +reflog``, verify it with ``git show`` or ``git log`` and run ``git +reset $COMMIT_ID``. Git stores the move of the branch's head in +reflog, so you can undo that undo later again. + +In a more complex situation you'd want to move some commits along with +resetting the head of the branch. Cherry-pick them to the new branch. +For example, if you want to reset the branch ``master`` back to the +original commit but preserve two commits created in the current branch +do something like:: + + $ git branch save-master # create a new branch saving master + $ git reflog # find the original place of master + $ git reset $COMMIT_ID + $ git cherry-pick save-master~ save-master + $ git branch -D save-master # remove temporary branch + + +git revert: revert a commit +--------------------------- + +``git revert`` reverts a commit or commits, that is, it creates a new +commit or commits that revert(s) the effects of the given commits. +It's the only way to undo published commits (``git commit --amend``, +``git rebase`` and ``git reset`` change the branch in +non-fast-forwardable ways so they should only be used for non-pushed +commits.) + +There is a problem with reverting a merge commit. ``git revert`` can +undo the code created by the merge commit but it cannot undo the fact +of merge. See the discussion `How to revert a faulty merge +`_. + + +One thing that cannot be undone +------------------------------- + +Whatever you undo, there is one thing that cannot be undone - +overwritten uncommitted changes. Uncommitted changes don't belong to +git so git cannot help preserving them. + +Most of the time git warns you when you're going to execute a command +that overwrites uncommitted changes. Git doesn't allow you to switch +branches with ``git checkout``. It stops you when you're going to +rebase with non-clean working tree. It refuses to pull new commits +over non-committed files. + +But there are commands that do exactly that - overwrite files in the +working tree. Commands like ``git checkout $PATHs`` or ``git reset +--hard`` silently overwrite files including your uncommitted changes. + +With that in mind you can understand the stance "commit early, commit +often". Commit as often as possible. Commit on every save in your +editor or IDE. You can edit your commits before pushing - edit commit +messages, change commits, reorder, combine, split, remove. But save +your changes in git database, either commit changes or at least stash +them with ``git stash``. + + +Merge or rebase? +================ + +Internet is full of heated discussions on the topic: "merge or +rebase?" Most of them are meaningless. When a DVCS is being used in a +big team with a big and complex project with many branches there is +simply no way to avoid merges. So the question's diminished to +"whether to use rebase, and if yes - when to use rebase?" Considering +that it is very much recommended not to rebase published commits the +question's diminished even further: "whether to use rebase on +non-pushed commits?" + +That small question is for the team to decide. The author of the PEP +recommends to use rebase when pulling, i.e. always do ``git pull +--rebase`` or even configure automatic setup of rebase for every new +branch:: + + $ git config branch.autosetuprebase always + +and configure rebase for existing branches:: + + $ git config branch.$NAME.rebase true + +For example:: + + $ git config branch.v1.rebase true + $ git config branch.master.rebase true + +After that ``git pull origin master`` becomes equivalent to ``git pull +--rebase origin master``. + +It is recommended to create new commits in a separate feature or topic +branch while using rebase to update the mainline branch. When the +topic branch is ready merge it into mainline. To avoid a tedious task +of resolving large number of conflicts at once you can merge the topic +branch to the mainline from time to time and switch back to the topic +branch to continue working on it. The entire workflow would be +something like:: + + $ git checkout -b issue-42 # create a new issue branch and switch to it + ...edit/test/commit... + $ git checkout master + $ git pull --rebase origin master # update master from the upstream + $ git merge issue-42 + $ git branch -d issue-42 # delete the topic branch + $ git push origin master + +When the topic branch is deleted only the label is removed, commits +are stayed in the database, they are now merged into master:: + + o--o--o--o--o--M--< master - the mainline branch + \ / + --*--*--* - the topic branch, now unnamed + +The topic branch is deleted to avoid cluttering branch namespace with +small topic branches. Information on what issue was fixed or what +feature was implemented should be in the commit messages. + + +Null-merges +=========== + +Git has a builtin merge strategy for what Python core developers call +"null-merge":: + + $ git merge -s ours v1 # null-merge v1 into master + + +Branching models +================ + +Git doesn't assume any particular development model regarding +branching and merging. Some projects prefer to graduate patches from +the oldest branch to the newest, some prefer to cherry-pick commits +backwards, some use squashing (combining a number of commits into +one). Anything is possible. + +There are a few examples to start with. `git help workflows +`_ +describes how the very git authors develop git. + +ProGit book has a few chapters devoted to branch management in +different projects: `Git Branching - Branching Workflows +`_ and +`Distributed Git - Contributing to a Project +`_. + +There is also a well-known article `A successful Git branching model +`_ by Vincent +Driessen. It recommends a set of very detailed rules on creating and +managing mainline, topic and bugfix branches. To support the model the +author implemented `git flow `_ +extension. + + +Advanced configuration +====================== + +Line endings +------------ + +Git has builtin mechanisms to handle line endings between platforms +with different end-of-line styles. To allow git to do CRLF conversion +assign ``text`` attribute to files using `.gitattributes +`_. +For files that have to have specific line endings assign ``eol`` +attribute. For binary files the attribute is, naturally, ``binary``. + +For example:: + + $ cat .gitattributes + *.py text + *.txt text + *.png binary + /readme.txt eol=CRLF + +To check what attributes git uses for files use ``git check-attr`` +command. For example:: + +$ git check-attr -a -- \*.py + + +Advanced topics +=============== + +Staging area +------------ + +Staging area aka index aka cache is a distinguishing feature of git. +Staging area is where git collects patches before committing them. +Separation between collecting patches and commit phases provides a +very useful feature of git: you can review collected patches before +commit and even edit them - remove some hunks, add new hunks and +review again. + +To add files to the index use ``git add``. Collecting patches before +committing means you need to do that for every change, not only to add +new (untracked) files. To simplify committing in case you just want to +commit everything without reviewing run ``git commit --all`` (or just +``-a``) - the command adds every changed tracked file to the index and +then commit. To commit a file or files regardless of patches collected +in the index run ``git commit [--only|-o] -- $FILE...``. + +To add hunks of patches to the index use ``git add --patch`` (or just +``-p``). To remove collected files from the index use ``git reset HEAD +-- $FILE...`` To add/inspect/remove collected hunks use ``git add +--interactive`` (``-i``). + +To see the diff between the index and the last commit (i.e., collected +patches) use ``git diff --cached``. To see the diff between the +working tree and the index (i.e., uncollected patches) use just ``git +diff``. To see the diff between the working tree and the last commit +(i.e., both collected and uncollected patches) run ``git diff HEAD``. + +See `WhatIsTheIndex +`_ and +`IndexCommandQuickref +`_ in Git +Wiki. + + +ReReRe +====== + +Rerere is a mechanism that helps to resolve repeated merge conflicts. +The most frequent source of recurring merge conflicts are topic +branches that are merged into mainline and then the merge commits are +removed; that's often performed to test the topic branches and train +rerere; merge commits are removed to have clean linear history and +finish the topic branch with only one last merge commit. + +Rerere works by remembering the states of tree before and after a +successful commit. That way rerere can automatically resolve conflicts +if they appear in the same files. + +Rerere can be used manually with ``git rerere`` command but most often +it's used automatically. Enable rerere with these commands in a +working tree:: + + $ git config rerere.enabled true + $ git config rerere.autoupdate true + +You don't need to turn rerere on globally - you don't want rerere in +bare repositories or single-branche repositories; you only need rerere +in repos where you often perform merges and resolve merge conflicts. + +See `Rerere `_ in The +Book. + + +Database maintenance +==================== + +Git object database and other files/directories under ``.git`` require +periodic maintenance and cleanup. For example, commit editing left +unreferenced objects (dangling objects, in git terminology) and these +objects should be pruned to avoid collecting cruft in the DB. The +command ``git gc`` is used for maintenance. Git automatically runs +``git gc --auto`` as a part of some commands to do quick maintenance. +Users are recommended to run ``git gc --aggressive`` from time to +time; ``git help gc`` recommends to run it every few hundred +changesets; for more intensive projects it should be something like +once a week and less frequently (biweekly or monthly) for lesser +active projects. + +``git gc --aggressive`` not only removes dangling objects, it also +repacks object database into indexed and better optimized pack(s); it +also packs symbolic references (branches and tags). Another way to do +it is to run ``git repack``. + +There is a well-known `message +`_ from Linus +Torvalds regarding "stupidity" of ``git gc --aggressive``. The message +can safely be ignored now. It is old and outdated, ``git gc +--aggressive`` became much better since that time. + +For those who still prefer ``git repack`` over ``git gc --aggressive`` +the recommended parameters are ``git repack -a -d -f --depth=20 +--window=250``. See `this detailed experiment +`_ +for explanation of the effects of these parameters. + +From time to time run ``git fsck [--strict]`` to verify integrity of +the database. ``git fsck`` may produce a list of dangling objects; +that's not an error, just a reminder to perform regular maintenance. + + +Tips and tricks +=============== + +Command-line options and arguments +---------------------------------- + +`git help cli +`_ +recommends not to combine short options/flags. Most of the times +combining works: ``git commit -av`` works perfectly, but there are +situations when it doesn't. E.g., ``git log -p -5`` cannot be combined +as ``git log -p5``. + +Some options have arguments, some even have default arguments. In that +case the argument for such option must be spelled in a sticky way: +``-Oarg``, never ``-O arg`` because for an option that has a default +argument the latter means "use default value for option ``-O`` and +pass ``arg`` further to the option parser". For example, ``git grep`` +has an option ``-O`` that passes a list of names of the found files to +a program; default program for ``-O`` is a pager (usually ``less``), +but you can use your editor:: + + $ git grep -Ovim # but not -O vim + +BTW, if git is instructed to use ``less`` as the pager (i.e., if pager +is not configured in git at all it uses ``less`` by default, or if it +gets ``less`` from GIT_PAGER or PAGER environment variables, or if it +was configured with ``git config --global core.pager less``, or +``less`` is used in the command ``git grep -Oless``) ``git grep`` +passes ``+/$pattern`` option to ``less`` which is quite convenient. +Unfortunately, ``git grep`` doesn't pass the pattern if the pager is +not exactly ``less``, even if it's ``less`` with parameters (something +like ``git config --global core.pager less -FRSXgimq``); fortunately, +``git grep -Oless`` always passes the pattern. + + +bash/zsh completion +------------------- + +It's a bit hard to type ``git rebase --interactive --preserve-merges +HEAD~5`` manually even for those who are happy to use command-line, +and this is where shell completion is of great help. Bash/zsh come +with programmable completion, often automatically installed and +enabled, so if you have bash/zsh and git installed, chances are you +are already done - just go and use it at the command-line. + +If you don't have necessary bits installed, install and enable +bash_completion package. If you want to upgrade your git completion to +the latest and greatest download necessary file from `git contrib +`_. + +Git-for-windows comes with git-bash for which bash completion is +installed and enabled. + + +bash/zsh prompt +--------------- + +For command-line lovers shell prompt can carry a lot of useful +information. To include git information in the prompt use +`git-prompt.sh +`_. +Read the detailed instructions in the file. + +Search the Net for "git prompt" to find other prompt variants. + + +git on server +============= + +The simplest way to publish a repository or a group of repositories is +``git daemon``. The daemon provides anonymous access, by default it is +read-only. The repositories are accessible by git protocol (git:// +URLs). Write access can be enabled but the protocol lacks any +authentication means, so it should be enabled only within a trusted +LAN. See ``git help daemon`` for details. + +Git over ssh provides authentication and repo-level authorisation as +repositories can be made user- or group-writeable (see parameter +``core.sharedRepository`` in ``git help config``). If that's too +permissive or too restrictive for some project's needs there is a +wrapper `gitolite `_ that can +be configured to allow access with great granularity; gitolite is +written in Perl and has a lot of documentation. + +Web interface to browse repositories can be created using `gitweb +`_ or `cgit +`_. Both are CGI scripts (written in +Perl and C). In addition to web interface both provide read-only dumb +http access for git (http(s):// URLs). + +There are also more advanced web-based development environments that +include ability to manage users, groups and projects; private, +group-accessible and public repositories; they often include issue +trackers, wiki pages, pull requests and other tools for development +and communication. Among these environments are `Kallithea +`_ and `pagure `_, +both are written in Python; pagure was written by Fedora developers +and is being used to develop some Fedora projects. `Gogs +`_ is written in Go; there is a fork `Gitea +`_. + +And last but not least, `Gitlab `_. It's +perhaps the most advanced web-based development environment for git. +Written in Ruby, community edition is free and open source (MIT +license). + + +From Mercurial to git +===================== + +There are many tools to convert Mercurial repositories to git. The +most famous are, probably, `hg-git `_ and +`fast-export `_ (many years ago +it was known under the name ``hg2git``). + +But a better tool, perhaps the best, is `git-remote-hg +`_. It provides transparent +bidirectional (pull and push) access to Mercurial repositories from +git. Its author wrote a `comparison of alternatives +`_ +that seems to be mostly objective. + +To use git-remote-hg, install or clone it, add to your PATH (or copy +script ``git-remote-hg`` to a directory that's already in PATH) and +prepend ``hg::`` to Mercurial URLs. For example:: + + $ git clone https://github.com/felipec/git-remote-hg.git + $ PATH=$PATH:"`pwd`"/git-remote-hg + $ git clone hg::https://hg.python.org/peps/ PEPs + +To work with the repository just use regular git commands including +``git fetch/pull/push``. + +To start converting your Mercurial habits to git see the page +`Mercurial for Git users +`_ at Mercurial wiki. +At the second half of the page there is a table that lists +corresponding Mercurial and git commands. Should work perfectly in +both directions. + +Python Developer's Guide also has a chapter `Mercurial for git +developers `_ that +documents a few differences between git and hg. + + +Copyright +========= + +This document has been placed in the public domain. + + + +.. + Local Variables: + mode: indented-text + indent-tabs-mode: nil + sentence-end-double-space: t + fill-column: 70 + coding: utf-8 + End: + vim: set fenc=us-ascii tw=70 : diff --git a/pep-0478.txt b/pep-0478.txt --- a/pep-0478.txt +++ b/pep-0478.txt @@ -66,7 +66,7 @@ * PEP 479, change StopIteration handling inside generators * PEP 484, the typing module, a new standard for type annotations * PEP 485, math.isclose(), a function for testing approximate equality -* PEP 486, making the Widnows Python launcher aware of virtual environments +* PEP 486, making the Windows Python launcher aware of virtual environments * PEP 488, eliminating .pyo files * PEP 489, a new and improved mechanism for loading extension modules * PEP 492, coroutines with async and await syntax diff --git a/pep-0495.txt b/pep-0495.txt --- a/pep-0495.txt +++ b/pep-0495.txt @@ -404,6 +404,19 @@ Temporal Arithmetic and Comparison Operators ============================================ +.. epigraph:: + + | In *mathematicks* he was greater + | Than Tycho Brahe, or Erra Pater: + | For he, by geometric scale, + | Could take the size of pots of ale; + | Resolve, by sines and tangents straight, + | If bread or butter wanted weight, + | And wisely tell what hour o' th' day + | The clock does strike by algebra. + + -- "Hudibras" by Samuel Butler + The value of the ``fold`` attribute will be ignored in all operations with naive datetime instances. As a consequence, naive ``datetime.datetime`` or ``datetime.time`` instances that differ only diff --git a/pep-0498.txt b/pep-0498.txt --- a/pep-0498.txt +++ b/pep-0498.txt @@ -8,7 +8,7 @@ Content-Type: text/x-rst Created: 01-Aug-2015 Python-Version: 3.6 -Post-History: 07-Aug-2015, 30-Aug-2015, 04-Sep-2015 +Post-History: 07-Aug-2015, 30-Aug-2015, 04-Sep-2015, 19-Sep-2015 Resolution: https://mail.python.org/pipermail/python-dev/2015-September/141526.html Abstract @@ -201,6 +201,11 @@ replaced by the corresponding single brace. Doubled opening braces do not signify the start of an expression. +Note that ``__format__()`` is not called directly on each value. The +actual code uses the equivalent of ``type(value).__format__(value, +format_spec)``, or ``format(value, format_spec)``. See the +documentation of the builtin ``format()`` function for more details. + Comments, using the ``'#'`` character, are not allowed inside an expression. @@ -209,7 +214,7 @@ ``'!a'``. These are treated the same as in ``str.format()``: ``'!s'`` calls ``str()`` on the expression, ``'!r'`` calls ``repr()`` on the expression, and ``'!a'`` calls ``ascii()`` on the expression. These -conversions are applied before the call to ``__format__``. The only +conversions are applied before the call to ``format()``. The only reason to use ``'!s'`` is if you want to specify a format specifier that applies to ``str``, not to the type of the expression. @@ -222,9 +227,9 @@ f ' { } ... ' -The resulting expression's ``__format__`` method is called with the -format specifier as an argument. The resulting value is used when -building the value of the f-string. +The expression is then formatted using the ``__format__`` protocol, +using the format specifier as an argument. The resulting value is +used when building the value of the f-string. Expressions cannot contain ``':'`` or ``'!'`` outside of strings or parentheses, brackets, or braces. The exception is that the ``'!='`` @@ -293,7 +298,7 @@ Might be be evaluated as:: - 'abc' + expr1.__format__(spec1) + repr(expr2).__format__(spec2) + 'def' + str(expr3).__format__('') + 'ghi' + 'abc' + format(expr1, spec1) + format(repr(expr2)) + 'def' + format(str(expr3)) + 'ghi' Expression evaluation --------------------- @@ -371,7 +376,7 @@ While the exact method of this run time concatenation is unspecified, the above code might evaluate to:: - 'ab' + x.__format__('') + '{c}' + 'str<' + y.__format__('^4') + '>de' + 'ab' + format(x) + '{c}' + 'str<' + format(y, '^4') + '>de' Each f-string is entirely evaluated before being concatenated to adjacent f-strings. That means that this:: diff --git a/pep-0502.txt b/pep-0502.txt --- a/pep-0502.txt +++ b/pep-0502.txt @@ -1,44 +1,46 @@ PEP: 502 -Title: String Interpolation Redux +Title: String Interpolation - Extended Discussion Version: $Revision$ Last-Modified: $Date$ Author: Mike G. Miller Status: Draft -Type: Standards Track +Type: Informational Content-Type: text/x-rst Created: 10-Aug-2015 Python-Version: 3.6 -Note: Open issues below are stated with a question mark (?), -and are therefore searchable. - Abstract ======== -This proposal describes a new string interpolation feature for Python, -called an *expression-string*, -that is both concise and powerful, -improves readability in most cases, -yet does not conflict with existing code. +PEP 498: *Literal String Interpolation*, which proposed "formatted strings" was +accepted September 9th, 2015. +Additional background and rationale given during its design phase is detailed +below. -To achieve this end, -a new string prefix is introduced, -which expands at compile-time into an equivalent expression-string object, -with requested variables from its context passed as keyword arguments. +To recap that PEP, +a string prefix was introduced that marks the string as a template to be +rendered. +These formatted strings may contain one or more expressions +built on `the existing syntax`_ of ``str.format()``. +The formatted string expands at compile-time into a conventional string format +operation, +with the given expressions from its text extracted and passed instead as +positional arguments. + At runtime, -the new object uses these passed values to render a string to given -specifications, building on `the existing syntax`_ of ``str.format()``:: +the resulting expressions are evaluated to render a string to given +specifications:: >>> location = 'World' - >>> e'Hello, {location} !' # new prefix: e'' - 'Hello, World !' # interpolated result + >>> f'Hello, {location} !' # new prefix: f'' + 'Hello, World !' # interpolated result + +Format-strings may be thought of as merely syntactic sugar to simplify traditional +calls to ``str.format()``. .. _the existing syntax: https://docs.python.org/3/library/string.html#format-string-syntax -This PEP does not recommend to remove or deprecate any of the existing string -formatting mechanisms. - Motivation ========== @@ -50,12 +52,16 @@ with similar use cases, the amount of code necessary to build similar strings is substantially higher, while at times offering lower readability due to verbosity, dense syntax, -or identifier duplication. [1]_ +or identifier duplication. + +These difficulties are described at moderate length in the original +`post to python-ideas`_ +that started the snowball (that became PEP 498) rolling. [1]_ Furthermore, replacement of the print statement with the more consistent print function of Python 3 (PEP 3105) has added one additional minor burden, an additional set of parentheses to type and read. -Combined with the verbosity of current formatting solutions, +Combined with the verbosity of current string formatting solutions, this puts an otherwise simple language at an unfortunate disadvantage to its peers:: @@ -66,7 +72,7 @@ # Python 3, str.format with named parameters print('Hello, user: {user}, id: {id}, on host: {hostname}'.format(**locals())) - # Python 3, variation B, worst case + # Python 3, worst case print('Hello, user: {user}, id: {id}, on host: {hostname}'.format(user=user, id=id, hostname= @@ -74,7 +80,7 @@ In Python, the formatting and printing of a string with multiple variables in a single line of code of standard width is noticeably harder and more verbose, -indentation often exacerbating the issue. +with indentation exacerbating the issue. For use cases such as smaller projects, systems programming, shell script replacements, and even one-liners, @@ -82,36 +88,17 @@ this verbosity has likely lead a significant number of developers and administrators to choose other languages over the years. +.. _post to python-ideas: https://mail.python.org/pipermail/python-ideas/2015-July/034659.html + Rationale ========= -Naming ------- - -The term expression-string was chosen because other applicable terms, -such as format-string and template are already well used in the Python standard -library. - -The string prefix itself, ``e''`` was chosen to demonstrate that the -specification enables expressions, -is not limited to ``str.format()`` syntax, -and also does not lend itself to `the shorthand term`_ "f-string". -It is also slightly easier to type than other choices such as ``_''`` and -``i''``, -while perhaps `less odd-looking`_ to C-developers. -``printf('')`` vs. ``print(f'')``. - -.. _the shorthand term: reference_needed -.. _less odd-looking: https://mail.python.org/pipermail/python-dev/2015-August/141147.html - - - Goals ------------- -The design goals of expression-strings are as follows: +The design goals of format strings are as follows: #. Eliminate need to pass variables manually. #. Eliminate repetition of identifiers and redundant parentheses. @@ -133,40 +120,44 @@ characters to enclose strings. It is not reasonable to choose one of them now to enable interpolation, while leaving the other for uninterpolated strings. -"Backtick" characters (`````) are also `constrained by history`_ as a shortcut -for ``repr()``. +Other characters, +such as the "Backtick" (or grave accent `````) are also +`constrained by history`_ +as a shortcut for ``repr()``. This leaves a few remaining options for the design of such a feature: * An operator, as in printf-style string formatting via ``%``. * A class, such as ``string.Template()``. -* A function, such as ``str.format()``. -* New syntax +* A method or function, such as ``str.format()``. +* New syntax, or * A new string prefix marker, such as the well-known ``r''`` or ``u''``. -The first three options above currently work well. +The first three options above are mature. Each has specific use cases and drawbacks, yet also suffer from the verbosity and visual noise mentioned previously. -All are discussed in the next section. +All options are discussed in the next sections. .. _constrained by history: https://mail.python.org/pipermail/python-ideas/2007-January/000054.html + Background ------------- -This proposal builds on several existing techniques and proposals and what +Formatted strings build on several existing techniques and proposals and what we've collectively learned from them. +In keeping with the design goals of readability and error-prevention, +the following examples therefore use named, +not positional arguments. -The following examples focus on the design goals of readability and -error-prevention using named parameters. Let's assume we have the following dictionary, and would like to print out its items as an informative string for end users:: >>> params = {'user': 'nobody', 'id': 9, 'hostname': 'darkstar'} -Printf-style formatting -''''''''''''''''''''''' +Printf-style formatting, via operator +''''''''''''''''''''''''''''''''''''' This `venerable technique`_ continues to have its uses, such as with byte-based protocols, @@ -178,7 +169,7 @@ In this form, considering the prerequisite dictionary creation, the technique is verbose, a tad noisy, -and relatively readable. +yet relatively readable. Additional issues are that an operator can only take one argument besides the original string, meaning multiple parameters must be passed in a tuple or dictionary. @@ -190,8 +181,8 @@ .. _venerable technique: https://docs.python.org/3/library/stdtypes.html#printf-style-string-formatting -string.Template -''''''''''''''' +string.Template Class +''''''''''''''''''''' The ``string.Template`` `class from`_ PEP 292 (Simpler String Substitutions) @@ -202,7 +193,7 @@ Template('Hello, user: $user, id: ${id}, on host: $hostname').substitute(params) -Also verbose, however the string itself is readable. +While also verbose, the string itself is readable. Though functionality is limited, it meets its requirements well. It isn't powerful enough for many cases, @@ -232,8 +223,8 @@ It was superseded by the following proposal. -str.format() -'''''''''''' +str.format() Method +''''''''''''''''''' The ``str.format()`` `syntax of`_ PEP 3101 is the most recent and modern of the existing options. @@ -253,36 +244,32 @@ host=hostname) 'Hello, user: nobody, id: 9, on host: darkstar' +The verbosity of the method-based approach is illustrated here. + .. _syntax of: https://docs.python.org/3/library/string.html#format-string-syntax PEP 498 -- Literal String Formatting '''''''''''''''''''''''''''''''''''' -PEP 498 discusses and delves partially into implementation details of -expression-strings, -which it calls f-strings, -the idea and syntax -(with exception of the prefix letter) -of which is identical to that discussed here. -The resulting compile-time transformation however -returns a string joined from parts at runtime, -rather than an object. +PEP 498 defines and discusses format strings, +as also described in the `Abstract`_ above. -It also, somewhat controversially to those first exposed to it, -introduces the idea that these strings shall be augmented with support for -arbitrary expressions, -which is discussed further in the following sections. - +It also, somewhat controversially to those first exposed, +introduces the idea that format-strings shall be augmented with support for +arbitrary expressions. +This is discussed further in the +Restricting Syntax section under +`Rejected Ideas`_. PEP 501 -- Translation ready string interpolation ''''''''''''''''''''''''''''''''''''''''''''''''' The complimentary PEP 501 brings internationalization into the discussion as a -first-class concern, with its proposal of i-strings, +first-class concern, with its proposal of the i-prefix, ``string.Template`` syntax integration compatible with ES6 (Javascript), deferred rendering, -and a similar object return value. +and an object return value. Implementations in Other Languages @@ -374,7 +361,8 @@ Designers of `Template strings`_ faced the same issue as Python where single and double quotes were taken. Unlike Python however, "backticks" were not. -They were chosen as part of the ECMAScript 2015 (ES6) standard:: +Despite `their issues`_, +they were chosen as part of the ECMAScript 2015 (ES6) standard:: console.log(`Fifteen is ${a + b} and\nnot ${2 * a + b}.`); @@ -391,8 +379,10 @@ * User implemented prefixes supported. * Arbitrary expressions are supported. +.. _their issues: https://mail.python.org/pipermail/python-ideas/2007-January/000054.html .. _Template strings: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/template_strings + C#, Version 6 ''''''''''''' @@ -428,13 +418,14 @@ Additional examples ''''''''''''''''''' -A number of additional examples may be `found at Wikipedia`_. +A number of additional examples of string interpolation may be +`found at Wikipedia`_. + +Now that background and history have been covered, +let's continue on for a solution. .. _found at Wikipedia: https://en.wikipedia.org/wiki/String_interpolation#Examples -Now that background and imlementation history have been covered, -let's continue on for a solution. - New Syntax ---------- @@ -442,178 +433,47 @@ This should be an option of last resort, as every new syntax feature has a cost in terms of real-estate in a brain it inhabits. -There is one alternative left on our list of possibilities, +There is however one alternative left on our list of possibilities, which follows. New String Prefix ----------------- -Given the history of string formatting in Python, -backwards-compatibility, +Given the history of string formatting in Python and backwards-compatibility, implementations in other languages, -and the avoidance of new syntax unless necessary, +avoidance of new syntax unless necessary, an acceptable design is reached through elimination rather than unique insight. -Therefore, we choose to explicitly mark interpolated string literals with a -string prefix. +Therefore, marking interpolated string literals with a string prefix is chosen. -We also choose an expression syntax that reuses and builds on the strongest of +We also choose an expression syntax that reuses and builds on the strongest of the existing choices, -``str.format()`` to avoid further duplication. - - -Specification -============= - -String literals with the prefix of ``e`` shall be converted at compile-time to -the construction of an ``estr`` (perhaps ``types.ExpressionString``?) object. -Strings and values are parsed from the literal and passed as tuples to the -constructor:: +``str.format()`` to avoid further duplication of functionality:: >>> location = 'World' - >>> e'Hello, {location} !' + >>> f'Hello, {location} !' # new prefix: f'' + 'Hello, World !' # interpolated result - # becomes - # estr('Hello, {location} !', # template - ('Hello, ', ' !'), # string fragments - ('location',), # expressions - ('World',), # values - ) +PEP 498 -- Literal String Formatting, delves into the mechanics and +implementation of this design. -The object interpolates its result immediately at run-time:: - 'Hello, World !' - - -ExpressionString Objects ------------------------- - -The ExpressionString object supports both immediate and deferred rendering of -its given template and parameters. -It does this by immediately rendering its inputs to its internal string and -``.rendered`` string member (still necessary?), -useful in the majority of use cases. -To allow for deferred rendering and caller-specified escaping, -all inputs are saved for later inspection, -with convenience methods available. - -Notes: - -* Inputs are saved to the object as ``.template`` and ``.context`` members - for later use. -* No explicit ``str(estr)`` call is necessary to render the result, - though doing so might be desired to free resources if significant. -* Additional or deferred rendering is available through the ``.render()`` - method, which allows template and context to be overriden for flexibility. -* Manual escaping of potentially dangerous input is available through the - ``.escape(escape_function)`` method, - the rules of which may therefore be specified by the caller. - The given function should both accept and return a single modified string. - -* A sample Python implementation can `found at Bitbucket`_: - -.. _found at Bitbucket: https://bitbucket.org/mixmastamyk/docs/src/default/pep/estring_demo.py - - -Inherits From ``str`` Type -''''''''''''''''''''''''''' - -Inheriting from the ``str`` class is one of the techniques available to improve -compatibility with code expecting a string object, -as it will pass an ``isinstance(obj, str)`` test. -ExpressionString implements this and also renders its result into the "raw" -string of its string superclass, -providing compatibility with a majority of code. - - -Interpolation Syntax --------------------- - -The strongest of the existing string formatting syntaxes is chosen, -``str.format()`` as a base to build on. [10]_ [11]_ - -.. - -* Additionally, single arbitrary expressions shall also be supported inside - braces as an extension:: - - >>> e'My age is {age + 1} years.' - - See below for section on safety. - -* Triple quoted strings with multiple lines shall be supported:: - - >>> e'''Hello, - {location} !''' - 'Hello,\n World !' - -* Adjacent implicit concatenation shall be supported; - interpolation does not `not bleed into`_ other strings:: - - >>> 'Hello {1, 2, 3} ' e'{location} !' - 'Hello {1, 2, 3} World !' - -* Additional implementation details, - for example expression and error-handling, - are specified in the compatible PEP 498. - -.. _not bleed into: https://mail.python.org/pipermail/python-ideas/2015-July/034763.html - - -Composition with Other Prefixes -------------------------------- - -* Expression-strings apply to unicode objects only, - therefore ``u''`` is never needed. - Should it be prevented? - -* Bytes objects are not included here and do not compose with e'' as they - do not support ``__format__()``. - -* Complimentary to raw strings, - backslash codes shall not be converted in the expression-string, - when combined with ``r''`` as ``re''``. - - -Examples --------- - -A more complicated example follows:: - - n = 5; # t0, t1 = ? TODO - a = e"Sliced {n} onions in {t1-t0:.3f} seconds." - # returns the equvalent of - estr("Sliced {n} onions in {t1-t0:.3f} seconds", # template - ('Sliced ', ' onions in ', ' seconds'), # strings - ('n', 't1-t0:.3f'), # expressions - (5, 0.555555) # values - ) - -With expressions only:: - - b = e"Three random numbers: {rand()}, {rand()}, {rand()}." - # returns the equvalent of - estr("Three random numbers: {rand():f}, {rand():f}, {rand():}.", # template - ('Three random numbers: ', ', ', ', ', '.'), # strings - ('rand():f', 'rand():f', 'rand():f'), # expressions - (rand(), rand(), rand()) # values - ) +Additional Topics +================= Safety ----------- In this section we will describe the safety situation and precautions taken -in support of expression-strings. +in support of format-strings. -#. Only string literals shall be considered here, +#. Only string literals have been considered for format-strings, not variables to be taken as input or passed around, making external attacks difficult to accomplish. - * ``str.format()`` `already handles`_ this use-case. - * Direct instantiation of the ExpressionString object with non-literal input - shall not be allowed. (Practicality?) + ``str.format()`` and alternatives `already handle`_ this use-case. #. Neither ``locals()`` nor ``globals()`` are necessary nor used during the transformation, @@ -622,37 +482,72 @@ #. To eliminate complexity as well as ``RuntimeError`` (s) due to recursion depth, recursive interpolation is not supported. -#. Restricted characters or expression classes?, such as ``=`` for assignment. - However, mistakes or malicious code could be missed inside string literals. Though that can be said of code in general, that these expressions are inside strings means they are a bit more likely to be obscured. -.. _already handles: https://mail.python.org/pipermail/python-ideas/2015-July/034729.html +.. _already handle: https://mail.python.org/pipermail/python-ideas/2015-July/034729.html -Mitigation via tools +Mitigation via Tools '''''''''''''''''''' The idea is that tools or linters such as pyflakes, pylint, or Pycharm, -could check inside strings for constructs that exceed project policy. -As this is a common task with languages these days, -tools won't have to implement this feature solely for Python, +may check inside strings with expressions and mark them up appropriately. +As this is a common task with programming languages today, +multi-language tools won't have to implement this feature solely for Python, significantly shortening time to implementation. -Additionally the Python interpreter could check(?) and warn with appropriate -command-line parameters passed. +Farther in the future, +strings might also be checked for constructs that exceed the safety policy of +a project. + + +Style Guide/Precautions +----------------------- + +As arbitrary expressions may accomplish anything a Python expression is +able to, +it is highly recommended to avoid constructs inside format-strings that could +cause side effects. + +Further guidelines may be written once usage patterns and true problems are +known. + + +Reference Implementation(s) +--------------------------- + +The `say module on PyPI`_ implements string interpolation as described here +with the small burden of a callable interface:: + + ? pip install say + + from say import say + nums = list(range(4)) + say("Nums has {len(nums)} items: {nums}") + +A Python implementation of Ruby interpolation `is also available`_. +It uses the codecs module to do its work:: + + ? pip install interpy + + # coding: interpy + location = 'World' + print("Hello #{location}.") + +.. _say module on PyPI: https://pypi.python.org/pypi/say/ +.. _is also available: https://github.com/syrusakbary/interpy Backwards Compatibility ----------------------- -By using existing syntax and avoiding use of current or historical features, -expression-strings (and any associated sub-features), -were designed so as to not interfere with existing code and is not expected -to cause any issues. +By using existing syntax and avoiding current or historical features, +format strings were designed so as to not interfere with existing code and are +not expected to cause any issues. Postponed Ideas @@ -666,20 +561,12 @@ the finer details diverge at almost every point, making a common solution unlikely: [15]_ -* Use-cases -* Compile and run-time tasks -* Interpolation Syntax +* Use-cases differ +* Compile vs. run-time tasks +* Interpolation syntax needs * Intended audience * Security policy -Rather than try to fit a "square peg in a round hole," -this PEP attempts to allow internationalization to be supported in the future -by not preventing it. -In this proposal, -expression-string inputs are saved for inspection and re-rendering at a later -time, -allowing for their use by an external library of any sort. - Rejected Ideas -------------- @@ -687,17 +574,24 @@ Restricting Syntax to ``str.format()`` Only ''''''''''''''''''''''''''''''''''''''''''' -This was deemed not enough of a solution to the problem. +The common `arguments against`_ support of arbitrary expresssions were: + +#. `YAGNI`_, "You aren't gonna need it." +#. The feature is not congruent with historical Python conservatism. +#. Postpone - can implement in a future version if need is demonstrated. + +.. _YAGNI: https://en.wikipedia.org/wiki/You_aren't_gonna_need_it +.. _arguments against: https://mail.python.org/pipermail/python-ideas/2015-August/034913.html + +Support of only ``str.format()`` syntax however, +was deemed not enough of a solution to the problem. +Often a simple length or increment of an object, for example, +is desired before printing. + It can be seen in the `Implementations in Other Languages`_ section that the developer community at large tends to agree. - -The common `arguments against`_ arbitrary expresssions were: - -#. YAGNI, "You ain't gonna need it." -#. The change is not congruent with historical Python conservatism. -#. Postpone - can implement in a future version if need is demonstrated. - -.. _arguments against: https://mail.python.org/pipermail/python-ideas/2015-August/034913.html +String interpolation with arbitrary expresssions is becoming an industry +standard in modern languages due to its utility. Additional/Custom String-Prefixes @@ -720,7 +614,7 @@ expressions could be used safely or not. The concept was also difficult to describe to others. [12]_ -Always consider expression-string variables to be unescaped, +Always consider format string variables to be unescaped, unless the developer has explicitly escaped them. @@ -735,33 +629,13 @@ which could encourage bad habits. [13]_ -Reference Implementation(s) -=========================== - -An expression-string implementation is currently attached to PEP 498, -under the ``f''`` prefix, -and may be available in nightly builds. - -A Python implementation of Ruby interpolation `is also available`_, -which is similar to this proposal. -It uses the codecs module to do its work:: - - ? pip install interpy - - # coding: interpy - location = 'World' - print("Hello #{location}.") - -.. _is also available: https://github.com/syrusakbary/interpy - - Acknowledgements ================ -* Eric V. Smith for providing invaluable implementation work and design - opinions, helping to focus this PEP. -* Others on the python-ideas mailing list for rejecting the craziest of ideas, - also helping to achieve focus. +* Eric V. Smith for the authoring and implementation of PEP 498. +* Everyone on the python-ideas mailing list for rejecting the various crazy + ideas that came up, + helping to keep the final design in focus. References @@ -771,7 +645,6 @@ (https://mail.python.org/pipermail/python-ideas/2015-July/034659.html) - .. [2] Briefer String Format (https://mail.python.org/pipermail/python-ideas/2015-July/034669.html) diff --git a/pep-0504.txt b/pep-0504.txt new file mode 100644 --- /dev/null +++ b/pep-0504.txt @@ -0,0 +1,396 @@ +PEP: 504 +Title: Using the System RNG by default +Version: $Revision$ +Last-Modified: $Date$ +Author: Nick Coghlan +Status: Withdrawn +Type: Standards Track +Content-Type: text/x-rst +Created: 15-Sep-2015 +Python-Version: 3.6 +Post-History: 15-Sep-2015 + +Abstract +======== + +Python currently defaults to using the deterministic Mersenne Twister random +number generator for the module level APIs in the ``random`` module, requiring +users to know that when they're performing "security sensitive" work, they +should instead switch to using the cryptographically secure ``os.urandom`` or +``random.SystemRandom`` interfaces or a third party library like +``cryptography``. + +Unfortunately, this approach has resulted in a situation where developers that +aren't aware that they're doing security sensitive work use the default module +level APIs, and thus expose their users to unnecessary risks. + +This isn't an acute problem, but it is a chronic one, and the often long +delays between the introduction of security flaws and their exploitation means +that it is difficult for developers to naturally learn from experience. + +In order to provide an eventually pervasive solution to the problem, this PEP +proposes that Python switch to using the system random number generator by +default in Python 3.6, and require developers to opt-in to using the +deterministic random number generator process wide either by using a new +``random.ensure_repeatable()`` API, or by explicitly creating their own +``random.Random()`` instance. + +To minimise the impact on existing code, module level APIs that require +determinism will implicitly switch to the deterministic PRNG. + +PEP Withdrawal +============== + +During discussion of this PEP, Steven D'Aprano proposed the simpler alternative +of offering a standardised ``secrets`` module that provides "one obvious way" +to handle security sensitive tasks like generating default passwords and other +tokens. + +Steven's proposal has the desired effect of aligning the easy way to generate +such tokens and the right way to generate them, without introducing any +compatibility risks for the existing ``random`` module API, so this PEP has +been withdrawn in favour of further work on refining Steven's proposal as +PEP 506. + + +Proposal +======== + +Currently, it is never correct to use the module level functions in the +``random`` module for security sensitive applications. This PEP proposes to +change that admonition in Python 3.6+ to instead be that it is not correct to +use the module level functions in the ``random`` module for security sensitive +applications if ``random.ensure_repeatable()`` is ever called (directly or +indirectly) in that process. + +To achieve this, rather than being bound methods of a ``random.Random`` +instance as they are today, the module level callables in ``random`` would +change to be functions that delegate to the corresponding method of the +existing ``random._inst`` module attribute. + +By default, this attribute will be bound to a ``random.SystemRandom`` instance. + +A new ``random.ensure_repeatable()`` API will then rebind the ``random._inst`` +attribute to a ``system.Random`` instance, restoring the same module level +API behaviour as existed in previous Python versions (aside from the +additional level of indirection):: + + def ensure_repeatable(): + """Switch to using random.Random() for the module level APIs + + This switches the default RNG instance from the crytographically + secure random.SystemRandom() to the deterministic random.Random(), + enabling the seed(), getstate() and setstate() operations. This means + a particular random scenario can be replayed later by providing the + same seed value or restoring a previously saved state. + + NOTE: Libraries implementing security sensitive operations should + always explicitly use random.SystemRandom() or os.urandom in order to + correctly handle applications that call this function. + """ + if not isinstance(_inst, Random): + _inst = random.Random() + +To minimise the impact on existing code, calling any of the following module +level functions will implicitly call ``random.ensure_repeatable()``: + +* ``random.seed`` +* ``random.getstate`` +* ``random.setstate`` + +There are no changes proposed to the ``random.Random`` or +``random.SystemRandom`` class APIs - applications that explicitly instantiate +their own random number generators will be entirely unaffected by this +proposal. + +Warning on implicit opt-in +-------------------------- + +In Python 3.6, implicitly opting in to the use of the deterministic PRNG will +emit a deprecation warning using the following check:: + + if not isinstance(_inst, Random): + warnings.warn(DeprecationWarning, + "Implicitly ensuring repeatability. " + "See help(random.ensure_repeatable) for details") + ensure_repeatable() + +The specific wording of the warning should have a suitable answer added to +Stack Overflow as was done for the custom error message that was added for +missing parentheses in a call to print [#print]_. + +In the first Python 3 release after Python 2.7 switches to security fix only +mode, the deprecation warning will be upgraded to a RuntimeWarning so it is +visible by default. + +This PEP does *not* propose ever removing the ability to ensure the default RNG +used process wide is a deterministic PRNG that will produce the same series of +outputs given a specific seed. That capability is widely used in modelling +and simulation scenarios, and requiring that ``ensure_repeatable()`` be called +either directly or indirectly is a sufficient enhancement to address the cases +where the module level random API is used for security sensitive tasks in web +applications without due consideration for the potential security implications +of using a deterministic PRNG. + +Performance impact +------------------ + +Due to the large performance difference between ``random.Random`` and +``random.SystemRandom``, applications ported to Python 3.6 will encounter a +significant performance regression in cases where: + +* the application is using the module level random API +* cryptographic quality randomness isn't needed +* the application doesn't already implicitly opt back in to the deterministic + PRNG by calling ``random.seed``, ``random.getstate``, or ``random.setstate`` +* the application isn't updated to explicitly call ``random.ensure_repeatable`` + +This would be noted in the Porting section of the Python 3.6 What's New guide, +with the recommendation to include the following code in the ``__main__`` +module of affected applications:: + + if hasattr(random, "ensure_repeatable"): + random.ensure_repeatable() + +Applications that do need cryptographic quality randomness should be using the +system random number generator regardless of speed considerations, so in those +cases the change proposed in this PEP will fix a previously latent security +defect. + +Documentation changes +--------------------- + +The ``random`` module documentation would be updated to move the documentation +of the ``seed``, ``getstate`` and ``setstate`` interfaces later in the module, +along with the documentation of the new ``ensure_repeatable`` function and the +associated security warning. + +That section of the module documentation would also gain a discussion of the +respective use cases for the deterministic PRNG enabled by +``ensure_repeatable`` (games, modelling & simulation, software testing) and the +system RNG that is used by default (cryptography, security token generation). +This discussion will also recommend the use of third party security libraries +for the latter task. + +Rationale +========= + +Writing secure software under deadline and budget pressures is a hard problem. +This is reflected in regular notifications of data breaches involving personally +identifiable information [#breaches]_, as well as with failures to take +security considerations into account when new systems, like motor vehicles +[#uconnect]_, are connected to the internet. It's also the case that a lot of +the programming advice readily available on the internet [#search] simply +doesn't take the mathemetical arcana of computer security into account. +Compounding these issues is the fact that defenders have to cover *all* of +their potential vulnerabilites, as a single mistake can make it possible to +subvert other defences [#bcrypt]_. + +One of the factors that contributes to making this last aspect particularly +difficult is APIs where using them inappropriately creates a *silent* security +failure - one where the only way to find out that what you're doing is +incorrect is for someone reviewing your code to say "that's a potential +security problem", or for a system you're responsible for to be compromised +through such an oversight (and you're not only still responsible for that +system when it is compromised, but your intrusion detection and auditing +mechanisms are good enough for you to be able to figure out after the event +how the compromise took place). + +This kind of situation is a significant contributor to "security fatigue", +where developers (often rightly [#owasptopten]_) feel that security engineers +spend all their time saying "don't do that the easy way, it creates a +security vulnerability". + +As the designers of one of the world's most popular languages [#ieeetopten]_, +we can help reduce that problem by making the easy way the right way (or at +least the "not wrong" way) in more circumstances, so developers and security +engineers can spend more time worrying about mitigating actually interesting +threats, and less time fighting with default language behaviours. + +Discussion +========== + +Why "ensure_repeatable" over "ensure_deterministic"? +---------------------------------------------------- + +This is a case where the meaning of a word as specialist jargon conflicts with +the typical meaning of the word, even though it's *technically* the same. + +From a technical perspective, a "deterministic RNG" means that given knowledge +of the algorithm and the current state, you can reliably compute arbitrary +future states. + +The problem is that "deterministic" on its own doesn't convey those qualifiers, +so it's likely to instead be interpreted as "predictable" or "not random" by +folks that are familiar with the conventional meaning, but aren't familiar with +the additional qualifiers on the technical meaning. + +A second problem with "deterministic" as a description for the traditional RNG +is that it doesn't really tell you what you can *do* with the traditional RNG +that you can't do with the system one. + +"ensure_repeatable" aims to address both of those problems, as its common +meaning accurately describes the main reason for preferring the deterministic +PRNG over the system RNG: ensuring you can repeat the same series of outputs +by providing the same seed value, or by restoring a previously saved PRNG state. + +Only changing the default for Python 3.6+ +----------------------------------------- + +Some other recent security changes, such as upgrading the capabilities of the +``ssl`` module and switching to properly verifying HTTPS certificates by +default, have been considered critical enough to justify backporting the +change to all currently supported versions of Python. + +The difference in this case is one of degree - the additional benefits from +rolling out this particular change a couple of years earlier than will +otherwise be the case aren't sufficient to justify either the additional effort +or the stability risks involved in making such an intrusive change in a +maintenance release. + +Keeping the module level functions +---------------------------------- + +In additional to general backwards compatibility considerations, Python is +widely used for educational purposes, and we specifically don't want to +invalidate the wide array of educational material that assumes the availabilty +of the current ``random`` module API. Accordingly, this proposal ensures that +most of the public API can continue to be used not only without modification, +but without generating any new warnings. + +Warning when implicitly opting in to the deterministic RNG +---------------------------------------------------------- + +It's necessary to implicitly opt in to the deterministic PRNG as Python is +widely used for modelling and simulation purposes where this is the right +thing to do, and in many cases, these software models won't have a dedicated +maintenance team tasked with ensuring they keep working on the latest versions +of Python. + +Unfortunately, explicitly calling ``random.seed`` with data from ``os.urandom`` +is also a mistake that appears in a number of the flawed "how to generate a +security token in Python" guides readily available online. + +Using first DeprecationWarning, and then eventually a RuntimeWarning, to +advise against implicitly switching to the deterministic PRNG aims to +nudge future users that need a cryptographically secure RNG away from +calling ``random.seed()`` and those that genuinely need a deterministic +generator towards explicitily calling ``random.ensure_repeatable()``. + +Avoiding the introduction of a userspace CSPRNG +----------------------------------------------- + +The original discussion of this proposal on python-ideas[#csprng]_ suggested +introducing a cryptographically secure pseudo-random number generator and using +that by default, rather than defaulting to the relatively slow system random +number generator. + +The problem [#nocsprng]_ with this approach is that it introduces an additional +point of failure in security sensitive situations, for the sake of applications +where the random number generation may not even be on a critical performance +path. + +Applications that do need cryptographic quality randomness should be using the +system random number generator regardless of speed considerations, so in those +cases. + +Isn't the deterministic PRNG "secure enough"? +--------------------------------------------- + +In a word, "No" - that's why there's a warning in the module documentation +that says not to use it for security sensitive purposes. While we're not +currently aware of any studies of Python's random number generator specifically, +studies of PHP's random number generator [#php]_ have demonstrated the ability +to use weaknesses in that subsystem to facilitate a practical attack on +password recovery tokens in popular PHP web applications. + +However, one of the rules of secure software development is that "attacks only +get better, never worse", so it may be that by the time Python 3.6 is released +we will actually see a practical attack on Python's deterministic PRNG publicly +documented. + +Security fatigue in the Python ecosystem +---------------------------------------- + +Over the past few years, the computing industry as a whole has been +making a concerted effort to upgrade the shared network infrastructure we all +depend on to a "secure by default" stance. As one of the most widely used +programming languages for network service development (including the OpenStack +Infrastructure-as-a-Service platform) and for systems administration +on Linux systems in general, a fair share of that burden has fallen on the +Python ecosystem, which is understandably frustrating for Pythonistas using +Python in other contexts where these issues aren't of as great a concern. + +This consideration is one of the primary factors driving the substantial +backwards compatibility improvements in this proposal relative to the initial +draft concept posted to python-ideas [#draft]_. + +Acknowledgements +================ + +* Theo de Raadt, for making the suggestion to Guido van Rossum that we + seriously consider defaulting to a cryptographically secure random number + generator +* Serhiy Storchaka, Terry Reedy, Petr Viktorin, and anyone else in the + python-ideas threads that suggested the approach of transparently switching + to the ``random.Random`` implementation when any of the functions that only + make sense for a deterministic RNG are called +* Nathaniel Smith for providing the reference on practical attacks against + PHP's random number generator when used to generate password reset tokens +* Donald Stufft for pursuing additional discussions with network security + experts that suggested the introduction of a userspace CSPRNG would mean + additional complexity for insufficient gain relative to just using the + system RNG directly +* Paul Moore for eloquently making the case for the current level of security + fatigue in the Python ecosystem + +References +========== + +.. [#breaches] Visualization of data breaches involving more than 30k records (each) + (http://www.informationisbeautiful.net/visualizations/worlds-biggest-data-breaches-hacks/) + +.. [#uconnect] Remote UConnect hack for Jeep Cherokee + (http://www.wired.com/2015/07/hackers-remotely-kill-jeep-highway/) + +.. [#php] PRNG based attack against password reset tokens in PHP applications + (https://media.blackhat.com/bh-us-12/Briefings/Argyros/BH_US_12_Argyros_PRNG_WP.pdf) + +.. [#search] Search link for "python password generator" + (https://www.google.com.au/search?q=python+password+generator) + +.. [#csprng] python-ideas thread discussing using a userspace CSPRNG + (https://mail.python.org/pipermail/python-ideas/2015-September/035886.html) + +.. [#draft] Initial draft concept that eventually became this PEP + (https://mail.python.org/pipermail/python-ideas/2015-September/036095.html) + +.. [#nocsprng] Safely generating random numbers + (http://sockpuppet.org/blog/2014/02/25/safely-generate-random-numbers/) + +.. [#ieeetopten] IEEE Spectrum 2015 Top Ten Programming Languages + (http://spectrum.ieee.org/computing/software/the-2015-top-ten-programming-languages) + +.. [#owasptopten] OWASP Top Ten Web Security Issues for 2013 + (https://www.owasp.org/index.php/OWASP_Top_Ten_Project#tab=OWASP_Top_10_for_2013) + +.. [#print] Stack Overflow answer for missing parentheses in call to print + (http://stackoverflow.com/questions/25445439/what-does-syntaxerror-missing-parentheses-in-call-to-print-mean-in-python/25445440#25445440) + +.. [#bcrypt] Bypassing bcrypt through an insecure data cache + (http://arstechnica.com/security/2015/09/once-seen-as-bulletproof-11-million-ashley-madison-passwords-already-cracked/) + +Copyright +========= + +This document has been placed in the public domain. + + +.. + Local Variables: + mode: indented-text + indent-tabs-mode: nil + sentence-end-double-space: t + fill-column: 70 + coding: utf-8 + End: diff --git a/pep-0505.txt b/pep-0505.txt new file mode 100644 --- /dev/null +++ b/pep-0505.txt @@ -0,0 +1,205 @@ +PEP: 505 +Title: None coalescing operators +Version: $Revision$ +Last-Modified: $Date$ +Author: Mark E. Haase +Status: Draft +Type: Standards Track +Content-Type: text/x-rst +Created: 18-Sep-2015 +Python-Version: 3.6 + +Abstract +======== + +Several modern programming languages have so-called "null coalescing" or +"null aware" operators, including C#, Dart, Perl, Swift, and PHP (starting in +version 7). These operators provide syntactic sugar for common patterns +involving null references. [1]_ [2]_ + +* The "null coalescing" operator is a binary operator that returns its first + first non-null operand. +* The "null aware member access" operator is a binary operator that accesses + an instance member only if that instance is non-null. It returns null + otherwise. +* The "null aware index access" operator is a binary operator that accesses a + member of a collection only if that collection is non-null. It returns null + otherwise. + +Python does not have any directly equivalent syntax. The ``or`` operator can +be used to similar effect but checks for a truthy value, not ``None`` +specifically. The ternary operator ``... if ... else ...`` can be used for +explicit null checks but is more verbose and typically duplicates part of the +expression in between ``if`` and ``else``. The proposed ``None`` coalescing +and ``None`` aware operators ofter an alternative syntax that is more +intuitive and concise. + + +Rationale +========= + +Null Coalescing Operator +------------------------ + +The following code illustrates how the ``None`` coalescing operators would +work in Python:: + + >>> title = 'My Title' + >>> title ?? 'Default Title' + 'My Title' + >>> title = None + >>> title ?? 'Default Title' + 'Default Title' + +Similar behavior can be achieved with the ``or`` operator, but ``or`` checks +whether its left operand is false-y, not specifically ``None``. This can lead +to surprising behavior. Consider the scenario of computing the price of some +products a customer has in his/her shopping cart:: + + >>> price = 100 + >>> requested_quantity = 5 + >>> default_quantity = 1 + >>> (requested_quantity or default_quantity) * price + 500 + >>> requested_quantity = None + >>> (requested_quantity or default_quantity) * price + 100 + >>> requested_quantity = 0 + >>> (requested_quantity or default_quantity) * price # oops! + 100 + +This type of bug is not possible with the ``None`` coalescing operator, +because there is no implicit type coersion to ``bool``:: + + >>> price = 100 + >>> requested_quantity = 0 + >>> default_quantity = 1 + >>> (requested_quantity ?? default_quantity) * price + 0 + +The same correct behavior can be achieved with the ternary operator. Here is +an excerpt from the popular Requests package:: + + data = [] if data is None else data + files = [] if files is None else files + headers = {} if headers is None else headers + params = {} if params is None else params + hooks = {} if hooks is None else hooks + +This particular formulation has the undesirable effect of putting the operands +in an unintuitive order: the brain thinks, "use ``data`` if possible and use +``[]`` as a fallback," but the code puts the fallback *before* the preferred +value. + +The author of this package could have written it like this instead:: + + data = data if data is not None else [] + files = files if files is not None else [] + headers = headers if headers is not None else {} + params = params if params is not None else {} + hooks = hooks if hooks is not None else {} + +This ordering of the operands is more intuitive, but it requires 4 extra +characters (for "not "). It also highlights the repetition of identifiers: +``data if data``, ``files if files``, etc. The ``None`` coalescing operator +improves readability:: + + data = data ?? [] + files = files ?? [] + headers = headers ?? {} + params = params ?? {} + hooks = hooks ?? {} + +The ``None`` coalescing operator also has a corresponding assignment shortcut. + +:: + + data ?= [] + files ?= [] + headers ?= {} + params ?= {} + hooks ?= {} + +The ``None`` coalescing operator is left-associative, which allows for easy +chaining:: + + >>> user_title = None + >>> local_default_title = None + >>> global_default_title = 'Global Default Title' + >>> title = user_title ?? local_default_title ?? global_default_title + 'Global Default Title' + +The direction of associativity is important because the ``None`` coalescing +operator short circuits: if its left operand is non-null, then the right +operand is not evaluated. + +:: + + >>> def get_default(): raise Exception() + >>> 'My Title' ?? get_default() + 'My Title' + + +Null-Aware Member Access Operator +--------------------------------- + +:: + + >>> title = 'My Title' + >>> title.upper() + 'MY TITLE' + >>> title = None + >>> title.upper() + Traceback (most recent call last): + File "", line 1, in + AttributeError: 'NoneType' object has no attribute 'upper' + >>> title?.upper() + None + + +Null-Aware Index Access Operator +--------------------------------- + +:: + + >>> person = {'name': 'Mark', 'age': 32} + >>> person['name'] + 'Mark' + >>> person = None + >>> person['name'] + Traceback (most recent call last): + File "", line 1, in + TypeError: 'NoneType' object is not subscriptable + >>> person?['name'] + None + + +Specification +============= + + +References +========== + +.. [1] Wikipedia: Null coalescing operator + (https://en.wikipedia.org/wiki/Null_coalescing_operator) + +.. [2] Seth Ladd's Blog: Null-aware operators in Dart + (http://blog.sethladd.com/2015/07/null-aware-operators-in-dart.html) + + +Copyright +========= + +This document has been placed in the public domain. + + + +.. + Local Variables: + mode: indented-text + indent-tabs-mode: nil + sentence-end-double-space: t + fill-column: 70 + coding: utf-8 + End: diff --git a/pep-0506.txt b/pep-0506.txt new file mode 100644 --- /dev/null +++ b/pep-0506.txt @@ -0,0 +1,356 @@ +PEP: 506 +Title: Adding A Secrets Module To The Standard Library +Version: $Revision$ +Last-Modified: $Date$ +Author: Steven D'Aprano +Status: Draft +Type: Standards Track +Content-Type: text/x-rst +Created: 19-Sep-2015 +Python-Version: 3.6 +Post-History: + + +Abstract +======== + +This PEP proposes the addition of a module for common security-related +functions such as generating tokens to the Python standard library. + + +Definitions +=========== + +Some common abbreviations used in this proposal: + +* PRNG: + + Pseudo Random Number Generator. A deterministic algorithm used + to produce random-looking numbers with certain desirable + statistical properties. + +* CSPRNG: + + Cryptographically Strong Pseudo Random Number Generator. An + algorithm used to produce random-looking numbers which are + resistant to prediction. + +* MT: + + Mersenne Twister. An extensively studied PRNG which is currently + used by the ``random`` module as the default. + + +Rationale +========= + +This proposal is motivated by concerns that Python's standard library +makes it too easy for developers to inadvertently make serious security +errors. Theo de Raadt, the founder of OpenBSD, contacted Guido van Rossum +and expressed some concern [1]_ about the use of MT for generating sensitive +information such as passwords, secure tokens, session keys and similar. + +Although the documentation for the random module explicitly states that +the default is not suitable for security purposes [2]_, it is strongly +believed that this warning may be missed, ignored or misunderstood by +many Python developers. In particular: + +* developers may not have read the documentation and consequently + not seen the warning; + +* they may not realise that their specific use of it has security + implications; or + +* not realising that there could be a problem, they have copied code + (or learned techniques) from websites which don't offer best + practises. + +The first [3]_ hit when searching for "python how to generate passwords" on +Google is a tutorial that uses the default functions from the ``random`` +module [4]_. Although it is not intended for use in web applications, it is +likely that similar techniques find themselves used in that situation. +The second hit is to a StackOverflow question about generating +passwords [5]_. Most of the answers given, including the accepted one, use +the default functions. When one user warned that the default could be +easily compromised, they were told "I think you worry too much." [6]_ + +This strongly suggests that the existing ``random`` module is an attractive +nuisance when it comes to generating (for example) passwords or secure +tokens. + +Additional motivation (of a more philosophical bent) can be found in the +post which first proposed this idea [7]_. + + +Proposal +======== + +Alternative proposals have focused on the default PRNG in the ``random`` +module, with the aim of providing "secure by default" cryptographically +strong primitives that developers can build upon without thinking about +security. (See Alternatives below.) This proposes a different approach: + +* The standard library already provides cryptographically strong + primitives, but many users don't know they exist or when to use them. + +* Instead of requiring crypto-naive users to write secure code, the + standard library should include a set of ready-to-use "batteries" for + the most common needs, such as generating secure tokens. This code + will both directly satisfy a need ("How do I generate a password reset + token?"), and act as an example of acceptable practises which + developers can learn from [8]_. + +To do this, this PEP proposes that we add a new module to the standard +library, with the suggested name ``secrets``. This module will contain a +set of ready-to-use functions for common activities with security +implications, together with some lower-level primitives. + +The suggestion is that ``secrets`` becomes the go-to module for dealing +with anything which should remain secret (passwords, tokens, etc.) +while the ``random`` module remains backward-compatible. + + +API and Implementation +====================== + +The contents of the ``secrets`` module is expected to evolve over time, and +likely will evolve between the time of writing this PEP and actual release +in the standard library [9]_. At the time of writing, the following functions +have been suggested: + +* A high-level function for generating secure tokens suitable for use + in (e.g.) password recovery, as session keys, etc. + +* A limited interface to the system CSPRNG, using either ``os.urandom`` + directly or ``random.SystemRandom``. Unlike the ``random`` module, this + does not need to provide methods for seeding, getting or setting the + state, or any non-uniform distributions. It should provide the + following: + + - A function for choosing items from a sequence, ``secrets.choice``. + - A function for generating an integer within some range, such as + ``secrets.randrange`` or ``secrets.randint``. + - A function for generating a given number of random bits and/or bytes + as an integer. + - A similar function which returns the value as a hex digit string. + +* ``hmac.compare_digest`` under the name ``equal``. + +The consensus appears to be that there is no need to add a new CSPRNG to +the ``random`` module to support these uses, ``SystemRandom`` will be +sufficient. + +Some illustrative implementations have been given by Nick Coghlan [10]_. +This idea has also been discussed on the issue tracker for the +"cryptography" module [11]_. + +The ``secrets`` module itself will be pure Python, and other Python +implementations can easily make use of it unchanged, or adapt it as +necessary. + + +Alternatives +============ + +One alternative is to change the default PRNG provided by the ``random`` +module [12]_. This received considerable scepticism and outright opposition: + +* There is fear that a CSPRNG may be slower than the current PRNG (which + in the case of MT is already quite slow). + +* Some applications (such as scientific simulations, and replaying + gameplay) require the ability to seed the PRNG into a known state, + which a CSPRNG lacks by design. + +* Another major use of the ``random`` module is for simple "guess a number" + games written by beginners, and many people are loath to make any + change to the ``random`` module which may make that harder. + +* Although there is no proposal to remove MT from the ``random`` module, + there was considerable hostility to the idea of having to opt-in to + a non-CSPRNG or any backwards-incompatible changes. + +* Demonstrated attacks against MT are typically against PHP applications. + It is believed that PHP's version of MT is a significantly softer target + than Python's version, due to a poor seeding technique [13]_. Consequently, + without a proven attack against Python applications, many people object + to a backwards-incompatible change. + +Nick Coghlan made an earlier suggestion for a globally configurable PRNG +which uses the system CSPRNG by default [14]_, but has since hinted that he +may withdraw it in favour of this proposal [15]_. + + +Comparison To Other Languages +============================= + +* PHP + + PHP includes a function ``uniqid`` [16]_ which by default returns a + thirteen character string based on the current time in microseconds. + Translated into Python syntax, it has the following signature:: + + def uniqid(prefix='', more_entropy=False)->str + + The PHP documentation warns that this function is not suitable for + security purposes. Nevertheless, various mature, well-known PHP + applications use it for that purpose (citation needed). + + PHP 5.3 and better also includes a function ``openssl_random_pseudo_bytes`` + [17]_. Translated into Python syntax, it has roughly the following + signature:: + + def openssl_random_pseudo_bytes(length:int)->Tuple[str, bool] + + This function returns a pseudo-random string of bytes of the given + length, and an boolean flag giving whether the string is considered + cryptographically strong. The PHP manual suggests that returning + anything but True should be rare except for old or broken platforms. + +* Javascript + + Based on a rather cursory search [18]_, there doesn't appear to be any + well-known standard functions for producing strong random values in + Javascript, although there may be good quality third-party libraries. + Standard Javascript doesn't seem to include an interface to the + system CSPRNG either, and people have extensively written about the + weaknesses of Javascript's ``Math.random`` [19]_. + +* Ruby + + The Ruby standard library includes a module ``SecureRandom`` [20]_ + which includes the following methods: + + * base64 - returns a Base64 encoded random string. + + * hex - returns a random hexadecimal string. + + * random_bytes - returns a random byte string. + + * random_number - depending on the argument, returns either a random + integer in the range(0, n), or a random float between 0.0 and 1.0. + + * urlsafe_base64 - returns a random URL-safe Base64 encoded string. + + * uuid - return a version 4 random Universally Unique IDentifier. + + +What Should Be The Name Of The Module? +====================================== + +There was a proposal to add a "random.safe" submodule, quoting the Zen +of Python "Namespaces are one honking great idea" koan. However, the +author of the Zen, Tim Peters, has come out against this idea [21]_, and +recommends a top-level module. + +In discussion on the python-ideas mailing list so far, the name "secrets" +has received some approval, and no strong opposition. + + +Frequently Asked Questions +========================== + +* Q: Is this a real problem? Surely MT is random enough that nobody can + predict its output. + + A: The consensus among security professionals is that MT is not safe + in security contexts. It is not difficult to reconstruct the internal + state of MT [22]_ [23]_ and so predict all past and future values. There + are a number of known, practical attacks on systems using MT for + randomness [24]_. + + While there are currently no known direct attacks on applications + written in Python due to the use of MT, there is widespread agreement + that such usage is unsafe. + +* Q: Is this an alternative to specialise cryptographic software such as SSL? + + A: No. This is a "batteries included" solution, not a full-featured + "nuclear reactor". It is intended to mitigate against some basic + security errors, not be a solution to all security-related issues. To + quote Nick Coghlan referring to his earlier proposal [25]_:: + + "...folks really are better off learning to use things like + cryptography.io for security sensitive software, so this change + is just about harm mitigation given that it's inevitable that a + non-trivial proportion of the millions of current and future + Python developers won't do that." + + +References +========== + +.. [1] https://mail.python.org/pipermail/python-ideas/2015-September/035820.html + +.. [2] https://docs.python.org/3/library/random.html + +.. [3] As of the date of writing. Also, as Google search terms may be + automatically customised for the user without their knowledge, some + readers may see different results. + +.. [4] http://interactivepython.org/runestone/static/everyday/2013/01/3_password.html + +.. [5] http://stackoverflow.com/questions/3854692/generate-password-in-python + +.. [6] http://stackoverflow.com/questions/3854692/generate-password-in-python/3854766#3854766 + +.. [7] https://mail.python.org/pipermail/python-ideas/2015-September/036238.html + +.. [8] At least those who are motivated to read the source code and documentation. + +.. [9] Tim Peters suggests that bike-shedding the contents of the module will + be 10000 times more time consuming than actually implementing the + module. Words do not begin to express how much I am looking forward to + this. + +.. [10] https://mail.python.org/pipermail/python-ideas/2015-September/036271.html + +.. [11] https://github.com/pyca/cryptography/issues/2347 + +.. [12] Link needed. + +.. [13] By default PHP seeds the MT PRNG with the time (citation needed), + which is exploitable by attackers, while Python seeds the PRNG with + output from the system CSPRNG, which is believed to be much harder to + exploit. + +.. [14] http://legacy.python.org/dev/peps/pep-0504/ + +.. [15] https://mail.python.org/pipermail/python-ideas/2015-September/036243.html + +.. [16] http://php.net/manual/en/function.uniqid.php + +.. [17] http://php.net/manual/en/function.openssl-random-pseudo-bytes.php + +.. [18] Volunteers and patches are welcome. + +.. [19] http://ifsec.blogspot.fr/2012/05/cross-domain-mathrandom-prediction.html + +.. [20] http://ruby-doc.org/stdlib-2.1.2/libdoc/securerandom/rdoc/SecureRandom.html + +.. [21] https://mail.python.org/pipermail/python-ideas/2015-September/036254.html + +.. [22] https://jazzy.id.au/2010/09/22/cracking_random_number_generators_part_3.html + +.. [23] https://mail.python.org/pipermail/python-ideas/2015-September/036077.html + +.. [24] https://media.blackhat.com/bh-us-12/Briefings/Argyros/BH_US_12_Argyros_PRNG_WP.pdf + +.. [25] https://mail.python.org/pipermail/python-ideas/2015-September/036157.html + + +Copyright +========= + +This document has been placed in the public domain. + + + +.. + Local Variables: + mode: indented-text + indent-tabs-mode: nil + sentence-end-double-space: t + fill-column: 70 + coding: utf-8 + End: diff --git a/pep-3140.txt b/pep-3140.txt --- a/pep-3140.txt +++ b/pep-3140.txt @@ -2,7 +2,7 @@ Title: str(container) should call str(item), not repr(item) Version: $Revision$ Last-Modified: $Date$ -Author: Oleg Broytmann , +Author: Oleg Broytman , Jim J. Jewett Discussions-To: python-3000 at python.org Status: Rejected -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Mon Sep 21 02:50:54 2015 From: python-checkins at python.org (alexander.belopolsky) Date: Mon, 21 Sep 2015 00:50:54 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_PEP_495=3A_Give_up_waiting_fo?= =?utf-8?q?r_sidebar_rendering_to_be_fixed=2E?= Message-ID: <20150921005054.115545.98761@psf.io> https://hg.python.org/peps/rev/835e2af8c0cc changeset: 6079:835e2af8c0cc user: Alexander Belopolsky date: Sun Sep 20 20:50:45 2015 -0400 summary: PEP 495: Give up waiting for sidebar rendering to be fixed. files: pep-0495.txt | 11 +++++------ 1 files changed, 5 insertions(+), 6 deletions(-) diff --git a/pep-0495.txt b/pep-0495.txt --- a/pep-0495.txt +++ b/pep-0495.txt @@ -21,12 +21,6 @@ with 0 corresponding to the earlier and 1 to the later of the two possible readings of an ambiguous local time. -.. sidebar:: US public service advertisement - - .. image:: pep-0495-daylightsavings.png - :align: center - :width: 95% - Rationale ========= @@ -40,6 +34,11 @@ attribute to the ``datetime`` instances taking values of 0 and 1 that will enumerate the two ambiguous times. +.. image:: pep-0495-daylightsavings.png + :align: center + :width: 30% + + .. [#] People who live in locations observing the Daylight Saving Time (DST) move their clocks back (usually one hour) every Fall. -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Mon Sep 21 02:53:05 2015 From: python-checkins at python.org (alexander.belopolsky) Date: Mon, 21 Sep 2015 00:53:05 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_PEP_495=3A_Removed_emacs-gene?= =?utf-8?q?rated_artifact=2E?= Message-ID: <20150921005305.81629.78764@psf.io> https://hg.python.org/peps/rev/9eb4d939920c changeset: 6080:9eb4d939920c user: Alexander Belopolsky date: Sun Sep 20 20:53:01 2015 -0400 summary: PEP 495: Removed emacs-generated artifact. files: pep-0495.txt | 2 -- 1 files changed, 0 insertions(+), 2 deletions(-) diff --git a/pep-0495.txt b/pep-0495.txt --- a/pep-0495.txt +++ b/pep-0495.txt @@ -800,5 +800,3 @@ employee, taken or made as part of that person's official duties. As a work of the U.S. federal government, the image is in the public domain. - - LocalWords: isdst Py tm -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Mon Sep 21 02:57:18 2015 From: python-checkins at python.org (alexander.belopolsky) Date: Mon, 21 Sep 2015 00:57:18 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_PEP_495=3A_Attempt_to_fix_the?= =?utf-8?q?_fold_sketch_by_renaming_the_image_file=2E?= Message-ID: <20150921005718.82660.60542@psf.io> https://hg.python.org/peps/rev/dbbae86648df changeset: 6081:dbbae86648df user: Alexander Belopolsky date: Sun Sep 20 20:57:14 2015 -0400 summary: PEP 495: Attempt to fix the fold sketch by renaming the image file. files: pep-0495-fold-2.png | 0 pep-0495.txt | 2 +- 2 files changed, 1 insertions(+), 1 deletions(-) diff --git a/pep-0495-fold.png b/pep-0495-fold-2.png rename from pep-0495-fold.png rename to pep-0495-fold-2.png diff --git a/pep-0495.txt b/pep-0495.txt --- a/pep-0495.txt +++ b/pep-0495.txt @@ -342,7 +342,7 @@ ``fromutc(u2)`` will return an instance with ``fold=1``. In all other cases the returned instance should have ``fold=0``. -.. image:: pep-0495-fold.png +.. image:: pep-0495-fold-2.png :align: center :width: 60% -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Mon Sep 21 04:42:49 2015 From: python-checkins at python.org (alexander.belopolsky) Date: Mon, 21 Sep 2015 02:42:49 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_PEP_495=3A_Rewrote_Guidelines?= =?utf-8?q?_for_New_tzinfo_Implementations=2E?= Message-ID: <20150921024249.16575.19312@psf.io> https://hg.python.org/peps/rev/1fa865cbe398 changeset: 6082:1fa865cbe398 user: Alexander Belopolsky date: Sun Sep 20 22:42:43 2015 -0400 summary: PEP 495: Rewrote Guidelines for New tzinfo Implementations. files: pep-0495.txt | 134 +++++++++++++++++++++++++------------- 1 files changed, 89 insertions(+), 45 deletions(-) diff --git a/pep-0495.txt b/pep-0495.txt --- a/pep-0495.txt +++ b/pep-0495.txt @@ -314,7 +314,7 @@ used anywhere in the stdlib because the only included ``tzinfo`` implementation (the ``datetime.timzeone`` class implementing fixed offset timezones) override ``fromutc()``. - + Guidelines for New tzinfo Implementations ========================================= @@ -332,20 +332,102 @@ the ambiguous or missing times. -In the DST Fold ---------------- +In the Fold +----------- New subclasses should override the base-class ``fromutc()`` method and -implement it so that in all cases where two UTC times ``u1`` and -``u2`` (``u1`` <``u2``) correspond to the same local time -``fromutc(u1)`` will return an instance with ``fold=0`` and -``fromutc(u2)`` will return an instance with ``fold=1``. In all +implement it so that in all cases where two UTC times ``u0`` and +``u1`` (``u0`` <``u1``) correspond to the same local time ``t``, +``fromutc(u0)`` will return an instance with ``fold=0`` and +``fromutc(u1)`` will return an instance with ``fold=1``. In all other cases the returned instance should have ``fold=0``. +The ``utcoffset()``, ``tzname()`` and ``dst()`` methods should use the +value of the fold attribute to determine whether an otherwise +ambiguous time ``t`` corresponds to the time before or after the +transition. By definition, ``utcoffset()`` is greater before and +smaller after any transition that creates a fold. The values returned +by ``tzname()`` and ``dst()`` may or may not depend on the value of +the fold attribute depending on the kind of the transition. + .. image:: pep-0495-fold-2.png :align: center :width: 60% +The sketch above illustrates the relationship between the UTC and +local time around a fall-back transition. The zig-zag line is a graph +of the function implemented by ``fromutc()``. Two intervals on the +UTC axis adjacent to the transition point and having the size of the +time shift at the transition are mapped to the same interval on the +local axis. New implementations of ``fromutc()`` method should set +the fold attribute to 1 when ``self`` is in the region marked in +yellow on the UTC axis. (All intervals should be treated as closed on +the left and open on the right.) + + +Mind the Gap +------------ + +The ``fromutc()`` method should never produce a time in the gap. + +If ``utcoffset()``, ``tzname()`` or ``dst()`` method is called on a +local time that falls in a gap, the rules in effect before the +transition should be used if ``fold=0``. Otherwise, the rules in +effect after the transition should be used. + +.. image:: pep-0495-gap.png + :align: center + :width: 60% + +The sketch above illustrates the relationship between the UTC and +local time around a spring-forward transition. At the transition, the +local clock is advanced skipping the times in the gap. For the +purposes of determining the values of ``utcoffset()``, ``tzname()`` +and ``dst()``, the line before the transition is extended forward to +find the UTC time corresponding to the time in the gap with ``fold=0`` +and for instances with ``fold=1``, the line after the transition is +extended back. + +Summary of Rules at a Transition +-------------------------------- + +On ambiguous/missing times ``utcoffset()`` should return values +according to the following table: + ++-----------------+----------------+-----------------------------+ +| | fold=0 | fold=1 | ++=================+================+=============================+ +| Fold | oldoff | newoff = oldoff - delta | ++-----------------+----------------+-----------------------------+ +| Gap | oldoff | newoff = oldoff + delta | ++-----------------+----------------+-----------------------------+ + +where ``oldoff`` (``newoff``) is the UTC offset before (after) the +transition and ``delta`` is the absolute size of the fold or the gap. + +Note that the interpretation of the fold attribute is consistent in +the fold and gap cases. In both cases, ``fold=0`` (``fold=1``) means +use ``fromutc()`` line before (after) the transition to find the UTC +time. Only in the "Fold" case, the UTC times ``u0`` and ``u1`` are +"real" solutions for the equation ``fromutc(u) == t``, while in the +"Gap" case they are "imaginary" solutions. + + +The DST Transitions +------------------- + +On a missing time introduced at the start of DST, the values returned +by ``utcoffset()`` and ``dst()`` methods should be as follows + ++-----------------+----------------+------------------+ +| | fold=0 | fold=1 | ++=================+================+==================+ +| utcoffset() | stdoff | stdoff + dstoff | ++-----------------+----------------+------------------+ +| dst() | zero | dstoff | ++-----------------+----------------+------------------+ + + On an ambiguous time introduced at the end of DST, the values returned by ``utcoffset()`` and ``dst()`` methods should be as follows @@ -362,44 +444,6 @@ = timedelta(0)``. -Mind the DST Gap ----------------- - -.. image:: pep-0495-gap.png - :align: center - :width: 60% - -On a missing time introduced at the start of DST, the values returned -by ``utcoffset()`` and ``dst()`` methods should be as follows - -+-----------------+----------------+------------------+ -| | fold=0 | fold=1 | -+=================+================+==================+ -| utcoffset() | stdoff | stdoff + dstoff | -+-----------------+----------------+------------------+ -| dst() | zero | dstoff | -+-----------------+----------------+------------------+ - - -Non-DST Folds and Gaps ----------------------- - -On ambiguous/missing times introduced by the change in the standard time -offset, the ``dst()`` method should return the same value regardless of -the value of ``fold`` and the ``utcoffset()`` should return values -according to the following table: - -+-----------------+----------------+-----------------------------+ -| | fold=0 | fold=1 | -+=================+================+=============================+ -| ambiguous | oldoff | newoff = oldoff - delta | -+-----------------+----------------+-----------------------------+ -| missing | oldoff | newoff = oldoff + delta | -+-----------------+----------------+-----------------------------+ - -where ``delta`` is the size of the fold or the gap. - - Temporal Arithmetic and Comparison Operators ============================================ -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Mon Sep 21 04:56:23 2015 From: python-checkins at python.org (terry.reedy) Date: Mon, 21 Sep 2015 02:56:23 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Merge_with_3=2E4?= Message-ID: <20150921025623.9939.9582@psf.io> https://hg.python.org/cpython/rev/84d500faf13c changeset: 98102:84d500faf13c branch: 3.5 parent: 98097:f4504ff121cc parent: 98101:a2e782188db6 user: Terry Jan Reedy date: Sun Sep 20 22:55:51 2015 -0400 summary: Merge with 3.4 files: Lib/idlelib/EditorWindow.py | 10 ++++++++-- Lib/idlelib/macosxSupport.py | 7 ++++++- 2 files changed, 14 insertions(+), 3 deletions(-) diff --git a/Lib/idlelib/EditorWindow.py b/Lib/idlelib/EditorWindow.py --- a/Lib/idlelib/EditorWindow.py +++ b/Lib/idlelib/EditorWindow.py @@ -537,16 +537,22 @@ return 'normal' def about_dialog(self, event=None): + "Handle Help 'About IDLE' event." + # Synchronize with macosxSupport.overrideRootMenu.about_dialog. aboutDialog.AboutDialog(self.top,'About IDLE') def config_dialog(self, event=None): + "Handle Options 'Configure IDLE' event." + # Synchronize with macosxSupport.overrideRootMenu.config_dialog. configDialog.ConfigDialog(self.top,'Settings') + def config_extensions_dialog(self, event=None): + "Handle Options 'Configure Extensions' event." configDialog.ConfigExtensionsDialog(self.top) def help_dialog(self, event=None): - "Handle help doc event." - # edit maxosxSupport.overrideRootMenu.help_dialog to match + "Handle Help 'IDLE Help' event." + # Synchronize with macosxSupport.overrideRootMenu.help_dialog. if self.root: parent = self.root else: diff --git a/Lib/idlelib/macosxSupport.py b/Lib/idlelib/macosxSupport.py --- a/Lib/idlelib/macosxSupport.py +++ b/Lib/idlelib/macosxSupport.py @@ -159,10 +159,14 @@ WindowList.register_callback(postwindowsmenu) def about_dialog(event=None): + "Handle Help 'About IDLE' event." + # Synchronize with EditorWindow.EditorWindow.about_dialog. from idlelib import aboutDialog aboutDialog.AboutDialog(root, 'About IDLE') def config_dialog(event=None): + "Handle Options 'Configure IDLE' event." + # Synchronize with EditorWindow.EditorWindow.config_dialog. from idlelib import configDialog # Ensure that the root object has an instance_dict attribute, @@ -170,10 +174,11 @@ # on an EditorWindow instance that is then passed as the first # argument to ConfigDialog) root.instance_dict = flist.inversedict - root.instance_dict = flist.inversedict configDialog.ConfigDialog(root, 'Settings') def help_dialog(event=None): + "Handle Help 'IDLE Help' event." + # Synchronize with EditorWindow.EditorWindow.help_dialog. from idlelib import help help.show_idlehelp(root) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 21 04:56:23 2015 From: python-checkins at python.org (terry.reedy) Date: Mon, 21 Sep 2015 02:56:23 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzI1MTk5?= =?utf-8?q?=3A_Idle=3A_add_synchronization_comments_for_future_maintainers?= =?utf-8?q?=2E?= Message-ID: <20150921025623.98370.79872@psf.io> https://hg.python.org/cpython/rev/884f15dc26f0 changeset: 98100:884f15dc26f0 branch: 2.7 user: Terry Jan Reedy date: Sun Sep 20 22:55:17 2015 -0400 summary: Issue #25199: Idle: add synchronization comments for future maintainers. files: Lib/idlelib/EditorWindow.py | 10 ++++++++-- Lib/idlelib/macosxSupport.py | 6 ++++++ 2 files changed, 14 insertions(+), 2 deletions(-) diff --git a/Lib/idlelib/EditorWindow.py b/Lib/idlelib/EditorWindow.py --- a/Lib/idlelib/EditorWindow.py +++ b/Lib/idlelib/EditorWindow.py @@ -564,16 +564,22 @@ return 'normal' def about_dialog(self, event=None): + "Handle Help 'About IDLE' event." + # Synchronize with macosxSupport.overrideRootMenu.about_dialog. aboutDialog.AboutDialog(self.top,'About IDLE') def config_dialog(self, event=None): + "Handle Options 'Configure IDLE' event." + # Synchronize with macosxSupport.overrideRootMenu.config_dialog. configDialog.ConfigDialog(self.top,'Settings') + def config_extensions_dialog(self, event=None): + "Handle Options 'Configure Extensions' event." configDialog.ConfigExtensionsDialog(self.top) def help_dialog(self, event=None): - "Handle help doc event." - # edit maxosxSupport.overrideRootMenu.help_dialog to match + "Handle Help 'IDLE Help' event." + # Synchronize with macosxSupport.overrideRootMenu.help_dialog. if self.root: parent = self.root else: diff --git a/Lib/idlelib/macosxSupport.py b/Lib/idlelib/macosxSupport.py --- a/Lib/idlelib/macosxSupport.py +++ b/Lib/idlelib/macosxSupport.py @@ -161,15 +161,21 @@ WindowList.register_callback(postwindowsmenu) def about_dialog(event=None): + "Handle Help 'About IDLE' event." + # Synchronize with EditorWindow.EditorWindow.about_dialog. from idlelib import aboutDialog aboutDialog.AboutDialog(root, 'About IDLE') def config_dialog(event=None): + "Handle Options 'Configure IDLE' event." + # Synchronize with EditorWindow.EditorWindow.config_dialog. from idlelib import configDialog root.instance_dict = flist.inversedict configDialog.ConfigDialog(root, 'Settings') def help_dialog(event=None): + "Handle Help 'IDLE Help' event." + # Synchronize with EditorWindow.EditorWindow.help_dialog. from idlelib import help help.show_idlehelp(root) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 21 04:56:24 2015 From: python-checkins at python.org (terry.reedy) Date: Mon, 21 Sep 2015 02:56:24 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzI1MTk5?= =?utf-8?q?=3A_Idle=3A_add_synchronization_comments_for_future_maintainers?= =?utf-8?q?=2E?= Message-ID: <20150921025623.115327.77263@psf.io> https://hg.python.org/cpython/rev/a2e782188db6 changeset: 98101:a2e782188db6 branch: 3.4 parent: 98096:9b79a4901069 user: Terry Jan Reedy date: Sun Sep 20 22:55:39 2015 -0400 summary: Issue #25199: Idle: add synchronization comments for future maintainers. files: Lib/idlelib/EditorWindow.py | 10 ++++++++-- Lib/idlelib/macosxSupport.py | 7 ++++++- 2 files changed, 14 insertions(+), 3 deletions(-) diff --git a/Lib/idlelib/EditorWindow.py b/Lib/idlelib/EditorWindow.py --- a/Lib/idlelib/EditorWindow.py +++ b/Lib/idlelib/EditorWindow.py @@ -537,16 +537,22 @@ return 'normal' def about_dialog(self, event=None): + "Handle Help 'About IDLE' event." + # Synchronize with macosxSupport.overrideRootMenu.about_dialog. aboutDialog.AboutDialog(self.top,'About IDLE') def config_dialog(self, event=None): + "Handle Options 'Configure IDLE' event." + # Synchronize with macosxSupport.overrideRootMenu.config_dialog. configDialog.ConfigDialog(self.top,'Settings') + def config_extensions_dialog(self, event=None): + "Handle Options 'Configure Extensions' event." configDialog.ConfigExtensionsDialog(self.top) def help_dialog(self, event=None): - "Handle help doc event." - # edit maxosxSupport.overrideRootMenu.help_dialog to match + "Handle Help 'IDLE Help' event." + # Synchronize with macosxSupport.overrideRootMenu.help_dialog. if self.root: parent = self.root else: diff --git a/Lib/idlelib/macosxSupport.py b/Lib/idlelib/macosxSupport.py --- a/Lib/idlelib/macosxSupport.py +++ b/Lib/idlelib/macosxSupport.py @@ -159,10 +159,14 @@ WindowList.register_callback(postwindowsmenu) def about_dialog(event=None): + "Handle Help 'About IDLE' event." + # Synchronize with EditorWindow.EditorWindow.about_dialog. from idlelib import aboutDialog aboutDialog.AboutDialog(root, 'About IDLE') def config_dialog(event=None): + "Handle Options 'Configure IDLE' event." + # Synchronize with EditorWindow.EditorWindow.config_dialog. from idlelib import configDialog # Ensure that the root object has an instance_dict attribute, @@ -170,10 +174,11 @@ # on an EditorWindow instance that is then passed as the first # argument to ConfigDialog) root.instance_dict = flist.inversedict - root.instance_dict = flist.inversedict configDialog.ConfigDialog(root, 'Settings') def help_dialog(event=None): + "Handle Help 'IDLE Help' event." + # Synchronize with EditorWindow.EditorWindow.help_dialog. from idlelib import help help.show_idlehelp(root) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 21 04:56:28 2015 From: python-checkins at python.org (terry.reedy) Date: Mon, 21 Sep 2015 02:56:28 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge_with_3=2E5?= Message-ID: <20150921025628.81647.30532@psf.io> https://hg.python.org/cpython/rev/33d51f2ad66f changeset: 98103:33d51f2ad66f parent: 98098:c0e9ca4254ef parent: 98102:84d500faf13c user: Terry Jan Reedy date: Sun Sep 20 22:56:03 2015 -0400 summary: Merge with 3.5 files: Lib/idlelib/EditorWindow.py | 10 ++++++++-- Lib/idlelib/macosxSupport.py | 7 ++++++- 2 files changed, 14 insertions(+), 3 deletions(-) diff --git a/Lib/idlelib/EditorWindow.py b/Lib/idlelib/EditorWindow.py --- a/Lib/idlelib/EditorWindow.py +++ b/Lib/idlelib/EditorWindow.py @@ -537,16 +537,22 @@ return 'normal' def about_dialog(self, event=None): + "Handle Help 'About IDLE' event." + # Synchronize with macosxSupport.overrideRootMenu.about_dialog. aboutDialog.AboutDialog(self.top,'About IDLE') def config_dialog(self, event=None): + "Handle Options 'Configure IDLE' event." + # Synchronize with macosxSupport.overrideRootMenu.config_dialog. configDialog.ConfigDialog(self.top,'Settings') + def config_extensions_dialog(self, event=None): + "Handle Options 'Configure Extensions' event." configDialog.ConfigExtensionsDialog(self.top) def help_dialog(self, event=None): - "Handle help doc event." - # edit maxosxSupport.overrideRootMenu.help_dialog to match + "Handle Help 'IDLE Help' event." + # Synchronize with macosxSupport.overrideRootMenu.help_dialog. if self.root: parent = self.root else: diff --git a/Lib/idlelib/macosxSupport.py b/Lib/idlelib/macosxSupport.py --- a/Lib/idlelib/macosxSupport.py +++ b/Lib/idlelib/macosxSupport.py @@ -159,10 +159,14 @@ WindowList.register_callback(postwindowsmenu) def about_dialog(event=None): + "Handle Help 'About IDLE' event." + # Synchronize with EditorWindow.EditorWindow.about_dialog. from idlelib import aboutDialog aboutDialog.AboutDialog(root, 'About IDLE') def config_dialog(event=None): + "Handle Options 'Configure IDLE' event." + # Synchronize with EditorWindow.EditorWindow.config_dialog. from idlelib import configDialog # Ensure that the root object has an instance_dict attribute, @@ -170,10 +174,11 @@ # on an EditorWindow instance that is then passed as the first # argument to ConfigDialog) root.instance_dict = flist.inversedict - root.instance_dict = flist.inversedict configDialog.ConfigDialog(root, 'Settings') def help_dialog(event=None): + "Handle Help 'IDLE Help' event." + # Synchronize with EditorWindow.EditorWindow.help_dialog. from idlelib import help help.show_idlehelp(root) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 21 05:06:13 2015 From: python-checkins at python.org (terry.reedy) Date: Mon, 21 Sep 2015 03:06:13 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge_with_3=2E5?= Message-ID: <20150921030613.11712.92951@psf.io> https://hg.python.org/cpython/rev/cc3e23297db7 changeset: 98107:cc3e23297db7 parent: 98103:33d51f2ad66f parent: 98106:1f602d6046d0 user: Terry Jan Reedy date: Sun Sep 20 23:05:52 2015 -0400 summary: Merge with 3.5 files: Lib/idlelib/EditorWindow.py | 4 +--- 1 files changed, 1 insertions(+), 3 deletions(-) diff --git a/Lib/idlelib/EditorWindow.py b/Lib/idlelib/EditorWindow.py --- a/Lib/idlelib/EditorWindow.py +++ b/Lib/idlelib/EditorWindow.py @@ -85,9 +85,7 @@ self.dlg = None self.parent = None -helpDialog = HelpDialog() # singleton instance -def _help_dialog(parent): # wrapper for htest - helpDialog.show_dialog(parent) +helpDialog = HelpDialog() # singleton instance, no longer used class EditorWindow(object): -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 21 05:06:12 2015 From: python-checkins at python.org (terry.reedy) Date: Mon, 21 Sep 2015 03:06:12 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzE2ODkz?= =?utf-8?q?=3A_finish_deprecation=2E?= Message-ID: <20150921030612.31187.56508@psf.io> https://hg.python.org/cpython/rev/e749080fa0f9 changeset: 98104:e749080fa0f9 branch: 2.7 parent: 98100:884f15dc26f0 user: Terry Jan Reedy date: Sun Sep 20 23:05:21 2015 -0400 summary: Issue #16893: finish deprecation. files: Lib/idlelib/EditorWindow.py | 4 +--- 1 files changed, 1 insertions(+), 3 deletions(-) diff --git a/Lib/idlelib/EditorWindow.py b/Lib/idlelib/EditorWindow.py --- a/Lib/idlelib/EditorWindow.py +++ b/Lib/idlelib/EditorWindow.py @@ -114,9 +114,7 @@ self.dlg = None self.parent = None -helpDialog = HelpDialog() # singleton instance -def _help_dialog(parent): # wrapper for htest - helpDialog.show_dialog(parent) +helpDialog = HelpDialog() # singleton instance, no longer used class EditorWindow(object): -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 21 05:06:13 2015 From: python-checkins at python.org (terry.reedy) Date: Mon, 21 Sep 2015 03:06:13 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzE2ODkz?= =?utf-8?q?=3A_finish_deprecation=2E?= Message-ID: <20150921030612.31191.75565@psf.io> https://hg.python.org/cpython/rev/ff0270e9bdfb changeset: 98105:ff0270e9bdfb branch: 3.4 parent: 98101:a2e782188db6 user: Terry Jan Reedy date: Sun Sep 20 23:05:25 2015 -0400 summary: Issue #16893: finish deprecation. files: Lib/idlelib/EditorWindow.py | 4 +--- 1 files changed, 1 insertions(+), 3 deletions(-) diff --git a/Lib/idlelib/EditorWindow.py b/Lib/idlelib/EditorWindow.py --- a/Lib/idlelib/EditorWindow.py +++ b/Lib/idlelib/EditorWindow.py @@ -85,9 +85,7 @@ self.dlg = None self.parent = None -helpDialog = HelpDialog() # singleton instance -def _help_dialog(parent): # wrapper for htest - helpDialog.show_dialog(parent) +helpDialog = HelpDialog() # singleton instance, no longer used class EditorWindow(object): -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 21 05:06:13 2015 From: python-checkins at python.org (terry.reedy) Date: Mon, 21 Sep 2015 03:06:13 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Merge_with_3=2E4?= Message-ID: <20150921030612.98352.64779@psf.io> https://hg.python.org/cpython/rev/1f602d6046d0 changeset: 98106:1f602d6046d0 branch: 3.5 parent: 98102:84d500faf13c parent: 98105:ff0270e9bdfb user: Terry Jan Reedy date: Sun Sep 20 23:05:41 2015 -0400 summary: Merge with 3.4 files: Lib/idlelib/EditorWindow.py | 4 +--- 1 files changed, 1 insertions(+), 3 deletions(-) diff --git a/Lib/idlelib/EditorWindow.py b/Lib/idlelib/EditorWindow.py --- a/Lib/idlelib/EditorWindow.py +++ b/Lib/idlelib/EditorWindow.py @@ -85,9 +85,7 @@ self.dlg = None self.parent = None -helpDialog = HelpDialog() # singleton instance -def _help_dialog(parent): # wrapper for htest - helpDialog.show_dialog(parent) +helpDialog = HelpDialog() # singleton instance, no longer used class EditorWindow(object): -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 21 05:13:32 2015 From: python-checkins at python.org (berker.peksag) Date: Mon, 21 Sep 2015 03:13:32 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Issue_=2325169=3A_os=2Egetppid=28=29_is_available_on_Windows_s?= =?utf-8?q?ince_Python_3=2E2=2E?= Message-ID: <20150921031332.3656.23147@psf.io> https://hg.python.org/cpython/rev/c6ccef432dc2 changeset: 98109:c6ccef432dc2 branch: 3.5 parent: 98106:1f602d6046d0 parent: 98108:60b0bc23c69e user: Berker Peksag date: Mon Sep 21 06:13:14 2015 +0300 summary: Issue #25169: os.getppid() is available on Windows since Python 3.2. Patch by Bar Harel. files: Doc/library/multiprocessing.rst | 3 +-- 1 files changed, 1 insertions(+), 2 deletions(-) diff --git a/Doc/library/multiprocessing.rst b/Doc/library/multiprocessing.rst --- a/Doc/library/multiprocessing.rst +++ b/Doc/library/multiprocessing.rst @@ -65,8 +65,7 @@ def info(title): print(title) print('module name:', __name__) - if hasattr(os, 'getppid'): # only available on Unix - print('parent process:', os.getppid()) + print('parent process:', os.getppid()) print('process id:', os.getpid()) def f(name): -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 21 05:13:32 2015 From: python-checkins at python.org (berker.peksag) Date: Mon, 21 Sep 2015 03:13:32 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzI1MTY5?= =?utf-8?q?=3A_os=2Egetppid=28=29_is_available_on_Windows_since_Python_3?= =?utf-8?b?LjIu?= Message-ID: <20150921031332.3666.64777@psf.io> https://hg.python.org/cpython/rev/60b0bc23c69e changeset: 98108:60b0bc23c69e branch: 3.4 parent: 98105:ff0270e9bdfb user: Berker Peksag date: Mon Sep 21 06:12:50 2015 +0300 summary: Issue #25169: os.getppid() is available on Windows since Python 3.2. Patch by Bar Harel. files: Doc/library/multiprocessing.rst | 3 +-- 1 files changed, 1 insertions(+), 2 deletions(-) diff --git a/Doc/library/multiprocessing.rst b/Doc/library/multiprocessing.rst --- a/Doc/library/multiprocessing.rst +++ b/Doc/library/multiprocessing.rst @@ -65,8 +65,7 @@ def info(title): print(title) print('module name:', __name__) - if hasattr(os, 'getppid'): # only available on Unix - print('parent process:', os.getppid()) + print('parent process:', os.getppid()) print('process id:', os.getpid()) def f(name): -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 21 05:13:32 2015 From: python-checkins at python.org (berker.peksag) Date: Mon, 21 Sep 2015 03:13:32 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2325169=3A_os=2Egetppid=28=29_is_available_on_Win?= =?utf-8?q?dows_since_Python_3=2E2=2E?= Message-ID: <20150921031332.11712.84954@psf.io> https://hg.python.org/cpython/rev/71a6d4c7cd01 changeset: 98110:71a6d4c7cd01 parent: 98107:cc3e23297db7 parent: 98109:c6ccef432dc2 user: Berker Peksag date: Mon Sep 21 06:13:36 2015 +0300 summary: Issue #25169: os.getppid() is available on Windows since Python 3.2. Patch by Bar Harel. files: Doc/library/multiprocessing.rst | 3 +-- 1 files changed, 1 insertions(+), 2 deletions(-) diff --git a/Doc/library/multiprocessing.rst b/Doc/library/multiprocessing.rst --- a/Doc/library/multiprocessing.rst +++ b/Doc/library/multiprocessing.rst @@ -65,8 +65,7 @@ def info(title): print(title) print('module name:', __name__) - if hasattr(os, 'getppid'): # only available on Unix - print('parent process:', os.getppid()) + print('parent process:', os.getppid()) print('process id:', os.getpid()) def f(name): -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 21 05:18:43 2015 From: python-checkins at python.org (alexander.belopolsky) Date: Mon, 21 Sep 2015 03:18:43 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_PEP_495=3A_Updated_the_Github?= =?utf-8?q?_link=2E?= Message-ID: <20150921031842.81631.25505@psf.io> https://hg.python.org/peps/rev/c6ccc93c9330 changeset: 6084:c6ccc93c9330 user: Alexander Belopolsky date: Sun Sep 20 23:18:38 2015 -0400 summary: PEP 495: Updated the Github link. files: pep-0495.txt | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/pep-0495.txt b/pep-0495.txt --- a/pep-0495.txt +++ b/pep-0495.txt @@ -831,7 +831,7 @@ Implementation ============== -* Github fork: https://github.com/abalkin/cpython +* Github fork: https://github.com/abalkin/cpython/tree/issue24773-s3 * Tracker issue: http://bugs.python.org/issue24773 -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Mon Sep 21 05:18:42 2015 From: python-checkins at python.org (alexander.belopolsky) Date: Mon, 21 Sep 2015 03:18:42 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_PEP_495=3A_New_pickle_format?= =?utf-8?q?=2E?= Message-ID: <20150921031842.3660.37633@psf.io> https://hg.python.org/peps/rev/f39bcfcb6db8 changeset: 6083:f39bcfcb6db8 user: Alexander Belopolsky date: Sun Sep 20 23:17:33 2015 -0400 summary: PEP 495: New pickle format. files: pep-0495.txt | 16 ++++++++++------ 1 files changed, 10 insertions(+), 6 deletions(-) diff --git a/pep-0495.txt b/pep-0495.txt --- a/pep-0495.txt +++ b/pep-0495.txt @@ -288,14 +288,18 @@ Pickles ....... +The value of the fold attribute will only be saved in pickles created +with protocol version 4 (introduced in Python 3.4) or greater. + Pickle sizes for the ``datetime.datetime`` and ``datetime.time`` objects will not change. The ``fold`` value will be encoded in the -first bit of the 5th byte of the ``datetime.datetime`` pickle payload -or the 2nd byte of the datetime.time. In the `current implementation`_ -these bytes are used to store minute value (0-59) and the first bit is -always 0. (This change only affects pickle format. In the C -implementation, the ``fold`` attribute will get a full byte to store its -value.) +first bit of the 3rd (1st) byte of ``datetime.datetime`` +(``datetime.time``) pickle payload. In the `current implementation`_ +these byte are used to store the month (1-12) and hour (0-23) values +and the first bit is always 0. We picked these bytes because they are +the only bytes that are checked by the current unpickle code. Thus +loading post-PEP ``fold=1`` pickles in a pre-PEP Python will result in +an exception rather than an instance with out of range components. .. _current implementation: https://hg.python.org/cpython/file/d3b20bff9c5d/Include/datetime.h#l17 -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Mon Sep 21 05:23:52 2015 From: python-checkins at python.org (alexander.belopolsky) Date: Mon, 21 Sep 2015 03:23:52 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_PEP_495=3A_Updated_the_dateti?= =?utf-8?q?me=2Eh_link=2E?= Message-ID: <20150921032351.9937.64634@psf.io> https://hg.python.org/peps/rev/39b7c1da05a2 changeset: 6085:39b7c1da05a2 user: Alexander Belopolsky date: Sun Sep 20 23:23:49 2015 -0400 summary: PEP 495: Updated the datetime.h link. files: pep-0495.txt | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/pep-0495.txt b/pep-0495.txt --- a/pep-0495.txt +++ b/pep-0495.txt @@ -301,7 +301,7 @@ loading post-PEP ``fold=1`` pickles in a pre-PEP Python will result in an exception rather than an instance with out of range components. -.. _current implementation: https://hg.python.org/cpython/file/d3b20bff9c5d/Include/datetime.h#l17 +.. _current implementation: https://hg.python.org/cpython/file/v3.5.0/Include/datetime.h#l10 Implementations of tzinfo in the Standard Library -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Mon Sep 21 05:24:17 2015 From: python-checkins at python.org (terry.reedy) Date: Mon, 21 Sep 2015 03:24:17 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_Add_NEWS_items?= =?utf-8?q?_for_Idle=2E?= Message-ID: <20150921032417.31201.50309@psf.io> https://hg.python.org/cpython/rev/a1e2ae8ec7b2 changeset: 98112:a1e2ae8ec7b2 branch: 3.4 parent: 98108:60b0bc23c69e user: Terry Jan Reedy date: Sun Sep 20 23:21:22 2015 -0400 summary: Add NEWS items for Idle. files: Misc/NEWS | 15 +++++++++++++++ 1 files changed, 15 insertions(+), 0 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -442,6 +442,21 @@ IDLE ---- +- Issue #25199: Idle: add synchronization comments for future maintainers. + +- Issue #16893: Replace help.txt with idle.html for Idle doc display. + The new idlelib/idle.html is copied from Doc/build/html/idle.html. + It looks better than help.txt and will better document Idle as released. + The tkinter html viewer that works for this file was written by Rose Roseman. + The now unused EditorWindow.HelpDialog class and helt.txt file are deprecated. + +- Issue #24199: Deprecate unused idlelib.idlever with possible removal in 3.6. + +- Issue #24782: In Idle extension config dialog, replace tabs with sorted list. + Patch by Mark Roseman. + +- Issue #24790: Remove extraneous code (which also create 2 & 3 conflicts). + - Issue #23672: Allow Idle to edit and run files with astral chars in name. Patch by Mohd Sanad Zaki Rizvi. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 21 05:24:17 2015 From: python-checkins at python.org (terry.reedy) Date: Mon, 21 Sep 2015 03:24:17 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_Add_NEWS_items?= =?utf-8?q?_for_Idle=2E?= Message-ID: <20150921032417.81623.40259@psf.io> https://hg.python.org/cpython/rev/af7437dd0637 changeset: 98111:af7437dd0637 branch: 2.7 parent: 98104:e749080fa0f9 user: Terry Jan Reedy date: Sun Sep 20 23:21:17 2015 -0400 summary: Add NEWS items for Idle. files: Misc/NEWS | 15 +++++++++++++++ 1 files changed, 15 insertions(+), 0 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -167,6 +167,21 @@ IDLE ---- +- Issue #25199: Idle: add synchronization comments for future maintainers. + +- Issue #16893: Replace help.txt with idle.html for Idle doc display. + The new idlelib/idle.html is copied from Doc/build/html/idle.html. + It looks better than help.txt and will better document Idle as released. + The tkinter html viewer that works for this file was written by Rose Roseman. + The now unused EditorWindow.HelpDialog class and helt.txt file are deprecated. + +- Issue #24199: Deprecate unused idlelib.idlever with possible removal in 3.6. + +- Issue #24782: In Idle extension config dialog, replace tabs with sorted list. + Patch by Mark Roseman. + +- Issue #24790: Remove extraneous code (which also create 2 & 3 conflicts). + - Issue #23672: Allow Idle to edit and run files with astral chars in name. Patch by Mohd Sanad Zaki Rizvi. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 21 05:24:17 2015 From: python-checkins at python.org (terry.reedy) Date: Mon, 21 Sep 2015 03:24:17 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Add_NEWS_items_for_Idle=2E?= Message-ID: <20150921032417.31181.2378@psf.io> https://hg.python.org/cpython/rev/0049d4e31c34 changeset: 98113:0049d4e31c34 branch: 3.5 parent: 98109:c6ccef432dc2 parent: 98112:a1e2ae8ec7b2 user: Terry Jan Reedy date: Sun Sep 20 23:23:44 2015 -0400 summary: Add NEWS items for Idle. files: Misc/NEWS | 18 ++++++++++++++++++ 1 files changed, 18 insertions(+), 0 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -91,6 +91,24 @@ - Issue #23572: Fixed functools.singledispatch on classes with falsy metaclasses. Patch by Ethan Furman. +IDLE +---- + +- Issue #25199: Idle: add synchronization comments for future maintainers. + +- Issue #16893: Replace help.txt with idle.html for Idle doc display. + The new idlelib/idle.html is copied from Doc/build/html/idle.html. + It looks better than help.txt and will better document Idle as released. + The tkinter html viewer that works for this file was written by Rose Roseman. + The now unused EditorWindow.HelpDialog class and helt.txt file are deprecated. + +- Issue #24199: Deprecate unused idlelib.idlever with possible removal in 3.6. + +- Issue #24782: In Idle extension config dialog, replace tabs with sorted list. + Patch by Mark Roseman. + +- Issue #24790: Remove extraneous code (which also create 2 & 3 conflicts). + Documentation ------------- -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 21 05:24:17 2015 From: python-checkins at python.org (terry.reedy) Date: Mon, 21 Sep 2015 03:24:17 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge_with_3=2E5?= Message-ID: <20150921032417.11694.4968@psf.io> https://hg.python.org/cpython/rev/d104333e0e8d changeset: 98114:d104333e0e8d parent: 98110:71a6d4c7cd01 parent: 98113:0049d4e31c34 user: Terry Jan Reedy date: Sun Sep 20 23:24:01 2015 -0400 summary: Merge with 3.5 files: Misc/NEWS | 18 ++++++++++++++++++ 1 files changed, 18 insertions(+), 0 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -182,6 +182,24 @@ - Issue #23572: Fixed functools.singledispatch on classes with falsy metaclasses. Patch by Ethan Furman. +IDLE +---- + +- Issue #25199: Idle: add synchronization comments for future maintainers. + +- Issue #16893: Replace help.txt with idle.html for Idle doc display. + The new idlelib/idle.html is copied from Doc/build/html/idle.html. + It looks better than help.txt and will better document Idle as released. + The tkinter html viewer that works for this file was written by Rose Roseman. + The now unused EditorWindow.HelpDialog class and helt.txt file are deprecated. + +- Issue #24199: Deprecate unused idlelib.idlever with possible removal in 3.6. + +- Issue #24782: In Idle extension config dialog, replace tabs with sorted list. + Patch by Mark Roseman. + +- Issue #24790: Remove extraneous code (which also create 2 & 3 conflicts). + Documentation ------------- -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 21 05:32:22 2015 From: python-checkins at python.org (terry.reedy) Date: Mon, 21 Sep 2015 03:32:22 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Add_NEWS_items_for_Idle_to?= =?utf-8?q?_3=2E6=2E0a1_section=2E?= Message-ID: <20150921033222.11700.55192@psf.io> https://hg.python.org/cpython/rev/b15bfa4da038 changeset: 98115:b15bfa4da038 user: Terry Jan Reedy date: Sun Sep 20 23:32:08 2015 -0400 summary: Add NEWS items for Idle to 3.6.0a1 section. files: Misc/NEWS | 18 ++++++++++++++++++ 1 files changed, 18 insertions(+), 0 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -78,6 +78,24 @@ - Issue #13248: Remove deprecated inspect.getargspec and inspect.getmoduleinfo functions. +IDLE +---- + +- Issue #25199: Idle: add synchronization comments for future maintainers. + +- Issue #16893: Replace help.txt with idle.html for Idle doc display. + The new idlelib/idle.html is copied from Doc/build/html/idle.html. + It looks better than help.txt and will better document Idle as released. + The tkinter html viewer that works for this file was written by Rose Roseman. + The now unused EditorWindow.HelpDialog class and helt.txt file are deprecated. + +- Issue #24199: Deprecate unused idlelib.idlever with possible removal in 3.6. + +- Issue #24782: In Idle extension config dialog, replace tabs with sorted list. + Patch by Mark Roseman. + +- Issue #24790: Remove extraneous code (which also create 2 & 3 conflicts). + Documentation ------------- -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 21 05:52:07 2015 From: python-checkins at python.org (berker.peksag) Date: Mon, 21 Sep 2015 03:52:07 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzIzNDg0?= =?utf-8?q?=3A_Document_differences_between_synchronization_primitives_of?= Message-ID: <20150921035207.115399.91312@psf.io> https://hg.python.org/cpython/rev/f3faf0f355e0 changeset: 98116:f3faf0f355e0 branch: 3.4 parent: 98112:a1e2ae8ec7b2 user: Berker Peksag date: Mon Sep 21 06:50:55 2015 +0300 summary: Issue #23484: Document differences between synchronization primitives of threading and multiprocessing modules. In multiprocessing, the name of the first parameter of the acquire methods is "block", but "blocking" in threading. This commit also improves documentation of Lock and RLock. Patch by Davin Potts. files: Doc/library/multiprocessing.rst | 133 ++++++++++++++++++- 1 files changed, 120 insertions(+), 13 deletions(-) diff --git a/Doc/library/multiprocessing.rst b/Doc/library/multiprocessing.rst --- a/Doc/library/multiprocessing.rst +++ b/Doc/library/multiprocessing.rst @@ -1129,10 +1129,15 @@ .. class:: BoundedSemaphore([value]) - A bounded semaphore object: a clone of :class:`threading.BoundedSemaphore`. - - (On Mac OS X, this is indistinguishable from :class:`Semaphore` because - ``sem_getvalue()`` is not implemented on that platform). + A bounded semaphore object: a close analog of + :class:`threading.BoundedSemaphore`. + + A solitary difference from its close analog exists: its ``acquire`` method's + first argument is named *block*, as is consistent with :meth:`Lock.acquire`. + + .. note:: + On Mac OS X, this is indistinguishable from :class:`Semaphore` because + ``sem_getvalue()`` is not implemented on that platform. .. class:: Condition([lock]) @@ -1148,26 +1153,128 @@ A clone of :class:`threading.Event`. + .. class:: Lock() - A non-recursive lock object: a clone of :class:`threading.Lock`. + A non-recursive lock object: a close analog of :class:`threading.Lock`. + Once a process or thread has acquired a lock, subsequent attempts to + acquire it from any process or thread will block until it is released; + any process or thread may release it. The concepts and behaviors of + :class:`threading.Lock` as it applies to threads are replicated here in + :class:`multiprocessing.Lock` as it applies to either processes or threads, + except as noted. + + Note that :class:`Lock` is actually a factory function which returns an + instance of ``multiprocessing.synchronize.Lock`` initialized with a + default context. + + :class:`Lock` supports the :term:`context manager` protocol and thus may be + used in :keyword:`with` statements. + + .. method:: acquire(block=True, timeout=None) + + Acquire a lock, blocking or non-blocking. + + With the *block* argument set to ``True`` (the default), the method call + will block until the lock is in an unlocked state, then set it to locked + and return ``True``. Note that the name of this first argument differs + from that in :meth:`threading.Lock.acquire`. + + With the *block* argument set to ``False``, the method call does not + block. If the lock is currently in a locked state, return ``False``; + otherwise set the lock to a locked state and return ``True``. + + When invoked with a positive, floating-point value for *timeout*, block + for at most the number of seconds specified by *timeout* as long as + the lock can not be acquired. Invocations with a negative value for + *timeout* are equivalent to a *timeout* of zero. Invocations with a + *timeout* value of ``None`` (the default) set the timeout period to + infinite. Note that the treatment of negative or ``None`` values for + *timeout* differs from the implemented behavior in + :meth:`threading.Lock.acquire`. The *timeout* argument has no practical + implications if the *block* argument is set to ``False`` and is thus + ignored. Returns ``True`` if the lock has been acquired or ``False`` if + the timeout period has elapsed. + + + .. method:: release() + + Release a lock. This can be called from any process or thread, not only + the process or thread which originally acquired the lock. + + Behavior is the same as in :meth:`threading.Lock.release` except that + when invoked on an unlocked lock, a :exc:`ValueError` is raised. + .. class:: RLock() - A recursive lock object: a clone of :class:`threading.RLock`. + A recursive lock object: a close analog of :class:`threading.RLock`. A + recursive lock must be released by the process or thread that acquired it. + Once a process or thread has acquired a recursive lock, the same process + or thread may acquire it again without blocking; that process or thread + must release it once for each time it has been acquired. + + Note that :class:`RLock` is actually a factory function which returns an + instance of ``multiprocessing.synchronize.RLock`` initialized with a + default context. + + :class:`RLock` supports the :term:`context manager` protocol and thus may be + used in :keyword:`with` statements. + + + .. method:: acquire(block=True, timeout=None) + + Acquire a lock, blocking or non-blocking. + + When invoked with the *block* argument set to ``True``, block until the + lock is in an unlocked state (not owned by any process or thread) unless + the lock is already owned by the current process or thread. The current + process or thread then takes ownership of the lock (if it does not + already have ownership) and the recursion level inside the lock increments + by one, resulting in a return value of ``True``. Note that there are + several differences in this first argument's behavior compared to the + implementation of :meth:`threading.RLock.acquire`, starting with the name + of the argument itself. + + When invoked with the *block* argument set to ``False``, do not block. + If the lock has already been acquired (and thus is owned) by another + process or thread, the current process or thread does not take ownership + and the recursion level within the lock is not changed, resulting in + a return value of ``False``. If the lock is in an unlocked state, the + current process or thread takes ownership and the recursion level is + incremented, resulting in a return value of ``True``. + + Use and behaviors of the *timeout* argument are the same as in + :meth:`Lock.acquire`. Note that some of these behaviors of *timeout* + differ from the implemented behaviors in :meth:`threading.RLock.acquire`. + + + .. method:: release() + + Release a lock, decrementing the recursion level. If after the + decrement the recursion level is zero, reset the lock to unlocked (not + owned by any process or thread) and if any other processes or threads + are blocked waiting for the lock to become unlocked, allow exactly one + of them to proceed. If after the decrement the recursion level is still + nonzero, the lock remains locked and owned by the calling process or + thread. + + Only call this method when the calling process or thread owns the lock. + An :exc:`AssertionError` is raised if this method is called by a process + or thread other than the owner or if the lock is in an unlocked (unowned) + state. Note that the type of exception raised in this situation + differs from the implemented behavior in :meth:`threading.RLock.release`. + .. class:: Semaphore([value]) - A semaphore object: a clone of :class:`threading.Semaphore`. + A semaphore object: a close analog of :class:`threading.Semaphore`. + + A solitary difference from its close analog exists: its ``acquire`` method's + first argument is named *block*, as is consistent with :meth:`Lock.acquire`. .. note:: - The :meth:`acquire` and :meth:`wait` methods of each of these types - treat negative timeouts as zero timeouts. This differs from - :mod:`threading` where, since version 3.2, the equivalent - :meth:`acquire` methods treat negative timeouts as infinite - timeouts. - On Mac OS X, ``sem_timedwait`` is unsupported, so calling ``acquire()`` with a timeout will emulate that function's behavior using a sleeping loop. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 21 05:52:08 2015 From: python-checkins at python.org (berker.peksag) Date: Mon, 21 Sep 2015 03:52:08 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2323484=3A_Document_differences_between_synchroni?= =?utf-8?q?zation_primitives_of?= Message-ID: <20150921035208.94111.9786@psf.io> https://hg.python.org/cpython/rev/020377a15708 changeset: 98118:020377a15708 parent: 98115:b15bfa4da038 parent: 98117:6cd030099966 user: Berker Peksag date: Mon Sep 21 06:52:11 2015 +0300 summary: Issue #23484: Document differences between synchronization primitives of threading and multiprocessing modules. In multiprocessing, the name of the first parameter of the acquire methods is "block", but "blocking" in threading. This commit also improves documentation of Lock and RLock. Patch by Davin Potts. files: Doc/library/multiprocessing.rst | 133 ++++++++++++++++++- 1 files changed, 120 insertions(+), 13 deletions(-) diff --git a/Doc/library/multiprocessing.rst b/Doc/library/multiprocessing.rst --- a/Doc/library/multiprocessing.rst +++ b/Doc/library/multiprocessing.rst @@ -1134,10 +1134,15 @@ .. class:: BoundedSemaphore([value]) - A bounded semaphore object: a clone of :class:`threading.BoundedSemaphore`. - - (On Mac OS X, this is indistinguishable from :class:`Semaphore` because - ``sem_getvalue()`` is not implemented on that platform). + A bounded semaphore object: a close analog of + :class:`threading.BoundedSemaphore`. + + A solitary difference from its close analog exists: its ``acquire`` method's + first argument is named *block*, as is consistent with :meth:`Lock.acquire`. + + .. note:: + On Mac OS X, this is indistinguishable from :class:`Semaphore` because + ``sem_getvalue()`` is not implemented on that platform. .. class:: Condition([lock]) @@ -1153,26 +1158,128 @@ A clone of :class:`threading.Event`. + .. class:: Lock() - A non-recursive lock object: a clone of :class:`threading.Lock`. + A non-recursive lock object: a close analog of :class:`threading.Lock`. + Once a process or thread has acquired a lock, subsequent attempts to + acquire it from any process or thread will block until it is released; + any process or thread may release it. The concepts and behaviors of + :class:`threading.Lock` as it applies to threads are replicated here in + :class:`multiprocessing.Lock` as it applies to either processes or threads, + except as noted. + + Note that :class:`Lock` is actually a factory function which returns an + instance of ``multiprocessing.synchronize.Lock`` initialized with a + default context. + + :class:`Lock` supports the :term:`context manager` protocol and thus may be + used in :keyword:`with` statements. + + .. method:: acquire(block=True, timeout=None) + + Acquire a lock, blocking or non-blocking. + + With the *block* argument set to ``True`` (the default), the method call + will block until the lock is in an unlocked state, then set it to locked + and return ``True``. Note that the name of this first argument differs + from that in :meth:`threading.Lock.acquire`. + + With the *block* argument set to ``False``, the method call does not + block. If the lock is currently in a locked state, return ``False``; + otherwise set the lock to a locked state and return ``True``. + + When invoked with a positive, floating-point value for *timeout*, block + for at most the number of seconds specified by *timeout* as long as + the lock can not be acquired. Invocations with a negative value for + *timeout* are equivalent to a *timeout* of zero. Invocations with a + *timeout* value of ``None`` (the default) set the timeout period to + infinite. Note that the treatment of negative or ``None`` values for + *timeout* differs from the implemented behavior in + :meth:`threading.Lock.acquire`. The *timeout* argument has no practical + implications if the *block* argument is set to ``False`` and is thus + ignored. Returns ``True`` if the lock has been acquired or ``False`` if + the timeout period has elapsed. + + + .. method:: release() + + Release a lock. This can be called from any process or thread, not only + the process or thread which originally acquired the lock. + + Behavior is the same as in :meth:`threading.Lock.release` except that + when invoked on an unlocked lock, a :exc:`ValueError` is raised. + .. class:: RLock() - A recursive lock object: a clone of :class:`threading.RLock`. + A recursive lock object: a close analog of :class:`threading.RLock`. A + recursive lock must be released by the process or thread that acquired it. + Once a process or thread has acquired a recursive lock, the same process + or thread may acquire it again without blocking; that process or thread + must release it once for each time it has been acquired. + + Note that :class:`RLock` is actually a factory function which returns an + instance of ``multiprocessing.synchronize.RLock`` initialized with a + default context. + + :class:`RLock` supports the :term:`context manager` protocol and thus may be + used in :keyword:`with` statements. + + + .. method:: acquire(block=True, timeout=None) + + Acquire a lock, blocking or non-blocking. + + When invoked with the *block* argument set to ``True``, block until the + lock is in an unlocked state (not owned by any process or thread) unless + the lock is already owned by the current process or thread. The current + process or thread then takes ownership of the lock (if it does not + already have ownership) and the recursion level inside the lock increments + by one, resulting in a return value of ``True``. Note that there are + several differences in this first argument's behavior compared to the + implementation of :meth:`threading.RLock.acquire`, starting with the name + of the argument itself. + + When invoked with the *block* argument set to ``False``, do not block. + If the lock has already been acquired (and thus is owned) by another + process or thread, the current process or thread does not take ownership + and the recursion level within the lock is not changed, resulting in + a return value of ``False``. If the lock is in an unlocked state, the + current process or thread takes ownership and the recursion level is + incremented, resulting in a return value of ``True``. + + Use and behaviors of the *timeout* argument are the same as in + :meth:`Lock.acquire`. Note that some of these behaviors of *timeout* + differ from the implemented behaviors in :meth:`threading.RLock.acquire`. + + + .. method:: release() + + Release a lock, decrementing the recursion level. If after the + decrement the recursion level is zero, reset the lock to unlocked (not + owned by any process or thread) and if any other processes or threads + are blocked waiting for the lock to become unlocked, allow exactly one + of them to proceed. If after the decrement the recursion level is still + nonzero, the lock remains locked and owned by the calling process or + thread. + + Only call this method when the calling process or thread owns the lock. + An :exc:`AssertionError` is raised if this method is called by a process + or thread other than the owner or if the lock is in an unlocked (unowned) + state. Note that the type of exception raised in this situation + differs from the implemented behavior in :meth:`threading.RLock.release`. + .. class:: Semaphore([value]) - A semaphore object: a clone of :class:`threading.Semaphore`. + A semaphore object: a close analog of :class:`threading.Semaphore`. + + A solitary difference from its close analog exists: its ``acquire`` method's + first argument is named *block*, as is consistent with :meth:`Lock.acquire`. .. note:: - The :meth:`acquire` and :meth:`wait` methods of each of these types - treat negative timeouts as zero timeouts. This differs from - :mod:`threading` where, since version 3.2, the equivalent - :meth:`acquire` methods treat negative timeouts as infinite - timeouts. - On Mac OS X, ``sem_timedwait`` is unsupported, so calling ``acquire()`` with a timeout will emulate that function's behavior using a sleeping loop. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 21 05:52:08 2015 From: python-checkins at python.org (berker.peksag) Date: Mon, 21 Sep 2015 03:52:08 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Issue_=2323484=3A_Document_differences_between_synchronization?= =?utf-8?q?_primitives_of?= Message-ID: <20150921035207.98376.95021@psf.io> https://hg.python.org/cpython/rev/6cd030099966 changeset: 98117:6cd030099966 branch: 3.5 parent: 98113:0049d4e31c34 parent: 98116:f3faf0f355e0 user: Berker Peksag date: Mon Sep 21 06:51:45 2015 +0300 summary: Issue #23484: Document differences between synchronization primitives of threading and multiprocessing modules. In multiprocessing, the name of the first parameter of the acquire methods is "block", but "blocking" in threading. This commit also improves documentation of Lock and RLock. Patch by Davin Potts. files: Doc/library/multiprocessing.rst | 133 ++++++++++++++++++- 1 files changed, 120 insertions(+), 13 deletions(-) diff --git a/Doc/library/multiprocessing.rst b/Doc/library/multiprocessing.rst --- a/Doc/library/multiprocessing.rst +++ b/Doc/library/multiprocessing.rst @@ -1129,10 +1129,15 @@ .. class:: BoundedSemaphore([value]) - A bounded semaphore object: a clone of :class:`threading.BoundedSemaphore`. - - (On Mac OS X, this is indistinguishable from :class:`Semaphore` because - ``sem_getvalue()`` is not implemented on that platform). + A bounded semaphore object: a close analog of + :class:`threading.BoundedSemaphore`. + + A solitary difference from its close analog exists: its ``acquire`` method's + first argument is named *block*, as is consistent with :meth:`Lock.acquire`. + + .. note:: + On Mac OS X, this is indistinguishable from :class:`Semaphore` because + ``sem_getvalue()`` is not implemented on that platform. .. class:: Condition([lock]) @@ -1148,26 +1153,128 @@ A clone of :class:`threading.Event`. + .. class:: Lock() - A non-recursive lock object: a clone of :class:`threading.Lock`. + A non-recursive lock object: a close analog of :class:`threading.Lock`. + Once a process or thread has acquired a lock, subsequent attempts to + acquire it from any process or thread will block until it is released; + any process or thread may release it. The concepts and behaviors of + :class:`threading.Lock` as it applies to threads are replicated here in + :class:`multiprocessing.Lock` as it applies to either processes or threads, + except as noted. + + Note that :class:`Lock` is actually a factory function which returns an + instance of ``multiprocessing.synchronize.Lock`` initialized with a + default context. + + :class:`Lock` supports the :term:`context manager` protocol and thus may be + used in :keyword:`with` statements. + + .. method:: acquire(block=True, timeout=None) + + Acquire a lock, blocking or non-blocking. + + With the *block* argument set to ``True`` (the default), the method call + will block until the lock is in an unlocked state, then set it to locked + and return ``True``. Note that the name of this first argument differs + from that in :meth:`threading.Lock.acquire`. + + With the *block* argument set to ``False``, the method call does not + block. If the lock is currently in a locked state, return ``False``; + otherwise set the lock to a locked state and return ``True``. + + When invoked with a positive, floating-point value for *timeout*, block + for at most the number of seconds specified by *timeout* as long as + the lock can not be acquired. Invocations with a negative value for + *timeout* are equivalent to a *timeout* of zero. Invocations with a + *timeout* value of ``None`` (the default) set the timeout period to + infinite. Note that the treatment of negative or ``None`` values for + *timeout* differs from the implemented behavior in + :meth:`threading.Lock.acquire`. The *timeout* argument has no practical + implications if the *block* argument is set to ``False`` and is thus + ignored. Returns ``True`` if the lock has been acquired or ``False`` if + the timeout period has elapsed. + + + .. method:: release() + + Release a lock. This can be called from any process or thread, not only + the process or thread which originally acquired the lock. + + Behavior is the same as in :meth:`threading.Lock.release` except that + when invoked on an unlocked lock, a :exc:`ValueError` is raised. + .. class:: RLock() - A recursive lock object: a clone of :class:`threading.RLock`. + A recursive lock object: a close analog of :class:`threading.RLock`. A + recursive lock must be released by the process or thread that acquired it. + Once a process or thread has acquired a recursive lock, the same process + or thread may acquire it again without blocking; that process or thread + must release it once for each time it has been acquired. + + Note that :class:`RLock` is actually a factory function which returns an + instance of ``multiprocessing.synchronize.RLock`` initialized with a + default context. + + :class:`RLock` supports the :term:`context manager` protocol and thus may be + used in :keyword:`with` statements. + + + .. method:: acquire(block=True, timeout=None) + + Acquire a lock, blocking or non-blocking. + + When invoked with the *block* argument set to ``True``, block until the + lock is in an unlocked state (not owned by any process or thread) unless + the lock is already owned by the current process or thread. The current + process or thread then takes ownership of the lock (if it does not + already have ownership) and the recursion level inside the lock increments + by one, resulting in a return value of ``True``. Note that there are + several differences in this first argument's behavior compared to the + implementation of :meth:`threading.RLock.acquire`, starting with the name + of the argument itself. + + When invoked with the *block* argument set to ``False``, do not block. + If the lock has already been acquired (and thus is owned) by another + process or thread, the current process or thread does not take ownership + and the recursion level within the lock is not changed, resulting in + a return value of ``False``. If the lock is in an unlocked state, the + current process or thread takes ownership and the recursion level is + incremented, resulting in a return value of ``True``. + + Use and behaviors of the *timeout* argument are the same as in + :meth:`Lock.acquire`. Note that some of these behaviors of *timeout* + differ from the implemented behaviors in :meth:`threading.RLock.acquire`. + + + .. method:: release() + + Release a lock, decrementing the recursion level. If after the + decrement the recursion level is zero, reset the lock to unlocked (not + owned by any process or thread) and if any other processes or threads + are blocked waiting for the lock to become unlocked, allow exactly one + of them to proceed. If after the decrement the recursion level is still + nonzero, the lock remains locked and owned by the calling process or + thread. + + Only call this method when the calling process or thread owns the lock. + An :exc:`AssertionError` is raised if this method is called by a process + or thread other than the owner or if the lock is in an unlocked (unowned) + state. Note that the type of exception raised in this situation + differs from the implemented behavior in :meth:`threading.RLock.release`. + .. class:: Semaphore([value]) - A semaphore object: a clone of :class:`threading.Semaphore`. + A semaphore object: a close analog of :class:`threading.Semaphore`. + + A solitary difference from its close analog exists: its ``acquire`` method's + first argument is named *block*, as is consistent with :meth:`Lock.acquire`. .. note:: - The :meth:`acquire` and :meth:`wait` methods of each of these types - treat negative timeouts as zero timeouts. This differs from - :mod:`threading` where, since version 3.2, the equivalent - :meth:`acquire` methods treat negative timeouts as infinite - timeouts. - On Mac OS X, ``sem_timedwait`` is unsupported, so calling ``acquire()`` with a timeout will emulate that function's behavior using a sleeping loop. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 21 05:52:27 2015 From: python-checkins at python.org (alexander.belopolsky) Date: Mon, 21 Sep 2015 03:52:27 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_PEP_495=3A_Corrected_a_typo?= =?utf-8?q?=2E?= Message-ID: <20150921035227.98368.84853@psf.io> https://hg.python.org/peps/rev/7b1223e44e81 changeset: 6086:7b1223e44e81 user: Alexander Belopolsky date: Sun Sep 20 23:52:23 2015 -0400 summary: PEP 495: Corrected a typo. files: pep-0495.txt | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/pep-0495.txt b/pep-0495.txt --- a/pep-0495.txt +++ b/pep-0495.txt @@ -295,7 +295,7 @@ objects will not change. The ``fold`` value will be encoded in the first bit of the 3rd (1st) byte of ``datetime.datetime`` (``datetime.time``) pickle payload. In the `current implementation`_ -these byte are used to store the month (1-12) and hour (0-23) values +these bytes are used to store the month (1-12) and hour (0-23) values and the first bit is always 0. We picked these bytes because they are the only bytes that are checked by the current unpickle code. Thus loading post-PEP ``fold=1`` pickles in a pre-PEP Python will result in -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Mon Sep 21 06:15:46 2015 From: python-checkins at python.org (berker.peksag) Date: Mon, 21 Sep 2015 04:15:46 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzIzNDg0?= =?utf-8?q?=3A_Document_differences_between_synchronization_primitives_of?= Message-ID: <20150921041546.3644.60382@psf.io> https://hg.python.org/cpython/rev/92350a2a8b08 changeset: 98119:92350a2a8b08 branch: 2.7 parent: 98111:af7437dd0637 user: Berker Peksag date: Mon Sep 21 07:15:52 2015 +0300 summary: Issue #23484: Document differences between synchronization primitives of threading and multiprocessing modules. In multiprocessing, the name of the first parameter of the acquire methods is "block", but "blocking" in threading. This commit also improves documentation of Lock and RLock. Patch by Davin Potts. files: Doc/library/multiprocessing.rst | 127 ++++++++++++++++++- 1 files changed, 120 insertions(+), 7 deletions(-) diff --git a/Doc/library/multiprocessing.rst b/Doc/library/multiprocessing.rst --- a/Doc/library/multiprocessing.rst +++ b/Doc/library/multiprocessing.rst @@ -908,10 +908,16 @@ .. class:: BoundedSemaphore([value]) - A bounded semaphore object: a clone of :class:`threading.BoundedSemaphore`. - - (On Mac OS X, this is indistinguishable from :class:`Semaphore` because - ``sem_getvalue()`` is not implemented on that platform). + A bounded semaphore object: a close analog of + :class:`threading.BoundedSemaphore`. + + A solitary difference from its close analog exists: its ``acquire`` method's + first argument is named *block* and it supports an optional second argument + *timeout*, as is consistent with :meth:`Lock.acquire`. + + .. note:: + On Mac OS X, this is indistinguishable from :class:`Semaphore` because + ``sem_getvalue()`` is not implemented on that platform. .. class:: Condition([lock]) @@ -930,17 +936,124 @@ .. versionchanged:: 2.7 Previously, the method always returned ``None``. + .. class:: Lock() - A non-recursive lock object: a clone of :class:`threading.Lock`. + A non-recursive lock object: a close analog of :class:`threading.Lock`. + Once a process or thread has acquired a lock, subsequent attempts to + acquire it from any process or thread will block until it is released; + any process or thread may release it. The concepts and behaviors of + :class:`threading.Lock` as it applies to threads are replicated here in + :class:`multiprocessing.Lock` as it applies to either processes or threads, + except as noted. + + Note that :class:`Lock` is actually a factory function which returns an + instance of ``multiprocessing.synchronize.Lock`` initialized with a + default context. + + :class:`Lock` supports the :term:`context manager` protocol and thus may be + used in :keyword:`with` statements. + + .. method:: acquire(block=True, timeout=None) + + Acquire a lock, blocking or non-blocking. + + With the *block* argument set to ``True`` (the default), the method call + will block until the lock is in an unlocked state, then set it to locked + and return ``True``. Note that the name of this first argument differs + from that in :meth:`threading.Lock.acquire`. + + With the *block* argument set to ``False``, the method call does not + block. If the lock is currently in a locked state, return ``False``; + otherwise set the lock to a locked state and return ``True``. + + When invoked with a positive, floating-point value for *timeout*, block + for at most the number of seconds specified by *timeout* as long as + the lock can not be acquired. Invocations with a negative value for + *timeout* are equivalent to a *timeout* of zero. Invocations with a + *timeout* value of ``None`` (the default) set the timeout period to + infinite. The *timeout* argument has no practical implications if the + *block* argument is set to ``False`` and is thus ignored. Returns + ``True`` if the lock has been acquired or ``False`` if the timeout period + has elapsed. Note that the *timeout* argument does not exist in this + method's analog, :meth:`threading.Lock.acquire`. + + .. method:: release() + + Release a lock. This can be called from any process or thread, not only + the process or thread which originally acquired the lock. + + Behavior is the same as in :meth:`threading.Lock.release` except that + when invoked on an unlocked lock, a :exc:`ValueError` is raised. + .. class:: RLock() - A recursive lock object: a clone of :class:`threading.RLock`. + A recursive lock object: a close analog of :class:`threading.RLock`. A + recursive lock must be released by the process or thread that acquired it. + Once a process or thread has acquired a recursive lock, the same process + or thread may acquire it again without blocking; that process or thread + must release it once for each time it has been acquired. + + Note that :class:`RLock` is actually a factory function which returns an + instance of ``multiprocessing.synchronize.RLock`` initialized with a + default context. + + :class:`RLock` supports the :term:`context manager` protocol and thus may be + used in :keyword:`with` statements. + + + .. method:: acquire(block=True, timeout=None) + + Acquire a lock, blocking or non-blocking. + + When invoked with the *block* argument set to ``True``, block until the + lock is in an unlocked state (not owned by any process or thread) unless + the lock is already owned by the current process or thread. The current + process or thread then takes ownership of the lock (if it does not + already have ownership) and the recursion level inside the lock increments + by one, resulting in a return value of ``True``. Note that there are + several differences in this first argument's behavior compared to the + implementation of :meth:`threading.RLock.acquire`, starting with the name + of the argument itself. + + When invoked with the *block* argument set to ``False``, do not block. + If the lock has already been acquired (and thus is owned) by another + process or thread, the current process or thread does not take ownership + and the recursion level within the lock is not changed, resulting in + a return value of ``False``. If the lock is in an unlocked state, the + current process or thread takes ownership and the recursion level is + incremented, resulting in a return value of ``True``. + + Use and behaviors of the *timeout* argument are the same as in + :meth:`Lock.acquire`. Note that the *timeout* argument does + not exist in this method's analog, :meth:`threading.RLock.acquire`. + + + .. method:: release() + + Release a lock, decrementing the recursion level. If after the + decrement the recursion level is zero, reset the lock to unlocked (not + owned by any process or thread) and if any other processes or threads + are blocked waiting for the lock to become unlocked, allow exactly one + of them to proceed. If after the decrement the recursion level is still + nonzero, the lock remains locked and owned by the calling process or + thread. + + Only call this method when the calling process or thread owns the lock. + An :exc:`AssertionError` is raised if this method is called by a process + or thread other than the owner or if the lock is in an unlocked (unowned) + state. Note that the type of exception raised in this situation + differs from the implemented behavior in :meth:`threading.RLock.release`. + .. class:: Semaphore([value]) - A semaphore object: a clone of :class:`threading.Semaphore`. + A semaphore object: a close analog of :class:`threading.Semaphore`. + + A solitary difference from its close analog exists: its ``acquire`` method's + first argument is named *block* and it supports an optional second argument + *timeout*, as is consistent with :meth:`Lock.acquire`. .. note:: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 21 07:11:43 2015 From: python-checkins at python.org (terry.reedy) Date: Mon, 21 Sep 2015 05:11:43 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_Move_items_fro?= =?utf-8?q?m_NEWS_to_idlelib/NEWS=2Etxt=2E__Standardize_headers_spacing=3A?= =?utf-8?q?_2_lines?= Message-ID: <20150921051142.98362.10608@psf.io> https://hg.python.org/cpython/rev/a108db755a59 changeset: 98120:a108db755a59 branch: 2.7 user: Terry Jan Reedy date: Mon Sep 21 01:07:54 2015 -0400 summary: Move items from NEWS to idlelib/NEWS.txt. Standardize headers spacing: 2 lines above "What's New and 0 lines above "Release date". Remove most old headers for non-final releases (they currently do not get carried forward. files: Lib/idlelib/NEWS.txt | 126 +++--------------------------- 1 files changed, 16 insertions(+), 110 deletions(-) diff --git a/Lib/idlelib/NEWS.txt b/Lib/idlelib/NEWS.txt --- a/Lib/idlelib/NEWS.txt +++ b/Lib/idlelib/NEWS.txt @@ -1,7 +1,21 @@ What's New in IDLE 2.7.11? ========================= +*Release date: -*Release date: +- Issue #25199: Idle: add synchronization comments for future maintainers. + +- Issue #16893: Replace help.txt with idle.html for Idle doc display. + The new idlelib/idle.html is copied from Doc/build/html/idle.html. + It looks better than help.txt and will better document Idle as released. + The tkinter html viewer that works for this file was written by Rose Roseman. + The now unused EditorWindow.HelpDialog class and helt.txt file are deprecated. + +- Issue #24199: Deprecate unused idlelib.idlever with possible removal in 3.6. + +- Issue #24782: In Idle extension config dialog, replace tabs with sorted list. + Patch by Mark Roseman. + +- Issue #24790: Remove extraneous code (which also create 2 & 3 conflicts). - Issue #23672: Allow Idle to edit and run files with astral chars in name. Patch by Mohd Sanad Zaki Rizvi. @@ -22,7 +36,6 @@ What's New in IDLE 2.7.10? ========================= - *Release date: 2015-05-23* - Issue #23583: Fixed writing unicode to standard output stream in IDLE. @@ -41,7 +54,6 @@ What's New in IDLE 2.7.9? ========================= - *Release date: 2014-12-10* - Issue #16893: Update Idle doc chapter to match current Idle and add new @@ -76,7 +88,6 @@ What's New in IDLE 2.7.8? ========================= - *Release date: 2014-06-29* - Issue #21940: Add unittest for WidgetRedirector. Initial patch by Saimadhav @@ -104,7 +115,6 @@ What's New in IDLE 2.7.7? ========================= - *Release date: 2014-05-31* - Issue #18104: Add idlelib/idle_test/htest.py with a few sample tests to begin @@ -142,7 +152,6 @@ What's New in IDLE 2.7.6? ========================= - *Release date: 2013-11-10* - Issue #19426: Fixed the opening of Python source file with specified encoding. @@ -190,7 +199,6 @@ What's New in IDLE 2.7.5? ========================= - *Release date: 2013-05-12* - Issue #17838: Allow sys.stdin to be reassigned. @@ -225,7 +233,6 @@ What's New in IDLE 2.7.4? ========================= - *Release date: 2013-04-06* - Issue #17625: In IDLE, close the replace dialog after it is used. @@ -296,7 +303,6 @@ What's New in IDLE 2.7.3? ========================= - *Release date: 2012-04-09* - Issue #964437 Make IDLE help window non-modal. @@ -329,7 +335,6 @@ What's New in IDLE 2.7.2? ========================= - *Release date: 2011-06-11* - Issue #11718: IDLE's open module dialog couldn't find the __init__.py @@ -374,7 +379,6 @@ What's New in Python 2.7.1? =========================== - *Release date: 2010-11-27* - Issue #6378: idle.bat now runs with the appropriate Python version rather than @@ -383,7 +387,6 @@ What's New in IDLE 2.7? ======================= - *Release date: 2010-07-03* - Issue #5150: IDLE's format menu now has an option to strip trailing @@ -415,7 +418,6 @@ What's New in IDLE 2.6? ======================= - *Release date: 01-Oct-2008* - Issue #2665: On Windows, an IDLE installation upgraded from an old version @@ -429,11 +431,6 @@ - Autocompletion of filenames now support alternate separators, e.g. the '/' char on Windows. Patch 2061 Tal Einat. -What's New in IDLE 2.6a1? -========================= - -*Release date: 29-Feb-2008* - - Configured selection highlighting colors were ignored; updating highlighting in the config dialog would cause non-Python files to be colored as if they were Python source; improve use of ColorDelagator. Patch 1334. Tal Einat. @@ -505,15 +502,8 @@ What's New in IDLE 1.2? ======================= - *Release date: 19-SEP-2006* - -What's New in IDLE 1.2c1? -========================= - -*Release date: 17-AUG-2006* - - File menu hotkeys: there were three 'p' assignments. Reassign the 'Save Copy As' and 'Print' hotkeys to 'y' and 't'. Change the Shell hotkey from 's' to 'l'. @@ -534,11 +524,6 @@ - When used w/o subprocess, all exceptions were preceded by an error message claiming they were IDLE internal errors (since 1.2a1). -What's New in IDLE 1.2b3? -========================= - -*Release date: 03-AUG-2006* - - Bug #1525817: Don't truncate short lines in IDLE's tool tips. - Bug #1517990: IDLE keybindings on MacOS X now work correctly @@ -562,26 +547,6 @@ 'as' keyword in comment directly following import command. Closes 1325071. Patch 1479219 Tal Einat -What's New in IDLE 1.2b2? -========================= - -*Release date: 11-JUL-2006* - -What's New in IDLE 1.2b1? -========================= - -*Release date: 20-JUN-2006* - -What's New in IDLE 1.2a2? -========================= - -*Release date: 27-APR-2006* - -What's New in IDLE 1.2a1? -========================= - -*Release date: 05-APR-2006* - - Patch #1162825: Support non-ASCII characters in IDLE window titles. - Source file f.flush() after writing; trying to avoid lossage if user @@ -661,19 +626,14 @@ - The remote procedure call module rpc.py can now access data attributes of remote registered objects. Changes to these attributes are local, however. + What's New in IDLE 1.1? ======================= - *Release date: 30-NOV-2004* - On OpenBSD, terminating IDLE with ctrl-c from the command line caused a stuck subprocess MainThread because only the SocketThread was exiting. -What's New in IDLE 1.1b3/rc1? -============================= - -*Release date: 18-NOV-2004* - - Saving a Keyset w/o making changes (by using the "Save as New Custom Key Set" button) caused IDLE to fail on restart (no new keyset was created in config-keys.cfg). Also true for Theme/highlights. Python Bug 1064535. @@ -681,28 +641,12 @@ - A change to the linecache.py API caused IDLE to exit when an exception was raised while running without the subprocess (-n switch). Python Bug 1063840. -What's New in IDLE 1.1b2? -========================= - -*Release date: 03-NOV-2004* - - When paragraph reformat width was made configurable, a bug was introduced that caused reformatting of comment blocks to ignore how far the block was indented, effectively adding the indentation width to the reformat width. This has been repaired, and the reformat width is again a bound on the total width of reformatted lines. -What's New in IDLE 1.1b1? -========================= - -*Release date: 15-OCT-2004* - - -What's New in IDLE 1.1a3? -========================= - -*Release date: 02-SEP-2004* - - Improve keyboard focus binding, especially in Windows menu. Improve window raising, especially in the Windows menu and in the debugger. IDLEfork 763524. @@ -710,24 +654,12 @@ - If user passes a non-existent filename on the commandline, just open a new file, don't raise a dialog. IDLEfork 854928. - -What's New in IDLE 1.1a2? -========================= - -*Release date: 05-AUG-2004* - - EditorWindow.py was not finding the .chm help file on Windows. Typo at Rev 1.54. Python Bug 990954 - checking sys.platform for substring 'win' was breaking IDLE docs on Mac (darwin). Also, Mac Safari browser requires full file:// URIs. SF 900580. - -What's New in IDLE 1.1a1? -========================= - -*Release date: 08-JUL-2004* - - Redirect the warning stream to the shell during the ScriptBinding check of user code and format the warning similarly to an exception for both that check and for runtime warnings raised in the subprocess. @@ -790,26 +722,10 @@ What's New in IDLE 1.0? ======================= - *Release date: 29-Jul-2003* -- Added a banner to the shell discussing warnings possibly raised by personal - firewall software. Added same comment to README.txt. - - -What's New in IDLE 1.0 release candidate 2? -=========================================== - -*Release date: 24-Jul-2003* - - Calltip error when docstring was None Python Bug 775541 - -What's New in IDLE 1.0 release candidate 1? -=========================================== - -*Release date: 18-Jul-2003* - - Updated extend.txt, help.txt, and config-extensions.def to correctly reflect the current status of the configuration system. Python Bug 768469 @@ -825,12 +741,6 @@ sys.std{in|out|err}.encoding, for both the local and the subprocess case. SF IDLEfork patch 682347. - -What's New in IDLE 1.0b2? -========================= - -*Release date: 29-Jun-2003* - - Extend AboutDialog.ViewFile() to support file encodings. Make the CREDITS file Latin-1. @@ -869,7 +779,6 @@ What's New in IDLEfork 0.9b1? ============================= - *Release date: 02-Jun-2003* - The current working directory of the execution environment (and shell @@ -971,10 +880,8 @@ exception formatting to the subprocess. - What's New in IDLEfork 0.9 Alpha 2? =================================== - *Release date: 27-Jan-2003* - Updated INSTALL.txt to claify use of the python2 rpm. @@ -1078,7 +985,6 @@ What's New in IDLEfork 0.9 Alpha 1? =================================== - *Release date: 31-Dec-2002* - First release of major new functionality. For further details refer to -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 21 07:11:42 2015 From: python-checkins at python.org (terry.reedy) Date: Mon, 21 Sep 2015 05:11:42 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_Move_items_fro?= =?utf-8?q?m_NEWS_to_idlelib/NEWS=2Etxt=2E__Standardize_headers_spacing=3A?= =?utf-8?q?_2_lines?= Message-ID: <20150921051142.31175.62491@psf.io> https://hg.python.org/cpython/rev/ae352d27618d changeset: 98121:ae352d27618d branch: 3.4 parent: 98116:f3faf0f355e0 user: Terry Jan Reedy date: Mon Sep 21 01:07:59 2015 -0400 summary: Move items from NEWS to idlelib/NEWS.txt. Standardize headers spacing: 2 lines above "What's New and 0 lines above "Release date". Remove most old headers for non-final releases (they currently do not get carried forward. files: Lib/idlelib/NEWS.txt | 120 +++++------------------------- 1 files changed, 20 insertions(+), 100 deletions(-) diff --git a/Lib/idlelib/NEWS.txt b/Lib/idlelib/NEWS.txt --- a/Lib/idlelib/NEWS.txt +++ b/Lib/idlelib/NEWS.txt @@ -2,6 +2,21 @@ ========================= *Release date: 2015-??-??* +- Issue #25199: Idle: add synchronization comments for future maintainers. + +- Issue #16893: Replace help.txt with idle.html for Idle doc display. + The new idlelib/idle.html is copied from Doc/build/html/idle.html. + It looks better than help.txt and will better document Idle as released. + The tkinter html viewer that works for this file was written by Rose Roseman. + The now unused EditorWindow.HelpDialog class and helt.txt file are deprecated. + +- Issue #24199: Deprecate unused idlelib.idlever with possible removal in 3.6. + +- Issue #24782: In Idle extension config dialog, replace tabs with sorted list. + Patch by Mark Roseman. + +- Issue #24790: Remove extraneous code (which also create 2 & 3 conflicts). + - Issue #23672: Allow Idle to edit and run files with astral chars in name. Patch by Mohd Sanad Zaki Rizvi. @@ -174,7 +189,6 @@ What's New in IDLE 3.2.1? ========================= - *Release date: 15-May-11* - Issue #6378: Further adjust idle.bat to start associated Python @@ -192,7 +206,6 @@ What's New in IDLE 3.1b1? ========================= - *Release date: 06-May-09* - Use of 'filter' in keybindingDialog.py was causing custom key assignment to @@ -201,7 +214,6 @@ What's New in IDLE 3.1a1? ========================= - *Release date: 07-Mar-09* - Issue #4815: Offer conversion to UTF-8 if source files have @@ -217,9 +229,9 @@ - Issue #2665: On Windows, an IDLE installation upgraded from an old version would not start if a custom theme was defined. + What's New in IDLE 2.7? (UNRELEASED, but merged into 3.1 releases above.) ======================= - *Release date: XX-XXX-2010* - idle.py modified and simplified to better support developing experimental @@ -243,9 +255,9 @@ - Issue #3549: On MacOS the preferences menu was not present + What's New in IDLE 3.0 final? ============================= - *Release date: 03-Dec-2008* - IDLE would print a "Unhandled server exception!" message when internal @@ -260,7 +272,6 @@ What's New in IDLE 3.0a3? ========================= - *Release date: 29-Feb-2008* - help() was not paging to the shell. Issue1650. @@ -277,7 +288,6 @@ What's New in IDLE 3.0a2? ========================= - *Release date: 06-Dec-2007* - Windows EOL sequence not converted correctly, encoding error. @@ -286,7 +296,6 @@ What's New in IDLE 3.0a1? ========================= - *Release date: 31-Aug-2007* - IDLE converted to Python 3000 syntax. @@ -302,9 +311,8 @@ be cleared before IDLE exits. -What's New in IDLE 2.6 final? -============================= - +What's New in IDLE 2.6 +====================== *Release date: 01-Oct-2008*, merged into 3.0 releases detailed above (3.0rc2) - Issue #2665: On Windows, an IDLE installation upgraded from an old version @@ -389,15 +397,8 @@ What's New in IDLE 1.2? ======================= - *Release date: 19-SEP-2006* - -What's New in IDLE 1.2c1? -========================= - -*Release date: 17-AUG-2006* - - File menu hotkeys: there were three 'p' assignments. Reassign the 'Save Copy As' and 'Print' hotkeys to 'y' and 't'. Change the Shell hotkey from 's' to 'l'. @@ -418,11 +419,6 @@ - When used w/o subprocess, all exceptions were preceded by an error message claiming they were IDLE internal errors (since 1.2a1). -What's New in IDLE 1.2b3? -========================= - -*Release date: 03-AUG-2006* - - Bug #1525817: Don't truncate short lines in IDLE's tool tips. - Bug #1517990: IDLE keybindings on MacOS X now work correctly @@ -446,26 +442,6 @@ 'as' keyword in comment directly following import command. Closes 1325071. Patch 1479219 Tal Einat -What's New in IDLE 1.2b2? -========================= - -*Release date: 11-JUL-2006* - -What's New in IDLE 1.2b1? -========================= - -*Release date: 20-JUN-2006* - -What's New in IDLE 1.2a2? -========================= - -*Release date: 27-APR-2006* - -What's New in IDLE 1.2a1? -========================= - -*Release date: 05-APR-2006* - - Patch #1162825: Support non-ASCII characters in IDLE window titles. - Source file f.flush() after writing; trying to avoid lossage if user @@ -545,19 +521,14 @@ - The remote procedure call module rpc.py can now access data attributes of remote registered objects. Changes to these attributes are local, however. + What's New in IDLE 1.1? ======================= - *Release date: 30-NOV-2004* - On OpenBSD, terminating IDLE with ctrl-c from the command line caused a stuck subprocess MainThread because only the SocketThread was exiting. -What's New in IDLE 1.1b3/rc1? -============================= - -*Release date: 18-NOV-2004* - - Saving a Keyset w/o making changes (by using the "Save as New Custom Key Set" button) caused IDLE to fail on restart (no new keyset was created in config-keys.cfg). Also true for Theme/highlights. Python Bug 1064535. @@ -565,28 +536,12 @@ - A change to the linecache.py API caused IDLE to exit when an exception was raised while running without the subprocess (-n switch). Python Bug 1063840. -What's New in IDLE 1.1b2? -========================= - -*Release date: 03-NOV-2004* - - When paragraph reformat width was made configurable, a bug was introduced that caused reformatting of comment blocks to ignore how far the block was indented, effectively adding the indentation width to the reformat width. This has been repaired, and the reformat width is again a bound on the total width of reformatted lines. -What's New in IDLE 1.1b1? -========================= - -*Release date: 15-OCT-2004* - - -What's New in IDLE 1.1a3? -========================= - -*Release date: 02-SEP-2004* - - Improve keyboard focus binding, especially in Windows menu. Improve window raising, especially in the Windows menu and in the debugger. IDLEfork 763524. @@ -594,24 +549,12 @@ - If user passes a non-existent filename on the commandline, just open a new file, don't raise a dialog. IDLEfork 854928. - -What's New in IDLE 1.1a2? -========================= - -*Release date: 05-AUG-2004* - - EditorWindow.py was not finding the .chm help file on Windows. Typo at Rev 1.54. Python Bug 990954 - checking sys.platform for substring 'win' was breaking IDLE docs on Mac (darwin). Also, Mac Safari browser requires full file:// URIs. SF 900580. - -What's New in IDLE 1.1a1? -========================= - -*Release date: 08-JUL-2004* - - Redirect the warning stream to the shell during the ScriptBinding check of user code and format the warning similarly to an exception for both that check and for runtime warnings raised in the subprocess. @@ -674,26 +617,13 @@ What's New in IDLE 1.0? ======================= - *Release date: 29-Jul-2003* - Added a banner to the shell discussing warnings possibly raised by personal firewall software. Added same comment to README.txt. - -What's New in IDLE 1.0 release candidate 2? -=========================================== - -*Release date: 24-Jul-2003* - - Calltip error when docstring was None Python Bug 775541 - -What's New in IDLE 1.0 release candidate 1? -=========================================== - -*Release date: 18-Jul-2003* - - Updated extend.txt, help.txt, and config-extensions.def to correctly reflect the current status of the configuration system. Python Bug 768469 @@ -709,12 +639,6 @@ sys.std{in|out|err}.encoding, for both the local and the subprocess case. SF IDLEfork patch 682347. - -What's New in IDLE 1.0b2? -========================= - -*Release date: 29-Jun-2003* - - Extend AboutDialog.ViewFile() to support file encodings. Make the CREDITS file Latin-1. @@ -753,7 +677,6 @@ What's New in IDLEfork 0.9b1? ============================= - *Release date: 02-Jun-2003* - The current working directory of the execution environment (and shell @@ -855,10 +778,8 @@ exception formatting to the subprocess. - What's New in IDLEfork 0.9 Alpha 2? =================================== - *Release date: 27-Jan-2003* - Updated INSTALL.txt to claify use of the python2 rpm. @@ -962,7 +883,6 @@ What's New in IDLEfork 0.9 Alpha 1? =================================== - *Release date: 31-Dec-2002* - First release of major new functionality. For further details refer to -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 21 07:11:43 2015 From: python-checkins at python.org (terry.reedy) Date: Mon, 21 Sep 2015 05:11:43 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Move_items_from_NEWS_to_idlelib/NEWS=2Etxt=2E__Standardize_hea?= =?utf-8?q?ders_spacing=3A_2_lines?= Message-ID: <20150921051143.94113.56007@psf.io> https://hg.python.org/cpython/rev/e758988851b7 changeset: 98122:e758988851b7 branch: 3.5 parent: 98117:6cd030099966 parent: 98121:ae352d27618d user: Terry Jan Reedy date: Mon Sep 21 01:10:21 2015 -0400 summary: Move items from NEWS to idlelib/NEWS.txt. Standardize headers spacing: 2 lines above "What's New and 0 lines above "Release date". Remove most old headers for non-final releases (they currently do not get carried forward. files: Lib/idlelib/NEWS.txt | 148 +++++------------------------- 1 files changed, 26 insertions(+), 122 deletions(-) diff --git a/Lib/idlelib/NEWS.txt b/Lib/idlelib/NEWS.txt --- a/Lib/idlelib/NEWS.txt +++ b/Lib/idlelib/NEWS.txt @@ -1,6 +1,26 @@ +What's New in IDLE 3.5.1? +========================= +*Release date: 2015-09-13* + +- Issue #25199: Idle: add synchronization comments for future maintainers. + +- Issue #16893: Replace help.txt with idle.html for Idle doc display. + The new idlelib/idle.html is copied from Doc/build/html/idle.html. + It looks better than help.txt and will better document Idle as released. + The tkinter html viewer that works for this file was written by Rose Roseman. + The now unused EditorWindow.HelpDialog class and helt.txt file are deprecated. + +- Issue #24199: Deprecate unused idlelib.idlever with possible removal in 3.6. + +- Issue #24782: In Idle extension config dialog, replace tabs with sorted list. + Patch by Mark Roseman. + +- Issue #24790: Remove extraneous code (which also create 2 & 3 conflicts). + + What's New in IDLE 3.5.0? ========================= -*Release date: 2015-09-13* ?? +*Release date: 2015-09-13* - Issue #23672: Allow Idle to edit and run files with astral chars in name. Patch by Mohd Sanad Zaki Rizvi. @@ -42,7 +62,7 @@ - Issue #21986: Code objects are not normally pickled by the pickle module. To match this, they are no longer pickled when running under Idle. - + - Issue #23180: Rename IDLE "Windows" menu item to "Window". Patch by Al Sweigart. @@ -180,11 +200,6 @@ - Use of 'filter' in keybindingDialog.py was causing custom key assignment to fail. Patch 5707 amaury.forgeotdarc. - -What's New in IDLE 3.1a1? -========================= -*Release date: 07-Mar-09* - - Issue #4815: Offer conversion to UTF-8 if source files have no encoding declaration and are not encoded in UTF-8. @@ -198,6 +213,7 @@ - Issue #2665: On Windows, an IDLE installation upgraded from an old version would not start if a custom theme was defined. + What's New in IDLE 2.7? (UNRELEASED, but merged into 3.1 releases above.) ======================= *Release date: XX-XXX-2010* @@ -223,11 +239,6 @@ - Issue #3549: On MacOS the preferences menu was not present -What's New in IDLE 3.0 final? -============================= - -*Release date: 03-Dec-2008* - - IDLE would print a "Unhandled server exception!" message when internal debugging is enabled. @@ -237,12 +248,6 @@ - Issue #4383: When IDLE cannot make the connection to its subprocess, it would fail to properly display the error message. - -What's New in IDLE 3.0a3? -========================= - -*Release date: 29-Feb-2008* - - help() was not paging to the shell. Issue1650. - CodeContext was not importing. @@ -254,21 +259,9 @@ - Issue #1585: IDLE uses non-existent xrange() function. - -What's New in IDLE 3.0a2? -========================= - -*Release date: 06-Dec-2007* - - Windows EOL sequence not converted correctly, encoding error. Caused file save to fail. Bug 1130. - -What's New in IDLE 3.0a1? -========================= - -*Release date: 31-Aug-2007* - - IDLE converted to Python 3000 syntax. - Strings became Unicode. @@ -282,9 +275,8 @@ be cleared before IDLE exits. -What's New in IDLE 2.6 final? -============================= - +What's New in IDLE 2.6 +====================== *Release date: 01-Oct-2008*, merged into 3.0 releases detailed above (3.0rc2) - Issue #2665: On Windows, an IDLE installation upgraded from an old version @@ -369,15 +361,8 @@ What's New in IDLE 1.2? ======================= - *Release date: 19-SEP-2006* - -What's New in IDLE 1.2c1? -========================= - -*Release date: 17-AUG-2006* - - File menu hotkeys: there were three 'p' assignments. Reassign the 'Save Copy As' and 'Print' hotkeys to 'y' and 't'. Change the Shell hotkey from 's' to 'l'. @@ -398,11 +383,6 @@ - When used w/o subprocess, all exceptions were preceded by an error message claiming they were IDLE internal errors (since 1.2a1). -What's New in IDLE 1.2b3? -========================= - -*Release date: 03-AUG-2006* - - Bug #1525817: Don't truncate short lines in IDLE's tool tips. - Bug #1517990: IDLE keybindings on MacOS X now work correctly @@ -426,26 +406,6 @@ 'as' keyword in comment directly following import command. Closes 1325071. Patch 1479219 Tal Einat -What's New in IDLE 1.2b2? -========================= - -*Release date: 11-JUL-2006* - -What's New in IDLE 1.2b1? -========================= - -*Release date: 20-JUN-2006* - -What's New in IDLE 1.2a2? -========================= - -*Release date: 27-APR-2006* - -What's New in IDLE 1.2a1? -========================= - -*Release date: 05-APR-2006* - - Patch #1162825: Support non-ASCII characters in IDLE window titles. - Source file f.flush() after writing; trying to avoid lossage if user @@ -525,19 +485,14 @@ - The remote procedure call module rpc.py can now access data attributes of remote registered objects. Changes to these attributes are local, however. + What's New in IDLE 1.1? ======================= - *Release date: 30-NOV-2004* - On OpenBSD, terminating IDLE with ctrl-c from the command line caused a stuck subprocess MainThread because only the SocketThread was exiting. -What's New in IDLE 1.1b3/rc1? -============================= - -*Release date: 18-NOV-2004* - - Saving a Keyset w/o making changes (by using the "Save as New Custom Key Set" button) caused IDLE to fail on restart (no new keyset was created in config-keys.cfg). Also true for Theme/highlights. Python Bug 1064535. @@ -545,28 +500,12 @@ - A change to the linecache.py API caused IDLE to exit when an exception was raised while running without the subprocess (-n switch). Python Bug 1063840. -What's New in IDLE 1.1b2? -========================= - -*Release date: 03-NOV-2004* - - When paragraph reformat width was made configurable, a bug was introduced that caused reformatting of comment blocks to ignore how far the block was indented, effectively adding the indentation width to the reformat width. This has been repaired, and the reformat width is again a bound on the total width of reformatted lines. -What's New in IDLE 1.1b1? -========================= - -*Release date: 15-OCT-2004* - - -What's New in IDLE 1.1a3? -========================= - -*Release date: 02-SEP-2004* - - Improve keyboard focus binding, especially in Windows menu. Improve window raising, especially in the Windows menu and in the debugger. IDLEfork 763524. @@ -574,24 +513,12 @@ - If user passes a non-existent filename on the commandline, just open a new file, don't raise a dialog. IDLEfork 854928. - -What's New in IDLE 1.1a2? -========================= - -*Release date: 05-AUG-2004* - - EditorWindow.py was not finding the .chm help file on Windows. Typo at Rev 1.54. Python Bug 990954 - checking sys.platform for substring 'win' was breaking IDLE docs on Mac (darwin). Also, Mac Safari browser requires full file:// URIs. SF 900580. - -What's New in IDLE 1.1a1? -========================= - -*Release date: 08-JUL-2004* - - Redirect the warning stream to the shell during the ScriptBinding check of user code and format the warning similarly to an exception for both that check and for runtime warnings raised in the subprocess. @@ -654,26 +581,13 @@ What's New in IDLE 1.0? ======================= - *Release date: 29-Jul-2003* - Added a banner to the shell discussing warnings possibly raised by personal firewall software. Added same comment to README.txt. - -What's New in IDLE 1.0 release candidate 2? -=========================================== - -*Release date: 24-Jul-2003* - - Calltip error when docstring was None Python Bug 775541 - -What's New in IDLE 1.0 release candidate 1? -=========================================== - -*Release date: 18-Jul-2003* - - Updated extend.txt, help.txt, and config-extensions.def to correctly reflect the current status of the configuration system. Python Bug 768469 @@ -689,12 +603,6 @@ sys.std{in|out|err}.encoding, for both the local and the subprocess case. SF IDLEfork patch 682347. - -What's New in IDLE 1.0b2? -========================= - -*Release date: 29-Jun-2003* - - Extend AboutDialog.ViewFile() to support file encodings. Make the CREDITS file Latin-1. @@ -733,7 +641,6 @@ What's New in IDLEfork 0.9b1? ============================= - *Release date: 02-Jun-2003* - The current working directory of the execution environment (and shell @@ -835,10 +742,8 @@ exception formatting to the subprocess. - What's New in IDLEfork 0.9 Alpha 2? =================================== - *Release date: 27-Jan-2003* - Updated INSTALL.txt to claify use of the python2 rpm. @@ -942,7 +847,6 @@ What's New in IDLEfork 0.9 Alpha 1? =================================== - *Release date: 31-Dec-2002* - First release of major new functionality. For further details refer to -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 21 07:11:43 2015 From: python-checkins at python.org (terry.reedy) Date: Mon, 21 Sep 2015 05:11:43 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Move_items_from_NEWS_to_idlelib/NEWS=2Etxt=2E__Standardi?= =?utf-8?q?ze_headers_spacing=3A_2_lines?= Message-ID: <20150921051143.9935.42550@psf.io> https://hg.python.org/cpython/rev/ab6f520f3637 changeset: 98123:ab6f520f3637 parent: 98118:020377a15708 parent: 98122:e758988851b7 user: Terry Jan Reedy date: Mon Sep 21 01:11:26 2015 -0400 summary: Move items from NEWS to idlelib/NEWS.txt. Standardize headers spacing: 2 lines above "What's New and 0 lines above "Release date". Remove most old headers for non-final releases (they currently do not get carried forward. files: Lib/idlelib/NEWS.txt | 142 ++++++------------------------ 1 files changed, 28 insertions(+), 114 deletions(-) diff --git a/Lib/idlelib/NEWS.txt b/Lib/idlelib/NEWS.txt --- a/Lib/idlelib/NEWS.txt +++ b/Lib/idlelib/NEWS.txt @@ -1,6 +1,26 @@ +What's New in IDLE 3.6.0a1? +=========================== +*Release date: 2017?* + +- Issue #25199: Idle: add synchronization comments for future maintainers. + +- Issue #16893: Replace help.txt with idle.html for Idle doc display. + The new idlelib/idle.html is copied from Doc/build/html/idle.html. + It looks better than help.txt and will better document Idle as released. + The tkinter html viewer that works for this file was written by Rose Roseman. + The now unused EditorWindow.HelpDialog class and helt.txt file are deprecated. + +- Issue #24199: Deprecate unused idlelib.idlever with possible removal in 3.6. + +- Issue #24782: In Idle extension config dialog, replace tabs with sorted list. + Patch by Mark Roseman. + +- Issue #24790: Remove extraneous code (which also create 2 & 3 conflicts). + + What's New in IDLE 3.5.0? ========================= -*Release date: 2015-09-13* ?? +*Release date: 2015-09-13* - Issue #23672: Allow Idle to edit and run files with astral chars in name. Patch by Mohd Sanad Zaki Rizvi. @@ -42,7 +62,7 @@ - Issue #21986: Code objects are not normally pickled by the pickle module. To match this, they are no longer pickled when running under Idle. - + - Issue #23180: Rename IDLE "Windows" menu item to "Window". Patch by Al Sweigart. @@ -198,6 +218,7 @@ - Issue #2665: On Windows, an IDLE installation upgraded from an old version would not start if a custom theme was defined. + What's New in IDLE 2.7? (UNRELEASED, but merged into 3.1 releases above.) ======================= *Release date: XX-XXX-2010* @@ -223,9 +244,9 @@ - Issue #3549: On MacOS the preferences menu was not present -What's New in IDLE 3.0 final? -============================= +What's New in IDLE 3.0? +======================= *Release date: 03-Dec-2008* - IDLE would print a "Unhandled server exception!" message when internal @@ -237,12 +258,6 @@ - Issue #4383: When IDLE cannot make the connection to its subprocess, it would fail to properly display the error message. - -What's New in IDLE 3.0a3? -========================= - -*Release date: 29-Feb-2008* - - help() was not paging to the shell. Issue1650. - CodeContext was not importing. @@ -254,21 +269,9 @@ - Issue #1585: IDLE uses non-existent xrange() function. - -What's New in IDLE 3.0a2? -========================= - -*Release date: 06-Dec-2007* - - Windows EOL sequence not converted correctly, encoding error. Caused file save to fail. Bug 1130. - -What's New in IDLE 3.0a1? -========================= - -*Release date: 31-Aug-2007* - - IDLE converted to Python 3000 syntax. - Strings became Unicode. @@ -282,9 +285,8 @@ be cleared before IDLE exits. -What's New in IDLE 2.6 final? -============================= - +What's New in IDLE 2.6 +====================== *Release date: 01-Oct-2008*, merged into 3.0 releases detailed above (3.0rc2) - Issue #2665: On Windows, an IDLE installation upgraded from an old version @@ -369,15 +371,8 @@ What's New in IDLE 1.2? ======================= - *Release date: 19-SEP-2006* - -What's New in IDLE 1.2c1? -========================= - -*Release date: 17-AUG-2006* - - File menu hotkeys: there were three 'p' assignments. Reassign the 'Save Copy As' and 'Print' hotkeys to 'y' and 't'. Change the Shell hotkey from 's' to 'l'. @@ -398,11 +393,6 @@ - When used w/o subprocess, all exceptions were preceded by an error message claiming they were IDLE internal errors (since 1.2a1). -What's New in IDLE 1.2b3? -========================= - -*Release date: 03-AUG-2006* - - Bug #1525817: Don't truncate short lines in IDLE's tool tips. - Bug #1517990: IDLE keybindings on MacOS X now work correctly @@ -426,26 +416,6 @@ 'as' keyword in comment directly following import command. Closes 1325071. Patch 1479219 Tal Einat -What's New in IDLE 1.2b2? -========================= - -*Release date: 11-JUL-2006* - -What's New in IDLE 1.2b1? -========================= - -*Release date: 20-JUN-2006* - -What's New in IDLE 1.2a2? -========================= - -*Release date: 27-APR-2006* - -What's New in IDLE 1.2a1? -========================= - -*Release date: 05-APR-2006* - - Patch #1162825: Support non-ASCII characters in IDLE window titles. - Source file f.flush() after writing; trying to avoid lossage if user @@ -525,19 +495,14 @@ - The remote procedure call module rpc.py can now access data attributes of remote registered objects. Changes to these attributes are local, however. + What's New in IDLE 1.1? ======================= - *Release date: 30-NOV-2004* - On OpenBSD, terminating IDLE with ctrl-c from the command line caused a stuck subprocess MainThread because only the SocketThread was exiting. -What's New in IDLE 1.1b3/rc1? -============================= - -*Release date: 18-NOV-2004* - - Saving a Keyset w/o making changes (by using the "Save as New Custom Key Set" button) caused IDLE to fail on restart (no new keyset was created in config-keys.cfg). Also true for Theme/highlights. Python Bug 1064535. @@ -545,28 +510,12 @@ - A change to the linecache.py API caused IDLE to exit when an exception was raised while running without the subprocess (-n switch). Python Bug 1063840. -What's New in IDLE 1.1b2? -========================= - -*Release date: 03-NOV-2004* - - When paragraph reformat width was made configurable, a bug was introduced that caused reformatting of comment blocks to ignore how far the block was indented, effectively adding the indentation width to the reformat width. This has been repaired, and the reformat width is again a bound on the total width of reformatted lines. -What's New in IDLE 1.1b1? -========================= - -*Release date: 15-OCT-2004* - - -What's New in IDLE 1.1a3? -========================= - -*Release date: 02-SEP-2004* - - Improve keyboard focus binding, especially in Windows menu. Improve window raising, especially in the Windows menu and in the debugger. IDLEfork 763524. @@ -574,24 +523,12 @@ - If user passes a non-existent filename on the commandline, just open a new file, don't raise a dialog. IDLEfork 854928. - -What's New in IDLE 1.1a2? -========================= - -*Release date: 05-AUG-2004* - - EditorWindow.py was not finding the .chm help file on Windows. Typo at Rev 1.54. Python Bug 990954 - checking sys.platform for substring 'win' was breaking IDLE docs on Mac (darwin). Also, Mac Safari browser requires full file:// URIs. SF 900580. - -What's New in IDLE 1.1a1? -========================= - -*Release date: 08-JUL-2004* - - Redirect the warning stream to the shell during the ScriptBinding check of user code and format the warning similarly to an exception for both that check and for runtime warnings raised in the subprocess. @@ -654,26 +591,13 @@ What's New in IDLE 1.0? ======================= - *Release date: 29-Jul-2003* - Added a banner to the shell discussing warnings possibly raised by personal firewall software. Added same comment to README.txt. - -What's New in IDLE 1.0 release candidate 2? -=========================================== - -*Release date: 24-Jul-2003* - - Calltip error when docstring was None Python Bug 775541 - -What's New in IDLE 1.0 release candidate 1? -=========================================== - -*Release date: 18-Jul-2003* - - Updated extend.txt, help.txt, and config-extensions.def to correctly reflect the current status of the configuration system. Python Bug 768469 @@ -689,12 +613,6 @@ sys.std{in|out|err}.encoding, for both the local and the subprocess case. SF IDLEfork patch 682347. - -What's New in IDLE 1.0b2? -========================= - -*Release date: 29-Jun-2003* - - Extend AboutDialog.ViewFile() to support file encodings. Make the CREDITS file Latin-1. @@ -733,7 +651,6 @@ What's New in IDLEfork 0.9b1? ============================= - *Release date: 02-Jun-2003* - The current working directory of the execution environment (and shell @@ -835,10 +752,8 @@ exception formatting to the subprocess. - What's New in IDLEfork 0.9 Alpha 2? =================================== - *Release date: 27-Jan-2003* - Updated INSTALL.txt to claify use of the python2 rpm. @@ -942,7 +857,6 @@ What's New in IDLEfork 0.9 Alpha 1? =================================== - *Release date: 31-Dec-2002* - First release of major new functionality. For further details refer to -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 21 07:44:47 2015 From: python-checkins at python.org (terry.reedy) Date: Mon, 21 Sep 2015 05:44:47 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzI0ODYx?= =?utf-8?q?=3A_Add_docstring_to_idlelib=2E=5F=5Finit=5F=5F_with_=27private?= =?utf-8?q?=27_warning=2E?= Message-ID: <20150921054447.16585.36224@psf.io> https://hg.python.org/cpython/rev/b7bbb2c1e1f9 changeset: 98124:b7bbb2c1e1f9 branch: 2.7 parent: 98120:a108db755a59 user: Terry Jan Reedy date: Mon Sep 21 01:44:00 2015 -0400 summary: Issue #24861: Add docstring to idlelib.__init__ with 'private' warning. files: Lib/idlelib/__init__.py | 9 ++++++++- 1 files changed, 8 insertions(+), 1 deletions(-) diff --git a/Lib/idlelib/__init__.py b/Lib/idlelib/__init__.py --- a/Lib/idlelib/__init__.py +++ b/Lib/idlelib/__init__.py @@ -1,1 +1,8 @@ -# Dummy file to make this a package. +"""The idlelib package implements the Idle application. + +Idle includes an interactive shell and editor. +Use the files named idle.* to start Idle. + +The other files are private implementations. Their details are subject +to change. See PEP 434 for more. Import them at your own risk. +""" -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 21 07:44:48 2015 From: python-checkins at python.org (terry.reedy) Date: Mon, 21 Sep 2015 05:44:48 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzI0ODYx?= =?utf-8?q?=3A_Add_docstring_to_idlelib=2E=5F=5Finit=5F=5F_with_=27private?= =?utf-8?q?=27_warning=2E?= Message-ID: <20150921054448.82652.72919@psf.io> https://hg.python.org/cpython/rev/084a8813f05f changeset: 98125:084a8813f05f branch: 3.4 parent: 98121:ae352d27618d user: Terry Jan Reedy date: Mon Sep 21 01:44:06 2015 -0400 summary: Issue #24861: Add docstring to idlelib.__init__ with 'private' warning. files: Lib/idlelib/__init__.py | 9 ++++++++- 1 files changed, 8 insertions(+), 1 deletions(-) diff --git a/Lib/idlelib/__init__.py b/Lib/idlelib/__init__.py --- a/Lib/idlelib/__init__.py +++ b/Lib/idlelib/__init__.py @@ -1,1 +1,8 @@ -# Dummy file to make this a package. +"""The idlelib package implements the Idle application. + +Idle includes an interactive shell and editor. +Use the files named idle.* to start Idle. + +The other files are private implementations. Their details are subject to +change. See PEP 434 for more. Import them at your own risk. +""" -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 21 07:44:48 2015 From: python-checkins at python.org (terry.reedy) Date: Mon, 21 Sep 2015 05:44:48 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Merge_with_3=2E4?= Message-ID: <20150921054448.98378.32020@psf.io> https://hg.python.org/cpython/rev/06d6c9b7fb4f changeset: 98126:06d6c9b7fb4f branch: 3.5 parent: 98122:e758988851b7 parent: 98125:084a8813f05f user: Terry Jan Reedy date: Mon Sep 21 01:44:20 2015 -0400 summary: Merge with 3.4 files: Lib/idlelib/__init__.py | 9 ++++++++- 1 files changed, 8 insertions(+), 1 deletions(-) diff --git a/Lib/idlelib/__init__.py b/Lib/idlelib/__init__.py --- a/Lib/idlelib/__init__.py +++ b/Lib/idlelib/__init__.py @@ -1,1 +1,8 @@ -# Dummy file to make this a package. +"""The idlelib package implements the Idle application. + +Idle includes an interactive shell and editor. +Use the files named idle.* to start Idle. + +The other files are private implementations. Their details are subject to +change. See PEP 434 for more. Import them at your own risk. +""" -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 21 07:44:48 2015 From: python-checkins at python.org (terry.reedy) Date: Mon, 21 Sep 2015 05:44:48 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge_with_3=2E5?= Message-ID: <20150921054448.94107.1870@psf.io> https://hg.python.org/cpython/rev/b31cc00d4363 changeset: 98127:b31cc00d4363 parent: 98123:ab6f520f3637 parent: 98126:06d6c9b7fb4f user: Terry Jan Reedy date: Mon Sep 21 01:44:33 2015 -0400 summary: Merge with 3.5 files: Lib/idlelib/__init__.py | 9 ++++++++- 1 files changed, 8 insertions(+), 1 deletions(-) diff --git a/Lib/idlelib/__init__.py b/Lib/idlelib/__init__.py --- a/Lib/idlelib/__init__.py +++ b/Lib/idlelib/__init__.py @@ -1,1 +1,8 @@ -# Dummy file to make this a package. +"""The idlelib package implements the Idle application. + +Idle includes an interactive shell and editor. +Use the files named idle.* to start Idle. + +The other files are private implementations. Their details are subject to +change. See PEP 434 for more. Import them at your own risk. +""" -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 21 09:09:02 2015 From: python-checkins at python.org (victor.stinner) Date: Mon, 21 Sep 2015 07:09:02 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Merge_3=2E4_=28test=5Fsocket=2C_issue_=2325138=29?= Message-ID: <20150921070902.31205.73968@psf.io> https://hg.python.org/cpython/rev/a7baccf0b1c2 changeset: 98129:a7baccf0b1c2 branch: 3.5 parent: 98126:06d6c9b7fb4f parent: 98128:f13a5b5a2824 user: Victor Stinner date: Mon Sep 21 09:04:17 2015 +0200 summary: Merge 3.4 (test_socket, issue #25138) files: Lib/test/test_socket.py | 9 ++++----- 1 files changed, 4 insertions(+), 5 deletions(-) diff --git a/Lib/test/test_socket.py b/Lib/test/test_socket.py --- a/Lib/test/test_socket.py +++ b/Lib/test/test_socket.py @@ -1289,12 +1289,11 @@ @unittest.skipUnless(support.is_resource_enabled('network'), 'network is not enabled') def test_idna(self): - # Check for internet access before running test (issue #12804). - try: + # Check for internet access before running test + # (issue #12804, issue #25138). + with support.transient_internet('python.org'): socket.gethostbyname('python.org') - except socket.gaierror as e: - if e.errno == socket.EAI_NODATA: - self.skipTest('internet access required for this test') + # these should all be successful domain = '?????????.pythontest.net' socket.gethostbyname(domain) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 21 09:09:02 2015 From: python-checkins at python.org (victor.stinner) Date: Mon, 21 Sep 2015 07:09:02 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge_3=2E5_=28test=5Fsocket=2C_issue_=2325138=29?= Message-ID: <20150921070902.9935.65141@psf.io> https://hg.python.org/cpython/rev/d8dd06ab00e4 changeset: 98130:d8dd06ab00e4 parent: 98127:b31cc00d4363 parent: 98129:a7baccf0b1c2 user: Victor Stinner date: Mon Sep 21 09:06:53 2015 +0200 summary: Merge 3.5 (test_socket, issue #25138) files: Lib/test/test_socket.py | 9 ++++----- 1 files changed, 4 insertions(+), 5 deletions(-) diff --git a/Lib/test/test_socket.py b/Lib/test/test_socket.py --- a/Lib/test/test_socket.py +++ b/Lib/test/test_socket.py @@ -1289,12 +1289,11 @@ @unittest.skipUnless(support.is_resource_enabled('network'), 'network is not enabled') def test_idna(self): - # Check for internet access before running test (issue #12804). - try: + # Check for internet access before running test + # (issue #12804, issue #25138). + with support.transient_internet('python.org'): socket.gethostbyname('python.org') - except socket.gaierror as e: - if e.errno == socket.EAI_NODATA: - self.skipTest('internet access required for this test') + # these should all be successful domain = '?????????.pythontest.net' socket.gethostbyname(domain) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 21 09:09:02 2015 From: python-checkins at python.org (victor.stinner) Date: Mon, 21 Sep 2015 07:09:02 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzI1MTM4?= =?utf-8?q?=3A_test=5Fsocket=2Etest=5Fidna=28=29_uses_support=2Etransient?= =?utf-8?q?=5Finternet=28=29_instead?= Message-ID: <20150921070902.98366.12710@psf.io> https://hg.python.org/cpython/rev/f13a5b5a2824 changeset: 98128:f13a5b5a2824 branch: 3.4 parent: 98125:084a8813f05f user: Victor Stinner date: Mon Sep 21 09:04:01 2015 +0200 summary: Issue #25138: test_socket.test_idna() uses support.transient_internet() instead of catching socket.EAI_NODATA error which doesn't exist on FreeBSD. files: Lib/test/test_socket.py | 9 ++++----- 1 files changed, 4 insertions(+), 5 deletions(-) diff --git a/Lib/test/test_socket.py b/Lib/test/test_socket.py --- a/Lib/test/test_socket.py +++ b/Lib/test/test_socket.py @@ -1284,12 +1284,11 @@ @unittest.skipUnless(support.is_resource_enabled('network'), 'network is not enabled') def test_idna(self): - # Check for internet access before running test (issue #12804). - try: + # Check for internet access before running test + # (issue #12804, issue #25138). + with support.transient_internet('python.org'): socket.gethostbyname('python.org') - except socket.gaierror as e: - if e.errno == socket.EAI_NODATA: - self.skipTest('internet access required for this test') + # these should all be successful domain = '?????????.pythontest.net' socket.gethostbyname(domain) -- Repository URL: https://hg.python.org/cpython From solipsis at pitrou.net Mon Sep 21 10:45:32 2015 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Mon, 21 Sep 2015 08:45:32 +0000 Subject: [Python-checkins] Daily reference leaks (b31cc00d4363): sum=17877 Message-ID: <20150921084532.115149.21084@psf.io> results for b31cc00d4363 on branch "default" -------------------------------------------- test_capi leaked [1598, 1598, 1598] references, sum=4794 test_capi leaked [387, 389, 389] memory blocks, sum=1165 test_functools leaked [0, 2, 2] memory blocks, sum=4 test_threading leaked [3196, 3196, 3196] references, sum=9588 test_threading leaked [774, 776, 776] memory blocks, sum=2326 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/psf-users/antoine/refleaks/reflogt0FUT9', '--timeout', '7200'] From python-checkins at python.org Mon Sep 21 14:05:24 2015 From: python-checkins at python.org (victor.stinner) Date: Mon, 21 Sep 2015 12:05:24 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge_3=2E5_=28test=5Feintr=2C_FreeBSD=29?= Message-ID: <20150921120524.11694.98741@psf.io> https://hg.python.org/cpython/rev/28d56d7deafc changeset: 98132:28d56d7deafc parent: 98130:d8dd06ab00e4 parent: 98131:3184c43627f5 user: Victor Stinner date: Mon Sep 21 14:05:18 2015 +0200 summary: Merge 3.5 (test_eintr, FreeBSD) files: Lib/test/eintrdata/eintr_tester.py | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Lib/test/eintrdata/eintr_tester.py b/Lib/test/eintrdata/eintr_tester.py --- a/Lib/test/eintrdata/eintr_tester.py +++ b/Lib/test/eintrdata/eintr_tester.py @@ -301,9 +301,9 @@ # Issue #25122: There is a race condition in the FreeBSD kernel on # handling signals in the FIFO device. Skip the test until the bug is - # fixed in the kernel. Bet that the bug will be fixed in FreeBSD 11. + # fixed in the kernel. # https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=203162 - @support.requires_freebsd_version(11) + @support.requires_freebsd_version(10, 3) @unittest.skipUnless(hasattr(os, 'mkfifo'), 'needs mkfifo()') def _test_open(self, do_open_close_reader, do_open_close_writer): filename = support.TESTFN -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 21 14:05:24 2015 From: python-checkins at python.org (victor.stinner) Date: Mon, 21 Sep 2015 12:05:24 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI1MTIy?= =?utf-8?q?=3A_test=5Feintr=3A_the_FreeBSD_fix_will_be_released_in_FreeBSD?= =?utf-8?q?_10=2E3?= Message-ID: <20150921120524.11684.65728@psf.io> https://hg.python.org/cpython/rev/3184c43627f5 changeset: 98131:3184c43627f5 branch: 3.5 parent: 98129:a7baccf0b1c2 user: Victor Stinner date: Mon Sep 21 14:05:02 2015 +0200 summary: Issue #25122: test_eintr: the FreeBSD fix will be released in FreeBSD 10.3 files: Lib/test/eintrdata/eintr_tester.py | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Lib/test/eintrdata/eintr_tester.py b/Lib/test/eintrdata/eintr_tester.py --- a/Lib/test/eintrdata/eintr_tester.py +++ b/Lib/test/eintrdata/eintr_tester.py @@ -301,9 +301,9 @@ # Issue #25122: There is a race condition in the FreeBSD kernel on # handling signals in the FIFO device. Skip the test until the bug is - # fixed in the kernel. Bet that the bug will be fixed in FreeBSD 11. + # fixed in the kernel. # https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=203162 - @support.requires_freebsd_version(11) + @support.requires_freebsd_version(10, 3) @unittest.skipUnless(hasattr(os, 'mkfifo'), 'needs mkfifo()') def _test_open(self, do_open_close_reader, do_open_close_writer): filename = support.TESTFN -- Repository URL: https://hg.python.org/cpython From lp_benchmark_robot at intel.com Mon Sep 21 16:39:32 2015 From: lp_benchmark_robot at intel.com (lp_benchmark_robot) Date: Mon, 21 Sep 2015 14:39:32 +0000 Subject: [Python-checkins] Benchmark Results for Python Default 2015-09-21 Message-ID: Results for project python_default-nightly, build date 2015-09-21 07:36:56 commit: d8dd06ab00e409d454e565d6a0ff6648dfe9bf5e revision date: 2015-09-21 07:06:53 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v3.4.3, with hash b4cbecbc0781e89a309d03b60a1f75f8499250e6 from 2015-02-25 12:15:33+00:00 ------------------------------------------------------------------------------------------ benchmark relative change since change since current rev with std_dev* last run v3.4.3 regrtest PGO ------------------------------------------------------------------------------------------ :-) django_v2 0.23543% 0.59668% 9.46779% 15.29038% :-| pybench 0.16008% 0.51273% -1.93772% 8.05169% :-( regex_v8 2.93551% -1.20108% -5.08906% 6.01129% :-| nbody 0.09854% -0.65144% -1.62090% 10.02030% :-( json_dump_v2 0.18227% -1.51844% -2.66127% 11.82505% :-| normal_startup 0.66716% 0.16060% 0.33732% 4.70905% ------------------------------------------------------------------------------------------ Note: Benchmark results are measured in seconds. * Relative Standard Deviation (Standard Deviation/Average) Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. From lp_benchmark_robot at intel.com Mon Sep 21 16:39:36 2015 From: lp_benchmark_robot at intel.com (lp_benchmark_robot) Date: Mon, 21 Sep 2015 14:39:36 +0000 Subject: [Python-checkins] Benchmark Results for Python 2.7 2015-09-21 Message-ID: Results for project python_2.7-nightly, build date 2015-09-21 12:51:46 commit: b7bbb2c1e1f92d3f7180b98ff0e63457636fe45c revision date: 2015-09-21 05:44:00 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v2.7.10, with hash 15c95b7d81dcf821daade360741e00714667653f from 2015-05-23 16:02:14+00:00 ------------------------------------------------------------------------------------------ benchmark relative change since change since current rev with std_dev* last run v2.7.10 regrtest PGO ------------------------------------------------------------------------------------------ :-) django_v2 0.25388% 1.40543% 4.53425% 8.77606% :-) pybench 0.17517% -0.02414% 6.71645% 7.12361% :-| regex_v8 0.55995% -0.04516% -1.19922% 7.19107% :-) nbody 0.11424% -0.01631% 9.03449% 3.74679% :-) json_dump_v2 0.29702% -0.29028% 4.15052% 12.92398% :-| normal_startup 1.98413% -0.04886% -1.71418% 3.56898% :-) ssbench 1.08811% 0.02315% 2.43770% 2.16185% ------------------------------------------------------------------------------------------ Note: Benchmark results for ssbench are measured in requests/second while all other are measured in seconds. * Relative Standard Deviation (Standard Deviation/Average) Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. From python-checkins at python.org Mon Sep 21 18:09:35 2015 From: python-checkins at python.org (victor.stinner) Date: Mon, 21 Sep 2015 16:09:35 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge_3=2E5_=28asyncio=29?= Message-ID: <20150921160935.9945.26577@psf.io> https://hg.python.org/cpython/rev/db13588ec523 changeset: 98135:db13588ec523 parent: 98132:28d56d7deafc parent: 98134:4115bc40b5ef user: Victor Stinner date: Mon Sep 21 18:08:27 2015 +0200 summary: Merge 3.5 (asyncio) files: Doc/library/asyncio-protocol.rst | 5 + Lib/asyncio/selector_events.py | 1 + Lib/asyncio/sslproto.py | 4 + Lib/test/test_asyncio/test_events.py | 75 +++++++++++++-- 4 files changed, 73 insertions(+), 12 deletions(-) diff --git a/Doc/library/asyncio-protocol.rst b/Doc/library/asyncio-protocol.rst --- a/Doc/library/asyncio-protocol.rst +++ b/Doc/library/asyncio-protocol.rst @@ -71,6 +71,8 @@ - ``'peercert'``: peer certificate; result of :meth:`ssl.SSLSocket.getpeercert` - ``'sslcontext'``: :class:`ssl.SSLContext` instance + - ``'ssl_object'``: :class:`ssl.SSLObject` or :class:`ssl.SSLSocket` + instance * pipe: @@ -80,6 +82,9 @@ - ``'subprocess'``: :class:`subprocess.Popen` instance + .. versionchanged:: 3.4.4 + ``'ssl_object'`` info was added to SSL sockets. + ReadTransport ------------- diff --git a/Lib/asyncio/selector_events.py b/Lib/asyncio/selector_events.py --- a/Lib/asyncio/selector_events.py +++ b/Lib/asyncio/selector_events.py @@ -843,6 +843,7 @@ self._extra.update(peercert=peercert, cipher=self._sock.cipher(), compression=self._sock.compression(), + ssl_object=self._sock, ) self._read_wants_write = False diff --git a/Lib/asyncio/sslproto.py b/Lib/asyncio/sslproto.py --- a/Lib/asyncio/sslproto.py +++ b/Lib/asyncio/sslproto.py @@ -295,6 +295,7 @@ def __init__(self, loop, ssl_protocol, app_protocol): self._loop = loop + # SSLProtocol instance self._ssl_protocol = ssl_protocol self._app_protocol = app_protocol self._closed = False @@ -425,10 +426,12 @@ self._app_protocol = app_protocol self._app_transport = _SSLProtocolTransport(self._loop, self, self._app_protocol) + # _SSLPipe instance (None until the connection is made) self._sslpipe = None self._session_established = False self._in_handshake = False self._in_shutdown = False + # transport, ex: SelectorSocketTransport self._transport = None def _wakeup_waiter(self, exc=None): @@ -591,6 +594,7 @@ self._extra.update(peercert=peercert, cipher=sslobj.cipher(), compression=sslobj.compression(), + ssl_object=sslobj, ) self._app_protocol.connection_made(self._app_transport) self._wakeup_waiter() diff --git a/Lib/test/test_asyncio/test_events.py b/Lib/test/test_asyncio/test_events.py --- a/Lib/test/test_asyncio/test_events.py +++ b/Lib/test/test_asyncio/test_events.py @@ -57,6 +57,17 @@ ONLYKEY = data_file('ssl_key.pem') SIGNED_CERTFILE = data_file('keycert3.pem') SIGNING_CA = data_file('pycacert.pem') +PEERCERT = {'serialNumber': 'B09264B1F2DA21D1', + 'version': 1, + 'subject': ((('countryName', 'XY'),), + (('localityName', 'Castle Anthrax'),), + (('organizationName', 'Python Software Foundation'),), + (('commonName', 'localhost'),)), + 'issuer': ((('countryName', 'XY'),), + (('organizationName', 'Python Software Foundation CA'),), + (('commonName', 'our-ca-server'),)), + 'notAfter': 'Nov 13 19:47:07 2022 GMT', + 'notBefore': 'Jan 4 19:47:07 2013 GMT'} class MyBaseProto(asyncio.Protocol): @@ -596,22 +607,56 @@ self.assertGreater(pr.nbytes, 0) tr.close() + def check_ssl_extra_info(self, client, check_sockname=True, + peername=None, peercert={}): + if check_sockname: + self.assertIsNotNone(client.get_extra_info('sockname')) + if peername: + self.assertEqual(peername, + client.get_extra_info('peername')) + else: + self.assertIsNotNone(client.get_extra_info('peername')) + self.assertEqual(peercert, + client.get_extra_info('peercert')) + + # Python disables compression to prevent CRIME attacks by default + self.assertIsNone(client.get_extra_info('compression')) + + # test SSL cipher + cipher = client.get_extra_info('cipher') + self.assertIsInstance(cipher, tuple) + self.assertEqual(len(cipher), 3, cipher) + self.assertIsInstance(cipher[0], str) + self.assertIsInstance(cipher[1], str) + self.assertIsInstance(cipher[2], int) + + # test SSL object + sslobj = client.get_extra_info('ssl_object') + self.assertIsNotNone(sslobj) + self.assertEqual(sslobj.compression(), + client.get_extra_info('compression')) + self.assertEqual(sslobj.cipher(), + client.get_extra_info('cipher')) + self.assertEqual(sslobj.getpeercert(), + client.get_extra_info('peercert')) + def _basetest_create_ssl_connection(self, connection_fut, - check_sockname=True): + check_sockname=True, + peername=None): tr, pr = self.loop.run_until_complete(connection_fut) self.assertIsInstance(tr, asyncio.Transport) self.assertIsInstance(pr, asyncio.Protocol) self.assertTrue('ssl' in tr.__class__.__name__.lower()) - if check_sockname: - self.assertIsNotNone(tr.get_extra_info('sockname')) + self.check_ssl_extra_info(tr, check_sockname, peername) self.loop.run_until_complete(pr.done) self.assertGreater(pr.nbytes, 0) tr.close() def _test_create_ssl_connection(self, httpd, create_connection, - check_sockname=True): + check_sockname=True, peername=None): conn_fut = create_connection(ssl=test_utils.dummy_ssl_context()) - self._basetest_create_ssl_connection(conn_fut, check_sockname) + self._basetest_create_ssl_connection(conn_fut, check_sockname, + peername) # ssl.Purpose was introduced in Python 3.4 if hasattr(ssl, 'Purpose'): @@ -629,7 +674,8 @@ with mock.patch('ssl.create_default_context', side_effect=_dummy_ssl_create_context) as m: conn_fut = create_connection(ssl=True) - self._basetest_create_ssl_connection(conn_fut, check_sockname) + self._basetest_create_ssl_connection(conn_fut, check_sockname, + peername) self.assertEqual(m.call_count, 1) # With the real ssl.create_default_context(), certificate @@ -638,7 +684,8 @@ conn_fut = create_connection(ssl=True) # Ignore the "SSL handshake failed" log in debug mode with test_utils.disable_logger(): - self._basetest_create_ssl_connection(conn_fut, check_sockname) + self._basetest_create_ssl_connection(conn_fut, check_sockname, + peername) self.assertEqual(cm.exception.reason, 'CERTIFICATE_VERIFY_FAILED') @@ -649,7 +696,8 @@ self.loop.create_connection, lambda: MyProto(loop=self.loop), *httpd.address) - self._test_create_ssl_connection(httpd, create_connection) + self._test_create_ssl_connection(httpd, create_connection, + peername=httpd.address) def test_legacy_create_ssl_connection(self): with test_utils.force_legacy_ssl_support(): @@ -669,7 +717,8 @@ server_hostname='127.0.0.1') self._test_create_ssl_connection(httpd, create_connection, - check_sockname) + check_sockname, + peername=httpd.address) def test_legacy_create_ssl_unix_connection(self): with test_utils.force_legacy_ssl_support(): @@ -819,9 +868,7 @@ self.assertEqual(3, proto.nbytes) # extra info is available - self.assertIsNotNone(proto.transport.get_extra_info('sockname')) - self.assertEqual('127.0.0.1', - proto.transport.get_extra_info('peername')[0]) + self.check_ssl_extra_info(client, peername=(host, port)) # close connection proto.transport.close() @@ -1023,6 +1070,10 @@ server_hostname='localhost') client, pr = self.loop.run_until_complete(f_c) + # extra info is available + self.check_ssl_extra_info(client,peername=(host, port), + peercert=PEERCERT) + # close connection proto.transport.close() client.close() -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 21 18:09:36 2015 From: python-checkins at python.org (victor.stinner) Date: Mon, 21 Sep 2015 16:09:36 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Merge_3=2E4_=28asyncio=29?= Message-ID: <20150921160935.9933.46425@psf.io> https://hg.python.org/cpython/rev/4115bc40b5ef changeset: 98134:4115bc40b5ef branch: 3.5 parent: 98131:3184c43627f5 parent: 98133:d7859e7e7071 user: Victor Stinner date: Mon Sep 21 18:08:06 2015 +0200 summary: Merge 3.4 (asyncio) files: Doc/library/asyncio-protocol.rst | 5 + Lib/asyncio/selector_events.py | 1 + Lib/asyncio/sslproto.py | 4 + Lib/test/test_asyncio/test_events.py | 75 +++++++++++++-- 4 files changed, 73 insertions(+), 12 deletions(-) diff --git a/Doc/library/asyncio-protocol.rst b/Doc/library/asyncio-protocol.rst --- a/Doc/library/asyncio-protocol.rst +++ b/Doc/library/asyncio-protocol.rst @@ -71,6 +71,8 @@ - ``'peercert'``: peer certificate; result of :meth:`ssl.SSLSocket.getpeercert` - ``'sslcontext'``: :class:`ssl.SSLContext` instance + - ``'ssl_object'``: :class:`ssl.SSLObject` or :class:`ssl.SSLSocket` + instance * pipe: @@ -80,6 +82,9 @@ - ``'subprocess'``: :class:`subprocess.Popen` instance + .. versionchanged:: 3.4.4 + ``'ssl_object'`` info was added to SSL sockets. + ReadTransport ------------- diff --git a/Lib/asyncio/selector_events.py b/Lib/asyncio/selector_events.py --- a/Lib/asyncio/selector_events.py +++ b/Lib/asyncio/selector_events.py @@ -843,6 +843,7 @@ self._extra.update(peercert=peercert, cipher=self._sock.cipher(), compression=self._sock.compression(), + ssl_object=self._sock, ) self._read_wants_write = False diff --git a/Lib/asyncio/sslproto.py b/Lib/asyncio/sslproto.py --- a/Lib/asyncio/sslproto.py +++ b/Lib/asyncio/sslproto.py @@ -295,6 +295,7 @@ def __init__(self, loop, ssl_protocol, app_protocol): self._loop = loop + # SSLProtocol instance self._ssl_protocol = ssl_protocol self._app_protocol = app_protocol self._closed = False @@ -425,10 +426,12 @@ self._app_protocol = app_protocol self._app_transport = _SSLProtocolTransport(self._loop, self, self._app_protocol) + # _SSLPipe instance (None until the connection is made) self._sslpipe = None self._session_established = False self._in_handshake = False self._in_shutdown = False + # transport, ex: SelectorSocketTransport self._transport = None def _wakeup_waiter(self, exc=None): @@ -591,6 +594,7 @@ self._extra.update(peercert=peercert, cipher=sslobj.cipher(), compression=sslobj.compression(), + ssl_object=sslobj, ) self._app_protocol.connection_made(self._app_transport) self._wakeup_waiter() diff --git a/Lib/test/test_asyncio/test_events.py b/Lib/test/test_asyncio/test_events.py --- a/Lib/test/test_asyncio/test_events.py +++ b/Lib/test/test_asyncio/test_events.py @@ -57,6 +57,17 @@ ONLYKEY = data_file('ssl_key.pem') SIGNED_CERTFILE = data_file('keycert3.pem') SIGNING_CA = data_file('pycacert.pem') +PEERCERT = {'serialNumber': 'B09264B1F2DA21D1', + 'version': 1, + 'subject': ((('countryName', 'XY'),), + (('localityName', 'Castle Anthrax'),), + (('organizationName', 'Python Software Foundation'),), + (('commonName', 'localhost'),)), + 'issuer': ((('countryName', 'XY'),), + (('organizationName', 'Python Software Foundation CA'),), + (('commonName', 'our-ca-server'),)), + 'notAfter': 'Nov 13 19:47:07 2022 GMT', + 'notBefore': 'Jan 4 19:47:07 2013 GMT'} class MyBaseProto(asyncio.Protocol): @@ -596,22 +607,56 @@ self.assertGreater(pr.nbytes, 0) tr.close() + def check_ssl_extra_info(self, client, check_sockname=True, + peername=None, peercert={}): + if check_sockname: + self.assertIsNotNone(client.get_extra_info('sockname')) + if peername: + self.assertEqual(peername, + client.get_extra_info('peername')) + else: + self.assertIsNotNone(client.get_extra_info('peername')) + self.assertEqual(peercert, + client.get_extra_info('peercert')) + + # Python disables compression to prevent CRIME attacks by default + self.assertIsNone(client.get_extra_info('compression')) + + # test SSL cipher + cipher = client.get_extra_info('cipher') + self.assertIsInstance(cipher, tuple) + self.assertEqual(len(cipher), 3, cipher) + self.assertIsInstance(cipher[0], str) + self.assertIsInstance(cipher[1], str) + self.assertIsInstance(cipher[2], int) + + # test SSL object + sslobj = client.get_extra_info('ssl_object') + self.assertIsNotNone(sslobj) + self.assertEqual(sslobj.compression(), + client.get_extra_info('compression')) + self.assertEqual(sslobj.cipher(), + client.get_extra_info('cipher')) + self.assertEqual(sslobj.getpeercert(), + client.get_extra_info('peercert')) + def _basetest_create_ssl_connection(self, connection_fut, - check_sockname=True): + check_sockname=True, + peername=None): tr, pr = self.loop.run_until_complete(connection_fut) self.assertIsInstance(tr, asyncio.Transport) self.assertIsInstance(pr, asyncio.Protocol) self.assertTrue('ssl' in tr.__class__.__name__.lower()) - if check_sockname: - self.assertIsNotNone(tr.get_extra_info('sockname')) + self.check_ssl_extra_info(tr, check_sockname, peername) self.loop.run_until_complete(pr.done) self.assertGreater(pr.nbytes, 0) tr.close() def _test_create_ssl_connection(self, httpd, create_connection, - check_sockname=True): + check_sockname=True, peername=None): conn_fut = create_connection(ssl=test_utils.dummy_ssl_context()) - self._basetest_create_ssl_connection(conn_fut, check_sockname) + self._basetest_create_ssl_connection(conn_fut, check_sockname, + peername) # ssl.Purpose was introduced in Python 3.4 if hasattr(ssl, 'Purpose'): @@ -629,7 +674,8 @@ with mock.patch('ssl.create_default_context', side_effect=_dummy_ssl_create_context) as m: conn_fut = create_connection(ssl=True) - self._basetest_create_ssl_connection(conn_fut, check_sockname) + self._basetest_create_ssl_connection(conn_fut, check_sockname, + peername) self.assertEqual(m.call_count, 1) # With the real ssl.create_default_context(), certificate @@ -638,7 +684,8 @@ conn_fut = create_connection(ssl=True) # Ignore the "SSL handshake failed" log in debug mode with test_utils.disable_logger(): - self._basetest_create_ssl_connection(conn_fut, check_sockname) + self._basetest_create_ssl_connection(conn_fut, check_sockname, + peername) self.assertEqual(cm.exception.reason, 'CERTIFICATE_VERIFY_FAILED') @@ -649,7 +696,8 @@ self.loop.create_connection, lambda: MyProto(loop=self.loop), *httpd.address) - self._test_create_ssl_connection(httpd, create_connection) + self._test_create_ssl_connection(httpd, create_connection, + peername=httpd.address) def test_legacy_create_ssl_connection(self): with test_utils.force_legacy_ssl_support(): @@ -669,7 +717,8 @@ server_hostname='127.0.0.1') self._test_create_ssl_connection(httpd, create_connection, - check_sockname) + check_sockname, + peername=httpd.address) def test_legacy_create_ssl_unix_connection(self): with test_utils.force_legacy_ssl_support(): @@ -819,9 +868,7 @@ self.assertEqual(3, proto.nbytes) # extra info is available - self.assertIsNotNone(proto.transport.get_extra_info('sockname')) - self.assertEqual('127.0.0.1', - proto.transport.get_extra_info('peername')[0]) + self.check_ssl_extra_info(client, peername=(host, port)) # close connection proto.transport.close() @@ -1023,6 +1070,10 @@ server_hostname='localhost') client, pr = self.loop.run_until_complete(f_c) + # extra info is available + self.check_ssl_extra_info(client,peername=(host, port), + peercert=PEERCERT) + # close connection proto.transport.close() client.close() -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 21 18:09:41 2015 From: python-checkins at python.org (victor.stinner) Date: Mon, 21 Sep 2015 16:09:41 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzI1MTE0?= =?utf-8?q?=2C_asyncio=3A_add_ssl=5Fobject_extra_info_to_SSL_transports?= Message-ID: <20150921160934.81631.76667@psf.io> https://hg.python.org/cpython/rev/d7859e7e7071 changeset: 98133:d7859e7e7071 branch: 3.4 parent: 98128:f13a5b5a2824 user: Victor Stinner date: Mon Sep 21 18:06:17 2015 +0200 summary: Issue #25114, asyncio: add ssl_object extra info to SSL transports This info is required on Python 3.5 and newer to get specific information on the SSL object, like getting the binary peer certificate (instead of getting it as text). files: Doc/library/asyncio-protocol.rst | 5 + Lib/asyncio/selector_events.py | 1 + Lib/asyncio/sslproto.py | 4 + Lib/test/test_asyncio/test_events.py | 75 +++++++++++++-- 4 files changed, 73 insertions(+), 12 deletions(-) diff --git a/Doc/library/asyncio-protocol.rst b/Doc/library/asyncio-protocol.rst --- a/Doc/library/asyncio-protocol.rst +++ b/Doc/library/asyncio-protocol.rst @@ -71,6 +71,8 @@ - ``'peercert'``: peer certificate; result of :meth:`ssl.SSLSocket.getpeercert` - ``'sslcontext'``: :class:`ssl.SSLContext` instance + - ``'ssl_object'``: :class:`ssl.SSLObject` or :class:`ssl.SSLSocket` + instance * pipe: @@ -80,6 +82,9 @@ - ``'subprocess'``: :class:`subprocess.Popen` instance + .. versionchanged:: 3.4.4 + ``'ssl_object'`` info was added to SSL sockets. + ReadTransport ------------- diff --git a/Lib/asyncio/selector_events.py b/Lib/asyncio/selector_events.py --- a/Lib/asyncio/selector_events.py +++ b/Lib/asyncio/selector_events.py @@ -843,6 +843,7 @@ self._extra.update(peercert=peercert, cipher=self._sock.cipher(), compression=self._sock.compression(), + ssl_object=self._sock, ) self._read_wants_write = False diff --git a/Lib/asyncio/sslproto.py b/Lib/asyncio/sslproto.py --- a/Lib/asyncio/sslproto.py +++ b/Lib/asyncio/sslproto.py @@ -295,6 +295,7 @@ def __init__(self, loop, ssl_protocol, app_protocol): self._loop = loop + # SSLProtocol instance self._ssl_protocol = ssl_protocol self._app_protocol = app_protocol self._closed = False @@ -425,10 +426,12 @@ self._app_protocol = app_protocol self._app_transport = _SSLProtocolTransport(self._loop, self, self._app_protocol) + # _SSLPipe instance (None until the connection is made) self._sslpipe = None self._session_established = False self._in_handshake = False self._in_shutdown = False + # transport, ex: SelectorSocketTransport self._transport = None def _wakeup_waiter(self, exc=None): @@ -591,6 +594,7 @@ self._extra.update(peercert=peercert, cipher=sslobj.cipher(), compression=sslobj.compression(), + ssl_object=sslobj, ) self._app_protocol.connection_made(self._app_transport) self._wakeup_waiter() diff --git a/Lib/test/test_asyncio/test_events.py b/Lib/test/test_asyncio/test_events.py --- a/Lib/test/test_asyncio/test_events.py +++ b/Lib/test/test_asyncio/test_events.py @@ -57,6 +57,17 @@ ONLYKEY = data_file('ssl_key.pem') SIGNED_CERTFILE = data_file('keycert3.pem') SIGNING_CA = data_file('pycacert.pem') +PEERCERT = {'serialNumber': 'B09264B1F2DA21D1', + 'version': 1, + 'subject': ((('countryName', 'XY'),), + (('localityName', 'Castle Anthrax'),), + (('organizationName', 'Python Software Foundation'),), + (('commonName', 'localhost'),)), + 'issuer': ((('countryName', 'XY'),), + (('organizationName', 'Python Software Foundation CA'),), + (('commonName', 'our-ca-server'),)), + 'notAfter': 'Nov 13 19:47:07 2022 GMT', + 'notBefore': 'Jan 4 19:47:07 2013 GMT'} class MyBaseProto(asyncio.Protocol): @@ -596,22 +607,56 @@ self.assertGreater(pr.nbytes, 0) tr.close() + def check_ssl_extra_info(self, client, check_sockname=True, + peername=None, peercert={}): + if check_sockname: + self.assertIsNotNone(client.get_extra_info('sockname')) + if peername: + self.assertEqual(peername, + client.get_extra_info('peername')) + else: + self.assertIsNotNone(client.get_extra_info('peername')) + self.assertEqual(peercert, + client.get_extra_info('peercert')) + + # Python disables compression to prevent CRIME attacks by default + self.assertIsNone(client.get_extra_info('compression')) + + # test SSL cipher + cipher = client.get_extra_info('cipher') + self.assertIsInstance(cipher, tuple) + self.assertEqual(len(cipher), 3, cipher) + self.assertIsInstance(cipher[0], str) + self.assertIsInstance(cipher[1], str) + self.assertIsInstance(cipher[2], int) + + # test SSL object + sslobj = client.get_extra_info('ssl_object') + self.assertIsNotNone(sslobj) + self.assertEqual(sslobj.compression(), + client.get_extra_info('compression')) + self.assertEqual(sslobj.cipher(), + client.get_extra_info('cipher')) + self.assertEqual(sslobj.getpeercert(), + client.get_extra_info('peercert')) + def _basetest_create_ssl_connection(self, connection_fut, - check_sockname=True): + check_sockname=True, + peername=None): tr, pr = self.loop.run_until_complete(connection_fut) self.assertIsInstance(tr, asyncio.Transport) self.assertIsInstance(pr, asyncio.Protocol) self.assertTrue('ssl' in tr.__class__.__name__.lower()) - if check_sockname: - self.assertIsNotNone(tr.get_extra_info('sockname')) + self.check_ssl_extra_info(tr, check_sockname, peername) self.loop.run_until_complete(pr.done) self.assertGreater(pr.nbytes, 0) tr.close() def _test_create_ssl_connection(self, httpd, create_connection, - check_sockname=True): + check_sockname=True, peername=None): conn_fut = create_connection(ssl=test_utils.dummy_ssl_context()) - self._basetest_create_ssl_connection(conn_fut, check_sockname) + self._basetest_create_ssl_connection(conn_fut, check_sockname, + peername) # ssl.Purpose was introduced in Python 3.4 if hasattr(ssl, 'Purpose'): @@ -629,7 +674,8 @@ with mock.patch('ssl.create_default_context', side_effect=_dummy_ssl_create_context) as m: conn_fut = create_connection(ssl=True) - self._basetest_create_ssl_connection(conn_fut, check_sockname) + self._basetest_create_ssl_connection(conn_fut, check_sockname, + peername) self.assertEqual(m.call_count, 1) # With the real ssl.create_default_context(), certificate @@ -638,7 +684,8 @@ conn_fut = create_connection(ssl=True) # Ignore the "SSL handshake failed" log in debug mode with test_utils.disable_logger(): - self._basetest_create_ssl_connection(conn_fut, check_sockname) + self._basetest_create_ssl_connection(conn_fut, check_sockname, + peername) self.assertEqual(cm.exception.reason, 'CERTIFICATE_VERIFY_FAILED') @@ -649,7 +696,8 @@ self.loop.create_connection, lambda: MyProto(loop=self.loop), *httpd.address) - self._test_create_ssl_connection(httpd, create_connection) + self._test_create_ssl_connection(httpd, create_connection, + peername=httpd.address) def test_legacy_create_ssl_connection(self): with test_utils.force_legacy_ssl_support(): @@ -669,7 +717,8 @@ server_hostname='127.0.0.1') self._test_create_ssl_connection(httpd, create_connection, - check_sockname) + check_sockname, + peername=httpd.address) def test_legacy_create_ssl_unix_connection(self): with test_utils.force_legacy_ssl_support(): @@ -819,9 +868,7 @@ self.assertEqual(3, proto.nbytes) # extra info is available - self.assertIsNotNone(proto.transport.get_extra_info('sockname')) - self.assertEqual('127.0.0.1', - proto.transport.get_extra_info('peername')[0]) + self.check_ssl_extra_info(client, peername=(host, port)) # close connection proto.transport.close() @@ -1023,6 +1070,10 @@ server_hostname='localhost') client, pr = self.loop.run_until_complete(f_c) + # extra info is available + self.check_ssl_extra_info(client,peername=(host, port), + peercert=PEERCERT) + # close connection proto.transport.close() client.close() -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 21 18:28:11 2015 From: python-checkins at python.org (victor.stinner) Date: Mon, 21 Sep 2015 16:28:11 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge_3=2E5_=28asyncio_doc=29?= Message-ID: <20150921162810.9939.46563@psf.io> https://hg.python.org/cpython/rev/e7f2a26f7087 changeset: 98137:e7f2a26f7087 parent: 98135:db13588ec523 parent: 98136:2f451651038c user: Victor Stinner date: Mon Sep 21 18:28:04 2015 +0200 summary: Merge 3.5 (asyncio doc) files: Doc/library/asyncio-protocol.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/library/asyncio-protocol.rst b/Doc/library/asyncio-protocol.rst --- a/Doc/library/asyncio-protocol.rst +++ b/Doc/library/asyncio-protocol.rst @@ -82,7 +82,7 @@ - ``'subprocess'``: :class:`subprocess.Popen` instance - .. versionchanged:: 3.4.4 + .. versionchanged:: 3.5.1 ``'ssl_object'`` info was added to SSL sockets. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 21 18:28:19 2015 From: python-checkins at python.org (victor.stinner) Date: Mon, 21 Sep 2015 16:28:19 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI1MTE0?= =?utf-8?q?=3A_Adjust_versionchanged_in_the_doc?= Message-ID: <20150921162810.81647.9184@psf.io> https://hg.python.org/cpython/rev/2f451651038c changeset: 98136:2f451651038c branch: 3.5 parent: 98134:4115bc40b5ef user: Victor Stinner date: Mon Sep 21 18:27:52 2015 +0200 summary: Issue #25114: Adjust versionchanged in the doc files: Doc/library/asyncio-protocol.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/library/asyncio-protocol.rst b/Doc/library/asyncio-protocol.rst --- a/Doc/library/asyncio-protocol.rst +++ b/Doc/library/asyncio-protocol.rst @@ -82,7 +82,7 @@ - ``'subprocess'``: :class:`subprocess.Popen` instance - .. versionchanged:: 3.4.4 + .. versionchanged:: 3.5.1 ``'ssl_object'`` info was added to SSL sockets. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 21 18:42:35 2015 From: python-checkins at python.org (victor.stinner) Date: Mon, 21 Sep 2015 16:42:35 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzIzNjMw?= =?utf-8?q?=2C_asyncio=3A_host_parameter_of_loop=2Ecreate=5Fserver=28=29_c?= =?utf-8?q?an_now_be_a?= Message-ID: <20150921164235.11704.79804@psf.io> https://hg.python.org/cpython/rev/682623e3e793 changeset: 98138:682623e3e793 branch: 3.4 parent: 98133:d7859e7e7071 user: Victor Stinner date: Mon Sep 21 18:33:43 2015 +0200 summary: Issue #23630, asyncio: host parameter of loop.create_server() can now be a sequence of strings. Patch written by Yann Sionneau. files: Doc/library/asyncio-eventloop.rst | 13 ++++- Lib/asyncio/base_events.py | 35 ++++++++++++--- Lib/asyncio/events.py | 3 +- Lib/test/test_asyncio/test_events.py | 33 +++++++++++++++ Misc/ACKS | 1 + 5 files changed, 74 insertions(+), 11 deletions(-) diff --git a/Doc/library/asyncio-eventloop.rst b/Doc/library/asyncio-eventloop.rst --- a/Doc/library/asyncio-eventloop.rst +++ b/Doc/library/asyncio-eventloop.rst @@ -331,9 +331,12 @@ Parameters: - * If *host* is an empty string or ``None``, all interfaces are assumed - and a list of multiple sockets will be returned (most likely - one for IPv4 and another one for IPv6). + * The *host* parameter can be a string, in that case the TCP server is + bound to *host* and *port*. The *host* parameter can also be a sequence + of strings and in that case the TCP server is bound to all hosts of the + sequence. If *host* is an empty string or ``None``, all interfaces are + assumed and a list of multiple sockets will be returned (most likely one + for IPv4 and another one for IPv6). * *family* can be set to either :data:`socket.AF_INET` or :data:`~socket.AF_INET6` to force the socket to use IPv4 or IPv6. If not set @@ -365,6 +368,10 @@ The function :func:`start_server` creates a (:class:`StreamReader`, :class:`StreamWriter`) pair and calls back a function with this pair. + .. versionchanged:: 3.4.4 + + The *host* parameter can now be a sequence of strings. + .. coroutinemethod:: BaseEventLoop.create_unix_server(protocol_factory, path=None, \*, sock=None, backlog=100, ssl=None) diff --git a/Lib/asyncio/base_events.py b/Lib/asyncio/base_events.py --- a/Lib/asyncio/base_events.py +++ b/Lib/asyncio/base_events.py @@ -18,6 +18,7 @@ import concurrent.futures import heapq import inspect +import itertools import logging import os import socket @@ -787,6 +788,15 @@ return transport, protocol @coroutine + def _create_server_getaddrinfo(self, host, port, family, flags): + infos = yield from self.getaddrinfo(host, port, family=family, + type=socket.SOCK_STREAM, + flags=flags) + if not infos: + raise OSError('getaddrinfo({!r}) returned empty list'.format(host)) + return infos + + @coroutine def create_server(self, protocol_factory, host=None, port=None, *, family=socket.AF_UNSPEC, @@ -795,7 +805,13 @@ backlog=100, ssl=None, reuse_address=None): - """Create a TCP server bound to host and port. + """Create a TCP server. + + The host parameter can be a string, in that case the TCP server is bound + to host and port. + + The host parameter can also be a sequence of strings and in that case + the TCP server is bound to all hosts of the sequence. Return a Server object which can be used to stop the service. @@ -813,13 +829,18 @@ reuse_address = os.name == 'posix' and sys.platform != 'cygwin' sockets = [] if host == '': - host = None + hosts = [None] + elif (isinstance(host, str) or + not isinstance(host, collections.Iterable)): + hosts = [host] + else: + hosts = host - infos = yield from self.getaddrinfo( - host, port, family=family, - type=socket.SOCK_STREAM, proto=0, flags=flags) - if not infos: - raise OSError('getaddrinfo() returned empty list') + fs = [self._create_server_getaddrinfo(host, port, family=family, + flags=flags) + for host in hosts] + infos = yield from tasks.gather(*fs, loop=self) + infos = itertools.chain.from_iterable(infos) completed = False try: diff --git a/Lib/asyncio/events.py b/Lib/asyncio/events.py --- a/Lib/asyncio/events.py +++ b/Lib/asyncio/events.py @@ -305,7 +305,8 @@ If host is an empty string or None all interfaces are assumed and a list of multiple sockets will be returned (most likely - one for IPv4 and another one for IPv6). + one for IPv4 and another one for IPv6). The host parameter can also be a + sequence (e.g. list) of hosts to bind to. family can be set to either AF_INET or AF_INET6 to force the socket to use IPv4 or IPv6. If not set it will be determined diff --git a/Lib/test/test_asyncio/test_events.py b/Lib/test/test_asyncio/test_events.py --- a/Lib/test/test_asyncio/test_events.py +++ b/Lib/test/test_asyncio/test_events.py @@ -745,6 +745,39 @@ self.assertEqual(cm.exception.errno, errno.EADDRINUSE) self.assertIn(str(httpd.address), cm.exception.strerror) + @mock.patch('asyncio.base_events.socket') + def create_server_multiple_hosts(self, family, hosts, mock_sock): + @asyncio.coroutine + def getaddrinfo(host, port, *args, **kw): + if family == socket.AF_INET: + return [[family, socket.SOCK_STREAM, 6, '', (host, port)]] + else: + return [[family, socket.SOCK_STREAM, 6, '', (host, port, 0, 0)]] + + def getaddrinfo_task(*args, **kwds): + return asyncio.Task(getaddrinfo(*args, **kwds), loop=self.loop) + + if family == socket.AF_INET: + mock_sock.socket().getsockbyname.side_effect = [(host, 80) + for host in hosts] + else: + mock_sock.socket().getsockbyname.side_effect = [(host, 80, 0, 0) + for host in hosts] + self.loop.getaddrinfo = getaddrinfo_task + self.loop._start_serving = mock.Mock() + f = self.loop.create_server(lambda: MyProto(self.loop), hosts, 80) + server = self.loop.run_until_complete(f) + self.addCleanup(server.close) + server_hosts = [sock.getsockbyname()[0] for sock in server.sockets] + self.assertEqual(server_hosts, hosts) + + def test_create_server_multiple_hosts_ipv4(self): + self.create_server_multiple_hosts(socket.AF_INET, + ['1.2.3.4', '5.6.7.8']) + + def test_create_server_multiple_hosts_ipv6(self): + self.create_server_multiple_hosts(socket.AF_INET6, ['::1', '::2']) + def test_create_server(self): proto = MyProto(self.loop) f = self.loop.create_server(lambda: proto, '0.0.0.0', 0) diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -1294,6 +1294,7 @@ Ravi Sinha Janne Sinkkonen Ng Pheng Siong +Yann Sionneau George Sipe J. Sipprell Kragen Sitaker -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 21 18:42:35 2015 From: python-checkins at python.org (victor.stinner) Date: Mon, 21 Sep 2015 16:42:35 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Merge_3=2E4_=28asyncio=29?= Message-ID: <20150921164235.115507.90037@psf.io> https://hg.python.org/cpython/rev/1253225519da changeset: 98139:1253225519da branch: 3.5 parent: 98136:2f451651038c parent: 98138:682623e3e793 user: Victor Stinner date: Mon Sep 21 18:41:05 2015 +0200 summary: Merge 3.4 (asyncio) files: Doc/library/asyncio-eventloop.rst | 13 ++++- Lib/asyncio/base_events.py | 35 ++++++++++++--- Lib/asyncio/events.py | 3 +- Lib/test/test_asyncio/test_events.py | 33 +++++++++++++++ Misc/ACKS | 1 + 5 files changed, 74 insertions(+), 11 deletions(-) diff --git a/Doc/library/asyncio-eventloop.rst b/Doc/library/asyncio-eventloop.rst --- a/Doc/library/asyncio-eventloop.rst +++ b/Doc/library/asyncio-eventloop.rst @@ -333,9 +333,12 @@ Parameters: - * If *host* is an empty string or ``None``, all interfaces are assumed - and a list of multiple sockets will be returned (most likely - one for IPv4 and another one for IPv6). + * The *host* parameter can be a string, in that case the TCP server is + bound to *host* and *port*. The *host* parameter can also be a sequence + of strings and in that case the TCP server is bound to all hosts of the + sequence. If *host* is an empty string or ``None``, all interfaces are + assumed and a list of multiple sockets will be returned (most likely one + for IPv4 and another one for IPv6). * *family* can be set to either :data:`socket.AF_INET` or :data:`~socket.AF_INET6` to force the socket to use IPv4 or IPv6. If not set @@ -369,6 +372,10 @@ The function :func:`start_server` creates a (:class:`StreamReader`, :class:`StreamWriter`) pair and calls back a function with this pair. + .. versionchanged:: 3.5.1 + + The *host* parameter can now be a sequence of strings. + .. coroutinemethod:: BaseEventLoop.create_unix_server(protocol_factory, path=None, \*, sock=None, backlog=100, ssl=None) diff --git a/Lib/asyncio/base_events.py b/Lib/asyncio/base_events.py --- a/Lib/asyncio/base_events.py +++ b/Lib/asyncio/base_events.py @@ -18,6 +18,7 @@ import concurrent.futures import heapq import inspect +import itertools import logging import os import socket @@ -787,6 +788,15 @@ return transport, protocol @coroutine + def _create_server_getaddrinfo(self, host, port, family, flags): + infos = yield from self.getaddrinfo(host, port, family=family, + type=socket.SOCK_STREAM, + flags=flags) + if not infos: + raise OSError('getaddrinfo({!r}) returned empty list'.format(host)) + return infos + + @coroutine def create_server(self, protocol_factory, host=None, port=None, *, family=socket.AF_UNSPEC, @@ -795,7 +805,13 @@ backlog=100, ssl=None, reuse_address=None): - """Create a TCP server bound to host and port. + """Create a TCP server. + + The host parameter can be a string, in that case the TCP server is bound + to host and port. + + The host parameter can also be a sequence of strings and in that case + the TCP server is bound to all hosts of the sequence. Return a Server object which can be used to stop the service. @@ -813,13 +829,18 @@ reuse_address = os.name == 'posix' and sys.platform != 'cygwin' sockets = [] if host == '': - host = None + hosts = [None] + elif (isinstance(host, str) or + not isinstance(host, collections.Iterable)): + hosts = [host] + else: + hosts = host - infos = yield from self.getaddrinfo( - host, port, family=family, - type=socket.SOCK_STREAM, proto=0, flags=flags) - if not infos: - raise OSError('getaddrinfo() returned empty list') + fs = [self._create_server_getaddrinfo(host, port, family=family, + flags=flags) + for host in hosts] + infos = yield from tasks.gather(*fs, loop=self) + infos = itertools.chain.from_iterable(infos) completed = False try: diff --git a/Lib/asyncio/events.py b/Lib/asyncio/events.py --- a/Lib/asyncio/events.py +++ b/Lib/asyncio/events.py @@ -305,7 +305,8 @@ If host is an empty string or None all interfaces are assumed and a list of multiple sockets will be returned (most likely - one for IPv4 and another one for IPv6). + one for IPv4 and another one for IPv6). The host parameter can also be a + sequence (e.g. list) of hosts to bind to. family can be set to either AF_INET or AF_INET6 to force the socket to use IPv4 or IPv6. If not set it will be determined diff --git a/Lib/test/test_asyncio/test_events.py b/Lib/test/test_asyncio/test_events.py --- a/Lib/test/test_asyncio/test_events.py +++ b/Lib/test/test_asyncio/test_events.py @@ -745,6 +745,39 @@ self.assertEqual(cm.exception.errno, errno.EADDRINUSE) self.assertIn(str(httpd.address), cm.exception.strerror) + @mock.patch('asyncio.base_events.socket') + def create_server_multiple_hosts(self, family, hosts, mock_sock): + @asyncio.coroutine + def getaddrinfo(host, port, *args, **kw): + if family == socket.AF_INET: + return [[family, socket.SOCK_STREAM, 6, '', (host, port)]] + else: + return [[family, socket.SOCK_STREAM, 6, '', (host, port, 0, 0)]] + + def getaddrinfo_task(*args, **kwds): + return asyncio.Task(getaddrinfo(*args, **kwds), loop=self.loop) + + if family == socket.AF_INET: + mock_sock.socket().getsockbyname.side_effect = [(host, 80) + for host in hosts] + else: + mock_sock.socket().getsockbyname.side_effect = [(host, 80, 0, 0) + for host in hosts] + self.loop.getaddrinfo = getaddrinfo_task + self.loop._start_serving = mock.Mock() + f = self.loop.create_server(lambda: MyProto(self.loop), hosts, 80) + server = self.loop.run_until_complete(f) + self.addCleanup(server.close) + server_hosts = [sock.getsockbyname()[0] for sock in server.sockets] + self.assertEqual(server_hosts, hosts) + + def test_create_server_multiple_hosts_ipv4(self): + self.create_server_multiple_hosts(socket.AF_INET, + ['1.2.3.4', '5.6.7.8']) + + def test_create_server_multiple_hosts_ipv6(self): + self.create_server_multiple_hosts(socket.AF_INET6, ['::1', '::2']) + def test_create_server(self): proto = MyProto(self.loop) f = self.loop.create_server(lambda: proto, '0.0.0.0', 0) diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -1331,6 +1331,7 @@ Ravi Sinha Janne Sinkkonen Ng Pheng Siong +Yann Sionneau George Sipe J. Sipprell Kragen Sitaker -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 21 18:42:36 2015 From: python-checkins at python.org (victor.stinner) Date: Mon, 21 Sep 2015 16:42:36 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge_3=2E5_=28asyncio=29?= Message-ID: <20150921164235.115327.38712@psf.io> https://hg.python.org/cpython/rev/f87f24923c83 changeset: 98140:f87f24923c83 parent: 98137:e7f2a26f7087 parent: 98139:1253225519da user: Victor Stinner date: Mon Sep 21 18:41:46 2015 +0200 summary: Merge 3.5 (asyncio) files: Doc/library/asyncio-eventloop.rst | 13 ++++- Lib/asyncio/base_events.py | 35 ++++++++++++--- Lib/asyncio/events.py | 3 +- Lib/test/test_asyncio/test_events.py | 33 +++++++++++++++ Misc/ACKS | 1 + 5 files changed, 74 insertions(+), 11 deletions(-) diff --git a/Doc/library/asyncio-eventloop.rst b/Doc/library/asyncio-eventloop.rst --- a/Doc/library/asyncio-eventloop.rst +++ b/Doc/library/asyncio-eventloop.rst @@ -333,9 +333,12 @@ Parameters: - * If *host* is an empty string or ``None``, all interfaces are assumed - and a list of multiple sockets will be returned (most likely - one for IPv4 and another one for IPv6). + * The *host* parameter can be a string, in that case the TCP server is + bound to *host* and *port*. The *host* parameter can also be a sequence + of strings and in that case the TCP server is bound to all hosts of the + sequence. If *host* is an empty string or ``None``, all interfaces are + assumed and a list of multiple sockets will be returned (most likely one + for IPv4 and another one for IPv6). * *family* can be set to either :data:`socket.AF_INET` or :data:`~socket.AF_INET6` to force the socket to use IPv4 or IPv6. If not set @@ -369,6 +372,10 @@ The function :func:`start_server` creates a (:class:`StreamReader`, :class:`StreamWriter`) pair and calls back a function with this pair. + .. versionchanged:: 3.5.1 + + The *host* parameter can now be a sequence of strings. + .. coroutinemethod:: BaseEventLoop.create_unix_server(protocol_factory, path=None, \*, sock=None, backlog=100, ssl=None) diff --git a/Lib/asyncio/base_events.py b/Lib/asyncio/base_events.py --- a/Lib/asyncio/base_events.py +++ b/Lib/asyncio/base_events.py @@ -18,6 +18,7 @@ import concurrent.futures import heapq import inspect +import itertools import logging import os import socket @@ -787,6 +788,15 @@ return transport, protocol @coroutine + def _create_server_getaddrinfo(self, host, port, family, flags): + infos = yield from self.getaddrinfo(host, port, family=family, + type=socket.SOCK_STREAM, + flags=flags) + if not infos: + raise OSError('getaddrinfo({!r}) returned empty list'.format(host)) + return infos + + @coroutine def create_server(self, protocol_factory, host=None, port=None, *, family=socket.AF_UNSPEC, @@ -795,7 +805,13 @@ backlog=100, ssl=None, reuse_address=None): - """Create a TCP server bound to host and port. + """Create a TCP server. + + The host parameter can be a string, in that case the TCP server is bound + to host and port. + + The host parameter can also be a sequence of strings and in that case + the TCP server is bound to all hosts of the sequence. Return a Server object which can be used to stop the service. @@ -813,13 +829,18 @@ reuse_address = os.name == 'posix' and sys.platform != 'cygwin' sockets = [] if host == '': - host = None + hosts = [None] + elif (isinstance(host, str) or + not isinstance(host, collections.Iterable)): + hosts = [host] + else: + hosts = host - infos = yield from self.getaddrinfo( - host, port, family=family, - type=socket.SOCK_STREAM, proto=0, flags=flags) - if not infos: - raise OSError('getaddrinfo() returned empty list') + fs = [self._create_server_getaddrinfo(host, port, family=family, + flags=flags) + for host in hosts] + infos = yield from tasks.gather(*fs, loop=self) + infos = itertools.chain.from_iterable(infos) completed = False try: diff --git a/Lib/asyncio/events.py b/Lib/asyncio/events.py --- a/Lib/asyncio/events.py +++ b/Lib/asyncio/events.py @@ -305,7 +305,8 @@ If host is an empty string or None all interfaces are assumed and a list of multiple sockets will be returned (most likely - one for IPv4 and another one for IPv6). + one for IPv4 and another one for IPv6). The host parameter can also be a + sequence (e.g. list) of hosts to bind to. family can be set to either AF_INET or AF_INET6 to force the socket to use IPv4 or IPv6. If not set it will be determined diff --git a/Lib/test/test_asyncio/test_events.py b/Lib/test/test_asyncio/test_events.py --- a/Lib/test/test_asyncio/test_events.py +++ b/Lib/test/test_asyncio/test_events.py @@ -745,6 +745,39 @@ self.assertEqual(cm.exception.errno, errno.EADDRINUSE) self.assertIn(str(httpd.address), cm.exception.strerror) + @mock.patch('asyncio.base_events.socket') + def create_server_multiple_hosts(self, family, hosts, mock_sock): + @asyncio.coroutine + def getaddrinfo(host, port, *args, **kw): + if family == socket.AF_INET: + return [[family, socket.SOCK_STREAM, 6, '', (host, port)]] + else: + return [[family, socket.SOCK_STREAM, 6, '', (host, port, 0, 0)]] + + def getaddrinfo_task(*args, **kwds): + return asyncio.Task(getaddrinfo(*args, **kwds), loop=self.loop) + + if family == socket.AF_INET: + mock_sock.socket().getsockbyname.side_effect = [(host, 80) + for host in hosts] + else: + mock_sock.socket().getsockbyname.side_effect = [(host, 80, 0, 0) + for host in hosts] + self.loop.getaddrinfo = getaddrinfo_task + self.loop._start_serving = mock.Mock() + f = self.loop.create_server(lambda: MyProto(self.loop), hosts, 80) + server = self.loop.run_until_complete(f) + self.addCleanup(server.close) + server_hosts = [sock.getsockbyname()[0] for sock in server.sockets] + self.assertEqual(server_hosts, hosts) + + def test_create_server_multiple_hosts_ipv4(self): + self.create_server_multiple_hosts(socket.AF_INET, + ['1.2.3.4', '5.6.7.8']) + + def test_create_server_multiple_hosts_ipv6(self): + self.create_server_multiple_hosts(socket.AF_INET6, ['::1', '::2']) + def test_create_server(self): proto = MyProto(self.loop) f = self.loop.create_server(lambda: proto, '0.0.0.0', 0) diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -1332,6 +1332,7 @@ Ravi Sinha Janne Sinkkonen Ng Pheng Siong +Yann Sionneau George Sipe J. Sipprell Kragen Sitaker -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 21 19:00:29 2015 From: python-checkins at python.org (alexander.belopolsky) Date: Mon, 21 Sep 2015 17:00:29 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_PEP_495=3A_Applied_a_patch_fr?= =?utf-8?q?om_Carl_Meyer=2E?= Message-ID: <20150921170019.98368.26587@psf.io> https://hg.python.org/peps/rev/d61277cfa2c6 changeset: 6087:d61277cfa2c6 user: Alexander Belopolsky date: Mon Sep 21 13:00:13 2015 -0400 summary: PEP 495: Applied a patch from Carl Meyer. files: pep-0495.txt | 72 +++++++++++++++++++++------------------ 1 files changed, 39 insertions(+), 33 deletions(-) diff --git a/pep-0495.txt b/pep-0495.txt --- a/pep-0495.txt +++ b/pep-0495.txt @@ -14,7 +14,7 @@ Abstract ======== -This PEP adds a new attribute ``fold`` to the instances of +This PEP adds a new attribute ``fold`` to instances of the ``datetime.time`` and ``datetime.datetime`` classes that can be used to differentiate between two moments in time for which local times are the same. The allowed values for the `fold` attribute will be 0 and 1 @@ -25,7 +25,7 @@ Rationale ========= -In the most world locations there have been and will be times when +In most world locations, there have been and will be times when local clocks are moved back. [#]_ In those times, intervals are introduced in which local clocks show the same time twice in the same day. In these situations, the information displayed on a local clock @@ -75,11 +75,11 @@ The "fold" attribute -------------------- -We propose adding an attribute called ``fold`` to the instances -of ``datetime.time`` and ``datetime.datetime`` classes. This attribute -should have the value 0 for all instances except those that -represent the second (chronologically) moment in time in an ambiguous -case. For those instances, the value will be 1. [#]_ +We propose adding an attribute called ``fold`` to instances of the +``datetime.time`` and ``datetime.datetime`` classes. This attribute +should have the value 0 for all instances except those that represent +the second (chronologically) moment in time in an ambiguous case. For +those instances, the value will be 1. [#]_ .. [#] An instance that has ``fold=1`` in a non-ambiguous case is said to represent an invalid time (or is invalid for short), but @@ -116,15 +116,18 @@ The ``replace()`` methods of the ``datetime.time`` and ``datetime.datetime`` classes will get a new keyword-only argument -called ``fold``. It will -behave similarly to the other ``replace()`` arguments: if the ``fold`` -argument is specified and given a value 0 or 1, the new instance -returned by ``replace()`` will have its ``fold`` attribute set -to that value. In CPython, any non-integer value of ``fold`` will -raise a ``TypeError``, but other implementations may allow the value -``None`` to behave the same as when ``fold`` is not given. If the -``fold`` argument is not specified, the original value of the ``fold`` -attribute is copied to the result. +called ``fold``. It will behave similarly to the other ``replace()`` +arguments: if the ``fold`` argument is specified and given a value 0 +or 1, the new instance returned by ``replace()`` will have its +``fold`` attribute set to that value. In CPython, any non-integer +value of ``fold`` will raise a ``TypeError``, but other +implementations may allow the value ``None`` to behave the same as +when ``fold`` is not given. [#]_ If the ``fold`` argument is not +specified, the original value of the ``fold`` attribute is copied to +the result. + +.. [#] PyPy already allows ``None`` to mean "no change to existing + attribute" for all other attributes in ``replace()``. C-API ..... @@ -132,14 +135,14 @@ Access macros will be defined to extract the value of ``fold`` from ``PyDateTime_DateTime`` and ``PyDateTime_Time`` objects. -.. code:: +.. code:: int PyDateTime_GET_FOLD(PyDateTime_DateTime *o) Return the value of ``fold`` as a C ``int``. -.. code:: - +.. code:: + int PyDateTime_TIME_GET_FOLD(PyDateTime_Time *o) Return the value of ``fold`` as a C ``int``. @@ -150,14 +153,17 @@ .. code:: - PyObject* PyDateTime_FromDateAndTimeAndFold(int year, int month, int day, int hour, int minute, int second, int usecond, int fold) + PyObject* PyDateTime_FromDateAndTimeAndFold( + int year, int month, int day, int hour, int minute, + int second, int usecond, int fold) Return a ``datetime.datetime`` object with the specified year, month, day, hour, minute, second, microsecond and fold. .. code:: - PyObject* PyTime_FromTimeAndFold(int hour, int minute, int second, int usecond, int fold) + PyObject* PyTime_FromTimeAndFold( + int hour, int minute, int second, int usecond, int fold) Return a ``datetime.time`` object with the specified hour, minute, second, microsecond and fold. @@ -267,9 +273,9 @@ not be distinguishable by any means other than an explicit access to the ``fold`` value. -On the other hand, if object's ``tzinfo`` is set to a fold-aware +On the other hand, if an object's ``tzinfo`` is set to a fold-aware implementation, then the value of ``fold`` will affect the result of -several methods but only if the corresponding time is in a fold or in +several methods, but only if the corresponding time is in a fold or in a gap: ``utcoffset()``, ``dst()``, ``tzname()``, ``astimezone()``, ``strftime()`` (if "%Z" or "%z" directive is used in the format specification), ``isoformat()``, and ``timetuple()``. @@ -311,12 +317,12 @@ proposed in this PEP. The existing (fixed offset) timezones do not introduce ambiguous local times and their ``utcoffset()`` implementation will return the same constant value as they do now -regardless of the value of ``fold``. +regardless of the value of ``fold``. The basic implementation of ``fromutc()`` in the abstract -``datetime.tzinfo`` class will not change. It is currently not -used anywhere in the stdlib because the only included ``tzinfo`` -implementation (the ``datetime.timzeone`` class implementing fixed +``datetime.tzinfo`` class will not change. It is currently not used +anywhere in the stdlib because the only included ``tzinfo`` +implementation (the ``datetime.timezone`` class implementing fixed offset timezones) override ``fromutc()``. @@ -590,7 +596,7 @@ between fold=0 and fold=1 when I set it for tomorrow 01:30 AM. What should I do? * Alice: I've never hear of a Py-O-Clock, but I guess fold=0 is - the first 01:30 AM and fold=1 is the second. + the first 01:30 AM and fold=1 is the second. A technical reason @@ -661,7 +667,7 @@ **repeated** Did not receive any support on the mailing list. - + **ltdf** (Local Time Disambiguation Flag) - short and no-one will attempt to guess what it means without reading the docs. (Feel free to @@ -761,13 +767,13 @@ Note that 12:00 was interpreted as 13:00 by ``mktime``. With the ``datetime.timestamp``, ``datetime.fromtimestamp``, it is currently -guaranteed that +guaranteed that .. code:: >>> t = datetime.datetime(2015, 6, 1, 12).timestamp() >>> datetime.datetime.fromtimestamp(t) - datetime.datetime(2015, 6, 1, 12, 0) + datetime.datetime(2015, 6, 1, 12, 0) This PEP extends the same guarantee to both values of ``fold``: @@ -775,13 +781,13 @@ >>> t = datetime.datetime(2015, 6, 1, 12, fold=0).timestamp() >>> datetime.datetime.fromtimestamp(t) - datetime.datetime(2015, 6, 1, 12, 0) + datetime.datetime(2015, 6, 1, 12, 0) .. code:: >>> t = datetime.datetime(2015, 6, 1, 12, fold=1).timestamp() >>> datetime.datetime.fromtimestamp(t) - datetime.datetime(2015, 6, 1, 12, 0) + datetime.datetime(2015, 6, 1, 12, 0) Thus one of the suggested uses for ``fold=-1`` -- to match the legacy behavior -- is not needed. Either choice of ``fold`` will match the -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Mon Sep 21 19:02:32 2015 From: python-checkins at python.org (alexander.belopolsky) Date: Mon, 21 Sep 2015 17:02:32 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_PEP_495=3A_Mention_pure_Pytho?= =?utf-8?q?n_datetime=2Epy_in_the_=2Ereplace=28fold=3DNone=29_rationale=2E?= Message-ID: <20150921170231.115214.97261@psf.io> https://hg.python.org/peps/rev/ed5764395275 changeset: 6088:ed5764395275 user: Alexander Belopolsky date: Mon Sep 21 13:02:27 2015 -0400 summary: PEP 495: Mention pure Python datetime.py in the .replace(fold=None) rationale. files: pep-0495.txt | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-) diff --git a/pep-0495.txt b/pep-0495.txt --- a/pep-0495.txt +++ b/pep-0495.txt @@ -126,7 +126,8 @@ specified, the original value of the ``fold`` attribute is copied to the result. -.. [#] PyPy already allows ``None`` to mean "no change to existing +.. [#] PyPy and pure Python implementation distributed with CPython + already allow ``None`` to mean "no change to existing attribute" for all other attributes in ``replace()``. C-API -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Mon Sep 21 19:33:27 2015 From: python-checkins at python.org (alexander.belopolsky) Date: Mon, 21 Sep 2015 17:33:27 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_PEP_495=3A_Corrected_a_typo?= =?utf-8?q?=2E__Thanks_Carl_Meyer=2E?= Message-ID: <20150921173324.115250.31508@psf.io> https://hg.python.org/peps/rev/1cc04cae951f changeset: 6089:1cc04cae951f user: Alexander Belopolsky date: Mon Sep 21 13:33:12 2015 -0400 summary: PEP 495: Corrected a typo. Thanks Carl Meyer. files: pep-0495.txt | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/pep-0495.txt b/pep-0495.txt --- a/pep-0495.txt +++ b/pep-0495.txt @@ -230,7 +230,7 @@ datetime.fromtimestamp(s0) == datetime.fromtimestamp(s1) == dt In this case, ``dt.timestamp()`` will return the smaller of ``s0`` -and ``s1`` values if ``dt.fold == True`` and the larger otherwise. +and ``s1`` values if ``dt.fold == 0`` and the larger otherwise. For example, on a system set to US/Eastern timezone:: -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Mon Sep 21 19:36:18 2015 From: python-checkins at python.org (eric.smith) Date: Mon, 21 Sep 2015 17:36:18 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2324779=3A_Remove_u?= =?utf-8?q?nused_rawmode_parameter_to_unicode=5Fdecode=2E?= Message-ID: <20150921173615.82662.941@psf.io> https://hg.python.org/cpython/rev/5230115de64d changeset: 98141:5230115de64d user: Eric V. Smith date: Mon Sep 21 13:36:09 2015 -0400 summary: Issue #24779: Remove unused rawmode parameter to unicode_decode. files: Python/ast.c | 9 +++------ 1 files changed, 3 insertions(+), 6 deletions(-) diff --git a/Python/ast.c b/Python/ast.c --- a/Python/ast.c +++ b/Python/ast.c @@ -3938,7 +3938,7 @@ } static PyObject * -decode_unicode(struct compiling *c, const char *s, size_t len, int rawmode, const char *encoding) +decode_unicode(struct compiling *c, const char *s, size_t len, const char *encoding) { PyObject *v, *u; char *buf; @@ -3994,10 +3994,7 @@ len = p - buf; s = buf; } - if (rawmode) - v = PyUnicode_DecodeRawUnicodeEscape(s, len, NULL); - else - v = PyUnicode_DecodeUnicodeEscape(s, len, NULL); + v = PyUnicode_DecodeUnicodeEscape(s, len, NULL); Py_XDECREF(u); return v; } @@ -4896,7 +4893,7 @@ } } if (!*bytesmode && !rawmode) { - return decode_unicode(c, s, len, rawmode, c->c_encoding); + return decode_unicode(c, s, len, c->c_encoding); } if (*bytesmode) { /* Disallow non-ascii characters (but not escapes) */ -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 21 22:21:58 2015 From: python-checkins at python.org (guido.van.rossum) Date: Mon, 21 Sep 2015 20:21:58 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_A_variety_of_small_edits_and_?= =?utf-8?q?clarifications_to_PEP_495=2E?= Message-ID: <20150921202158.3650.18287@psf.io> https://hg.python.org/peps/rev/c9a0cdfd8187 changeset: 6090:c9a0cdfd8187 user: Guido van Rossum date: Mon Sep 21 13:21:26 2015 -0700 summary: A variety of small edits and clarifications to PEP 495. files: pep-0495.txt | 74 ++++++++++++++++++++++++++------------- 1 files changed, 49 insertions(+), 25 deletions(-) diff --git a/pep-0495.txt b/pep-0495.txt --- a/pep-0495.txt +++ b/pep-0495.txt @@ -41,7 +41,7 @@ .. [#] People who live in locations observing the Daylight Saving Time (DST) move their clocks back (usually one hour) every Fall. - + It is less common, but occasionally clocks can be moved back for other reasons. For example, Ukraine skipped the spring-forward transition in March 1990 and instead, moved their clocks back on @@ -122,7 +122,11 @@ ``fold`` attribute set to that value. In CPython, any non-integer value of ``fold`` will raise a ``TypeError``, but other implementations may allow the value ``None`` to behave the same as -when ``fold`` is not given. [#]_ If the ``fold`` argument is not +when ``fold`` is not given. [#]_ (This is +a nod to the existing difference in treatment of ``None`` arguments +in other positions of this method across Python implementations; +it is not intended to leave the door open for future alternative +interpretation of ``fold=None``.) If the ``fold`` argument is not specified, the original value of the ``fold`` attribute is copied to the result. @@ -176,18 +180,23 @@ What time is it? ................ -The ``datetime.now()`` method called with no arguments, will set +The ``datetime.now()`` method called without arguments will set ``fold=1`` when returning the second of the two ambiguous times in a system local time fold. When called with a ``tzinfo`` argument, the value of the ``fold`` will be determined by the ``tzinfo.fromutc()`` -implementation. If an instance of the ``datetime.timezone`` class -(*e.g.* ``datetime.timezone.utc``) is passed as ``tzinfo``, the +implementation. When an instance of the ``datetime.timezone`` class +(the stdlib's fixed-offset ``tzinfo`` subclass, +*e.g.* ``datetime.timezone.utc``) is passed as ``tzinfo``, the returned datetime instance will always have ``fold=0``. +The ``datetime.utcnow()`` method is unaffected. Conversion from naive to aware .............................. +A new feature is proposed to facilitate conversion from naive datetime +instances to aware. + The ``astimezone()`` method will now work for naive ``self``. The system local timezone will be assumed in this case and the ``fold`` flag will be used to determine which local timezone is in effect @@ -201,6 +210,11 @@ >>> dt.replace(fold=1).astimezone().strftime('%D %T %Z%z') '11/02/14 01:30:00 EST-0500' +An implication is that ``datetime.now(tz)`` is fully equivalent to +``datetime.now().astimezone(tz)`` (assuming ``tz`` is an instance of a +post-PEP ``tzinfo`` implementation, i.e. one that correctly handles +and sets ``fold``). + Conversion from POSIX seconds from EPOCH ........................................ @@ -229,6 +243,8 @@ datetime.fromtimestamp(s0) == datetime.fromtimestamp(s1) == dt +(This is because ``==`` disregards the value of fold -- see below.) + In this case, ``dt.timestamp()`` will return the smaller of ``s0`` and ``s1`` values if ``dt.fold == 0`` and the larger otherwise. @@ -240,7 +256,6 @@ >>> datetime(2014, 11, 2, 1, 30, fold=1).timestamp() 1414909800.0 - When a ``datetime.datetime`` instance ``dt`` represents a missing time, there is no value ``s`` for which:: @@ -256,6 +271,8 @@ The value returned by ``dt.timestamp()`` given a missing ``dt`` will be the greater of the two "nice to know" values if ``dt.fold == 0`` and the smaller otherwise. +(This is not a typo -- it's intentionally backwards from the rule for +ambiguous times.) For example, on a system set to US/Eastern timezone:: @@ -272,13 +289,14 @@ changes in the behavior of their aware datetime instances. Two such instances that differ only by the value of the ``fold`` attribute will not be distinguishable by any means other than an explicit access to -the ``fold`` value. +the ``fold`` value. (This is because these pre-PEP implementations +are not using the ``fold`` attribute.) On the other hand, if an object's ``tzinfo`` is set to a fold-aware -implementation, then the value of ``fold`` will affect the result of -several methods, but only if the corresponding time is in a fold or in -a gap: ``utcoffset()``, ``dst()``, ``tzname()``, ``astimezone()``, -``strftime()`` (if "%Z" or "%z" directive is used in the format +implementation, then in a fold or gap the value of ``fold`` will +affect the result of several methods: +``utcoffset()``, ``dst()``, ``tzname()``, ``astimezone()``, +``strftime()`` (if the "%Z" or "%z" directive is used in the format specification), ``isoformat()``, and ``timetuple()``. @@ -300,8 +318,9 @@ Pickle sizes for the ``datetime.datetime`` and ``datetime.time`` objects will not change. The ``fold`` value will be encoded in the -first bit of the 3rd (1st) byte of ``datetime.datetime`` -(``datetime.time``) pickle payload. In the `current implementation`_ +first bit of the 3rd byte of the ``datetime.datetime`` +pickle payload; and in the first bit of the 1st byte of the +``datetime.time`` payload. In the `current implementation`_ these bytes are used to store the month (1-12) and hour (0-23) values and the first bit is always 0. We picked these bytes because they are the only bytes that are checked by the current unpickle code. Thus @@ -324,8 +343,11 @@ ``datetime.tzinfo`` class will not change. It is currently not used anywhere in the stdlib because the only included ``tzinfo`` implementation (the ``datetime.timezone`` class implementing fixed -offset timezones) override ``fromutc()``. - +offset timezones) override ``fromutc()``. Keeping the default +implementation unchanged has the benefit that pre-PEP 3rd party +implementations that inherit the default ``fromutc()`` are not +accidentally affected. + Guidelines for New tzinfo Implementations ========================================= @@ -381,7 +403,7 @@ The ``fromutc()`` method should never produce a time in the gap. -If ``utcoffset()``, ``tzname()`` or ``dst()`` method is called on a +If the ``utcoffset()``, ``tzname()`` or ``dst()`` method is called on a local time that falls in a gap, the rules in effect before the transition should be used if ``fold=0``. Otherwise, the rules in effect after the transition should be used. @@ -477,7 +499,7 @@ by the value of ``fold`` will compare as equal. Applications that need to differentiate between such instances should check the value of ``fold`` explicitly or convert those instances to a timezone that does -not have ambiguous times. +not have ambiguous times (such as UTC). The value of ``fold`` will also be ignored whenever a timedelta is added to or subtracted from a datetime instance which may be either @@ -495,7 +517,9 @@ subtraction. Naive and intra-zone comparisons will ignore the value of ``fold`` and -return the same results as they do now. +return the same results as they do now. (This is the only way to +preserve backward compatibility. If you need an aware intra-zone +comparison that uses the fold, convert both sides to UTC first.) The inter-zone subtraction will be defined as it is now: ``t - s`` is computed as ``(t - t.utcoffset()) - (s - @@ -506,14 +530,14 @@ .. [#] Note that the new rules may result in a paradoxical situation when ``s == t`` but ``s - u != t - u``. Such paradoxes are not really new and are inherent in the overloading of the minus - operator as two different intra- and inter-zone operations. For + operator differently for intra- and inter-zone operations. For example, one can easily construct datetime instances ``t`` and ``s`` with some variable offset ``tzinfo`` and a datetime ``u`` with ``tzinfo=timezone.utc`` such that ``(t - u) - (s - u) != t - s``. The explanation for this paradox is that the minuses inside the parentheses and the two other minuses are really three different - operations: inter-zone datetime subtraction, timedelta subtraction - and intra-zone datetime subtraction which have the mathematical + operations: inter-zone datetime subtraction, timedelta subtraction, + and intra-zone datetime subtraction, which each have the mathematical properties of subtraction separately, but not when combined in a single expression. @@ -522,9 +546,9 @@ ---------------------------------- The aware datetime comparison operators will work the same as they do -now with results indirectly affected by the value of ``fold`` whenever -``utcoffset()`` value of one of the operands depends on it, with one -exception. Whenever one of the operands in inter-zone comparison is +now, with results indirectly affected by the value of ``fold`` whenever +the ``utcoffset()`` value of one of the operands depends on it, with one +exception. Whenever one or both of the operands in inter-zone comparison is such that its ``utcoffset()`` depends on the value of its ``fold`` fold attribute, the result is ``False``. [#]_ @@ -541,7 +565,7 @@ def toutc(t, fold): u = t - t.replace(fold=fold).utcoffset() - return u.replace(tzinfo=None) + return u.replace(tzinfo=None) Then ``t == s`` is equivalent to -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Mon Sep 21 22:22:25 2015 From: python-checkins at python.org (victor.stinner) Date: Mon, 21 Sep 2015 20:22:25 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzI1MTE0?= =?utf-8?q?=3A_Fix_test=5Fasyncio?= Message-ID: <20150921202225.115080.66921@psf.io> https://hg.python.org/cpython/rev/32e3a9e46f70 changeset: 98142:32e3a9e46f70 branch: 3.4 parent: 98138:682623e3e793 user: Victor Stinner date: Mon Sep 21 22:20:19 2015 +0200 summary: Issue #25114: Fix test_asyncio ssl.SSLContext() does not always disable compression. Fix unit test. files: Lib/test/test_asyncio/test_events.py | 5 ++--- 1 files changed, 2 insertions(+), 3 deletions(-) diff --git a/Lib/test/test_asyncio/test_events.py b/Lib/test/test_asyncio/test_events.py --- a/Lib/test/test_asyncio/test_events.py +++ b/Lib/test/test_asyncio/test_events.py @@ -619,9 +619,6 @@ self.assertEqual(peercert, client.get_extra_info('peercert')) - # Python disables compression to prevent CRIME attacks by default - self.assertIsNone(client.get_extra_info('compression')) - # test SSL cipher cipher = client.get_extra_info('cipher') self.assertIsInstance(cipher, tuple) @@ -639,6 +636,8 @@ client.get_extra_info('cipher')) self.assertEqual(sslobj.getpeercert(), client.get_extra_info('peercert')) + self.assertEqual(sslobj.compression(), + client.get_extra_info('compression')) def _basetest_create_ssl_connection(self, connection_fut, check_sockname=True, -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 21 22:22:26 2015 From: python-checkins at python.org (victor.stinner) Date: Mon, 21 Sep 2015 20:22:26 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Merge_3=2E4_=28test=5Fasyncio=29?= Message-ID: <20150921202225.11696.38601@psf.io> https://hg.python.org/cpython/rev/09d27433983f changeset: 98143:09d27433983f branch: 3.5 parent: 98139:1253225519da parent: 98142:32e3a9e46f70 user: Victor Stinner date: Mon Sep 21 22:20:36 2015 +0200 summary: Merge 3.4 (test_asyncio) files: Lib/test/test_asyncio/test_events.py | 5 ++--- 1 files changed, 2 insertions(+), 3 deletions(-) diff --git a/Lib/test/test_asyncio/test_events.py b/Lib/test/test_asyncio/test_events.py --- a/Lib/test/test_asyncio/test_events.py +++ b/Lib/test/test_asyncio/test_events.py @@ -619,9 +619,6 @@ self.assertEqual(peercert, client.get_extra_info('peercert')) - # Python disables compression to prevent CRIME attacks by default - self.assertIsNone(client.get_extra_info('compression')) - # test SSL cipher cipher = client.get_extra_info('cipher') self.assertIsInstance(cipher, tuple) @@ -639,6 +636,8 @@ client.get_extra_info('cipher')) self.assertEqual(sslobj.getpeercert(), client.get_extra_info('peercert')) + self.assertEqual(sslobj.compression(), + client.get_extra_info('compression')) def _basetest_create_ssl_connection(self, connection_fut, check_sockname=True, -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 21 22:22:25 2015 From: python-checkins at python.org (victor.stinner) Date: Mon, 21 Sep 2015 20:22:25 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogTWVyZ2UgMy41ICh0ZXN0X2FzeW5jaW8p?= Message-ID: <20150921202225.98358.71159@psf.io> https://hg.python.org/cpython/rev/59ad04ce3c29 changeset: 98144:59ad04ce3c29 parent: 98141:5230115de64d parent: 98143:09d27433983f user: Victor Stinner date: Mon Sep 21 22:20:52 2015 +0200 summary: Merge 3.5 (test_asyncio) files: Lib/test/test_asyncio/test_events.py | 5 ++--- 1 files changed, 2 insertions(+), 3 deletions(-) diff --git a/Lib/test/test_asyncio/test_events.py b/Lib/test/test_asyncio/test_events.py --- a/Lib/test/test_asyncio/test_events.py +++ b/Lib/test/test_asyncio/test_events.py @@ -619,9 +619,6 @@ self.assertEqual(peercert, client.get_extra_info('peercert')) - # Python disables compression to prevent CRIME attacks by default - self.assertIsNone(client.get_extra_info('compression')) - # test SSL cipher cipher = client.get_extra_info('cipher') self.assertIsInstance(cipher, tuple) @@ -639,6 +636,8 @@ client.get_extra_info('cipher')) self.assertEqual(sslobj.getpeercert(), client.get_extra_info('peercert')) + self.assertEqual(sslobj.compression(), + client.get_extra_info('compression')) def _basetest_create_ssl_connection(self, connection_fut, check_sockname=True, -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 21 22:32:25 2015 From: python-checkins at python.org (victor.stinner) Date: Mon, 21 Sep 2015 20:32:25 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Merge_3=2E4_=28Issue_=2323630=2C_fix_test=5Fasyncio=29?= Message-ID: <20150921203225.16583.38031@psf.io> https://hg.python.org/cpython/rev/840942a3f6aa changeset: 98146:840942a3f6aa branch: 3.5 parent: 98143:09d27433983f parent: 98145:42e7334e0e4c user: Victor Stinner date: Mon Sep 21 22:29:30 2015 +0200 summary: Merge 3.4 (Issue #23630, fix test_asyncio) files: Lib/test/test_asyncio/test_events.py | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diff --git a/Lib/test/test_asyncio/test_events.py b/Lib/test/test_asyncio/test_events.py --- a/Lib/test/test_asyncio/test_events.py +++ b/Lib/test/test_asyncio/test_events.py @@ -764,6 +764,7 @@ for host in hosts] self.loop.getaddrinfo = getaddrinfo_task self.loop._start_serving = mock.Mock() + self.loop._stop_serving = mock.Mock() f = self.loop.create_server(lambda: MyProto(self.loop), hosts, 80) server = self.loop.run_until_complete(f) self.addCleanup(server.close) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 21 22:32:27 2015 From: python-checkins at python.org (victor.stinner) Date: Mon, 21 Sep 2015 20:32:27 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Merge_3=2E5_=28Issue_=2323?= =?utf-8?q?630=2C_fix_test=5Fasyncio=29?= Message-ID: <20150921203225.9929.98688@psf.io> https://hg.python.org/cpython/rev/1f979a573f8d changeset: 98147:1f979a573f8d parent: 98144:59ad04ce3c29 user: Victor Stinner date: Mon Sep 21 22:29:43 2015 +0200 summary: Merge 3.5 (Issue #23630, fix test_asyncio) files: Lib/test/test_asyncio/test_events.py | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diff --git a/Lib/test/test_asyncio/test_events.py b/Lib/test/test_asyncio/test_events.py --- a/Lib/test/test_asyncio/test_events.py +++ b/Lib/test/test_asyncio/test_events.py @@ -764,6 +764,7 @@ for host in hosts] self.loop.getaddrinfo = getaddrinfo_task self.loop._start_serving = mock.Mock() + self.loop._stop_serving = mock.Mock() f = self.loop.create_server(lambda: MyProto(self.loop), hosts, 80) server = self.loop.run_until_complete(f) self.addCleanup(server.close) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 21 22:32:30 2015 From: python-checkins at python.org (victor.stinner) Date: Mon, 21 Sep 2015 20:32:30 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzIzNjMw?= =?utf-8?q?=3A_Fix_test=5Fasyncio_on_Windows?= Message-ID: <20150921203225.81649.25932@psf.io> https://hg.python.org/cpython/rev/42e7334e0e4c changeset: 98145:42e7334e0e4c branch: 3.4 parent: 98142:32e3a9e46f70 user: Victor Stinner date: Mon Sep 21 22:28:44 2015 +0200 summary: Issue #23630: Fix test_asyncio on Windows The proactor event loop requires also to mock loop._stop_serving. files: Lib/test/test_asyncio/test_events.py | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diff --git a/Lib/test/test_asyncio/test_events.py b/Lib/test/test_asyncio/test_events.py --- a/Lib/test/test_asyncio/test_events.py +++ b/Lib/test/test_asyncio/test_events.py @@ -764,6 +764,7 @@ for host in hosts] self.loop.getaddrinfo = getaddrinfo_task self.loop._start_serving = mock.Mock() + self.loop._stop_serving = mock.Mock() f = self.loop.create_server(lambda: MyProto(self.loop), hosts, 80) server = self.loop.run_until_complete(f) self.addCleanup(server.close) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 21 22:37:56 2015 From: python-checkins at python.org (victor.stinner) Date: Mon, 21 Sep 2015 20:37:56 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2325207=2C_=2314626?= =?utf-8?q?=3A_Fix_ICC_compiler_warnings_in_posixmodule=2Ec?= Message-ID: <20150921203756.3646.28210@psf.io> https://hg.python.org/cpython/rev/a138a1131bae changeset: 98148:a138a1131bae user: Victor Stinner date: Mon Sep 21 22:37:15 2015 +0200 summary: Issue #25207, #14626: Fix ICC compiler warnings in posixmodule.c Replace "#if XXX" with #ifdef XXX". files: Modules/posixmodule.c | 6 +++--- 1 files changed, 3 insertions(+), 3 deletions(-) diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c --- a/Modules/posixmodule.c +++ b/Modules/posixmodule.c @@ -4609,7 +4609,7 @@ #define UTIME_HAVE_NOFOLLOW_SYMLINKS \ (defined(HAVE_UTIMENSAT) || defined(HAVE_LUTIMES)) -#if UTIME_HAVE_NOFOLLOW_SYMLINKS +#ifdef UTIME_HAVE_NOFOLLOW_SYMLINKS static int utime_nofollow_symlinks(utime_t *ut, char *path) @@ -4771,7 +4771,7 @@ utime.now = 1; } -#if !UTIME_HAVE_NOFOLLOW_SYMLINKS +#if !defined(UTIME_HAVE_NOFOLLOW_SYMLINKS) if (follow_symlinks_specified("utime", follow_symlinks)) goto exit; #endif @@ -4825,7 +4825,7 @@ #else /* MS_WINDOWS */ Py_BEGIN_ALLOW_THREADS -#if UTIME_HAVE_NOFOLLOW_SYMLINKS +#ifdef UTIME_HAVE_NOFOLLOW_SYMLINKS if ((!follow_symlinks) && (dir_fd == DEFAULT_DIR_FD)) result = utime_nofollow_symlinks(&utime, path->narrow); else -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 21 22:45:14 2015 From: python-checkins at python.org (victor.stinner) Date: Mon, 21 Sep 2015 20:45:14 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_ssue_=2325207=3A_fix_ICC_c?= =?utf-8?q?ompiler_warning_in_msvcrtmodule=2Ec?= Message-ID: <20150921204513.81647.37334@psf.io> https://hg.python.org/cpython/rev/73867bf953e6 changeset: 98149:73867bf953e6 user: Victor Stinner date: Mon Sep 21 22:40:28 2015 +0200 summary: ssue #25207: fix ICC compiler warning in msvcrtmodule.c files: PC/msvcrtmodule.c | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-) diff --git a/PC/msvcrtmodule.c b/PC/msvcrtmodule.c --- a/PC/msvcrtmodule.c +++ b/PC/msvcrtmodule.c @@ -541,7 +541,6 @@ #endif /* constants for the crt versions */ - (void)st; #ifdef _VC_ASSEMBLY_PUBLICKEYTOKEN st = PyModule_AddStringConstant(m, "VC_ASSEMBLY_PUBLICKEYTOKEN", _VC_ASSEMBLY_PUBLICKEYTOKEN); @@ -567,6 +566,8 @@ st = PyModule_AddObject(m, "CRT_ASSEMBLY_VERSION", version); if (st < 0) return NULL; #endif + /* make compiler warning quiet if st is unused */ + (void)st; return m; } -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 21 23:12:18 2015 From: python-checkins at python.org (alexander.belopolsky) Date: Mon, 21 Sep 2015 21:12:18 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_PEP_495_=28minor=29=3A_Corect?= =?utf-8?q?ed_markup=2E?= Message-ID: <20150921211217.98354.71129@psf.io> https://hg.python.org/peps/rev/f14bd86db3bc changeset: 6091:f14bd86db3bc user: Alexander Belopolsky date: Mon Sep 21 17:12:12 2015 -0400 summary: PEP 495 (minor): Corected markup. files: pep-0495.txt | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/pep-0495.txt b/pep-0495.txt --- a/pep-0495.txt +++ b/pep-0495.txt @@ -17,7 +17,7 @@ This PEP adds a new attribute ``fold`` to instances of the ``datetime.time`` and ``datetime.datetime`` classes that can be used to differentiate between two moments in time for which local times are -the same. The allowed values for the `fold` attribute will be 0 and 1 +the same. The allowed values for the ``fold`` attribute will be 0 and 1 with 0 corresponding to the earlier and 1 to the later of the two possible readings of an ambiguous local time. -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Mon Sep 21 23:16:33 2015 From: python-checkins at python.org (eric.smith) Date: Mon, 21 Sep 2015 21:16:33 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_Partial_fix_for_issue_25206?= =?utf-8?q?=3A_Fixed_format=28=29_call=2E?= Message-ID: <20150921211632.16587.96851@psf.io> https://hg.python.org/peps/rev/831276834e4b changeset: 6092:831276834e4b user: Eric V. Smith date: Mon Sep 21 17:16:30 2015 -0400 summary: Partial fix for issue 25206: Fixed format() call. files: pep-0498.txt | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/pep-0498.txt b/pep-0498.txt --- a/pep-0498.txt +++ b/pep-0498.txt @@ -298,7 +298,7 @@ Might be be evaluated as:: - 'abc' + format(expr1, spec1) + format(repr(expr2)) + 'def' + format(str(expr3)) + 'ghi' + 'abc' + format(expr1, spec1) + format(repr(expr2), spec2) + 'def' + format(str(expr3)) + 'ghi' Expression evaluation --------------------- -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Mon Sep 21 23:52:57 2015 From: python-checkins at python.org (alexander.belopolsky) Date: Mon, 21 Sep 2015 21:52:57 +0000 Subject: [Python-checkins] =?utf-8?q?peps_=28merge_default_-=3E_default=29?= =?utf-8?q?=3A_merge?= Message-ID: <20150921215257.16573.53156@psf.io> https://hg.python.org/peps/rev/25b6c610eaba changeset: 6094:25b6c610eaba parent: 6093:b32bb8775d14 parent: 6092:831276834e4b user: Alexander Belopolsky date: Mon Sep 21 17:52:52 2015 -0400 summary: merge files: pep-0498.txt | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/pep-0498.txt b/pep-0498.txt --- a/pep-0498.txt +++ b/pep-0498.txt @@ -298,7 +298,7 @@ Might be be evaluated as:: - 'abc' + format(expr1, spec1) + format(repr(expr2)) + 'def' + format(str(expr3)) + 'ghi' + 'abc' + format(expr1, spec1) + format(repr(expr2), spec2) + 'def' + format(str(expr3)) + 'ghi' Expression evaluation --------------------- -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Mon Sep 21 23:52:57 2015 From: python-checkins at python.org (alexander.belopolsky) Date: Mon, 21 Sep 2015 21:52:57 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_PEP_495_=28minor=29=3A_Final_?= =?utf-8?q?edits=2E?= Message-ID: <20150921215257.81635.91743@psf.io> https://hg.python.org/peps/rev/b32bb8775d14 changeset: 6093:b32bb8775d14 parent: 6091:f14bd86db3bc user: Alexander Belopolsky date: Mon Sep 21 17:52:10 2015 -0400 summary: PEP 495 (minor): Final edits. files: pep-0495.txt | 13 ++++++------- 1 files changed, 6 insertions(+), 7 deletions(-) diff --git a/pep-0495.txt b/pep-0495.txt --- a/pep-0495.txt +++ b/pep-0495.txt @@ -343,7 +343,7 @@ ``datetime.tzinfo`` class will not change. It is currently not used anywhere in the stdlib because the only included ``tzinfo`` implementation (the ``datetime.timezone`` class implementing fixed -offset timezones) override ``fromutc()``. Keeping the default +offset timezones) overrides ``fromutc()``. Keeping the default implementation unchanged has the benefit that pre-PEP 3rd party implementations that inherit the default ``fromutc()`` are not accidentally affected. @@ -369,7 +369,7 @@ ----------- New subclasses should override the base-class ``fromutc()`` method and -implement it so that in all cases where two UTC times ``u0`` and +implement it so that in all cases where two different UTC times ``u0`` and ``u1`` (``u0`` <``u1``) correspond to the same local time ``t``, ``fromutc(u0)`` will return an instance with ``fold=0`` and ``fromutc(u1)`` will return an instance with ``fold=1``. In all @@ -381,7 +381,7 @@ transition. By definition, ``utcoffset()`` is greater before and smaller after any transition that creates a fold. The values returned by ``tzname()`` and ``dst()`` may or may not depend on the value of -the fold attribute depending on the kind of the transition. +the ``fold`` attribute depending on the kind of the transition. .. image:: pep-0495-fold-2.png :align: center @@ -695,10 +695,9 @@ **ltdf** (Local Time Disambiguation Flag) - short and no-one will attempt - to guess what it means without reading the docs. (Feel free to - use it in discussions with the meaning ltdf=False is the - earlier if you don't want to endorse any of the alternatives - above.) + to guess what it means without reading the docs. (This abbreviation + was used in PEP discussions with the meaning ``ltdf=False`` is the + earlier by those who didn't want to endorse any of the alternatives.) .. _original: https://mail.python.org/pipermail/python-dev/2015-April/139099.html .. _independently proposed: https://mail.python.org/pipermail/datetime-sig/2015-August/000479.html -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Tue Sep 22 00:11:16 2015 From: python-checkins at python.org (victor.stinner) Date: Mon, 21 Sep 2015 22:11:16 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2324870=3A_Optimize?= =?utf-8?q?_the_ASCII_decoder_for_error_handlers=3A_surrogateescape=2C?= Message-ID: <20150921221115.94131.1360@psf.io> https://hg.python.org/cpython/rev/3c430259873e changeset: 98150:3c430259873e user: Victor Stinner date: Mon Sep 21 23:06:27 2015 +0200 summary: Issue #24870: Optimize the ASCII decoder for error handlers: surrogateescape, ignore and replace. Initial patch written by Naoki Inada. The decoder is now up to 60 times as fast for these error handlers. Add also unit tests for the ASCII decoder. files: Doc/whatsnew/3.6.rst | 3 +- Lib/test/test_codecs.py | 32 +++++++++++++ Objects/unicodeobject.c | 68 ++++++++++++++++++++++++++-- 3 files changed, 96 insertions(+), 7 deletions(-) diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst --- a/Doc/whatsnew/3.6.rst +++ b/Doc/whatsnew/3.6.rst @@ -106,7 +106,8 @@ Optimizations ============= -* None yet. +* The ASCII decoder is now up to 60 times as fast for error handlers: + ``surrogateescape``, ``ignore`` and ``replace``. Build and C API Changes diff --git a/Lib/test/test_codecs.py b/Lib/test/test_codecs.py --- a/Lib/test/test_codecs.py +++ b/Lib/test/test_codecs.py @@ -27,6 +27,7 @@ self.assertEqual(coder(input), (expect, len(input))) return check + class Queue(object): """ queue: write bytes at one end, read bytes from the other end @@ -47,6 +48,7 @@ self._buffer = self._buffer[size:] return s + class MixInCheckStateHandling: def check_state_handling_decode(self, encoding, u, s): for i in range(len(s)+1): @@ -80,6 +82,7 @@ part2 = d.encode(u[i:], True) self.assertEqual(s, part1+part2) + class ReadTest(MixInCheckStateHandling): def check_partial(self, input, partialresults): # get a StreamReader for the encoding and feed the bytestring version @@ -383,6 +386,7 @@ self.assertEqual(test_sequence.decode(self.encoding, "backslashreplace"), before + backslashreplace + after) + class UTF32Test(ReadTest, unittest.TestCase): encoding = "utf-32" if sys.byteorder == 'little': @@ -478,6 +482,7 @@ self.assertEqual('\U00010000' * 1024, codecs.utf_32_decode(encoded_be)[0]) + class UTF32LETest(ReadTest, unittest.TestCase): encoding = "utf-32-le" ill_formed_sequence = b"\x80\xdc\x00\x00" @@ -523,6 +528,7 @@ self.assertEqual('\U00010000' * 1024, codecs.utf_32_le_decode(encoded)[0]) + class UTF32BETest(ReadTest, unittest.TestCase): encoding = "utf-32-be" ill_formed_sequence = b"\x00\x00\xdc\x80" @@ -797,6 +803,7 @@ with self.assertRaises(UnicodeDecodeError): b"abc\xed\xa0z".decode("utf-8", "surrogatepass") + @unittest.skipUnless(sys.platform == 'win32', 'cp65001 is a Windows-only codec') class CP65001Test(ReadTest, unittest.TestCase): @@ -1136,6 +1143,7 @@ self.assertEqual(decode(br"[\x0]\x0", "ignore"), (b"[]", 8)) self.assertEqual(decode(br"[\x0]\x0", "replace"), (b"[?]?", 8)) + class RecodingTest(unittest.TestCase): def test_recoding(self): f = io.BytesIO() @@ -1255,6 +1263,7 @@ if len(i)!=2: print(repr(i)) + class PunycodeTest(unittest.TestCase): def test_encode(self): for uni, puny in punycode_testcases: @@ -1274,6 +1283,7 @@ puny = puny.decode("ascii").encode("ascii") self.assertEqual(uni, puny.decode("punycode")) + class UnicodeInternalTest(unittest.TestCase): @unittest.skipUnless(SIZEOF_WCHAR_T == 4, 'specific to 32-bit wchar_t') def test_bug1251300(self): @@ -1528,6 +1538,7 @@ except Exception as e: raise support.TestFailed("Test 3.%d: %s" % (pos+1, str(e))) + class IDNACodecTest(unittest.TestCase): def test_builtin_decode(self): self.assertEqual(str(b"python.org", "idna"), "python.org") @@ -1614,6 +1625,7 @@ self.assertRaises(Exception, b"python.org".decode, "idna", errors) + class CodecsModuleTest(unittest.TestCase): def test_decode(self): @@ -1722,6 +1734,7 @@ self.assertRaises(UnicodeError, codecs.decode, b'abc', 'undefined', errors) + class StreamReaderTest(unittest.TestCase): def setUp(self): @@ -1732,6 +1745,7 @@ f = self.reader(self.stream) self.assertEqual(f.readlines(), ['\ud55c\n', '\uae00']) + class EncodedFileTest(unittest.TestCase): def test_basic(self): @@ -1862,6 +1876,7 @@ "unicode_internal" ] + class BasicUnicodeTest(unittest.TestCase, MixInCheckStateHandling): def test_basics(self): s = "abc123" # all codecs should be able to encode these @@ -2024,6 +2039,7 @@ self.check_state_handling_decode(encoding, u, u.encode(encoding)) self.check_state_handling_encode(encoding, u, u.encode(encoding)) + class CharmapTest(unittest.TestCase): def test_decode_with_string_map(self): self.assertEqual( @@ -2274,6 +2290,7 @@ info.streamwriter, 'strict') as srw: self.assertEqual(srw.read(), "\xfc") + class TypesTest(unittest.TestCase): def test_decode_unicode(self): # Most decoders don't accept unicode input @@ -2564,6 +2581,7 @@ bytes_transform_encodings.append("bz2_codec") transform_aliases["bz2_codec"] = ["bz2"] + class TransformCodecTest(unittest.TestCase): def test_basics(self): @@ -3041,5 +3059,19 @@ self.assertEqual(decoded, ('abc', 3)) +class ASCIITest(unittest.TestCase): + def test_decode(self): + for data, error_handler, expected in ( + (b'[\x80\xff]', 'ignore', '[]'), + (b'[\x80\xff]', 'replace', '[\ufffd\ufffd]'), + (b'[\x80\xff]', 'surrogateescape', '[\udc80\udcff]'), + (b'[\x80\xff]', 'backslashreplace', '[\\x80\\xff]'), + ): + with self.subTest(data=data, error_handler=error_handler, + expected=expected): + self.assertEqual(data.decode('ascii', error_handler), + expected) + + if __name__ == "__main__": unittest.main() diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c --- a/Objects/unicodeobject.c +++ b/Objects/unicodeobject.c @@ -6644,6 +6644,28 @@ /* --- 7-bit ASCII Codec -------------------------------------------------- */ +typedef enum { + _Py_ERROR_UNKNOWN=0, + _Py_ERROR_SURROGATEESCAPE, + _Py_ERROR_REPLACE, + _Py_ERROR_IGNORE, + _Py_ERROR_OTHER +} _Py_error_handler; + +static _Py_error_handler +get_error_handler(const char *errors) +{ + if (errors == NULL) + return _Py_ERROR_OTHER; + if (strcmp(errors, "surrogateescape") == 0) + return _Py_ERROR_SURROGATEESCAPE; + if (strcmp(errors, "ignore") == 0) + return _Py_ERROR_IGNORE; + if (strcmp(errors, "replace") == 0) + return _Py_ERROR_REPLACE; + return _Py_ERROR_OTHER; +} + PyObject * PyUnicode_DecodeASCII(const char *s, Py_ssize_t size, @@ -6657,8 +6679,9 @@ Py_ssize_t endinpos; Py_ssize_t outpos; const char *e; - PyObject *errorHandler = NULL; + PyObject *error_handler_obj = NULL; PyObject *exc = NULL; + _Py_error_handler error_handler = _Py_ERROR_UNKNOWN; if (size == 0) _Py_RETURN_UNICODE_EMPTY(); @@ -6687,12 +6710,45 @@ PyUnicode_WRITE(kind, data, writer.pos, c); writer.pos++; ++s; - } - else { + continue; + } + + /* byte outsize range 0x00..0x7f: call the error handler */ + + if (error_handler == _Py_ERROR_UNKNOWN) + error_handler = get_error_handler(errors); + + switch (error_handler) + { + case _Py_ERROR_REPLACE: + case _Py_ERROR_SURROGATEESCAPE: + /* Fast-path: the error handler only writes one character, + but we must switch to UCS2 at the first write */ + if (kind < PyUnicode_2BYTE_KIND) { + if (_PyUnicodeWriter_Prepare(&writer, size - writer.pos, + 0xffff) < 0) + return NULL; + kind = writer.kind; + data = writer.data; + } + + if (error_handler == _Py_ERROR_REPLACE) + PyUnicode_WRITE(kind, data, writer.pos, 0xfffd); + else + PyUnicode_WRITE(kind, data, writer.pos, c + 0xdc00); + writer.pos++; + ++s; + break; + + case _Py_ERROR_IGNORE: + ++s; + break; + + default: startinpos = s-starts; endinpos = startinpos + 1; if (unicode_decode_call_errorhandler_writer( - errors, &errorHandler, + errors, &error_handler_obj, "ascii", "ordinal not in range(128)", &starts, &e, &startinpos, &endinpos, &exc, &s, &writer)) @@ -6701,13 +6757,13 @@ data = writer.data; } } - Py_XDECREF(errorHandler); + Py_XDECREF(error_handler_obj); Py_XDECREF(exc); return _PyUnicodeWriter_Finish(&writer); onError: _PyUnicodeWriter_Dealloc(&writer); - Py_XDECREF(errorHandler); + Py_XDECREF(error_handler_obj); Py_XDECREF(exc); return NULL; } -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 22 01:00:03 2015 From: python-checkins at python.org (victor.stinner) Date: Mon, 21 Sep 2015 23:00:03 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2324870=3A_Add_=5FP?= =?utf-8?q?yUnicodeWriter=5FPrepareKind=28=29_macro?= Message-ID: <20150921230003.3652.21189@psf.io> https://hg.python.org/cpython/rev/aa247150a8b1 changeset: 98152:aa247150a8b1 user: Victor Stinner date: Tue Sep 22 00:58:32 2015 +0200 summary: Issue #24870: Add _PyUnicodeWriter_PrepareKind() macro Add a macro which ensures that the writer has at least the requested kind. files: Include/unicodeobject.h | 17 ++++++++++++ Objects/unicodeobject.c | 38 ++++++++++++++++++++++------ 2 files changed, 46 insertions(+), 9 deletions(-) diff --git a/Include/unicodeobject.h b/Include/unicodeobject.h --- a/Include/unicodeobject.h +++ b/Include/unicodeobject.h @@ -942,6 +942,23 @@ _PyUnicodeWriter_PrepareInternal(_PyUnicodeWriter *writer, Py_ssize_t length, Py_UCS4 maxchar); +/* Prepare the buffer to have at least the kind KIND. + For example, kind=PyUnicode_2BYTE_KIND ensures that the writer will + support characters in range U+000-U+FFFF. + + Return 0 on success, raise an exception and return -1 on error. */ +#define _PyUnicodeWriter_PrepareKind(WRITER, KIND) \ + (assert((KIND) != PyUnicode_WCHAR_KIND), \ + (KIND) <= (WRITER)->kind \ + ? 0 \ + : _PyUnicodeWriter_PrepareKindInternal((WRITER), (KIND))) + +/* Don't call this function directly, use the _PyUnicodeWriter_PrepareKind() + macro instead. */ +PyAPI_FUNC(int) +_PyUnicodeWriter_PrepareKindInternal(_PyUnicodeWriter *writer, + enum PyUnicode_Kind kind); + /* Append a Unicode character. Return 0 on success, raise an exception and return -1 on error. */ PyAPI_FUNC(int) diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c --- a/Objects/unicodeobject.c +++ b/Objects/unicodeobject.c @@ -6722,14 +6722,11 @@ case _Py_ERROR_REPLACE: case _Py_ERROR_SURROGATEESCAPE: /* Fast-path: the error handler only writes one character, - but we must switch to UCS2 at the first write */ - if (kind < PyUnicode_2BYTE_KIND) { - if (_PyUnicodeWriter_Prepare(&writer, size - writer.pos, - 0xffff) < 0) - return NULL; - kind = writer.kind; - data = writer.data; - } + but we may switch to UCS2 at the first write */ + if (_PyUnicodeWriter_PrepareKind(&writer, PyUnicode_2BYTE_KIND) < 0) + goto onError; + kind = writer.kind; + data = writer.data; if (error_handler == _Py_ERROR_REPLACE) PyUnicode_WRITE(kind, data, writer.pos, 0xfffd); @@ -13309,7 +13306,8 @@ Py_ssize_t newlen; PyObject *newbuffer; - assert(length > 0); + /* ensure that the _PyUnicodeWriter_Prepare macro was used */ + assert(maxchar > writer->maxchar || length > 0); if (length > PY_SSIZE_T_MAX - writer->pos) { PyErr_NoMemory(); @@ -13375,6 +13373,28 @@ #undef OVERALLOCATE_FACTOR } +int +_PyUnicodeWriter_PrepareKindInternal(_PyUnicodeWriter *writer, + enum PyUnicode_Kind kind) +{ + Py_UCS4 maxchar; + + /* ensure that the _PyUnicodeWriter_PrepareKind macro was used */ + assert(writer->kind < kind); + + switch (kind) + { + case PyUnicode_1BYTE_KIND: maxchar = 0xff; break; + case PyUnicode_2BYTE_KIND: maxchar = 0xffff; break; + case PyUnicode_4BYTE_KIND: maxchar = 0x10ffff; break; + default: + assert(0 && "invalid kind"); + return -1; + } + + return _PyUnicodeWriter_PrepareInternal(writer, 0, maxchar); +} + Py_LOCAL_INLINE(int) _PyUnicodeWriter_WriteCharInline(_PyUnicodeWriter *writer, Py_UCS4 ch) { -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 22 01:00:06 2015 From: python-checkins at python.org (victor.stinner) Date: Mon, 21 Sep 2015 23:00:06 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2324870=3A_Reuse_th?= =?utf-8?q?e_new_=5FPy=5Ferror=5Fhandler_enum?= Message-ID: <20150921230003.98350.97160@psf.io> https://hg.python.org/cpython/rev/2cf85e2834c2 changeset: 98151:2cf85e2834c2 user: Victor Stinner date: Tue Sep 22 00:26:54 2015 +0200 summary: Issue #24870: Reuse the new _Py_error_handler enum Factorize code with the new get_error_handler() function. Add some empty lines for readability. files: Objects/unicodeobject.c | 166 +++++++++++++-------------- 1 files changed, 78 insertions(+), 88 deletions(-) diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c --- a/Objects/unicodeobject.c +++ b/Objects/unicodeobject.c @@ -293,6 +293,34 @@ #include "clinic/unicodeobject.c.h" +typedef enum { + _Py_ERROR_UNKNOWN=0, + _Py_ERROR_STRICT, + _Py_ERROR_SURROGATEESCAPE, + _Py_ERROR_REPLACE, + _Py_ERROR_IGNORE, + _Py_ERROR_XMLCHARREFREPLACE, + _Py_ERROR_OTHER +} _Py_error_handler; + +static _Py_error_handler +get_error_handler(const char *errors) +{ + if (errors == NULL) + return _Py_ERROR_STRICT; + if (strcmp(errors, "strict") == 0) + return _Py_ERROR_STRICT; + if (strcmp(errors, "surrogateescape") == 0) + return _Py_ERROR_SURROGATEESCAPE; + if (strcmp(errors, "ignore") == 0) + return _Py_ERROR_IGNORE; + if (strcmp(errors, "replace") == 0) + return _Py_ERROR_REPLACE; + if (strcmp(errors, "xmlcharrefreplace") == 0) + return _Py_ERROR_XMLCHARREFREPLACE; + return _Py_ERROR_OTHER; +} + /* The max unicode value is always 0x10FFFF while using the PEP-393 API. This function is kept for backward compatibility with the old API. */ Py_UNICODE @@ -3163,24 +3191,22 @@ static int locale_error_handler(const char *errors, int *surrogateescape) { - if (errors == NULL) { + _Py_error_handler error_handler = get_error_handler(errors); + switch (error_handler) + { + case _Py_ERROR_STRICT: *surrogateescape = 0; return 0; - } - - if (strcmp(errors, "strict") == 0) { - *surrogateescape = 0; - return 0; - } - if (strcmp(errors, "surrogateescape") == 0) { + case _Py_ERROR_SURROGATEESCAPE: *surrogateescape = 1; return 0; - } - PyErr_Format(PyExc_ValueError, - "only 'strict' and 'surrogateescape' error handlers " - "are supported, not '%s'", - errors); - return -1; + default: + PyErr_Format(PyExc_ValueError, + "only 'strict' and 'surrogateescape' error handlers " + "are supported, not '%s'", + errors); + return -1; + } } PyObject * @@ -6403,11 +6429,9 @@ Py_ssize_t ressize; const char *encoding = (limit == 256) ? "latin-1" : "ascii"; const char *reason = (limit == 256) ? "ordinal not in range(256)" : "ordinal not in range(128)"; - PyObject *errorHandler = NULL; + PyObject *error_handler_obj = NULL; PyObject *exc = NULL; - /* the following variable is used for caching string comparisons - * -1=not initialized, 0=unknown, 1=strict, 2=replace, 3=ignore, 4=xmlcharrefreplace */ - int known_errorHandler = -1; + _Py_error_handler error_handler = _Py_ERROR_UNKNOWN; if (PyUnicode_READY(unicode) == -1) return NULL; @@ -6441,32 +6465,28 @@ Py_ssize_t collstart = pos; Py_ssize_t collend = pos; /* find all unecodable characters */ + while ((collend < size) && (PyUnicode_READ(kind, data, collend) >= limit)) ++collend; + /* cache callback name lookup (if not done yet, i.e. it's the first error) */ - if (known_errorHandler==-1) { - if ((errors==NULL) || (!strcmp(errors, "strict"))) - known_errorHandler = 1; - else if (!strcmp(errors, "replace")) - known_errorHandler = 2; - else if (!strcmp(errors, "ignore")) - known_errorHandler = 3; - else if (!strcmp(errors, "xmlcharrefreplace")) - known_errorHandler = 4; - else - known_errorHandler = 0; - } - switch (known_errorHandler) { - case 1: /* strict */ + if (error_handler == _Py_ERROR_UNKNOWN) + error_handler = get_error_handler(errors); + + switch (error_handler) { + case _Py_ERROR_STRICT: raise_encode_exception(&exc, encoding, unicode, collstart, collend, reason); goto onError; - case 2: /* replace */ + + case _Py_ERROR_REPLACE: while (collstart++ < collend) - *str++ = '?'; /* fall through */ - case 3: /* ignore */ + *str++ = '?'; + /* fall through */ + case _Py_ERROR_IGNORE: pos = collend; break; - case 4: /* xmlcharrefreplace */ + + case _Py_ERROR_XMLCHARREFREPLACE: respos = str - PyBytes_AS_STRING(res); requiredsize = respos; /* determine replacement size */ @@ -6510,8 +6530,9 @@ } pos = collend; break; + default: - repunicode = unicode_encode_call_errorhandler(errors, &errorHandler, + repunicode = unicode_encode_call_errorhandler(errors, &error_handler_obj, encoding, reason, unicode, &exc, collstart, collend, &newpos); if (repunicode == NULL || (PyUnicode_Check(repunicode) && @@ -6587,7 +6608,7 @@ goto onError; } - Py_XDECREF(errorHandler); + Py_XDECREF(error_handler_obj); Py_XDECREF(exc); return res; @@ -6597,7 +6618,7 @@ onError: Py_XDECREF(res); - Py_XDECREF(errorHandler); + Py_XDECREF(error_handler_obj); Py_XDECREF(exc); return NULL; } @@ -6644,28 +6665,6 @@ /* --- 7-bit ASCII Codec -------------------------------------------------- */ -typedef enum { - _Py_ERROR_UNKNOWN=0, - _Py_ERROR_SURROGATEESCAPE, - _Py_ERROR_REPLACE, - _Py_ERROR_IGNORE, - _Py_ERROR_OTHER -} _Py_error_handler; - -static _Py_error_handler -get_error_handler(const char *errors) -{ - if (errors == NULL) - return _Py_ERROR_OTHER; - if (strcmp(errors, "surrogateescape") == 0) - return _Py_ERROR_SURROGATEESCAPE; - if (strcmp(errors, "ignore") == 0) - return _Py_ERROR_IGNORE; - if (strcmp(errors, "replace") == 0) - return _Py_ERROR_REPLACE; - return _Py_ERROR_OTHER; -} - PyObject * PyUnicode_DecodeASCII(const char *s, Py_ssize_t size, @@ -8129,7 +8128,7 @@ charmap_encoding_error( PyObject *unicode, Py_ssize_t *inpos, PyObject *mapping, PyObject **exceptionObject, - int *known_errorHandler, PyObject **errorHandler, const char *errors, + _Py_error_handler *error_handler, PyObject **error_handler_obj, const char *errors, PyObject **res, Py_ssize_t *respos) { PyObject *repunicode = NULL; /* initialize to prevent gcc warning */ @@ -8176,23 +8175,15 @@ } /* cache callback name lookup * (if not done yet, i.e. it's the first error) */ - if (*known_errorHandler==-1) { - if ((errors==NULL) || (!strcmp(errors, "strict"))) - *known_errorHandler = 1; - else if (!strcmp(errors, "replace")) - *known_errorHandler = 2; - else if (!strcmp(errors, "ignore")) - *known_errorHandler = 3; - else if (!strcmp(errors, "xmlcharrefreplace")) - *known_errorHandler = 4; - else - *known_errorHandler = 0; - } - switch (*known_errorHandler) { - case 1: /* strict */ + if (*error_handler == _Py_ERROR_UNKNOWN) + *error_handler = get_error_handler(errors); + + switch (*error_handler) { + case _Py_ERROR_STRICT: raise_encode_exception(exceptionObject, encoding, unicode, collstartpos, collendpos, reason); return -1; - case 2: /* replace */ + + case _Py_ERROR_REPLACE: for (collpos = collstartpos; collpos https://hg.python.org/cpython/rev/ecee4ff91bf8 changeset: 98153:ecee4ff91bf8 user: Victor Stinner date: Tue Sep 22 01:01:17 2015 +0200 summary: _PyUnicodeWriter_PrepareInternal(): make the assertion more strict files: Objects/unicodeobject.c | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-) diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c --- a/Objects/unicodeobject.c +++ b/Objects/unicodeobject.c @@ -13307,7 +13307,8 @@ PyObject *newbuffer; /* ensure that the _PyUnicodeWriter_Prepare macro was used */ - assert(maxchar > writer->maxchar || length > 0); + assert((maxchar > writer->maxchar && length >= 0) + || length > 0); if (length > PY_SSIZE_T_MAX - writer->pos) { PyErr_NoMemory(); -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 22 01:29:42 2015 From: python-checkins at python.org (victor.stinner) Date: Mon, 21 Sep 2015 23:29:42 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2325207=2C_=2314626?= =?utf-8?q?=3A_Fix_my_commit=2E?= Message-ID: <20150921232942.82648.52888@psf.io> https://hg.python.org/cpython/rev/170cd0104267 changeset: 98154:170cd0104267 user: Victor Stinner date: Tue Sep 22 01:29:33 2015 +0200 summary: Issue #25207, #14626: Fix my commit. It doesn't work to use #define XXX defined(YYY)" and then "#ifdef XXX" to check YYY. files: Modules/posixmodule.c | 6 +- Objects/unicodeobject.c | 52 +++++++++++++++++++++++----- 2 files changed, 46 insertions(+), 12 deletions(-) diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c --- a/Modules/posixmodule.c +++ b/Modules/posixmodule.c @@ -4605,9 +4605,9 @@ #define PATH_UTIME_HAVE_FD 0 #endif - -#define UTIME_HAVE_NOFOLLOW_SYMLINKS \ - (defined(HAVE_UTIMENSAT) || defined(HAVE_LUTIMES)) +#if defined(HAVE_UTIMENSAT) || defined(HAVE_LUTIMES) +# define UTIME_HAVE_NOFOLLOW_SYMLINKS +#endif #ifdef UTIME_HAVE_NOFOLLOW_SYMLINKS diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c --- a/Objects/unicodeobject.c +++ b/Objects/unicodeobject.c @@ -4709,8 +4709,9 @@ Py_ssize_t startinpos; Py_ssize_t endinpos; const char *errmsg = ""; - PyObject *errorHandler = NULL; + PyObject *error_handler_obj = NULL; PyObject *exc = NULL; + _Py_error_handler error_handler = _Py_ERROR_UNKNOWN; if (size == 0) { if (consumed) @@ -4773,24 +4774,57 @@ continue; } - if (unicode_decode_call_errorhandler_writer( - errors, &errorHandler, - "utf-8", errmsg, - &starts, &end, &startinpos, &endinpos, &exc, &s, - &writer)) - goto onError; + /* undecodable byte: call the error handler */ + + if (error_handler == _Py_ERROR_UNKNOWN) + error_handler = get_error_handler(errors); + + switch (error_handler) + { + case _Py_ERROR_REPLACE: + case _Py_ERROR_SURROGATEESCAPE: + { + unsigned char ch = (unsigned char)*s; + + /* Fast-path: the error handler only writes one character, + but we may switch to UCS2 at the first write */ + if (_PyUnicodeWriter_PrepareKind(&writer, PyUnicode_2BYTE_KIND) < 0) + goto onError; + kind = writer.kind; + + if (error_handler == _Py_ERROR_REPLACE) + PyUnicode_WRITE(kind, writer.data, writer.pos, 0xfffd); + else + PyUnicode_WRITE(kind, writer.data, writer.pos, ch + 0xdc00); + writer.pos++; + ++s; + break; + } + + case _Py_ERROR_IGNORE: + s++; + break; + + default: + if (unicode_decode_call_errorhandler_writer( + errors, &error_handler_obj, + "utf-8", errmsg, + &starts, &end, &startinpos, &endinpos, &exc, &s, + &writer)) + goto onError; + } } End: if (consumed) *consumed = s - starts; - Py_XDECREF(errorHandler); + Py_XDECREF(error_handler_obj); Py_XDECREF(exc); return _PyUnicodeWriter_Finish(&writer); onError: - Py_XDECREF(errorHandler); + Py_XDECREF(error_handler_obj); Py_XDECREF(exc); _PyUnicodeWriter_Dealloc(&writer); return NULL; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 22 01:36:09 2015 From: python-checkins at python.org (terry.reedy) Date: Mon, 21 Sep 2015 23:36:09 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzI0ODYx?= =?utf-8?q?=3A_add_Idle_news_item_and_correct_previous_errors=2E?= Message-ID: <20150921233609.11692.57449@psf.io> https://hg.python.org/cpython/rev/5087ca9e6002 changeset: 98156:5087ca9e6002 branch: 3.4 parent: 98145:42e7334e0e4c user: Terry Jan Reedy date: Mon Sep 21 19:28:22 2015 -0400 summary: Issue #24861: add Idle news item and correct previous errors. files: Lib/idlelib/NEWS.txt | 7 +++++-- Misc/NEWS | 7 +++++-- 2 files changed, 10 insertions(+), 4 deletions(-) diff --git a/Lib/idlelib/NEWS.txt b/Lib/idlelib/NEWS.txt --- a/Lib/idlelib/NEWS.txt +++ b/Lib/idlelib/NEWS.txt @@ -2,12 +2,15 @@ ========================= *Release date: 2015-??-??* +- Issue #24861: Most of idlelib is private and subject to change. + Use idleib.idle.* to start Idle. See idlelib.__init__.__doc__. + - Issue #25199: Idle: add synchronization comments for future maintainers. - Issue #16893: Replace help.txt with idle.html for Idle doc display. - The new idlelib/idle.html is copied from Doc/build/html/idle.html. + The new idlelib/idle.html is copied from Doc/build/html/library/idle.html. It looks better than help.txt and will better document Idle as released. - The tkinter html viewer that works for this file was written by Rose Roseman. + The tkinter html viewer that works for this file was written by Mark Roseman. The now unused EditorWindow.HelpDialog class and helt.txt file are deprecated. - Issue #24199: Deprecate unused idlelib.idlever with possible removal in 3.6. diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -442,12 +442,15 @@ IDLE ---- +- Issue #24861: Most of idlelib is private and subject to change. + Use idleib.idle.* to start Idle. See idlelib.__init__.__doc__. + - Issue #25199: Idle: add synchronization comments for future maintainers. - Issue #16893: Replace help.txt with idle.html for Idle doc display. - The new idlelib/idle.html is copied from Doc/build/html/idle.html. + The new idlelib/idle.html is copied from Doc/build/html/library/idle.html. It looks better than help.txt and will better document Idle as released. - The tkinter html viewer that works for this file was written by Rose Roseman. + The tkinter html viewer that works for this file was written by Mark Roseman. The now unused EditorWindow.HelpDialog class and helt.txt file are deprecated. - Issue #24199: Deprecate unused idlelib.idlever with possible removal in 3.6. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 22 01:36:09 2015 From: python-checkins at python.org (terry.reedy) Date: Mon, 21 Sep 2015 23:36:09 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge_with_3=2E5?= Message-ID: <20150921233609.11690.30224@psf.io> https://hg.python.org/cpython/rev/dd221ffecf92 changeset: 98158:dd221ffecf92 parent: 98153:ecee4ff91bf8 parent: 98157:28583bc0680e user: Terry Jan Reedy date: Mon Sep 21 19:33:46 2015 -0400 summary: Merge with 3.5 files: Lib/idlelib/NEWS.txt | 7 +++++-- Misc/NEWS | 7 +++++-- 2 files changed, 10 insertions(+), 4 deletions(-) diff --git a/Lib/idlelib/NEWS.txt b/Lib/idlelib/NEWS.txt --- a/Lib/idlelib/NEWS.txt +++ b/Lib/idlelib/NEWS.txt @@ -2,12 +2,15 @@ =========================== *Release date: 2017?* +- Issue #24861: Most of idlelib is private and subject to change. + Use idleib.idle.* to start Idle. See idlelib.__init__.__doc__. + - Issue #25199: Idle: add synchronization comments for future maintainers. - Issue #16893: Replace help.txt with idle.html for Idle doc display. - The new idlelib/idle.html is copied from Doc/build/html/idle.html. + The new idlelib/idle.html is copied from Doc/build/html/library/idle.html. It looks better than help.txt and will better document Idle as released. - The tkinter html viewer that works for this file was written by Rose Roseman. + The tkinter html viewer that works for this file was written by Mark Roseman. The now unused EditorWindow.HelpDialog class and helt.txt file are deprecated. - Issue #24199: Deprecate unused idlelib.idlever with possible removal in 3.6. diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -203,12 +203,15 @@ IDLE ---- +- Issue #24861: Most of idlelib is private and subject to change. + Use idleib.idle.* to start Idle. See idlelib.__init__.__doc__. + - Issue #25199: Idle: add synchronization comments for future maintainers. - Issue #16893: Replace help.txt with idle.html for Idle doc display. - The new idlelib/idle.html is copied from Doc/build/html/idle.html. + The new idlelib/idle.html is copied from Doc/build/html/library/idle.html. It looks better than help.txt and will better document Idle as released. - The tkinter html viewer that works for this file was written by Rose Roseman. + The tkinter html viewer that works for this file was written by Mark Roseman. The now unused EditorWindow.HelpDialog class and helt.txt file are deprecated. - Issue #24199: Deprecate unused idlelib.idlever with possible removal in 3.6. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 22 01:36:09 2015 From: python-checkins at python.org (terry.reedy) Date: Mon, 21 Sep 2015 23:36:09 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Marge_3=2E4?= Message-ID: <20150921233609.94127.74863@psf.io> https://hg.python.org/cpython/rev/28583bc0680e changeset: 98157:28583bc0680e branch: 3.5 parent: 98146:840942a3f6aa parent: 98156:5087ca9e6002 user: Terry Jan Reedy date: Mon Sep 21 19:33:14 2015 -0400 summary: Marge 3.4 files: Lib/idlelib/NEWS.txt | 7 +++++-- Misc/NEWS | 7 +++++-- 2 files changed, 10 insertions(+), 4 deletions(-) diff --git a/Lib/idlelib/NEWS.txt b/Lib/idlelib/NEWS.txt --- a/Lib/idlelib/NEWS.txt +++ b/Lib/idlelib/NEWS.txt @@ -2,12 +2,15 @@ ========================= *Release date: 2015-09-13* +- Issue #24861: Most of idlelib is private and subject to change. + Use idleib.idle.* to start Idle. See idlelib.__init__.__doc__. + - Issue #25199: Idle: add synchronization comments for future maintainers. - Issue #16893: Replace help.txt with idle.html for Idle doc display. - The new idlelib/idle.html is copied from Doc/build/html/idle.html. + The new idlelib/idle.html is copied from Doc/build/html/library/idle.html. It looks better than help.txt and will better document Idle as released. - The tkinter html viewer that works for this file was written by Rose Roseman. + The tkinter html viewer that works for this file was written by Mark Roseman. The now unused EditorWindow.HelpDialog class and helt.txt file are deprecated. - Issue #24199: Deprecate unused idlelib.idlever with possible removal in 3.6. diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -94,12 +94,15 @@ IDLE ---- +- Issue #24861: Most of idlelib is private and subject to change. + Use idleib.idle.* to start Idle. See idlelib.__init__.__doc__. + - Issue #25199: Idle: add synchronization comments for future maintainers. - Issue #16893: Replace help.txt with idle.html for Idle doc display. - The new idlelib/idle.html is copied from Doc/build/html/idle.html. + The new idlelib/idle.html is copied from Doc/build/html/library/idle.html. It looks better than help.txt and will better document Idle as released. - The tkinter html viewer that works for this file was written by Rose Roseman. + The tkinter html viewer that works for this file was written by Mark Roseman. The now unused EditorWindow.HelpDialog class and helt.txt file are deprecated. - Issue #24199: Deprecate unused idlelib.idlever with possible removal in 3.6. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 22 01:36:09 2015 From: python-checkins at python.org (terry.reedy) Date: Mon, 21 Sep 2015 23:36:09 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_default_-=3E_default?= =?utf-8?q?=29=3A_Merge?= Message-ID: <20150921233609.3642.57193@psf.io> https://hg.python.org/cpython/rev/6988771a1084 changeset: 98159:6988771a1084 parent: 98154:170cd0104267 parent: 98158:dd221ffecf92 user: Terry Jan Reedy date: Mon Sep 21 19:35:51 2015 -0400 summary: Merge files: Lib/idlelib/NEWS.txt | 7 +++++-- Misc/NEWS | 7 +++++-- 2 files changed, 10 insertions(+), 4 deletions(-) diff --git a/Lib/idlelib/NEWS.txt b/Lib/idlelib/NEWS.txt --- a/Lib/idlelib/NEWS.txt +++ b/Lib/idlelib/NEWS.txt @@ -2,12 +2,15 @@ =========================== *Release date: 2017?* +- Issue #24861: Most of idlelib is private and subject to change. + Use idleib.idle.* to start Idle. See idlelib.__init__.__doc__. + - Issue #25199: Idle: add synchronization comments for future maintainers. - Issue #16893: Replace help.txt with idle.html for Idle doc display. - The new idlelib/idle.html is copied from Doc/build/html/idle.html. + The new idlelib/idle.html is copied from Doc/build/html/library/idle.html. It looks better than help.txt and will better document Idle as released. - The tkinter html viewer that works for this file was written by Rose Roseman. + The tkinter html viewer that works for this file was written by Mark Roseman. The now unused EditorWindow.HelpDialog class and helt.txt file are deprecated. - Issue #24199: Deprecate unused idlelib.idlever with possible removal in 3.6. diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -203,12 +203,15 @@ IDLE ---- +- Issue #24861: Most of idlelib is private and subject to change. + Use idleib.idle.* to start Idle. See idlelib.__init__.__doc__. + - Issue #25199: Idle: add synchronization comments for future maintainers. - Issue #16893: Replace help.txt with idle.html for Idle doc display. - The new idlelib/idle.html is copied from Doc/build/html/idle.html. + The new idlelib/idle.html is copied from Doc/build/html/library/idle.html. It looks better than help.txt and will better document Idle as released. - The tkinter html viewer that works for this file was written by Rose Roseman. + The tkinter html viewer that works for this file was written by Mark Roseman. The now unused EditorWindow.HelpDialog class and helt.txt file are deprecated. - Issue #24199: Deprecate unused idlelib.idlever with possible removal in 3.6. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 22 01:36:10 2015 From: python-checkins at python.org (terry.reedy) Date: Mon, 21 Sep 2015 23:36:10 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzI0ODYx?= =?utf-8?q?=3A_add_Idle_news_items_and_correct_previous_errors=2E?= Message-ID: <20150921233609.11704.42668@psf.io> https://hg.python.org/cpython/rev/e9edb95ca8b6 changeset: 98155:e9edb95ca8b6 branch: 2.7 parent: 98124:b7bbb2c1e1f9 user: Terry Jan Reedy date: Mon Sep 21 19:28:18 2015 -0400 summary: Issue #24861: add Idle news items and correct previous errors. files: Lib/idlelib/NEWS.txt | 7 +++++-- Misc/NEWS | 7 +++++-- 2 files changed, 10 insertions(+), 4 deletions(-) diff --git a/Lib/idlelib/NEWS.txt b/Lib/idlelib/NEWS.txt --- a/Lib/idlelib/NEWS.txt +++ b/Lib/idlelib/NEWS.txt @@ -2,12 +2,15 @@ ========================= *Release date: +- Issue #24861: Most of idlelib is private and subject to change. + Use idleib.idle.* to start Idle. See idlelib.__init__.__doc__. + - Issue #25199: Idle: add synchronization comments for future maintainers. - Issue #16893: Replace help.txt with idle.html for Idle doc display. - The new idlelib/idle.html is copied from Doc/build/html/idle.html. + The new idlelib/idle.html is copied from Doc/build/html/library/idle.html. It looks better than help.txt and will better document Idle as released. - The tkinter html viewer that works for this file was written by Rose Roseman. + The tkinter html viewer that works for this file was written by Mark Roseman. The now unused EditorWindow.HelpDialog class and helt.txt file are deprecated. - Issue #24199: Deprecate unused idlelib.idlever with possible removal in 3.6. diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -167,12 +167,15 @@ IDLE ---- +- Issue #24861: Most of idlelib is private and subject to change. + Use idleib.idle.* to start Idle. See idlelib.__init__.__doc__. + - Issue #25199: Idle: add synchronization comments for future maintainers. - Issue #16893: Replace help.txt with idle.html for Idle doc display. - The new idlelib/idle.html is copied from Doc/build/html/idle.html. + The new idlelib/idle.html is copied from Doc/build/html/library/idle.html. It looks better than help.txt and will better document Idle as released. - The tkinter html viewer that works for this file was written by Rose Roseman. + The tkinter html viewer that works for this file was written by Mark Roseman. The now unused EditorWindow.HelpDialog class and helt.txt file are deprecated. - Issue #24199: Deprecate unused idlelib.idlever with possible removal in 3.6. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 22 01:42:58 2015 From: python-checkins at python.org (alexander.belopolsky) Date: Mon, 21 Sep 2015 23:42:58 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_PEP_495=3A_Changed_status_to_?= =?utf-8?b?IkFjY2VwdGVkLiI=?= Message-ID: <20150921234258.31189.84534@psf.io> https://hg.python.org/peps/rev/13bbd922195f changeset: 6095:13bbd922195f user: Alexander Belopolsky date: Mon Sep 21 19:42:51 2015 -0400 summary: PEP 495: Changed status to "Accepted." files: pep-0495.txt | 5 +++-- 1 files changed, 3 insertions(+), 2 deletions(-) diff --git a/pep-0495.txt b/pep-0495.txt --- a/pep-0495.txt +++ b/pep-0495.txt @@ -4,11 +4,12 @@ Last-Modified: $Date$ Author: Alexander Belopolsky , Tim Peters Discussions-To: Datetime-SIG -Status: Draft +Status: Accepted Type: Standards Track Content-Type: text/x-rst Created: 02-Aug-2015 - +Python-Version: 3.6 +Resolution: https://mail.python.org/pipermail/datetime-sig/2015-September/000900.html Abstract -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Tue Sep 22 01:50:21 2015 From: python-checkins at python.org (alexander.belopolsky) Date: Mon, 21 Sep 2015 23:50:21 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_PEP_500=3A_Changed_status_to_?= =?utf-8?b?IlJlamVjdGVkLiI=?= Message-ID: <20150921235019.31193.14559@psf.io> https://hg.python.org/peps/rev/d43fe1069360 changeset: 6096:d43fe1069360 user: Alexander Belopolsky date: Mon Sep 21 19:50:15 2015 -0400 summary: PEP 500: Changed status to "Rejected." files: pep-0500.txt | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/pep-0500.txt b/pep-0500.txt --- a/pep-0500.txt +++ b/pep-0500.txt @@ -5,12 +5,12 @@ Last-Modified: $Date$ Author: Alexander Belopolsky , Tim Peters Discussions-To: Datetime-SIG -Status: Draft +Status: Rejected Type: Standards Track Content-Type: text/x-rst Requires: 495 Created: 08-Aug-2015 - +Resolution: https://mail.python.org/pipermail/datetime-sig/2015-August/000354.html Abstract ======== -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Tue Sep 22 04:43:19 2015 From: python-checkins at python.org (terry.reedy) Date: Tue, 22 Sep 2015 02:43:19 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_whitespace?= Message-ID: <20150922024318.9941.27800@psf.io> https://hg.python.org/cpython/rev/c48a5234c142 changeset: 98165:c48a5234c142 branch: 3.4 parent: 98161:df987a0bc350 user: Terry Jan Reedy date: Mon Sep 21 22:42:32 2015 -0400 summary: whitespace files: Lib/idlelib/help.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Lib/idlelib/help.py b/Lib/idlelib/help.py --- a/Lib/idlelib/help.py +++ b/Lib/idlelib/help.py @@ -242,7 +242,7 @@ filename = join(abspath(dirname(__file__)), 'help.html') if not isfile(filename): # try copy_strip, present message - return + return HelpWindow(parent, filename, 'IDLE Help') if __name__ == '__main__': -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 22 04:43:19 2015 From: python-checkins at python.org (terry.reedy) Date: Tue, 22 Sep 2015 02:43:19 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Merge_with_3=2E4?= Message-ID: <20150922024319.9953.83981@psf.io> https://hg.python.org/cpython/rev/9a66286b970c changeset: 98166:9a66286b970c branch: 3.5 parent: 98162:f08437278049 parent: 98165:c48a5234c142 user: Terry Jan Reedy date: Mon Sep 21 22:42:43 2015 -0400 summary: Merge with 3.4 files: Lib/idlelib/help.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Lib/idlelib/help.py b/Lib/idlelib/help.py --- a/Lib/idlelib/help.py +++ b/Lib/idlelib/help.py @@ -242,7 +242,7 @@ filename = join(abspath(dirname(__file__)), 'help.html') if not isfile(filename): # try copy_strip, present message - return + return HelpWindow(parent, filename, 'IDLE Help') if __name__ == '__main__': -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 22 04:43:19 2015 From: python-checkins at python.org (terry.reedy) Date: Tue, 22 Sep 2015 02:43:19 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge_with_3=2E5?= Message-ID: <20150922024319.9951.35479@psf.io> https://hg.python.org/cpython/rev/2c3f9b7b457d changeset: 98167:2c3f9b7b457d parent: 98163:09ebed6a8cb8 parent: 98166:9a66286b970c user: Terry Jan Reedy date: Mon Sep 21 22:42:55 2015 -0400 summary: Merge with 3.5 files: Lib/idlelib/help.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Lib/idlelib/help.py b/Lib/idlelib/help.py --- a/Lib/idlelib/help.py +++ b/Lib/idlelib/help.py @@ -242,7 +242,7 @@ filename = join(abspath(dirname(__file__)), 'help.html') if not isfile(filename): # try copy_strip, present message - return + return HelpWindow(parent, filename, 'IDLE Help') if __name__ == '__main__': -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 22 04:43:18 2015 From: python-checkins at python.org (terry.reedy) Date: Tue, 22 Sep 2015 02:43:18 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_whitespace?= Message-ID: <20150922024318.9947.51445@psf.io> https://hg.python.org/cpython/rev/9d77345adcc9 changeset: 98164:9d77345adcc9 branch: 2.7 parent: 98160:855484b55da3 user: Terry Jan Reedy date: Mon Sep 21 22:42:17 2015 -0400 summary: whitespace files: Lib/idlelib/help.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Lib/idlelib/help.py b/Lib/idlelib/help.py --- a/Lib/idlelib/help.py +++ b/Lib/idlelib/help.py @@ -245,7 +245,7 @@ filename = join(abspath(dirname(__file__)), 'help.html') if not isfile(filename): # try copy_strip, present message - return + return HelpWindow(parent, filename, 'IDLE Help') if __name__ == '__main__': -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 22 04:43:18 2015 From: python-checkins at python.org (terry.reedy) Date: Tue, 22 Sep 2015 02:43:18 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzE2ODkz?= =?utf-8?q?=3A_Add_idlelib=2Ehelp=2Ecopy=5Fstrip=28=29_to_copy-rstrip_Doc/?= =?utf-8?b?Li4uL2lkbGUuaHRtbC4=?= Message-ID: <20150922024318.31191.55543@psf.io> https://hg.python.org/cpython/rev/df987a0bc350 changeset: 98161:df987a0bc350 branch: 3.4 parent: 98156:5087ca9e6002 user: Terry Jan Reedy date: Mon Sep 21 22:36:42 2015 -0400 summary: Issue #16893: Add idlelib.help.copy_strip() to copy-rstrip Doc/.../idle.html. Change destination to help.html. Adjust NEWS entries. files: Lib/idlelib/NEWS.txt | 4 +- Lib/idlelib/help.html | 0 Lib/idlelib/help.py | 53 ++++++++++++++++++++---------- Misc/NEWS | 4 +- 4 files changed, 39 insertions(+), 22 deletions(-) diff --git a/Lib/idlelib/NEWS.txt b/Lib/idlelib/NEWS.txt --- a/Lib/idlelib/NEWS.txt +++ b/Lib/idlelib/NEWS.txt @@ -7,8 +7,8 @@ - Issue #25199: Idle: add synchronization comments for future maintainers. -- Issue #16893: Replace help.txt with idle.html for Idle doc display. - The new idlelib/idle.html is copied from Doc/build/html/library/idle.html. +- Issue #16893: Replace help.txt with help.html for Idle doc display. + The new idlelib/help.html is rstripped Doc/build/html/library/idle.html. It looks better than help.txt and will better document Idle as released. The tkinter html viewer that works for this file was written by Mark Roseman. The now unused EditorWindow.HelpDialog class and helt.txt file are deprecated. diff --git a/Lib/idlelib/idle.html b/Lib/idlelib/help.html rename from Lib/idlelib/idle.html rename to Lib/idlelib/help.html diff --git a/Lib/idlelib/help.py b/Lib/idlelib/help.py --- a/Lib/idlelib/help.py +++ b/Lib/idlelib/help.py @@ -1,23 +1,26 @@ -""" -help.py implements the Idle help menu and is subject to change. +""" help.py: Implement the Idle help menu. +Contents are subject to revision at any time, without notice. -The contents are subject to revision at any time, without notice. Help => About IDLE: diplay About Idle dialog -Help => IDLE Help: display idle.html with proper formatting -HelpParser - Parses idle.html generated from idle.rst by Sphinx -and renders to tk Text. +Help => IDLE Help: Display help.html with proper formatting. +Doc/library/idle.rst (Sphinx)=> Doc/build/html/library/idle.html +(help.copy_strip)=> Lib/idlelib/help.html -HelpText - Displays formatted idle.html. +HelpParser - Parse help.html and and render to tk Text. -HelpFrame - Contains text, scrollbar, and table-of-contents. +HelpText - Display formatted help.html. + +HelpFrame - Contain text, scrollbar, and table-of-contents. (This will be needed for display in a future tabbed window.) -HelpWindow - Display idleframe in a standalone window. +HelpWindow - Display HelpFrame in a standalone window. + +copy_strip - Copy idle.html to help.html, rstripping each line. show_idlehelp - Create HelpWindow. Called in EditorWindow.help_dialog. """ @@ -36,7 +39,7 @@ ## IDLE Help ## class HelpParser(HTMLParser): - """Render idle.html generated by Sphinx from idle.rst. + """Render help.html into a text widget. The overridden handle_xyz methods handle a subset of html tags. The supplied text should have the needed tag configurations. @@ -62,7 +65,7 @@ self.tags = '' if self.level == 0 else 'l'+str(self.level) def handle_starttag(self, tag, attrs): - "Handle starttags in idle.html." + "Handle starttags in help.html." class_ = '' for a, v in attrs: if a == 'class': @@ -120,7 +123,7 @@ self.text.insert('end', s, self.tags) def handle_endtag(self, tag): - "Handle endtags in idle.html." + "Handle endtags in help.html." if tag in ['h1', 'h2', 'h3', 'span', 'em']: self.indent(0) # clear tag, reset indent if self.show and tag in ['h1', 'h2', 'h3']: @@ -136,7 +139,7 @@ self.indent(amt=-1) def handle_data(self, data): - "Handle date segments in idle.html." + "Handle date segments in help.html." if self.show and not self.hdrlink: d = data if self.pre else data.replace('\n', ' ') if self.tags == 'h1': @@ -149,7 +152,7 @@ class HelpText(Text): - "Display idle.html." + "Display help.html." def __init__(self, parent, filename): "Configure tags and feed file to parser." Text.__init__(self, parent, wrap='word', highlightthickness=0, @@ -188,6 +191,7 @@ class HelpFrame(Frame): + "Display html text, scrollbar, and toc." def __init__(self, parent, filename): Frame.__init__(self, parent) text = HelpText(self, filename) @@ -202,6 +206,7 @@ toc.grid(column=0, row=0, sticky='nw') def contents_widget(self, text): + "Create table of contents." toc = Menubutton(self, text='TOC') drop = Menu(toc, tearoff=False) for tag, lbl in text.parser.contents: @@ -211,7 +216,7 @@ class HelpWindow(Toplevel): - + "Display frame with rendered html." def __init__(self, parent, filename, title): Toplevel.__init__(self, parent) self.wm_title(title) @@ -221,11 +226,23 @@ self.grid_rowconfigure(0, weight=1) +def copy_strip(): + "Copy idle.html to idlelib/help.html, stripping trailing whitespace." + src = join(abspath(dirname(dirname(dirname(__file__)))), + 'Doc', 'build', 'html', 'library', 'idle.html') + dst = join(abspath(dirname(__file__)), 'help.html') + with open(src, 'rb') as inn,\ + open(dst, 'wb') as out: + for line in inn: + out.write(line.rstrip() + '\n') + print('idle.html copied to help.html') + def show_idlehelp(parent): - filename = join(abspath(dirname(__file__)), 'idle.html') + "Create HelpWindow; called from Idle Help event handler." + filename = join(abspath(dirname(__file__)), 'help.html') if not isfile(filename): - dirpath = join(abspath(dirname(dirname(dirname(__file__)))), - 'Doc', 'build', 'html', 'library') + # try copy_strip, present message + return HelpWindow(parent, filename, 'IDLE Help') if __name__ == '__main__': diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -447,8 +447,8 @@ - Issue #25199: Idle: add synchronization comments for future maintainers. -- Issue #16893: Replace help.txt with idle.html for Idle doc display. - The new idlelib/idle.html is copied from Doc/build/html/library/idle.html. +- Issue #16893: Replace help.txt with help.html for Idle doc display. + The new idlelib/help.html is rstripped Doc/build/html/library/idle.html. It looks better than help.txt and will better document Idle as released. The tkinter html viewer that works for this file was written by Mark Roseman. The now unused EditorWindow.HelpDialog class and helt.txt file are deprecated. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 22 04:43:18 2015 From: python-checkins at python.org (terry.reedy) Date: Tue, 22 Sep 2015 02:43:18 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzE2ODkz?= =?utf-8?q?=3A_Add_idlelib=2Ehelp=2Ecopy=5Fstrip=28=29_to_copy-rstrip_Doc/?= =?utf-8?b?Li4uL2lkbGUuaHRtbC4=?= Message-ID: <20150922024318.115149.36455@psf.io> https://hg.python.org/cpython/rev/855484b55da3 changeset: 98160:855484b55da3 branch: 2.7 parent: 98155:e9edb95ca8b6 user: Terry Jan Reedy date: Mon Sep 21 22:36:36 2015 -0400 summary: Issue #16893: Add idlelib.help.copy_strip() to copy-rstrip Doc/.../idle.html. Change destination to help.html. Adjust NEWS entries. files: Lib/idlelib/NEWS.txt | 4 +- Lib/idlelib/help.html | 0 Lib/idlelib/help.py | 53 ++++++++++++++++++++---------- Misc/NEWS | 4 +- 4 files changed, 39 insertions(+), 22 deletions(-) diff --git a/Lib/idlelib/NEWS.txt b/Lib/idlelib/NEWS.txt --- a/Lib/idlelib/NEWS.txt +++ b/Lib/idlelib/NEWS.txt @@ -7,8 +7,8 @@ - Issue #25199: Idle: add synchronization comments for future maintainers. -- Issue #16893: Replace help.txt with idle.html for Idle doc display. - The new idlelib/idle.html is copied from Doc/build/html/library/idle.html. +- Issue #16893: Replace help.txt with help.html for Idle doc display. + The new idlelib/help.html is rstripped Doc/build/html/library/idle.html. It looks better than help.txt and will better document Idle as released. The tkinter html viewer that works for this file was written by Mark Roseman. The now unused EditorWindow.HelpDialog class and helt.txt file are deprecated. diff --git a/Lib/idlelib/idle.html b/Lib/idlelib/help.html rename from Lib/idlelib/idle.html rename to Lib/idlelib/help.html diff --git a/Lib/idlelib/help.py b/Lib/idlelib/help.py --- a/Lib/idlelib/help.py +++ b/Lib/idlelib/help.py @@ -1,23 +1,26 @@ -""" -help.py implements the Idle help menu and is subject to change. +""" help.py: Implement the Idle help menu. +Contents are subject to revision at any time, without notice. -The contents are subject to revision at any time, without notice. Help => About IDLE: diplay About Idle dialog -Help => IDLE Help: display idle.html with proper formatting -HelpParser - Parses idle.html generated from idle.rst by Sphinx -and renders to tk Text. +Help => IDLE Help: Display help.html with proper formatting. +Doc/library/idle.rst (Sphinx)=> Doc/build/html/library/idle.html +(help.copy_strip)=> Lib/idlelib/help.html -HelpText - Displays formatted idle.html. +HelpParser - Parse help.html and and render to tk Text. -HelpFrame - Contains text, scrollbar, and table-of-contents. +HelpText - Display formatted help.html. + +HelpFrame - Contain text, scrollbar, and table-of-contents. (This will be needed for display in a future tabbed window.) -HelpWindow - Display idleframe in a standalone window. +HelpWindow - Display HelpFrame in a standalone window. + +copy_strip - Copy idle.html to help.html, rstripping each line. show_idlehelp - Create HelpWindow. Called in EditorWindow.help_dialog. """ @@ -36,7 +39,7 @@ ## IDLE Help ## class HelpParser(HTMLParser): - """Render idle.html generated by Sphinx from idle.rst. + """Render help.html into a text widget. The overridden handle_xyz methods handle a subset of html tags. The supplied text should have the needed tag configurations. @@ -62,7 +65,7 @@ self.tags = '' if self.level == 0 else 'l'+str(self.level) def handle_starttag(self, tag, attrs): - "Handle starttags in idle.html." + "Handle starttags in help.html." class_ = '' for a, v in attrs: if a == 'class': @@ -120,7 +123,7 @@ self.text.insert('end', s, self.tags) def handle_endtag(self, tag): - "Handle endtags in idle.html." + "Handle endtags in help.html." if tag in ['h1', 'h2', 'h3', 'span', 'em']: self.indent(0) # clear tag, reset indent if self.show and tag in ['h1', 'h2', 'h3']: @@ -136,7 +139,7 @@ self.indent(amt=-1) def handle_data(self, data): - "Handle date segments in idle.html." + "Handle date segments in help.html." if self.show and not self.hdrlink: d = data if self.pre else data.replace('\n', ' ') if self.tags == 'h1': @@ -152,7 +155,7 @@ class HelpText(Text): - "Display idle.html." + "Display help.html." def __init__(self, parent, filename): "Configure tags and feed file to parser." Text.__init__(self, parent, wrap='word', highlightthickness=0, @@ -191,6 +194,7 @@ class HelpFrame(Frame): + "Display html text, scrollbar, and toc." def __init__(self, parent, filename): Frame.__init__(self, parent) text = HelpText(self, filename) @@ -205,6 +209,7 @@ toc.grid(column=0, row=0, sticky='nw') def contents_widget(self, text): + "Create table of contents." toc = Menubutton(self, text='TOC') drop = Menu(toc, tearoff=False) for tag, lbl in text.parser.contents: @@ -214,7 +219,7 @@ class HelpWindow(Toplevel): - + "Display frame with rendered html." def __init__(self, parent, filename, title): Toplevel.__init__(self, parent) self.wm_title(title) @@ -224,11 +229,23 @@ self.grid_rowconfigure(0, weight=1) +def copy_strip(): + "Copy idle.html to idlelib/help.html, stripping trailing whitespace." + src = join(abspath(dirname(dirname(dirname(__file__)))), + 'Doc', 'build', 'html', 'library', 'idle.html') + dst = join(abspath(dirname(__file__)), 'help.html') + with open(src, 'r') as inn,\ + open(dst, 'w') as out: + for line in inn: + out.write(line.rstrip() + '\n') + print('idle.html copied to help.html') + def show_idlehelp(parent): - filename = join(abspath(dirname(__file__)), 'idle.html') + "Create HelpWindow; called from Idle Help event handler." + filename = join(abspath(dirname(__file__)), 'help.html') if not isfile(filename): - dirpath = join(abspath(dirname(dirname(dirname(__file__)))), - 'Doc', 'build', 'html', 'library') + # try copy_strip, present message + return HelpWindow(parent, filename, 'IDLE Help') if __name__ == '__main__': diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -172,8 +172,8 @@ - Issue #25199: Idle: add synchronization comments for future maintainers. -- Issue #16893: Replace help.txt with idle.html for Idle doc display. - The new idlelib/idle.html is copied from Doc/build/html/library/idle.html. +- Issue #16893: Replace help.txt with help.html for Idle doc display. + The new idlelib/help.html is rstripped Doc/build/html/library/idle.html. It looks better than help.txt and will better document Idle as released. The tkinter html viewer that works for this file was written by Mark Roseman. The now unused EditorWindow.HelpDialog class and helt.txt file are deprecated. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 22 04:43:19 2015 From: python-checkins at python.org (terry.reedy) Date: Tue, 22 Sep 2015 02:43:19 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogSXNzdWUgIzE2ODkzOiBBZGQgaWRsZWxpYi5oZWxwLmNvcHlfc3RyaXAo?= =?utf-8?q?=29_to_copy-rstrip_Doc/=2E=2E=2E/idle=2Ehtml=2E?= Message-ID: <20150922024318.94121.92090@psf.io> https://hg.python.org/cpython/rev/09ebed6a8cb8 changeset: 98163:09ebed6a8cb8 parent: 98159:6988771a1084 parent: 98162:f08437278049 user: Terry Jan Reedy date: Mon Sep 21 22:40:31 2015 -0400 summary: Issue #16893: Add idlelib.help.copy_strip() to copy-rstrip Doc/.../idle.html. Change destination to help.html. Adjust NEWS entries. files: Lib/idlelib/NEWS.txt | 4 +- Lib/idlelib/help.html | 0 Lib/idlelib/help.py | 53 ++++++++++++++++++++---------- Misc/NEWS | 28 ++------------- 4 files changed, 42 insertions(+), 43 deletions(-) diff --git a/Lib/idlelib/NEWS.txt b/Lib/idlelib/NEWS.txt --- a/Lib/idlelib/NEWS.txt +++ b/Lib/idlelib/NEWS.txt @@ -7,8 +7,8 @@ - Issue #25199: Idle: add synchronization comments for future maintainers. -- Issue #16893: Replace help.txt with idle.html for Idle doc display. - The new idlelib/idle.html is copied from Doc/build/html/library/idle.html. +- Issue #16893: Replace help.txt with help.html for Idle doc display. + The new idlelib/help.html is rstripped Doc/build/html/library/idle.html. It looks better than help.txt and will better document Idle as released. The tkinter html viewer that works for this file was written by Mark Roseman. The now unused EditorWindow.HelpDialog class and helt.txt file are deprecated. diff --git a/Lib/idlelib/idle.html b/Lib/idlelib/help.html rename from Lib/idlelib/idle.html rename to Lib/idlelib/help.html diff --git a/Lib/idlelib/help.py b/Lib/idlelib/help.py --- a/Lib/idlelib/help.py +++ b/Lib/idlelib/help.py @@ -1,23 +1,26 @@ -""" -help.py implements the Idle help menu and is subject to change. +""" help.py: Implement the Idle help menu. +Contents are subject to revision at any time, without notice. -The contents are subject to revision at any time, without notice. Help => About IDLE: diplay About Idle dialog -Help => IDLE Help: display idle.html with proper formatting -HelpParser - Parses idle.html generated from idle.rst by Sphinx -and renders to tk Text. +Help => IDLE Help: Display help.html with proper formatting. +Doc/library/idle.rst (Sphinx)=> Doc/build/html/library/idle.html +(help.copy_strip)=> Lib/idlelib/help.html -HelpText - Displays formatted idle.html. +HelpParser - Parse help.html and and render to tk Text. -HelpFrame - Contains text, scrollbar, and table-of-contents. +HelpText - Display formatted help.html. + +HelpFrame - Contain text, scrollbar, and table-of-contents. (This will be needed for display in a future tabbed window.) -HelpWindow - Display idleframe in a standalone window. +HelpWindow - Display HelpFrame in a standalone window. + +copy_strip - Copy idle.html to help.html, rstripping each line. show_idlehelp - Create HelpWindow. Called in EditorWindow.help_dialog. """ @@ -36,7 +39,7 @@ ## IDLE Help ## class HelpParser(HTMLParser): - """Render idle.html generated by Sphinx from idle.rst. + """Render help.html into a text widget. The overridden handle_xyz methods handle a subset of html tags. The supplied text should have the needed tag configurations. @@ -62,7 +65,7 @@ self.tags = '' if self.level == 0 else 'l'+str(self.level) def handle_starttag(self, tag, attrs): - "Handle starttags in idle.html." + "Handle starttags in help.html." class_ = '' for a, v in attrs: if a == 'class': @@ -120,7 +123,7 @@ self.text.insert('end', s, self.tags) def handle_endtag(self, tag): - "Handle endtags in idle.html." + "Handle endtags in help.html." if tag in ['h1', 'h2', 'h3', 'span', 'em']: self.indent(0) # clear tag, reset indent if self.show and tag in ['h1', 'h2', 'h3']: @@ -136,7 +139,7 @@ self.indent(amt=-1) def handle_data(self, data): - "Handle date segments in idle.html." + "Handle date segments in help.html." if self.show and not self.hdrlink: d = data if self.pre else data.replace('\n', ' ') if self.tags == 'h1': @@ -149,7 +152,7 @@ class HelpText(Text): - "Display idle.html." + "Display help.html." def __init__(self, parent, filename): "Configure tags and feed file to parser." Text.__init__(self, parent, wrap='word', highlightthickness=0, @@ -188,6 +191,7 @@ class HelpFrame(Frame): + "Display html text, scrollbar, and toc." def __init__(self, parent, filename): Frame.__init__(self, parent) text = HelpText(self, filename) @@ -202,6 +206,7 @@ toc.grid(column=0, row=0, sticky='nw') def contents_widget(self, text): + "Create table of contents." toc = Menubutton(self, text='TOC') drop = Menu(toc, tearoff=False) for tag, lbl in text.parser.contents: @@ -211,7 +216,7 @@ class HelpWindow(Toplevel): - + "Display frame with rendered html." def __init__(self, parent, filename, title): Toplevel.__init__(self, parent) self.wm_title(title) @@ -221,11 +226,23 @@ self.grid_rowconfigure(0, weight=1) +def copy_strip(): + "Copy idle.html to idlelib/help.html, stripping trailing whitespace." + src = join(abspath(dirname(dirname(dirname(__file__)))), + 'Doc', 'build', 'html', 'library', 'idle.html') + dst = join(abspath(dirname(__file__)), 'help.html') + with open(src, 'rb') as inn,\ + open(dst, 'wb') as out: + for line in inn: + out.write(line.rstrip() + '\n') + print('idle.html copied to help.html') + def show_idlehelp(parent): - filename = join(abspath(dirname(__file__)), 'idle.html') + "Create HelpWindow; called from Idle Help event handler." + filename = join(abspath(dirname(__file__)), 'help.html') if not isfile(filename): - dirpath = join(abspath(dirname(dirname(dirname(__file__)))), - 'Doc', 'build', 'html', 'library') + # try copy_strip, present message + return HelpWindow(parent, filename, 'IDLE Help') if __name__ == '__main__': diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -81,10 +81,13 @@ IDLE ---- +- Issue #24861: Most of idlelib is private and subject to change. + Use idleib.idle.* to start Idle. See idlelib.__init__.__doc__. + - Issue #25199: Idle: add synchronization comments for future maintainers. -- Issue #16893: Replace help.txt with idle.html for Idle doc display. - The new idlelib/idle.html is copied from Doc/build/html/idle.html. +- Issue #16893: Replace help.txt with help.html for Idle doc display. + The new idlelib/help.html is rstripped Doc/build/html/library/idle.html. It looks better than help.txt and will better document Idle as released. The tkinter html viewer that works for this file was written by Rose Roseman. The now unused EditorWindow.HelpDialog class and helt.txt file are deprecated. @@ -200,27 +203,6 @@ - Issue #23572: Fixed functools.singledispatch on classes with falsy metaclasses. Patch by Ethan Furman. -IDLE ----- - -- Issue #24861: Most of idlelib is private and subject to change. - Use idleib.idle.* to start Idle. See idlelib.__init__.__doc__. - -- Issue #25199: Idle: add synchronization comments for future maintainers. - -- Issue #16893: Replace help.txt with idle.html for Idle doc display. - The new idlelib/idle.html is copied from Doc/build/html/library/idle.html. - It looks better than help.txt and will better document Idle as released. - The tkinter html viewer that works for this file was written by Mark Roseman. - The now unused EditorWindow.HelpDialog class and helt.txt file are deprecated. - -- Issue #24199: Deprecate unused idlelib.idlever with possible removal in 3.6. - -- Issue #24782: In Idle extension config dialog, replace tabs with sorted list. - Patch by Mark Roseman. - -- Issue #24790: Remove extraneous code (which also create 2 & 3 conflicts). - Documentation ------------- -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 22 04:43:19 2015 From: python-checkins at python.org (terry.reedy) Date: Tue, 22 Sep 2015 02:43:19 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Issue_=2316893=3A_Add_idlelib=2Ehelp=2Ecopy=5Fstrip=28=29_to_c?= =?utf-8?q?opy-rstrip_Doc/=2E=2E=2E/idle=2Ehtml=2E?= Message-ID: <20150922024318.16569.1451@psf.io> https://hg.python.org/cpython/rev/f08437278049 changeset: 98162:f08437278049 branch: 3.5 parent: 98157:28583bc0680e parent: 98161:df987a0bc350 user: Terry Jan Reedy date: Mon Sep 21 22:38:47 2015 -0400 summary: Issue #16893: Add idlelib.help.copy_strip() to copy-rstrip Doc/.../idle.html. Change destination to help.html. Adjust NEWS entries. files: Lib/idlelib/NEWS.txt | 4 +- Lib/idlelib/help.html | 0 Lib/idlelib/help.py | 53 ++++++++++++++++++++---------- Misc/NEWS | 4 +- 4 files changed, 39 insertions(+), 22 deletions(-) diff --git a/Lib/idlelib/NEWS.txt b/Lib/idlelib/NEWS.txt --- a/Lib/idlelib/NEWS.txt +++ b/Lib/idlelib/NEWS.txt @@ -7,8 +7,8 @@ - Issue #25199: Idle: add synchronization comments for future maintainers. -- Issue #16893: Replace help.txt with idle.html for Idle doc display. - The new idlelib/idle.html is copied from Doc/build/html/library/idle.html. +- Issue #16893: Replace help.txt with help.html for Idle doc display. + The new idlelib/help.html is rstripped Doc/build/html/library/idle.html. It looks better than help.txt and will better document Idle as released. The tkinter html viewer that works for this file was written by Mark Roseman. The now unused EditorWindow.HelpDialog class and helt.txt file are deprecated. diff --git a/Lib/idlelib/idle.html b/Lib/idlelib/help.html rename from Lib/idlelib/idle.html rename to Lib/idlelib/help.html diff --git a/Lib/idlelib/help.py b/Lib/idlelib/help.py --- a/Lib/idlelib/help.py +++ b/Lib/idlelib/help.py @@ -1,23 +1,26 @@ -""" -help.py implements the Idle help menu and is subject to change. +""" help.py: Implement the Idle help menu. +Contents are subject to revision at any time, without notice. -The contents are subject to revision at any time, without notice. Help => About IDLE: diplay About Idle dialog -Help => IDLE Help: display idle.html with proper formatting -HelpParser - Parses idle.html generated from idle.rst by Sphinx -and renders to tk Text. +Help => IDLE Help: Display help.html with proper formatting. +Doc/library/idle.rst (Sphinx)=> Doc/build/html/library/idle.html +(help.copy_strip)=> Lib/idlelib/help.html -HelpText - Displays formatted idle.html. +HelpParser - Parse help.html and and render to tk Text. -HelpFrame - Contains text, scrollbar, and table-of-contents. +HelpText - Display formatted help.html. + +HelpFrame - Contain text, scrollbar, and table-of-contents. (This will be needed for display in a future tabbed window.) -HelpWindow - Display idleframe in a standalone window. +HelpWindow - Display HelpFrame in a standalone window. + +copy_strip - Copy idle.html to help.html, rstripping each line. show_idlehelp - Create HelpWindow. Called in EditorWindow.help_dialog. """ @@ -36,7 +39,7 @@ ## IDLE Help ## class HelpParser(HTMLParser): - """Render idle.html generated by Sphinx from idle.rst. + """Render help.html into a text widget. The overridden handle_xyz methods handle a subset of html tags. The supplied text should have the needed tag configurations. @@ -62,7 +65,7 @@ self.tags = '' if self.level == 0 else 'l'+str(self.level) def handle_starttag(self, tag, attrs): - "Handle starttags in idle.html." + "Handle starttags in help.html." class_ = '' for a, v in attrs: if a == 'class': @@ -120,7 +123,7 @@ self.text.insert('end', s, self.tags) def handle_endtag(self, tag): - "Handle endtags in idle.html." + "Handle endtags in help.html." if tag in ['h1', 'h2', 'h3', 'span', 'em']: self.indent(0) # clear tag, reset indent if self.show and tag in ['h1', 'h2', 'h3']: @@ -136,7 +139,7 @@ self.indent(amt=-1) def handle_data(self, data): - "Handle date segments in idle.html." + "Handle date segments in help.html." if self.show and not self.hdrlink: d = data if self.pre else data.replace('\n', ' ') if self.tags == 'h1': @@ -149,7 +152,7 @@ class HelpText(Text): - "Display idle.html." + "Display help.html." def __init__(self, parent, filename): "Configure tags and feed file to parser." Text.__init__(self, parent, wrap='word', highlightthickness=0, @@ -188,6 +191,7 @@ class HelpFrame(Frame): + "Display html text, scrollbar, and toc." def __init__(self, parent, filename): Frame.__init__(self, parent) text = HelpText(self, filename) @@ -202,6 +206,7 @@ toc.grid(column=0, row=0, sticky='nw') def contents_widget(self, text): + "Create table of contents." toc = Menubutton(self, text='TOC') drop = Menu(toc, tearoff=False) for tag, lbl in text.parser.contents: @@ -211,7 +216,7 @@ class HelpWindow(Toplevel): - + "Display frame with rendered html." def __init__(self, parent, filename, title): Toplevel.__init__(self, parent) self.wm_title(title) @@ -221,11 +226,23 @@ self.grid_rowconfigure(0, weight=1) +def copy_strip(): + "Copy idle.html to idlelib/help.html, stripping trailing whitespace." + src = join(abspath(dirname(dirname(dirname(__file__)))), + 'Doc', 'build', 'html', 'library', 'idle.html') + dst = join(abspath(dirname(__file__)), 'help.html') + with open(src, 'rb') as inn,\ + open(dst, 'wb') as out: + for line in inn: + out.write(line.rstrip() + '\n') + print('idle.html copied to help.html') + def show_idlehelp(parent): - filename = join(abspath(dirname(__file__)), 'idle.html') + "Create HelpWindow; called from Idle Help event handler." + filename = join(abspath(dirname(__file__)), 'help.html') if not isfile(filename): - dirpath = join(abspath(dirname(dirname(dirname(__file__)))), - 'Doc', 'build', 'html', 'library') + # try copy_strip, present message + return HelpWindow(parent, filename, 'IDLE Help') if __name__ == '__main__': diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -99,8 +99,8 @@ - Issue #25199: Idle: add synchronization comments for future maintainers. -- Issue #16893: Replace help.txt with idle.html for Idle doc display. - The new idlelib/idle.html is copied from Doc/build/html/library/idle.html. +- Issue #16893: Replace help.txt with help.html for Idle doc display. + The new idlelib/help.html is rstripped Doc/build/html/library/idle.html. It looks better than help.txt and will better document Idle as released. The tkinter html viewer that works for this file was written by Mark Roseman. The now unused EditorWindow.HelpDialog class and helt.txt file are deprecated. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 22 08:42:02 2015 From: python-checkins at python.org (raymond.hettinger) Date: Tue, 22 Sep 2015 06:42:02 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Minor_consistency_improvem?= =?utf-8?q?ents_to_negative_value_comparisons=2E?= Message-ID: <20150922064202.94109.13983@psf.io> https://hg.python.org/cpython/rev/0209061804da changeset: 98168:0209061804da user: Raymond Hettinger date: Mon Sep 21 23:41:56 2015 -0700 summary: Minor consistency improvements to negative value comparisons. files: Modules/_collectionsmodule.c | 18 +++++++++--------- 1 files changed, 9 insertions(+), 9 deletions(-) diff --git a/Modules/_collectionsmodule.c b/Modules/_collectionsmodule.c --- a/Modules/_collectionsmodule.c +++ b/Modules/_collectionsmodule.c @@ -209,7 +209,7 @@ Py_SIZE(deque)--; deque->state++; - if (deque->rightindex == -1) { + if (deque->rightindex < 0) { if (Py_SIZE(deque)) { prevblock = deque->rightblock->leftlink; assert(deque->leftblock != deque->rightblock); @@ -715,7 +715,7 @@ *(dest++) = *(src++); } while (--m); } - if (rightindex == -1) { + if (rightindex < 0) { assert(leftblock != rightblock); assert(b == NULL); b = rightblock; @@ -827,7 +827,7 @@ /* Step backwards with the right block/index pair */ rightindex--; - if (rightindex == -1) { + if (rightindex < 0) { rightblock = rightblock->leftlink; rightindex = BLOCKLEN - 1; } @@ -1234,7 +1234,7 @@ Py_DECREF(new_deque); return NULL; } - if (old_deque->maxlen == -1) + if (old_deque->maxlen < 0) return PyObject_CallFunction((PyObject *)(Py_TYPE(deque)), "O", deque, NULL); else return PyObject_CallFunction((PyObject *)(Py_TYPE(deque)), "Oi", @@ -1258,12 +1258,12 @@ return NULL; } if (dict == NULL) { - if (deque->maxlen == -1) + if (deque->maxlen < 0) result = Py_BuildValue("O(O)", Py_TYPE(deque), aslist); else result = Py_BuildValue("O(On)", Py_TYPE(deque), aslist, deque->maxlen); } else { - if (deque->maxlen == -1) + if (deque->maxlen < 0) result = Py_BuildValue("O(OO)O", Py_TYPE(deque), aslist, Py_None, dict); else result = Py_BuildValue("O(On)O", Py_TYPE(deque), aslist, deque->maxlen, dict); @@ -1354,7 +1354,7 @@ } Py_DECREF(x); Py_DECREF(y); - if (b == -1) + if (b < 0) goto done; } /* We reached the end of one deque or both */ @@ -1437,7 +1437,7 @@ static PyObject * deque_get_maxlen(dequeobject *deque) { - if (deque->maxlen == -1) + if (deque->maxlen < 0) Py_RETURN_NONE; return PyLong_FromSsize_t(deque->maxlen); } @@ -1778,7 +1778,7 @@ item = it->b->data[it->index]; it->index--; it->counter--; - if (it->index == -1 && it->counter > 0) { + if (it->index < 0 && it->counter > 0) { CHECK_NOT_END(it->b->leftlink); it->b = it->b->leftlink; it->index = BLOCKLEN - 1; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 22 10:20:44 2015 From: python-checkins at python.org (raymond.hettinger) Date: Tue, 22 Sep 2015 08:20:44 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Eliminate_unnecessary_vari?= =?utf-8?q?able?= Message-ID: <20150922082041.3654.47376@psf.io> https://hg.python.org/cpython/rev/2ec2b11ed157 changeset: 98169:2ec2b11ed157 user: Raymond Hettinger date: Tue Sep 22 01:20:36 2015 -0700 summary: Eliminate unnecessary variable files: Modules/_collectionsmodule.c | 3 +-- 1 files changed, 1 insertions(+), 2 deletions(-) diff --git a/Modules/_collectionsmodule.c b/Modules/_collectionsmodule.c --- a/Modules/_collectionsmodule.c +++ b/Modules/_collectionsmodule.c @@ -804,10 +804,9 @@ Py_ssize_t leftindex = deque->leftindex; Py_ssize_t rightindex = deque->rightindex; Py_ssize_t n = Py_SIZE(deque) >> 1; - Py_ssize_t i; PyObject *tmp; - for (i=0 ; i 0) { /* Validate that pointers haven't met in the middle */ assert(leftblock != rightblock || leftindex < rightindex); CHECK_NOT_END(leftblock); -- Repository URL: https://hg.python.org/cpython From solipsis at pitrou.net Tue Sep 22 10:42:05 2015 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Tue, 22 Sep 2015 08:42:05 +0000 Subject: [Python-checkins] Daily reference leaks (2c3f9b7b457d): sum=61923 Message-ID: <20150922084204.9945.60965@psf.io> results for 2c3f9b7b457d on branch "default" -------------------------------------------- test_capi leaked [5446, 5446, 5446] references, sum=16338 test_capi leaked [1433, 1435, 1435] memory blocks, sum=4303 test_functools leaked [0, 2, 2] memory blocks, sum=4 test_threading leaked [10892, 10892, 10892] references, sum=32676 test_threading leaked [2866, 2868, 2868] memory blocks, sum=8602 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/psf-users/antoine/refleaks/refloguiajv5', '--timeout', '7200'] From python-checkins at python.org Tue Sep 22 10:47:40 2015 From: python-checkins at python.org (victor.stinner) Date: Tue, 22 Sep 2015 08:47:40 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2324870=3A_revert_u?= =?utf-8?q?nwanted_change?= Message-ID: <20150922084723.82660.41447@psf.io> https://hg.python.org/cpython/rev/8317796ca004 changeset: 98170:8317796ca004 user: Victor Stinner date: Tue Sep 22 10:46:52 2015 +0200 summary: Issue #24870: revert unwanted change Sorry, I pushed the patch on the UTF-8 decoder by mistake :-( files: Objects/unicodeobject.c | 52 +++++----------------------- 1 files changed, 9 insertions(+), 43 deletions(-) diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c --- a/Objects/unicodeobject.c +++ b/Objects/unicodeobject.c @@ -4709,9 +4709,8 @@ Py_ssize_t startinpos; Py_ssize_t endinpos; const char *errmsg = ""; - PyObject *error_handler_obj = NULL; + PyObject *errorHandler = NULL; PyObject *exc = NULL; - _Py_error_handler error_handler = _Py_ERROR_UNKNOWN; if (size == 0) { if (consumed) @@ -4774,57 +4773,24 @@ continue; } - /* undecodable byte: call the error handler */ - - if (error_handler == _Py_ERROR_UNKNOWN) - error_handler = get_error_handler(errors); - - switch (error_handler) - { - case _Py_ERROR_REPLACE: - case _Py_ERROR_SURROGATEESCAPE: - { - unsigned char ch = (unsigned char)*s; - - /* Fast-path: the error handler only writes one character, - but we may switch to UCS2 at the first write */ - if (_PyUnicodeWriter_PrepareKind(&writer, PyUnicode_2BYTE_KIND) < 0) - goto onError; - kind = writer.kind; - - if (error_handler == _Py_ERROR_REPLACE) - PyUnicode_WRITE(kind, writer.data, writer.pos, 0xfffd); - else - PyUnicode_WRITE(kind, writer.data, writer.pos, ch + 0xdc00); - writer.pos++; - ++s; - break; - } - - case _Py_ERROR_IGNORE: - s++; - break; - - default: - if (unicode_decode_call_errorhandler_writer( - errors, &error_handler_obj, - "utf-8", errmsg, - &starts, &end, &startinpos, &endinpos, &exc, &s, - &writer)) - goto onError; - } + if (unicode_decode_call_errorhandler_writer( + errors, &errorHandler, + "utf-8", errmsg, + &starts, &end, &startinpos, &endinpos, &exc, &s, + &writer)) + goto onError; } End: if (consumed) *consumed = s - starts; - Py_XDECREF(error_handler_obj); + Py_XDECREF(errorHandler); Py_XDECREF(exc); return _PyUnicodeWriter_Finish(&writer); onError: - Py_XDECREF(error_handler_obj); + Py_XDECREF(errorHandler); Py_XDECREF(exc); _PyUnicodeWriter_Dealloc(&writer); return NULL; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 22 12:18:56 2015 From: python-checkins at python.org (berker.peksag) Date: Tue, 22 Sep 2015 10:18:56 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2325137=3A_Add_a_note_to_whatsnew/3=2E5=2Erst_for?= =?utf-8?q?_nested_functools=2Epartial_calls?= Message-ID: <20150922100848.31197.92336@psf.io> https://hg.python.org/cpython/rev/ed694938c61a changeset: 98172:ed694938c61a parent: 98170:8317796ca004 parent: 98171:fa766b6f12b5 user: Berker Peksag date: Tue Sep 22 13:08:42 2015 +0300 summary: Issue #25137: Add a note to whatsnew/3.5.rst for nested functools.partial calls Also, properly skip the test_nested_optimization test for partial subclasses and add a test for the suggested usage. files: Doc/whatsnew/3.5.rst | 6 ++++++ Lib/test/test_functools.py | 18 +++++++++++++++--- 2 files changed, 21 insertions(+), 3 deletions(-) diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -2441,6 +2441,12 @@ module and the :func:`help` function. (Contributed by Serhiy Storchaka in :issue:`15582`.) +* Nested :func:`functools.partial` calls are now flattened. If you were + relying on the previous behavior, you can now either add an attribute to a + :func:`functools.partial` object or you can create a subclass of + :func:`functools.partial`. + (Contributed by Alexander Belopolsky in :issue:`7830`.) + Changes in the C API -------------------- diff --git a/Lib/test/test_functools.py b/Lib/test/test_functools.py --- a/Lib/test/test_functools.py +++ b/Lib/test/test_functools.py @@ -139,14 +139,23 @@ def test_nested_optimization(self): partial = self.partial - # Only "true" partial is optimized - if partial.__name__ != 'partial': - return inner = partial(signature, 'asdf') nested = partial(inner, bar=True) flat = partial(signature, 'asdf', bar=True) self.assertEqual(signature(nested), signature(flat)) + def test_nested_partial_with_attribute(self): + # see issue 25137 + partial = self.partial + + def foo(bar): + return bar + + p = partial(foo, 'first') + p2 = partial(p, 'second') + p2.new_attr = 'spam' + self.assertEqual(p2.new_attr, 'spam') + @unittest.skipUnless(c_functools, 'requires the C _functools module') class TestPartialC(TestPartial, unittest.TestCase): @@ -238,6 +247,9 @@ if c_functools: partial = PartialSubclass + # partial subclasses are not optimized for nested calls + test_nested_optimization = None + class TestPartialMethod(unittest.TestCase): -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 22 12:24:40 2015 From: python-checkins at python.org (berker.peksag) Date: Tue, 22 Sep 2015 10:24:40 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI1MTM3?= =?utf-8?q?=3A_Add_a_note_to_whatsnew/3=2E5=2Erst_for_nested_functools=2Ep?= =?utf-8?q?artial_calls?= Message-ID: <20150922100848.98364.23056@psf.io> https://hg.python.org/cpython/rev/fa766b6f12b5 changeset: 98171:fa766b6f12b5 branch: 3.5 parent: 98166:9a66286b970c user: Berker Peksag date: Tue Sep 22 13:08:16 2015 +0300 summary: Issue #25137: Add a note to whatsnew/3.5.rst for nested functools.partial calls Also, properly skip the test_nested_optimization test for partial subclasses and add a test for the suggested usage. files: Doc/whatsnew/3.5.rst | 6 ++++++ Lib/test/test_functools.py | 18 +++++++++++++++--- 2 files changed, 21 insertions(+), 3 deletions(-) diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -2441,6 +2441,12 @@ module and the :func:`help` function. (Contributed by Serhiy Storchaka in :issue:`15582`.) +* Nested :func:`functools.partial` calls are now flattened. If you were + relying on the previous behavior, you can now either add an attribute to a + :func:`functools.partial` object or you can create a subclass of + :func:`functools.partial`. + (Contributed by Alexander Belopolsky in :issue:`7830`.) + Changes in the C API -------------------- diff --git a/Lib/test/test_functools.py b/Lib/test/test_functools.py --- a/Lib/test/test_functools.py +++ b/Lib/test/test_functools.py @@ -139,14 +139,23 @@ def test_nested_optimization(self): partial = self.partial - # Only "true" partial is optimized - if partial.__name__ != 'partial': - return inner = partial(signature, 'asdf') nested = partial(inner, bar=True) flat = partial(signature, 'asdf', bar=True) self.assertEqual(signature(nested), signature(flat)) + def test_nested_partial_with_attribute(self): + # see issue 25137 + partial = self.partial + + def foo(bar): + return bar + + p = partial(foo, 'first') + p2 = partial(p, 'second') + p2.new_attr = 'spam' + self.assertEqual(p2.new_attr, 'spam') + @unittest.skipUnless(c_functools, 'requires the C _functools module') class TestPartialC(TestPartial, unittest.TestCase): @@ -238,6 +247,9 @@ if c_functools: partial = PartialSubclass + # partial subclasses are not optimized for nested calls + test_nested_optimization = None + class TestPartialMethod(unittest.TestCase): -- Repository URL: https://hg.python.org/cpython From lp_benchmark_robot at intel.com Tue Sep 22 15:17:35 2015 From: lp_benchmark_robot at intel.com (lp_benchmark_robot) Date: Tue, 22 Sep 2015 13:17:35 +0000 Subject: [Python-checkins] Benchmark Results for Python Default 2015-09-22 Message-ID: Results for project python_default-nightly, build date 2015-09-22 03:02:31 commit: 2c3f9b7b457d003dafa44d0e95c34eea8b1416e8 revision date: 2015-09-22 02:42:55 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v3.4.3, with hash b4cbecbc0781e89a309d03b60a1f75f8499250e6 from 2015-02-25 12:15:33+00:00 ------------------------------------------------------------------------------------------ benchmark relative change since change since current rev with std_dev* last run v3.4.3 regrtest PGO ------------------------------------------------------------------------------------------ :-) django_v2 0.32124% -1.56411% 8.05177% 16.77520% :-( pybench 0.19146% -0.68342% -2.63438% 9.44362% :-( regex_v8 2.88003% -0.04566% -5.13704% 7.16398% :-| nbody 0.18329% 0.52519% -1.08720% 9.20507% :-( json_dump_v2 0.34187% 0.24384% -2.41094% 10.38769% :-| normal_startup 0.74327% -0.42928% -0.11989% 5.74466% ------------------------------------------------------------------------------------------ Note: Benchmark results are measured in seconds. * Relative Standard Deviation (Standard Deviation/Average) Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. From lp_benchmark_robot at intel.com Tue Sep 22 15:17:34 2015 From: lp_benchmark_robot at intel.com (lp_benchmark_robot) Date: Tue, 22 Sep 2015 13:17:34 +0000 Subject: [Python-checkins] Benchmark Results for Python 2.7 2015-09-22 Message-ID: Results for project python_2.7-nightly, build date 2015-09-22 03:43:23 commit: 9d77345adcc998360815b77942645e491e0ae268 revision date: 2015-09-22 02:42:17 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v2.7.10, with hash 15c95b7d81dcf821daade360741e00714667653f from 2015-05-23 16:02:14+00:00 ------------------------------------------------------------------------------------------ benchmark relative change since change since current rev with std_dev* last run v2.7.10 regrtest PGO ------------------------------------------------------------------------------------------ :-) django_v2 0.15162% 0.35735% 4.87540% 9.21415% :-) pybench 0.15863% 0.10056% 6.81025% 7.30794% :-| regex_v8 0.60278% 0.00711% -1.19203% 7.00860% :-) nbody 0.13649% 0.02179% 9.05431% 5.84603% :-) json_dump_v2 0.30262% 0.39097% 4.52526% 13.34282% :-| normal_startup 1.88948% -0.12240% -1.83868% 3.38542% :-| ssbench 0.96389% -0.60694% 1.81597% 2.23808% ------------------------------------------------------------------------------------------ Note: Benchmark results for ssbench are measured in requests/second while all other are measured in seconds. * Relative Standard Deviation (Standard Deviation/Average) Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. From python-checkins at python.org Tue Sep 22 17:27:50 2015 From: python-checkins at python.org (eric.smith) Date: Tue, 22 Sep 2015 15:27:50 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_More_on_issue_25206=3A_typo?= =?utf-8?q?=2E?= Message-ID: <20150922152748.31185.2535@psf.io> https://hg.python.org/peps/rev/df19ba9217e4 changeset: 6097:df19ba9217e4 user: Eric V. Smith date: Tue Sep 22 11:27:43 2015 -0400 summary: More on issue 25206: typo. files: pep-0498.txt | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/pep-0498.txt b/pep-0498.txt --- a/pep-0498.txt +++ b/pep-0498.txt @@ -294,7 +294,7 @@ For example, this code:: - f'abc{expr1:spec1}{expr2!r:spec2}def{expr3:!s}ghi' + f'abc{expr1:spec1}{expr2!r:spec2}def{expr3!s}ghi' Might be be evaluated as:: -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Tue Sep 22 23:57:34 2015 From: python-checkins at python.org (eric.smith) Date: Tue, 22 Sep 2015 21:57:34 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_Simplified_example=2E?= Message-ID: <20150922215733.31193.57891@psf.io> https://hg.python.org/peps/rev/f9313604d92a changeset: 6098:f9313604d92a user: Eric V. Smith date: Tue Sep 22 17:57:36 2015 -0400 summary: Simplified example. files: pep-0498.txt | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/pep-0498.txt b/pep-0498.txt --- a/pep-0498.txt +++ b/pep-0498.txt @@ -294,11 +294,11 @@ For example, this code:: - f'abc{expr1:spec1}{expr2!r:spec2}def{expr3!s}ghi' + f'abc{expr1:spec1}{expr2!r:spec2}def{expr3}ghi' Might be be evaluated as:: - 'abc' + format(expr1, spec1) + format(repr(expr2), spec2) + 'def' + format(str(expr3)) + 'ghi' + 'abc' + format(expr1, spec1) + format(repr(expr2), spec2) + 'def' + format(expr3) + 'ghi' Expression evaluation --------------------- -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Wed Sep 23 02:01:47 2015 From: python-checkins at python.org (steve.dower) Date: Wed, 23 Sep 2015 00:01:47 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI1MTI2?= =?utf-8?q?=3A_Clarifies_that_the_non-web_installer_will_download_some?= Message-ID: <20150923000147.115366.91036@psf.io> https://hg.python.org/cpython/rev/4e98c622ab20 changeset: 98177:4e98c622ab20 branch: 3.5 user: Steve Dower date: Tue Sep 22 16:36:33 2015 -0700 summary: Issue #25126: Clarifies that the non-web installer will download some components. files: Misc/NEWS | 3 +++ Tools/msi/bundle/Default.wxl | 4 ++-- 2 files changed, 5 insertions(+), 2 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -152,6 +152,9 @@ - Issue #25091: Increases font size of the installer. +- Issue #25126: Clarifies that the non-web installer will download some + components. + - Issue #25213: Restores requestedExecutionLevel to manifest to disable UAC virtualization. diff --git a/Tools/msi/bundle/Default.wxl b/Tools/msi/bundle/Default.wxl --- a/Tools/msi/bundle/Default.wxl +++ b/Tools/msi/bundle/Default.wxl @@ -86,8 +86,8 @@ for &all users (requires elevation) Install &launcher for all users (recommended) &Precompile standard library - Install debugging &symbols - Install debu&g binaries (requires VS 2015 or later) + Download debugging &symbols + Download debu&g binaries (requires VS 2015 or later) [ActionLikeInstallation] Progress [ActionLikeInstalling]: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 23 02:01:48 2015 From: python-checkins at python.org (steve.dower) Date: Wed, 23 Sep 2015 00:01:48 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI1MTAy?= =?utf-8?q?=3A_Windows_installer_does_not_precompile_for_-O_or_-OO=2E?= Message-ID: <20150923000147.3654.99695@psf.io> https://hg.python.org/cpython/rev/31b230e5517e changeset: 98179:31b230e5517e branch: 3.5 user: Steve Dower date: Tue Sep 22 16:45:19 2015 -0700 summary: Issue #25102: Windows installer does not precompile for -O or -OO. files: Misc/NEWS | 2 + Tools/msi/bundle/bundle.wxl | 2 + Tools/msi/bundle/packagegroups/postinstall.wxs | 51 +++++++++- 3 files changed, 50 insertions(+), 5 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -150,6 +150,8 @@ Windows ------- +- Issue #25102: Windows installer does not precompile for -O or -OO. + - Issue #25081: Makes Back button in installer go back to upgrade page when upgrading. diff --git a/Tools/msi/bundle/bundle.wxl b/Tools/msi/bundle/bundle.wxl --- a/Tools/msi/bundle/bundle.wxl +++ b/Tools/msi/bundle/bundle.wxl @@ -2,4 +2,6 @@ C Runtime Update (KB2999226) Precompiling standard library + Precompiling standard library (-O) + Precompiling standard library (-OO) diff --git a/Tools/msi/bundle/packagegroups/postinstall.wxs b/Tools/msi/bundle/packagegroups/postinstall.wxs --- a/Tools/msi/bundle/packagegroups/postinstall.wxs +++ b/Tools/msi/bundle/packagegroups/postinstall.wxs @@ -40,23 +40,64 @@ - + + + + + + https://hg.python.org/cpython/rev/94ea3e05817f changeset: 98178:94ea3e05817f branch: 3.5 user: Steve Dower date: Tue Sep 22 16:36:33 2015 -0700 summary: Issue #25081: Makes Back button in installer go back to upgrade page when upgrading. files: Misc/NEWS | 3 +++ Tools/msi/bundle/bootstrap/PythonBootstrapperApplication.cpp | 5 +++++ 2 files changed, 8 insertions(+), 0 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -150,6 +150,9 @@ Windows ------- +- Issue #25081: Makes Back button in installer go back to upgrade page when + upgrading. + - Issue #25091: Increases font size of the installer. - Issue #25126: Clarifies that the non-web installer will download some diff --git a/Tools/msi/bundle/bootstrap/PythonBootstrapperApplication.cpp b/Tools/msi/bundle/bootstrap/PythonBootstrapperApplication.cpp --- a/Tools/msi/bundle/bootstrap/PythonBootstrapperApplication.cpp +++ b/Tools/msi/bundle/bootstrap/PythonBootstrapperApplication.cpp @@ -323,6 +323,8 @@ SavePageSettings(); if (_modifying) { GoToPage(PAGE_MODIFY); + } else if (_upgrading) { + GoToPage(PAGE_UPGRADE); } else { GoToPage(PAGE_INSTALL); } @@ -2524,6 +2526,7 @@ case BOOTSTRAPPER_ACTION_INSTALL: if (_upgradingOldVersion) { _installPage = PAGE_UPGRADE; + _upgrading = TRUE; } else if (SUCCEEDED(BalGetNumericVariable(L"SimpleInstall", &simple)) && simple) { _installPage = PAGE_SIMPLE_INSTALL; } else { @@ -3029,6 +3032,7 @@ _suppressDowngradeFailure = FALSE; _suppressRepair = FALSE; _modifying = FALSE; + _upgrading = FALSE; _overridableVariables = nullptr; _taskbarList = nullptr; @@ -3113,6 +3117,7 @@ BOOL _suppressDowngradeFailure; BOOL _suppressRepair; BOOL _modifying; + BOOL _upgrading; int _crtInstalledToken; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 23 02:01:57 2015 From: python-checkins at python.org (steve.dower) Date: Wed, 23 Sep 2015 00:01:57 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI1MjEz?= =?utf-8?q?=3A_Restores_requestedExecutionLevel_to_manifest_to_disable_UAC?= Message-ID: <20150923000146.82652.55604@psf.io> https://hg.python.org/cpython/rev/b7f0f1d1e923 changeset: 98173:b7f0f1d1e923 branch: 3.5 parent: 98171:fa766b6f12b5 user: Steve Dower date: Tue Sep 22 14:33:31 2015 -0700 summary: Issue #25213: Restores requestedExecutionLevel to manifest to disable UAC virtualization. files: Misc/NEWS | 3 +++ PC/python.manifest | 7 +++++++ 2 files changed, 10 insertions(+), 0 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -147,6 +147,9 @@ Windows ------- +- Issue #25213: Restores requestedExecutionLevel to manifest to disable + UAC virtualization. + - Issue #25022: Removed very outdated PC/example_nt/ directory. What's New in Python 3.5.0 final? diff --git a/PC/python.manifest b/PC/python.manifest --- a/PC/python.manifest +++ b/PC/python.manifest @@ -1,5 +1,12 @@ + + + + + + + -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 23 02:01:58 2015 From: python-checkins at python.org (steve.dower) Date: Wed, 23 Sep 2015 00:01:58 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge_with_3=2E5?= Message-ID: <20150923000148.82662.28581@psf.io> https://hg.python.org/cpython/rev/ff1c6e9eeed4 changeset: 98180:ff1c6e9eeed4 parent: 98172:ed694938c61a parent: 98179:31b230e5517e user: Steve Dower date: Tue Sep 22 17:01:17 2015 -0700 summary: Merge with 3.5 files: Misc/NEWS | 24 + Modules/timemodule.c | 3 + PC/python.manifest | 7 + Tools/msi/bundle/Default.thm | 156 +++++----- Tools/msi/bundle/Default.wxl | 4 +- Tools/msi/bundle/SideBar.png | Bin Tools/msi/bundle/bootstrap/PythonBootstrapperApplication.cpp | 5 + Tools/msi/bundle/bundle.wxl | 2 + Tools/msi/bundle/packagegroups/postinstall.wxs | 51 ++- Tools/msi/make_zip.proj | 2 + Tools/msi/make_zip.py | 23 +- 11 files changed, 190 insertions(+), 87 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -138,6 +138,17 @@ Library ------- +- Issue #25092: Fix datetime.strftime() failure when errno was already set to + EINVAL. + +- Issue #23517: Fix rounding in fromtimestamp() and utcfromtimestamp() methods + of datetime.datetime: microseconds are now rounded to nearest with ties + going to nearest even integer (ROUND_HALF_EVEN), instead of being rounding + towards minus infinity (ROUND_FLOOR). It's important that these methods use + the same rounding mode than datetime.timedelta to keep the property: + (datetime(1970,1,1) + timedelta(seconds=t)) == datetime.utcfromtimestamp(t). + It also the rounding mode used by round(float) for example. + - Issue #25155: Fix datetime.datetime.now() and datetime.datetime.utcnow() on Windows to support date after year 2038. It was a regression introduced in Python 3.5.0. @@ -239,6 +250,19 @@ Windows ------- +- Issue #25102: Windows installer does not precompile for -O or -OO. + +- Issue #25081: Makes Back button in installer go back to upgrade page when + upgrading. + +- Issue #25091: Increases font size of the installer. + +- Issue #25126: Clarifies that the non-web installer will download some + components. + +- Issue #25213: Restores requestedExecutionLevel to manifest to disable + UAC virtualization. + - Issue #25022: Removed very outdated PC/example_nt/ directory. diff --git a/Modules/timemodule.c b/Modules/timemodule.c --- a/Modules/timemodule.c +++ b/Modules/timemodule.c @@ -653,6 +653,9 @@ PyErr_NoMemory(); break; } +#if defined _MSC_VER && _MSC_VER >= 1400 && defined(__STDC_SECURE_LIB__) + errno = 0; +#endif _Py_BEGIN_SUPPRESS_IPH buflen = format_time(outbuf, i, fmt, &buf); _Py_END_SUPPRESS_IPH diff --git a/PC/python.manifest b/PC/python.manifest --- a/PC/python.manifest +++ b/PC/python.manifest @@ -1,5 +1,12 @@ + + + + + + + diff --git a/Tools/msi/bundle/Default.thm b/Tools/msi/bundle/Default.thm --- a/Tools/msi/bundle/Default.thm +++ b/Tools/msi/bundle/Default.thm @@ -1,136 +1,136 @@ - #(loc.Caption) - Segoe UI - Segoe UI - Segoe UI - Segoe UI - Segoe UI - Segoe UI + #(loc.Caption) + Segoe UI + Segoe UI + Segoe UI + Segoe UI + Segoe UI + Segoe UI - #(loc.HelpHeader) - + #(loc.HelpHeader) + #(loc.HelpText) - + - #(loc.InstallHeader) - + #(loc.InstallHeader) + #(loc.InstallMessage) - - + + - #(loc.ShortPrependPathLabel) - #(loc.ShortInstallLauncherAllUsersLabel) + #(loc.ShortPrependPathLabel) + #(loc.ShortInstallLauncherAllUsersLabel) - + - #(loc.InstallUpgradeHeader) - + #(loc.InstallUpgradeHeader) + #(loc.InstallUpgradeMessage) - - + + - + - #(loc.InstallHeader) - + #(loc.InstallHeader) + - + - + - #(loc.Custom1Header) - + #(loc.Custom1Header) + - #(loc.Include_docLabel) - #(loc.Include_docHelpLabel) + #(loc.Include_docLabel) + #(loc.Include_docHelpLabel) - #(loc.Include_pipLabel) - #(loc.Include_pipHelpLabel) + #(loc.Include_pipLabel) + #(loc.Include_pipHelpLabel) - #(loc.Include_tcltkLabel) - #(loc.Include_tcltkHelpLabel) + #(loc.Include_tcltkLabel) + #(loc.Include_tcltkHelpLabel) - #(loc.Include_testLabel) - #(loc.Include_testHelpLabel) + #(loc.Include_testLabel) + #(loc.Include_testHelpLabel) - #(loc.Include_launcherLabel) - #(loc.InstallLauncherAllUsersLabel) - #(loc.Include_launcherHelpLabel) + #(loc.Include_launcherLabel) + #(loc.InstallLauncherAllUsersLabel) + #(loc.Include_launcherHelpLabel) - - - + + + - #(loc.Custom2Header) - + #(loc.Custom2Header) + - #(loc.InstallAllUsersLabel) - #(loc.AssociateFilesLabel) - #(loc.ShortcutsLabel) - #(loc.PrependPathLabel) - #(loc.PrecompileLabel) - #(loc.Include_symbolsLabel) - #(loc.Include_debugLabel) + #(loc.InstallAllUsersLabel) + #(loc.AssociateFilesLabel) + #(loc.ShortcutsLabel) + #(loc.PrependPathLabel) + #(loc.PrecompileLabel) + #(loc.Include_symbolsLabel) + #(loc.Include_debugLabel) - #(loc.CustomLocationLabel) - - - #(loc.CustomLocationHelpLabel) + #(loc.CustomLocationLabel) + + + #(loc.CustomLocationHelpLabel) - - - + + + - #(loc.ProgressHeader) - + #(loc.ProgressHeader) + - #(loc.ProgressLabel) - #(loc.OverallProgressPackageText) - - + #(loc.ProgressLabel) + #(loc.OverallProgressPackageText) + + - #(loc.ModifyHeader) - + #(loc.ModifyHeader) + - - - + + + - + - #(loc.SuccessHeader) - + #(loc.SuccessHeader) + #(loc.SuccessRestartText) - - + + - #(loc.FailureHeader) - + #(loc.FailureHeader) + #(loc.FailureHyperlinkLogText) #(loc.FailureRestartText) - - + + \ No newline at end of file diff --git a/Tools/msi/bundle/Default.wxl b/Tools/msi/bundle/Default.wxl --- a/Tools/msi/bundle/Default.wxl +++ b/Tools/msi/bundle/Default.wxl @@ -86,8 +86,8 @@ for &all users (requires elevation) Install &launcher for all users (recommended) &Precompile standard library - Install debugging &symbols - Install debu&g binaries (requires VS 2015 or later) + Download debugging &symbols + Download debu&g binaries (requires VS 2015 or later) [ActionLikeInstallation] Progress [ActionLikeInstalling]: diff --git a/Tools/msi/bundle/SideBar.png b/Tools/msi/bundle/SideBar.png index 9c18fff33a6ea50e3514ea938c5e5bfe4c5fe4c6..a23ce5e145848836c6ee21cc0cb4603263630455 GIT binary patch [stripped] diff --git a/Tools/msi/bundle/bootstrap/PythonBootstrapperApplication.cpp b/Tools/msi/bundle/bootstrap/PythonBootstrapperApplication.cpp --- a/Tools/msi/bundle/bootstrap/PythonBootstrapperApplication.cpp +++ b/Tools/msi/bundle/bootstrap/PythonBootstrapperApplication.cpp @@ -323,6 +323,8 @@ SavePageSettings(); if (_modifying) { GoToPage(PAGE_MODIFY); + } else if (_upgrading) { + GoToPage(PAGE_UPGRADE); } else { GoToPage(PAGE_INSTALL); } @@ -2524,6 +2526,7 @@ case BOOTSTRAPPER_ACTION_INSTALL: if (_upgradingOldVersion) { _installPage = PAGE_UPGRADE; + _upgrading = TRUE; } else if (SUCCEEDED(BalGetNumericVariable(L"SimpleInstall", &simple)) && simple) { _installPage = PAGE_SIMPLE_INSTALL; } else { @@ -3029,6 +3032,7 @@ _suppressDowngradeFailure = FALSE; _suppressRepair = FALSE; _modifying = FALSE; + _upgrading = FALSE; _overridableVariables = nullptr; _taskbarList = nullptr; @@ -3113,6 +3117,7 @@ BOOL _suppressDowngradeFailure; BOOL _suppressRepair; BOOL _modifying; + BOOL _upgrading; int _crtInstalledToken; diff --git a/Tools/msi/bundle/bundle.wxl b/Tools/msi/bundle/bundle.wxl --- a/Tools/msi/bundle/bundle.wxl +++ b/Tools/msi/bundle/bundle.wxl @@ -2,4 +2,6 @@ C Runtime Update (KB2999226) Precompiling standard library + Precompiling standard library (-O) + Precompiling standard library (-OO) diff --git a/Tools/msi/bundle/packagegroups/postinstall.wxs b/Tools/msi/bundle/packagegroups/postinstall.wxs --- a/Tools/msi/bundle/packagegroups/postinstall.wxs +++ b/Tools/msi/bundle/packagegroups/postinstall.wxs @@ -40,23 +40,64 @@ - + + + + + + python-$(PythonVersion)-embed-$(ArchName) .zip $(OutputPath)\en-us\$(TargetName)$(TargetExt) + rmdir /q/s "$(IntermediateOutputPath)\zip_$(ArchName)" "$(PythonExe)" "$(MSBuildThisFileDirectory)\make_zip.py" $(Arguments) -e -o "$(TargetPath)" -t "$(IntermediateOutputPath)\zip_$(ArchName)" -a $(ArchName) set DOC_FILENAME=python$(PythonVersion).chm @@ -23,6 +24,7 @@ diff --git a/Tools/msi/make_zip.py b/Tools/msi/make_zip.py --- a/Tools/msi/make_zip.py +++ b/Tools/msi/make_zip.py @@ -15,6 +15,20 @@ DEBUG_RE = re.compile(r'_d\.(pyd|dll|exe)$', re.IGNORECASE) PYTHON_DLL_RE = re.compile(r'python\d\d?\.dll$', re.IGNORECASE) +EXCLUDE_FROM_LIBRARY = { + '__pycache__', + 'ensurepip', + 'idlelib', + 'pydoc_data', + 'site-packages', + 'tkinter', + 'turtledemo', +} + +EXCLUDE_FILE_FROM_LIBRARY = { + 'bdist_wininst.py', +} + def is_not_debug(p): if DEBUG_RE.search(p.name): return False @@ -37,16 +51,21 @@ def include_in_lib(p): name = p.name.lower() if p.is_dir(): - if name in {'__pycache__', 'ensurepip', 'idlelib', 'pydoc_data', 'tkinter', 'turtledemo'}: + if name in EXCLUDE_FROM_LIBRARY: return False if name.startswith('plat-'): return False if name == 'test' and p.parts[-2].lower() == 'lib': return False + if name in {'test', 'tests'} and p.parts[-3].lower() == 'lib': + return False return True + if name in EXCLUDE_FILE_FROM_LIBRARY: + return False + suffix = p.suffix.lower() - return suffix not in {'.pyc', '.pyo'} + return suffix not in {'.pyc', '.pyo', '.exe'} def include_in_tools(p): if p.is_dir() and p.name.lower() in {'scripts', 'i18n', 'pynche', 'demo', 'parser'}: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 23 02:01:57 2015 From: python-checkins at python.org (steve.dower) Date: Wed, 23 Sep 2015 00:01:57 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI1MDky?= =?utf-8?q?=3A_Fix_datetime=2Estrftime=28=29_failure_when_errno_was_alread?= =?utf-8?q?y_set_to?= Message-ID: <20150923000147.82648.49738@psf.io> https://hg.python.org/cpython/rev/aa6b9205c120 changeset: 98174:aa6b9205c120 branch: 3.5 user: Steve Dower date: Tue Sep 22 14:51:42 2015 -0700 summary: Issue #25092: Fix datetime.strftime() failure when errno was already set to EINVAL. files: Misc/NEWS | 3 +++ Modules/timemodule.c | 3 +++ 2 files changed, 6 insertions(+), 0 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -18,6 +18,9 @@ Library ------- +- Issue #25092: Fix datetime.strftime() failure when errno was already set to + EINVAL. + - Issue #23517: Fix rounding in fromtimestamp() and utcfromtimestamp() methods of datetime.datetime: microseconds are now rounded to nearest with ties going to nearest even integer (ROUND_HALF_EVEN), instead of being rounding diff --git a/Modules/timemodule.c b/Modules/timemodule.c --- a/Modules/timemodule.c +++ b/Modules/timemodule.c @@ -653,6 +653,9 @@ PyErr_NoMemory(); break; } +#if defined _MSC_VER && _MSC_VER >= 1400 && defined(__STDC_SECURE_LIB__) + errno = 0; +#endif _Py_BEGIN_SUPPRESS_IPH buflen = format_time(outbuf, i, fmt, &buf); _Py_END_SUPPRESS_IPH -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 23 02:02:09 2015 From: python-checkins at python.org (steve.dower) Date: Wed, 23 Sep 2015 00:02:09 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogQ2xvc2VzICMyNTA4?= =?utf-8?q?5_and_=2325086=3A_Exclude_distutils_and_test_directories_from?= Message-ID: <20150923000147.11690.12156@psf.io> https://hg.python.org/cpython/rev/812e30f67d6e changeset: 98175:812e30f67d6e branch: 3.5 user: Steve Dower date: Tue Sep 22 15:03:54 2015 -0700 summary: Closes #25085 and #25086: Exclude distutils and test directories from embeddable distro. files: Tools/msi/make_zip.proj | 2 ++ Tools/msi/make_zip.py | 23 +++++++++++++++++++++-- 2 files changed, 23 insertions(+), 2 deletions(-) diff --git a/Tools/msi/make_zip.proj b/Tools/msi/make_zip.proj --- a/Tools/msi/make_zip.proj +++ b/Tools/msi/make_zip.proj @@ -14,6 +14,7 @@ python-$(PythonVersion)-embed-$(ArchName) .zip $(OutputPath)\en-us\$(TargetName)$(TargetExt) + rmdir /q/s "$(IntermediateOutputPath)\zip_$(ArchName)" "$(PythonExe)" "$(MSBuildThisFileDirectory)\make_zip.py" $(Arguments) -e -o "$(TargetPath)" -t "$(IntermediateOutputPath)\zip_$(ArchName)" -a $(ArchName) set DOC_FILENAME=python$(PythonVersion).chm @@ -23,6 +24,7 @@ diff --git a/Tools/msi/make_zip.py b/Tools/msi/make_zip.py --- a/Tools/msi/make_zip.py +++ b/Tools/msi/make_zip.py @@ -15,6 +15,20 @@ DEBUG_RE = re.compile(r'_d\.(pyd|dll|exe)$', re.IGNORECASE) PYTHON_DLL_RE = re.compile(r'python\d\d?\.dll$', re.IGNORECASE) +EXCLUDE_FROM_LIBRARY = { + '__pycache__', + 'ensurepip', + 'idlelib', + 'pydoc_data', + 'site-packages', + 'tkinter', + 'turtledemo', +} + +EXCLUDE_FILE_FROM_LIBRARY = { + 'bdist_wininst.py', +} + def is_not_debug(p): if DEBUG_RE.search(p.name): return False @@ -37,16 +51,21 @@ def include_in_lib(p): name = p.name.lower() if p.is_dir(): - if name in {'__pycache__', 'ensurepip', 'idlelib', 'pydoc_data', 'tkinter', 'turtledemo'}: + if name in EXCLUDE_FROM_LIBRARY: return False if name.startswith('plat-'): return False if name == 'test' and p.parts[-2].lower() == 'lib': return False + if name in {'test', 'tests'} and p.parts[-3].lower() == 'lib': + return False return True + if name in EXCLUDE_FILE_FROM_LIBRARY: + return False + suffix = p.suffix.lower() - return suffix not in {'.pyc', '.pyo'} + return suffix not in {'.pyc', '.pyo', '.exe'} def include_in_tools(p): if p.is_dir() and p.name.lower() in {'scripts', 'i18n', 'pynche', 'demo', 'parser'}: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 23 02:02:09 2015 From: python-checkins at python.org (steve.dower) Date: Wed, 23 Sep 2015 00:02:09 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI1MDkx?= =?utf-8?q?=3A_Increases_font_size_of_the_installer=2E?= Message-ID: <20150923000147.9927.57387@psf.io> https://hg.python.org/cpython/rev/07a3d804c6ea changeset: 98176:07a3d804c6ea branch: 3.5 user: Steve Dower date: Tue Sep 22 16:36:29 2015 -0700 summary: Issue #25091: Increases font size of the installer. files: Misc/NEWS | 2 + Tools/msi/bundle/Default.thm | 156 +++++++++++----------- Tools/msi/bundle/SideBar.png | Bin 3 files changed, 80 insertions(+), 78 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -150,6 +150,8 @@ Windows ------- +- Issue #25091: Increases font size of the installer. + - Issue #25213: Restores requestedExecutionLevel to manifest to disable UAC virtualization. diff --git a/Tools/msi/bundle/Default.thm b/Tools/msi/bundle/Default.thm --- a/Tools/msi/bundle/Default.thm +++ b/Tools/msi/bundle/Default.thm @@ -1,136 +1,136 @@ - #(loc.Caption) - Segoe UI - Segoe UI - Segoe UI - Segoe UI - Segoe UI - Segoe UI + #(loc.Caption) + Segoe UI + Segoe UI + Segoe UI + Segoe UI + Segoe UI + Segoe UI - #(loc.HelpHeader) - + #(loc.HelpHeader) + #(loc.HelpText) - + - #(loc.InstallHeader) - + #(loc.InstallHeader) + #(loc.InstallMessage) - - + + - #(loc.ShortPrependPathLabel) - #(loc.ShortInstallLauncherAllUsersLabel) + #(loc.ShortPrependPathLabel) + #(loc.ShortInstallLauncherAllUsersLabel) - + - #(loc.InstallUpgradeHeader) - + #(loc.InstallUpgradeHeader) + #(loc.InstallUpgradeMessage) - - + + - + - #(loc.InstallHeader) - + #(loc.InstallHeader) + - + - + - #(loc.Custom1Header) - + #(loc.Custom1Header) + - #(loc.Include_docLabel) - #(loc.Include_docHelpLabel) + #(loc.Include_docLabel) + #(loc.Include_docHelpLabel) - #(loc.Include_pipLabel) - #(loc.Include_pipHelpLabel) + #(loc.Include_pipLabel) + #(loc.Include_pipHelpLabel) - #(loc.Include_tcltkLabel) - #(loc.Include_tcltkHelpLabel) + #(loc.Include_tcltkLabel) + #(loc.Include_tcltkHelpLabel) - #(loc.Include_testLabel) - #(loc.Include_testHelpLabel) + #(loc.Include_testLabel) + #(loc.Include_testHelpLabel) - #(loc.Include_launcherLabel) - #(loc.InstallLauncherAllUsersLabel) - #(loc.Include_launcherHelpLabel) + #(loc.Include_launcherLabel) + #(loc.InstallLauncherAllUsersLabel) + #(loc.Include_launcherHelpLabel) - - - + + + - #(loc.Custom2Header) - + #(loc.Custom2Header) + - #(loc.InstallAllUsersLabel) - #(loc.AssociateFilesLabel) - #(loc.ShortcutsLabel) - #(loc.PrependPathLabel) - #(loc.PrecompileLabel) - #(loc.Include_symbolsLabel) - #(loc.Include_debugLabel) + #(loc.InstallAllUsersLabel) + #(loc.AssociateFilesLabel) + #(loc.ShortcutsLabel) + #(loc.PrependPathLabel) + #(loc.PrecompileLabel) + #(loc.Include_symbolsLabel) + #(loc.Include_debugLabel) - #(loc.CustomLocationLabel) - - - #(loc.CustomLocationHelpLabel) + #(loc.CustomLocationLabel) + + + #(loc.CustomLocationHelpLabel) - - - + + + - #(loc.ProgressHeader) - + #(loc.ProgressHeader) + - #(loc.ProgressLabel) - #(loc.OverallProgressPackageText) - - + #(loc.ProgressLabel) + #(loc.OverallProgressPackageText) + + - #(loc.ModifyHeader) - + #(loc.ModifyHeader) + - - - + + + - + - #(loc.SuccessHeader) - + #(loc.SuccessHeader) + #(loc.SuccessRestartText) - - + + - #(loc.FailureHeader) - + #(loc.FailureHeader) + #(loc.FailureHyperlinkLogText) #(loc.FailureRestartText) - - + + \ No newline at end of file diff --git a/Tools/msi/bundle/SideBar.png b/Tools/msi/bundle/SideBar.png index 9c18fff33a6ea50e3514ea938c5e5bfe4c5fe4c6..a23ce5e145848836c6ee21cc0cb4603263630455 GIT binary patch [stripped] -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 23 02:26:08 2015 From: python-checkins at python.org (steve.dower) Date: Wed, 23 Sep 2015 00:26:08 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzE5MTQz?= =?utf-8?q?=3A_platform_module_now_reads_Windows_version_from_kernel32=2Ed?= =?utf-8?q?ll_to?= Message-ID: <20150923002607.115474.20945@psf.io> https://hg.python.org/cpython/rev/d8453733cc0c changeset: 98181:d8453733cc0c branch: 2.7 parent: 98164:9d77345adcc9 user: Steve Dower date: Tue Sep 22 17:25:30 2015 -0700 summary: Issue #19143: platform module now reads Windows version from kernel32.dll to avoid compatibility shims. files: Lib/platform.py | 302 +++++++++++++++-------------------- Misc/NEWS | 3 + 2 files changed, 129 insertions(+), 176 deletions(-) diff --git a/Lib/platform.py b/Lib/platform.py --- a/Lib/platform.py +++ b/Lib/platform.py @@ -28,12 +28,14 @@ # Betancourt, Randall Hopper, Karl Putland, John Farrell, Greg # Andruk, Just van Rossum, Thomas Heller, Mark R. Levinson, Mark # Hammond, Bill Tutt, Hans Nowak, Uwe Zessin (OpenVMS support), -# Colin Kong, Trent Mick, Guido van Rossum, Anthony Baxter +# Colin Kong, Trent Mick, Guido van Rossum, Anthony Baxter, Steve +# Dower # # History: # # # +# 1.0.8 - changed Windows support to read version from kernel32.dll # 1.0.7 - added DEV_NULL # 1.0.6 - added linux_distribution() # 1.0.5 - fixed Java support to allow running the module on Jython @@ -531,189 +533,137 @@ version = _norm_version(version) return system,release,version -def _win32_getvalue(key,name,default=''): +_WIN32_CLIENT_RELEASES = { + (5, 0): "2000", + (5, 1): "XP", + # Strictly, 5.2 client is XP 64-bit, but platform.py historically + # has always called it 2003 Server + (5, 2): "2003Server", + (5, None): "post2003", - """ Read a value for name from the registry key. + (6, 0): "Vista", + (6, 1): "7", + (6, 2): "8", + (6, 3): "8.1", + (6, None): "post8.1", - In case this fails, default is returned. + (10, 0): "10", + (10, None): "post10", +} - """ +# Server release name lookup will default to client names if necessary +_WIN32_SERVER_RELEASES = { + (5, 2): "2003Server", + + (6, 0): "2008Server", + (6, 1): "2008ServerR2", + (6, 2): "2012Server", + (6, 3): "2012ServerR2", + (6, None): "post2012ServerR2", +} + +def _get_real_winver(maj, min, build): + if maj < 6 or (maj == 6 and min < 2): + return maj, min, build + + from ctypes import (c_buffer, POINTER, byref, create_unicode_buffer, + Structure, WinDLL) + from ctypes.wintypes import DWORD, HANDLE + + class VS_FIXEDFILEINFO(Structure): + _fields_ = [ + ("dwSignature", DWORD), + ("dwStrucVersion", DWORD), + ("dwFileVersionMS", DWORD), + ("dwFileVersionLS", DWORD), + ("dwProductVersionMS", DWORD), + ("dwProductVersionLS", DWORD), + ("dwFileFlagsMask", DWORD), + ("dwFileFlags", DWORD), + ("dwFileOS", DWORD), + ("dwFileType", DWORD), + ("dwFileSubtype", DWORD), + ("dwFileDateMS", DWORD), + ("dwFileDateLS", DWORD), + ] + + kernel32 = WinDLL('kernel32') + version = WinDLL('version') + + # We will immediately double the length up to MAX_PATH, but the + # path may be longer, so we retry until the returned string is + # shorter than our buffer. + name_len = actual_len = 130 + while actual_len == name_len: + name_len *= 2 + name = create_unicode_buffer(name_len) + actual_len = kernel32.GetModuleFileNameW(HANDLE(kernel32._handle), + name, len(name)) + if not actual_len: + return maj, min, build + + size = version.GetFileVersionInfoSizeW(name, None) + if not size: + return maj, min, build + + ver_block = c_buffer(size) + if (not version.GetFileVersionInfoW(name, None, size, ver_block) or + not ver_block): + return maj, min, build + + pvi = POINTER(VS_FIXEDFILEINFO)() + if not version.VerQueryValueW(ver_block, "", byref(pvi), byref(DWORD())): + return maj, min, build + + maj = pvi.contents.dwProductVersionMS >> 16 + min = pvi.contents.dwProductVersionMS & 0xFFFF + build = pvi.contents.dwProductVersionLS >> 16 + + return maj, min, build + +def win32_ver(release='', version='', csd='', ptype=''): + from sys import getwindowsversion try: - # Use win32api if available - from win32api import RegQueryValueEx + from winreg import OpenKeyEx, QueryValueEx, CloseKey, HKEY_LOCAL_MACHINE except ImportError: - # On Python 2.0 and later, emulate using _winreg - import _winreg - RegQueryValueEx = _winreg.QueryValueEx + from _winreg import OpenKeyEx, QueryValueEx, CloseKey, HKEY_LOCAL_MACHINE + + winver = getwindowsversion() + maj, min, build = _get_real_winver(*winver[:3]) + version = '{0}.{1}.{2}'.format(maj, min, build) + + release = (_WIN32_CLIENT_RELEASES.get((maj, min)) or + _WIN32_CLIENT_RELEASES.get((maj, None)) or + release) + + # getwindowsversion() reflect the compatibility mode Python is + # running under, and so the service pack value is only going to be + # valid if the versions match. + if winver[:2] == (maj, min): + try: + csd = 'SP{}'.format(winver.service_pack_major) + except AttributeError: + if csd[:13] == 'Service Pack ': + csd = 'SP' + csd[13:] + + # VER_NT_SERVER = 3 + if getattr(winver, 'product_type', None) == 3: + release = (_WIN32_SERVER_RELEASES.get((maj, min)) or + _WIN32_SERVER_RELEASES.get((maj, None)) or + release) + + key = None try: - return RegQueryValueEx(key,name) + key = OpenKeyEx(HKEY_LOCAL_MACHINE, + r'SOFTWARE\Microsoft\Windows NT\CurrentVersion') + ptype = QueryValueEx(key, 'CurrentType')[0] except: - return default + pass + finally: + if key: + CloseKey(key) -def win32_ver(release='',version='',csd='',ptype=''): - - """ Get additional version information from the Windows Registry - and return a tuple (version,csd,ptype) referring to version - number, CSD level (service pack), and OS type (multi/single - processor). - - As a hint: ptype returns 'Uniprocessor Free' on single - processor NT machines and 'Multiprocessor Free' on multi - processor machines. The 'Free' refers to the OS version being - free of debugging code. It could also state 'Checked' which - means the OS version uses debugging code, i.e. code that - checks arguments, ranges, etc. (Thomas Heller). - - Note: this function works best with Mark Hammond's win32 - package installed, but also on Python 2.3 and later. It - obviously only runs on Win32 compatible platforms. - - """ - # XXX Is there any way to find out the processor type on WinXX ? - # XXX Is win32 available on Windows CE ? - # - # Adapted from code posted by Karl Putland to comp.lang.python. - # - # The mappings between reg. values and release names can be found - # here: http://msdn.microsoft.com/library/en-us/sysinfo/base/osversioninfo_str.asp - - # Import the needed APIs - try: - import win32api - from win32api import RegQueryValueEx, RegOpenKeyEx, \ - RegCloseKey, GetVersionEx - from win32con import HKEY_LOCAL_MACHINE, VER_PLATFORM_WIN32_NT, \ - VER_PLATFORM_WIN32_WINDOWS, VER_NT_WORKSTATION - except ImportError: - # Emulate the win32api module using Python APIs - try: - sys.getwindowsversion - except AttributeError: - # No emulation possible, so return the defaults... - return release,version,csd,ptype - else: - # Emulation using _winreg (added in Python 2.0) and - # sys.getwindowsversion() (added in Python 2.3) - import _winreg - GetVersionEx = sys.getwindowsversion - RegQueryValueEx = _winreg.QueryValueEx - RegOpenKeyEx = _winreg.OpenKeyEx - RegCloseKey = _winreg.CloseKey - HKEY_LOCAL_MACHINE = _winreg.HKEY_LOCAL_MACHINE - VER_PLATFORM_WIN32_WINDOWS = 1 - VER_PLATFORM_WIN32_NT = 2 - VER_NT_WORKSTATION = 1 - VER_NT_SERVER = 3 - REG_SZ = 1 - - # Find out the registry key and some general version infos - winver = GetVersionEx() - maj,min,buildno,plat,csd = winver - version = '%i.%i.%i' % (maj,min,buildno & 0xFFFF) - if hasattr(winver, "service_pack"): - if winver.service_pack != "": - csd = 'SP%s' % winver.service_pack_major - else: - if csd[:13] == 'Service Pack ': - csd = 'SP' + csd[13:] - - if plat == VER_PLATFORM_WIN32_WINDOWS: - regkey = 'SOFTWARE\\Microsoft\\Windows\\CurrentVersion' - # Try to guess the release name - if maj == 4: - if min == 0: - release = '95' - elif min == 10: - release = '98' - elif min == 90: - release = 'Me' - else: - release = 'postMe' - elif maj == 5: - release = '2000' - - elif plat == VER_PLATFORM_WIN32_NT: - regkey = 'SOFTWARE\\Microsoft\\Windows NT\\CurrentVersion' - if maj <= 4: - release = 'NT' - elif maj == 5: - if min == 0: - release = '2000' - elif min == 1: - release = 'XP' - elif min == 2: - release = '2003Server' - else: - release = 'post2003' - elif maj == 6: - if hasattr(winver, "product_type"): - product_type = winver.product_type - else: - product_type = VER_NT_WORKSTATION - # Without an OSVERSIONINFOEX capable sys.getwindowsversion(), - # or help from the registry, we cannot properly identify - # non-workstation versions. - try: - key = RegOpenKeyEx(HKEY_LOCAL_MACHINE, regkey) - name, type = RegQueryValueEx(key, "ProductName") - # Discard any type that isn't REG_SZ - if type == REG_SZ and name.find("Server") != -1: - product_type = VER_NT_SERVER - except WindowsError: - # Use default of VER_NT_WORKSTATION - pass - - if min == 0: - if product_type == VER_NT_WORKSTATION: - release = 'Vista' - else: - release = '2008Server' - elif min == 1: - if product_type == VER_NT_WORKSTATION: - release = '7' - else: - release = '2008ServerR2' - elif min == 2: - if product_type == VER_NT_WORKSTATION: - release = '8' - else: - release = '2012Server' - else: - release = 'post2012Server' - - else: - if not release: - # E.g. Win3.1 with win32s - release = '%i.%i' % (maj,min) - return release,version,csd,ptype - - # Open the registry key - try: - keyCurVer = RegOpenKeyEx(HKEY_LOCAL_MACHINE, regkey) - # Get a value to make sure the key exists... - RegQueryValueEx(keyCurVer, 'SystemRoot') - except: - return release,version,csd,ptype - - # Parse values - #subversion = _win32_getvalue(keyCurVer, - # 'SubVersionNumber', - # ('',1))[0] - #if subversion: - # release = release + subversion # 95a, 95b, etc. - build = _win32_getvalue(keyCurVer, - 'CurrentBuildNumber', - ('',1))[0] - ptype = _win32_getvalue(keyCurVer, - 'CurrentType', - (ptype,1))[0] - - # Normalize version - version = _norm_version(version,build) - - # Close key - RegCloseKey(keyCurVer) - return release,version,csd,ptype + return release, version, csd, ptype def _mac_ver_lookup(selectors,default=None): diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -37,6 +37,9 @@ Library ------- +- Issue #19143: platform module now reads Windows version from kernel32.dll to + avoid compatibility shims. + - Issue #24684: socket.socket.getaddrinfo() now calls PyUnicode_AsEncodedString() instead of calling the encode() method of the host, to handle correctly custom unicode string with an encode() method -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 23 02:27:07 2015 From: python-checkins at python.org (steve.dower) Date: Wed, 23 Sep 2015 00:27:07 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2319143=3A_platform_module_now_reads_Windows_vers?= =?utf-8?q?ion_from_kernel32=2Edll_to?= Message-ID: <20150923002707.9935.49552@psf.io> https://hg.python.org/cpython/rev/2f55d73e5ad6 changeset: 98183:2f55d73e5ad6 parent: 98180:ff1c6e9eeed4 parent: 98182:fa869ccf9368 user: Steve Dower date: Tue Sep 22 17:24:01 2015 -0700 summary: Issue #19143: platform module now reads Windows version from kernel32.dll to avoid compatibility shims. files: Lib/platform.py | 292 +++++++++++++++-------------------- Misc/NEWS | 3 + 2 files changed, 125 insertions(+), 170 deletions(-) diff --git a/Lib/platform.py b/Lib/platform.py --- a/Lib/platform.py +++ b/Lib/platform.py @@ -26,12 +26,14 @@ # Betancourt, Randall Hopper, Karl Putland, John Farrell, Greg # Andruk, Just van Rossum, Thomas Heller, Mark R. Levinson, Mark # Hammond, Bill Tutt, Hans Nowak, Uwe Zessin (OpenVMS support), -# Colin Kong, Trent Mick, Guido van Rossum, Anthony Baxter +# Colin Kong, Trent Mick, Guido van Rossum, Anthony Baxter, Steve +# Dower # # History: # # # +# 1.0.8 - changed Windows support to read version from kernel32.dll # 1.0.7 - added DEV_NULL # 1.0.6 - added linux_distribution() # 1.0.5 - fixed Java support to allow running the module on Jython @@ -469,188 +471,138 @@ version = _norm_version(version) return system, release, version -def _win32_getvalue(key, name, default=''): +_WIN32_CLIENT_RELEASES = { + (5, 0): "2000", + (5, 1): "XP", + # Strictly, 5.2 client is XP 64-bit, but platform.py historically + # has always called it 2003 Server + (5, 2): "2003Server", + (5, None): "post2003", - """ Read a value for name from the registry key. + (6, 0): "Vista", + (6, 1): "7", + (6, 2): "8", + (6, 3): "8.1", + (6, None): "post8.1", - In case this fails, default is returned. + (10, 0): "10", + (10, None): "post10", +} - """ - try: - # Use win32api if available - from win32api import RegQueryValueEx - except ImportError: - # On Python 2.0 and later, emulate using winreg - import winreg - RegQueryValueEx = winreg.QueryValueEx - try: - return RegQueryValueEx(key, name) - except: - return default +# Server release name lookup will default to client names if necessary +_WIN32_SERVER_RELEASES = { + (5, 2): "2003Server", + + (6, 0): "2008Server", + (6, 1): "2008ServerR2", + (6, 2): "2012Server", + (6, 3): "2012ServerR2", + (6, None): "post2012ServerR2", +} + +def _get_real_winver(maj, min, build): + if maj < 6 or (maj == 6 and min < 2): + return maj, min, build + + from ctypes import (c_buffer, POINTER, byref, create_unicode_buffer, + Structure, WinDLL) + from ctypes.wintypes import DWORD, HANDLE + + class VS_FIXEDFILEINFO(Structure): + _fields_ = [ + ("dwSignature", DWORD), + ("dwStrucVersion", DWORD), + ("dwFileVersionMS", DWORD), + ("dwFileVersionLS", DWORD), + ("dwProductVersionMS", DWORD), + ("dwProductVersionLS", DWORD), + ("dwFileFlagsMask", DWORD), + ("dwFileFlags", DWORD), + ("dwFileOS", DWORD), + ("dwFileType", DWORD), + ("dwFileSubtype", DWORD), + ("dwFileDateMS", DWORD), + ("dwFileDateLS", DWORD), + ] + + kernel32 = WinDLL('kernel32') + version = WinDLL('version') + + # We will immediately double the length up to MAX_PATH, but the + # path may be longer, so we retry until the returned string is + # shorter than our buffer. + name_len = actual_len = 130 + while actual_len == name_len: + name_len *= 2 + name = create_unicode_buffer(name_len) + actual_len = kernel32.GetModuleFileNameW(HANDLE(kernel32._handle), + name, len(name)) + if not actual_len: + return maj, min, build + + size = version.GetFileVersionInfoSizeW(name, None) + if not size: + return maj, min, build + + ver_block = c_buffer(size) + if (not version.GetFileVersionInfoW(name, None, size, ver_block) or + not ver_block): + return maj, min, build + + pvi = POINTER(VS_FIXEDFILEINFO)() + if not version.VerQueryValueW(ver_block, "", byref(pvi), byref(DWORD())): + return maj, min, build + + maj = pvi.contents.dwProductVersionMS >> 16 + min = pvi.contents.dwProductVersionMS & 0xFFFF + build = pvi.contents.dwProductVersionLS >> 16 + + return maj, min, build def win32_ver(release='', version='', csd='', ptype=''): + from sys import getwindowsversion + try: + from winreg import OpenKeyEx, QueryValueEx, CloseKey, HKEY_LOCAL_MACHINE + except ImportError: + from _winreg import OpenKeyEx, QueryValueEx, CloseKey, HKEY_LOCAL_MACHINE - """ Get additional version information from the Windows Registry - and return a tuple (version, csd, ptype) referring to version - number, CSD level (service pack), and OS type (multi/single - processor). + winver = getwindowsversion() + maj, min, build = _get_real_winver(*winver[:3]) + version = '{0}.{1}.{2}'.format(maj, min, build) - As a hint: ptype returns 'Uniprocessor Free' on single - processor NT machines and 'Multiprocessor Free' on multi - processor machines. The 'Free' refers to the OS version being - free of debugging code. It could also state 'Checked' which - means the OS version uses debugging code, i.e. code that - checks arguments, ranges, etc. (Thomas Heller). + release = (_WIN32_CLIENT_RELEASES.get((maj, min)) or + _WIN32_CLIENT_RELEASES.get((maj, None)) or + release) - Note: this function works best with Mark Hammond's win32 - package installed, but also on Python 2.3 and later. It - obviously only runs on Win32 compatible platforms. + # getwindowsversion() reflect the compatibility mode Python is + # running under, and so the service pack value is only going to be + # valid if the versions match. + if winver[:2] == (maj, min): + try: + csd = 'SP{}'.format(winver.service_pack_major) + except AttributeError: + if csd[:13] == 'Service Pack ': + csd = 'SP' + csd[13:] - """ - # XXX Is there any way to find out the processor type on WinXX ? - # XXX Is win32 available on Windows CE ? - # - # Adapted from code posted by Karl Putland to comp.lang.python. - # - # The mappings between reg. values and release names can be found - # here: http://msdn.microsoft.com/library/en-us/sysinfo/base/osversioninfo_str.asp + # VER_NT_SERVER = 3 + if getattr(winver, 'product_type', None) == 3: + release = (_WIN32_SERVER_RELEASES.get((maj, min)) or + _WIN32_SERVER_RELEASES.get((maj, None)) or + release) - # Import the needed APIs + key = None try: - from win32api import RegQueryValueEx, RegOpenKeyEx, \ - RegCloseKey, GetVersionEx - from win32con import HKEY_LOCAL_MACHINE, VER_PLATFORM_WIN32_NT, \ - VER_PLATFORM_WIN32_WINDOWS, VER_NT_WORKSTATION - except ImportError: - # Emulate the win32api module using Python APIs - try: - sys.getwindowsversion - except AttributeError: - # No emulation possible, so return the defaults... - return release, version, csd, ptype - else: - # Emulation using winreg (added in Python 2.0) and - # sys.getwindowsversion() (added in Python 2.3) - import winreg - GetVersionEx = sys.getwindowsversion - RegQueryValueEx = winreg.QueryValueEx - RegOpenKeyEx = winreg.OpenKeyEx - RegCloseKey = winreg.CloseKey - HKEY_LOCAL_MACHINE = winreg.HKEY_LOCAL_MACHINE - VER_PLATFORM_WIN32_WINDOWS = 1 - VER_PLATFORM_WIN32_NT = 2 - VER_NT_WORKSTATION = 1 - VER_NT_SERVER = 3 - REG_SZ = 1 + key = OpenKeyEx(HKEY_LOCAL_MACHINE, + r'SOFTWARE\Microsoft\Windows NT\CurrentVersion') + ptype = QueryValueEx(key, 'CurrentType')[0] + except: + pass + finally: + if key: + CloseKey(key) - # Find out the registry key and some general version infos - winver = GetVersionEx() - maj, min, buildno, plat, csd = winver - version = '%i.%i.%i' % (maj, min, buildno & 0xFFFF) - if hasattr(winver, "service_pack"): - if winver.service_pack != "": - csd = 'SP%s' % winver.service_pack_major - else: - if csd[:13] == 'Service Pack ': - csd = 'SP' + csd[13:] + return release, version, csd, ptype - if plat == VER_PLATFORM_WIN32_WINDOWS: - regkey = 'SOFTWARE\\Microsoft\\Windows\\CurrentVersion' - # Try to guess the release name - if maj == 4: - if min == 0: - release = '95' - elif min == 10: - release = '98' - elif min == 90: - release = 'Me' - else: - release = 'postMe' - elif maj == 5: - release = '2000' - - elif plat == VER_PLATFORM_WIN32_NT: - regkey = 'SOFTWARE\\Microsoft\\Windows NT\\CurrentVersion' - if maj <= 4: - release = 'NT' - elif maj == 5: - if min == 0: - release = '2000' - elif min == 1: - release = 'XP' - elif min == 2: - release = '2003Server' - else: - release = 'post2003' - elif maj == 6: - if hasattr(winver, "product_type"): - product_type = winver.product_type - else: - product_type = VER_NT_WORKSTATION - # Without an OSVERSIONINFOEX capable sys.getwindowsversion(), - # or help from the registry, we cannot properly identify - # non-workstation versions. - try: - key = RegOpenKeyEx(HKEY_LOCAL_MACHINE, regkey) - name, type = RegQueryValueEx(key, "ProductName") - # Discard any type that isn't REG_SZ - if type == REG_SZ and name.find("Server") != -1: - product_type = VER_NT_SERVER - except OSError: - # Use default of VER_NT_WORKSTATION - pass - - if min == 0: - if product_type == VER_NT_WORKSTATION: - release = 'Vista' - else: - release = '2008Server' - elif min == 1: - if product_type == VER_NT_WORKSTATION: - release = '7' - else: - release = '2008ServerR2' - elif min == 2: - if product_type == VER_NT_WORKSTATION: - release = '8' - else: - release = '2012Server' - else: - release = 'post2012Server' - - else: - if not release: - # E.g. Win3.1 with win32s - release = '%i.%i' % (maj, min) - return release, version, csd, ptype - - # Open the registry key - try: - keyCurVer = RegOpenKeyEx(HKEY_LOCAL_MACHINE, regkey) - # Get a value to make sure the key exists... - RegQueryValueEx(keyCurVer, 'SystemRoot') - except: - return release, version, csd, ptype - - # Parse values - #subversion = _win32_getvalue(keyCurVer, - # 'SubVersionNumber', - # ('',1))[0] - #if subversion: - # release = release + subversion # 95a, 95b, etc. - build = _win32_getvalue(keyCurVer, - 'CurrentBuildNumber', - ('', 1))[0] - ptype = _win32_getvalue(keyCurVer, - 'CurrentType', - (ptype, 1))[0] - - # Normalize version - version = _norm_version(version, build) - - # Close key - RegCloseKey(keyCurVer) - return release, version, csd, ptype def _mac_ver_xml(): fn = '/System/Library/CoreServices/SystemVersion.plist' diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -138,6 +138,9 @@ Library ------- +- Issue #19143: platform module now reads Windows version from kernel32.dll to + avoid compatibility shims. + - Issue #25092: Fix datetime.strftime() failure when errno was already set to EINVAL. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 23 02:27:09 2015 From: python-checkins at python.org (steve.dower) Date: Wed, 23 Sep 2015 00:27:09 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzE5MTQz?= =?utf-8?q?=3A_platform_module_now_reads_Windows_version_from_kernel32=2Ed?= =?utf-8?q?ll_to?= Message-ID: <20150923002707.82654.9543@psf.io> https://hg.python.org/cpython/rev/fa869ccf9368 changeset: 98182:fa869ccf9368 branch: 3.5 parent: 98179:31b230e5517e user: Steve Dower date: Tue Sep 22 17:23:39 2015 -0700 summary: Issue #19143: platform module now reads Windows version from kernel32.dll to avoid compatibility shims. files: Lib/platform.py | 292 +++++++++++++++-------------------- Misc/NEWS | 3 + 2 files changed, 125 insertions(+), 170 deletions(-) diff --git a/Lib/platform.py b/Lib/platform.py --- a/Lib/platform.py +++ b/Lib/platform.py @@ -26,12 +26,14 @@ # Betancourt, Randall Hopper, Karl Putland, John Farrell, Greg # Andruk, Just van Rossum, Thomas Heller, Mark R. Levinson, Mark # Hammond, Bill Tutt, Hans Nowak, Uwe Zessin (OpenVMS support), -# Colin Kong, Trent Mick, Guido van Rossum, Anthony Baxter +# Colin Kong, Trent Mick, Guido van Rossum, Anthony Baxter, Steve +# Dower # # History: # # # +# 1.0.8 - changed Windows support to read version from kernel32.dll # 1.0.7 - added DEV_NULL # 1.0.6 - added linux_distribution() # 1.0.5 - fixed Java support to allow running the module on Jython @@ -469,188 +471,138 @@ version = _norm_version(version) return system, release, version -def _win32_getvalue(key, name, default=''): +_WIN32_CLIENT_RELEASES = { + (5, 0): "2000", + (5, 1): "XP", + # Strictly, 5.2 client is XP 64-bit, but platform.py historically + # has always called it 2003 Server + (5, 2): "2003Server", + (5, None): "post2003", - """ Read a value for name from the registry key. + (6, 0): "Vista", + (6, 1): "7", + (6, 2): "8", + (6, 3): "8.1", + (6, None): "post8.1", - In case this fails, default is returned. + (10, 0): "10", + (10, None): "post10", +} - """ - try: - # Use win32api if available - from win32api import RegQueryValueEx - except ImportError: - # On Python 2.0 and later, emulate using winreg - import winreg - RegQueryValueEx = winreg.QueryValueEx - try: - return RegQueryValueEx(key, name) - except: - return default +# Server release name lookup will default to client names if necessary +_WIN32_SERVER_RELEASES = { + (5, 2): "2003Server", + + (6, 0): "2008Server", + (6, 1): "2008ServerR2", + (6, 2): "2012Server", + (6, 3): "2012ServerR2", + (6, None): "post2012ServerR2", +} + +def _get_real_winver(maj, min, build): + if maj < 6 or (maj == 6 and min < 2): + return maj, min, build + + from ctypes import (c_buffer, POINTER, byref, create_unicode_buffer, + Structure, WinDLL) + from ctypes.wintypes import DWORD, HANDLE + + class VS_FIXEDFILEINFO(Structure): + _fields_ = [ + ("dwSignature", DWORD), + ("dwStrucVersion", DWORD), + ("dwFileVersionMS", DWORD), + ("dwFileVersionLS", DWORD), + ("dwProductVersionMS", DWORD), + ("dwProductVersionLS", DWORD), + ("dwFileFlagsMask", DWORD), + ("dwFileFlags", DWORD), + ("dwFileOS", DWORD), + ("dwFileType", DWORD), + ("dwFileSubtype", DWORD), + ("dwFileDateMS", DWORD), + ("dwFileDateLS", DWORD), + ] + + kernel32 = WinDLL('kernel32') + version = WinDLL('version') + + # We will immediately double the length up to MAX_PATH, but the + # path may be longer, so we retry until the returned string is + # shorter than our buffer. + name_len = actual_len = 130 + while actual_len == name_len: + name_len *= 2 + name = create_unicode_buffer(name_len) + actual_len = kernel32.GetModuleFileNameW(HANDLE(kernel32._handle), + name, len(name)) + if not actual_len: + return maj, min, build + + size = version.GetFileVersionInfoSizeW(name, None) + if not size: + return maj, min, build + + ver_block = c_buffer(size) + if (not version.GetFileVersionInfoW(name, None, size, ver_block) or + not ver_block): + return maj, min, build + + pvi = POINTER(VS_FIXEDFILEINFO)() + if not version.VerQueryValueW(ver_block, "", byref(pvi), byref(DWORD())): + return maj, min, build + + maj = pvi.contents.dwProductVersionMS >> 16 + min = pvi.contents.dwProductVersionMS & 0xFFFF + build = pvi.contents.dwProductVersionLS >> 16 + + return maj, min, build def win32_ver(release='', version='', csd='', ptype=''): + from sys import getwindowsversion + try: + from winreg import OpenKeyEx, QueryValueEx, CloseKey, HKEY_LOCAL_MACHINE + except ImportError: + from _winreg import OpenKeyEx, QueryValueEx, CloseKey, HKEY_LOCAL_MACHINE - """ Get additional version information from the Windows Registry - and return a tuple (version, csd, ptype) referring to version - number, CSD level (service pack), and OS type (multi/single - processor). + winver = getwindowsversion() + maj, min, build = _get_real_winver(*winver[:3]) + version = '{0}.{1}.{2}'.format(maj, min, build) - As a hint: ptype returns 'Uniprocessor Free' on single - processor NT machines and 'Multiprocessor Free' on multi - processor machines. The 'Free' refers to the OS version being - free of debugging code. It could also state 'Checked' which - means the OS version uses debugging code, i.e. code that - checks arguments, ranges, etc. (Thomas Heller). + release = (_WIN32_CLIENT_RELEASES.get((maj, min)) or + _WIN32_CLIENT_RELEASES.get((maj, None)) or + release) - Note: this function works best with Mark Hammond's win32 - package installed, but also on Python 2.3 and later. It - obviously only runs on Win32 compatible platforms. + # getwindowsversion() reflect the compatibility mode Python is + # running under, and so the service pack value is only going to be + # valid if the versions match. + if winver[:2] == (maj, min): + try: + csd = 'SP{}'.format(winver.service_pack_major) + except AttributeError: + if csd[:13] == 'Service Pack ': + csd = 'SP' + csd[13:] - """ - # XXX Is there any way to find out the processor type on WinXX ? - # XXX Is win32 available on Windows CE ? - # - # Adapted from code posted by Karl Putland to comp.lang.python. - # - # The mappings between reg. values and release names can be found - # here: http://msdn.microsoft.com/library/en-us/sysinfo/base/osversioninfo_str.asp + # VER_NT_SERVER = 3 + if getattr(winver, 'product_type', None) == 3: + release = (_WIN32_SERVER_RELEASES.get((maj, min)) or + _WIN32_SERVER_RELEASES.get((maj, None)) or + release) - # Import the needed APIs + key = None try: - from win32api import RegQueryValueEx, RegOpenKeyEx, \ - RegCloseKey, GetVersionEx - from win32con import HKEY_LOCAL_MACHINE, VER_PLATFORM_WIN32_NT, \ - VER_PLATFORM_WIN32_WINDOWS, VER_NT_WORKSTATION - except ImportError: - # Emulate the win32api module using Python APIs - try: - sys.getwindowsversion - except AttributeError: - # No emulation possible, so return the defaults... - return release, version, csd, ptype - else: - # Emulation using winreg (added in Python 2.0) and - # sys.getwindowsversion() (added in Python 2.3) - import winreg - GetVersionEx = sys.getwindowsversion - RegQueryValueEx = winreg.QueryValueEx - RegOpenKeyEx = winreg.OpenKeyEx - RegCloseKey = winreg.CloseKey - HKEY_LOCAL_MACHINE = winreg.HKEY_LOCAL_MACHINE - VER_PLATFORM_WIN32_WINDOWS = 1 - VER_PLATFORM_WIN32_NT = 2 - VER_NT_WORKSTATION = 1 - VER_NT_SERVER = 3 - REG_SZ = 1 + key = OpenKeyEx(HKEY_LOCAL_MACHINE, + r'SOFTWARE\Microsoft\Windows NT\CurrentVersion') + ptype = QueryValueEx(key, 'CurrentType')[0] + except: + pass + finally: + if key: + CloseKey(key) - # Find out the registry key and some general version infos - winver = GetVersionEx() - maj, min, buildno, plat, csd = winver - version = '%i.%i.%i' % (maj, min, buildno & 0xFFFF) - if hasattr(winver, "service_pack"): - if winver.service_pack != "": - csd = 'SP%s' % winver.service_pack_major - else: - if csd[:13] == 'Service Pack ': - csd = 'SP' + csd[13:] + return release, version, csd, ptype - if plat == VER_PLATFORM_WIN32_WINDOWS: - regkey = 'SOFTWARE\\Microsoft\\Windows\\CurrentVersion' - # Try to guess the release name - if maj == 4: - if min == 0: - release = '95' - elif min == 10: - release = '98' - elif min == 90: - release = 'Me' - else: - release = 'postMe' - elif maj == 5: - release = '2000' - - elif plat == VER_PLATFORM_WIN32_NT: - regkey = 'SOFTWARE\\Microsoft\\Windows NT\\CurrentVersion' - if maj <= 4: - release = 'NT' - elif maj == 5: - if min == 0: - release = '2000' - elif min == 1: - release = 'XP' - elif min == 2: - release = '2003Server' - else: - release = 'post2003' - elif maj == 6: - if hasattr(winver, "product_type"): - product_type = winver.product_type - else: - product_type = VER_NT_WORKSTATION - # Without an OSVERSIONINFOEX capable sys.getwindowsversion(), - # or help from the registry, we cannot properly identify - # non-workstation versions. - try: - key = RegOpenKeyEx(HKEY_LOCAL_MACHINE, regkey) - name, type = RegQueryValueEx(key, "ProductName") - # Discard any type that isn't REG_SZ - if type == REG_SZ and name.find("Server") != -1: - product_type = VER_NT_SERVER - except OSError: - # Use default of VER_NT_WORKSTATION - pass - - if min == 0: - if product_type == VER_NT_WORKSTATION: - release = 'Vista' - else: - release = '2008Server' - elif min == 1: - if product_type == VER_NT_WORKSTATION: - release = '7' - else: - release = '2008ServerR2' - elif min == 2: - if product_type == VER_NT_WORKSTATION: - release = '8' - else: - release = '2012Server' - else: - release = 'post2012Server' - - else: - if not release: - # E.g. Win3.1 with win32s - release = '%i.%i' % (maj, min) - return release, version, csd, ptype - - # Open the registry key - try: - keyCurVer = RegOpenKeyEx(HKEY_LOCAL_MACHINE, regkey) - # Get a value to make sure the key exists... - RegQueryValueEx(keyCurVer, 'SystemRoot') - except: - return release, version, csd, ptype - - # Parse values - #subversion = _win32_getvalue(keyCurVer, - # 'SubVersionNumber', - # ('',1))[0] - #if subversion: - # release = release + subversion # 95a, 95b, etc. - build = _win32_getvalue(keyCurVer, - 'CurrentBuildNumber', - ('', 1))[0] - ptype = _win32_getvalue(keyCurVer, - 'CurrentType', - (ptype, 1))[0] - - # Normalize version - version = _norm_version(version, build) - - # Close key - RegCloseKey(keyCurVer) - return release, version, csd, ptype def _mac_ver_xml(): fn = '/System/Library/CoreServices/SystemVersion.plist' diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -18,6 +18,9 @@ Library ------- +- Issue #19143: platform module now reads Windows version from kernel32.dll to + avoid compatibility shims. + - Issue #25092: Fix datetime.strftime() failure when errno was already set to EINVAL. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 23 02:32:06 2015 From: python-checkins at python.org (steve.dower) Date: Wed, 23 Sep 2015 00:32:06 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge_from_3=2E5?= Message-ID: <20150923003206.31187.36099@psf.io> https://hg.python.org/cpython/rev/33d9921f5065 changeset: 98186:33d9921f5065 parent: 98183:2f55d73e5ad6 parent: 98185:68f1d424a9a5 user: Steve Dower date: Tue Sep 22 17:31:33 2015 -0700 summary: Merge from 3.5 files: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 23 02:32:06 2015 From: python-checkins at python.org (steve.dower) Date: Wed, 23 Sep 2015 00:32:06 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzE5MTQz?= =?utf-8?q?=3A_platform_module_now_reads_Windows_version_from_kernel32=2Ed?= =?utf-8?q?ll_to?= Message-ID: <20150923003206.9951.89998@psf.io> https://hg.python.org/cpython/rev/2f57270374f7 changeset: 98184:2f57270374f7 branch: 3.4 parent: 98165:c48a5234c142 user: Steve Dower date: Tue Sep 22 17:29:51 2015 -0700 summary: Issue #19143: platform module now reads Windows version from kernel32.dll to avoid compatibility shims. files: Lib/platform.py | 292 +++++++++++++++-------------------- Misc/NEWS | 3 + 2 files changed, 125 insertions(+), 170 deletions(-) diff --git a/Lib/platform.py b/Lib/platform.py --- a/Lib/platform.py +++ b/Lib/platform.py @@ -26,12 +26,14 @@ # Betancourt, Randall Hopper, Karl Putland, John Farrell, Greg # Andruk, Just van Rossum, Thomas Heller, Mark R. Levinson, Mark # Hammond, Bill Tutt, Hans Nowak, Uwe Zessin (OpenVMS support), -# Colin Kong, Trent Mick, Guido van Rossum, Anthony Baxter +# Colin Kong, Trent Mick, Guido van Rossum, Anthony Baxter, Steve +# Dower # # History: # # # +# 1.0.8 - changed Windows support to read version from kernel32.dll # 1.0.7 - added DEV_NULL # 1.0.6 - added linux_distribution() # 1.0.5 - fixed Java support to allow running the module on Jython @@ -455,188 +457,138 @@ version = _norm_version(version) return system, release, version -def _win32_getvalue(key, name, default=''): +_WIN32_CLIENT_RELEASES = { + (5, 0): "2000", + (5, 1): "XP", + # Strictly, 5.2 client is XP 64-bit, but platform.py historically + # has always called it 2003 Server + (5, 2): "2003Server", + (5, None): "post2003", - """ Read a value for name from the registry key. + (6, 0): "Vista", + (6, 1): "7", + (6, 2): "8", + (6, 3): "8.1", + (6, None): "post8.1", - In case this fails, default is returned. + (10, 0): "10", + (10, None): "post10", +} - """ - try: - # Use win32api if available - from win32api import RegQueryValueEx - except ImportError: - # On Python 2.0 and later, emulate using winreg - import winreg - RegQueryValueEx = winreg.QueryValueEx - try: - return RegQueryValueEx(key, name) - except: - return default +# Server release name lookup will default to client names if necessary +_WIN32_SERVER_RELEASES = { + (5, 2): "2003Server", + + (6, 0): "2008Server", + (6, 1): "2008ServerR2", + (6, 2): "2012Server", + (6, 3): "2012ServerR2", + (6, None): "post2012ServerR2", +} + +def _get_real_winver(maj, min, build): + if maj < 6 or (maj == 6 and min < 2): + return maj, min, build + + from ctypes import (c_buffer, POINTER, byref, create_unicode_buffer, + Structure, WinDLL) + from ctypes.wintypes import DWORD, HANDLE + + class VS_FIXEDFILEINFO(Structure): + _fields_ = [ + ("dwSignature", DWORD), + ("dwStrucVersion", DWORD), + ("dwFileVersionMS", DWORD), + ("dwFileVersionLS", DWORD), + ("dwProductVersionMS", DWORD), + ("dwProductVersionLS", DWORD), + ("dwFileFlagsMask", DWORD), + ("dwFileFlags", DWORD), + ("dwFileOS", DWORD), + ("dwFileType", DWORD), + ("dwFileSubtype", DWORD), + ("dwFileDateMS", DWORD), + ("dwFileDateLS", DWORD), + ] + + kernel32 = WinDLL('kernel32') + version = WinDLL('version') + + # We will immediately double the length up to MAX_PATH, but the + # path may be longer, so we retry until the returned string is + # shorter than our buffer. + name_len = actual_len = 130 + while actual_len == name_len: + name_len *= 2 + name = create_unicode_buffer(name_len) + actual_len = kernel32.GetModuleFileNameW(HANDLE(kernel32._handle), + name, len(name)) + if not actual_len: + return maj, min, build + + size = version.GetFileVersionInfoSizeW(name, None) + if not size: + return maj, min, build + + ver_block = c_buffer(size) + if (not version.GetFileVersionInfoW(name, None, size, ver_block) or + not ver_block): + return maj, min, build + + pvi = POINTER(VS_FIXEDFILEINFO)() + if not version.VerQueryValueW(ver_block, "", byref(pvi), byref(DWORD())): + return maj, min, build + + maj = pvi.contents.dwProductVersionMS >> 16 + min = pvi.contents.dwProductVersionMS & 0xFFFF + build = pvi.contents.dwProductVersionLS >> 16 + + return maj, min, build def win32_ver(release='', version='', csd='', ptype=''): + from sys import getwindowsversion + try: + from winreg import OpenKeyEx, QueryValueEx, CloseKey, HKEY_LOCAL_MACHINE + except ImportError: + from _winreg import OpenKeyEx, QueryValueEx, CloseKey, HKEY_LOCAL_MACHINE - """ Get additional version information from the Windows Registry - and return a tuple (version, csd, ptype) referring to version - number, CSD level (service pack), and OS type (multi/single - processor). + winver = getwindowsversion() + maj, min, build = _get_real_winver(*winver[:3]) + version = '{0}.{1}.{2}'.format(maj, min, build) - As a hint: ptype returns 'Uniprocessor Free' on single - processor NT machines and 'Multiprocessor Free' on multi - processor machines. The 'Free' refers to the OS version being - free of debugging code. It could also state 'Checked' which - means the OS version uses debugging code, i.e. code that - checks arguments, ranges, etc. (Thomas Heller). + release = (_WIN32_CLIENT_RELEASES.get((maj, min)) or + _WIN32_CLIENT_RELEASES.get((maj, None)) or + release) - Note: this function works best with Mark Hammond's win32 - package installed, but also on Python 2.3 and later. It - obviously only runs on Win32 compatible platforms. + # getwindowsversion() reflect the compatibility mode Python is + # running under, and so the service pack value is only going to be + # valid if the versions match. + if winver[:2] == (maj, min): + try: + csd = 'SP{}'.format(winver.service_pack_major) + except AttributeError: + if csd[:13] == 'Service Pack ': + csd = 'SP' + csd[13:] - """ - # XXX Is there any way to find out the processor type on WinXX ? - # XXX Is win32 available on Windows CE ? - # - # Adapted from code posted by Karl Putland to comp.lang.python. - # - # The mappings between reg. values and release names can be found - # here: http://msdn.microsoft.com/library/en-us/sysinfo/base/osversioninfo_str.asp + # VER_NT_SERVER = 3 + if getattr(winver, 'product_type', None) == 3: + release = (_WIN32_SERVER_RELEASES.get((maj, min)) or + _WIN32_SERVER_RELEASES.get((maj, None)) or + release) - # Import the needed APIs + key = None try: - from win32api import RegQueryValueEx, RegOpenKeyEx, \ - RegCloseKey, GetVersionEx - from win32con import HKEY_LOCAL_MACHINE, VER_PLATFORM_WIN32_NT, \ - VER_PLATFORM_WIN32_WINDOWS, VER_NT_WORKSTATION - except ImportError: - # Emulate the win32api module using Python APIs - try: - sys.getwindowsversion - except AttributeError: - # No emulation possible, so return the defaults... - return release, version, csd, ptype - else: - # Emulation using winreg (added in Python 2.0) and - # sys.getwindowsversion() (added in Python 2.3) - import winreg - GetVersionEx = sys.getwindowsversion - RegQueryValueEx = winreg.QueryValueEx - RegOpenKeyEx = winreg.OpenKeyEx - RegCloseKey = winreg.CloseKey - HKEY_LOCAL_MACHINE = winreg.HKEY_LOCAL_MACHINE - VER_PLATFORM_WIN32_WINDOWS = 1 - VER_PLATFORM_WIN32_NT = 2 - VER_NT_WORKSTATION = 1 - VER_NT_SERVER = 3 - REG_SZ = 1 + key = OpenKeyEx(HKEY_LOCAL_MACHINE, + r'SOFTWARE\Microsoft\Windows NT\CurrentVersion') + ptype = QueryValueEx(key, 'CurrentType')[0] + except: + pass + finally: + if key: + CloseKey(key) - # Find out the registry key and some general version infos - winver = GetVersionEx() - maj, min, buildno, plat, csd = winver - version = '%i.%i.%i' % (maj, min, buildno & 0xFFFF) - if hasattr(winver, "service_pack"): - if winver.service_pack != "": - csd = 'SP%s' % winver.service_pack_major - else: - if csd[:13] == 'Service Pack ': - csd = 'SP' + csd[13:] + return release, version, csd, ptype - if plat == VER_PLATFORM_WIN32_WINDOWS: - regkey = 'SOFTWARE\\Microsoft\\Windows\\CurrentVersion' - # Try to guess the release name - if maj == 4: - if min == 0: - release = '95' - elif min == 10: - release = '98' - elif min == 90: - release = 'Me' - else: - release = 'postMe' - elif maj == 5: - release = '2000' - - elif plat == VER_PLATFORM_WIN32_NT: - regkey = 'SOFTWARE\\Microsoft\\Windows NT\\CurrentVersion' - if maj <= 4: - release = 'NT' - elif maj == 5: - if min == 0: - release = '2000' - elif min == 1: - release = 'XP' - elif min == 2: - release = '2003Server' - else: - release = 'post2003' - elif maj == 6: - if hasattr(winver, "product_type"): - product_type = winver.product_type - else: - product_type = VER_NT_WORKSTATION - # Without an OSVERSIONINFOEX capable sys.getwindowsversion(), - # or help from the registry, we cannot properly identify - # non-workstation versions. - try: - key = RegOpenKeyEx(HKEY_LOCAL_MACHINE, regkey) - name, type = RegQueryValueEx(key, "ProductName") - # Discard any type that isn't REG_SZ - if type == REG_SZ and name.find("Server") != -1: - product_type = VER_NT_SERVER - except OSError: - # Use default of VER_NT_WORKSTATION - pass - - if min == 0: - if product_type == VER_NT_WORKSTATION: - release = 'Vista' - else: - release = '2008Server' - elif min == 1: - if product_type == VER_NT_WORKSTATION: - release = '7' - else: - release = '2008ServerR2' - elif min == 2: - if product_type == VER_NT_WORKSTATION: - release = '8' - else: - release = '2012Server' - else: - release = 'post2012Server' - - else: - if not release: - # E.g. Win3.1 with win32s - release = '%i.%i' % (maj, min) - return release, version, csd, ptype - - # Open the registry key - try: - keyCurVer = RegOpenKeyEx(HKEY_LOCAL_MACHINE, regkey) - # Get a value to make sure the key exists... - RegQueryValueEx(keyCurVer, 'SystemRoot') - except: - return release, version, csd, ptype - - # Parse values - #subversion = _win32_getvalue(keyCurVer, - # 'SubVersionNumber', - # ('',1))[0] - #if subversion: - # release = release + subversion # 95a, 95b, etc. - build = _win32_getvalue(keyCurVer, - 'CurrentBuildNumber', - ('', 1))[0] - ptype = _win32_getvalue(keyCurVer, - 'CurrentType', - (ptype, 1))[0] - - # Normalize version - version = _norm_version(version, build) - - # Close key - RegCloseKey(keyCurVer) - return release, version, csd, ptype def _mac_ver_xml(): fn = '/System/Library/CoreServices/SystemVersion.plist' diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -81,6 +81,9 @@ Library ------- +- Issue #19143: platform module now reads Windows version from kernel32.dll to + avoid compatibility shims. + - Issue #23517: Fix rounding in fromtimestamp() and utcfromtimestamp() methods of datetime.datetime: microseconds are now rounded to nearest with ties going to nearest even integer (ROUND_HALF_EVEN), instead of being rounding -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 23 02:32:08 2015 From: python-checkins at python.org (steve.dower) Date: Wed, 23 Sep 2015 00:32:08 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Merge_from_3=2E4?= Message-ID: <20150923003206.98364.52229@psf.io> https://hg.python.org/cpython/rev/68f1d424a9a5 changeset: 98185:68f1d424a9a5 branch: 3.5 parent: 98182:fa869ccf9368 parent: 98184:2f57270374f7 user: Steve Dower date: Tue Sep 22 17:30:28 2015 -0700 summary: Merge from 3.4 files: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 23 02:36:54 2015 From: python-checkins at python.org (steve.dower) Date: Wed, 23 Sep 2015 00:36:54 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_Handle_calls_t?= =?utf-8?q?o_win32=5Fver_from_non-Windows_platform?= Message-ID: <20150923003654.9947.9370@psf.io> https://hg.python.org/cpython/rev/a8dc0b6e2b63 changeset: 98187:a8dc0b6e2b63 branch: 3.4 parent: 98184:2f57270374f7 user: Steve Dower date: Tue Sep 22 17:35:24 2015 -0700 summary: Handle calls to win32_ver from non-Windows platform files: Lib/platform.py | 5 ++++- 1 files changed, 4 insertions(+), 1 deletions(-) diff --git a/Lib/platform.py b/Lib/platform.py --- a/Lib/platform.py +++ b/Lib/platform.py @@ -546,7 +546,10 @@ return maj, min, build def win32_ver(release='', version='', csd='', ptype=''): - from sys import getwindowsversion + try: + from sys import getwindowsversion + except ImportError: + return release, version, csd, ptype try: from winreg import OpenKeyEx, QueryValueEx, CloseKey, HKEY_LOCAL_MACHINE except ImportError: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 23 02:36:55 2015 From: python-checkins at python.org (steve.dower) Date: Wed, 23 Sep 2015 00:36:55 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_Handle_calls_t?= =?utf-8?q?o_win32=5Fver_from_non-Windows_platform?= Message-ID: <20150923003654.115080.61353@psf.io> https://hg.python.org/cpython/rev/c9db70693f7c changeset: 98190:c9db70693f7c branch: 2.7 parent: 98181:d8453733cc0c user: Steve Dower date: Tue Sep 22 17:35:24 2015 -0700 summary: Handle calls to win32_ver from non-Windows platform files: Lib/platform.py | 5 ++++- 1 files changed, 4 insertions(+), 1 deletions(-) diff --git a/Lib/platform.py b/Lib/platform.py --- a/Lib/platform.py +++ b/Lib/platform.py @@ -622,7 +622,10 @@ return maj, min, build def win32_ver(release='', version='', csd='', ptype=''): - from sys import getwindowsversion + try: + from sys import getwindowsversion + except ImportError: + return release, version, csd, ptype try: from winreg import OpenKeyEx, QueryValueEx, CloseKey, HKEY_LOCAL_MACHINE except ImportError: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 23 02:36:54 2015 From: python-checkins at python.org (steve.dower) Date: Wed, 23 Sep 2015 00:36:54 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge_from_3=2E5?= Message-ID: <20150923003654.3648.77439@psf.io> https://hg.python.org/cpython/rev/a200ab3e074b changeset: 98189:a200ab3e074b parent: 98186:33d9921f5065 parent: 98188:404368d05549 user: Steve Dower date: Tue Sep 22 17:35:55 2015 -0700 summary: Merge from 3.5 files: Lib/platform.py | 5 ++++- 1 files changed, 4 insertions(+), 1 deletions(-) diff --git a/Lib/platform.py b/Lib/platform.py --- a/Lib/platform.py +++ b/Lib/platform.py @@ -560,7 +560,10 @@ return maj, min, build def win32_ver(release='', version='', csd='', ptype=''): - from sys import getwindowsversion + try: + from sys import getwindowsversion + except ImportError: + return release, version, csd, ptype try: from winreg import OpenKeyEx, QueryValueEx, CloseKey, HKEY_LOCAL_MACHINE except ImportError: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 23 02:36:54 2015 From: python-checkins at python.org (steve.dower) Date: Wed, 23 Sep 2015 00:36:54 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Merge_from_3=2E4?= Message-ID: <20150923003654.98350.87129@psf.io> https://hg.python.org/cpython/rev/404368d05549 changeset: 98188:404368d05549 branch: 3.5 parent: 98185:68f1d424a9a5 parent: 98187:a8dc0b6e2b63 user: Steve Dower date: Tue Sep 22 17:35:42 2015 -0700 summary: Merge from 3.4 files: Lib/platform.py | 5 ++++- 1 files changed, 4 insertions(+), 1 deletions(-) diff --git a/Lib/platform.py b/Lib/platform.py --- a/Lib/platform.py +++ b/Lib/platform.py @@ -560,7 +560,10 @@ return maj, min, build def win32_ver(release='', version='', csd='', ptype=''): - from sys import getwindowsversion + try: + from sys import getwindowsversion + except ImportError: + return release, version, csd, ptype try: from winreg import OpenKeyEx, QueryValueEx, CloseKey, HKEY_LOCAL_MACHINE except ImportError: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 23 03:14:15 2015 From: python-checkins at python.org (terry.reedy) Date: Wed, 23 Sep 2015 01:14:15 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzI0NTcw?= =?utf-8?q?=3A_Right-click_for_context_menus_now_work_on_Mac_Aqual_also=2E?= Message-ID: <20150923011414.9937.52702@psf.io> https://hg.python.org/cpython/rev/51b2b1a821b7 changeset: 98192:51b2b1a821b7 branch: 3.4 parent: 98187:a8dc0b6e2b63 user: Terry Jan Reedy date: Tue Sep 22 21:10:27 2015 -0400 summary: Issue #24570: Right-click for context menus now work on Mac Aqual also. Patch by Mark Roseman. files: Lib/idlelib/EditorWindow.py | 10 +++++----- Lib/idlelib/PyShell.py | 8 ++++++++ 2 files changed, 13 insertions(+), 5 deletions(-) diff --git a/Lib/idlelib/EditorWindow.py b/Lib/idlelib/EditorWindow.py --- a/Lib/idlelib/EditorWindow.py +++ b/Lib/idlelib/EditorWindow.py @@ -175,13 +175,13 @@ if macosxSupport.isAquaTk(): # Command-W on editorwindows doesn't work without this. text.bind('<>', self.close_event) - # Some OS X systems have only one mouse button, - # so use control-click for pulldown menus there. - # (Note, AquaTk defines <2> as the right button if - # present and the Tk Text widget already binds <2>.) + # Some OS X systems have only one mouse button, so use + # control-click for popup context menus there. For two + # buttons, AquaTk defines <2> as the right button, not <3>. text.bind("",self.right_menu_event) + text.bind("<2>", self.right_menu_event) else: - # Elsewhere, use right-click for pulldown menus. + # Elsewhere, use right-click for popup menus. text.bind("<3>",self.right_menu_event) text.bind("<>", self.cut) text.bind("<>", self.copy) diff --git a/Lib/idlelib/PyShell.py b/Lib/idlelib/PyShell.py --- a/Lib/idlelib/PyShell.py +++ b/Lib/idlelib/PyShell.py @@ -1547,6 +1547,14 @@ root.withdraw() flist = PyShellFileList(root) macosxSupport.setupApp(root, flist) + + if macosxSupport.isAquaTk(): + # There are some screwed up <2> class bindings for text + # widgets defined in Tk which we need to do away with. + # See issue #24801. + root.unbind_class('Text', '') + root.unbind_class('Text', '') + root.unbind_class('Text', '<>') if enable_edit: if not (cmd or script): -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 23 03:14:14 2015 From: python-checkins at python.org (terry.reedy) Date: Wed, 23 Sep 2015 01:14:14 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzI0NTcw?= =?utf-8?q?=3A_Right-click_for_context_menus_now_work_on_Mac_Aqual_also=2E?= Message-ID: <20150923011414.31191.32010@psf.io> https://hg.python.org/cpython/rev/b79dd0d7dc98 changeset: 98191:b79dd0d7dc98 branch: 2.7 user: Terry Jan Reedy date: Tue Sep 22 21:10:22 2015 -0400 summary: Issue #24570: Right-click for context menus now work on Mac Aqual also. Patch by Mark Roseman. files: Lib/idlelib/EditorWindow.py | 10 +++++----- Lib/idlelib/PyShell.py | 8 ++++++++ 2 files changed, 13 insertions(+), 5 deletions(-) diff --git a/Lib/idlelib/EditorWindow.py b/Lib/idlelib/EditorWindow.py --- a/Lib/idlelib/EditorWindow.py +++ b/Lib/idlelib/EditorWindow.py @@ -202,13 +202,13 @@ if macosxSupport.isAquaTk(): # Command-W on editorwindows doesn't work without this. text.bind('<>', self.close_event) - # Some OS X systems have only one mouse button, - # so use control-click for pulldown menus there. - # (Note, AquaTk defines <2> as the right button if - # present and the Tk Text widget already binds <2>.) + # Some OS X systems have only one mouse button, so use + # control-click for popup context menus there. For two + # buttons, AquaTk defines <2> as the right button, not <3>. text.bind("",self.right_menu_event) + text.bind("<2>", self.right_menu_event) else: - # Elsewhere, use right-click for pulldown menus. + # Elsewhere, use right-click for popup menus. text.bind("<3>",self.right_menu_event) text.bind("<>", self.cut) text.bind("<>", self.copy) diff --git a/Lib/idlelib/PyShell.py b/Lib/idlelib/PyShell.py --- a/Lib/idlelib/PyShell.py +++ b/Lib/idlelib/PyShell.py @@ -1556,6 +1556,14 @@ root.withdraw() flist = PyShellFileList(root) macosxSupport.setupApp(root, flist) + + if macosxSupport.isAquaTk(): + # There are some screwed up <2> class bindings for text + # widgets defined in Tk which we need to do away with. + # See issue #24801. + root.unbind_class('Text', '') + root.unbind_class('Text', '') + root.unbind_class('Text', '<>') if enable_edit: if not (cmd or script): -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 23 03:14:15 2015 From: python-checkins at python.org (terry.reedy) Date: Wed, 23 Sep 2015 01:14:15 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_whitespace?= Message-ID: <20150923011415.16585.12590@psf.io> https://hg.python.org/cpython/rev/86dcaadc7957 changeset: 98196:86dcaadc7957 branch: 3.4 parent: 98192:51b2b1a821b7 user: Terry Jan Reedy date: Tue Sep 22 21:13:28 2015 -0400 summary: whitespace files: Lib/idlelib/PyShell.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Lib/idlelib/PyShell.py b/Lib/idlelib/PyShell.py --- a/Lib/idlelib/PyShell.py +++ b/Lib/idlelib/PyShell.py @@ -1547,7 +1547,7 @@ root.withdraw() flist = PyShellFileList(root) macosxSupport.setupApp(root, flist) - + if macosxSupport.isAquaTk(): # There are some screwed up <2> class bindings for text # widgets defined in Tk which we need to do away with. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 23 03:14:15 2015 From: python-checkins at python.org (terry.reedy) Date: Wed, 23 Sep 2015 01:14:15 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Merge_with_3=2E4?= Message-ID: <20150923011415.3662.77691@psf.io> https://hg.python.org/cpython/rev/ac35626f0f14 changeset: 98193:ac35626f0f14 branch: 3.5 parent: 98188:404368d05549 parent: 98192:51b2b1a821b7 user: Terry Jan Reedy date: Tue Sep 22 21:10:49 2015 -0400 summary: Merge with 3.4 files: Lib/idlelib/EditorWindow.py | 10 +++++----- Lib/idlelib/PyShell.py | 8 ++++++++ 2 files changed, 13 insertions(+), 5 deletions(-) diff --git a/Lib/idlelib/EditorWindow.py b/Lib/idlelib/EditorWindow.py --- a/Lib/idlelib/EditorWindow.py +++ b/Lib/idlelib/EditorWindow.py @@ -175,13 +175,13 @@ if macosxSupport.isAquaTk(): # Command-W on editorwindows doesn't work without this. text.bind('<>', self.close_event) - # Some OS X systems have only one mouse button, - # so use control-click for pulldown menus there. - # (Note, AquaTk defines <2> as the right button if - # present and the Tk Text widget already binds <2>.) + # Some OS X systems have only one mouse button, so use + # control-click for popup context menus there. For two + # buttons, AquaTk defines <2> as the right button, not <3>. text.bind("",self.right_menu_event) + text.bind("<2>", self.right_menu_event) else: - # Elsewhere, use right-click for pulldown menus. + # Elsewhere, use right-click for popup menus. text.bind("<3>",self.right_menu_event) text.bind("<>", self.cut) text.bind("<>", self.copy) diff --git a/Lib/idlelib/PyShell.py b/Lib/idlelib/PyShell.py --- a/Lib/idlelib/PyShell.py +++ b/Lib/idlelib/PyShell.py @@ -1547,6 +1547,14 @@ root.withdraw() flist = PyShellFileList(root) macosxSupport.setupApp(root, flist) + + if macosxSupport.isAquaTk(): + # There are some screwed up <2> class bindings for text + # widgets defined in Tk which we need to do away with. + # See issue #24801. + root.unbind_class('Text', '') + root.unbind_class('Text', '') + root.unbind_class('Text', '<>') if enable_edit: if not (cmd or script): -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 23 03:14:15 2015 From: python-checkins at python.org (terry.reedy) Date: Wed, 23 Sep 2015 01:14:15 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_whitespace?= Message-ID: <20150923011415.16573.93563@psf.io> https://hg.python.org/cpython/rev/b627f5145961 changeset: 98195:b627f5145961 branch: 2.7 parent: 98191:b79dd0d7dc98 user: Terry Jan Reedy date: Tue Sep 22 21:13:09 2015 -0400 summary: whitespace files: Lib/idlelib/PyShell.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Lib/idlelib/PyShell.py b/Lib/idlelib/PyShell.py --- a/Lib/idlelib/PyShell.py +++ b/Lib/idlelib/PyShell.py @@ -1556,7 +1556,7 @@ root.withdraw() flist = PyShellFileList(root) macosxSupport.setupApp(root, flist) - + if macosxSupport.isAquaTk(): # There are some screwed up <2> class bindings for text # widgets defined in Tk which we need to do away with. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 23 03:14:16 2015 From: python-checkins at python.org (terry.reedy) Date: Wed, 23 Sep 2015 01:14:16 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge_with_3=2E5?= Message-ID: <20150923011415.82666.85441@psf.io> https://hg.python.org/cpython/rev/c548859fd591 changeset: 98194:c548859fd591 parent: 98189:a200ab3e074b parent: 98193:ac35626f0f14 user: Terry Jan Reedy date: Tue Sep 22 21:11:06 2015 -0400 summary: Merge with 3.5 files: Lib/idlelib/EditorWindow.py | 10 +++++----- Lib/idlelib/PyShell.py | 8 ++++++++ 2 files changed, 13 insertions(+), 5 deletions(-) diff --git a/Lib/idlelib/EditorWindow.py b/Lib/idlelib/EditorWindow.py --- a/Lib/idlelib/EditorWindow.py +++ b/Lib/idlelib/EditorWindow.py @@ -175,13 +175,13 @@ if macosxSupport.isAquaTk(): # Command-W on editorwindows doesn't work without this. text.bind('<>', self.close_event) - # Some OS X systems have only one mouse button, - # so use control-click for pulldown menus there. - # (Note, AquaTk defines <2> as the right button if - # present and the Tk Text widget already binds <2>.) + # Some OS X systems have only one mouse button, so use + # control-click for popup context menus there. For two + # buttons, AquaTk defines <2> as the right button, not <3>. text.bind("",self.right_menu_event) + text.bind("<2>", self.right_menu_event) else: - # Elsewhere, use right-click for pulldown menus. + # Elsewhere, use right-click for popup menus. text.bind("<3>",self.right_menu_event) text.bind("<>", self.cut) text.bind("<>", self.copy) diff --git a/Lib/idlelib/PyShell.py b/Lib/idlelib/PyShell.py --- a/Lib/idlelib/PyShell.py +++ b/Lib/idlelib/PyShell.py @@ -1547,6 +1547,14 @@ root.withdraw() flist = PyShellFileList(root) macosxSupport.setupApp(root, flist) + + if macosxSupport.isAquaTk(): + # There are some screwed up <2> class bindings for text + # widgets defined in Tk which we need to do away with. + # See issue #24801. + root.unbind_class('Text', '') + root.unbind_class('Text', '') + root.unbind_class('Text', '<>') if enable_edit: if not (cmd or script): -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 23 03:14:17 2015 From: python-checkins at python.org (terry.reedy) Date: Wed, 23 Sep 2015 01:14:17 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge_with_3=2E5?= Message-ID: <20150923011415.31189.88940@psf.io> https://hg.python.org/cpython/rev/cad65e43c717 changeset: 98198:cad65e43c717 parent: 98194:c548859fd591 parent: 98197:ab08ec59c706 user: Terry Jan Reedy date: Tue Sep 22 21:13:53 2015 -0400 summary: Merge with 3.5 files: Lib/idlelib/PyShell.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Lib/idlelib/PyShell.py b/Lib/idlelib/PyShell.py --- a/Lib/idlelib/PyShell.py +++ b/Lib/idlelib/PyShell.py @@ -1547,7 +1547,7 @@ root.withdraw() flist = PyShellFileList(root) macosxSupport.setupApp(root, flist) - + if macosxSupport.isAquaTk(): # There are some screwed up <2> class bindings for text # widgets defined in Tk which we need to do away with. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 23 03:14:16 2015 From: python-checkins at python.org (terry.reedy) Date: Wed, 23 Sep 2015 01:14:16 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Merge_with_3=2E4?= Message-ID: <20150923011415.81625.73334@psf.io> https://hg.python.org/cpython/rev/ab08ec59c706 changeset: 98197:ab08ec59c706 branch: 3.5 parent: 98193:ac35626f0f14 parent: 98196:86dcaadc7957 user: Terry Jan Reedy date: Tue Sep 22 21:13:39 2015 -0400 summary: Merge with 3.4 files: Lib/idlelib/PyShell.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Lib/idlelib/PyShell.py b/Lib/idlelib/PyShell.py --- a/Lib/idlelib/PyShell.py +++ b/Lib/idlelib/PyShell.py @@ -1547,7 +1547,7 @@ root.withdraw() flist = PyShellFileList(root) macosxSupport.setupApp(root, flist) - + if macosxSupport.isAquaTk(): # There are some screwed up <2> class bindings for text # widgets defined in Tk which we need to do away with. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 23 03:21:40 2015 From: python-checkins at python.org (steve.dower) Date: Wed, 23 Sep 2015 01:21:40 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWVzICMyNTEx?= =?utf-8?q?2=3A_py=2Eexe_launcher_is_missing_icons?= Message-ID: <20150923012139.82666.581@psf.io> https://hg.python.org/cpython/rev/4d0d987bf6a8 changeset: 98199:4d0d987bf6a8 branch: 3.5 parent: 98197:ab08ec59c706 user: Steve Dower date: Tue Sep 22 18:20:58 2015 -0700 summary: Issues #25112: py.exe launcher is missing icons files: Misc/NEWS | 2 + PC/pylauncher.rc | 55 +++++++++++++++++------------------ 2 files changed, 29 insertions(+), 28 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -153,6 +153,8 @@ Windows ------- +- Issues #25112: py.exe launcher is missing icons + - Issue #25102: Windows installer does not precompile for -O or -OO. - Issue #25081: Makes Back button in installer go back to upgrade page when diff --git a/PC/pylauncher.rc b/PC/pylauncher.rc --- a/PC/pylauncher.rc +++ b/PC/pylauncher.rc @@ -1,51 +1,50 @@ #include -#define MS_WINDOWS -#include "..\Include\modsupport.h" -#include "..\Include\patchlevel.h" -#ifdef _DEBUG -# include "pythonnt_rc_d.h" -#else -# include "pythonnt_rc.h" -#endif +#include "python_ver_rc.h" -#define PYTHON_VERSION PY_VERSION "\0" -#define PYVERSION64 PY_MAJOR_VERSION, PY_MINOR_VERSION, FIELD3, PYTHON_API_VERSION +// Include the manifest file that indicates we support all +// current versions of Windows. +#include +1 RT_MANIFEST "python.manifest" + +1 ICON DISCARDABLE "launcher.ico" +2 ICON DISCARDABLE "py.ico" +3 ICON DISCARDABLE "pyc.ico" + +///////////////////////////////////////////////////////////////////////////// +// +// Version +// VS_VERSION_INFO VERSIONINFO FILEVERSION PYVERSION64 PRODUCTVERSION PYVERSION64 - FILEFLAGSMASK 0x17L + FILEFLAGSMASK 0x3fL #ifdef _DEBUG - FILEFLAGS 0x1L + FILEFLAGS VS_FF_DEBUG #else FILEFLAGS 0x0L #endif - FILEOS 0x4L - FILETYPE 0x1L + FILEOS VOS__WINDOWS32 + FILETYPE VFT_APP FILESUBTYPE 0x0L BEGIN BLOCK "StringFileInfo" BEGIN - BLOCK "080904b0" + BLOCK "000004b0" BEGIN - VALUE "Comments", "Python Launcher for Windows" - VALUE "CompanyName", "Python Software Foundation" - VALUE "FileDescription", "Python Launcher for Windows (Console)" + VALUE "CompanyName", PYTHON_COMPANY "\0" + VALUE "FileDescription", "Python\0" VALUE "FileVersion", PYTHON_VERSION - VALUE "InternalName", "py" - VALUE "LegalCopyright", "Copyright (C) 2011-2014 Python Software Foundation" - VALUE "OriginalFilename", "py" - VALUE "ProductName", "Python Launcher for Windows" + VALUE "InternalName", "Python Launcher\0" + VALUE "LegalCopyright", PYTHON_COPYRIGHT "\0" + VALUE "OriginalFilename", "py" PYTHON_DEBUG_EXT ".exe\0" + VALUE "ProductName", "Python\0" VALUE "ProductVersion", PYTHON_VERSION END END BLOCK "VarFileInfo" BEGIN - VALUE "Translation", 0x809, 1200 + VALUE "Translation", 0x0, 1200 END -END - -IDI_ICON1 ICON "launcher.ico" - - +END \ No newline at end of file -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 23 03:21:40 2015 From: python-checkins at python.org (steve.dower) Date: Wed, 23 Sep 2015 01:21:40 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge_from_3=2E5?= Message-ID: <20150923012140.82640.68659@psf.io> https://hg.python.org/cpython/rev/1438ac605924 changeset: 98200:1438ac605924 parent: 98198:cad65e43c717 parent: 98199:4d0d987bf6a8 user: Steve Dower date: Tue Sep 22 18:21:13 2015 -0700 summary: Merge from 3.5 files: Misc/NEWS | 2 + PC/pylauncher.rc | 55 +++++++++++++++++------------------ 2 files changed, 29 insertions(+), 28 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -253,6 +253,8 @@ Windows ------- +- Issues #25112: py.exe launcher is missing icons + - Issue #25102: Windows installer does not precompile for -O or -OO. - Issue #25081: Makes Back button in installer go back to upgrade page when diff --git a/PC/pylauncher.rc b/PC/pylauncher.rc --- a/PC/pylauncher.rc +++ b/PC/pylauncher.rc @@ -1,51 +1,50 @@ #include -#define MS_WINDOWS -#include "..\Include\modsupport.h" -#include "..\Include\patchlevel.h" -#ifdef _DEBUG -# include "pythonnt_rc_d.h" -#else -# include "pythonnt_rc.h" -#endif +#include "python_ver_rc.h" -#define PYTHON_VERSION PY_VERSION "\0" -#define PYVERSION64 PY_MAJOR_VERSION, PY_MINOR_VERSION, FIELD3, PYTHON_API_VERSION +// Include the manifest file that indicates we support all +// current versions of Windows. +#include +1 RT_MANIFEST "python.manifest" + +1 ICON DISCARDABLE "launcher.ico" +2 ICON DISCARDABLE "py.ico" +3 ICON DISCARDABLE "pyc.ico" + +///////////////////////////////////////////////////////////////////////////// +// +// Version +// VS_VERSION_INFO VERSIONINFO FILEVERSION PYVERSION64 PRODUCTVERSION PYVERSION64 - FILEFLAGSMASK 0x17L + FILEFLAGSMASK 0x3fL #ifdef _DEBUG - FILEFLAGS 0x1L + FILEFLAGS VS_FF_DEBUG #else FILEFLAGS 0x0L #endif - FILEOS 0x4L - FILETYPE 0x1L + FILEOS VOS__WINDOWS32 + FILETYPE VFT_APP FILESUBTYPE 0x0L BEGIN BLOCK "StringFileInfo" BEGIN - BLOCK "080904b0" + BLOCK "000004b0" BEGIN - VALUE "Comments", "Python Launcher for Windows" - VALUE "CompanyName", "Python Software Foundation" - VALUE "FileDescription", "Python Launcher for Windows (Console)" + VALUE "CompanyName", PYTHON_COMPANY "\0" + VALUE "FileDescription", "Python\0" VALUE "FileVersion", PYTHON_VERSION - VALUE "InternalName", "py" - VALUE "LegalCopyright", "Copyright (C) 2011-2014 Python Software Foundation" - VALUE "OriginalFilename", "py" - VALUE "ProductName", "Python Launcher for Windows" + VALUE "InternalName", "Python Launcher\0" + VALUE "LegalCopyright", PYTHON_COPYRIGHT "\0" + VALUE "OriginalFilename", "py" PYTHON_DEBUG_EXT ".exe\0" + VALUE "ProductName", "Python\0" VALUE "ProductVersion", PYTHON_VERSION END END BLOCK "VarFileInfo" BEGIN - VALUE "Translation", 0x809, 1200 + VALUE "Translation", 0x0, 1200 END -END - -IDI_ICON1 ICON "launcher.ico" - - +END \ No newline at end of file -- Repository URL: https://hg.python.org/cpython From tjreedy at udel.edu Wed Sep 23 03:04:59 2015 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 22 Sep 2015 21:04:59 -0400 Subject: [Python-checkins] cpython (3.5): Issue #25102: Windows installer does not precompile for -O or -OO. In-Reply-To: <20150923000147.3654.99695@psf.io> References: <20150923000147.3654.99695@psf.io> Message-ID: <5601FABB.1090605@udel.edu> On 9/22/2015 8:01 PM, steve.dower wrote: > https://hg.python.org/cpython/rev/31b230e5517e > changeset: 98179:31b230e5517e > branch: 3.5 > user: Steve Dower > date: Tue Sep 22 16:45:19 2015 -0700 > summary: > Issue #25102: Windows installer does not precompile for -O or -OO. I believe this should say 'now precompiles'. Before you came on board, we agreed that to avoid confusion, commit messages should be written to be true after the patch. From python-checkins at python.org Wed Sep 23 04:08:28 2015 From: python-checkins at python.org (martin.panter) Date: Wed, 23 Sep 2015 02:08:28 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzI1MDQ3?= =?utf-8?q?=3A_Respect_case_writing_XML_encoding_declarations?= Message-ID: <20150923020828.82640.55714@psf.io> https://hg.python.org/cpython/rev/ff7aba08ada6 changeset: 98201:ff7aba08ada6 branch: 3.4 parent: 98196:86dcaadc7957 user: Martin Panter date: Wed Sep 23 01:14:35 2015 +0000 summary: Issue #25047: Respect case writing XML encoding declarations This restores the ability to write encoding names in uppercase like "UTF-8", which worked in Python 2. files: Lib/test/test_xml_etree.py | 21 ++++++++++++++------- Lib/xml/etree/ElementTree.py | 9 ++++----- Misc/NEWS | 4 ++++ 3 files changed, 22 insertions(+), 12 deletions(-) diff --git a/Lib/test/test_xml_etree.py b/Lib/test/test_xml_etree.py --- a/Lib/test/test_xml_etree.py +++ b/Lib/test/test_xml_etree.py @@ -2396,14 +2396,21 @@ elem = ET.Element("tag") elem.text = "abc" self.assertEqual(serialize(elem), 'abc') - self.assertEqual(serialize(elem, encoding="utf-8"), - b'abc') - self.assertEqual(serialize(elem, encoding="us-ascii"), - b'abc') + for enc in ("utf-8", "us-ascii"): + with self.subTest(enc): + self.assertEqual(serialize(elem, encoding=enc), + b'abc') + self.assertEqual(serialize(elem, encoding=enc.upper()), + b'abc') for enc in ("iso-8859-1", "utf-16", "utf-32"): - self.assertEqual(serialize(elem, encoding=enc), - ("\n" - "abc" % enc).encode(enc)) + with self.subTest(enc): + self.assertEqual(serialize(elem, encoding=enc), + ("\n" + "abc" % enc).encode(enc)) + upper = enc.upper() + self.assertEqual(serialize(elem, encoding=upper), + ("\n" + "abc" % upper).encode(enc)) elem = ET.Element("tag") elem.text = "<&\"\'>" diff --git a/Lib/xml/etree/ElementTree.py b/Lib/xml/etree/ElementTree.py --- a/Lib/xml/etree/ElementTree.py +++ b/Lib/xml/etree/ElementTree.py @@ -756,14 +756,13 @@ encoding = "utf-8" else: encoding = "us-ascii" - else: - encoding = encoding.lower() - with _get_writer(file_or_filename, encoding) as write: + enc_lower = encoding.lower() + with _get_writer(file_or_filename, enc_lower) as write: if method == "xml" and (xml_declaration or (xml_declaration is None and - encoding not in ("utf-8", "us-ascii", "unicode"))): + enc_lower not in ("utf-8", "us-ascii", "unicode"))): declared_encoding = encoding - if encoding == "unicode": + if enc_lower == "unicode": # Retrieve the default encoding for the xml declaration import locale declared_encoding = locale.getpreferredencoding() diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -81,6 +81,10 @@ Library ------- +- Issue #25047: The XML encoding declaration written by Element Tree now + respects the letter case given by the user. This restores the ability to + write encoding names in uppercase like "UTF-8", which worked in Python 2. + - Issue #19143: platform module now reads Windows version from kernel32.dll to avoid compatibility shims. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 23 04:08:28 2015 From: python-checkins at python.org (martin.panter) Date: Wed, 23 Sep 2015 02:08:28 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2325047=3A_Merge_Element_Tree_encoding_from_3=2E5?= Message-ID: <20150923020828.82656.91874@psf.io> https://hg.python.org/cpython/rev/409bab2181d3 changeset: 98203:409bab2181d3 parent: 98200:1438ac605924 parent: 98202:9c248233754c user: Martin Panter date: Wed Sep 23 01:49:24 2015 +0000 summary: Issue #25047: Merge Element Tree encoding from 3.5 files: Lib/test/test_xml_etree.py | 21 ++++++++++++++------- Lib/xml/etree/ElementTree.py | 9 ++++----- Misc/NEWS | 4 ++++ 3 files changed, 22 insertions(+), 12 deletions(-) diff --git a/Lib/test/test_xml_etree.py b/Lib/test/test_xml_etree.py --- a/Lib/test/test_xml_etree.py +++ b/Lib/test/test_xml_etree.py @@ -2396,14 +2396,21 @@ elem = ET.Element("tag") elem.text = "abc" self.assertEqual(serialize(elem), 'abc') - self.assertEqual(serialize(elem, encoding="utf-8"), - b'abc') - self.assertEqual(serialize(elem, encoding="us-ascii"), - b'abc') + for enc in ("utf-8", "us-ascii"): + with self.subTest(enc): + self.assertEqual(serialize(elem, encoding=enc), + b'abc') + self.assertEqual(serialize(elem, encoding=enc.upper()), + b'abc') for enc in ("iso-8859-1", "utf-16", "utf-32"): - self.assertEqual(serialize(elem, encoding=enc), - ("\n" - "abc" % enc).encode(enc)) + with self.subTest(enc): + self.assertEqual(serialize(elem, encoding=enc), + ("\n" + "abc" % enc).encode(enc)) + upper = enc.upper() + self.assertEqual(serialize(elem, encoding=upper), + ("\n" + "abc" % upper).encode(enc)) elem = ET.Element("tag") elem.text = "<&\"\'>" diff --git a/Lib/xml/etree/ElementTree.py b/Lib/xml/etree/ElementTree.py --- a/Lib/xml/etree/ElementTree.py +++ b/Lib/xml/etree/ElementTree.py @@ -752,14 +752,13 @@ encoding = "utf-8" else: encoding = "us-ascii" - else: - encoding = encoding.lower() - with _get_writer(file_or_filename, encoding) as write: + enc_lower = encoding.lower() + with _get_writer(file_or_filename, enc_lower) as write: if method == "xml" and (xml_declaration or (xml_declaration is None and - encoding not in ("utf-8", "us-ascii", "unicode"))): + enc_lower not in ("utf-8", "us-ascii", "unicode"))): declared_encoding = encoding - if encoding == "unicode": + if enc_lower == "unicode": # Retrieve the default encoding for the xml declaration import locale declared_encoding = locale.getpreferredencoding() diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -138,6 +138,10 @@ Library ------- +- Issue #25047: The XML encoding declaration written by Element Tree now + respects the letter case given by the user. This restores the ability to + write encoding names in uppercase like "UTF-8", which worked in Python 2. + - Issue #19143: platform module now reads Windows version from kernel32.dll to avoid compatibility shims. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 23 04:08:28 2015 From: python-checkins at python.org (martin.panter) Date: Wed, 23 Sep 2015 02:08:28 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Issue_=2325047=3A_Merge_Element_Tree_encoding_from_3=2E4_into_?= =?utf-8?b?My41?= Message-ID: <20150923020828.81635.8769@psf.io> https://hg.python.org/cpython/rev/9c248233754c changeset: 98202:9c248233754c branch: 3.5 parent: 98199:4d0d987bf6a8 parent: 98201:ff7aba08ada6 user: Martin Panter date: Wed Sep 23 01:43:08 2015 +0000 summary: Issue #25047: Merge Element Tree encoding from 3.4 into 3.5 files: Lib/test/test_xml_etree.py | 21 ++++++++++++++------- Lib/xml/etree/ElementTree.py | 9 ++++----- Misc/NEWS | 4 ++++ 3 files changed, 22 insertions(+), 12 deletions(-) diff --git a/Lib/test/test_xml_etree.py b/Lib/test/test_xml_etree.py --- a/Lib/test/test_xml_etree.py +++ b/Lib/test/test_xml_etree.py @@ -2396,14 +2396,21 @@ elem = ET.Element("tag") elem.text = "abc" self.assertEqual(serialize(elem), 'abc') - self.assertEqual(serialize(elem, encoding="utf-8"), - b'abc') - self.assertEqual(serialize(elem, encoding="us-ascii"), - b'abc') + for enc in ("utf-8", "us-ascii"): + with self.subTest(enc): + self.assertEqual(serialize(elem, encoding=enc), + b'abc') + self.assertEqual(serialize(elem, encoding=enc.upper()), + b'abc') for enc in ("iso-8859-1", "utf-16", "utf-32"): - self.assertEqual(serialize(elem, encoding=enc), - ("\n" - "abc" % enc).encode(enc)) + with self.subTest(enc): + self.assertEqual(serialize(elem, encoding=enc), + ("\n" + "abc" % enc).encode(enc)) + upper = enc.upper() + self.assertEqual(serialize(elem, encoding=upper), + ("\n" + "abc" % upper).encode(enc)) elem = ET.Element("tag") elem.text = "<&\"\'>" diff --git a/Lib/xml/etree/ElementTree.py b/Lib/xml/etree/ElementTree.py --- a/Lib/xml/etree/ElementTree.py +++ b/Lib/xml/etree/ElementTree.py @@ -752,14 +752,13 @@ encoding = "utf-8" else: encoding = "us-ascii" - else: - encoding = encoding.lower() - with _get_writer(file_or_filename, encoding) as write: + enc_lower = encoding.lower() + with _get_writer(file_or_filename, enc_lower) as write: if method == "xml" and (xml_declaration or (xml_declaration is None and - encoding not in ("utf-8", "us-ascii", "unicode"))): + enc_lower not in ("utf-8", "us-ascii", "unicode"))): declared_encoding = encoding - if encoding == "unicode": + if enc_lower == "unicode": # Retrieve the default encoding for the xml declaration import locale declared_encoding = locale.getpreferredencoding() diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -18,6 +18,10 @@ Library ------- +- Issue #25047: The XML encoding declaration written by Element Tree now + respects the letter case given by the user. This restores the ability to + write encoding names in uppercase like "UTF-8", which worked in Python 2. + - Issue #19143: platform module now reads Windows version from kernel32.dll to avoid compatibility shims. -- Repository URL: https://hg.python.org/cpython From tjreedy at udel.edu Wed Sep 23 03:24:36 2015 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 22 Sep 2015 21:24:36 -0400 Subject: [Python-checkins] cpython (3.4): Issue #24570: Right-click for context menus now work on Mac Aqual also. In-Reply-To: <20150923011414.9937.52702@psf.io> References: <20150923011414.9937.52702@psf.io> Message-ID: <5601FF53.1070605@udel.edu> On 9/22/2015 9:14 PM, terry.reedy wrote: > https://hg.python.org/cpython/rev/51b2b1a821b7 > changeset: 98192:51b2b1a821b7 > branch: 3.4 > parent: 98187:a8dc0b6e2b63 > user: Terry Jan Reedy > date: Tue Sep 22 21:10:27 2015 -0400 > summary: > Issue #24570: Right-click for context menus now work on Mac Aqual also. > Patch by Mark Roseman. Actually #24801. Moved commit messages and will fix for news entry. From python-checkins at python.org Wed Sep 23 05:00:26 2015 From: python-checkins at python.org (terry.reedy) Date: Wed, 23 Sep 2015 03:00:26 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzE2ODkz?= =?utf-8?q?=3A_Move_idlelib=2EEditorWindow=2EHelpDialog_deprecation_warnin?= =?utf-8?q?g?= Message-ID: <20150923030026.94119.12680@psf.io> https://hg.python.org/cpython/rev/c607004a98bf changeset: 98205:c607004a98bf branch: 3.4 parent: 98201:ff7aba08ada6 user: Terry Jan Reedy date: Tue Sep 22 22:59:40 2015 -0400 summary: Issue #16893: Move idlelib.EditorWindow.HelpDialog deprecation warning so it is not triggered on import. The problem is creation of a now-unused instance "helpDialog = HelpDialog()", left for back compatibility. So instead trigger the warning when that instance or another is used. files: Lib/idlelib/EditorWindow.py | 10 +++++----- 1 files changed, 5 insertions(+), 5 deletions(-) diff --git a/Lib/idlelib/EditorWindow.py b/Lib/idlelib/EditorWindow.py --- a/Lib/idlelib/EditorWindow.py +++ b/Lib/idlelib/EditorWindow.py @@ -43,11 +43,6 @@ class HelpDialog(object): def __init__(self): - import warnings as w - w.warn("EditorWindow.HelpDialog is no longer used by Idle.\n" - "It will be removed in 3.6 or later.\n" - "It has been replaced by private help.HelpWindow\n", - DeprecationWarning, stacklevel=2) self.parent = None # parent of help window self.dlg = None # the help window iteself @@ -59,6 +54,11 @@ near - a Toplevel widget (e.g. EditorWindow or PyShell) to use as a reference for placing the help window """ + import warnings as w + w.warn("EditorWindow.HelpDialog is no longer used by Idle.\n" + "It will be removed in 3.6 or later.\n" + "It has been replaced by private help.HelpWindow\n", + DeprecationWarning, stacklevel=2) if self.dlg is None: self.show_dialog(parent) if near: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 23 05:00:27 2015 From: python-checkins at python.org (terry.reedy) Date: Wed, 23 Sep 2015 03:00:27 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzE2ODkz?= =?utf-8?q?=3A_Move_idlelib=2EEditorWindow=2EHelpDialog_deprecation_warnin?= =?utf-8?q?g?= Message-ID: <20150923030026.9939.75997@psf.io> https://hg.python.org/cpython/rev/26e819909891 changeset: 98204:26e819909891 branch: 2.7 parent: 98195:b627f5145961 user: Terry Jan Reedy date: Tue Sep 22 22:59:35 2015 -0400 summary: Issue #16893: Move idlelib.EditorWindow.HelpDialog deprecation warning so it is not triggered on import. The problem is creation of a now-unused instance "helpDialog = HelpDialog()", left for back compatibility. So instead trigger the warning when that instance or another is used. files: Lib/idlelib/EditorWindow.py | 10 +++++----- 1 files changed, 5 insertions(+), 5 deletions(-) diff --git a/Lib/idlelib/EditorWindow.py b/Lib/idlelib/EditorWindow.py --- a/Lib/idlelib/EditorWindow.py +++ b/Lib/idlelib/EditorWindow.py @@ -72,11 +72,6 @@ class HelpDialog(object): def __init__(self): - import warnings as w - w.warn("EditorWindow.HelpDialog is no longer used by Idle.\n" - "It will be removed in 3.6 or later.\n" - "It has been replaced by private help.HelpWindow\n", - DeprecationWarning, stacklevel=2) self.parent = None # parent of help window self.dlg = None # the help window iteself @@ -88,6 +83,11 @@ near - a Toplevel widget (e.g. EditorWindow or PyShell) to use as a reference for placing the help window """ + import warnings as w + w.warn("EditorWindow.HelpDialog is no longer used by Idle.\n" + "It will be removed in 3.6 or later.\n" + "It has been replaced by private help.HelpWindow\n", + DeprecationWarning, stacklevel=2) if self.dlg is None: self.show_dialog(parent) if near: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 23 05:00:29 2015 From: python-checkins at python.org (terry.reedy) Date: Wed, 23 Sep 2015 03:00:29 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge_with_3=2E5?= Message-ID: <20150923030026.3650.59815@psf.io> https://hg.python.org/cpython/rev/47fe144fc24a changeset: 98207:47fe144fc24a parent: 98203:409bab2181d3 parent: 98206:16b9207225d4 user: Terry Jan Reedy date: Tue Sep 22 23:00:07 2015 -0400 summary: Merge with 3.5 files: Lib/idlelib/EditorWindow.py | 10 +++++----- 1 files changed, 5 insertions(+), 5 deletions(-) diff --git a/Lib/idlelib/EditorWindow.py b/Lib/idlelib/EditorWindow.py --- a/Lib/idlelib/EditorWindow.py +++ b/Lib/idlelib/EditorWindow.py @@ -43,11 +43,6 @@ class HelpDialog(object): def __init__(self): - import warnings as w - w.warn("EditorWindow.HelpDialog is no longer used by Idle.\n" - "It will be removed in 3.6 or later.\n" - "It has been replaced by private help.HelpWindow\n", - DeprecationWarning, stacklevel=2) self.parent = None # parent of help window self.dlg = None # the help window iteself @@ -59,6 +54,11 @@ near - a Toplevel widget (e.g. EditorWindow or PyShell) to use as a reference for placing the help window """ + import warnings as w + w.warn("EditorWindow.HelpDialog is no longer used by Idle.\n" + "It will be removed in 3.6 or later.\n" + "It has been replaced by private help.HelpWindow\n", + DeprecationWarning, stacklevel=2) if self.dlg is None: self.show_dialog(parent) if near: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 23 05:00:31 2015 From: python-checkins at python.org (terry.reedy) Date: Wed, 23 Sep 2015 03:00:31 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Merge_with_3=2E4?= Message-ID: <20150923030026.11688.52634@psf.io> https://hg.python.org/cpython/rev/16b9207225d4 changeset: 98206:16b9207225d4 branch: 3.5 parent: 98202:9c248233754c parent: 98205:c607004a98bf user: Terry Jan Reedy date: Tue Sep 22 22:59:53 2015 -0400 summary: Merge with 3.4 files: Lib/idlelib/EditorWindow.py | 10 +++++----- 1 files changed, 5 insertions(+), 5 deletions(-) diff --git a/Lib/idlelib/EditorWindow.py b/Lib/idlelib/EditorWindow.py --- a/Lib/idlelib/EditorWindow.py +++ b/Lib/idlelib/EditorWindow.py @@ -43,11 +43,6 @@ class HelpDialog(object): def __init__(self): - import warnings as w - w.warn("EditorWindow.HelpDialog is no longer used by Idle.\n" - "It will be removed in 3.6 or later.\n" - "It has been replaced by private help.HelpWindow\n", - DeprecationWarning, stacklevel=2) self.parent = None # parent of help window self.dlg = None # the help window iteself @@ -59,6 +54,11 @@ near - a Toplevel widget (e.g. EditorWindow or PyShell) to use as a reference for placing the help window """ + import warnings as w + w.warn("EditorWindow.HelpDialog is no longer used by Idle.\n" + "It will be removed in 3.6 or later.\n" + "It has been replaced by private help.HelpWindow\n", + DeprecationWarning, stacklevel=2) if self.dlg is None: self.show_dialog(parent) if near: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 23 07:53:35 2015 From: python-checkins at python.org (martin.panter) Date: Wed, 23 Sep 2015 05:53:35 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Issue_=2312067=3A_Merge_comparisons_doc_from_3=2E4_into_3=2E5?= Message-ID: <20150923055335.3646.96388@psf.io> https://hg.python.org/cpython/rev/b6698c00265b changeset: 98209:b6698c00265b branch: 3.5 parent: 98206:16b9207225d4 parent: 98208:1fc049e5ec14 user: Martin Panter date: Wed Sep 23 05:34:48 2015 +0000 summary: Issue #12067: Merge comparisons doc from 3.4 into 3.5 files: Doc/reference/expressions.rst | 236 +++++++++++++++++---- Misc/NEWS | 7 + 2 files changed, 191 insertions(+), 52 deletions(-) diff --git a/Doc/reference/expressions.rst b/Doc/reference/expressions.rst --- a/Doc/reference/expressions.rst +++ b/Doc/reference/expressions.rst @@ -1036,10 +1036,6 @@ .. _comparisons: -.. _is: -.. _is not: -.. _in: -.. _not in: Comparisons =========== @@ -1075,66 +1071,183 @@ *c*, so that, e.g., ``x < y > z`` is perfectly legal (though perhaps not pretty). +Value comparisons +----------------- + The operators ``<``, ``>``, ``==``, ``>=``, ``<=``, and ``!=`` compare the -values of two objects. The objects need not have the same type. If both are -numbers, they are converted to a common type. Otherwise, the ``==`` and ``!=`` -operators *always* consider objects of different types to be unequal, while the -``<``, ``>``, ``>=`` and ``<=`` operators raise a :exc:`TypeError` when -comparing objects of different types that do not implement these operators for -the given pair of types. You can control comparison behavior of objects of -non-built-in types by defining rich comparison methods like :meth:`__gt__`, -described in section :ref:`customization`. +values of two objects. The objects do not need to have the same type. -Comparison of objects of the same type depends on the type: +Chapter :ref:`objects` states that objects have a value (in addition to type +and identity). The value of an object is a rather abstract notion in Python: +For example, there is no canonical access method for an object's value. Also, +there is no requirement that the value of an object should be constructed in a +particular way, e.g. comprised of all its data attributes. Comparison operators +implement a particular notion of what the value of an object is. One can think +of them as defining the value of an object indirectly, by means of their +comparison implementation. -* Numbers are compared arithmetically. +Because all types are (direct or indirect) subtypes of :class:`object`, they +inherit the default comparison behavior from :class:`object`. Types can +customize their comparison behavior by implementing +:dfn:`rich comparison methods` like :meth:`__lt__`, described in +:ref:`customization`. -* The values :const:`float('NaN')` and :const:`Decimal('NaN')` are special. - They are identical to themselves, ``x is x`` but are not equal to themselves, - ``x != x``. Additionally, comparing any value to a not-a-number value +The default behavior for equality comparison (``==`` and ``!=``) is based on +the identity of the objects. Hence, equality comparison of instances with the +same identity results in equality, and equality comparison of instances with +different identities results in inequality. A motivation for this default +behavior is the desire that all objects should be reflexive (i.e. ``x is y`` +implies ``x == y``). + +A default order comparison (``<``, ``>``, ``<=``, and ``>=``) is not provided; +an attempt raises :exc:`TypeError`. A motivation for this default behavior is +the lack of a similar invariant as for equality. + +The behavior of the default equality comparison, that instances with different +identities are always unequal, may be in contrast to what types will need that +have a sensible definition of object value and value-based equality. Such +types will need to customize their comparison behavior, and in fact, a number +of built-in types have done that. + +The following list describes the comparison behavior of the most important +built-in types. + +* Numbers of built-in numeric types (:ref:`typesnumeric`) and of the standard + library types :class:`fractions.Fraction` and :class:`decimal.Decimal` can be + compared within and across their types, with the restriction that complex + numbers do not support order comparison. Within the limits of the types + involved, they compare mathematically (algorithmically) correct without loss + of precision. + + The not-a-number values :const:`float('NaN')` and :const:`Decimal('NaN')` + are special. They are identical to themselves (``x is x`` is true) but + are not equal to themselves (``x == x`` is false). Additionally, + comparing any number to a not-a-number value will return ``False``. For example, both ``3 < float('NaN')`` and ``float('NaN') < 3`` will return ``False``. -* Bytes objects are compared lexicographically using the numeric values of their - elements. +* Binary sequences (instances of :class:`bytes` or :class:`bytearray`) can be + compared within and across their types. They compare lexicographically using + the numeric values of their elements. -* Strings are compared lexicographically using the numeric equivalents (the - result of the built-in function :func:`ord`) of their characters. [#]_ String - and bytes object can't be compared! +* Strings (instances of :class:`str`) compare lexicographically using the + numerical Unicode code points (the result of the built-in function + :func:`ord`) of their characters. [#]_ -* Tuples and lists are compared lexicographically using comparison of - corresponding elements. This means that to compare equal, each element must - compare equal and the two sequences must be of the same type and have the same - length. + Strings and binary sequences cannot be directly compared. - If not equal, the sequences are ordered the same as their first differing - elements. For example, ``[1,2,x] <= [1,2,y]`` has the same value as - ``x <= y``. If the corresponding element does not exist, the shorter - sequence is ordered first (for example, ``[1,2] < [1,2,3]``). +* Sequences (instances of :class:`tuple`, :class:`list`, or :class:`range`) can + be compared only within each of their types, with the restriction that ranges + do not support order comparison. Equality comparison across these types + results in unequality, and ordering comparison across these types raises + :exc:`TypeError`. -* Mappings (dictionaries) compare equal if and only if they have the same - ``(key, value)`` pairs. Order comparisons ``('<', '<=', '>=', '>')`` - raise :exc:`TypeError`. + Sequences compare lexicographically using comparison of corresponding + elements, whereby reflexivity of the elements is enforced. -* Sets and frozensets define comparison operators to mean subset and superset - tests. Those relations do not define total orderings (the two sets ``{1,2}`` - and ``{2,3}`` are not equal, nor subsets of one another, nor supersets of one + In enforcing reflexivity of elements, the comparison of collections assumes + that for a collection element ``x``, ``x == x`` is always true. Based on + that assumption, element identity is compared first, and element comparison + is performed only for distinct elements. This approach yields the same + result as a strict element comparison would, if the compared elements are + reflexive. For non-reflexive elements, the result is different than for + strict element comparison, and may be surprising: The non-reflexive + not-a-number values for example result in the following comparison behavior + when used in a list:: + + >>> nan = float('NaN') + >>> nan is nan + True + >>> nan == nan + False <-- the defined non-reflexive behavior of NaN + >>> [nan] == [nan] + True <-- list enforces reflexivity and tests identity first + + Lexicographical comparison between built-in collections works as follows: + + - For two collections to compare equal, they must be of the same type, have + the same length, and each pair of corresponding elements must compare + equal (for example, ``[1,2] == (1,2)`` is false because the type is not the + same). + + - Collections that support order comparison are ordered the same as their + first unequal elements (for example, ``[1,2,x] <= [1,2,y]`` has the same + value as ``x <= y``). If a corresponding element does not exist, the + shorter collection is ordered first (for example, ``[1,2] < [1,2,3]`` is + true). + +* Mappings (instances of :class:`dict`) compare equal if and only if they have + equal `(key, value)` pairs. Equality comparison of the keys and elements + enforces reflexivity. + + Order comparisons (``<``, ``>``, ``<=``, and ``>=``) raise :exc:`TypeError`. + +* Sets (instances of :class:`set` or :class:`frozenset`) can be compared within + and across their types. + + They define order + comparison operators to mean subset and superset tests. Those relations do + not define total orderings (for example, the two sets ``{1,2}`` and ``{2,3}`` + are not equal, nor subsets of one another, nor supersets of one another). Accordingly, sets are not appropriate arguments for functions - which depend on total ordering. For example, :func:`min`, :func:`max`, and - :func:`sorted` produce undefined results given a list of sets as inputs. + which depend on total ordering (for example, :func:`min`, :func:`max`, and + :func:`sorted` produce undefined results given a list of sets as inputs). -* Most other objects of built-in types compare unequal unless they are the same - object; the choice whether one object is considered smaller or larger than - another one is made arbitrarily but consistently within one execution of a - program. + Comparison of sets enforces reflexivity of its elements. -Comparison of objects of differing types depends on whether either of the -types provide explicit support for the comparison. Most numeric types can be -compared with one another. When cross-type comparison is not supported, the -comparison method returns ``NotImplemented``. +* Most other built-in types have no comparison methods implemented, so they + inherit the default comparison behavior. +User-defined classes that customize their comparison behavior should follow +some consistency rules, if possible: + +* Equality comparison should be reflexive. + In other words, identical objects should compare equal: + + ``x is y`` implies ``x == y`` + +* Comparison should be symmetric. + In other words, the following expressions should have the same result: + + ``x == y`` and ``y == x`` + + ``x != y`` and ``y != x`` + + ``x < y`` and ``y > x`` + + ``x <= y`` and ``y >= x`` + +* Comparison should be transitive. + The following (non-exhaustive) examples illustrate that: + + ``x > y and y > z`` implies ``x > z`` + + ``x < y and y <= z`` implies ``x < z`` + +* Inverse comparison should result in the boolean negation. + In other words, the following expressions should have the same result: + + ``x == y`` and ``not x != y`` + + ``x < y`` and ``not x >= y`` (for total ordering) + + ``x > y`` and ``not x <= y`` (for total ordering) + + The last two expressions apply to totally ordered collections (e.g. to + sequences, but not to sets or mappings). See also the + :func:`~functools.total_ordering` decorator. + +Python does not enforce these consistency rules. In fact, the not-a-number +values are an example for not following these rules. + + +.. _in: +.. _not in: .. _membership-test-details: +Membership test operations +-------------------------- + The operators :keyword:`in` and :keyword:`not in` test for membership. ``x in s`` evaluates to true if *x* is a member of *s*, and false otherwise. ``x not in s`` returns the negation of ``x in s``. All built-in sequences and set types @@ -1176,6 +1289,13 @@ operator: is not pair: identity; test + +.. _is: +.. _is not: + +Identity comparisons +-------------------- + The operators :keyword:`is` and :keyword:`is not` test for object identity: ``x is y`` is true if and only if *x* and *y* are the same object. ``x is not y`` yields the inverse truth value. [#]_ @@ -1405,12 +1525,24 @@ cases, Python returns the latter result, in order to preserve that ``divmod(x,y)[0] * y + x % y`` be very close to ``x``. -.. [#] While comparisons between strings make sense at the byte level, they may - be counter-intuitive to users. For example, the strings ``"\u00C7"`` and - ``"\u0043\u0327"`` compare differently, even though they both represent the - same unicode character (LATIN CAPITAL LETTER C WITH CEDILLA). To compare - strings in a human recognizable way, compare using - :func:`unicodedata.normalize`. +.. [#] The Unicode standard distinguishes between :dfn:`code points` + (e.g. U+0041) and :dfn:`abstract characters` (e.g. "LATIN CAPITAL LETTER A"). + While most abstract characters in Unicode are only represented using one + code point, there is a number of abstract characters that can in addition be + represented using a sequence of more than one code point. For example, the + abstract character "LATIN CAPITAL LETTER C WITH CEDILLA" can be represented + as a single :dfn:`precomposed character` at code position U+00C7, or as a + sequence of a :dfn:`base character` at code position U+0043 (LATIN CAPITAL + LETTER C), followed by a :dfn:`combining character` at code position U+0327 + (COMBINING CEDILLA). + + The comparison operators on strings compare at the level of Unicode code + points. This may be counter-intuitive to humans. For example, + ``"\u00C7" == "\u0043\u0327"`` is ``False``, even though both strings + represent the same abstract character "LATIN CAPITAL LETTER C WITH CEDILLA". + + To compare strings at the level of abstract characters (that is, in a way + intuitive to humans), use :func:`unicodedata.normalize`. .. [#] Due to automatic garbage-collection, free lists, and the dynamic nature of descriptors, you may notice seemingly unusual behaviour in certain uses of diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -125,6 +125,13 @@ Documentation ------------- +- Issue #12067: Rewrite Comparisons section in the Expressions chapter of the + language reference. Some of the details of comparing mixed types were + incorrect or ambiguous. NotImplemented is only relevant at a lower level + than the Expressions chapter. Added details of comparing range() objects, + and default behaviour and consistency suggestions for user-defined classes. + Patch from Andy Maier. + - Issue #24952: Clarify the default size argument of stack_size() in the "threading" and "_thread" modules. Patch from Mattip. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 23 07:53:35 2015 From: python-checkins at python.org (martin.panter) Date: Wed, 23 Sep 2015 05:53:35 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzEyMDY3?= =?utf-8?q?=3A_Rewrite_Comparisons_section_in_the_language_reference?= Message-ID: <20150923055335.11692.86332@psf.io> https://hg.python.org/cpython/rev/1fc049e5ec14 changeset: 98208:1fc049e5ec14 branch: 3.4 parent: 98205:c607004a98bf user: Martin Panter date: Wed Sep 23 05:28:13 2015 +0000 summary: Issue #12067: Rewrite Comparisons section in the language reference Some of the details of comparing mixed types were incorrect or ambiguous. NotImplemented is only relevant at a lower level than the Expressions chapter. Added details of comparing range() objects, and default behaviour and consistency suggestions for user-defined classes. Patch from Andy Maier. files: Doc/reference/expressions.rst | 236 +++++++++++++++++---- Misc/NEWS | 7 + 2 files changed, 191 insertions(+), 52 deletions(-) diff --git a/Doc/reference/expressions.rst b/Doc/reference/expressions.rst --- a/Doc/reference/expressions.rst +++ b/Doc/reference/expressions.rst @@ -1013,10 +1013,6 @@ .. _comparisons: -.. _is: -.. _is not: -.. _in: -.. _not in: Comparisons =========== @@ -1052,66 +1048,183 @@ *c*, so that, e.g., ``x < y > z`` is perfectly legal (though perhaps not pretty). +Value comparisons +----------------- + The operators ``<``, ``>``, ``==``, ``>=``, ``<=``, and ``!=`` compare the -values of two objects. The objects need not have the same type. If both are -numbers, they are converted to a common type. Otherwise, the ``==`` and ``!=`` -operators *always* consider objects of different types to be unequal, while the -``<``, ``>``, ``>=`` and ``<=`` operators raise a :exc:`TypeError` when -comparing objects of different types that do not implement these operators for -the given pair of types. You can control comparison behavior of objects of -non-built-in types by defining rich comparison methods like :meth:`__gt__`, -described in section :ref:`customization`. +values of two objects. The objects do not need to have the same type. -Comparison of objects of the same type depends on the type: +Chapter :ref:`objects` states that objects have a value (in addition to type +and identity). The value of an object is a rather abstract notion in Python: +For example, there is no canonical access method for an object's value. Also, +there is no requirement that the value of an object should be constructed in a +particular way, e.g. comprised of all its data attributes. Comparison operators +implement a particular notion of what the value of an object is. One can think +of them as defining the value of an object indirectly, by means of their +comparison implementation. -* Numbers are compared arithmetically. +Because all types are (direct or indirect) subtypes of :class:`object`, they +inherit the default comparison behavior from :class:`object`. Types can +customize their comparison behavior by implementing +:dfn:`rich comparison methods` like :meth:`__lt__`, described in +:ref:`customization`. -* The values :const:`float('NaN')` and :const:`Decimal('NaN')` are special. - They are identical to themselves, ``x is x`` but are not equal to themselves, - ``x != x``. Additionally, comparing any value to a not-a-number value +The default behavior for equality comparison (``==`` and ``!=``) is based on +the identity of the objects. Hence, equality comparison of instances with the +same identity results in equality, and equality comparison of instances with +different identities results in inequality. A motivation for this default +behavior is the desire that all objects should be reflexive (i.e. ``x is y`` +implies ``x == y``). + +A default order comparison (``<``, ``>``, ``<=``, and ``>=``) is not provided; +an attempt raises :exc:`TypeError`. A motivation for this default behavior is +the lack of a similar invariant as for equality. + +The behavior of the default equality comparison, that instances with different +identities are always unequal, may be in contrast to what types will need that +have a sensible definition of object value and value-based equality. Such +types will need to customize their comparison behavior, and in fact, a number +of built-in types have done that. + +The following list describes the comparison behavior of the most important +built-in types. + +* Numbers of built-in numeric types (:ref:`typesnumeric`) and of the standard + library types :class:`fractions.Fraction` and :class:`decimal.Decimal` can be + compared within and across their types, with the restriction that complex + numbers do not support order comparison. Within the limits of the types + involved, they compare mathematically (algorithmically) correct without loss + of precision. + + The not-a-number values :const:`float('NaN')` and :const:`Decimal('NaN')` + are special. They are identical to themselves (``x is x`` is true) but + are not equal to themselves (``x == x`` is false). Additionally, + comparing any number to a not-a-number value will return ``False``. For example, both ``3 < float('NaN')`` and ``float('NaN') < 3`` will return ``False``. -* Bytes objects are compared lexicographically using the numeric values of their - elements. +* Binary sequences (instances of :class:`bytes` or :class:`bytearray`) can be + compared within and across their types. They compare lexicographically using + the numeric values of their elements. -* Strings are compared lexicographically using the numeric equivalents (the - result of the built-in function :func:`ord`) of their characters. [#]_ String - and bytes object can't be compared! +* Strings (instances of :class:`str`) compare lexicographically using the + numerical Unicode code points (the result of the built-in function + :func:`ord`) of their characters. [#]_ -* Tuples and lists are compared lexicographically using comparison of - corresponding elements. This means that to compare equal, each element must - compare equal and the two sequences must be of the same type and have the same - length. + Strings and binary sequences cannot be directly compared. - If not equal, the sequences are ordered the same as their first differing - elements. For example, ``[1,2,x] <= [1,2,y]`` has the same value as - ``x <= y``. If the corresponding element does not exist, the shorter - sequence is ordered first (for example, ``[1,2] < [1,2,3]``). +* Sequences (instances of :class:`tuple`, :class:`list`, or :class:`range`) can + be compared only within each of their types, with the restriction that ranges + do not support order comparison. Equality comparison across these types + results in unequality, and ordering comparison across these types raises + :exc:`TypeError`. -* Mappings (dictionaries) compare equal if and only if they have the same - ``(key, value)`` pairs. Order comparisons ``('<', '<=', '>=', '>')`` - raise :exc:`TypeError`. + Sequences compare lexicographically using comparison of corresponding + elements, whereby reflexivity of the elements is enforced. -* Sets and frozensets define comparison operators to mean subset and superset - tests. Those relations do not define total orderings (the two sets ``{1,2}`` - and ``{2,3}`` are not equal, nor subsets of one another, nor supersets of one + In enforcing reflexivity of elements, the comparison of collections assumes + that for a collection element ``x``, ``x == x`` is always true. Based on + that assumption, element identity is compared first, and element comparison + is performed only for distinct elements. This approach yields the same + result as a strict element comparison would, if the compared elements are + reflexive. For non-reflexive elements, the result is different than for + strict element comparison, and may be surprising: The non-reflexive + not-a-number values for example result in the following comparison behavior + when used in a list:: + + >>> nan = float('NaN') + >>> nan is nan + True + >>> nan == nan + False <-- the defined non-reflexive behavior of NaN + >>> [nan] == [nan] + True <-- list enforces reflexivity and tests identity first + + Lexicographical comparison between built-in collections works as follows: + + - For two collections to compare equal, they must be of the same type, have + the same length, and each pair of corresponding elements must compare + equal (for example, ``[1,2] == (1,2)`` is false because the type is not the + same). + + - Collections that support order comparison are ordered the same as their + first unequal elements (for example, ``[1,2,x] <= [1,2,y]`` has the same + value as ``x <= y``). If a corresponding element does not exist, the + shorter collection is ordered first (for example, ``[1,2] < [1,2,3]`` is + true). + +* Mappings (instances of :class:`dict`) compare equal if and only if they have + equal `(key, value)` pairs. Equality comparison of the keys and elements + enforces reflexivity. + + Order comparisons (``<``, ``>``, ``<=``, and ``>=``) raise :exc:`TypeError`. + +* Sets (instances of :class:`set` or :class:`frozenset`) can be compared within + and across their types. + + They define order + comparison operators to mean subset and superset tests. Those relations do + not define total orderings (for example, the two sets ``{1,2}`` and ``{2,3}`` + are not equal, nor subsets of one another, nor supersets of one another). Accordingly, sets are not appropriate arguments for functions - which depend on total ordering. For example, :func:`min`, :func:`max`, and - :func:`sorted` produce undefined results given a list of sets as inputs. + which depend on total ordering (for example, :func:`min`, :func:`max`, and + :func:`sorted` produce undefined results given a list of sets as inputs). -* Most other objects of built-in types compare unequal unless they are the same - object; the choice whether one object is considered smaller or larger than - another one is made arbitrarily but consistently within one execution of a - program. + Comparison of sets enforces reflexivity of its elements. -Comparison of objects of differing types depends on whether either of the -types provide explicit support for the comparison. Most numeric types can be -compared with one another. When cross-type comparison is not supported, the -comparison method returns ``NotImplemented``. +* Most other built-in types have no comparison methods implemented, so they + inherit the default comparison behavior. +User-defined classes that customize their comparison behavior should follow +some consistency rules, if possible: + +* Equality comparison should be reflexive. + In other words, identical objects should compare equal: + + ``x is y`` implies ``x == y`` + +* Comparison should be symmetric. + In other words, the following expressions should have the same result: + + ``x == y`` and ``y == x`` + + ``x != y`` and ``y != x`` + + ``x < y`` and ``y > x`` + + ``x <= y`` and ``y >= x`` + +* Comparison should be transitive. + The following (non-exhaustive) examples illustrate that: + + ``x > y and y > z`` implies ``x > z`` + + ``x < y and y <= z`` implies ``x < z`` + +* Inverse comparison should result in the boolean negation. + In other words, the following expressions should have the same result: + + ``x == y`` and ``not x != y`` + + ``x < y`` and ``not x >= y`` (for total ordering) + + ``x > y`` and ``not x <= y`` (for total ordering) + + The last two expressions apply to totally ordered collections (e.g. to + sequences, but not to sets or mappings). See also the + :func:`~functools.total_ordering` decorator. + +Python does not enforce these consistency rules. In fact, the not-a-number +values are an example for not following these rules. + + +.. _in: +.. _not in: .. _membership-test-details: +Membership test operations +-------------------------- + The operators :keyword:`in` and :keyword:`not in` test for membership. ``x in s`` evaluates to true if *x* is a member of *s*, and false otherwise. ``x not in s`` returns the negation of ``x in s``. All built-in sequences and set types @@ -1153,6 +1266,13 @@ operator: is not pair: identity; test + +.. _is: +.. _is not: + +Identity comparisons +-------------------- + The operators :keyword:`is` and :keyword:`is not` test for object identity: ``x is y`` is true if and only if *x* and *y* are the same object. ``x is not y`` yields the inverse truth value. [#]_ @@ -1379,12 +1499,24 @@ cases, Python returns the latter result, in order to preserve that ``divmod(x,y)[0] * y + x % y`` be very close to ``x``. -.. [#] While comparisons between strings make sense at the byte level, they may - be counter-intuitive to users. For example, the strings ``"\u00C7"`` and - ``"\u0043\u0327"`` compare differently, even though they both represent the - same unicode character (LATIN CAPITAL LETTER C WITH CEDILLA). To compare - strings in a human recognizable way, compare using - :func:`unicodedata.normalize`. +.. [#] The Unicode standard distinguishes between :dfn:`code points` + (e.g. U+0041) and :dfn:`abstract characters` (e.g. "LATIN CAPITAL LETTER A"). + While most abstract characters in Unicode are only represented using one + code point, there is a number of abstract characters that can in addition be + represented using a sequence of more than one code point. For example, the + abstract character "LATIN CAPITAL LETTER C WITH CEDILLA" can be represented + as a single :dfn:`precomposed character` at code position U+00C7, or as a + sequence of a :dfn:`base character` at code position U+0043 (LATIN CAPITAL + LETTER C), followed by a :dfn:`combining character` at code position U+0327 + (COMBINING CEDILLA). + + The comparison operators on strings compare at the level of Unicode code + points. This may be counter-intuitive to humans. For example, + ``"\u00C7" == "\u0043\u0327"`` is ``False``, even though both strings + represent the same abstract character "LATIN CAPITAL LETTER C WITH CEDILLA". + + To compare strings at the level of abstract characters (that is, in a way + intuitive to humans), use :func:`unicodedata.normalize`. .. [#] Due to automatic garbage-collection, free lists, and the dynamic nature of descriptors, you may notice seemingly unusual behaviour in certain uses of diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -518,6 +518,13 @@ Documentation ------------- +- Issue #12067: Rewrite Comparisons section in the Expressions chapter of the + language reference. Some of the details of comparing mixed types were + incorrect or ambiguous. NotImplemented is only relevant at a lower level + than the Expressions chapter. Added details of comparing range() objects, + and default behaviour and consistency suggestions for user-defined classes. + Patch from Andy Maier. + - Issue #24952: Clarify the default size argument of stack_size() in the "threading" and "_thread" modules. Patch from Mattip. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 23 07:53:35 2015 From: python-checkins at python.org (martin.panter) Date: Wed, 23 Sep 2015 05:53:35 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2312067=3A_Merge_comparisons_doc_from_3=2E5?= Message-ID: <20150923055335.3668.6725@psf.io> https://hg.python.org/cpython/rev/294b8a7957e9 changeset: 98210:294b8a7957e9 parent: 98207:47fe144fc24a parent: 98209:b6698c00265b user: Martin Panter date: Wed Sep 23 05:41:52 2015 +0000 summary: Issue #12067: Merge comparisons doc from 3.5 files: Doc/reference/expressions.rst | 236 +++++++++++++++++---- Misc/NEWS | 7 + 2 files changed, 191 insertions(+), 52 deletions(-) diff --git a/Doc/reference/expressions.rst b/Doc/reference/expressions.rst --- a/Doc/reference/expressions.rst +++ b/Doc/reference/expressions.rst @@ -1036,10 +1036,6 @@ .. _comparisons: -.. _is: -.. _is not: -.. _in: -.. _not in: Comparisons =========== @@ -1075,66 +1071,183 @@ *c*, so that, e.g., ``x < y > z`` is perfectly legal (though perhaps not pretty). +Value comparisons +----------------- + The operators ``<``, ``>``, ``==``, ``>=``, ``<=``, and ``!=`` compare the -values of two objects. The objects need not have the same type. If both are -numbers, they are converted to a common type. Otherwise, the ``==`` and ``!=`` -operators *always* consider objects of different types to be unequal, while the -``<``, ``>``, ``>=`` and ``<=`` operators raise a :exc:`TypeError` when -comparing objects of different types that do not implement these operators for -the given pair of types. You can control comparison behavior of objects of -non-built-in types by defining rich comparison methods like :meth:`__gt__`, -described in section :ref:`customization`. +values of two objects. The objects do not need to have the same type. -Comparison of objects of the same type depends on the type: +Chapter :ref:`objects` states that objects have a value (in addition to type +and identity). The value of an object is a rather abstract notion in Python: +For example, there is no canonical access method for an object's value. Also, +there is no requirement that the value of an object should be constructed in a +particular way, e.g. comprised of all its data attributes. Comparison operators +implement a particular notion of what the value of an object is. One can think +of them as defining the value of an object indirectly, by means of their +comparison implementation. -* Numbers are compared arithmetically. +Because all types are (direct or indirect) subtypes of :class:`object`, they +inherit the default comparison behavior from :class:`object`. Types can +customize their comparison behavior by implementing +:dfn:`rich comparison methods` like :meth:`__lt__`, described in +:ref:`customization`. -* The values :const:`float('NaN')` and :const:`Decimal('NaN')` are special. - They are identical to themselves, ``x is x`` but are not equal to themselves, - ``x != x``. Additionally, comparing any value to a not-a-number value +The default behavior for equality comparison (``==`` and ``!=``) is based on +the identity of the objects. Hence, equality comparison of instances with the +same identity results in equality, and equality comparison of instances with +different identities results in inequality. A motivation for this default +behavior is the desire that all objects should be reflexive (i.e. ``x is y`` +implies ``x == y``). + +A default order comparison (``<``, ``>``, ``<=``, and ``>=``) is not provided; +an attempt raises :exc:`TypeError`. A motivation for this default behavior is +the lack of a similar invariant as for equality. + +The behavior of the default equality comparison, that instances with different +identities are always unequal, may be in contrast to what types will need that +have a sensible definition of object value and value-based equality. Such +types will need to customize their comparison behavior, and in fact, a number +of built-in types have done that. + +The following list describes the comparison behavior of the most important +built-in types. + +* Numbers of built-in numeric types (:ref:`typesnumeric`) and of the standard + library types :class:`fractions.Fraction` and :class:`decimal.Decimal` can be + compared within and across their types, with the restriction that complex + numbers do not support order comparison. Within the limits of the types + involved, they compare mathematically (algorithmically) correct without loss + of precision. + + The not-a-number values :const:`float('NaN')` and :const:`Decimal('NaN')` + are special. They are identical to themselves (``x is x`` is true) but + are not equal to themselves (``x == x`` is false). Additionally, + comparing any number to a not-a-number value will return ``False``. For example, both ``3 < float('NaN')`` and ``float('NaN') < 3`` will return ``False``. -* Bytes objects are compared lexicographically using the numeric values of their - elements. +* Binary sequences (instances of :class:`bytes` or :class:`bytearray`) can be + compared within and across their types. They compare lexicographically using + the numeric values of their elements. -* Strings are compared lexicographically using the numeric equivalents (the - result of the built-in function :func:`ord`) of their characters. [#]_ String - and bytes object can't be compared! +* Strings (instances of :class:`str`) compare lexicographically using the + numerical Unicode code points (the result of the built-in function + :func:`ord`) of their characters. [#]_ -* Tuples and lists are compared lexicographically using comparison of - corresponding elements. This means that to compare equal, each element must - compare equal and the two sequences must be of the same type and have the same - length. + Strings and binary sequences cannot be directly compared. - If not equal, the sequences are ordered the same as their first differing - elements. For example, ``[1,2,x] <= [1,2,y]`` has the same value as - ``x <= y``. If the corresponding element does not exist, the shorter - sequence is ordered first (for example, ``[1,2] < [1,2,3]``). +* Sequences (instances of :class:`tuple`, :class:`list`, or :class:`range`) can + be compared only within each of their types, with the restriction that ranges + do not support order comparison. Equality comparison across these types + results in unequality, and ordering comparison across these types raises + :exc:`TypeError`. -* Mappings (dictionaries) compare equal if and only if they have the same - ``(key, value)`` pairs. Order comparisons ``('<', '<=', '>=', '>')`` - raise :exc:`TypeError`. + Sequences compare lexicographically using comparison of corresponding + elements, whereby reflexivity of the elements is enforced. -* Sets and frozensets define comparison operators to mean subset and superset - tests. Those relations do not define total orderings (the two sets ``{1,2}`` - and ``{2,3}`` are not equal, nor subsets of one another, nor supersets of one + In enforcing reflexivity of elements, the comparison of collections assumes + that for a collection element ``x``, ``x == x`` is always true. Based on + that assumption, element identity is compared first, and element comparison + is performed only for distinct elements. This approach yields the same + result as a strict element comparison would, if the compared elements are + reflexive. For non-reflexive elements, the result is different than for + strict element comparison, and may be surprising: The non-reflexive + not-a-number values for example result in the following comparison behavior + when used in a list:: + + >>> nan = float('NaN') + >>> nan is nan + True + >>> nan == nan + False <-- the defined non-reflexive behavior of NaN + >>> [nan] == [nan] + True <-- list enforces reflexivity and tests identity first + + Lexicographical comparison between built-in collections works as follows: + + - For two collections to compare equal, they must be of the same type, have + the same length, and each pair of corresponding elements must compare + equal (for example, ``[1,2] == (1,2)`` is false because the type is not the + same). + + - Collections that support order comparison are ordered the same as their + first unequal elements (for example, ``[1,2,x] <= [1,2,y]`` has the same + value as ``x <= y``). If a corresponding element does not exist, the + shorter collection is ordered first (for example, ``[1,2] < [1,2,3]`` is + true). + +* Mappings (instances of :class:`dict`) compare equal if and only if they have + equal `(key, value)` pairs. Equality comparison of the keys and elements + enforces reflexivity. + + Order comparisons (``<``, ``>``, ``<=``, and ``>=``) raise :exc:`TypeError`. + +* Sets (instances of :class:`set` or :class:`frozenset`) can be compared within + and across their types. + + They define order + comparison operators to mean subset and superset tests. Those relations do + not define total orderings (for example, the two sets ``{1,2}`` and ``{2,3}`` + are not equal, nor subsets of one another, nor supersets of one another). Accordingly, sets are not appropriate arguments for functions - which depend on total ordering. For example, :func:`min`, :func:`max`, and - :func:`sorted` produce undefined results given a list of sets as inputs. + which depend on total ordering (for example, :func:`min`, :func:`max`, and + :func:`sorted` produce undefined results given a list of sets as inputs). -* Most other objects of built-in types compare unequal unless they are the same - object; the choice whether one object is considered smaller or larger than - another one is made arbitrarily but consistently within one execution of a - program. + Comparison of sets enforces reflexivity of its elements. -Comparison of objects of differing types depends on whether either of the -types provide explicit support for the comparison. Most numeric types can be -compared with one another. When cross-type comparison is not supported, the -comparison method returns ``NotImplemented``. +* Most other built-in types have no comparison methods implemented, so they + inherit the default comparison behavior. +User-defined classes that customize their comparison behavior should follow +some consistency rules, if possible: + +* Equality comparison should be reflexive. + In other words, identical objects should compare equal: + + ``x is y`` implies ``x == y`` + +* Comparison should be symmetric. + In other words, the following expressions should have the same result: + + ``x == y`` and ``y == x`` + + ``x != y`` and ``y != x`` + + ``x < y`` and ``y > x`` + + ``x <= y`` and ``y >= x`` + +* Comparison should be transitive. + The following (non-exhaustive) examples illustrate that: + + ``x > y and y > z`` implies ``x > z`` + + ``x < y and y <= z`` implies ``x < z`` + +* Inverse comparison should result in the boolean negation. + In other words, the following expressions should have the same result: + + ``x == y`` and ``not x != y`` + + ``x < y`` and ``not x >= y`` (for total ordering) + + ``x > y`` and ``not x <= y`` (for total ordering) + + The last two expressions apply to totally ordered collections (e.g. to + sequences, but not to sets or mappings). See also the + :func:`~functools.total_ordering` decorator. + +Python does not enforce these consistency rules. In fact, the not-a-number +values are an example for not following these rules. + + +.. _in: +.. _not in: .. _membership-test-details: +Membership test operations +-------------------------- + The operators :keyword:`in` and :keyword:`not in` test for membership. ``x in s`` evaluates to true if *x* is a member of *s*, and false otherwise. ``x not in s`` returns the negation of ``x in s``. All built-in sequences and set types @@ -1176,6 +1289,13 @@ operator: is not pair: identity; test + +.. _is: +.. _is not: + +Identity comparisons +-------------------- + The operators :keyword:`is` and :keyword:`is not` test for object identity: ``x is y`` is true if and only if *x* and *y* are the same object. ``x is not y`` yields the inverse truth value. [#]_ @@ -1405,12 +1525,24 @@ cases, Python returns the latter result, in order to preserve that ``divmod(x,y)[0] * y + x % y`` be very close to ``x``. -.. [#] While comparisons between strings make sense at the byte level, they may - be counter-intuitive to users. For example, the strings ``"\u00C7"`` and - ``"\u0043\u0327"`` compare differently, even though they both represent the - same unicode character (LATIN CAPITAL LETTER C WITH CEDILLA). To compare - strings in a human recognizable way, compare using - :func:`unicodedata.normalize`. +.. [#] The Unicode standard distinguishes between :dfn:`code points` + (e.g. U+0041) and :dfn:`abstract characters` (e.g. "LATIN CAPITAL LETTER A"). + While most abstract characters in Unicode are only represented using one + code point, there is a number of abstract characters that can in addition be + represented using a sequence of more than one code point. For example, the + abstract character "LATIN CAPITAL LETTER C WITH CEDILLA" can be represented + as a single :dfn:`precomposed character` at code position U+00C7, or as a + sequence of a :dfn:`base character` at code position U+0043 (LATIN CAPITAL + LETTER C), followed by a :dfn:`combining character` at code position U+0327 + (COMBINING CEDILLA). + + The comparison operators on strings compare at the level of Unicode code + points. This may be counter-intuitive to humans. For example, + ``"\u00C7" == "\u0043\u0327"`` is ``False``, even though both strings + represent the same abstract character "LATIN CAPITAL LETTER C WITH CEDILLA". + + To compare strings at the level of abstract characters (that is, in a way + intuitive to humans), use :func:`unicodedata.normalize`. .. [#] Due to automatic garbage-collection, free lists, and the dynamic nature of descriptors, you may notice seemingly unusual behaviour in certain uses of diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -224,6 +224,13 @@ Documentation ------------- +- Issue #12067: Rewrite Comparisons section in the Expressions chapter of the + language reference. Some of the details of comparing mixed types were + incorrect or ambiguous. NotImplemented is only relevant at a lower level + than the Expressions chapter. Added details of comparing range() objects, + and default behaviour and consistency suggestions for user-defined classes. + Patch from Andy Maier. + - Issue #24952: Clarify the default size argument of stack_size() in the "threading" and "_thread" modules. Patch from Mattip. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 23 09:46:06 2015 From: python-checkins at python.org (terry.reedy) Date: Wed, 23 Sep 2015 07:46:06 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge_with_3=2E5?= Message-ID: <20150923074606.81627.25146@psf.io> https://hg.python.org/cpython/rev/48cc97dbb95c changeset: 98213:48cc97dbb95c parent: 98210:294b8a7957e9 parent: 98212:728c9d896b21 user: Terry Jan Reedy date: Wed Sep 23 03:45:49 2015 -0400 summary: Merge with 3.5 files: Lib/idlelib/help.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Lib/idlelib/help.py b/Lib/idlelib/help.py --- a/Lib/idlelib/help.py +++ b/Lib/idlelib/help.py @@ -234,7 +234,7 @@ with open(src, 'rb') as inn,\ open(dst, 'wb') as out: for line in inn: - out.write(line.rstrip() + '\n') + out.write(line.rstrip() + b'\n') print('idle.html copied to help.html') def show_idlehelp(parent): -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 23 09:46:06 2015 From: python-checkins at python.org (terry.reedy) Date: Wed, 23 Sep 2015 07:46:06 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSWRsZWxpYi5oZWxw?= =?utf-8?q?=3A_add_=27b=27_prefix_needed_for_bytes_+_bytes=2E?= Message-ID: <20150923074606.11710.55618@psf.io> https://hg.python.org/cpython/rev/e7f5ecfd86fc changeset: 98211:e7f5ecfd86fc branch: 3.4 parent: 98208:1fc049e5ec14 user: Terry Jan Reedy date: Wed Sep 23 03:45:13 2015 -0400 summary: Idlelib.help: add 'b' prefix needed for bytes + bytes. files: Lib/idlelib/help.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Lib/idlelib/help.py b/Lib/idlelib/help.py --- a/Lib/idlelib/help.py +++ b/Lib/idlelib/help.py @@ -234,7 +234,7 @@ with open(src, 'rb') as inn,\ open(dst, 'wb') as out: for line in inn: - out.write(line.rstrip() + '\n') + out.write(line.rstrip() + b'\n') print('idle.html copied to help.html') def show_idlehelp(parent): -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 23 09:46:07 2015 From: python-checkins at python.org (terry.reedy) Date: Wed, 23 Sep 2015 07:46:07 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Merge_with_3=2E4?= Message-ID: <20150923074606.16573.14480@psf.io> https://hg.python.org/cpython/rev/728c9d896b21 changeset: 98212:728c9d896b21 branch: 3.5 parent: 98209:b6698c00265b parent: 98211:e7f5ecfd86fc user: Terry Jan Reedy date: Wed Sep 23 03:45:33 2015 -0400 summary: Merge with 3.4 files: Lib/idlelib/help.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Lib/idlelib/help.py b/Lib/idlelib/help.py --- a/Lib/idlelib/help.py +++ b/Lib/idlelib/help.py @@ -234,7 +234,7 @@ with open(src, 'rb') as inn,\ open(dst, 'wb') as out: for line in inn: - out.write(line.rstrip() + '\n') + out.write(line.rstrip() + b'\n') print('idle.html copied to help.html') def show_idlehelp(parent): -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 23 09:53:10 2015 From: python-checkins at python.org (terry.reedy) Date: Wed, 23 Sep 2015 07:53:10 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzI1MjE5?= =?utf-8?q?=3A_Update_doc_for_Idle_command_line_options=2E?= Message-ID: <20150923075310.98368.64319@psf.io> https://hg.python.org/cpython/rev/97f3d7749d3f changeset: 98215:97f3d7749d3f branch: 3.4 parent: 98211:e7f5ecfd86fc user: Terry Jan Reedy date: Wed Sep 23 03:52:23 2015 -0400 summary: Issue #25219: Update doc for Idle command line options. Some were missing and notes were not correct. files: Doc/library/idle.rst | 29 ++++++++++++++------------- Lib/idlelib/help.html | 33 +++++++++++++++--------------- 2 files changed, 32 insertions(+), 30 deletions(-) diff --git a/Doc/library/idle.rst b/Doc/library/idle.rst --- a/Doc/library/idle.rst +++ b/Doc/library/idle.rst @@ -504,27 +504,28 @@ :: - idle.py [-c command] [-d] [-e] [-s] [-t title] [arg] ... + idle.py [-c command] [-d] [-e] [-h] [-i] [-r file] [-s] [-t title] [-] [arg] ... - -c command run this command - -d enable debugger - -e edit mode; arguments are files to be edited - -s run $IDLESTARTUP or $PYTHONSTARTUP first + -c command run command in the shell window + -d enable debugger and open shell window + -e open editor window + -h print help message with legal combinatios and exit + -i open shell window + -r file run file in shell window + -s run $IDLESTARTUP or $PYTHONSTARTUP first, in shell window -t title set title of shell window + - run stdin in shell (- must be last option before args) If there are arguments: -#. If ``-e`` is used, arguments are files opened for editing and - ``sys.argv`` reflects the arguments passed to IDLE itself. +* If ``-``, ``-c``, or ``r`` is used, all arguments are placed in + ``sys.argv[1:...]`` and ``sys.argv[0]`` is set to ``''``, ``'-c'``, + or ``'-r'``. No editor window is opened, even if that is the default + set in the Options dialog. -#. Otherwise, if ``-c`` is used, all arguments are placed in - ``sys.argv[1:...]``, with ``sys.argv[0]`` set to ``'-c'``. +* Otherwise, arguments are files opened for editing and + ``sys.argv`` reflects the arguments passed to IDLE itself. -#. Otherwise, if neither ``-e`` nor ``-c`` is used, the first - argument is a script which is executed with the remaining arguments in - ``sys.argv[1:...]`` and ``sys.argv[0]`` set to the script name. If the - script name is '-', no script is executed but an interactive Python session - is started; the arguments are still available in ``sys.argv``. Running without a subprocess ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ diff --git a/Lib/idlelib/help.html b/Lib/idlelib/help.html --- a/Lib/idlelib/help.html +++ b/Lib/idlelib/help.html @@ -478,27 +478,28 @@ functions to be used from IDLE’s Python shell.

25.5.4.1. Command line usage?

-
idle.py [-c command] [-d] [-e] [-s] [-t title] [arg] ...
+
idle.py [-c command] [-d] [-e] [-h] [-i] [-r file] [-s] [-t title] [-] [arg] ...
 
--c command  run this command
--d          enable debugger
--e          edit mode; arguments are files to be edited
--s          run $IDLESTARTUP or $PYTHONSTARTUP first
+-c command  run command in the shell window
+-d          enable debugger and open shell window
+-e          open editor window
+-h          print help message with legal combinatios and exit
+-i          open shell window
+-r file     run file in shell window
+-s          run $IDLESTARTUP or $PYTHONSTARTUP first, in shell window
 -t title    set title of shell window
+-           run stdin in shell (- must be last option before args)
 

If there are arguments:

-
    -
  1. If -e is used, arguments are files opened for editing and +
      +
    • If -, -c, or r is used, all arguments are placed in +sys.argv[1:...] and sys.argv[0] is set to '', '-c', +or '-r'. No editor window is opened, even if that is the default +set in the Options dialog.
    • +
    • Otherwise, arguments are files opened for editing and sys.argv reflects the arguments passed to IDLE itself.
    • -
    • Otherwise, if -c is used, all arguments are placed in -sys.argv[1:...], with sys.argv[0] set to '-c'.
    • -
    • Otherwise, if neither -e nor -c is used, the first -argument is a script which is executed with the remaining arguments in -sys.argv[1:...] and sys.argv[0] set to the script name. If the -script name is ‘-‘, no script is executed but an interactive Python session -is started; the arguments are still available in sys.argv.
    • -
+

25.5.4.2. Running without a subprocess?

@@ -661,7 +662,7 @@ The Python Software Foundation is a non-profit corporation. Please donate.
- Last updated on Sep 12, 2015. + Last updated on Sep 23, 2015. Found a bug?
Created using Sphinx 1.2.3. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 23 09:53:10 2015 From: python-checkins at python.org (terry.reedy) Date: Wed, 23 Sep 2015 07:53:10 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzI1MjE5?= =?utf-8?q?=3A_Update_doc_for_Idle_command_line_options=2E?= Message-ID: <20150923075309.98354.99498@psf.io> https://hg.python.org/cpython/rev/80e92eba23e0 changeset: 98214:80e92eba23e0 branch: 2.7 parent: 98204:26e819909891 user: Terry Jan Reedy date: Wed Sep 23 03:52:18 2015 -0400 summary: Issue #25219: Update doc for Idle command line options. Some were missing and notes were not correct. files: Doc/library/idle.rst | 29 ++++++++++++++------------- Lib/idlelib/help.html | 33 +++++++++++++++--------------- 2 files changed, 32 insertions(+), 30 deletions(-) diff --git a/Doc/library/idle.rst b/Doc/library/idle.rst --- a/Doc/library/idle.rst +++ b/Doc/library/idle.rst @@ -504,27 +504,28 @@ :: - idle.py [-c command] [-d] [-e] [-s] [-t title] [arg] ... + idle.py [-c command] [-d] [-e] [-h] [-i] [-r file] [-s] [-t title] [-] [arg] ... - -c command run this command - -d enable debugger - -e edit mode; arguments are files to be edited - -s run $IDLESTARTUP or $PYTHONSTARTUP first + -c command run command in the shell window + -d enable debugger and open shell window + -e open editor window + -h print help message with legal combinatios and exit + -i open shell window + -r file run file in shell window + -s run $IDLESTARTUP or $PYTHONSTARTUP first, in shell window -t title set title of shell window + - run stdin in shell (- must be last option before args) If there are arguments: -#. If ``-e`` is used, arguments are files opened for editing and - ``sys.argv`` reflects the arguments passed to IDLE itself. +* If ``-``, ``-c``, or ``r`` is used, all arguments are placed in + ``sys.argv[1:...]`` and ``sys.argv[0]`` is set to ``''``, ``'-c'``, + or ``'-r'``. No editor window is opened, even if that is the default + set in the Options dialog. -#. Otherwise, if ``-c`` is used, all arguments are placed in - ``sys.argv[1:...]``, with ``sys.argv[0]`` set to ``'-c'``. +* Otherwise, arguments are files opened for editing and + ``sys.argv`` reflects the arguments passed to IDLE itself. -#. Otherwise, if neither ``-e`` nor ``-c`` is used, the first - argument is a script which is executed with the remaining arguments in - ``sys.argv[1:...]`` and ``sys.argv[0]`` set to the script name. If the - script name is '-', no script is executed but an interactive Python session - is started; the arguments are still available in ``sys.argv``. Running without a subprocess ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ diff --git a/Lib/idlelib/help.html b/Lib/idlelib/help.html --- a/Lib/idlelib/help.html +++ b/Lib/idlelib/help.html @@ -478,27 +478,28 @@ functions to be used from IDLE’s Python shell.

24.6.4.1. Command line usage?

-
idle.py [-c command] [-d] [-e] [-s] [-t title] [arg] ...
+
idle.py [-c command] [-d] [-e] [-h] [-i] [-r file] [-s] [-t title] [-] [arg] ...
 
--c command  run this command
--d          enable debugger
--e          edit mode; arguments are files to be edited
--s          run $IDLESTARTUP or $PYTHONSTARTUP first
+-c command  run command in the shell window
+-d          enable debugger and open shell window
+-e          open editor window
+-h          print help message with legal combinatios and exit
+-i          open shell window
+-r file     run file in shell window
+-s          run $IDLESTARTUP or $PYTHONSTARTUP first, in shell window
 -t title    set title of shell window
+-           run stdin in shell (- must be last option before args)
 

If there are arguments:

-
    -
  1. If -e is used, arguments are files opened for editing and +
      +
    • If -, -c, or r is used, all arguments are placed in +sys.argv[1:...] and sys.argv[0] is set to '', '-c', +or '-r'. No editor window is opened, even if that is the default +set in the Options dialog.
    • +
    • Otherwise, arguments are files opened for editing and sys.argv reflects the arguments passed to IDLE itself.
    • -
    • Otherwise, if -c is used, all arguments are placed in -sys.argv[1:...], with sys.argv[0] set to '-c'.
    • -
    • Otherwise, if neither -e nor -c is used, the first -argument is a script which is executed with the remaining arguments in -sys.argv[1:...] and sys.argv[0] set to the script name. If the -script name is ‘-‘, no script is executed but an interactive Python session -is started; the arguments are still available in sys.argv.
    • -
+

24.6.4.2. Running without a subprocess?

@@ -661,7 +662,7 @@ The Python Software Foundation is a non-profit corporation. Please donate.
- Last updated on Sep 20, 2015. + Last updated on Sep 23, 2015. Found a bug?
Created using Sphinx 1.2.3. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 23 09:53:10 2015 From: python-checkins at python.org (terry.reedy) Date: Wed, 23 Sep 2015 07:53:10 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge_with_3=2E5?= Message-ID: <20150923075310.98368.58880@psf.io> https://hg.python.org/cpython/rev/e69fbc53db61 changeset: 98217:e69fbc53db61 parent: 98213:48cc97dbb95c parent: 98216:5d5fca739abf user: Terry Jan Reedy date: Wed Sep 23 03:52:50 2015 -0400 summary: Merge with 3.5 files: Doc/library/idle.rst | 29 ++++++++++++++------------- Lib/idlelib/help.html | 33 +++++++++++++++--------------- 2 files changed, 32 insertions(+), 30 deletions(-) diff --git a/Doc/library/idle.rst b/Doc/library/idle.rst --- a/Doc/library/idle.rst +++ b/Doc/library/idle.rst @@ -504,27 +504,28 @@ :: - idle.py [-c command] [-d] [-e] [-s] [-t title] [arg] ... + idle.py [-c command] [-d] [-e] [-h] [-i] [-r file] [-s] [-t title] [-] [arg] ... - -c command run this command - -d enable debugger - -e edit mode; arguments are files to be edited - -s run $IDLESTARTUP or $PYTHONSTARTUP first + -c command run command in the shell window + -d enable debugger and open shell window + -e open editor window + -h print help message with legal combinatios and exit + -i open shell window + -r file run file in shell window + -s run $IDLESTARTUP or $PYTHONSTARTUP first, in shell window -t title set title of shell window + - run stdin in shell (- must be last option before args) If there are arguments: -#. If ``-e`` is used, arguments are files opened for editing and - ``sys.argv`` reflects the arguments passed to IDLE itself. +* If ``-``, ``-c``, or ``r`` is used, all arguments are placed in + ``sys.argv[1:...]`` and ``sys.argv[0]`` is set to ``''``, ``'-c'``, + or ``'-r'``. No editor window is opened, even if that is the default + set in the Options dialog. -#. Otherwise, if ``-c`` is used, all arguments are placed in - ``sys.argv[1:...]``, with ``sys.argv[0]`` set to ``'-c'``. +* Otherwise, arguments are files opened for editing and + ``sys.argv`` reflects the arguments passed to IDLE itself. -#. Otherwise, if neither ``-e`` nor ``-c`` is used, the first - argument is a script which is executed with the remaining arguments in - ``sys.argv[1:...]`` and ``sys.argv[0]`` set to the script name. If the - script name is '-', no script is executed but an interactive Python session - is started; the arguments are still available in ``sys.argv``. Running without a subprocess ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ diff --git a/Lib/idlelib/help.html b/Lib/idlelib/help.html --- a/Lib/idlelib/help.html +++ b/Lib/idlelib/help.html @@ -478,27 +478,28 @@ functions to be used from IDLE’s Python shell.

25.5.4.1. Command line usage?

-
idle.py [-c command] [-d] [-e] [-s] [-t title] [arg] ...
+
idle.py [-c command] [-d] [-e] [-h] [-i] [-r file] [-s] [-t title] [-] [arg] ...
 
--c command  run this command
--d          enable debugger
--e          edit mode; arguments are files to be edited
--s          run $IDLESTARTUP or $PYTHONSTARTUP first
+-c command  run command in the shell window
+-d          enable debugger and open shell window
+-e          open editor window
+-h          print help message with legal combinatios and exit
+-i          open shell window
+-r file     run file in shell window
+-s          run $IDLESTARTUP or $PYTHONSTARTUP first, in shell window
 -t title    set title of shell window
+-           run stdin in shell (- must be last option before args)
 

If there are arguments:

-
    -
  1. If -e is used, arguments are files opened for editing and +
      +
    • If -, -c, or r is used, all arguments are placed in +sys.argv[1:...] and sys.argv[0] is set to '', '-c', +or '-r'. No editor window is opened, even if that is the default +set in the Options dialog.
    • +
    • Otherwise, arguments are files opened for editing and sys.argv reflects the arguments passed to IDLE itself.
    • -
    • Otherwise, if -c is used, all arguments are placed in -sys.argv[1:...], with sys.argv[0] set to '-c'.
    • -
    • Otherwise, if neither -e nor -c is used, the first -argument is a script which is executed with the remaining arguments in -sys.argv[1:...] and sys.argv[0] set to the script name. If the -script name is ‘-‘, no script is executed but an interactive Python session -is started; the arguments are still available in sys.argv.
    • -
+

25.5.4.2. Running without a subprocess?

@@ -661,7 +662,7 @@ The Python Software Foundation is a non-profit corporation. Please donate.
- Last updated on Sep 12, 2015. + Last updated on Sep 23, 2015. Found a bug?
Created using Sphinx 1.2.3. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 23 09:53:11 2015 From: python-checkins at python.org (terry.reedy) Date: Wed, 23 Sep 2015 07:53:11 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Merge_with_3=2E4?= Message-ID: <20150923075310.3644.83611@psf.io> https://hg.python.org/cpython/rev/5d5fca739abf changeset: 98216:5d5fca739abf branch: 3.5 parent: 98212:728c9d896b21 parent: 98215:97f3d7749d3f user: Terry Jan Reedy date: Wed Sep 23 03:52:36 2015 -0400 summary: Merge with 3.4 files: Doc/library/idle.rst | 29 ++++++++++++++------------- Lib/idlelib/help.html | 33 +++++++++++++++--------------- 2 files changed, 32 insertions(+), 30 deletions(-) diff --git a/Doc/library/idle.rst b/Doc/library/idle.rst --- a/Doc/library/idle.rst +++ b/Doc/library/idle.rst @@ -504,27 +504,28 @@ :: - idle.py [-c command] [-d] [-e] [-s] [-t title] [arg] ... + idle.py [-c command] [-d] [-e] [-h] [-i] [-r file] [-s] [-t title] [-] [arg] ... - -c command run this command - -d enable debugger - -e edit mode; arguments are files to be edited - -s run $IDLESTARTUP or $PYTHONSTARTUP first + -c command run command in the shell window + -d enable debugger and open shell window + -e open editor window + -h print help message with legal combinatios and exit + -i open shell window + -r file run file in shell window + -s run $IDLESTARTUP or $PYTHONSTARTUP first, in shell window -t title set title of shell window + - run stdin in shell (- must be last option before args) If there are arguments: -#. If ``-e`` is used, arguments are files opened for editing and - ``sys.argv`` reflects the arguments passed to IDLE itself. +* If ``-``, ``-c``, or ``r`` is used, all arguments are placed in + ``sys.argv[1:...]`` and ``sys.argv[0]`` is set to ``''``, ``'-c'``, + or ``'-r'``. No editor window is opened, even if that is the default + set in the Options dialog. -#. Otherwise, if ``-c`` is used, all arguments are placed in - ``sys.argv[1:...]``, with ``sys.argv[0]`` set to ``'-c'``. +* Otherwise, arguments are files opened for editing and + ``sys.argv`` reflects the arguments passed to IDLE itself. -#. Otherwise, if neither ``-e`` nor ``-c`` is used, the first - argument is a script which is executed with the remaining arguments in - ``sys.argv[1:...]`` and ``sys.argv[0]`` set to the script name. If the - script name is '-', no script is executed but an interactive Python session - is started; the arguments are still available in ``sys.argv``. Running without a subprocess ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ diff --git a/Lib/idlelib/help.html b/Lib/idlelib/help.html --- a/Lib/idlelib/help.html +++ b/Lib/idlelib/help.html @@ -478,27 +478,28 @@ functions to be used from IDLE’s Python shell.

25.5.4.1. Command line usage?

-
idle.py [-c command] [-d] [-e] [-s] [-t title] [arg] ...
+
idle.py [-c command] [-d] [-e] [-h] [-i] [-r file] [-s] [-t title] [-] [arg] ...
 
--c command  run this command
--d          enable debugger
--e          edit mode; arguments are files to be edited
--s          run $IDLESTARTUP or $PYTHONSTARTUP first
+-c command  run command in the shell window
+-d          enable debugger and open shell window
+-e          open editor window
+-h          print help message with legal combinatios and exit
+-i          open shell window
+-r file     run file in shell window
+-s          run $IDLESTARTUP or $PYTHONSTARTUP first, in shell window
 -t title    set title of shell window
+-           run stdin in shell (- must be last option before args)
 

If there are arguments:

-
    -
  1. If -e is used, arguments are files opened for editing and +
      +
    • If -, -c, or r is used, all arguments are placed in +sys.argv[1:...] and sys.argv[0] is set to '', '-c', +or '-r'. No editor window is opened, even if that is the default +set in the Options dialog.
    • +
    • Otherwise, arguments are files opened for editing and sys.argv reflects the arguments passed to IDLE itself.
    • -
    • Otherwise, if -c is used, all arguments are placed in -sys.argv[1:...], with sys.argv[0] set to '-c'.
    • -
    • Otherwise, if neither -e nor -c is used, the first -argument is a script which is executed with the remaining arguments in -sys.argv[1:...] and sys.argv[0] set to the script name. If the -script name is ‘-‘, no script is executed but an interactive Python session -is started; the arguments are still available in sys.argv.
    • -
+

25.5.4.2. Running without a subprocess?

@@ -661,7 +662,7 @@ The Python Software Foundation is a non-profit corporation. Please donate.
- Last updated on Sep 12, 2015. + Last updated on Sep 23, 2015. Found a bug?
Created using Sphinx 1.2.3. -- Repository URL: https://hg.python.org/cpython From solipsis at pitrou.net Wed Sep 23 10:47:05 2015 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Wed, 23 Sep 2015 08:47:05 +0000 Subject: [Python-checkins] Daily reference leaks (47fe144fc24a): sum=61926 Message-ID: <20150923084705.3650.47058@psf.io> results for 47fe144fc24a on branch "default" -------------------------------------------- test_asyncio leaked [0, 0, 3] memory blocks, sum=3 test_capi leaked [5446, 5446, 5446] references, sum=16338 test_capi leaked [1433, 1435, 1435] memory blocks, sum=4303 test_functools leaked [0, 2, 2] memory blocks, sum=4 test_threading leaked [10892, 10892, 10892] references, sum=32676 test_threading leaked [2866, 2868, 2868] memory blocks, sum=8602 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/psf-users/antoine/refleaks/reflogAZIZZp', '--timeout', '7200'] From python-checkins at python.org Wed Sep 23 11:42:08 2015 From: python-checkins at python.org (raymond.hettinger) Date: Wed, 23 Sep 2015 09:42:08 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Eliminate_unnecessary_vari?= =?utf-8?q?ables?= Message-ID: <20150923094207.94111.98005@psf.io> https://hg.python.org/cpython/rev/723003947972 changeset: 98218:723003947972 user: Raymond Hettinger date: Wed Sep 23 02:42:02 2015 -0700 summary: Eliminate unnecessary variables files: Modules/_collectionsmodule.c | 6 ++---- 1 files changed, 2 insertions(+), 4 deletions(-) diff --git a/Modules/_collectionsmodule.c b/Modules/_collectionsmodule.c --- a/Modules/_collectionsmodule.c +++ b/Modules/_collectionsmodule.c @@ -843,13 +843,12 @@ block *b = deque->leftblock; Py_ssize_t index = deque->leftindex; Py_ssize_t n = Py_SIZE(deque); - Py_ssize_t i; Py_ssize_t count = 0; size_t start_state = deque->state; PyObject *item; int cmp; - for (i=0 ; idata[index]; cmp = PyObject_RichCompareBool(item, v, Py_EQ); @@ -883,12 +882,11 @@ block *b = deque->leftblock; Py_ssize_t index = deque->leftindex; Py_ssize_t n = Py_SIZE(deque); - Py_ssize_t i; size_t start_state = deque->state; PyObject *item; int cmp; - for (i=0 ; idata[index]; cmp = PyObject_RichCompareBool(item, v, Py_EQ); -- Repository URL: https://hg.python.org/cpython From lp_benchmark_robot at intel.com Wed Sep 23 13:42:55 2015 From: lp_benchmark_robot at intel.com (lp_benchmark_robot) Date: Wed, 23 Sep 2015 11:42:55 +0000 Subject: [Python-checkins] Benchmark Results for Python 2.7 2015-09-23 Message-ID: Results for project python_2.7-nightly, build date 2015-09-23 03:43:50 commit: 26e81990989182f56c319b04aee072c099947891 revision date: 2015-09-23 02:59:35 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v2.7.10, with hash 15c95b7d81dcf821daade360741e00714667653f from 2015-05-23 16:02:14+00:00 ------------------------------------------------------------------------------------------ benchmark relative change since change since current rev with std_dev* last run v2.7.10 regrtest PGO ------------------------------------------------------------------------------------------ :-) django_v2 0.21662% -0.36076% 4.53223% 9.62297% :-) pybench 0.18384% -0.04429% 6.76898% 6.42734% :-| regex_v8 0.63377% 0.02303% -1.16873% 7.61713% :-) nbody 0.09783% -0.00925% 9.04590% 2.82610% :-) json_dump_v2 0.23271% -0.46904% 4.07745% 13.64134% :-| normal_startup 1.70069% 0.17924% -1.65614% 2.86074% :-| ssbench 0.70774% 0.06615% 1.88332% 1.76400% ------------------------------------------------------------------------------------------ Note: Benchmark results for ssbench are measured in requests/second while all other are measured in seconds. * Relative Standard Deviation (Standard Deviation/Average) Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. From lp_benchmark_robot at intel.com Wed Sep 23 13:43:01 2015 From: lp_benchmark_robot at intel.com (lp_benchmark_robot) Date: Wed, 23 Sep 2015 11:43:01 +0000 Subject: [Python-checkins] Benchmark Results for Python Default 2015-09-23 Message-ID: Results for project python_default-nightly, build date 2015-09-23 03:03:04 commit: 47fe144fc24af14f9c2990918a7b3e707ea952bf revision date: 2015-09-23 03:00:07 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v3.4.3, with hash b4cbecbc0781e89a309d03b60a1f75f8499250e6 from 2015-02-25 12:15:33+00:00 ------------------------------------------------------------------------------------------ benchmark relative change since change since current rev with std_dev* last run v3.4.3 regrtest PGO ------------------------------------------------------------------------------------------ :-) django_v2 0.31792% 0.21980% 8.25387% 15.74786% :-( pybench 0.08705% 0.35237% -2.27273% 9.19412% :-( regex_v8 2.78253% -0.33535% -5.48961% 7.19399% :-| nbody 0.13701% 1.44192% 0.37040% 6.25317% :-( json_dump_v2 0.19448% -0.34730% -2.76662% 13.09287% :-| normal_startup 0.75188% -0.01675% -0.29316% 5.76918% ------------------------------------------------------------------------------------------ Note: Benchmark results are measured in seconds. * Relative Standard Deviation (Standard Deviation/Average) Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. From python-checkins at python.org Wed Sep 23 13:49:05 2015 From: python-checkins at python.org (eric.smith) Date: Wed, 23 Sep 2015 11:49:05 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Move_f-string_compilation_?= =?utf-8?q?of_the_expression_earlier=2C_before_the_conversion?= Message-ID: <20150923114905.16581.7322@psf.io> https://hg.python.org/cpython/rev/25e106dbc336 changeset: 98219:25e106dbc336 user: Eric V. Smith date: Wed Sep 23 07:49:00 2015 -0400 summary: Move f-string compilation of the expression earlier, before the conversion character and format_spec are checked. This allows for error messages that more closely match what a user would expect. files: Lib/test/test_fstring.py | 9 +++ Python/ast.c | 66 +++++++++++++++++++++------ 2 files changed, 60 insertions(+), 15 deletions(-) diff --git a/Lib/test/test_fstring.py b/Lib/test/test_fstring.py --- a/Lib/test/test_fstring.py +++ b/Lib/test/test_fstring.py @@ -287,6 +287,15 @@ "f' { } '", r"f'{\n}'", r"f'{\n \n}'", + + # Catch the empty expression before the + # invalid conversion. + "f'{!x}'", + "f'{ !xr}'", + "f'{!x:}'", + "f'{!x:a}'", + "f'{ !xr:}'", + "f'{ !xr:a}'", ]) def test_parens_in_expressions(self): diff --git a/Python/ast.c b/Python/ast.c --- a/Python/ast.c +++ b/Python/ast.c @@ -4007,13 +4007,14 @@ expression. This is to allow strings with embedded newlines, for example. */ static expr_ty -fstring_expression_compile(PyObject *str, Py_ssize_t expr_start, - Py_ssize_t expr_end, PyArena *arena) +fstring_compile_expr(PyObject *str, Py_ssize_t expr_start, + Py_ssize_t expr_end, PyArena *arena) { PyCompilerFlags cf; mod_ty mod; char *utf_expr; Py_ssize_t i; + Py_UCS4 end_ch = -1; int all_whitespace; PyObject *sub = NULL; @@ -4023,6 +4024,16 @@ assert(str); + assert(expr_start >= 0 && expr_start < PyUnicode_GET_LENGTH(str)); + assert(expr_end >= 0 && expr_end < PyUnicode_GET_LENGTH(str)); + assert(expr_end >= expr_start); + + /* There has to be at least on character on each side of the + expression inside this str. This will have been caught before + we're called. */ + assert(expr_start >= 1); + assert(expr_end <= PyUnicode_GET_LENGTH(str)-1); + /* If the substring is all whitespace, it's an error. We need to catch this here, and not when we call PyParser_ASTFromString, because turning the expression '' in to '()' would go from @@ -4049,10 +4060,17 @@ string directly. */ if (expr_start-1 == 0 && expr_end+1 == PyUnicode_GET_LENGTH(str)) { - /* No need to actually remember these characters, because we - know they must be braces. */ + /* If str is well formed, then the first and last chars must + be '{' and '}', respectively. But, if there's a syntax + error, for example f'{3!', then the last char won't be a + closing brace. So, remember the last character we read in + order for us to restore it. */ + end_ch = PyUnicode_ReadChar(str, expr_end-expr_start+1); + assert(end_ch != (Py_UCS4)-1); + + /* In all cases, however, start_ch must be '{'. */ assert(PyUnicode_ReadChar(str, 0) == '{'); - assert(PyUnicode_ReadChar(str, expr_end-expr_start+1) == '}'); + sub = str; } else { /* Create a substring object. It must be a new object, with @@ -4064,21 +4082,23 @@ decref_sub = 1; /* Remember to deallocate it on error. */ } + /* Put () around the expression. */ if (PyUnicode_WriteChar(sub, 0, '(') < 0 || PyUnicode_WriteChar(sub, expr_end-expr_start+1, ')') < 0) goto error; - cf.cf_flags = PyCF_ONLY_AST; - /* No need to free the memory returned here: it's managed by the string. */ utf_expr = PyUnicode_AsUTF8(sub); if (!utf_expr) goto error; + + cf.cf_flags = PyCF_ONLY_AST; mod = PyParser_ASTFromString(utf_expr, "", Py_eval_input, &cf, arena); if (!mod) goto error; + if (sub != str) /* Clear instead of decref in case we ever modify this code to change the error handling: this is safest because the XDECREF won't try @@ -4089,9 +4109,10 @@ Py_CLEAR(sub); else { assert(!decref_sub); + assert(end_ch != (Py_UCS4)-1); /* Restore str, which we earlier modified directly. */ if (PyUnicode_WriteChar(str, 0, '{') < 0 || - PyUnicode_WriteChar(str, expr_end-expr_start+1, '}') < 0) + PyUnicode_WriteChar(str, expr_end-expr_start+1, end_ch) < 0) goto error; } return mod->v.Expression.body; @@ -4100,6 +4121,18 @@ /* Only decref sub if it was the result of a call to SubString. */ if (decref_sub) Py_XDECREF(sub); + + if (end_ch != (Py_UCS4)-1) { + /* We only get here if we modified str. Make sure that's the + case: str will be equal to sub. */ + if (str == sub) { + /* Don't check the error, because we've already set the + error state (that's why we're in 'error', after + all). */ + PyUnicode_WriteChar(str, 0, '{'); + PyUnicode_WriteChar(str, expr_end-expr_start+1, end_ch); + } + } return NULL; } @@ -4331,9 +4364,18 @@ return -1; } - /* Check for a conversion char, if present. */ if (*ofs >= PyUnicode_GET_LENGTH(str)) goto unexpected_end_of_string; + + /* Compile the expression as soon as possible, so we show errors + related to the expression before errors related to the + conversion or format_spec. */ + simple_expression = fstring_compile_expr(str, expr_start, expr_end, + c->c_arena); + if (!simple_expression) + return -1; + + /* Check for a conversion char, if present. */ if (PyUnicode_READ(kind, data, *ofs) == '!') { *ofs += 1; if (*ofs >= PyUnicode_GET_LENGTH(str)) @@ -4374,12 +4416,6 @@ assert(PyUnicode_READ(kind, data, *ofs) == '}'); *ofs += 1; - /* Compile the expression. */ - simple_expression = fstring_expression_compile(str, expr_start, expr_end, - c->c_arena); - if (!simple_expression) - return -1; - /* And now create the FormattedValue node that represents this entire expression with the conversion and format spec. */ *expression = FormattedValue(simple_expression, (int)conversion, -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 23 14:00:08 2015 From: python-checkins at python.org (eric.smith) Date: Wed, 23 Sep 2015 12:00:08 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Added_more_f-string_test_f?= =?utf-8?q?or_empty_expressions=2E?= Message-ID: <20150923120006.3652.14681@psf.io> https://hg.python.org/cpython/rev/5a6d4fc4224f changeset: 98220:5a6d4fc4224f user: Eric V. Smith date: Wed Sep 23 08:00:01 2015 -0400 summary: Added more f-string test for empty expressions. files: Lib/test/test_fstring.py | 3 +++ 1 files changed, 3 insertions(+), 0 deletions(-) diff --git a/Lib/test/test_fstring.py b/Lib/test/test_fstring.py --- a/Lib/test/test_fstring.py +++ b/Lib/test/test_fstring.py @@ -296,6 +296,9 @@ "f'{!x:a}'", "f'{ !xr:}'", "f'{ !xr:a}'", + + "f'{!}'", + "f'{:}'", ]) def test_parens_in_expressions(self): -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 23 16:24:54 2015 From: python-checkins at python.org (eric.smith) Date: Wed, 23 Sep 2015 14:24:54 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_f-strings=3A_More_tests_fo?= =?utf-8?q?r_empty_expressions_along_with_missing_closing_braces=2E?= Message-ID: <20150923142450.81619.99002@psf.io> https://hg.python.org/cpython/rev/288681785d26 changeset: 98221:288681785d26 user: Eric V. Smith date: Wed Sep 23 10:24:43 2015 -0400 summary: f-strings: More tests for empty expressions along with missing closing braces. files: Lib/test/test_fstring.py | 7 +++++++ 1 files changed, 7 insertions(+), 0 deletions(-) diff --git a/Lib/test/test_fstring.py b/Lib/test/test_fstring.py --- a/Lib/test/test_fstring.py +++ b/Lib/test/test_fstring.py @@ -299,6 +299,13 @@ "f'{!}'", "f'{:}'", + + # We find the empty expression before the + # missing closing brace. + "f'{!'", + "f'{!s:'", + "f'{:'", + "f'{:x'", ]) def test_parens_in_expressions(self): -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 23 23:11:50 2015 From: python-checkins at python.org (victor.stinner) Date: Wed, 23 Sep 2015 21:11:50 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2325220=3A_Create_L?= =?utf-8?q?ib/test/libregrtest/?= Message-ID: <20150923211149.115474.31297@psf.io> https://hg.python.org/cpython/rev/eaf9a99b6bb8 changeset: 98222:eaf9a99b6bb8 user: Victor Stinner date: Wed Sep 23 23:04:18 2015 +0200 summary: Issue #25220: Create Lib/test/libregrtest/ Start to split regrtest.py into smaller parts with the creation of Lib/test/libregrtest/cmdline.py. files: Lib/test/libregrtest/__init__.py | 1 + Lib/test/libregrtest/cmdline.py | 1268 +----------------- Lib/test/regrtest.py | 332 +---- Lib/test/test_regrtest.py | 4 +- 4 files changed, 14 insertions(+), 1591 deletions(-) diff --git a/Lib/test/libregrtest/__init__.py b/Lib/test/libregrtest/__init__.py new file mode 100644 --- /dev/null +++ b/Lib/test/libregrtest/__init__.py @@ -0,0 +1,1 @@ +from test.libregrtest.cmdline import _parse_args, RESOURCE_NAMES diff --git a/Lib/test/regrtest.py b/Lib/test/libregrtest/cmdline.py old mode 100755 new mode 100644 copy from Lib/test/regrtest.py copy to Lib/test/libregrtest/cmdline.py --- a/Lib/test/regrtest.py +++ b/Lib/test/libregrtest/cmdline.py @@ -1,10 +1,8 @@ -#! /usr/bin/env python3 +import argparse +import faulthandler +import os -""" -Script to run Python regression tests. - -Run this script with -h or --help for documentation. -""" +from test import support USAGE = """\ python -m test [options] [test_name1 [test_name2 ...]] @@ -119,102 +117,16 @@ option '-uall,-gui'. """ -# We import importlib *ASAP* in order to test #15386 -import importlib - -import argparse -import builtins -import faulthandler -import io -import json -import locale -import logging -import os -import platform -import random -import re -import shutil -import signal -import sys -import sysconfig -import tempfile -import time -import traceback -import unittest -import warnings -from inspect import isabstract - -try: - import threading -except ImportError: - threading = None -try: - import _multiprocessing, multiprocessing.process -except ImportError: - multiprocessing = None - - -# Some times __path__ and __file__ are not absolute (e.g. while running from -# Lib/) and, if we change the CWD to run the tests in a temporary dir, some -# imports might fail. This affects only the modules imported before os.chdir(). -# These modules are searched first in sys.path[0] (so '' -- the CWD) and if -# they are found in the CWD their __file__ and __path__ will be relative (this -# happens before the chdir). All the modules imported after the chdir, are -# not found in the CWD, and since the other paths in sys.path[1:] are absolute -# (site.py absolutize them), the __file__ and __path__ will be absolute too. -# Therefore it is necessary to absolutize manually the __file__ and __path__ of -# the packages to prevent later imports to fail when the CWD is different. -for module in sys.modules.values(): - if hasattr(module, '__path__'): - module.__path__ = [os.path.abspath(path) for path in module.__path__] - if hasattr(module, '__file__'): - module.__file__ = os.path.abspath(module.__file__) - - -# MacOSX (a.k.a. Darwin) has a default stack size that is too small -# for deeply recursive regular expressions. We see this as crashes in -# the Python test suite when running test_re.py and test_sre.py. The -# fix is to set the stack limit to 2048. -# This approach may also be useful for other Unixy platforms that -# suffer from small default stack limits. -if sys.platform == 'darwin': - try: - import resource - except ImportError: - pass - else: - soft, hard = resource.getrlimit(resource.RLIMIT_STACK) - newsoft = min(hard, max(soft, 1024*2048)) - resource.setrlimit(resource.RLIMIT_STACK, (newsoft, hard)) - -# Test result constants. -PASSED = 1 -FAILED = 0 -ENV_CHANGED = -1 -SKIPPED = -2 -RESOURCE_DENIED = -3 -INTERRUPTED = -4 -CHILD_ERROR = -5 # error in a child process - -from test import support RESOURCE_NAMES = ('audio', 'curses', 'largefile', 'network', 'decimal', 'cpu', 'subprocess', 'urlfetch', 'gui') -# When tests are run from the Python build directory, it is best practice -# to keep the test files in a subfolder. This eases the cleanup of leftover -# files using the "make distclean" command. -if sysconfig.is_python_build(): - TEMPDIR = os.path.join(sysconfig.get_config_var('srcdir'), 'build') -else: - TEMPDIR = tempfile.gettempdir() -TEMPDIR = os.path.abspath(TEMPDIR) - class _ArgParser(argparse.ArgumentParser): def error(self, message): super().error(message + "\nPass -h or --help for complete help.") + def _create_parser(): # Set prog to prevent the uninformative "__main__.py" from displaying in # error messages when using "python -m test ...". @@ -328,11 +240,13 @@ return parser + def relative_filename(string): # CWD is replaced with a temporary dir before calling main(), so we # join it with the saved CWD so it ends up where the user expects. return os.path.join(support.SAVEDCWD, string) + def huntrleaks(string): args = string.split(':') if len(args) not in (2, 3): @@ -343,6 +257,7 @@ fname = args[2] if len(args) > 2 and args[2] else 'reflog.txt' return nwarmup, ntracked, fname + def resources_list(string): u = [x.lower() for x in string.split(',')] for r in u: @@ -354,6 +269,7 @@ raise argparse.ArgumentTypeError('invalid resource: ' + r) return u + def _parse_args(args, **kwargs): # Defaults ns = argparse.Namespace(testdir=None, verbose=0, quiet=False, @@ -422,1169 +338,3 @@ ns.randomize = True return ns - - -def run_test_in_subprocess(testname, ns): - """Run the given test in a subprocess with --slaveargs. - - ns is the option Namespace parsed from command-line arguments. regrtest - is invoked in a subprocess with the --slaveargs argument; when the - subprocess exits, its return code, stdout and stderr are returned as a - 3-tuple. - """ - from subprocess import Popen, PIPE - base_cmd = ([sys.executable] + support.args_from_interpreter_flags() + - ['-X', 'faulthandler', '-m', 'test.regrtest']) - - slaveargs = ( - (testname, ns.verbose, ns.quiet), - dict(huntrleaks=ns.huntrleaks, - use_resources=ns.use_resources, - output_on_failure=ns.verbose3, - timeout=ns.timeout, failfast=ns.failfast, - match_tests=ns.match_tests)) - # Running the child from the same working directory as regrtest's original - # invocation ensures that TEMPDIR for the child is the same when - # sysconfig.is_python_build() is true. See issue 15300. - popen = Popen(base_cmd + ['--slaveargs', json.dumps(slaveargs)], - stdout=PIPE, stderr=PIPE, - universal_newlines=True, - close_fds=(os.name != 'nt'), - cwd=support.SAVEDCWD) - stdout, stderr = popen.communicate() - retcode = popen.wait() - return retcode, stdout, stderr - - -def main(tests=None, **kwargs): - """Execute a test suite. - - This also parses command-line options and modifies its behavior - accordingly. - - tests -- a list of strings containing test names (optional) - testdir -- the directory in which to look for tests (optional) - - Users other than the Python test suite will certainly want to - specify testdir; if it's omitted, the directory containing the - Python test suite is searched for. - - If the tests argument is omitted, the tests listed on the - command-line will be used. If that's empty, too, then all *.py - files beginning with test_ will be used. - - The other default arguments (verbose, quiet, exclude, - single, randomize, findleaks, use_resources, trace, coverdir, - print_slow, and random_seed) allow programmers calling main() - directly to set the values that would normally be set by flags - on the command line. - """ - # Display the Python traceback on fatal errors (e.g. segfault) - faulthandler.enable(all_threads=True) - - # Display the Python traceback on SIGALRM or SIGUSR1 signal - signals = [] - if hasattr(signal, 'SIGALRM'): - signals.append(signal.SIGALRM) - if hasattr(signal, 'SIGUSR1'): - signals.append(signal.SIGUSR1) - for signum in signals: - faulthandler.register(signum, chain=True) - - replace_stdout() - - support.record_original_stdout(sys.stdout) - - ns = _parse_args(sys.argv[1:], **kwargs) - - if ns.huntrleaks: - # Avoid false positives due to various caches - # filling slowly with random data: - warm_caches() - if ns.memlimit is not None: - support.set_memlimit(ns.memlimit) - if ns.threshold is not None: - import gc - gc.set_threshold(ns.threshold) - if ns.nowindows: - import msvcrt - msvcrt.SetErrorMode(msvcrt.SEM_FAILCRITICALERRORS| - msvcrt.SEM_NOALIGNMENTFAULTEXCEPT| - msvcrt.SEM_NOGPFAULTERRORBOX| - msvcrt.SEM_NOOPENFILEERRORBOX) - try: - msvcrt.CrtSetReportMode - except AttributeError: - # release build - pass - else: - for m in [msvcrt.CRT_WARN, msvcrt.CRT_ERROR, msvcrt.CRT_ASSERT]: - msvcrt.CrtSetReportMode(m, msvcrt.CRTDBG_MODE_FILE) - msvcrt.CrtSetReportFile(m, msvcrt.CRTDBG_FILE_STDERR) - if ns.wait: - input("Press any key to continue...") - - if ns.slaveargs is not None: - args, kwargs = json.loads(ns.slaveargs) - if kwargs.get('huntrleaks'): - unittest.BaseTestSuite._cleanup = False - try: - result = runtest(*args, **kwargs) - except KeyboardInterrupt: - result = INTERRUPTED, '' - except BaseException as e: - traceback.print_exc() - result = CHILD_ERROR, str(e) - sys.stdout.flush() - print() # Force a newline (just in case) - print(json.dumps(result)) - sys.exit(0) - - good = [] - bad = [] - skipped = [] - resource_denieds = [] - environment_changed = [] - interrupted = False - - if ns.findleaks: - try: - import gc - except ImportError: - print('No GC available, disabling findleaks.') - ns.findleaks = False - else: - # Uncomment the line below to report garbage that is not - # freeable by reference counting alone. By default only - # garbage that is not collectable by the GC is reported. - #gc.set_debug(gc.DEBUG_SAVEALL) - found_garbage = [] - - if ns.huntrleaks: - unittest.BaseTestSuite._cleanup = False - - if ns.single: - filename = os.path.join(TEMPDIR, 'pynexttest') - try: - with open(filename, 'r') as fp: - next_test = fp.read().strip() - tests = [next_test] - except OSError: - pass - - if ns.fromfile: - tests = [] - with open(os.path.join(support.SAVEDCWD, ns.fromfile)) as fp: - count_pat = re.compile(r'\[\s*\d+/\s*\d+\]') - for line in fp: - line = count_pat.sub('', line) - guts = line.split() # assuming no test has whitespace in its name - if guts and not guts[0].startswith('#'): - tests.extend(guts) - - # Strip .py extensions. - removepy(ns.args) - removepy(tests) - - stdtests = STDTESTS[:] - nottests = NOTTESTS.copy() - if ns.exclude: - for arg in ns.args: - if arg in stdtests: - stdtests.remove(arg) - nottests.add(arg) - ns.args = [] - - # For a partial run, we do not need to clutter the output. - if ns.verbose or ns.header or not (ns.quiet or ns.single or tests or ns.args): - # Print basic platform information - print("==", platform.python_implementation(), *sys.version.split()) - print("== ", platform.platform(aliased=True), - "%s-endian" % sys.byteorder) - print("== ", "hash algorithm:", sys.hash_info.algorithm, - "64bit" if sys.maxsize > 2**32 else "32bit") - print("== ", os.getcwd()) - print("Testing with flags:", sys.flags) - - # if testdir is set, then we are not running the python tests suite, so - # don't add default tests to be executed or skipped (pass empty values) - if ns.testdir: - alltests = findtests(ns.testdir, list(), set()) - else: - alltests = findtests(ns.testdir, stdtests, nottests) - - selected = tests or ns.args or alltests - if ns.single: - selected = selected[:1] - try: - next_single_test = alltests[alltests.index(selected[0])+1] - except IndexError: - next_single_test = None - # Remove all the selected tests that precede start if it's set. - if ns.start: - try: - del selected[:selected.index(ns.start)] - except ValueError: - print("Couldn't find starting test (%s), using all tests" % ns.start) - if ns.randomize: - if ns.random_seed is None: - ns.random_seed = random.randrange(10000000) - random.seed(ns.random_seed) - print("Using random seed", ns.random_seed) - random.shuffle(selected) - if ns.trace: - import trace, tempfile - tracer = trace.Trace(ignoredirs=[sys.base_prefix, sys.base_exec_prefix, - tempfile.gettempdir()], - trace=False, count=True) - - test_times = [] - support.verbose = ns.verbose # Tell tests to be moderately quiet - support.use_resources = ns.use_resources - save_modules = sys.modules.keys() - - def accumulate_result(test, result): - ok, test_time = result - test_times.append((test_time, test)) - if ok == PASSED: - good.append(test) - elif ok == FAILED: - bad.append(test) - elif ok == ENV_CHANGED: - environment_changed.append(test) - elif ok == SKIPPED: - skipped.append(test) - elif ok == RESOURCE_DENIED: - skipped.append(test) - resource_denieds.append(test) - - if ns.forever: - def test_forever(tests=list(selected)): - while True: - for test in tests: - yield test - if bad: - return - tests = test_forever() - test_count = '' - test_count_width = 3 - else: - tests = iter(selected) - test_count = '/{}'.format(len(selected)) - test_count_width = len(test_count) - 1 - - if ns.use_mp: - try: - from threading import Thread - except ImportError: - print("Multiprocess option requires thread support") - sys.exit(2) - from queue import Queue - debug_output_pat = re.compile(r"\[\d+ refs, \d+ blocks\]$") - output = Queue() - pending = MultiprocessTests(tests) - def work(): - # A worker thread. - try: - while True: - try: - test = next(pending) - except StopIteration: - output.put((None, None, None, None)) - return - retcode, stdout, stderr = run_test_in_subprocess(test, ns) - # Strip last refcount output line if it exists, since it - # comes from the shutdown of the interpreter in the subcommand. - stderr = debug_output_pat.sub("", stderr) - stdout, _, result = stdout.strip().rpartition("\n") - if retcode != 0: - result = (CHILD_ERROR, "Exit code %s" % retcode) - output.put((test, stdout.rstrip(), stderr.rstrip(), result)) - return - if not result: - output.put((None, None, None, None)) - return - result = json.loads(result) - output.put((test, stdout.rstrip(), stderr.rstrip(), result)) - except BaseException: - output.put((None, None, None, None)) - raise - workers = [Thread(target=work) for i in range(ns.use_mp)] - for worker in workers: - worker.start() - finished = 0 - test_index = 1 - try: - while finished < ns.use_mp: - test, stdout, stderr, result = output.get() - if test is None: - finished += 1 - continue - accumulate_result(test, result) - if not ns.quiet: - fmt = "[{1:{0}}{2}/{3}] {4}" if bad else "[{1:{0}}{2}] {4}" - print(fmt.format( - test_count_width, test_index, test_count, - len(bad), test)) - if stdout: - print(stdout) - if stderr: - print(stderr, file=sys.stderr) - sys.stdout.flush() - sys.stderr.flush() - if result[0] == INTERRUPTED: - raise KeyboardInterrupt - if result[0] == CHILD_ERROR: - raise Exception("Child error on {}: {}".format(test, result[1])) - test_index += 1 - except KeyboardInterrupt: - interrupted = True - pending.interrupted = True - for worker in workers: - worker.join() - else: - for test_index, test in enumerate(tests, 1): - if not ns.quiet: - fmt = "[{1:{0}}{2}/{3}] {4}" if bad else "[{1:{0}}{2}] {4}" - print(fmt.format( - test_count_width, test_index, test_count, len(bad), test)) - sys.stdout.flush() - if ns.trace: - # If we're tracing code coverage, then we don't exit with status - # if on a false return value from main. - tracer.runctx('runtest(test, ns.verbose, ns.quiet, timeout=ns.timeout)', - globals=globals(), locals=vars()) - else: - try: - result = runtest(test, ns.verbose, ns.quiet, - ns.huntrleaks, - output_on_failure=ns.verbose3, - timeout=ns.timeout, failfast=ns.failfast, - match_tests=ns.match_tests) - accumulate_result(test, result) - except KeyboardInterrupt: - interrupted = True - break - if ns.findleaks: - gc.collect() - if gc.garbage: - print("Warning: test created", len(gc.garbage), end=' ') - print("uncollectable object(s).") - # move the uncollectable objects somewhere so we don't see - # them again - found_garbage.extend(gc.garbage) - del gc.garbage[:] - # Unload the newly imported modules (best effort finalization) - for module in sys.modules.keys(): - if module not in save_modules and module.startswith("test."): - support.unload(module) - - if interrupted: - # print a newline after ^C - print() - print("Test suite interrupted by signal SIGINT.") - omitted = set(selected) - set(good) - set(bad) - set(skipped) - print(count(len(omitted), "test"), "omitted:") - printlist(omitted) - if good and not ns.quiet: - if not bad and not skipped and not interrupted and len(good) > 1: - print("All", end=' ') - print(count(len(good), "test"), "OK.") - if ns.print_slow: - test_times.sort(reverse=True) - print("10 slowest tests:") - for time, test in test_times[:10]: - print("%s: %.1fs" % (test, time)) - if bad: - print(count(len(bad), "test"), "failed:") - printlist(bad) - if environment_changed: - print("{} altered the execution environment:".format( - count(len(environment_changed), "test"))) - printlist(environment_changed) - if skipped and not ns.quiet: - print(count(len(skipped), "test"), "skipped:") - printlist(skipped) - - if ns.verbose2 and bad: - print("Re-running failed tests in verbose mode") - for test in bad[:]: - print("Re-running test %r in verbose mode" % test) - sys.stdout.flush() - try: - ns.verbose = True - ok = runtest(test, True, ns.quiet, ns.huntrleaks, - timeout=ns.timeout) - except KeyboardInterrupt: - # print a newline separate from the ^C - print() - break - else: - if ok[0] in {PASSED, ENV_CHANGED, SKIPPED, RESOURCE_DENIED}: - bad.remove(test) - else: - if bad: - print(count(len(bad), 'test'), "failed again:") - printlist(bad) - - if ns.single: - if next_single_test: - with open(filename, 'w') as fp: - fp.write(next_single_test + '\n') - else: - os.unlink(filename) - - if ns.trace: - r = tracer.results() - r.write_results(show_missing=True, summary=True, coverdir=ns.coverdir) - - if ns.runleaks: - os.system("leaks %d" % os.getpid()) - - sys.exit(len(bad) > 0 or interrupted) - - -# small set of tests to determine if we have a basically functioning interpreter -# (i.e. if any of these fail, then anything else is likely to follow) -STDTESTS = [ - 'test_grammar', - 'test_opcodes', - 'test_dict', - 'test_builtin', - 'test_exceptions', - 'test_types', - 'test_unittest', - 'test_doctest', - 'test_doctest2', - 'test_support' -] - -# set of tests that we don't want to be executed when using regrtest -NOTTESTS = set() - -def findtests(testdir=None, stdtests=STDTESTS, nottests=NOTTESTS): - """Return a list of all applicable test modules.""" - testdir = findtestdir(testdir) - names = os.listdir(testdir) - tests = [] - others = set(stdtests) | nottests - for name in names: - mod, ext = os.path.splitext(name) - if mod[:5] == "test_" and ext in (".py", "") and mod not in others: - tests.append(mod) - return stdtests + sorted(tests) - -# We do not use a generator so multiple threads can call next(). -class MultiprocessTests(object): - - """A thread-safe iterator over tests for multiprocess mode.""" - - def __init__(self, tests): - self.interrupted = False - self.lock = threading.Lock() - self.tests = tests - - def __iter__(self): - return self - - def __next__(self): - with self.lock: - if self.interrupted: - raise StopIteration('tests interrupted') - return next(self.tests) - -def replace_stdout(): - """Set stdout encoder error handler to backslashreplace (as stderr error - handler) to avoid UnicodeEncodeError when printing a traceback""" - import atexit - - stdout = sys.stdout - sys.stdout = open(stdout.fileno(), 'w', - encoding=stdout.encoding, - errors="backslashreplace", - closefd=False, - newline='\n') - - def restore_stdout(): - sys.stdout.close() - sys.stdout = stdout - atexit.register(restore_stdout) - -def runtest(test, verbose, quiet, - huntrleaks=False, use_resources=None, - output_on_failure=False, failfast=False, match_tests=None, - timeout=None): - """Run a single test. - - test -- the name of the test - verbose -- if true, print more messages - quiet -- if true, don't print 'skipped' messages (probably redundant) - huntrleaks -- run multiple times to test for leaks; requires a debug - build; a triple corresponding to -R's three arguments - use_resources -- list of extra resources to use - output_on_failure -- if true, display test output on failure - timeout -- dump the traceback and exit if a test takes more than - timeout seconds - failfast, match_tests -- See regrtest command-line flags for these. - - Returns the tuple result, test_time, where result is one of the constants: - INTERRUPTED KeyboardInterrupt when run under -j - RESOURCE_DENIED test skipped because resource denied - SKIPPED test skipped for some other reason - ENV_CHANGED test failed because it changed the execution environment - FAILED test failed - PASSED test passed - """ - - if use_resources is not None: - support.use_resources = use_resources - use_timeout = (timeout is not None) - if use_timeout: - faulthandler.dump_traceback_later(timeout, exit=True) - try: - support.match_tests = match_tests - if failfast: - support.failfast = True - if output_on_failure: - support.verbose = True - - # Reuse the same instance to all calls to runtest(). Some - # tests keep a reference to sys.stdout or sys.stderr - # (eg. test_argparse). - if runtest.stringio is None: - stream = io.StringIO() - runtest.stringio = stream - else: - stream = runtest.stringio - stream.seek(0) - stream.truncate() - - orig_stdout = sys.stdout - orig_stderr = sys.stderr - try: - sys.stdout = stream - sys.stderr = stream - result = runtest_inner(test, verbose, quiet, huntrleaks, - display_failure=False) - if result[0] == FAILED: - output = stream.getvalue() - orig_stderr.write(output) - orig_stderr.flush() - finally: - sys.stdout = orig_stdout - sys.stderr = orig_stderr - else: - support.verbose = verbose # Tell tests to be moderately quiet - result = runtest_inner(test, verbose, quiet, huntrleaks, - display_failure=not verbose) - return result - finally: - if use_timeout: - faulthandler.cancel_dump_traceback_later() - cleanup_test_droppings(test, verbose) -runtest.stringio = None - -# Unit tests are supposed to leave the execution environment unchanged -# once they complete. But sometimes tests have bugs, especially when -# tests fail, and the changes to environment go on to mess up other -# tests. This can cause issues with buildbot stability, since tests -# are run in random order and so problems may appear to come and go. -# There are a few things we can save and restore to mitigate this, and -# the following context manager handles this task. - -class saved_test_environment: - """Save bits of the test environment and restore them at block exit. - - with saved_test_environment(testname, verbose, quiet): - #stuff - - Unless quiet is True, a warning is printed to stderr if any of - the saved items was changed by the test. The attribute 'changed' - is initially False, but is set to True if a change is detected. - - If verbose is more than 1, the before and after state of changed - items is also printed. - """ - - changed = False - - def __init__(self, testname, verbose=0, quiet=False): - self.testname = testname - self.verbose = verbose - self.quiet = quiet - - # To add things to save and restore, add a name XXX to the resources list - # and add corresponding get_XXX/restore_XXX functions. get_XXX should - # return the value to be saved and compared against a second call to the - # get function when test execution completes. restore_XXX should accept - # the saved value and restore the resource using it. It will be called if - # and only if a change in the value is detected. - # - # Note: XXX will have any '.' replaced with '_' characters when determining - # the corresponding method names. - - resources = ('sys.argv', 'cwd', 'sys.stdin', 'sys.stdout', 'sys.stderr', - 'os.environ', 'sys.path', 'sys.path_hooks', '__import__', - 'warnings.filters', 'asyncore.socket_map', - 'logging._handlers', 'logging._handlerList', 'sys.gettrace', - 'sys.warnoptions', - # multiprocessing.process._cleanup() may release ref - # to a thread, so check processes first. - 'multiprocessing.process._dangling', 'threading._dangling', - 'sysconfig._CONFIG_VARS', 'sysconfig._INSTALL_SCHEMES', - 'files', 'locale', 'warnings.showwarning', - ) - - def get_sys_argv(self): - return id(sys.argv), sys.argv, sys.argv[:] - def restore_sys_argv(self, saved_argv): - sys.argv = saved_argv[1] - sys.argv[:] = saved_argv[2] - - def get_cwd(self): - return os.getcwd() - def restore_cwd(self, saved_cwd): - os.chdir(saved_cwd) - - def get_sys_stdout(self): - return sys.stdout - def restore_sys_stdout(self, saved_stdout): - sys.stdout = saved_stdout - - def get_sys_stderr(self): - return sys.stderr - def restore_sys_stderr(self, saved_stderr): - sys.stderr = saved_stderr - - def get_sys_stdin(self): - return sys.stdin - def restore_sys_stdin(self, saved_stdin): - sys.stdin = saved_stdin - - def get_os_environ(self): - return id(os.environ), os.environ, dict(os.environ) - def restore_os_environ(self, saved_environ): - os.environ = saved_environ[1] - os.environ.clear() - os.environ.update(saved_environ[2]) - - def get_sys_path(self): - return id(sys.path), sys.path, sys.path[:] - def restore_sys_path(self, saved_path): - sys.path = saved_path[1] - sys.path[:] = saved_path[2] - - def get_sys_path_hooks(self): - return id(sys.path_hooks), sys.path_hooks, sys.path_hooks[:] - def restore_sys_path_hooks(self, saved_hooks): - sys.path_hooks = saved_hooks[1] - sys.path_hooks[:] = saved_hooks[2] - - def get_sys_gettrace(self): - return sys.gettrace() - def restore_sys_gettrace(self, trace_fxn): - sys.settrace(trace_fxn) - - def get___import__(self): - return builtins.__import__ - def restore___import__(self, import_): - builtins.__import__ = import_ - - def get_warnings_filters(self): - return id(warnings.filters), warnings.filters, warnings.filters[:] - def restore_warnings_filters(self, saved_filters): - warnings.filters = saved_filters[1] - warnings.filters[:] = saved_filters[2] - - def get_asyncore_socket_map(self): - asyncore = sys.modules.get('asyncore') - # XXX Making a copy keeps objects alive until __exit__ gets called. - return asyncore and asyncore.socket_map.copy() or {} - def restore_asyncore_socket_map(self, saved_map): - asyncore = sys.modules.get('asyncore') - if asyncore is not None: - asyncore.close_all(ignore_all=True) - asyncore.socket_map.update(saved_map) - - def get_shutil_archive_formats(self): - # we could call get_archives_formats() but that only returns the - # registry keys; we want to check the values too (the functions that - # are registered) - return shutil._ARCHIVE_FORMATS, shutil._ARCHIVE_FORMATS.copy() - def restore_shutil_archive_formats(self, saved): - shutil._ARCHIVE_FORMATS = saved[0] - shutil._ARCHIVE_FORMATS.clear() - shutil._ARCHIVE_FORMATS.update(saved[1]) - - def get_shutil_unpack_formats(self): - return shutil._UNPACK_FORMATS, shutil._UNPACK_FORMATS.copy() - def restore_shutil_unpack_formats(self, saved): - shutil._UNPACK_FORMATS = saved[0] - shutil._UNPACK_FORMATS.clear() - shutil._UNPACK_FORMATS.update(saved[1]) - - def get_logging__handlers(self): - # _handlers is a WeakValueDictionary - return id(logging._handlers), logging._handlers, logging._handlers.copy() - def restore_logging__handlers(self, saved_handlers): - # Can't easily revert the logging state - pass - - def get_logging__handlerList(self): - # _handlerList is a list of weakrefs to handlers - return id(logging._handlerList), logging._handlerList, logging._handlerList[:] - def restore_logging__handlerList(self, saved_handlerList): - # Can't easily revert the logging state - pass - - def get_sys_warnoptions(self): - return id(sys.warnoptions), sys.warnoptions, sys.warnoptions[:] - def restore_sys_warnoptions(self, saved_options): - sys.warnoptions = saved_options[1] - sys.warnoptions[:] = saved_options[2] - - # Controlling dangling references to Thread objects can make it easier - # to track reference leaks. - def get_threading__dangling(self): - if not threading: - return None - # This copies the weakrefs without making any strong reference - return threading._dangling.copy() - def restore_threading__dangling(self, saved): - if not threading: - return - threading._dangling.clear() - threading._dangling.update(saved) - - # Same for Process objects - def get_multiprocessing_process__dangling(self): - if not multiprocessing: - return None - # Unjoined process objects can survive after process exits - multiprocessing.process._cleanup() - # This copies the weakrefs without making any strong reference - return multiprocessing.process._dangling.copy() - def restore_multiprocessing_process__dangling(self, saved): - if not multiprocessing: - return - multiprocessing.process._dangling.clear() - multiprocessing.process._dangling.update(saved) - - def get_sysconfig__CONFIG_VARS(self): - # make sure the dict is initialized - sysconfig.get_config_var('prefix') - return (id(sysconfig._CONFIG_VARS), sysconfig._CONFIG_VARS, - dict(sysconfig._CONFIG_VARS)) - def restore_sysconfig__CONFIG_VARS(self, saved): - sysconfig._CONFIG_VARS = saved[1] - sysconfig._CONFIG_VARS.clear() - sysconfig._CONFIG_VARS.update(saved[2]) - - def get_sysconfig__INSTALL_SCHEMES(self): - return (id(sysconfig._INSTALL_SCHEMES), sysconfig._INSTALL_SCHEMES, - sysconfig._INSTALL_SCHEMES.copy()) - def restore_sysconfig__INSTALL_SCHEMES(self, saved): - sysconfig._INSTALL_SCHEMES = saved[1] - sysconfig._INSTALL_SCHEMES.clear() - sysconfig._INSTALL_SCHEMES.update(saved[2]) - - def get_files(self): - return sorted(fn + ('/' if os.path.isdir(fn) else '') - for fn in os.listdir()) - def restore_files(self, saved_value): - fn = support.TESTFN - if fn not in saved_value and (fn + '/') not in saved_value: - if os.path.isfile(fn): - support.unlink(fn) - elif os.path.isdir(fn): - support.rmtree(fn) - - _lc = [getattr(locale, lc) for lc in dir(locale) - if lc.startswith('LC_')] - def get_locale(self): - pairings = [] - for lc in self._lc: - try: - pairings.append((lc, locale.setlocale(lc, None))) - except (TypeError, ValueError): - continue - return pairings - def restore_locale(self, saved): - for lc, setting in saved: - locale.setlocale(lc, setting) - - def get_warnings_showwarning(self): - return warnings.showwarning - def restore_warnings_showwarning(self, fxn): - warnings.showwarning = fxn - - def resource_info(self): - for name in self.resources: - method_suffix = name.replace('.', '_') - get_name = 'get_' + method_suffix - restore_name = 'restore_' + method_suffix - yield name, getattr(self, get_name), getattr(self, restore_name) - - def __enter__(self): - self.saved_values = dict((name, get()) for name, get, restore - in self.resource_info()) - return self - - def __exit__(self, exc_type, exc_val, exc_tb): - saved_values = self.saved_values - del self.saved_values - for name, get, restore in self.resource_info(): - current = get() - original = saved_values.pop(name) - # Check for changes to the resource's value - if current != original: - self.changed = True - restore(original) - if not self.quiet: - print("Warning -- {} was modified by {}".format( - name, self.testname), - file=sys.stderr) - if self.verbose > 1: - print(" Before: {}\n After: {} ".format( - original, current), - file=sys.stderr) - return False - - -def runtest_inner(test, verbose, quiet, - huntrleaks=False, display_failure=True): - support.unload(test) - - test_time = 0.0 - refleak = False # True if the test leaked references. - try: - if test.startswith('test.'): - abstest = test - else: - # Always import it from the test package - abstest = 'test.' + test - with saved_test_environment(test, verbose, quiet) as environment: - start_time = time.time() - the_module = importlib.import_module(abstest) - # If the test has a test_main, that will run the appropriate - # tests. If not, use normal unittest test loading. - test_runner = getattr(the_module, "test_main", None) - if test_runner is None: - def test_runner(): - loader = unittest.TestLoader() - tests = loader.loadTestsFromModule(the_module) - for error in loader.errors: - print(error, file=sys.stderr) - if loader.errors: - raise Exception("errors while loading tests") - support.run_unittest(tests) - test_runner() - if huntrleaks: - refleak = dash_R(the_module, test, test_runner, huntrleaks) - test_time = time.time() - start_time - except support.ResourceDenied as msg: - if not quiet: - print(test, "skipped --", msg) - sys.stdout.flush() - return RESOURCE_DENIED, test_time - except unittest.SkipTest as msg: - if not quiet: - print(test, "skipped --", msg) - sys.stdout.flush() - return SKIPPED, test_time - except KeyboardInterrupt: - raise - except support.TestFailed as msg: - if display_failure: - print("test", test, "failed --", msg, file=sys.stderr) - else: - print("test", test, "failed", file=sys.stderr) - sys.stderr.flush() - return FAILED, test_time - except: - msg = traceback.format_exc() - print("test", test, "crashed --", msg, file=sys.stderr) - sys.stderr.flush() - return FAILED, test_time - else: - if refleak: - return FAILED, test_time - if environment.changed: - return ENV_CHANGED, test_time - return PASSED, test_time - -def cleanup_test_droppings(testname, verbose): - import shutil - import stat - import gc - - # First kill any dangling references to open files etc. - # This can also issue some ResourceWarnings which would otherwise get - # triggered during the following test run, and possibly produce failures. - gc.collect() - - # Try to clean up junk commonly left behind. While tests shouldn't leave - # any files or directories behind, when a test fails that can be tedious - # for it to arrange. The consequences can be especially nasty on Windows, - # since if a test leaves a file open, it cannot be deleted by name (while - # there's nothing we can do about that here either, we can display the - # name of the offending test, which is a real help). - for name in (support.TESTFN, - "db_home", - ): - if not os.path.exists(name): - continue - - if os.path.isdir(name): - kind, nuker = "directory", shutil.rmtree - elif os.path.isfile(name): - kind, nuker = "file", os.unlink - else: - raise SystemError("os.path says %r exists but is neither " - "directory nor file" % name) - - if verbose: - print("%r left behind %s %r" % (testname, kind, name)) - try: - # if we have chmod, fix possible permissions problems - # that might prevent cleanup - if (hasattr(os, 'chmod')): - os.chmod(name, stat.S_IRWXU | stat.S_IRWXG | stat.S_IRWXO) - nuker(name) - except Exception as msg: - print(("%r left behind %s %r and it couldn't be " - "removed: %s" % (testname, kind, name, msg)), file=sys.stderr) - -def dash_R(the_module, test, indirect_test, huntrleaks): - """Run a test multiple times, looking for reference leaks. - - Returns: - False if the test didn't leak references; True if we detected refleaks. - """ - # This code is hackish and inelegant, but it seems to do the job. - import copyreg - import collections.abc - - if not hasattr(sys, 'gettotalrefcount'): - raise Exception("Tracking reference leaks requires a debug build " - "of Python") - - # Save current values for dash_R_cleanup() to restore. - fs = warnings.filters[:] - ps = copyreg.dispatch_table.copy() - pic = sys.path_importer_cache.copy() - try: - import zipimport - except ImportError: - zdc = None # Run unmodified on platforms without zipimport support - else: - zdc = zipimport._zip_directory_cache.copy() - abcs = {} - for abc in [getattr(collections.abc, a) for a in collections.abc.__all__]: - if not isabstract(abc): - continue - for obj in abc.__subclasses__() + [abc]: - abcs[obj] = obj._abc_registry.copy() - - nwarmup, ntracked, fname = huntrleaks - fname = os.path.join(support.SAVEDCWD, fname) - repcount = nwarmup + ntracked - rc_deltas = [0] * repcount - alloc_deltas = [0] * repcount - - print("beginning", repcount, "repetitions", file=sys.stderr) - print(("1234567890"*(repcount//10 + 1))[:repcount], file=sys.stderr) - sys.stderr.flush() - for i in range(repcount): - indirect_test() - alloc_after, rc_after = dash_R_cleanup(fs, ps, pic, zdc, abcs) - sys.stderr.write('.') - sys.stderr.flush() - if i >= nwarmup: - rc_deltas[i] = rc_after - rc_before - alloc_deltas[i] = alloc_after - alloc_before - alloc_before, rc_before = alloc_after, rc_after - print(file=sys.stderr) - # These checkers return False on success, True on failure - def check_rc_deltas(deltas): - return any(deltas) - def check_alloc_deltas(deltas): - # At least 1/3rd of 0s - if 3 * deltas.count(0) < len(deltas): - return True - # Nothing else than 1s, 0s and -1s - if not set(deltas) <= {1,0,-1}: - return True - return False - failed = False - for deltas, item_name, checker in [ - (rc_deltas, 'references', check_rc_deltas), - (alloc_deltas, 'memory blocks', check_alloc_deltas)]: - if checker(deltas): - msg = '%s leaked %s %s, sum=%s' % ( - test, deltas[nwarmup:], item_name, sum(deltas)) - print(msg, file=sys.stderr) - sys.stderr.flush() - with open(fname, "a") as refrep: - print(msg, file=refrep) - refrep.flush() - failed = True - return failed - -def dash_R_cleanup(fs, ps, pic, zdc, abcs): - import gc, copyreg - import _strptime, linecache - import urllib.parse, urllib.request, mimetypes, doctest - import struct, filecmp, collections.abc - from distutils.dir_util import _path_created - from weakref import WeakSet - - # Clear the warnings registry, so they can be displayed again - for mod in sys.modules.values(): - if hasattr(mod, '__warningregistry__'): - del mod.__warningregistry__ - - # Restore some original values. - warnings.filters[:] = fs - copyreg.dispatch_table.clear() - copyreg.dispatch_table.update(ps) - sys.path_importer_cache.clear() - sys.path_importer_cache.update(pic) - try: - import zipimport - except ImportError: - pass # Run unmodified on platforms without zipimport support - else: - zipimport._zip_directory_cache.clear() - zipimport._zip_directory_cache.update(zdc) - - # clear type cache - sys._clear_type_cache() - - # Clear ABC registries, restoring previously saved ABC registries. - for abc in [getattr(collections.abc, a) for a in collections.abc.__all__]: - if not isabstract(abc): - continue - for obj in abc.__subclasses__() + [abc]: - obj._abc_registry = abcs.get(obj, WeakSet()).copy() - obj._abc_cache.clear() - obj._abc_negative_cache.clear() - - # Flush standard output, so that buffered data is sent to the OS and - # associated Python objects are reclaimed. - for stream in (sys.stdout, sys.stderr, sys.__stdout__, sys.__stderr__): - if stream is not None: - stream.flush() - - # Clear assorted module caches. - _path_created.clear() - re.purge() - _strptime._regex_cache.clear() - urllib.parse.clear_cache() - urllib.request.urlcleanup() - linecache.clearcache() - mimetypes._default_mime_types() - filecmp._cache.clear() - struct._clearcache() - doctest.master = None - try: - import ctypes - except ImportError: - # Don't worry about resetting the cache if ctypes is not supported - pass - else: - ctypes._reset_cache() - - # Collect cyclic trash and read memory statistics immediately after. - func1 = sys.getallocatedblocks - func2 = sys.gettotalrefcount - gc.collect() - return func1(), func2() - -def warm_caches(): - # char cache - s = bytes(range(256)) - for i in range(256): - s[i:i+1] - # unicode cache - x = [chr(i) for i in range(256)] - # int cache - x = list(range(-5, 257)) - -def findtestdir(path=None): - return path or os.path.dirname(__file__) or os.curdir - -def removepy(names): - if not names: - return - for idx, name in enumerate(names): - basename, ext = os.path.splitext(name) - if ext == '.py': - names[idx] = basename - -def count(n, word): - if n == 1: - return "%d %s" % (n, word) - else: - return "%d %ss" % (n, word) - -def printlist(x, width=70, indent=4): - """Print the elements of iterable x to stdout. - - Optional arg width (default 70) is the maximum line length. - Optional arg indent (default 4) is the number of blanks with which to - begin each line. - """ - - from textwrap import fill - blanks = ' ' * indent - # Print the sorted list: 'x' may be a '--random' list or a set() - print(fill(' '.join(str(elt) for elt in sorted(x)), width, - initial_indent=blanks, subsequent_indent=blanks)) - - -def main_in_temp_cwd(): - """Run main() in a temporary working directory.""" - if sysconfig.is_python_build(): - try: - os.mkdir(TEMPDIR) - except FileExistsError: - pass - - # Define a writable temp dir that will be used as cwd while running - # the tests. The name of the dir includes the pid to allow parallel - # testing (see the -j option). - test_cwd = 'test_python_{}'.format(os.getpid()) - test_cwd = os.path.join(TEMPDIR, test_cwd) - - # Run the tests in a context manager that temporarily changes the CWD to a - # temporary and writable directory. If it's not possible to create or - # change the CWD, the original CWD will be used. The original CWD is - # available from support.SAVEDCWD. - with support.temp_cwd(test_cwd, quiet=True): - main() - - -if __name__ == '__main__': - # Remove regrtest.py's own directory from the module search path. Despite - # the elimination of implicit relative imports, this is still needed to - # ensure that submodules of the test package do not inappropriately appear - # as top-level modules even when people (or buildbots!) invoke regrtest.py - # directly instead of using the -m switch - mydir = os.path.abspath(os.path.normpath(os.path.dirname(sys.argv[0]))) - i = len(sys.path) - while i >= 0: - i -= 1 - if os.path.abspath(os.path.normpath(sys.path[i])) == mydir: - del sys.path[i] - - # findtestdir() gets the dirname out of __file__, so we have to make it - # absolute before changing the working directory. - # For example __file__ may be relative when running trace or profile. - # See issue #9323. - __file__ = os.path.abspath(__file__) - - # sanity check - assert __file__ == os.path.abspath(sys.argv[0]) - - main_in_temp_cwd() diff --git a/Lib/test/regrtest.py b/Lib/test/regrtest.py --- a/Lib/test/regrtest.py +++ b/Lib/test/regrtest.py @@ -6,119 +6,6 @@ Run this script with -h or --help for documentation. """ -USAGE = """\ -python -m test [options] [test_name1 [test_name2 ...]] -python path/to/Lib/test/regrtest.py [options] [test_name1 [test_name2 ...]] -""" - -DESCRIPTION = """\ -Run Python regression tests. - -If no arguments or options are provided, finds all files matching -the pattern "test_*" in the Lib/test subdirectory and runs -them in alphabetical order (but see -M and -u, below, for exceptions). - -For more rigorous testing, it is useful to use the following -command line: - -python -E -Wd -m test [options] [test_name1 ...] -""" - -EPILOG = """\ -Additional option details: - --r randomizes test execution order. You can use --randseed=int to provide a -int seed value for the randomizer; this is useful for reproducing troublesome -test orders. - --s On the first invocation of regrtest using -s, the first test file found -or the first test file given on the command line is run, and the name of -the next test is recorded in a file named pynexttest. If run from the -Python build directory, pynexttest is located in the 'build' subdirectory, -otherwise it is located in tempfile.gettempdir(). On subsequent runs, -the test in pynexttest is run, and the next test is written to pynexttest. -When the last test has been run, pynexttest is deleted. In this way it -is possible to single step through the test files. This is useful when -doing memory analysis on the Python interpreter, which process tends to -consume too many resources to run the full regression test non-stop. - --S is used to continue running tests after an aborted run. It will -maintain the order a standard run (ie, this assumes -r is not used). -This is useful after the tests have prematurely stopped for some external -reason and you want to start running from where you left off rather -than starting from the beginning. - --f reads the names of tests from the file given as f's argument, one -or more test names per line. Whitespace is ignored. Blank lines and -lines beginning with '#' are ignored. This is especially useful for -whittling down failures involving interactions among tests. - --L causes the leaks(1) command to be run just before exit if it exists. -leaks(1) is available on Mac OS X and presumably on some other -FreeBSD-derived systems. - --R runs each test several times and examines sys.gettotalrefcount() to -see if the test appears to be leaking references. The argument should -be of the form stab:run:fname where 'stab' is the number of times the -test is run to let gettotalrefcount settle down, 'run' is the number -of times further it is run and 'fname' is the name of the file the -reports are written to. These parameters all have defaults (5, 4 and -"reflog.txt" respectively), and the minimal invocation is '-R :'. - --M runs tests that require an exorbitant amount of memory. These tests -typically try to ascertain containers keep working when containing more than -2 billion objects, which only works on 64-bit systems. There are also some -tests that try to exhaust the address space of the process, which only makes -sense on 32-bit systems with at least 2Gb of memory. The passed-in memlimit, -which is a string in the form of '2.5Gb', determines howmuch memory the -tests will limit themselves to (but they may go slightly over.) The number -shouldn't be more memory than the machine has (including swap memory). You -should also keep in mind that swap memory is generally much, much slower -than RAM, and setting memlimit to all available RAM or higher will heavily -tax the machine. On the other hand, it is no use running these tests with a -limit of less than 2.5Gb, and many require more than 20Gb. Tests that expect -to use more than memlimit memory will be skipped. The big-memory tests -generally run very, very long. - --u is used to specify which special resource intensive tests to run, -such as those requiring large file support or network connectivity. -The argument is a comma-separated list of words indicating the -resources to test. Currently only the following are defined: - - all - Enable all special resources. - - none - Disable all special resources (this is the default). - - audio - Tests that use the audio device. (There are known - cases of broken audio drivers that can crash Python or - even the Linux kernel.) - - curses - Tests that use curses and will modify the terminal's - state and output modes. - - largefile - It is okay to run some test that may create huge - files. These tests can take a long time and may - consume >2GB of disk space temporarily. - - network - It is okay to run tests that use external network - resource, e.g. testing SSL support for sockets. - - decimal - Test the decimal module against a large suite that - verifies compliance with standards. - - cpu - Used for certain CPU-heavy tests. - - subprocess Run all tests for the subprocess module. - - urlfetch - It is okay to download files required on testing. - - gui - Run tests that require a running GUI. - -To enable all resources except one, use '-uall,-'. For -example, to run all the tests except for the gui tests, give the -option '-uall,-gui'. -""" - # We import importlib *ASAP* in order to test #15386 import importlib @@ -153,6 +40,8 @@ except ImportError: multiprocessing = None +from test.libregrtest import _parse_args + # Some times __path__ and __file__ are not absolute (e.g. while running from # Lib/) and, if we change the CWD to run the tests in a temporary dir, some @@ -198,9 +87,6 @@ from test import support -RESOURCE_NAMES = ('audio', 'curses', 'largefile', 'network', - 'decimal', 'cpu', 'subprocess', 'urlfetch', 'gui') - # When tests are run from the Python build directory, it is best practice # to keep the test files in a subfolder. This eases the cleanup of leftover # files using the "make distclean" command. @@ -210,220 +96,6 @@ TEMPDIR = tempfile.gettempdir() TEMPDIR = os.path.abspath(TEMPDIR) -class _ArgParser(argparse.ArgumentParser): - - def error(self, message): - super().error(message + "\nPass -h or --help for complete help.") - -def _create_parser(): - # Set prog to prevent the uninformative "__main__.py" from displaying in - # error messages when using "python -m test ...". - parser = _ArgParser(prog='regrtest.py', - usage=USAGE, - description=DESCRIPTION, - epilog=EPILOG, - add_help=False, - formatter_class=argparse.RawDescriptionHelpFormatter) - - # Arguments with this clause added to its help are described further in - # the epilog's "Additional option details" section. - more_details = ' See the section at bottom for more details.' - - group = parser.add_argument_group('General options') - # We add help explicitly to control what argument group it renders under. - group.add_argument('-h', '--help', action='help', - help='show this help message and exit') - group.add_argument('--timeout', metavar='TIMEOUT', type=float, - help='dump the traceback and exit if a test takes ' - 'more than TIMEOUT seconds; disabled if TIMEOUT ' - 'is negative or equals to zero') - group.add_argument('--wait', action='store_true', - help='wait for user input, e.g., allow a debugger ' - 'to be attached') - group.add_argument('--slaveargs', metavar='ARGS') - group.add_argument('-S', '--start', metavar='START', - help='the name of the test at which to start.' + - more_details) - - group = parser.add_argument_group('Verbosity') - group.add_argument('-v', '--verbose', action='count', - help='run tests in verbose mode with output to stdout') - group.add_argument('-w', '--verbose2', action='store_true', - help='re-run failed tests in verbose mode') - group.add_argument('-W', '--verbose3', action='store_true', - help='display test output on failure') - group.add_argument('-q', '--quiet', action='store_true', - help='no output unless one or more tests fail') - group.add_argument('-o', '--slow', action='store_true', dest='print_slow', - help='print the slowest 10 tests') - group.add_argument('--header', action='store_true', - help='print header with interpreter info') - - group = parser.add_argument_group('Selecting tests') - group.add_argument('-r', '--randomize', action='store_true', - help='randomize test execution order.' + more_details) - group.add_argument('--randseed', metavar='SEED', - dest='random_seed', type=int, - help='pass a random seed to reproduce a previous ' - 'random run') - group.add_argument('-f', '--fromfile', metavar='FILE', - help='read names of tests to run from a file.' + - more_details) - group.add_argument('-x', '--exclude', action='store_true', - help='arguments are tests to *exclude*') - group.add_argument('-s', '--single', action='store_true', - help='single step through a set of tests.' + - more_details) - group.add_argument('-m', '--match', metavar='PAT', - dest='match_tests', - help='match test cases and methods with glob pattern PAT') - group.add_argument('-G', '--failfast', action='store_true', - help='fail as soon as a test fails (only with -v or -W)') - group.add_argument('-u', '--use', metavar='RES1,RES2,...', - action='append', type=resources_list, - help='specify which special resource intensive tests ' - 'to run.' + more_details) - group.add_argument('-M', '--memlimit', metavar='LIMIT', - help='run very large memory-consuming tests.' + - more_details) - group.add_argument('--testdir', metavar='DIR', - type=relative_filename, - help='execute test files in the specified directory ' - '(instead of the Python stdlib test suite)') - - group = parser.add_argument_group('Special runs') - group.add_argument('-l', '--findleaks', action='store_true', - help='if GC is available detect tests that leak memory') - group.add_argument('-L', '--runleaks', action='store_true', - help='run the leaks(1) command just before exit.' + - more_details) - group.add_argument('-R', '--huntrleaks', metavar='RUNCOUNTS', - type=huntrleaks, - help='search for reference leaks (needs debug build, ' - 'very slow).' + more_details) - group.add_argument('-j', '--multiprocess', metavar='PROCESSES', - dest='use_mp', type=int, - help='run PROCESSES processes at once') - group.add_argument('-T', '--coverage', action='store_true', - dest='trace', - help='turn on code coverage tracing using the trace ' - 'module') - group.add_argument('-D', '--coverdir', metavar='DIR', - type=relative_filename, - help='directory where coverage files are put') - group.add_argument('-N', '--nocoverdir', - action='store_const', const=None, dest='coverdir', - help='put coverage files alongside modules') - group.add_argument('-t', '--threshold', metavar='THRESHOLD', - type=int, - help='call gc.set_threshold(THRESHOLD)') - group.add_argument('-n', '--nowindows', action='store_true', - help='suppress error message boxes on Windows') - group.add_argument('-F', '--forever', action='store_true', - help='run the specified tests in a loop, until an ' - 'error happens') - - parser.add_argument('args', nargs=argparse.REMAINDER, - help=argparse.SUPPRESS) - - return parser - -def relative_filename(string): - # CWD is replaced with a temporary dir before calling main(), so we - # join it with the saved CWD so it ends up where the user expects. - return os.path.join(support.SAVEDCWD, string) - -def huntrleaks(string): - args = string.split(':') - if len(args) not in (2, 3): - raise argparse.ArgumentTypeError( - 'needs 2 or 3 colon-separated arguments') - nwarmup = int(args[0]) if args[0] else 5 - ntracked = int(args[1]) if args[1] else 4 - fname = args[2] if len(args) > 2 and args[2] else 'reflog.txt' - return nwarmup, ntracked, fname - -def resources_list(string): - u = [x.lower() for x in string.split(',')] - for r in u: - if r == 'all' or r == 'none': - continue - if r[0] == '-': - r = r[1:] - if r not in RESOURCE_NAMES: - raise argparse.ArgumentTypeError('invalid resource: ' + r) - return u - -def _parse_args(args, **kwargs): - # Defaults - ns = argparse.Namespace(testdir=None, verbose=0, quiet=False, - exclude=False, single=False, randomize=False, fromfile=None, - findleaks=False, use_resources=None, trace=False, coverdir='coverage', - runleaks=False, huntrleaks=False, verbose2=False, print_slow=False, - random_seed=None, use_mp=None, verbose3=False, forever=False, - header=False, failfast=False, match_tests=None) - for k, v in kwargs.items(): - if not hasattr(ns, k): - raise TypeError('%r is an invalid keyword argument ' - 'for this function' % k) - setattr(ns, k, v) - if ns.use_resources is None: - ns.use_resources = [] - - parser = _create_parser() - parser.parse_args(args=args, namespace=ns) - - if ns.single and ns.fromfile: - parser.error("-s and -f don't go together!") - if ns.use_mp and ns.trace: - parser.error("-T and -j don't go together!") - if ns.use_mp and ns.findleaks: - parser.error("-l and -j don't go together!") - if ns.use_mp and ns.memlimit: - parser.error("-M and -j don't go together!") - if ns.failfast and not (ns.verbose or ns.verbose3): - parser.error("-G/--failfast needs either -v or -W") - - if ns.quiet: - ns.verbose = 0 - if ns.timeout is not None: - if hasattr(faulthandler, 'dump_traceback_later'): - if ns.timeout <= 0: - ns.timeout = None - else: - print("Warning: The timeout option requires " - "faulthandler.dump_traceback_later") - ns.timeout = None - if ns.use_mp is not None: - if ns.use_mp <= 0: - # Use all cores + extras for tests that like to sleep - ns.use_mp = 2 + (os.cpu_count() or 1) - if ns.use_mp == 1: - ns.use_mp = None - if ns.use: - for a in ns.use: - for r in a: - if r == 'all': - ns.use_resources[:] = RESOURCE_NAMES - continue - if r == 'none': - del ns.use_resources[:] - continue - remove = False - if r[0] == '-': - remove = True - r = r[1:] - if remove: - if r in ns.use_resources: - ns.use_resources.remove(r) - elif r not in ns.use_resources: - ns.use_resources.append(r) - if ns.random_seed is not None: - ns.randomize = True - - return ns - - def run_test_in_subprocess(testname, ns): """Run the given test in a subprocess with --slaveargs. diff --git a/Lib/test/test_regrtest.py b/Lib/test/test_regrtest.py --- a/Lib/test/test_regrtest.py +++ b/Lib/test/test_regrtest.py @@ -7,7 +7,7 @@ import getopt import os.path import unittest -from test import regrtest, support +from test import regrtest, support, libregrtest class ParseArgsTestCase(unittest.TestCase): @@ -148,7 +148,7 @@ self.assertEqual(ns.use_resources, ['gui', 'network']) ns = regrtest._parse_args([opt, 'gui,none,network']) self.assertEqual(ns.use_resources, ['network']) - expected = list(regrtest.RESOURCE_NAMES) + expected = list(libregrtest.RESOURCE_NAMES) expected.remove('gui') ns = regrtest._parse_args([opt, 'all,-gui']) self.assertEqual(ns.use_resources, expected) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 23 23:17:03 2015 From: python-checkins at python.org (victor.stinner) Date: Wed, 23 Sep 2015 21:17:03 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2325220=3A_Backed_o?= =?utf-8?q?ut_changeset_eaf9a99b6bb8?= Message-ID: <20150923211702.115182.85068@psf.io> https://hg.python.org/cpython/rev/c92d893fd3c8 changeset: 98223:c92d893fd3c8 user: Victor Stinner date: Wed Sep 23 23:16:47 2015 +0200 summary: Issue #25220: Backed out changeset eaf9a99b6bb8 files: Lib/test/libregrtest/__init__.py | 1 - Lib/test/libregrtest/cmdline.py | 340 ------------------- Lib/test/regrtest.py | 332 ++++++++++++++++++- Lib/test/test_regrtest.py | 4 +- 4 files changed, 332 insertions(+), 345 deletions(-) diff --git a/Lib/test/libregrtest/__init__.py b/Lib/test/libregrtest/__init__.py deleted file mode 100644 --- a/Lib/test/libregrtest/__init__.py +++ /dev/null @@ -1,1 +0,0 @@ -from test.libregrtest.cmdline import _parse_args, RESOURCE_NAMES diff --git a/Lib/test/libregrtest/cmdline.py b/Lib/test/libregrtest/cmdline.py deleted file mode 100644 --- a/Lib/test/libregrtest/cmdline.py +++ /dev/null @@ -1,340 +0,0 @@ -import argparse -import faulthandler -import os - -from test import support - -USAGE = """\ -python -m test [options] [test_name1 [test_name2 ...]] -python path/to/Lib/test/regrtest.py [options] [test_name1 [test_name2 ...]] -""" - -DESCRIPTION = """\ -Run Python regression tests. - -If no arguments or options are provided, finds all files matching -the pattern "test_*" in the Lib/test subdirectory and runs -them in alphabetical order (but see -M and -u, below, for exceptions). - -For more rigorous testing, it is useful to use the following -command line: - -python -E -Wd -m test [options] [test_name1 ...] -""" - -EPILOG = """\ -Additional option details: - --r randomizes test execution order. You can use --randseed=int to provide a -int seed value for the randomizer; this is useful for reproducing troublesome -test orders. - --s On the first invocation of regrtest using -s, the first test file found -or the first test file given on the command line is run, and the name of -the next test is recorded in a file named pynexttest. If run from the -Python build directory, pynexttest is located in the 'build' subdirectory, -otherwise it is located in tempfile.gettempdir(). On subsequent runs, -the test in pynexttest is run, and the next test is written to pynexttest. -When the last test has been run, pynexttest is deleted. In this way it -is possible to single step through the test files. This is useful when -doing memory analysis on the Python interpreter, which process tends to -consume too many resources to run the full regression test non-stop. - --S is used to continue running tests after an aborted run. It will -maintain the order a standard run (ie, this assumes -r is not used). -This is useful after the tests have prematurely stopped for some external -reason and you want to start running from where you left off rather -than starting from the beginning. - --f reads the names of tests from the file given as f's argument, one -or more test names per line. Whitespace is ignored. Blank lines and -lines beginning with '#' are ignored. This is especially useful for -whittling down failures involving interactions among tests. - --L causes the leaks(1) command to be run just before exit if it exists. -leaks(1) is available on Mac OS X and presumably on some other -FreeBSD-derived systems. - --R runs each test several times and examines sys.gettotalrefcount() to -see if the test appears to be leaking references. The argument should -be of the form stab:run:fname where 'stab' is the number of times the -test is run to let gettotalrefcount settle down, 'run' is the number -of times further it is run and 'fname' is the name of the file the -reports are written to. These parameters all have defaults (5, 4 and -"reflog.txt" respectively), and the minimal invocation is '-R :'. - --M runs tests that require an exorbitant amount of memory. These tests -typically try to ascertain containers keep working when containing more than -2 billion objects, which only works on 64-bit systems. There are also some -tests that try to exhaust the address space of the process, which only makes -sense on 32-bit systems with at least 2Gb of memory. The passed-in memlimit, -which is a string in the form of '2.5Gb', determines howmuch memory the -tests will limit themselves to (but they may go slightly over.) The number -shouldn't be more memory than the machine has (including swap memory). You -should also keep in mind that swap memory is generally much, much slower -than RAM, and setting memlimit to all available RAM or higher will heavily -tax the machine. On the other hand, it is no use running these tests with a -limit of less than 2.5Gb, and many require more than 20Gb. Tests that expect -to use more than memlimit memory will be skipped. The big-memory tests -generally run very, very long. - --u is used to specify which special resource intensive tests to run, -such as those requiring large file support or network connectivity. -The argument is a comma-separated list of words indicating the -resources to test. Currently only the following are defined: - - all - Enable all special resources. - - none - Disable all special resources (this is the default). - - audio - Tests that use the audio device. (There are known - cases of broken audio drivers that can crash Python or - even the Linux kernel.) - - curses - Tests that use curses and will modify the terminal's - state and output modes. - - largefile - It is okay to run some test that may create huge - files. These tests can take a long time and may - consume >2GB of disk space temporarily. - - network - It is okay to run tests that use external network - resource, e.g. testing SSL support for sockets. - - decimal - Test the decimal module against a large suite that - verifies compliance with standards. - - cpu - Used for certain CPU-heavy tests. - - subprocess Run all tests for the subprocess module. - - urlfetch - It is okay to download files required on testing. - - gui - Run tests that require a running GUI. - -To enable all resources except one, use '-uall,-'. For -example, to run all the tests except for the gui tests, give the -option '-uall,-gui'. -""" - - -RESOURCE_NAMES = ('audio', 'curses', 'largefile', 'network', - 'decimal', 'cpu', 'subprocess', 'urlfetch', 'gui') - -class _ArgParser(argparse.ArgumentParser): - - def error(self, message): - super().error(message + "\nPass -h or --help for complete help.") - - -def _create_parser(): - # Set prog to prevent the uninformative "__main__.py" from displaying in - # error messages when using "python -m test ...". - parser = _ArgParser(prog='regrtest.py', - usage=USAGE, - description=DESCRIPTION, - epilog=EPILOG, - add_help=False, - formatter_class=argparse.RawDescriptionHelpFormatter) - - # Arguments with this clause added to its help are described further in - # the epilog's "Additional option details" section. - more_details = ' See the section at bottom for more details.' - - group = parser.add_argument_group('General options') - # We add help explicitly to control what argument group it renders under. - group.add_argument('-h', '--help', action='help', - help='show this help message and exit') - group.add_argument('--timeout', metavar='TIMEOUT', type=float, - help='dump the traceback and exit if a test takes ' - 'more than TIMEOUT seconds; disabled if TIMEOUT ' - 'is negative or equals to zero') - group.add_argument('--wait', action='store_true', - help='wait for user input, e.g., allow a debugger ' - 'to be attached') - group.add_argument('--slaveargs', metavar='ARGS') - group.add_argument('-S', '--start', metavar='START', - help='the name of the test at which to start.' + - more_details) - - group = parser.add_argument_group('Verbosity') - group.add_argument('-v', '--verbose', action='count', - help='run tests in verbose mode with output to stdout') - group.add_argument('-w', '--verbose2', action='store_true', - help='re-run failed tests in verbose mode') - group.add_argument('-W', '--verbose3', action='store_true', - help='display test output on failure') - group.add_argument('-q', '--quiet', action='store_true', - help='no output unless one or more tests fail') - group.add_argument('-o', '--slow', action='store_true', dest='print_slow', - help='print the slowest 10 tests') - group.add_argument('--header', action='store_true', - help='print header with interpreter info') - - group = parser.add_argument_group('Selecting tests') - group.add_argument('-r', '--randomize', action='store_true', - help='randomize test execution order.' + more_details) - group.add_argument('--randseed', metavar='SEED', - dest='random_seed', type=int, - help='pass a random seed to reproduce a previous ' - 'random run') - group.add_argument('-f', '--fromfile', metavar='FILE', - help='read names of tests to run from a file.' + - more_details) - group.add_argument('-x', '--exclude', action='store_true', - help='arguments are tests to *exclude*') - group.add_argument('-s', '--single', action='store_true', - help='single step through a set of tests.' + - more_details) - group.add_argument('-m', '--match', metavar='PAT', - dest='match_tests', - help='match test cases and methods with glob pattern PAT') - group.add_argument('-G', '--failfast', action='store_true', - help='fail as soon as a test fails (only with -v or -W)') - group.add_argument('-u', '--use', metavar='RES1,RES2,...', - action='append', type=resources_list, - help='specify which special resource intensive tests ' - 'to run.' + more_details) - group.add_argument('-M', '--memlimit', metavar='LIMIT', - help='run very large memory-consuming tests.' + - more_details) - group.add_argument('--testdir', metavar='DIR', - type=relative_filename, - help='execute test files in the specified directory ' - '(instead of the Python stdlib test suite)') - - group = parser.add_argument_group('Special runs') - group.add_argument('-l', '--findleaks', action='store_true', - help='if GC is available detect tests that leak memory') - group.add_argument('-L', '--runleaks', action='store_true', - help='run the leaks(1) command just before exit.' + - more_details) - group.add_argument('-R', '--huntrleaks', metavar='RUNCOUNTS', - type=huntrleaks, - help='search for reference leaks (needs debug build, ' - 'very slow).' + more_details) - group.add_argument('-j', '--multiprocess', metavar='PROCESSES', - dest='use_mp', type=int, - help='run PROCESSES processes at once') - group.add_argument('-T', '--coverage', action='store_true', - dest='trace', - help='turn on code coverage tracing using the trace ' - 'module') - group.add_argument('-D', '--coverdir', metavar='DIR', - type=relative_filename, - help='directory where coverage files are put') - group.add_argument('-N', '--nocoverdir', - action='store_const', const=None, dest='coverdir', - help='put coverage files alongside modules') - group.add_argument('-t', '--threshold', metavar='THRESHOLD', - type=int, - help='call gc.set_threshold(THRESHOLD)') - group.add_argument('-n', '--nowindows', action='store_true', - help='suppress error message boxes on Windows') - group.add_argument('-F', '--forever', action='store_true', - help='run the specified tests in a loop, until an ' - 'error happens') - - parser.add_argument('args', nargs=argparse.REMAINDER, - help=argparse.SUPPRESS) - - return parser - - -def relative_filename(string): - # CWD is replaced with a temporary dir before calling main(), so we - # join it with the saved CWD so it ends up where the user expects. - return os.path.join(support.SAVEDCWD, string) - - -def huntrleaks(string): - args = string.split(':') - if len(args) not in (2, 3): - raise argparse.ArgumentTypeError( - 'needs 2 or 3 colon-separated arguments') - nwarmup = int(args[0]) if args[0] else 5 - ntracked = int(args[1]) if args[1] else 4 - fname = args[2] if len(args) > 2 and args[2] else 'reflog.txt' - return nwarmup, ntracked, fname - - -def resources_list(string): - u = [x.lower() for x in string.split(',')] - for r in u: - if r == 'all' or r == 'none': - continue - if r[0] == '-': - r = r[1:] - if r not in RESOURCE_NAMES: - raise argparse.ArgumentTypeError('invalid resource: ' + r) - return u - - -def _parse_args(args, **kwargs): - # Defaults - ns = argparse.Namespace(testdir=None, verbose=0, quiet=False, - exclude=False, single=False, randomize=False, fromfile=None, - findleaks=False, use_resources=None, trace=False, coverdir='coverage', - runleaks=False, huntrleaks=False, verbose2=False, print_slow=False, - random_seed=None, use_mp=None, verbose3=False, forever=False, - header=False, failfast=False, match_tests=None) - for k, v in kwargs.items(): - if not hasattr(ns, k): - raise TypeError('%r is an invalid keyword argument ' - 'for this function' % k) - setattr(ns, k, v) - if ns.use_resources is None: - ns.use_resources = [] - - parser = _create_parser() - parser.parse_args(args=args, namespace=ns) - - if ns.single and ns.fromfile: - parser.error("-s and -f don't go together!") - if ns.use_mp and ns.trace: - parser.error("-T and -j don't go together!") - if ns.use_mp and ns.findleaks: - parser.error("-l and -j don't go together!") - if ns.use_mp and ns.memlimit: - parser.error("-M and -j don't go together!") - if ns.failfast and not (ns.verbose or ns.verbose3): - parser.error("-G/--failfast needs either -v or -W") - - if ns.quiet: - ns.verbose = 0 - if ns.timeout is not None: - if hasattr(faulthandler, 'dump_traceback_later'): - if ns.timeout <= 0: - ns.timeout = None - else: - print("Warning: The timeout option requires " - "faulthandler.dump_traceback_later") - ns.timeout = None - if ns.use_mp is not None: - if ns.use_mp <= 0: - # Use all cores + extras for tests that like to sleep - ns.use_mp = 2 + (os.cpu_count() or 1) - if ns.use_mp == 1: - ns.use_mp = None - if ns.use: - for a in ns.use: - for r in a: - if r == 'all': - ns.use_resources[:] = RESOURCE_NAMES - continue - if r == 'none': - del ns.use_resources[:] - continue - remove = False - if r[0] == '-': - remove = True - r = r[1:] - if remove: - if r in ns.use_resources: - ns.use_resources.remove(r) - elif r not in ns.use_resources: - ns.use_resources.append(r) - if ns.random_seed is not None: - ns.randomize = True - - return ns diff --git a/Lib/test/regrtest.py b/Lib/test/regrtest.py --- a/Lib/test/regrtest.py +++ b/Lib/test/regrtest.py @@ -6,6 +6,119 @@ Run this script with -h or --help for documentation. """ +USAGE = """\ +python -m test [options] [test_name1 [test_name2 ...]] +python path/to/Lib/test/regrtest.py [options] [test_name1 [test_name2 ...]] +""" + +DESCRIPTION = """\ +Run Python regression tests. + +If no arguments or options are provided, finds all files matching +the pattern "test_*" in the Lib/test subdirectory and runs +them in alphabetical order (but see -M and -u, below, for exceptions). + +For more rigorous testing, it is useful to use the following +command line: + +python -E -Wd -m test [options] [test_name1 ...] +""" + +EPILOG = """\ +Additional option details: + +-r randomizes test execution order. You can use --randseed=int to provide a +int seed value for the randomizer; this is useful for reproducing troublesome +test orders. + +-s On the first invocation of regrtest using -s, the first test file found +or the first test file given on the command line is run, and the name of +the next test is recorded in a file named pynexttest. If run from the +Python build directory, pynexttest is located in the 'build' subdirectory, +otherwise it is located in tempfile.gettempdir(). On subsequent runs, +the test in pynexttest is run, and the next test is written to pynexttest. +When the last test has been run, pynexttest is deleted. In this way it +is possible to single step through the test files. This is useful when +doing memory analysis on the Python interpreter, which process tends to +consume too many resources to run the full regression test non-stop. + +-S is used to continue running tests after an aborted run. It will +maintain the order a standard run (ie, this assumes -r is not used). +This is useful after the tests have prematurely stopped for some external +reason and you want to start running from where you left off rather +than starting from the beginning. + +-f reads the names of tests from the file given as f's argument, one +or more test names per line. Whitespace is ignored. Blank lines and +lines beginning with '#' are ignored. This is especially useful for +whittling down failures involving interactions among tests. + +-L causes the leaks(1) command to be run just before exit if it exists. +leaks(1) is available on Mac OS X and presumably on some other +FreeBSD-derived systems. + +-R runs each test several times and examines sys.gettotalrefcount() to +see if the test appears to be leaking references. The argument should +be of the form stab:run:fname where 'stab' is the number of times the +test is run to let gettotalrefcount settle down, 'run' is the number +of times further it is run and 'fname' is the name of the file the +reports are written to. These parameters all have defaults (5, 4 and +"reflog.txt" respectively), and the minimal invocation is '-R :'. + +-M runs tests that require an exorbitant amount of memory. These tests +typically try to ascertain containers keep working when containing more than +2 billion objects, which only works on 64-bit systems. There are also some +tests that try to exhaust the address space of the process, which only makes +sense on 32-bit systems with at least 2Gb of memory. The passed-in memlimit, +which is a string in the form of '2.5Gb', determines howmuch memory the +tests will limit themselves to (but they may go slightly over.) The number +shouldn't be more memory than the machine has (including swap memory). You +should also keep in mind that swap memory is generally much, much slower +than RAM, and setting memlimit to all available RAM or higher will heavily +tax the machine. On the other hand, it is no use running these tests with a +limit of less than 2.5Gb, and many require more than 20Gb. Tests that expect +to use more than memlimit memory will be skipped. The big-memory tests +generally run very, very long. + +-u is used to specify which special resource intensive tests to run, +such as those requiring large file support or network connectivity. +The argument is a comma-separated list of words indicating the +resources to test. Currently only the following are defined: + + all - Enable all special resources. + + none - Disable all special resources (this is the default). + + audio - Tests that use the audio device. (There are known + cases of broken audio drivers that can crash Python or + even the Linux kernel.) + + curses - Tests that use curses and will modify the terminal's + state and output modes. + + largefile - It is okay to run some test that may create huge + files. These tests can take a long time and may + consume >2GB of disk space temporarily. + + network - It is okay to run tests that use external network + resource, e.g. testing SSL support for sockets. + + decimal - Test the decimal module against a large suite that + verifies compliance with standards. + + cpu - Used for certain CPU-heavy tests. + + subprocess Run all tests for the subprocess module. + + urlfetch - It is okay to download files required on testing. + + gui - Run tests that require a running GUI. + +To enable all resources except one, use '-uall,-'. For +example, to run all the tests except for the gui tests, give the +option '-uall,-gui'. +""" + # We import importlib *ASAP* in order to test #15386 import importlib @@ -40,8 +153,6 @@ except ImportError: multiprocessing = None -from test.libregrtest import _parse_args - # Some times __path__ and __file__ are not absolute (e.g. while running from # Lib/) and, if we change the CWD to run the tests in a temporary dir, some @@ -87,6 +198,9 @@ from test import support +RESOURCE_NAMES = ('audio', 'curses', 'largefile', 'network', + 'decimal', 'cpu', 'subprocess', 'urlfetch', 'gui') + # When tests are run from the Python build directory, it is best practice # to keep the test files in a subfolder. This eases the cleanup of leftover # files using the "make distclean" command. @@ -96,6 +210,220 @@ TEMPDIR = tempfile.gettempdir() TEMPDIR = os.path.abspath(TEMPDIR) +class _ArgParser(argparse.ArgumentParser): + + def error(self, message): + super().error(message + "\nPass -h or --help for complete help.") + +def _create_parser(): + # Set prog to prevent the uninformative "__main__.py" from displaying in + # error messages when using "python -m test ...". + parser = _ArgParser(prog='regrtest.py', + usage=USAGE, + description=DESCRIPTION, + epilog=EPILOG, + add_help=False, + formatter_class=argparse.RawDescriptionHelpFormatter) + + # Arguments with this clause added to its help are described further in + # the epilog's "Additional option details" section. + more_details = ' See the section at bottom for more details.' + + group = parser.add_argument_group('General options') + # We add help explicitly to control what argument group it renders under. + group.add_argument('-h', '--help', action='help', + help='show this help message and exit') + group.add_argument('--timeout', metavar='TIMEOUT', type=float, + help='dump the traceback and exit if a test takes ' + 'more than TIMEOUT seconds; disabled if TIMEOUT ' + 'is negative or equals to zero') + group.add_argument('--wait', action='store_true', + help='wait for user input, e.g., allow a debugger ' + 'to be attached') + group.add_argument('--slaveargs', metavar='ARGS') + group.add_argument('-S', '--start', metavar='START', + help='the name of the test at which to start.' + + more_details) + + group = parser.add_argument_group('Verbosity') + group.add_argument('-v', '--verbose', action='count', + help='run tests in verbose mode with output to stdout') + group.add_argument('-w', '--verbose2', action='store_true', + help='re-run failed tests in verbose mode') + group.add_argument('-W', '--verbose3', action='store_true', + help='display test output on failure') + group.add_argument('-q', '--quiet', action='store_true', + help='no output unless one or more tests fail') + group.add_argument('-o', '--slow', action='store_true', dest='print_slow', + help='print the slowest 10 tests') + group.add_argument('--header', action='store_true', + help='print header with interpreter info') + + group = parser.add_argument_group('Selecting tests') + group.add_argument('-r', '--randomize', action='store_true', + help='randomize test execution order.' + more_details) + group.add_argument('--randseed', metavar='SEED', + dest='random_seed', type=int, + help='pass a random seed to reproduce a previous ' + 'random run') + group.add_argument('-f', '--fromfile', metavar='FILE', + help='read names of tests to run from a file.' + + more_details) + group.add_argument('-x', '--exclude', action='store_true', + help='arguments are tests to *exclude*') + group.add_argument('-s', '--single', action='store_true', + help='single step through a set of tests.' + + more_details) + group.add_argument('-m', '--match', metavar='PAT', + dest='match_tests', + help='match test cases and methods with glob pattern PAT') + group.add_argument('-G', '--failfast', action='store_true', + help='fail as soon as a test fails (only with -v or -W)') + group.add_argument('-u', '--use', metavar='RES1,RES2,...', + action='append', type=resources_list, + help='specify which special resource intensive tests ' + 'to run.' + more_details) + group.add_argument('-M', '--memlimit', metavar='LIMIT', + help='run very large memory-consuming tests.' + + more_details) + group.add_argument('--testdir', metavar='DIR', + type=relative_filename, + help='execute test files in the specified directory ' + '(instead of the Python stdlib test suite)') + + group = parser.add_argument_group('Special runs') + group.add_argument('-l', '--findleaks', action='store_true', + help='if GC is available detect tests that leak memory') + group.add_argument('-L', '--runleaks', action='store_true', + help='run the leaks(1) command just before exit.' + + more_details) + group.add_argument('-R', '--huntrleaks', metavar='RUNCOUNTS', + type=huntrleaks, + help='search for reference leaks (needs debug build, ' + 'very slow).' + more_details) + group.add_argument('-j', '--multiprocess', metavar='PROCESSES', + dest='use_mp', type=int, + help='run PROCESSES processes at once') + group.add_argument('-T', '--coverage', action='store_true', + dest='trace', + help='turn on code coverage tracing using the trace ' + 'module') + group.add_argument('-D', '--coverdir', metavar='DIR', + type=relative_filename, + help='directory where coverage files are put') + group.add_argument('-N', '--nocoverdir', + action='store_const', const=None, dest='coverdir', + help='put coverage files alongside modules') + group.add_argument('-t', '--threshold', metavar='THRESHOLD', + type=int, + help='call gc.set_threshold(THRESHOLD)') + group.add_argument('-n', '--nowindows', action='store_true', + help='suppress error message boxes on Windows') + group.add_argument('-F', '--forever', action='store_true', + help='run the specified tests in a loop, until an ' + 'error happens') + + parser.add_argument('args', nargs=argparse.REMAINDER, + help=argparse.SUPPRESS) + + return parser + +def relative_filename(string): + # CWD is replaced with a temporary dir before calling main(), so we + # join it with the saved CWD so it ends up where the user expects. + return os.path.join(support.SAVEDCWD, string) + +def huntrleaks(string): + args = string.split(':') + if len(args) not in (2, 3): + raise argparse.ArgumentTypeError( + 'needs 2 or 3 colon-separated arguments') + nwarmup = int(args[0]) if args[0] else 5 + ntracked = int(args[1]) if args[1] else 4 + fname = args[2] if len(args) > 2 and args[2] else 'reflog.txt' + return nwarmup, ntracked, fname + +def resources_list(string): + u = [x.lower() for x in string.split(',')] + for r in u: + if r == 'all' or r == 'none': + continue + if r[0] == '-': + r = r[1:] + if r not in RESOURCE_NAMES: + raise argparse.ArgumentTypeError('invalid resource: ' + r) + return u + +def _parse_args(args, **kwargs): + # Defaults + ns = argparse.Namespace(testdir=None, verbose=0, quiet=False, + exclude=False, single=False, randomize=False, fromfile=None, + findleaks=False, use_resources=None, trace=False, coverdir='coverage', + runleaks=False, huntrleaks=False, verbose2=False, print_slow=False, + random_seed=None, use_mp=None, verbose3=False, forever=False, + header=False, failfast=False, match_tests=None) + for k, v in kwargs.items(): + if not hasattr(ns, k): + raise TypeError('%r is an invalid keyword argument ' + 'for this function' % k) + setattr(ns, k, v) + if ns.use_resources is None: + ns.use_resources = [] + + parser = _create_parser() + parser.parse_args(args=args, namespace=ns) + + if ns.single and ns.fromfile: + parser.error("-s and -f don't go together!") + if ns.use_mp and ns.trace: + parser.error("-T and -j don't go together!") + if ns.use_mp and ns.findleaks: + parser.error("-l and -j don't go together!") + if ns.use_mp and ns.memlimit: + parser.error("-M and -j don't go together!") + if ns.failfast and not (ns.verbose or ns.verbose3): + parser.error("-G/--failfast needs either -v or -W") + + if ns.quiet: + ns.verbose = 0 + if ns.timeout is not None: + if hasattr(faulthandler, 'dump_traceback_later'): + if ns.timeout <= 0: + ns.timeout = None + else: + print("Warning: The timeout option requires " + "faulthandler.dump_traceback_later") + ns.timeout = None + if ns.use_mp is not None: + if ns.use_mp <= 0: + # Use all cores + extras for tests that like to sleep + ns.use_mp = 2 + (os.cpu_count() or 1) + if ns.use_mp == 1: + ns.use_mp = None + if ns.use: + for a in ns.use: + for r in a: + if r == 'all': + ns.use_resources[:] = RESOURCE_NAMES + continue + if r == 'none': + del ns.use_resources[:] + continue + remove = False + if r[0] == '-': + remove = True + r = r[1:] + if remove: + if r in ns.use_resources: + ns.use_resources.remove(r) + elif r not in ns.use_resources: + ns.use_resources.append(r) + if ns.random_seed is not None: + ns.randomize = True + + return ns + + def run_test_in_subprocess(testname, ns): """Run the given test in a subprocess with --slaveargs. diff --git a/Lib/test/test_regrtest.py b/Lib/test/test_regrtest.py --- a/Lib/test/test_regrtest.py +++ b/Lib/test/test_regrtest.py @@ -7,7 +7,7 @@ import getopt import os.path import unittest -from test import regrtest, support, libregrtest +from test import regrtest, support class ParseArgsTestCase(unittest.TestCase): @@ -148,7 +148,7 @@ self.assertEqual(ns.use_resources, ['gui', 'network']) ns = regrtest._parse_args([opt, 'gui,none,network']) self.assertEqual(ns.use_resources, ['network']) - expected = list(libregrtest.RESOURCE_NAMES) + expected = list(regrtest.RESOURCE_NAMES) expected.remove('gui') ns = regrtest._parse_args([opt, 'all,-gui']) self.assertEqual(ns.use_resources, expected) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 24 02:02:47 2015 From: python-checkins at python.org (terry.reedy) Date: Thu, 24 Sep 2015 00:02:47 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgMjUyMjQ6?= =?utf-8?q?_Augment_Idle_doc_feature_list_and_no-subprocess_section?= Message-ID: <20150924000246.98378.86812@psf.io> https://hg.python.org/cpython/rev/5a60fdd9af8c changeset: 98224:5a60fdd9af8c branch: 2.7 parent: 98214:80e92eba23e0 user: Terry Jan Reedy date: Wed Sep 23 20:00:22 2015 -0400 summary: Issue 25224: Augment Idle doc feature list and no-subprocess section to finish making current README.txt obsolete. files: Doc/library/idle.rst | 25 +++++++++++++++++++++---- 1 files changed, 21 insertions(+), 4 deletions(-) diff --git a/Doc/library/idle.rst b/Doc/library/idle.rst --- a/Doc/library/idle.rst +++ b/Doc/library/idle.rst @@ -8,7 +8,7 @@ single: Python Editor single: Integrated Development Environment -.. moduleauthor:: Guido van Rossum +.. moduleauthor:: Guido van Rossum IDLE is the Python IDE built with the :mod:`tkinter` GUI toolkit. @@ -18,13 +18,19 @@ * cross-platform: works on Windows, Unix, and Mac OS X +* Python shell window (interactive interpreter) with colorizing + of code input, output, and error messages + * multi-window text editor with multiple undo, Python colorizing, - smart indent, call tips, and many other features + smart indent, call tips, auto completion, and other features -* Python shell window (a.k.a. interactive interpreter) +* search within any window, replace within editor windows, and search + through multiple files (grep) -* debugger (not complete, but you can set breakpoints, view and step) +* debugger with persistent breakpoints, stepping, and viewing + of global and local namespaces +* configuration, browsers, and other dialogs Menus ----- @@ -530,6 +536,17 @@ Running without a subprocess ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +By default, Idle executes user code in a separate subprocess via a socket, +which uses the internal loopback interface. This connection is not +externally visible and no data is sent to or received from the Internet. +If firewall software complains anyway, you can ignore it. + +If the attempt to make the socket connection fails, Idle will notify you. +Such failures are sometimes transient, but if persistent, the problem +may be either a firewall blocking the connecton or misconfiguration of +a particular system. Until the problem is fixed, one can run Idle with +the -n command line switch. + If IDLE is started with the -n command line switch it will run in a single process and will not create the subprocess which runs the RPC Python execution server. This can be useful if Python cannot create -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 24 02:02:47 2015 From: python-checkins at python.org (terry.reedy) Date: Thu, 24 Sep 2015 00:02:47 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgMjUyMjQ6?= =?utf-8?q?_Augment_Idle_doc_feature_list_and_no-subprocess_section?= Message-ID: <20150924000247.94127.12711@psf.io> https://hg.python.org/cpython/rev/b5c8a40d4240 changeset: 98228:b5c8a40d4240 branch: 2.7 parent: 98224:5a60fdd9af8c user: Terry Jan Reedy date: Wed Sep 23 20:02:25 2015 -0400 summary: Issue 25224: Augment Idle doc feature list and no-subprocess section to finish making current README.txt obsolete. files: Lib/idlelib/help.html | 20 +++++++++++++++++--- 1 files changed, 17 insertions(+), 3 deletions(-) diff --git a/Lib/idlelib/help.html b/Lib/idlelib/help.html --- a/Lib/idlelib/help.html +++ b/Lib/idlelib/help.html @@ -80,10 +80,15 @@
  • coded in 100% pure Python, using the tkinter GUI toolkit
  • cross-platform: works on Windows, Unix, and Mac OS X
  • +
  • Python shell window (interactive interpreter) with colorizing +of code input, output, and error messages
  • multi-window text editor with multiple undo, Python colorizing, -smart indent, call tips, and many other features
  • -
  • Python shell window (a.k.a. interactive interpreter)
  • -
  • debugger (not complete, but you can set breakpoints, view and step)
  • +smart indent, call tips, auto completion, and other features +
  • search within any window, replace within editor windows, and search +through multiple files (grep)
  • +
  • debugger with persistent breakpoints, stepping, and viewing +of global and local namespaces
  • +
  • configuration, browsers, and other dialogs

24.6.4.2. Running without a subprocess?

+

By default, Idle executes user code in a separate subprocess via a socket, +which uses the internal loopback interface. This connection is not +externally visible and no data is sent to or received from the Internet. +If firewall software complains anyway, you can ignore it.

+

If the attempt to make the socket connection fails, Idle will notify you. +Such failures are sometimes transient, but if persistent, the problem +may be either a firewall blocking the connecton or misconfiguration of +a particular system. Until the problem is fixed, one can run Idle with +the -n command line switch.

If IDLE is started with the -n command line switch it will run in a single process and will not create the subprocess which runs the RPC Python execution server. This can be useful if Python cannot create -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 24 02:02:47 2015 From: python-checkins at python.org (terry.reedy) Date: Thu, 24 Sep 2015 00:02:47 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Merge_with_3=2E4?= Message-ID: <20150924000247.94117.28344@psf.io> https://hg.python.org/cpython/rev/82822f4556c9 changeset: 98226:82822f4556c9 branch: 3.5 parent: 98216:5d5fca739abf parent: 98225:0f20f3fe7ab4 user: Terry Jan Reedy date: Wed Sep 23 20:00:55 2015 -0400 summary: Merge with 3.4 files: Doc/library/idle.rst | 25 +++++++++++++++++++++---- Lib/idlelib/help.html | 20 +++++++++++++++++--- 2 files changed, 38 insertions(+), 7 deletions(-) diff --git a/Doc/library/idle.rst b/Doc/library/idle.rst --- a/Doc/library/idle.rst +++ b/Doc/library/idle.rst @@ -8,7 +8,7 @@ single: Python Editor single: Integrated Development Environment -.. moduleauthor:: Guido van Rossum +.. moduleauthor:: Guido van Rossum IDLE is the Python IDE built with the :mod:`tkinter` GUI toolkit. @@ -18,13 +18,19 @@ * cross-platform: works on Windows, Unix, and Mac OS X +* Python shell window (interactive interpreter) with colorizing + of code input, output, and error messages + * multi-window text editor with multiple undo, Python colorizing, - smart indent, call tips, and many other features + smart indent, call tips, auto completion, and other features -* Python shell window (a.k.a. interactive interpreter) +* search within any window, replace within editor windows, and search + through multiple files (grep) -* debugger (not complete, but you can set breakpoints, view and step) +* debugger with persistent breakpoints, stepping, and viewing + of global and local namespaces +* configuration, browsers, and other dialogs Menus ----- @@ -530,6 +536,17 @@ Running without a subprocess ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +By default, Idle executes user code in a separate subprocess via a socket, +which uses the internal loopback interface. This connection is not +externally visible and no data is sent to or received from the Internet. +If firewall software complains anyway, you can ignore it. + +If the attempt to make the socket connection fails, Idle will notify you. +Such failures are sometimes transient, but if persistent, the problem +may be either a firewall blocking the connecton or misconfiguration of +a particular system. Until the problem is fixed, one can run Idle with +the -n command line switch. + If IDLE is started with the -n command line switch it will run in a single process and will not create the subprocess which runs the RPC Python execution server. This can be useful if Python cannot create diff --git a/Lib/idlelib/help.html b/Lib/idlelib/help.html --- a/Lib/idlelib/help.html +++ b/Lib/idlelib/help.html @@ -80,10 +80,15 @@

  • coded in 100% pure Python, using the tkinter GUI toolkit
  • cross-platform: works on Windows, Unix, and Mac OS X
  • +
  • Python shell window (interactive interpreter) with colorizing +of code input, output, and error messages
  • multi-window text editor with multiple undo, Python colorizing, -smart indent, call tips, and many other features
  • -
  • Python shell window (a.k.a. interactive interpreter)
  • -
  • debugger (not complete, but you can set breakpoints, view and step)
  • +smart indent, call tips, auto completion, and other features +
  • search within any window, replace within editor windows, and search +through multiple files (grep)
  • +
  • debugger with persistent breakpoints, stepping, and viewing +of global and local namespaces
  • +
  • configuration, browsers, and other dialogs

25.5.4.2. Running without a subprocess?

+

By default, Idle executes user code in a separate subprocess via a socket, +which uses the internal loopback interface. This connection is not +externally visible and no data is sent to or received from the Internet. +If firewall software complains anyway, you can ignore it.

+

If the attempt to make the socket connection fails, Idle will notify you. +Such failures are sometimes transient, but if persistent, the problem +may be either a firewall blocking the connecton or misconfiguration of +a particular system. Until the problem is fixed, one can run Idle with +the -n command line switch.

If IDLE is started with the -n command line switch it will run in a single process and will not create the subprocess which runs the RPC Python execution server. This can be useful if Python cannot create -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 24 02:02:47 2015 From: python-checkins at python.org (terry.reedy) Date: Thu, 24 Sep 2015 00:02:47 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgMjUyMjQ6?= =?utf-8?q?_Augment_Idle_doc_feature_list_and_no-subprocess_section?= Message-ID: <20150924000246.9957.70878@psf.io> https://hg.python.org/cpython/rev/0f20f3fe7ab4 changeset: 98225:0f20f3fe7ab4 branch: 3.4 parent: 98215:97f3d7749d3f user: Terry Jan Reedy date: Wed Sep 23 20:00:33 2015 -0400 summary: Issue 25224: Augment Idle doc feature list and no-subprocess section to finish making current README.txt obsolete. files: Doc/library/idle.rst | 25 +++++++++++++++++++++---- Lib/idlelib/help.html | 20 +++++++++++++++++--- 2 files changed, 38 insertions(+), 7 deletions(-) diff --git a/Doc/library/idle.rst b/Doc/library/idle.rst --- a/Doc/library/idle.rst +++ b/Doc/library/idle.rst @@ -8,7 +8,7 @@ single: Python Editor single: Integrated Development Environment -.. moduleauthor:: Guido van Rossum +.. moduleauthor:: Guido van Rossum IDLE is the Python IDE built with the :mod:`tkinter` GUI toolkit. @@ -18,13 +18,19 @@ * cross-platform: works on Windows, Unix, and Mac OS X +* Python shell window (interactive interpreter) with colorizing + of code input, output, and error messages + * multi-window text editor with multiple undo, Python colorizing, - smart indent, call tips, and many other features + smart indent, call tips, auto completion, and other features -* Python shell window (a.k.a. interactive interpreter) +* search within any window, replace within editor windows, and search + through multiple files (grep) -* debugger (not complete, but you can set breakpoints, view and step) +* debugger with persistent breakpoints, stepping, and viewing + of global and local namespaces +* configuration, browsers, and other dialogs Menus ----- @@ -530,6 +536,17 @@ Running without a subprocess ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +By default, Idle executes user code in a separate subprocess via a socket, +which uses the internal loopback interface. This connection is not +externally visible and no data is sent to or received from the Internet. +If firewall software complains anyway, you can ignore it. + +If the attempt to make the socket connection fails, Idle will notify you. +Such failures are sometimes transient, but if persistent, the problem +may be either a firewall blocking the connecton or misconfiguration of +a particular system. Until the problem is fixed, one can run Idle with +the -n command line switch. + If IDLE is started with the -n command line switch it will run in a single process and will not create the subprocess which runs the RPC Python execution server. This can be useful if Python cannot create diff --git a/Lib/idlelib/help.html b/Lib/idlelib/help.html --- a/Lib/idlelib/help.html +++ b/Lib/idlelib/help.html @@ -80,10 +80,15 @@

  • coded in 100% pure Python, using the tkinter GUI toolkit
  • cross-platform: works on Windows, Unix, and Mac OS X
  • +
  • Python shell window (interactive interpreter) with colorizing +of code input, output, and error messages
  • multi-window text editor with multiple undo, Python colorizing, -smart indent, call tips, and many other features
  • -
  • Python shell window (a.k.a. interactive interpreter)
  • -
  • debugger (not complete, but you can set breakpoints, view and step)
  • +smart indent, call tips, auto completion, and other features +
  • search within any window, replace within editor windows, and search +through multiple files (grep)
  • +
  • debugger with persistent breakpoints, stepping, and viewing +of global and local namespaces
  • +
  • configuration, browsers, and other dialogs

25.5.4.2. Running without a subprocess?

+

By default, Idle executes user code in a separate subprocess via a socket, +which uses the internal loopback interface. This connection is not +externally visible and no data is sent to or received from the Internet. +If firewall software complains anyway, you can ignore it.

+

If the attempt to make the socket connection fails, Idle will notify you. +Such failures are sometimes transient, but if persistent, the problem +may be either a firewall blocking the connecton or misconfiguration of +a particular system. Until the problem is fixed, one can run Idle with +the -n command line switch.

If IDLE is started with the -n command line switch it will run in a single process and will not create the subprocess which runs the RPC Python execution server. This can be useful if Python cannot create -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 24 02:02:47 2015 From: python-checkins at python.org (terry.reedy) Date: Thu, 24 Sep 2015 00:02:47 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge_with_3=2E5?= Message-ID: <20150924000247.115438.13078@psf.io> https://hg.python.org/cpython/rev/e426969e7e1d changeset: 98227:e426969e7e1d parent: 98223:c92d893fd3c8 parent: 98226:82822f4556c9 user: Terry Jan Reedy date: Wed Sep 23 20:01:09 2015 -0400 summary: Merge with 3.5 files: Doc/library/idle.rst | 25 +++++++++++++++++++++---- Lib/idlelib/help.html | 20 +++++++++++++++++--- 2 files changed, 38 insertions(+), 7 deletions(-) diff --git a/Doc/library/idle.rst b/Doc/library/idle.rst --- a/Doc/library/idle.rst +++ b/Doc/library/idle.rst @@ -8,7 +8,7 @@ single: Python Editor single: Integrated Development Environment -.. moduleauthor:: Guido van Rossum +.. moduleauthor:: Guido van Rossum IDLE is the Python IDE built with the :mod:`tkinter` GUI toolkit. @@ -18,13 +18,19 @@ * cross-platform: works on Windows, Unix, and Mac OS X +* Python shell window (interactive interpreter) with colorizing + of code input, output, and error messages + * multi-window text editor with multiple undo, Python colorizing, - smart indent, call tips, and many other features + smart indent, call tips, auto completion, and other features -* Python shell window (a.k.a. interactive interpreter) +* search within any window, replace within editor windows, and search + through multiple files (grep) -* debugger (not complete, but you can set breakpoints, view and step) +* debugger with persistent breakpoints, stepping, and viewing + of global and local namespaces +* configuration, browsers, and other dialogs Menus ----- @@ -530,6 +536,17 @@ Running without a subprocess ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +By default, Idle executes user code in a separate subprocess via a socket, +which uses the internal loopback interface. This connection is not +externally visible and no data is sent to or received from the Internet. +If firewall software complains anyway, you can ignore it. + +If the attempt to make the socket connection fails, Idle will notify you. +Such failures are sometimes transient, but if persistent, the problem +may be either a firewall blocking the connecton or misconfiguration of +a particular system. Until the problem is fixed, one can run Idle with +the -n command line switch. + If IDLE is started with the -n command line switch it will run in a single process and will not create the subprocess which runs the RPC Python execution server. This can be useful if Python cannot create diff --git a/Lib/idlelib/help.html b/Lib/idlelib/help.html --- a/Lib/idlelib/help.html +++ b/Lib/idlelib/help.html @@ -80,10 +80,15 @@

  • coded in 100% pure Python, using the tkinter GUI toolkit
  • cross-platform: works on Windows, Unix, and Mac OS X
  • +
  • Python shell window (interactive interpreter) with colorizing +of code input, output, and error messages
  • multi-window text editor with multiple undo, Python colorizing, -smart indent, call tips, and many other features
  • -
  • Python shell window (a.k.a. interactive interpreter)
  • -
  • debugger (not complete, but you can set breakpoints, view and step)
  • +smart indent, call tips, auto completion, and other features +
  • search within any window, replace within editor windows, and search +through multiple files (grep)
  • +
  • debugger with persistent breakpoints, stepping, and viewing +of global and local namespaces
  • +
  • configuration, browsers, and other dialogs

25.5.4.2. Running without a subprocess?

+

By default, Idle executes user code in a separate subprocess via a socket, +which uses the internal loopback interface. This connection is not +externally visible and no data is sent to or received from the Internet. +If firewall software complains anyway, you can ignore it.

+

If the attempt to make the socket connection fails, Idle will notify you. +Such failures are sometimes transient, but if persistent, the problem +may be either a firewall blocking the connecton or misconfiguration of +a particular system. Until the problem is fixed, one can run Idle with +the -n command line switch.

If IDLE is started with the -n command line switch it will run in a single process and will not create the subprocess which runs the RPC Python execution server. This can be useful if Python cannot create -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 24 02:38:57 2015 From: python-checkins at python.org (martin.panter) Date: Thu, 24 Sep 2015 00:38:57 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzI1MjEx?= =?utf-8?q?=3A_Fix_error_message_code_in_test=5Flong=3B_patch_from_s-wakab?= =?utf-8?q?a?= Message-ID: <20150924003857.81621.48671@psf.io> https://hg.python.org/cpython/rev/e2f1f69d0618 changeset: 98229:e2f1f69d0618 branch: 2.7 user: Martin Panter date: Thu Sep 24 00:19:42 2015 +0000 summary: Issue #25211: Fix error message code in test_long; patch from s-wakaba files: Lib/test/test_long.py | 38 +++++++++++++++--------------- 1 files changed, 19 insertions(+), 19 deletions(-) diff --git a/Lib/test/test_long.py b/Lib/test/test_long.py --- a/Lib/test/test_long.py +++ b/Lib/test/test_long.py @@ -216,43 +216,43 @@ for n in xrange(2*SHIFT): p2 = 2L ** n eq(x << n >> n, x, - Frm("x << n >> n != x for x=%r, n=%r", (x, n))) + Frm("x << n >> n != x for x=%r, n=%r", x, n)) eq(x // p2, x >> n, - Frm("x // p2 != x >> n for x=%r n=%r p2=%r", (x, n, p2))) + Frm("x // p2 != x >> n for x=%r n=%r p2=%r", x, n, p2)) eq(x * p2, x << n, - Frm("x * p2 != x << n for x=%r n=%r p2=%r", (x, n, p2))) + Frm("x * p2 != x << n for x=%r n=%r p2=%r", x, n, p2)) eq(x & -p2, x >> n << n, - Frm("not x & -p2 == x >> n << n for x=%r n=%r p2=%r", (x, n, p2))) + Frm("not x & -p2 == x >> n << n for x=%r n=%r p2=%r", x, n, p2)) eq(x & -p2, x & ~(p2 - 1), - Frm("not x & -p2 == x & ~(p2 - 1) for x=%r n=%r p2=%r", (x, n, p2))) + Frm("not x & -p2 == x & ~(p2 - 1) for x=%r n=%r p2=%r", x, n, p2)) def check_bitop_identities_2(self, x, y): eq = self.assertEqual - eq(x & y, y & x, Frm("x & y != y & x for x=%r, y=%r", (x, y))) - eq(x | y, y | x, Frm("x | y != y | x for x=%r, y=%r", (x, y))) - eq(x ^ y, y ^ x, Frm("x ^ y != y ^ x for x=%r, y=%r", (x, y))) - eq(x ^ y ^ x, y, Frm("x ^ y ^ x != y for x=%r, y=%r", (x, y))) - eq(x & y, ~(~x | ~y), Frm("x & y != ~(~x | ~y) for x=%r, y=%r", (x, y))) - eq(x | y, ~(~x & ~y), Frm("x | y != ~(~x & ~y) for x=%r, y=%r", (x, y))) + eq(x & y, y & x, Frm("x & y != y & x for x=%r, y=%r", x, y)) + eq(x | y, y | x, Frm("x | y != y | x for x=%r, y=%r", x, y)) + eq(x ^ y, y ^ x, Frm("x ^ y != y ^ x for x=%r, y=%r", x, y)) + eq(x ^ y ^ x, y, Frm("x ^ y ^ x != y for x=%r, y=%r", x, y)) + eq(x & y, ~(~x | ~y), Frm("x & y != ~(~x | ~y) for x=%r, y=%r", x, y)) + eq(x | y, ~(~x & ~y), Frm("x | y != ~(~x & ~y) for x=%r, y=%r", x, y)) eq(x ^ y, (x | y) & ~(x & y), - Frm("x ^ y != (x | y) & ~(x & y) for x=%r, y=%r", (x, y))) + Frm("x ^ y != (x | y) & ~(x & y) for x=%r, y=%r", x, y)) eq(x ^ y, (x & ~y) | (~x & y), - Frm("x ^ y == (x & ~y) | (~x & y) for x=%r, y=%r", (x, y))) + Frm("x ^ y == (x & ~y) | (~x & y) for x=%r, y=%r", x, y)) eq(x ^ y, (x | y) & (~x | ~y), - Frm("x ^ y == (x | y) & (~x | ~y) for x=%r, y=%r", (x, y))) + Frm("x ^ y == (x | y) & (~x | ~y) for x=%r, y=%r", x, y)) def check_bitop_identities_3(self, x, y, z): eq = self.assertEqual eq((x & y) & z, x & (y & z), - Frm("(x & y) & z != x & (y & z) for x=%r, y=%r, z=%r", (x, y, z))) + Frm("(x & y) & z != x & (y & z) for x=%r, y=%r, z=%r", x, y, z)) eq((x | y) | z, x | (y | z), - Frm("(x | y) | z != x | (y | z) for x=%r, y=%r, z=%r", (x, y, z))) + Frm("(x | y) | z != x | (y | z) for x=%r, y=%r, z=%r", x, y, z)) eq((x ^ y) ^ z, x ^ (y ^ z), - Frm("(x ^ y) ^ z != x ^ (y ^ z) for x=%r, y=%r, z=%r", (x, y, z))) + Frm("(x ^ y) ^ z != x ^ (y ^ z) for x=%r, y=%r, z=%r", x, y, z)) eq(x & (y | z), (x & y) | (x & z), - Frm("x & (y | z) != (x & y) | (x & z) for x=%r, y=%r, z=%r", (x, y, z))) + Frm("x & (y | z) != (x & y) | (x & z) for x=%r, y=%r, z=%r", x, y, z)) eq(x | (y & z), (x | y) & (x | z), - Frm("x | (y & z) != (x | y) & (x | z) for x=%r, y=%r, z=%r", (x, y, z))) + Frm("x | (y & z) != (x | y) & (x | z) for x=%r, y=%r, z=%r", x, y, z)) def test_bitop_identities(self): for x in special: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 24 04:15:50 2015 From: python-checkins at python.org (raymond.hettinger) Date: Thu, 24 Sep 2015 02:15:50 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Replace_an_unpredictable_b?= =?utf-8?q?ranch_with_a_simple_addition=2E?= Message-ID: <20150924021549.16571.631@psf.io> https://hg.python.org/cpython/rev/2b71c9db17a5 changeset: 98230:2b71c9db17a5 parent: 98227:e426969e7e1d user: Raymond Hettinger date: Wed Sep 23 19:15:44 2015 -0700 summary: Replace an unpredictable branch with a simple addition. files: Modules/_collectionsmodule.c | 5 ++--- 1 files changed, 2 insertions(+), 3 deletions(-) diff --git a/Modules/_collectionsmodule.c b/Modules/_collectionsmodule.c --- a/Modules/_collectionsmodule.c +++ b/Modules/_collectionsmodule.c @@ -852,10 +852,9 @@ CHECK_NOT_END(b); item = b->data[index]; cmp = PyObject_RichCompareBool(item, v, Py_EQ); - if (cmp > 0) - count++; - else if (cmp < 0) + if (cmp < 0) return NULL; + count += cmp; if (start_state != deque->state) { PyErr_SetString(PyExc_RuntimeError, -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 24 07:40:26 2015 From: python-checkins at python.org (terry.reedy) Date: Thu, 24 Sep 2015 05:40:26 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzIyODIw?= =?utf-8?q?=3A_Explain_need_for_*print*_when_running_file_from_Idle_editor?= =?utf-8?q?=2E?= Message-ID: <20150924054026.9949.62889@psf.io> https://hg.python.org/cpython/rev/824cb25448ab changeset: 98232:824cb25448ab branch: 3.4 parent: 98225:0f20f3fe7ab4 user: Terry Jan Reedy date: Thu Sep 24 01:39:30 2015 -0400 summary: Issue #22820: Explain need for *print* when running file from Idle editor. files: Doc/library/idle.rst | 7 ++++++- 1 files changed, 6 insertions(+), 1 deletions(-) diff --git a/Doc/library/idle.rst b/Doc/library/idle.rst --- a/Doc/library/idle.rst +++ b/Doc/library/idle.rst @@ -206,7 +206,12 @@ Run Module Do Check Module (above). If no error, restart the shell to clean the - environment, then execute the module. + environment, then execute the module. Output is displayed in the Shell + window. Note that output requires use of ``print`` or ``write``. + When execution is complete, the Shell retains focus and displays a prompt. + At this point, one may interactively explore the result of execution. + This is similar to executing a file with ``python -i file`` at a command + line. Shell menu (Shell window only) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 24 07:40:27 2015 From: python-checkins at python.org (terry.reedy) Date: Thu, 24 Sep 2015 05:40:27 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzIyODIw?= =?utf-8?q?=3A_Explain_need_for_*print*_when_running_file_from_Idle_editor?= =?utf-8?q?=2E?= Message-ID: <20150924054026.3668.7650@psf.io> https://hg.python.org/cpython/rev/e7bf0727f7df changeset: 98231:e7bf0727f7df branch: 2.7 parent: 98229:e2f1f69d0618 user: Terry Jan Reedy date: Thu Sep 24 01:39:25 2015 -0400 summary: Issue #22820: Explain need for *print* when running file from Idle editor. files: Doc/library/idle.rst | 7 ++++++- Lib/idlelib/help.html | 9 +++++++-- 2 files changed, 13 insertions(+), 3 deletions(-) diff --git a/Doc/library/idle.rst b/Doc/library/idle.rst --- a/Doc/library/idle.rst +++ b/Doc/library/idle.rst @@ -206,7 +206,12 @@ Run Module Do Check Module (above). If no error, restart the shell to clean the - environment, then execute the module. + environment, then execute the module. Output is displayed in the Shell + window. Note that output requires use of ``print`` or ``write``. + When execution is complete, the Shell retains focus and displays a prompt. + At this point, one may interactively explore the result of execution. + This is similar to executing a file with ``python -i file`` at a command + line. Shell menu (Shell window only) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ diff --git a/Lib/idlelib/help.html b/Lib/idlelib/help.html --- a/Lib/idlelib/help.html +++ b/Lib/idlelib/help.html @@ -223,7 +223,12 @@ Editor window.

Run Module
Do Check Module (above). If no error, restart the shell to clean the -environment, then execute the module.
+environment, then execute the module. Output is displayed in the Shell +window. Note that output requires use of print or write. +When execution is complete, the Shell retains focus and displays a prompt. +At this point, one may interactively explore the result of execution. +This is similar to executing a file with python -i file at a command +line.
@@ -676,7 +681,7 @@ The Python Software Foundation is a non-profit corporation. Please donate.
- Last updated on Sep 23, 2015. + Last updated on Sep 24, 2015. Found a bug?
Created using Sphinx 1.2.3. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 24 07:40:28 2015 From: python-checkins at python.org (terry.reedy) Date: Thu, 24 Sep 2015 05:40:28 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge_with_3=2E5?= Message-ID: <20150924054027.94123.26035@psf.io> https://hg.python.org/cpython/rev/d20deb8601c1 changeset: 98234:d20deb8601c1 parent: 98230:2b71c9db17a5 parent: 98233:264ca0caa3c2 user: Terry Jan Reedy date: Thu Sep 24 01:40:07 2015 -0400 summary: Merge with 3.5 files: Doc/library/idle.rst | 7 ++++++- 1 files changed, 6 insertions(+), 1 deletions(-) diff --git a/Doc/library/idle.rst b/Doc/library/idle.rst --- a/Doc/library/idle.rst +++ b/Doc/library/idle.rst @@ -206,7 +206,12 @@ Run Module Do Check Module (above). If no error, restart the shell to clean the - environment, then execute the module. + environment, then execute the module. Output is displayed in the Shell + window. Note that output requires use of ``print`` or ``write``. + When execution is complete, the Shell retains focus and displays a prompt. + At this point, one may interactively explore the result of execution. + This is similar to executing a file with ``python -i file`` at a command + line. Shell menu (Shell window only) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 24 07:40:29 2015 From: python-checkins at python.org (terry.reedy) Date: Thu, 24 Sep 2015 05:40:29 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Merge_with_3=2E4?= Message-ID: <20150924054027.115182.65836@psf.io> https://hg.python.org/cpython/rev/264ca0caa3c2 changeset: 98233:264ca0caa3c2 branch: 3.5 parent: 98226:82822f4556c9 parent: 98232:824cb25448ab user: Terry Jan Reedy date: Thu Sep 24 01:39:48 2015 -0400 summary: Merge with 3.4 files: Doc/library/idle.rst | 7 ++++++- 1 files changed, 6 insertions(+), 1 deletions(-) diff --git a/Doc/library/idle.rst b/Doc/library/idle.rst --- a/Doc/library/idle.rst +++ b/Doc/library/idle.rst @@ -206,7 +206,12 @@ Run Module Do Check Module (above). If no error, restart the shell to clean the - environment, then execute the module. + environment, then execute the module. Output is displayed in the Shell + window. Note that output requires use of ``print`` or ``write``. + When execution is complete, the Shell retains focus and displays a prompt. + At this point, one may interactively explore the result of execution. + This is similar to executing a file with ``python -i file`` at a command + line. Shell menu (Shell window only) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 24 09:05:37 2015 From: python-checkins at python.org (victor.stinner) Date: Thu, 24 Sep 2015 07:05:37 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzI0ODk0?= =?utf-8?q?=3A_Document_the_codec_iso8859=5F11?= Message-ID: <20150924070536.11700.2780@psf.io> https://hg.python.org/cpython/rev/aa9e0dcbbcc2 changeset: 98235:aa9e0dcbbcc2 branch: 3.4 parent: 98232:824cb25448ab user: Victor Stinner date: Thu Sep 24 09:04:05 2015 +0200 summary: Issue #24894: Document the codec iso8859_11 Patch written by Prashant Tyagi. files: Doc/library/codecs.rst | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) diff --git a/Doc/library/codecs.rst b/Doc/library/codecs.rst --- a/Doc/library/codecs.rst +++ b/Doc/library/codecs.rst @@ -1130,6 +1130,8 @@ +-----------------+--------------------------------+--------------------------------+ | iso8859_10 | iso-8859-10, latin6, L6 | Nordic languages | +-----------------+--------------------------------+--------------------------------+ +| iso8859_11 | iso-8859-11, thai | Thai languages | ++-----------------+--------------------------------+--------------------------------+ | iso8859_13 | iso-8859-13, latin7, L7 | Baltic languages | +-----------------+--------------------------------+--------------------------------+ | iso8859_14 | iso-8859-14, latin8, L8 | Celtic languages | -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 24 09:05:37 2015 From: python-checkins at python.org (victor.stinner) Date: Thu, 24 Sep 2015 07:05:37 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Merge_3=2E4_=28codecs=2C_issue_=2324894=29?= Message-ID: <20150924070536.11702.64993@psf.io> https://hg.python.org/cpython/rev/84a918335fe5 changeset: 98236:84a918335fe5 branch: 3.5 parent: 98233:264ca0caa3c2 parent: 98235:aa9e0dcbbcc2 user: Victor Stinner date: Thu Sep 24 09:04:26 2015 +0200 summary: Merge 3.4 (codecs, issue #24894) files: Doc/library/codecs.rst | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) diff --git a/Doc/library/codecs.rst b/Doc/library/codecs.rst --- a/Doc/library/codecs.rst +++ b/Doc/library/codecs.rst @@ -1148,6 +1148,8 @@ +-----------------+--------------------------------+--------------------------------+ | iso8859_10 | iso-8859-10, latin6, L6 | Nordic languages | +-----------------+--------------------------------+--------------------------------+ +| iso8859_11 | iso-8859-11, thai | Thai languages | ++-----------------+--------------------------------+--------------------------------+ | iso8859_13 | iso-8859-13, latin7, L7 | Baltic languages | +-----------------+--------------------------------+--------------------------------+ | iso8859_14 | iso-8859-14, latin8, L8 | Celtic languages | -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 24 09:05:41 2015 From: python-checkins at python.org (victor.stinner) Date: Thu, 24 Sep 2015 07:05:41 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogTWVyZ2UgMy41IChjb2RlY3MsIGlzc3VlICMyNDg5NCk=?= Message-ID: <20150924070541.16583.64558@psf.io> https://hg.python.org/cpython/rev/97842831c635 changeset: 98237:97842831c635 parent: 98234:d20deb8601c1 parent: 98236:84a918335fe5 user: Victor Stinner date: Thu Sep 24 09:04:44 2015 +0200 summary: Merge 3.5 (codecs, issue #24894) files: Doc/library/codecs.rst | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) diff --git a/Doc/library/codecs.rst b/Doc/library/codecs.rst --- a/Doc/library/codecs.rst +++ b/Doc/library/codecs.rst @@ -1148,6 +1148,8 @@ +-----------------+--------------------------------+--------------------------------+ | iso8859_10 | iso-8859-10, latin6, L6 | Nordic languages | +-----------------+--------------------------------+--------------------------------+ +| iso8859_11 | iso-8859-11, thai | Thai languages | ++-----------------+--------------------------------+--------------------------------+ | iso8859_13 | iso-8859-13, latin7, L7 | Baltic languages | +-----------------+--------------------------------+--------------------------------+ | iso8859_14 | iso-8859-14, latin8, L8 | Celtic languages | -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 24 09:05:43 2015 From: python-checkins at python.org (victor.stinner) Date: Thu, 24 Sep 2015 07:05:43 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzI0ODk0?= =?utf-8?q?=3A_Document_the_codec_iso8859=5F11?= Message-ID: <20150924070541.81641.94082@psf.io> https://hg.python.org/cpython/rev/ad560409c7d6 changeset: 98238:ad560409c7d6 branch: 2.7 parent: 98231:e7bf0727f7df user: Victor Stinner date: Thu Sep 24 09:05:19 2015 +0200 summary: Issue #24894: Document the codec iso8859_11 Patch written by Prashant Tyagi. files: Doc/library/codecs.rst | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) diff --git a/Doc/library/codecs.rst b/Doc/library/codecs.rst --- a/Doc/library/codecs.rst +++ b/Doc/library/codecs.rst @@ -1067,6 +1067,8 @@ +-----------------+--------------------------------+--------------------------------+ | iso8859_10 | iso-8859-10, latin6, L6 | Nordic languages | +-----------------+--------------------------------+--------------------------------+ +| iso8859_11 | iso-8859-11, thai | Thai languages | ++-----------------+--------------------------------+--------------------------------+ | iso8859_13 | iso-8859-13, latin7, L7 | Baltic languages | +-----------------+--------------------------------+--------------------------------+ | iso8859_14 | iso-8859-14, latin8, L8 | Celtic languages | -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 24 09:10:25 2015 From: python-checkins at python.org (terry.reedy) Date: Thu, 24 Sep 2015 07:10:25 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgMjE5OTU6?= =?utf-8?q?_Explain_some_differences_between_IDLE_and_console_Python=2E?= Message-ID: <20150924071025.16589.41863@psf.io> https://hg.python.org/cpython/rev/ac6ade0c5927 changeset: 98239:ac6ade0c5927 branch: 2.7 user: Terry Jan Reedy date: Thu Sep 24 03:09:38 2015 -0400 summary: Issue 21995: Explain some differences between IDLE and console Python. files: Doc/library/idle.rst | 25 +++++++++++++++++++++---- Lib/idlelib/help.html | 29 ++++++++++++++++++++++------- 2 files changed, 43 insertions(+), 11 deletions(-) diff --git a/Doc/library/idle.rst b/Doc/library/idle.rst --- a/Doc/library/idle.rst +++ b/Doc/library/idle.rst @@ -16,7 +16,7 @@ * coded in 100% pure Python, using the :mod:`tkinter` GUI toolkit -* cross-platform: works on Windows, Unix, and Mac OS X +* cross-platform: works mostly the same on Windows, Unix, and Mac OS X * Python shell window (interactive interpreter) with colorizing of code input, output, and error messages @@ -492,8 +492,8 @@ black -Startup -------- +Startup and code execution +-------------------------- Upon startup with the ``-s`` option, IDLE will execute the file referenced by the environment variables :envvar:`IDLESTARTUP` or :envvar:`PYTHONSTARTUP`. @@ -538,10 +538,27 @@ ``sys.argv`` reflects the arguments passed to IDLE itself. +IDLE-console differences +^^^^^^^^^^^^^^^^^^^^^^^^ + +As much as possible, the result of executing Python code with IDLE is the +same as executing the same code in a console window. However, the different +interface and operation occasionally affects results. + +For instance, IDLE normally executes user code in a separate process from +the IDLE GUI itself. The IDLE versions of sys.stdin, .stdout, and .stderr in the +execution process get input from and send output to the GUI process, +which keeps control of the keyboard and screen. This is normally transparent, +but code that access these object will see different attribute values. +Also, functions that directly access the keyboard and screen will not work. + +With IDLE's Shell, one enters, edits, and recalls complete statements. +Some consoles only work with a single physical line at a time. + Running without a subprocess ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -By default, Idle executes user code in a separate subprocess via a socket, +By default, IDLE executes user code in a separate subprocess via a socket, which uses the internal loopback interface. This connection is not externally visible and no data is sent to or received from the Internet. If firewall software complains anyway, you can ignore it. diff --git a/Lib/idlelib/help.html b/Lib/idlelib/help.html --- a/Lib/idlelib/help.html +++ b/Lib/idlelib/help.html @@ -79,7 +79,7 @@

IDLE has the following features:

  • coded in 100% pure Python, using the tkinter GUI toolkit
  • -
  • cross-platform: works on Windows, Unix, and Mac OS X
  • +
  • cross-platform: works mostly the same on Windows, Unix, and Mac OS X
  • Python shell window (interactive interpreter) with colorizing of code input, output, and error messages
  • multi-window text editor with multiple undo, Python colorizing, @@ -472,8 +472,8 @@
-
-

24.6.4. Startup?

+
+

24.6.4. Startup and code execution?

Upon startup with the -s option, IDLE will execute the file referenced by the environment variables IDLESTARTUP or PYTHONSTARTUP. IDLE first checks for IDLESTARTUP; if IDLESTARTUP is present the file @@ -511,9 +511,23 @@ sys.argv reflects the arguments passed to IDLE itself.

+
+

24.6.4.2. IDLE-console differences?

+

As much as possible, the result of executing Python code with IDLE is the +same as executing the same code in a console window. However, the different +interface and operation occasionally affects results.

+

For instance, IDLE normally executes user code in a separate process from +the IDLE GUI itself. The IDLE versions of sys.stdin, .stdout, and .stderr in the +execution process get input from and send output to the GUI process, +which keeps control of the keyboard and screen. This is normally transparent, +but code that access these object will see different attribute values. +Also, functions that directly access the keyboard and screen will not work.

+

With IDLE’s Shell, one enters, edits, and recalls complete statements. +Some consoles only work with a single physical line at a time.

+
-

24.6.4.2. Running without a subprocess?

-

By default, Idle executes user code in a separate subprocess via a socket, +

24.6.4.3. Running without a subprocess?

+

By default, IDLE executes user code in a separate subprocess via a socket, which uses the internal loopback interface. This connection is not externally visible and no data is sent to or received from the Internet. If firewall software complains anyway, you can ignore it.

@@ -604,9 +618,10 @@
  • 24.6.3. Syntax colors
  • -
  • 24.6.4. Startup
      +
    • 24.6.4. Startup and code execution
    • 24.6.5. Help and preferences
        -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 24 09:10:25 2015 From: python-checkins at python.org (terry.reedy) Date: Thu, 24 Sep 2015 07:10:25 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Merge_with_3=2E4?= Message-ID: <20150924071025.9931.95883@psf.io> https://hg.python.org/cpython/rev/203efe78b3f2 changeset: 98241:203efe78b3f2 branch: 3.5 parent: 98236:84a918335fe5 parent: 98240:ca6c9cc77c20 user: Terry Jan Reedy date: Thu Sep 24 03:09:56 2015 -0400 summary: Merge with 3.4 files: Doc/library/idle.rst | 25 +++++++++++++++++--- Lib/idlelib/help.html | 38 +++++++++++++++++++++++------- 2 files changed, 50 insertions(+), 13 deletions(-) diff --git a/Doc/library/idle.rst b/Doc/library/idle.rst --- a/Doc/library/idle.rst +++ b/Doc/library/idle.rst @@ -16,7 +16,7 @@ * coded in 100% pure Python, using the :mod:`tkinter` GUI toolkit -* cross-platform: works on Windows, Unix, and Mac OS X +* cross-platform: works mostly the same on Windows, Unix, and Mac OS X * Python shell window (interactive interpreter) with colorizing of code input, output, and error messages @@ -492,8 +492,8 @@ black -Startup -------- +Startup and code execution +-------------------------- Upon startup with the ``-s`` option, IDLE will execute the file referenced by the environment variables :envvar:`IDLESTARTUP` or :envvar:`PYTHONSTARTUP`. @@ -538,10 +538,27 @@ ``sys.argv`` reflects the arguments passed to IDLE itself. +IDLE-console differences +^^^^^^^^^^^^^^^^^^^^^^^^ + +As much as possible, the result of executing Python code with IDLE is the +same as executing the same code in a console window. However, the different +interface and operation occasionally affects results. + +For instance, IDLE normally executes user code in a separate process from +the IDLE GUI itself. The IDLE versions of sys.stdin, .stdout, and .stderr in the +execution process get input from and send output to the GUI process, +which keeps control of the keyboard and screen. This is normally transparent, +but code that access these object will see different attribute values. +Also, functions that directly access the keyboard and screen will not work. + +With IDLE's Shell, one enters, edits, and recalls complete statements. +Some consoles only work with a single physical line at a time. + Running without a subprocess ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -By default, Idle executes user code in a separate subprocess via a socket, +By default, IDLE executes user code in a separate subprocess via a socket, which uses the internal loopback interface. This connection is not externally visible and no data is sent to or received from the Internet. If firewall software complains anyway, you can ignore it. diff --git a/Lib/idlelib/help.html b/Lib/idlelib/help.html --- a/Lib/idlelib/help.html +++ b/Lib/idlelib/help.html @@ -79,7 +79,7 @@

        IDLE has the following features:

        • coded in 100% pure Python, using the tkinter GUI toolkit
        • -
        • cross-platform: works on Windows, Unix, and Mac OS X
        • +
        • cross-platform: works mostly the same on Windows, Unix, and Mac OS X
        • Python shell window (interactive interpreter) with colorizing of code input, output, and error messages
        • multi-window text editor with multiple undo, Python colorizing, @@ -223,7 +223,12 @@ Editor window.
          Run Module
          Do Check Module (above). If no error, restart the shell to clean the -environment, then execute the module.
          +environment, then execute the module. Output is displayed in the Shell +window. Note that output requires use of print or write. +When execution is complete, the Shell retains focus and displays a prompt. +At this point, one may interactively explore the result of execution. +This is similar to executing a file with python -i file at a command +line.
  • @@ -467,8 +472,8 @@
    -
    -

    25.5.4. Startup?

    +
    +

    25.5.4. Startup and code execution?

    Upon startup with the -s option, IDLE will execute the file referenced by the environment variables IDLESTARTUP or PYTHONSTARTUP. IDLE first checks for IDLESTARTUP; if IDLESTARTUP is present the file @@ -506,9 +511,23 @@ sys.argv reflects the arguments passed to IDLE itself.

    +
    +

    25.5.4.2. IDLE-console differences?

    +

    As much as possible, the result of executing Python code with IDLE is the +same as executing the same code in a console window. However, the different +interface and operation occasionally affects results.

    +

    For instance, IDLE normally executes user code in a separate process from +the IDLE GUI itself. The IDLE versions of sys.stdin, .stdout, and .stderr in the +execution process get input from and send output to the GUI process, +which keeps control of the keyboard and screen. This is normally transparent, +but code that access these object will see different attribute values. +Also, functions that directly access the keyboard and screen will not work.

    +

    With IDLE’s Shell, one enters, edits, and recalls complete statements. +Some consoles only work with a single physical line at a time.

    +
    -

    25.5.4.2. Running without a subprocess?

    -

    By default, Idle executes user code in a separate subprocess via a socket, +

    25.5.4.3. Running without a subprocess?

    +

    By default, IDLE executes user code in a separate subprocess via a socket, which uses the internal loopback interface. This connection is not externally visible and no data is sent to or received from the Internet. If firewall software complains anyway, you can ignore it.

    @@ -599,9 +618,10 @@
  • 25.5.3. Syntax colors
  • -
  • 25.5.4. Startup
      +
    • 25.5.4. Startup and code execution
    • 25.5.5. Help and preferences
        @@ -676,7 +696,7 @@ The Python Software Foundation is a non-profit corporation. Please donate.
        - Last updated on Sep 23, 2015. + Last updated on Sep 24, 2015. Found a bug?
        Created using Sphinx 1.2.3. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 24 09:10:25 2015 From: python-checkins at python.org (terry.reedy) Date: Thu, 24 Sep 2015 07:10:25 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgMjE5OTU6?= =?utf-8?q?_Explain_some_differences_between_IDLE_and_console_Python=2E?= Message-ID: <20150924071025.9957.16393@psf.io> https://hg.python.org/cpython/rev/ca6c9cc77c20 changeset: 98240:ca6c9cc77c20 branch: 3.4 parent: 98235:aa9e0dcbbcc2 user: Terry Jan Reedy date: Thu Sep 24 03:09:43 2015 -0400 summary: Issue 21995: Explain some differences between IDLE and console Python. files: Doc/library/idle.rst | 25 +++++++++++++++++--- Lib/idlelib/help.html | 38 +++++++++++++++++++++++------- 2 files changed, 50 insertions(+), 13 deletions(-) diff --git a/Doc/library/idle.rst b/Doc/library/idle.rst --- a/Doc/library/idle.rst +++ b/Doc/library/idle.rst @@ -16,7 +16,7 @@ * coded in 100% pure Python, using the :mod:`tkinter` GUI toolkit -* cross-platform: works on Windows, Unix, and Mac OS X +* cross-platform: works mostly the same on Windows, Unix, and Mac OS X * Python shell window (interactive interpreter) with colorizing of code input, output, and error messages @@ -492,8 +492,8 @@ black -Startup -------- +Startup and code execution +-------------------------- Upon startup with the ``-s`` option, IDLE will execute the file referenced by the environment variables :envvar:`IDLESTARTUP` or :envvar:`PYTHONSTARTUP`. @@ -538,10 +538,27 @@ ``sys.argv`` reflects the arguments passed to IDLE itself. +IDLE-console differences +^^^^^^^^^^^^^^^^^^^^^^^^ + +As much as possible, the result of executing Python code with IDLE is the +same as executing the same code in a console window. However, the different +interface and operation occasionally affects results. + +For instance, IDLE normally executes user code in a separate process from +the IDLE GUI itself. The IDLE versions of sys.stdin, .stdout, and .stderr in the +execution process get input from and send output to the GUI process, +which keeps control of the keyboard and screen. This is normally transparent, +but code that access these object will see different attribute values. +Also, functions that directly access the keyboard and screen will not work. + +With IDLE's Shell, one enters, edits, and recalls complete statements. +Some consoles only work with a single physical line at a time. + Running without a subprocess ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -By default, Idle executes user code in a separate subprocess via a socket, +By default, IDLE executes user code in a separate subprocess via a socket, which uses the internal loopback interface. This connection is not externally visible and no data is sent to or received from the Internet. If firewall software complains anyway, you can ignore it. diff --git a/Lib/idlelib/help.html b/Lib/idlelib/help.html --- a/Lib/idlelib/help.html +++ b/Lib/idlelib/help.html @@ -79,7 +79,7 @@

        IDLE has the following features:

        • coded in 100% pure Python, using the tkinter GUI toolkit
        • -
        • cross-platform: works on Windows, Unix, and Mac OS X
        • +
        • cross-platform: works mostly the same on Windows, Unix, and Mac OS X
        • Python shell window (interactive interpreter) with colorizing of code input, output, and error messages
        • multi-window text editor with multiple undo, Python colorizing, @@ -223,7 +223,12 @@ Editor window.
          Run Module
          Do Check Module (above). If no error, restart the shell to clean the -environment, then execute the module.
          +environment, then execute the module. Output is displayed in the Shell +window. Note that output requires use of print or write. +When execution is complete, the Shell retains focus and displays a prompt. +At this point, one may interactively explore the result of execution. +This is similar to executing a file with python -i file at a command +line.
  • @@ -467,8 +472,8 @@
    -
    -

    25.5.4. Startup?

    +
    +

    25.5.4. Startup and code execution?

    Upon startup with the -s option, IDLE will execute the file referenced by the environment variables IDLESTARTUP or PYTHONSTARTUP. IDLE first checks for IDLESTARTUP; if IDLESTARTUP is present the file @@ -506,9 +511,23 @@ sys.argv reflects the arguments passed to IDLE itself.

    +
    +

    25.5.4.2. IDLE-console differences?

    +

    As much as possible, the result of executing Python code with IDLE is the +same as executing the same code in a console window. However, the different +interface and operation occasionally affects results.

    +

    For instance, IDLE normally executes user code in a separate process from +the IDLE GUI itself. The IDLE versions of sys.stdin, .stdout, and .stderr in the +execution process get input from and send output to the GUI process, +which keeps control of the keyboard and screen. This is normally transparent, +but code that access these object will see different attribute values. +Also, functions that directly access the keyboard and screen will not work.

    +

    With IDLE’s Shell, one enters, edits, and recalls complete statements. +Some consoles only work with a single physical line at a time.

    +
    -

    25.5.4.2. Running without a subprocess?

    -

    By default, Idle executes user code in a separate subprocess via a socket, +

    25.5.4.3. Running without a subprocess?

    +

    By default, IDLE executes user code in a separate subprocess via a socket, which uses the internal loopback interface. This connection is not externally visible and no data is sent to or received from the Internet. If firewall software complains anyway, you can ignore it.

    @@ -599,9 +618,10 @@
  • 25.5.3. Syntax colors
  • -
  • 25.5.4. Startup
      +
    • 25.5.4. Startup and code execution
    • 25.5.5. Help and preferences
        @@ -676,7 +696,7 @@ The Python Software Foundation is a non-profit corporation. Please donate.
        - Last updated on Sep 23, 2015. + Last updated on Sep 24, 2015. Found a bug?
        Created using Sphinx 1.2.3. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 24 09:10:26 2015 From: python-checkins at python.org (terry.reedy) Date: Thu, 24 Sep 2015 07:10:26 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge_with_3=2E5?= Message-ID: <20150924071025.82654.79073@psf.io> https://hg.python.org/cpython/rev/ba738eedf871 changeset: 98242:ba738eedf871 parent: 98237:97842831c635 parent: 98241:203efe78b3f2 user: Terry Jan Reedy date: Thu Sep 24 03:10:07 2015 -0400 summary: Merge with 3.5 files: Doc/library/idle.rst | 25 +++++++++++++++++--- Lib/idlelib/help.html | 38 +++++++++++++++++++++++------- 2 files changed, 50 insertions(+), 13 deletions(-) diff --git a/Doc/library/idle.rst b/Doc/library/idle.rst --- a/Doc/library/idle.rst +++ b/Doc/library/idle.rst @@ -16,7 +16,7 @@ * coded in 100% pure Python, using the :mod:`tkinter` GUI toolkit -* cross-platform: works on Windows, Unix, and Mac OS X +* cross-platform: works mostly the same on Windows, Unix, and Mac OS X * Python shell window (interactive interpreter) with colorizing of code input, output, and error messages @@ -492,8 +492,8 @@ black -Startup -------- +Startup and code execution +-------------------------- Upon startup with the ``-s`` option, IDLE will execute the file referenced by the environment variables :envvar:`IDLESTARTUP` or :envvar:`PYTHONSTARTUP`. @@ -538,10 +538,27 @@ ``sys.argv`` reflects the arguments passed to IDLE itself. +IDLE-console differences +^^^^^^^^^^^^^^^^^^^^^^^^ + +As much as possible, the result of executing Python code with IDLE is the +same as executing the same code in a console window. However, the different +interface and operation occasionally affects results. + +For instance, IDLE normally executes user code in a separate process from +the IDLE GUI itself. The IDLE versions of sys.stdin, .stdout, and .stderr in the +execution process get input from and send output to the GUI process, +which keeps control of the keyboard and screen. This is normally transparent, +but code that access these object will see different attribute values. +Also, functions that directly access the keyboard and screen will not work. + +With IDLE's Shell, one enters, edits, and recalls complete statements. +Some consoles only work with a single physical line at a time. + Running without a subprocess ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -By default, Idle executes user code in a separate subprocess via a socket, +By default, IDLE executes user code in a separate subprocess via a socket, which uses the internal loopback interface. This connection is not externally visible and no data is sent to or received from the Internet. If firewall software complains anyway, you can ignore it. diff --git a/Lib/idlelib/help.html b/Lib/idlelib/help.html --- a/Lib/idlelib/help.html +++ b/Lib/idlelib/help.html @@ -79,7 +79,7 @@

        IDLE has the following features:

        • coded in 100% pure Python, using the tkinter GUI toolkit
        • -
        • cross-platform: works on Windows, Unix, and Mac OS X
        • +
        • cross-platform: works mostly the same on Windows, Unix, and Mac OS X
        • Python shell window (interactive interpreter) with colorizing of code input, output, and error messages
        • multi-window text editor with multiple undo, Python colorizing, @@ -223,7 +223,12 @@ Editor window.
          Run Module
          Do Check Module (above). If no error, restart the shell to clean the -environment, then execute the module.
          +environment, then execute the module. Output is displayed in the Shell +window. Note that output requires use of print or write. +When execution is complete, the Shell retains focus and displays a prompt. +At this point, one may interactively explore the result of execution. +This is similar to executing a file with python -i file at a command +line.
  • @@ -467,8 +472,8 @@
    -
    -

    25.5.4. Startup?

    +
    +

    25.5.4. Startup and code execution?

    Upon startup with the -s option, IDLE will execute the file referenced by the environment variables IDLESTARTUP or PYTHONSTARTUP. IDLE first checks for IDLESTARTUP; if IDLESTARTUP is present the file @@ -506,9 +511,23 @@ sys.argv reflects the arguments passed to IDLE itself.

    +
    +

    25.5.4.2. IDLE-console differences?

    +

    As much as possible, the result of executing Python code with IDLE is the +same as executing the same code in a console window. However, the different +interface and operation occasionally affects results.

    +

    For instance, IDLE normally executes user code in a separate process from +the IDLE GUI itself. The IDLE versions of sys.stdin, .stdout, and .stderr in the +execution process get input from and send output to the GUI process, +which keeps control of the keyboard and screen. This is normally transparent, +but code that access these object will see different attribute values. +Also, functions that directly access the keyboard and screen will not work.

    +

    With IDLE’s Shell, one enters, edits, and recalls complete statements. +Some consoles only work with a single physical line at a time.

    +
    -

    25.5.4.2. Running without a subprocess?

    -

    By default, Idle executes user code in a separate subprocess via a socket, +

    25.5.4.3. Running without a subprocess?

    +

    By default, IDLE executes user code in a separate subprocess via a socket, which uses the internal loopback interface. This connection is not externally visible and no data is sent to or received from the Internet. If firewall software complains anyway, you can ignore it.

    @@ -599,9 +618,10 @@
  • 25.5.3. Syntax colors
  • -
  • 25.5.4. Startup
      +
    • 25.5.4. Startup and code execution
    • 25.5.5. Help and preferences
        @@ -676,7 +696,7 @@ The Python Software Foundation is a non-profit corporation. Please donate.
        - Last updated on Sep 23, 2015. + Last updated on Sep 24, 2015. Found a bug?
        Created using Sphinx 1.2.3. -- Repository URL: https://hg.python.org/cpython From solipsis at pitrou.net Thu Sep 24 10:43:47 2015 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Thu, 24 Sep 2015 08:43:47 +0000 Subject: [Python-checkins] Daily reference leaks (d20deb8601c1): sum=17880 Message-ID: <20150924084346.94109.47920@psf.io> results for d20deb8601c1 on branch "default" -------------------------------------------- test_asyncio leaked [0, 3, 0] memory blocks, sum=3 test_capi leaked [1598, 1598, 1598] references, sum=4794 test_capi leaked [387, 389, 389] memory blocks, sum=1165 test_functools leaked [0, 2, 2] memory blocks, sum=4 test_threading leaked [3196, 3196, 3196] references, sum=9588 test_threading leaked [774, 776, 776] memory blocks, sum=2326 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/psf-users/antoine/refleaks/reflogYwGSWV', '--timeout', '7200'] From python-checkins at python.org Thu Sep 24 13:34:27 2015 From: python-checkins at python.org (andrew.svetlov) Date: Thu, 24 Sep 2015 11:34:27 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogRml4ICMyNTIwODog?= =?utf-8?q?Improve_=22Develop_with_asyncio=22_doc_page=2E?= Message-ID: <20150924113427.16583.41541@psf.io> https://hg.python.org/cpython/rev/3909d29d29fc changeset: 98243:3909d29d29fc branch: 3.4 parent: 98240:ca6c9cc77c20 user: Andrew Svetlov date: Thu Sep 24 14:32:39 2015 +0300 summary: Fix #25208: Improve "Develop with asyncio" doc page. Patch by Benjamin Hodgson. files: Doc/library/asyncio-dev.rst | 10 +++++----- 1 files changed, 5 insertions(+), 5 deletions(-) diff --git a/Doc/library/asyncio-dev.rst b/Doc/library/asyncio-dev.rst --- a/Doc/library/asyncio-dev.rst +++ b/Doc/library/asyncio-dev.rst @@ -14,14 +14,14 @@ Debug mode of asyncio --------------------- -The implementation of :mod:`asyncio` module has been written for performances. -To development with asyncio, it's required to enable the debug checks to ease -the development of asynchronous code. +The implementation of :mod:`asyncio` has been written for performance. +In order to ease the development of asynchronous code, you may wish to +enable *debug mode*. -Setup an application to enable all debug checks: +To enable all debug checks for an application: * Enable the asyncio debug mode globally by setting the environment variable - :envvar:`PYTHONASYNCIODEBUG` to ``1`` + :envvar:`PYTHONASYNCIODEBUG` to ``1``, or by calling :meth:`BaseEventLoop.set_debug`. * Set the log level of the :ref:`asyncio logger ` to :py:data:`logging.DEBUG`. For example, call ``logging.basicConfig(level=logging.DEBUG)`` at startup. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 24 13:34:31 2015 From: python-checkins at python.org (andrew.svetlov) Date: Thu, 24 Sep 2015 11:34:31 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_Add_Benjamin_H?= =?utf-8?q?odgson_to_Misc/ACK?= Message-ID: <20150924113427.81619.37004@psf.io> https://hg.python.org/cpython/rev/6b2c27d29ed6 changeset: 98244:6b2c27d29ed6 branch: 3.4 user: Andrew Svetlov date: Thu Sep 24 14:34:07 2015 +0300 summary: Add Benjamin Hodgson to Misc/ACK files: Misc/ACKS | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -579,6 +579,7 @@ Konrad Hinsen David Hobley Tim Hochberg +Benjamin Hodgson Joerg-Cyril Hoehle Gregor Hoffleit Chris Hoffman -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 24 13:35:25 2015 From: python-checkins at python.org (andrew.svetlov) Date: Thu, 24 Sep 2015 11:35:25 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Merge_3=2E4_-=3E_3=2E5?= Message-ID: <20150924113525.9939.63943@psf.io> https://hg.python.org/cpython/rev/a76a61d6fb21 changeset: 98245:a76a61d6fb21 branch: 3.5 parent: 98241:203efe78b3f2 parent: 98244:6b2c27d29ed6 user: Andrew Svetlov date: Thu Sep 24 14:35:16 2015 +0300 summary: Merge 3.4 -> 3.5 files: Doc/library/asyncio-dev.rst | 10 +++++----- Misc/ACKS | 1 + 2 files changed, 6 insertions(+), 5 deletions(-) diff --git a/Doc/library/asyncio-dev.rst b/Doc/library/asyncio-dev.rst --- a/Doc/library/asyncio-dev.rst +++ b/Doc/library/asyncio-dev.rst @@ -14,14 +14,14 @@ Debug mode of asyncio --------------------- -The implementation of :mod:`asyncio` module has been written for performances. -To development with asyncio, it's required to enable the debug checks to ease -the development of asynchronous code. +The implementation of :mod:`asyncio` has been written for performance. +In order to ease the development of asynchronous code, you may wish to +enable *debug mode*. -Setup an application to enable all debug checks: +To enable all debug checks for an application: * Enable the asyncio debug mode globally by setting the environment variable - :envvar:`PYTHONASYNCIODEBUG` to ``1`` + :envvar:`PYTHONASYNCIODEBUG` to ``1``, or by calling :meth:`BaseEventLoop.set_debug`. * Set the log level of the :ref:`asyncio logger ` to :py:data:`logging.DEBUG`. For example, call ``logging.basicConfig(level=logging.DEBUG)`` at startup. diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -590,6 +590,7 @@ Konrad Hinsen David Hobley Tim Hochberg +Benjamin Hodgson Joerg-Cyril Hoehle Gregor Hoffleit Chris Hoffman -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 24 13:36:06 2015 From: python-checkins at python.org (andrew.svetlov) Date: Thu, 24 Sep 2015 11:36:06 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge_3=2E5_-=3E_default?= Message-ID: <20150924113605.3660.5983@psf.io> https://hg.python.org/cpython/rev/8f91b475c2fd changeset: 98246:8f91b475c2fd parent: 98242:ba738eedf871 parent: 98245:a76a61d6fb21 user: Andrew Svetlov date: Thu Sep 24 14:36:00 2015 +0300 summary: Merge 3.5 -> default files: Doc/library/asyncio-dev.rst | 10 +++++----- Misc/ACKS | 1 + 2 files changed, 6 insertions(+), 5 deletions(-) diff --git a/Doc/library/asyncio-dev.rst b/Doc/library/asyncio-dev.rst --- a/Doc/library/asyncio-dev.rst +++ b/Doc/library/asyncio-dev.rst @@ -14,14 +14,14 @@ Debug mode of asyncio --------------------- -The implementation of :mod:`asyncio` module has been written for performances. -To development with asyncio, it's required to enable the debug checks to ease -the development of asynchronous code. +The implementation of :mod:`asyncio` has been written for performance. +In order to ease the development of asynchronous code, you may wish to +enable *debug mode*. -Setup an application to enable all debug checks: +To enable all debug checks for an application: * Enable the asyncio debug mode globally by setting the environment variable - :envvar:`PYTHONASYNCIODEBUG` to ``1`` + :envvar:`PYTHONASYNCIODEBUG` to ``1``, or by calling :meth:`BaseEventLoop.set_debug`. * Set the log level of the :ref:`asyncio logger ` to :py:data:`logging.DEBUG`. For example, call ``logging.basicConfig(level=logging.DEBUG)`` at startup. diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -590,6 +590,7 @@ Konrad Hinsen David Hobley Tim Hochberg +Benjamin Hodgson Joerg-Cyril Hoehle Gregor Hoffleit Chris Hoffman -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 24 14:54:38 2015 From: python-checkins at python.org (eric.smith) Date: Thu, 24 Sep 2015 12:54:38 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_Issue_25206=3A_fix_f-string_e?= =?utf-8?q?xceptions_to_match_the_code=2E?= Message-ID: <20150924125420.31179.87125@psf.io> https://hg.python.org/peps/rev/7e32da8da58e changeset: 6099:7e32da8da58e user: Eric V. Smith date: Thu Sep 24 08:54:17 2015 -0400 summary: Issue 25206: fix f-string exceptions to match the code. files: pep-0498.txt | 8 +++----- 1 files changed, 3 insertions(+), 5 deletions(-) diff --git a/pep-0498.txt b/pep-0498.txt --- a/pep-0498.txt +++ b/pep-0498.txt @@ -398,15 +398,13 @@ >>> f'x={x' File "", line 1 - SyntaxError: missing '}' in format string expression + SyntaxError: f-string: expecting '}' Invalid expressions:: >>> f'x={!x}' - File "", line 1 - !x - ^ - SyntaxError: invalid syntax + File "", line 1 + SyntaxError: f-string: empty expression not allowed Run time errors occur when evaluating the expressions inside an f-string. Note that an f-string can be evaluated multiple times, and -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Thu Sep 24 14:54:38 2015 From: python-checkins at python.org (victor.stinner) Date: Thu, 24 Sep 2015 12:54:38 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2325227=3A_Cleanup_?= =?utf-8?q?unicode=5Fencode=5Fucs1=28=29_error_handler?= Message-ID: <20150924125420.3662.9480@psf.io> https://hg.python.org/cpython/rev/fa65c32d7134 changeset: 98248:fa65c32d7134 user: Victor Stinner date: Thu Sep 24 14:45:00 2015 +0200 summary: Issue #25227: Cleanup unicode_encode_ucs1() error handler * Change limit type from unsigned int to Py_UCS4, to use the same type than the "ch" variable (an Unicode character). * Reuse ch variable for _Py_ERROR_XMLCHARREFREPLACE * Add some newlines for readability files: Objects/unicodeobject.c | 22 +++++++++++++--------- 1 files changed, 13 insertions(+), 9 deletions(-) diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c --- a/Objects/unicodeobject.c +++ b/Objects/unicodeobject.c @@ -6415,7 +6415,7 @@ static PyObject * unicode_encode_ucs1(PyObject *unicode, const char *errors, - unsigned int limit) + const Py_UCS4 limit) { /* input state */ Py_ssize_t pos=0, size; @@ -6449,12 +6449,12 @@ ressize = size; while (pos < size) { - Py_UCS4 c = PyUnicode_READ(kind, data, pos); + Py_UCS4 ch = PyUnicode_READ(kind, data, pos); /* can we encode this? */ - if (c0; ++i, ++str) { - c = PyUnicode_READ_CHAR(repunicode, i); - if (c >= limit) { + ch = PyUnicode_READ_CHAR(repunicode, i); + if (ch >= limit) { raise_encode_exception(&exc, encoding, unicode, pos, pos+1, reason); Py_DECREF(repunicode); goto onError; } - *str = (char)c; + *str = (char)ch; } pos = newpos; Py_DECREF(repunicode); -- Repository URL: https://hg.python.org/cpython From lp_benchmark_robot at intel.com Thu Sep 24 15:31:44 2015 From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com) Date: Thu, 24 Sep 2015 14:31:44 +0100 Subject: [Python-checkins] Benchmark Results for Python Default 2015-09-24 Message-ID: <35ef9654-5f9c-4140-bce9-b80864d48e14@irsmsx104.ger.corp.intel.com> Results for project python_default-nightly, build date 2015-09-24 03:02:02 commit: 2b71c9db17a55e48b3fc04f15f3ec867f364085c revision date: 2015-09-24 02:15:44 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v3.4.3, with hash b4cbecbc0781e89a309d03b60a1f75f8499250e6 from 2015-02-25 12:15:33+00:00 ------------------------------------------------------------------------------------------ benchmark relative change since change since current rev with std_dev* last run v3.4.3 regrtest PGO ------------------------------------------------------------------------------------------ :-) django_v2 0.18264% -0.75203% 7.56391% 16.35872% :-( pybench 0.08466% -0.05211% -2.32602% 9.02563% :-( regex_v8 2.80228% 0.07997% -5.40526% 7.50645% :-| nbody 0.26428% -0.03359% 0.33694% 10.33234% :-( json_dump_v2 0.26005% -0.93397% -3.72643% 13.20410% :-| normal_startup 0.57887% 0.15483% 0.36088% 5.33375% ------------------------------------------------------------------------------------------ Note: Benchmark results are measured in seconds. * Relative Standard Deviation (Standard Deviation/Average) Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. From lp_benchmark_robot at intel.com Thu Sep 24 15:33:42 2015 From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com) Date: Thu, 24 Sep 2015 14:33:42 +0100 Subject: [Python-checkins] Benchmark Results for Python 2.7 2015-09-24 Message-ID: <1e17b951-e4f2-4163-b8bc-1715204e0f0e@irsmsx104.ger.corp.intel.com> Results for project python_2.7-nightly, build date 2015-09-24 03:42:58 commit: e2f1f69d0618f70a72b9dc0d6cdbce162d72f11e revision date: 2015-09-24 00:19:42 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v2.7.10, with hash 15c95b7d81dcf821daade360741e00714667653f from 2015-05-23 16:02:14+00:00 ------------------------------------------------------------------------------------------ benchmark relative change since change since current rev with std_dev* last run v2.7.10 regrtest PGO ------------------------------------------------------------------------------------------ :-) django_v2 0.28025% -0.55377% 4.00355% 8.72489% :-) pybench 0.14593% 0.02415% 6.79149% 6.91599% :-| regex_v8 0.61847% 0.02126% -1.14722% 7.78544% :-) nbody 0.11601% 0.01084% 9.05576% 4.12003% :-) json_dump_v2 0.25821% 0.37153% 4.43383% 13.09859% :-| normal_startup 1.59986% 0.54183% -1.10534% 2.32355% :-) ssbench 0.45407% 0.64119% 2.53659% 1.46234% ------------------------------------------------------------------------------------------ Note: Benchmark results for ssbench are measured in requests/second while all other are measured in seconds. * Relative Standard Deviation (Standard Deviation/Average) Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. From python-checkins at python.org Thu Sep 24 16:04:53 2015 From: python-checkins at python.org (eric.smith) Date: Thu, 24 Sep 2015 14:04:53 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Fixed_error_creation_if_th?= =?utf-8?q?e_problem_is_an_empty_expression_in_an_f-string=3A_use?= Message-ID: <20150924125211.31183.4270@psf.io> https://hg.python.org/cpython/rev/e671c2a25f05 changeset: 98247:e671c2a25f05 user: Eric V. Smith date: Thu Sep 24 08:52:04 2015 -0400 summary: Fixed error creation if the problem is an empty expression in an f-string: use ast_error instead of PyErr_SetString. files: Python/ast.c | 11 +++++------ 1 files changed, 5 insertions(+), 6 deletions(-) diff --git a/Python/ast.c b/Python/ast.c --- a/Python/ast.c +++ b/Python/ast.c @@ -4008,7 +4008,8 @@ example. */ static expr_ty fstring_compile_expr(PyObject *str, Py_ssize_t expr_start, - Py_ssize_t expr_end, PyArena *arena) + Py_ssize_t expr_end, struct compiling *c, const node *n) + { PyCompilerFlags cf; mod_ty mod; @@ -4048,8 +4049,7 @@ } } if (all_whitespace) { - PyErr_SetString(PyExc_SyntaxError, "f-string: empty expression " - "not allowed"); + ast_error(c, n, "f-string: empty expression not allowed"); goto error; } @@ -4095,7 +4095,7 @@ cf.cf_flags = PyCF_ONLY_AST; mod = PyParser_ASTFromString(utf_expr, "", - Py_eval_input, &cf, arena); + Py_eval_input, &cf, c->c_arena); if (!mod) goto error; @@ -4370,8 +4370,7 @@ /* Compile the expression as soon as possible, so we show errors related to the expression before errors related to the conversion or format_spec. */ - simple_expression = fstring_compile_expr(str, expr_start, expr_end, - c->c_arena); + simple_expression = fstring_compile_expr(str, expr_start, expr_end, c, n); if (!simple_expression) return -1; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 24 18:02:04 2015 From: python-checkins at python.org (donald.stufft) Date: Thu, 24 Sep 2015 16:02:04 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_Add_the_data-gpg-sig_attribut?= =?utf-8?q?e_to_file_links?= Message-ID: <20150924160138.94137.79436@psf.io> https://hg.python.org/peps/rev/9090e66cc8c7 changeset: 6100:9090e66cc8c7 user: Donald Stufft date: Thu Sep 24 12:01:20 2015 -0400 summary: Add the data-gpg-sig attribute to file links files: pep-0503.txt | 4 ++++ 1 files changed, 4 insertions(+), 0 deletions(-) diff --git a/pep-0503.txt b/pep-0503.txt --- a/pep-0503.txt +++ b/pep-0503.txt @@ -91,6 +91,10 @@ associated signature, the signature would be located at ``/packages/HolyGrail-1.0.tar.gz.asc``. +* A repository **MAY** include a ``data-gpg-sig`` attribute on a file link with + a value of either ``true`` or ``false`` to indicate whether or not there is a + GPG signature. Repositories that do this **SHOULD** include it on every link. + Normalized Names ---------------- -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Thu Sep 24 18:22:01 2015 From: python-checkins at python.org (donald.stufft) Date: Thu, 24 Sep 2015 16:22:01 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_Accept_the_PEP?= Message-ID: <20150924160517.16587.24025@psf.io> https://hg.python.org/peps/rev/0fa30cba8a69 changeset: 6101:0fa30cba8a69 user: Donald Stufft date: Thu Sep 24 12:05:15 2015 -0400 summary: Accept the PEP files: pep-0503.txt | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-) diff --git a/pep-0503.txt b/pep-0503.txt --- a/pep-0503.txt +++ b/pep-0503.txt @@ -5,11 +5,12 @@ Author: Donald Stufft BDFL-Delegate: Donald Stufft Discussions-To: distutils-sig at python.org -Status: Draft +Status: Accepted Type: Informational Content-Type: text/x-rst Created: 04-Sep-2015 Post-History: 04-Sep-2015 +Resolution: https://mail.python.org/pipermail/distutils-sig/2015-September/026899.html Abstract -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Thu Sep 24 23:33:12 2015 From: python-checkins at python.org (terry.reedy) Date: Thu, 24 Sep 2015 21:33:12 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Merge_with_3=2E4?= Message-ID: <20150924213312.82658.21172@psf.io> https://hg.python.org/cpython/rev/9025f5c58d00 changeset: 98251:9025f5c58d00 branch: 3.5 parent: 98245:a76a61d6fb21 parent: 98250:1d0f4b94066b user: Terry Jan Reedy date: Thu Sep 24 17:32:38 2015 -0400 summary: Merge with 3.4 files: Lib/idlelib/help.py | 19 +++++++++++-------- 1 files changed, 11 insertions(+), 8 deletions(-) diff --git a/Lib/idlelib/help.py b/Lib/idlelib/help.py --- a/Lib/idlelib/help.py +++ b/Lib/idlelib/help.py @@ -48,7 +48,8 @@ def __init__(self, text): HTMLParser.__init__(self, convert_charrefs=True) self.text = text # text widget we're rendering into - self.tags = '' # current text tags to apply + self.tags = '' # current block level text tags to apply + self.chartags = '' # current character level text tags self.show = False # used so we exclude page navigation self.hdrlink = False # used so we don't show header links self.level = 0 # indentation level @@ -78,11 +79,11 @@ elif tag == 'p' and class_ != 'first': s = '\n\n' elif tag == 'span' and class_ == 'pre': - self.tags = 'pre' + self.chartags = 'pre' elif tag == 'span' and class_ == 'versionmodified': - self.tags = 'em' + self.chartags = 'em' elif tag == 'em': - self.tags = 'em' + self.chartags = 'em' elif tag in ['ul', 'ol']: if class_.find('simple') != -1: s = '\n' @@ -120,16 +121,18 @@ self.text.insert('end', '\n\n') self.tags = tag if self.show: - self.text.insert('end', s, self.tags) + self.text.insert('end', s, (self.tags, self.chartags)) def handle_endtag(self, tag): "Handle endtags in help.html." - if tag in ['h1', 'h2', 'h3', 'span', 'em']: + if tag in ['h1', 'h2', 'h3']: self.indent(0) # clear tag, reset indent if self.show and tag in ['h1', 'h2', 'h3']: title = self.data self.contents.append(('toc'+str(self.tocid), title)) self.tocid += 1 + elif tag in ['span', 'em']: + self.chartags = '' elif tag == 'a': self.hdrlink = False elif tag == 'pre': @@ -148,7 +151,7 @@ if d[0:len(self.hprefix)] == self.hprefix: d = d[len(self.hprefix):].strip() self.data += d - self.text.insert('end', d, self.tags) + self.text.insert('end', d, (self.tags, self.chartags)) class HelpText(Text): @@ -165,7 +168,7 @@ self.tag_configure('h1', font=(normalfont, 20, 'bold')) self.tag_configure('h2', font=(normalfont, 18, 'bold')) self.tag_configure('h3', font=(normalfont, 15, 'bold')) - self.tag_configure('pre', font=(fixedfont, 12)) + self.tag_configure('pre', font=(fixedfont, 12), background='#f6f6ff') self.tag_configure('preblock', font=(fixedfont, 10), lmargin1=25, borderwidth=1, relief='solid', background='#eeffcc') self.tag_configure('l1', lmargin1=25, lmargin2=25) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 24 23:33:12 2015 From: python-checkins at python.org (terry.reedy) Date: Thu, 24 Sep 2015 21:33:12 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzI1MTk4?= =?utf-8?q?=3A_In_Idle_doc_viewer=2C_fix_indent_of_fixed-pitch_=3Cpre=3E_t?= =?utf-8?q?ext?= Message-ID: <20150924213312.82652.44279@psf.io> https://hg.python.org/cpython/rev/1d0f4b94066b changeset: 98250:1d0f4b94066b branch: 3.4 parent: 98244:6b2c27d29ed6 user: Terry Jan Reedy date: Thu Sep 24 17:32:01 2015 -0400 summary: Issue #25198: In Idle doc viewer, fix indent of fixed-pitch
         text
        by adding a new tag.  Patch by Mark Roseman. Also give 
         text a very
        light blueish-gray background similar to that used by Sphinx html.
        
        files:
          Lib/idlelib/help.py |  19 +++++++++++--------
          1 files changed, 11 insertions(+), 8 deletions(-)
        
        
        diff --git a/Lib/idlelib/help.py b/Lib/idlelib/help.py
        --- a/Lib/idlelib/help.py
        +++ b/Lib/idlelib/help.py
        @@ -48,7 +48,8 @@
             def __init__(self, text):
                 HTMLParser.__init__(self, convert_charrefs=True)
                 self.text = text         # text widget we're rendering into
        -        self.tags = ''           # current text tags to apply
        +        self.tags = ''           # current block level text tags to apply
        +        self.chartags = ''       # current character level text tags
                 self.show = False        # used so we exclude page navigation
                 self.hdrlink = False     # used so we don't show header links
                 self.level = 0           # indentation level
        @@ -78,11 +79,11 @@
                 elif tag == 'p' and class_ != 'first':
                     s = '\n\n'
                 elif tag == 'span' and class_ == 'pre':
        -            self.tags = 'pre'
        +            self.chartags = 'pre'
                 elif tag == 'span' and class_ == 'versionmodified':
        -            self.tags = 'em'
        +            self.chartags = 'em'
                 elif tag == 'em':
        -            self.tags = 'em'
        +            self.chartags = 'em'
                 elif tag in ['ul', 'ol']:
                     if class_.find('simple') != -1:
                         s = '\n'
        @@ -120,16 +121,18 @@
                         self.text.insert('end', '\n\n')
                     self.tags = tag
                 if self.show:
        -            self.text.insert('end', s, self.tags)
        +            self.text.insert('end', s, (self.tags, self.chartags))
         
             def handle_endtag(self, tag):
                 "Handle endtags in help.html."
        -        if tag in ['h1', 'h2', 'h3', 'span', 'em']:
        +        if tag in ['h1', 'h2', 'h3']:
                     self.indent(0)  # clear tag, reset indent
                     if self.show and tag in ['h1', 'h2', 'h3']:
                         title = self.data
                         self.contents.append(('toc'+str(self.tocid), title))
                         self.tocid += 1
        +        elif tag in ['span', 'em']:
        +            self.chartags = ''
                 elif tag == 'a':
                     self.hdrlink = False
                 elif tag == 'pre':
        @@ -148,7 +151,7 @@
                         if d[0:len(self.hprefix)] == self.hprefix:
                             d = d[len(self.hprefix):].strip()
                         self.data += d
        -            self.text.insert('end', d, self.tags)
        +            self.text.insert('end', d, (self.tags, self.chartags))
         
         
         class HelpText(Text):
        @@ -165,7 +168,7 @@
                 self.tag_configure('h1', font=(normalfont, 20, 'bold'))
                 self.tag_configure('h2', font=(normalfont, 18, 'bold'))
                 self.tag_configure('h3', font=(normalfont, 15, 'bold'))
        -        self.tag_configure('pre', font=(fixedfont, 12))
        +        self.tag_configure('pre', font=(fixedfont, 12), background='#f6f6ff')
                 self.tag_configure('preblock', font=(fixedfont, 10), lmargin1=25,
                         borderwidth=1, relief='solid', background='#eeffcc')
                 self.tag_configure('l1', lmargin1=25, lmargin2=25)
        
        -- 
        Repository URL: https://hg.python.org/cpython
        
        From python-checkins at python.org  Thu Sep 24 23:33:12 2015
        From: python-checkins at python.org (terry.reedy)
        Date: Thu, 24 Sep 2015 21:33:12 +0000
        Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzI1MTk4?=
         =?utf-8?q?=3A_In_Idle_doc_viewer=2C_fix_indent_of_fixed-pitch_=3Cpre=3E_t?=
         =?utf-8?q?ext?=
        Message-ID: <20150924213311.115399.87287@psf.io>
        
        https://hg.python.org/cpython/rev/8b3dc527a62c
        changeset:   98249:8b3dc527a62c
        branch:      2.7
        parent:      98239:ac6ade0c5927
        user:        Terry Jan Reedy 
        date:        Thu Sep 24 17:31:54 2015 -0400
        summary:
          Issue #25198: In Idle doc viewer, fix indent of fixed-pitch 
         text
        by adding a new tag.  Patch by Mark Roseman. Also give 
         text a very
        light blueish-gray background similar to that used by Sphinx html.
        
        files:
          Lib/idlelib/help.py |  19 +++++++++++--------
          1 files changed, 11 insertions(+), 8 deletions(-)
        
        
        diff --git a/Lib/idlelib/help.py b/Lib/idlelib/help.py
        --- a/Lib/idlelib/help.py
        +++ b/Lib/idlelib/help.py
        @@ -48,7 +48,8 @@
             def __init__(self, text):
                 HTMLParser.__init__(self)
                 self.text = text         # text widget we're rendering into
        -        self.tags = ''           # current text tags to apply
        +        self.tags = ''           # current block level text tags to apply
        +        self.chartags = ''       # current character level text tags
                 self.show = False        # used so we exclude page navigation
                 self.hdrlink = False     # used so we don't show header links
                 self.level = 0           # indentation level
        @@ -78,11 +79,11 @@
                 elif tag == 'p' and class_ != 'first':
                     s = '\n\n'
                 elif tag == 'span' and class_ == 'pre':
        -            self.tags = 'pre'
        +            self.chartags = 'pre'
                 elif tag == 'span' and class_ == 'versionmodified':
        -            self.tags = 'em'
        +            self.chartags = 'em'
                 elif tag == 'em':
        -            self.tags = 'em'
        +            self.chartags = 'em'
                 elif tag in ['ul', 'ol']:
                     if class_.find('simple') != -1:
                         s = '\n'
        @@ -120,16 +121,18 @@
                         self.text.insert('end', '\n\n')
                     self.tags = tag
                 if self.show:
        -            self.text.insert('end', s, self.tags)
        +            self.text.insert('end', s, (self.tags, self.chartags))
         
             def handle_endtag(self, tag):
                 "Handle endtags in help.html."
        -        if tag in ['h1', 'h2', 'h3', 'span', 'em']:
        +        if tag in ['h1', 'h2', 'h3']:
                     self.indent(0)  # clear tag, reset indent
                     if self.show and tag in ['h1', 'h2', 'h3']:
                         title = self.data
                         self.contents.append(('toc'+str(self.tocid), title))
                         self.tocid += 1
        +        elif tag in ['span', 'em']:
        +            self.chartags = ''
                 elif tag == 'a':
                     self.hdrlink = False
                 elif tag == 'pre':
        @@ -148,7 +151,7 @@
                         if d[0:len(self.hprefix)] == self.hprefix:
                             d = d[len(self.hprefix):].strip()
                         self.data += d
        -            self.text.insert('end', d, self.tags)
        +            self.text.insert('end', d, (self.tags, self.chartags))
         
             def handle_charref(self, name):
                 self.text.insert('end', unichr(int(name)))
        @@ -168,7 +171,7 @@
                 self.tag_configure('h1', font=(normalfont, 20, 'bold'))
                 self.tag_configure('h2', font=(normalfont, 18, 'bold'))
                 self.tag_configure('h3', font=(normalfont, 15, 'bold'))
        -        self.tag_configure('pre', font=(fixedfont, 12))
        +        self.tag_configure('pre', font=(fixedfont, 12), background='#f6f6ff')
                 self.tag_configure('preblock', font=(fixedfont, 10), lmargin1=25,
                         borderwidth=1, relief='solid', background='#eeffcc')
                 self.tag_configure('l1', lmargin1=25, lmargin2=25)
        
        -- 
        Repository URL: https://hg.python.org/cpython
        
        From python-checkins at python.org  Thu Sep 24 23:33:13 2015
        From: python-checkins at python.org (terry.reedy)
        Date: Thu, 24 Sep 2015 21:33:13 +0000
        Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?=
        	=?utf-8?q?=29=3A_Merge_with_3=2E5?=
        Message-ID: <20150924213312.82658.18726@psf.io>
        
        https://hg.python.org/cpython/rev/7efb00bace60
        changeset:   98252:7efb00bace60
        parent:      98248:fa65c32d7134
        parent:      98251:9025f5c58d00
        user:        Terry Jan Reedy 
        date:        Thu Sep 24 17:32:50 2015 -0400
        summary:
          Merge with 3.5
        
        files:
          Lib/idlelib/help.py |  19 +++++++++++--------
          1 files changed, 11 insertions(+), 8 deletions(-)
        
        
        diff --git a/Lib/idlelib/help.py b/Lib/idlelib/help.py
        --- a/Lib/idlelib/help.py
        +++ b/Lib/idlelib/help.py
        @@ -48,7 +48,8 @@
             def __init__(self, text):
                 HTMLParser.__init__(self, convert_charrefs=True)
                 self.text = text         # text widget we're rendering into
        -        self.tags = ''           # current text tags to apply
        +        self.tags = ''           # current block level text tags to apply
        +        self.chartags = ''       # current character level text tags
                 self.show = False        # used so we exclude page navigation
                 self.hdrlink = False     # used so we don't show header links
                 self.level = 0           # indentation level
        @@ -78,11 +79,11 @@
                 elif tag == 'p' and class_ != 'first':
                     s = '\n\n'
                 elif tag == 'span' and class_ == 'pre':
        -            self.tags = 'pre'
        +            self.chartags = 'pre'
                 elif tag == 'span' and class_ == 'versionmodified':
        -            self.tags = 'em'
        +            self.chartags = 'em'
                 elif tag == 'em':
        -            self.tags = 'em'
        +            self.chartags = 'em'
                 elif tag in ['ul', 'ol']:
                     if class_.find('simple') != -1:
                         s = '\n'
        @@ -120,16 +121,18 @@
                         self.text.insert('end', '\n\n')
                     self.tags = tag
                 if self.show:
        -            self.text.insert('end', s, self.tags)
        +            self.text.insert('end', s, (self.tags, self.chartags))
         
             def handle_endtag(self, tag):
                 "Handle endtags in help.html."
        -        if tag in ['h1', 'h2', 'h3', 'span', 'em']:
        +        if tag in ['h1', 'h2', 'h3']:
                     self.indent(0)  # clear tag, reset indent
                     if self.show and tag in ['h1', 'h2', 'h3']:
                         title = self.data
                         self.contents.append(('toc'+str(self.tocid), title))
                         self.tocid += 1
        +        elif tag in ['span', 'em']:
        +            self.chartags = ''
                 elif tag == 'a':
                     self.hdrlink = False
                 elif tag == 'pre':
        @@ -148,7 +151,7 @@
                         if d[0:len(self.hprefix)] == self.hprefix:
                             d = d[len(self.hprefix):].strip()
                         self.data += d
        -            self.text.insert('end', d, self.tags)
        +            self.text.insert('end', d, (self.tags, self.chartags))
         
         
         class HelpText(Text):
        @@ -165,7 +168,7 @@
                 self.tag_configure('h1', font=(normalfont, 20, 'bold'))
                 self.tag_configure('h2', font=(normalfont, 18, 'bold'))
                 self.tag_configure('h3', font=(normalfont, 15, 'bold'))
        -        self.tag_configure('pre', font=(fixedfont, 12))
        +        self.tag_configure('pre', font=(fixedfont, 12), background='#f6f6ff')
                 self.tag_configure('preblock', font=(fixedfont, 10), lmargin1=25,
                         borderwidth=1, relief='solid', background='#eeffcc')
                 self.tag_configure('l1', lmargin1=25, lmargin2=25)
        
        -- 
        Repository URL: https://hg.python.org/cpython
        
        From python-checkins at python.org  Fri Sep 25 01:23:55 2015
        From: python-checkins at python.org (alexander.belopolsky)
        Date: Thu, 24 Sep 2015 23:23:55 +0000
        Subject: [Python-checkins] =?utf-8?q?peps=3A_PEP_495_=28minor=29=3A_Correc?=
        	=?utf-8?q?ted_a_typo=2E?=
        Message-ID: <20150924232354.16585.16540@psf.io>
        
        https://hg.python.org/peps/rev/c7712f920a8f
        changeset:   6102:c7712f920a8f
        user:        Alexander Belopolsky 
        date:        Thu Sep 24 19:23:49 2015 -0400
        summary:
          PEP 495 (minor): Corrected a typo.
        
        files:
          pep-0495.txt |  2 +-
          1 files changed, 1 insertions(+), 1 deletions(-)
        
        
        diff --git a/pep-0495.txt b/pep-0495.txt
        --- a/pep-0495.txt
        +++ b/pep-0495.txt
        @@ -739,7 +739,7 @@
         naive datetimes.
         
         This leaves us with only one situation where an existing program can
        -start producing diferent results after the implementation of this PEP:
        +start producing different results after the implementation of this PEP:
         when a ``datetime.timestamp()`` method is called on a naive datetime
         instance that happen to be in the fold or the gap.  In the current
         implementation, the result is undefined.  Depending on the system
        
        -- 
        Repository URL: https://hg.python.org/peps
        
        From python-checkins at python.org  Fri Sep 25 05:14:36 2015
        From: python-checkins at python.org (terry.reedy)
        Date: Fri, 25 Sep 2015 03:14:36 +0000
        Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzI1MjI1?=
         =?utf-8?q?=3A_Condense_and_rewrite_Idle_doc_section_on_text_colors=2E?=
        Message-ID: <20150925031435.94133.21260@psf.io>
        
        https://hg.python.org/cpython/rev/2f6aa20c05b3
        changeset:   98253:2f6aa20c05b3
        branch:      2.7
        parent:      98249:8b3dc527a62c
        user:        Terry Jan Reedy 
        date:        Thu Sep 24 23:13:43 2015 -0400
        summary:
          Issue #25225: Condense and rewrite Idle doc section on text colors.
        
        files:
          Doc/library/idle.rst |  42 +++++++++----------------------
          1 files changed, 12 insertions(+), 30 deletions(-)
        
        
        diff --git a/Doc/library/idle.rst b/Doc/library/idle.rst
        --- a/Doc/library/idle.rst
        +++ b/Doc/library/idle.rst
        @@ -458,38 +458,20 @@
           * :kbd:`Return` while on any previous command retrieves that command
         
         
        -Syntax colors
        --------------
        +Text colors
        +^^^^^^^^^^^
         
        -The coloring is applied in a background "thread," so you may occasionally see
        -uncolorized text.  To change the color scheme, edit the ``[Colors]`` section in
        -:file:`config.txt`.
        +Idle defaults to black on white text, but colors text with special meanings.
        +For the shell, these are shell output, shell error, user output, and
        +user error.  For Python code, at the shell prompt or in an editor, these are
        +keywords, builtin class and function names, names following ``class`` and
        +``def``, strings, and comments. For any text window, these are the cursor (when
        +present), found text (when possible), and selected text.
         
        -Python syntax colors:
        -   Keywords
        -      orange
        -
        -   Strings
        -      green
        -
        -   Comments
        -      red
        -
        -   Definitions
        -      blue
        -
        -Shell colors:
        -   Console output
        -      brown
        -
        -   stdout
        -      blue
        -
        -   stderr
        -      dark green
        -
        -   stdin
        -      black
        +Text coloring is done in the background, so uncolorized text is occasionally
        +visible.  To change the color scheme, use the Configure IDLE dialog
        +Highlighting tab.  The marking of debugger breakpoint lines in the editor and
        +text in popups and dialogs is not user-configurable.
         
         
         Startup and code execution
        
        -- 
        Repository URL: https://hg.python.org/cpython
        
        From python-checkins at python.org  Fri Sep 25 05:14:36 2015
        From: python-checkins at python.org (terry.reedy)
        Date: Fri, 25 Sep 2015 03:14:36 +0000
        Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzI1MjI1?=
         =?utf-8?q?=3A_Condense_and_rewrite_Idle_doc_section_on_text_colors=2E?=
        Message-ID: <20150925031436.9929.90997@psf.io>
        
        https://hg.python.org/cpython/rev/f5502ff7ae2d
        changeset:   98254:f5502ff7ae2d
        branch:      3.4
        parent:      98250:1d0f4b94066b
        user:        Terry Jan Reedy 
        date:        Thu Sep 24 23:13:49 2015 -0400
        summary:
          Issue #25225: Condense and rewrite Idle doc section on text colors.
        
        files:
          Doc/library/idle.rst |  42 +++++++++----------------------
          1 files changed, 12 insertions(+), 30 deletions(-)
        
        
        diff --git a/Doc/library/idle.rst b/Doc/library/idle.rst
        --- a/Doc/library/idle.rst
        +++ b/Doc/library/idle.rst
        @@ -458,38 +458,20 @@
           * :kbd:`Return` while on any previous command retrieves that command
         
         
        -Syntax colors
        --------------
        +Text colors
        +^^^^^^^^^^^
         
        -The coloring is applied in a background "thread," so you may occasionally see
        -uncolorized text.  To change the color scheme, edit the ``[Colors]`` section in
        -:file:`config.txt`.
        +Idle defaults to black on white text, but colors text with special meanings.
        +For the shell, these are shell output, shell error, user output, and
        +user error.  For Python code, at the shell prompt or in an editor, these are
        +keywords, builtin class and function names, names following ``class`` and
        +``def``, strings, and comments. For any text window, these are the cursor (when
        +present), found text (when possible), and selected text.
         
        -Python syntax colors:
        -   Keywords
        -      orange
        -
        -   Strings
        -      green
        -
        -   Comments
        -      red
        -
        -   Definitions
        -      blue
        -
        -Shell colors:
        -   Console output
        -      brown
        -
        -   stdout
        -      blue
        -
        -   stderr
        -      dark green
        -
        -   stdin
        -      black
        +Text coloring is done in the background, so uncolorized text is occasionally
        +visible.  To change the color scheme, use the Configure IDLE dialog
        +Highlighting tab.  The marking of debugger breakpoint lines in the editor and
        +text in popups and dialogs is not user-configurable.
         
         
         Startup and code execution
        
        -- 
        Repository URL: https://hg.python.org/cpython
        
        From python-checkins at python.org  Fri Sep 25 05:14:38 2015
        From: python-checkins at python.org (terry.reedy)
        Date: Fri, 25 Sep 2015 03:14:38 +0000
        Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?=
        	=?utf-8?q?_Merge_with_3=2E4?=
        Message-ID: <20150925031436.81633.97583@psf.io>
        
        https://hg.python.org/cpython/rev/6badd794cff1
        changeset:   98255:6badd794cff1
        branch:      3.5
        parent:      98251:9025f5c58d00
        parent:      98254:f5502ff7ae2d
        user:        Terry Jan Reedy 
        date:        Thu Sep 24 23:14:02 2015 -0400
        summary:
          Merge with 3.4
        
        files:
          Doc/library/idle.rst |  42 +++++++++----------------------
          1 files changed, 12 insertions(+), 30 deletions(-)
        
        
        diff --git a/Doc/library/idle.rst b/Doc/library/idle.rst
        --- a/Doc/library/idle.rst
        +++ b/Doc/library/idle.rst
        @@ -458,38 +458,20 @@
           * :kbd:`Return` while on any previous command retrieves that command
         
         
        -Syntax colors
        --------------
        +Text colors
        +^^^^^^^^^^^
         
        -The coloring is applied in a background "thread," so you may occasionally see
        -uncolorized text.  To change the color scheme, edit the ``[Colors]`` section in
        -:file:`config.txt`.
        +Idle defaults to black on white text, but colors text with special meanings.
        +For the shell, these are shell output, shell error, user output, and
        +user error.  For Python code, at the shell prompt or in an editor, these are
        +keywords, builtin class and function names, names following ``class`` and
        +``def``, strings, and comments. For any text window, these are the cursor (when
        +present), found text (when possible), and selected text.
         
        -Python syntax colors:
        -   Keywords
        -      orange
        -
        -   Strings
        -      green
        -
        -   Comments
        -      red
        -
        -   Definitions
        -      blue
        -
        -Shell colors:
        -   Console output
        -      brown
        -
        -   stdout
        -      blue
        -
        -   stderr
        -      dark green
        -
        -   stdin
        -      black
        +Text coloring is done in the background, so uncolorized text is occasionally
        +visible.  To change the color scheme, use the Configure IDLE dialog
        +Highlighting tab.  The marking of debugger breakpoint lines in the editor and
        +text in popups and dialogs is not user-configurable.
         
         
         Startup and code execution
        
        -- 
        Repository URL: https://hg.python.org/cpython
        
        From python-checkins at python.org  Fri Sep 25 05:14:41 2015
        From: python-checkins at python.org (terry.reedy)
        Date: Fri, 25 Sep 2015 03:14:41 +0000
        Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?=
        	=?utf-8?q?=29=3A_Merge_with_3=2E5?=
        Message-ID: <20150925031441.94135.17630@psf.io>
        
        https://hg.python.org/cpython/rev/2d7ec044f090
        changeset:   98256:2d7ec044f090
        parent:      98252:7efb00bace60
        parent:      98255:6badd794cff1
        user:        Terry Jan Reedy 
        date:        Thu Sep 24 23:14:15 2015 -0400
        summary:
          Merge with 3.5
        
        files:
          Doc/library/idle.rst |  42 +++++++++----------------------
          1 files changed, 12 insertions(+), 30 deletions(-)
        
        
        diff --git a/Doc/library/idle.rst b/Doc/library/idle.rst
        --- a/Doc/library/idle.rst
        +++ b/Doc/library/idle.rst
        @@ -458,38 +458,20 @@
           * :kbd:`Return` while on any previous command retrieves that command
         
         
        -Syntax colors
        --------------
        +Text colors
        +^^^^^^^^^^^
         
        -The coloring is applied in a background "thread," so you may occasionally see
        -uncolorized text.  To change the color scheme, edit the ``[Colors]`` section in
        -:file:`config.txt`.
        +Idle defaults to black on white text, but colors text with special meanings.
        +For the shell, these are shell output, shell error, user output, and
        +user error.  For Python code, at the shell prompt or in an editor, these are
        +keywords, builtin class and function names, names following ``class`` and
        +``def``, strings, and comments. For any text window, these are the cursor (when
        +present), found text (when possible), and selected text.
         
        -Python syntax colors:
        -   Keywords
        -      orange
        -
        -   Strings
        -      green
        -
        -   Comments
        -      red
        -
        -   Definitions
        -      blue
        -
        -Shell colors:
        -   Console output
        -      brown
        -
        -   stdout
        -      blue
        -
        -   stderr
        -      dark green
        -
        -   stdin
        -      black
        +Text coloring is done in the background, so uncolorized text is occasionally
        +visible.  To change the color scheme, use the Configure IDLE dialog
        +Highlighting tab.  The marking of debugger breakpoint lines in the editor and
        +text in popups and dialogs is not user-configurable.
         
         
         Startup and code execution
        
        -- 
        Repository URL: https://hg.python.org/cpython
        
        From python-checkins at python.org  Fri Sep 25 05:19:42 2015
        From: python-checkins at python.org (terry.reedy)
        Date: Fri, 25 Sep 2015 03:19:42 +0000
        Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?=
        	=?utf-8?q?_Merge_with_3=2E4?=
        Message-ID: <20150925031942.3644.88776@psf.io>
        
        https://hg.python.org/cpython/rev/e289a81c871e
        changeset:   98259:e289a81c871e
        branch:      3.5
        parent:      98255:6badd794cff1
        parent:      98258:a43d79b2434c
        user:        Terry Jan Reedy 
        date:        Thu Sep 24 23:19:09 2015 -0400
        summary:
          Merge with 3.4
        
        files:
          Lib/idlelib/help.html |  77 +++++++++++-------------------
          1 files changed, 29 insertions(+), 48 deletions(-)
        
        
        diff --git a/Lib/idlelib/help.html b/Lib/idlelib/help.html
        --- a/Lib/idlelib/help.html
        +++ b/Lib/idlelib/help.html
        @@ -439,41 +439,22 @@
         
         
  • +
    +

    25.5.2.4. Text colors?

    +

    Idle defaults to black on white text, but colors text with special meanings. +For the shell, these are shell output, shell error, user output, and +user error. For Python code, at the shell prompt or in an editor, these are +keywords, builtin class and function names, names following class and +def, strings, and comments. For any text window, these are the cursor (when +present), found text (when possible), and selected text.

    +

    Text coloring is done in the background, so uncolorized text is occasionally +visible. To change the color scheme, use the Configure IDLE dialog +Highlighting tab. The marking of debugger breakpoint lines in the editor and +text in popups and dialogs is not user-configurable.

    -
    -

    25.5.3. Syntax colors?

    -

    The coloring is applied in a background “thread,” so you may occasionally see -uncolorized text. To change the color scheme, edit the [Colors] section in -config.txt.

    -
    -
    Python syntax colors:
    -
    -
    Keywords
    -
    orange
    -
    Strings
    -
    green
    -
    Comments
    -
    red
    -
    Definitions
    -
    blue
    -
    -
    -
    Shell colors:
    -
    -
    Console output
    -
    brown
    -
    stdout
    -
    blue
    -
    stderr
    -
    dark green
    -
    stdin
    -
    black
    -
    -
    -
    -

    25.5.4. Startup and code execution?

    +

    25.5.3. Startup and code execution?

    Upon startup with the -s option, IDLE will execute the file referenced by the environment variables IDLESTARTUP or PYTHONSTARTUP. IDLE first checks for IDLESTARTUP; if IDLESTARTUP is present the file @@ -487,7 +468,7 @@ executed in the Tk namespace, so this file is not useful for importing functions to be used from IDLE’s Python shell.

    -

    25.5.4.1. Command line usage?

    +

    25.5.3.1. Command line usage?

    idle.py [-c command] [-d] [-e] [-h] [-i] [-r file] [-s] [-t title] [-] [arg] ...
     
     -c command  run command in the shell window
    @@ -512,7 +493,7 @@
     
     
    -

    25.5.4.2. IDLE-console differences?

    +

    25.5.3.2. IDLE-console differences?

    As much as possible, the result of executing Python code with IDLE is the same as executing the same code in a console window. However, the different interface and operation occasionally affects results.

    @@ -526,7 +507,7 @@ Some consoles only work with a single physical line at a time.

    -

    25.5.4.3. Running without a subprocess?

    +

    25.5.3.3. Running without a subprocess?

    By default, IDLE executes user code in a separate subprocess via a socket, which uses the internal loopback interface. This connection is not externally visible and no data is sent to or received from the Internet. @@ -552,9 +533,9 @@

    -

    25.5.5. Help and preferences?

    +

    25.5.4. Help and preferences?

    -

    25.5.5.1. Additional help sources?

    +

    25.5.4.1. Additional help sources?

    IDLE includes a help menu entry called “Python Docs” that will open the extensive sources of help, including tutorials, available at docs.python.org. Selected URLs can be added or removed from the help menu at any time using the @@ -562,14 +543,14 @@ more information.

    -

    25.5.5.2. Setting preferences?

    +

    25.5.4.2. Setting preferences?

    The font preferences, highlighting, keys, and general preferences can be changed via Configure IDLE on the Option menu. Keys can be user defined; IDLE ships with four built in key sets. In addition a user can create a custom key set in the Configure IDLE dialog under the keys tab.

    -

    25.5.5.3. Extensions?

    +

    25.5.4.3. Extensions?

    IDLE contains an extension facility. Peferences for extensions can be changed with Configure Extensions. See the beginning of config-extensions.def in the idlelib directory for further information. The default extensions @@ -615,19 +596,19 @@

  • 25.5.2.1. Automatic indentation
  • 25.5.2.2. Completions
  • 25.5.2.3. Python Shell window
  • +
  • 25.5.2.4. Text colors
  • -
  • 25.5.3. Syntax colors
  • -
  • 25.5.4. Startup and code execution
  • +
    +

    24.6.2.4. Text colors?

    +

    Idle defaults to black on white text, but colors text with special meanings. +For the shell, these are shell output, shell error, user output, and +user error. For Python code, at the shell prompt or in an editor, these are +keywords, builtin class and function names, names following class and +def, strings, and comments. For any text window, these are the cursor (when +present), found text (when possible), and selected text.

    +

    Text coloring is done in the background, so uncolorized text is occasionally +visible. To change the color scheme, use the Configure IDLE dialog +Highlighting tab. The marking of debugger breakpoint lines in the editor and +text in popups and dialogs is not user-configurable.

    -
    -

    24.6.3. Syntax colors?

    -

    The coloring is applied in a background “thread,” so you may occasionally see -uncolorized text. To change the color scheme, edit the [Colors] section in -config.txt.

    -
    -
    Python syntax colors:
    -
    -
    Keywords
    -
    orange
    -
    Strings
    -
    green
    -
    Comments
    -
    red
    -
    Definitions
    -
    blue
    -
    -
    -
    Shell colors:
    -
    -
    Console output
    -
    brown
    -
    stdout
    -
    blue
    -
    stderr
    -
    dark green
    -
    stdin
    -
    black
    -
    -
    -
    -

    24.6.4. Startup and code execution?

    +

    24.6.3. Startup and code execution?

    Upon startup with the -s option, IDLE will execute the file referenced by the environment variables IDLESTARTUP or PYTHONSTARTUP. IDLE first checks for IDLESTARTUP; if IDLESTARTUP is present the file @@ -487,7 +468,7 @@ executed in the Tk namespace, so this file is not useful for importing functions to be used from IDLE’s Python shell.

    -

    24.6.4.1. Command line usage?

    +

    24.6.3.1. Command line usage?

    idle.py [-c command] [-d] [-e] [-h] [-i] [-r file] [-s] [-t title] [-] [arg] ...
     
     -c command  run command in the shell window
    @@ -512,7 +493,7 @@
     
     
    -

    24.6.4.2. IDLE-console differences?

    +

    24.6.3.2. IDLE-console differences?

    As much as possible, the result of executing Python code with IDLE is the same as executing the same code in a console window. However, the different interface and operation occasionally affects results.

    @@ -526,7 +507,7 @@ Some consoles only work with a single physical line at a time.

    -

    24.6.4.3. Running without a subprocess?

    +

    24.6.3.3. Running without a subprocess?

    By default, IDLE executes user code in a separate subprocess via a socket, which uses the internal loopback interface. This connection is not externally visible and no data is sent to or received from the Internet. @@ -552,9 +533,9 @@

    -

    24.6.5. Help and preferences?

    +

    24.6.4. Help and preferences?

    -

    24.6.5.1. Additional help sources?

    +

    24.6.4.1. Additional help sources?

    IDLE includes a help menu entry called “Python Docs” that will open the extensive sources of help, including tutorials, available at docs.python.org. Selected URLs can be added or removed from the help menu at any time using the @@ -562,14 +543,14 @@ more information.

    -

    24.6.5.2. Setting preferences?

    +

    24.6.4.2. Setting preferences?

    The font preferences, highlighting, keys, and general preferences can be changed via Configure IDLE on the Option menu. Keys can be user defined; IDLE ships with four built in key sets. In addition a user can create a custom key set in the Configure IDLE dialog under the keys tab.

    -

    24.6.5.3. Extensions?

    +

    24.6.4.3. Extensions?

    IDLE contains an extension facility. Peferences for extensions can be changed with Configure Extensions. See the beginning of config-extensions.def in the idlelib directory for further information. The default extensions @@ -615,19 +596,19 @@

  • 24.6.2.1. Automatic indentation
  • 24.6.2.2. Completions
  • 24.6.2.3. Python Shell window
  • +
  • 24.6.2.4. Text colors
  • -
  • 24.6.3. Syntax colors
  • -
  • 24.6.4. Startup and code execution
  • +
    +

    25.5.2.4. Text colors?

    +

    Idle defaults to black on white text, but colors text with special meanings. +For the shell, these are shell output, shell error, user output, and +user error. For Python code, at the shell prompt or in an editor, these are +keywords, builtin class and function names, names following class and +def, strings, and comments. For any text window, these are the cursor (when +present), found text (when possible), and selected text.

    +

    Text coloring is done in the background, so uncolorized text is occasionally +visible. To change the color scheme, use the Configure IDLE dialog +Highlighting tab. The marking of debugger breakpoint lines in the editor and +text in popups and dialogs is not user-configurable.

    -
    -

    25.5.3. Syntax colors?

    -

    The coloring is applied in a background “thread,” so you may occasionally see -uncolorized text. To change the color scheme, edit the [Colors] section in -config.txt.

    -
    -
    Python syntax colors:
    -
    -
    Keywords
    -
    orange
    -
    Strings
    -
    green
    -
    Comments
    -
    red
    -
    Definitions
    -
    blue
    -
    -
    -
    Shell colors:
    -
    -
    Console output
    -
    brown
    -
    stdout
    -
    blue
    -
    stderr
    -
    dark green
    -
    stdin
    -
    black
    -
    -
    -
    -

    25.5.4. Startup and code execution?

    +

    25.5.3. Startup and code execution?

    Upon startup with the -s option, IDLE will execute the file referenced by the environment variables IDLESTARTUP or PYTHONSTARTUP. IDLE first checks for IDLESTARTUP; if IDLESTARTUP is present the file @@ -487,7 +468,7 @@ executed in the Tk namespace, so this file is not useful for importing functions to be used from IDLE’s Python shell.

    -

    25.5.4.1. Command line usage?

    +

    25.5.3.1. Command line usage?

    idle.py [-c command] [-d] [-e] [-h] [-i] [-r file] [-s] [-t title] [-] [arg] ...
     
     -c command  run command in the shell window
    @@ -512,7 +493,7 @@
     
     
    -

    25.5.4.2. IDLE-console differences?

    +

    25.5.3.2. IDLE-console differences?

    As much as possible, the result of executing Python code with IDLE is the same as executing the same code in a console window. However, the different interface and operation occasionally affects results.

    @@ -526,7 +507,7 @@ Some consoles only work with a single physical line at a time.

    -

    25.5.4.3. Running without a subprocess?

    +

    25.5.3.3. Running without a subprocess?

    By default, IDLE executes user code in a separate subprocess via a socket, which uses the internal loopback interface. This connection is not externally visible and no data is sent to or received from the Internet. @@ -552,9 +533,9 @@

    -

    25.5.5. Help and preferences?

    +

    25.5.4. Help and preferences?

    -

    25.5.5.1. Additional help sources?

    +

    25.5.4.1. Additional help sources?

    IDLE includes a help menu entry called “Python Docs” that will open the extensive sources of help, including tutorials, available at docs.python.org. Selected URLs can be added or removed from the help menu at any time using the @@ -562,14 +543,14 @@ more information.

    -

    25.5.5.2. Setting preferences?

    +

    25.5.4.2. Setting preferences?

    The font preferences, highlighting, keys, and general preferences can be changed via Configure IDLE on the Option menu. Keys can be user defined; IDLE ships with four built in key sets. In addition a user can create a custom key set in the Configure IDLE dialog under the keys tab.

    -

    25.5.5.3. Extensions?

    +

    25.5.4.3. Extensions?

    IDLE contains an extension facility. Peferences for extensions can be changed with Configure Extensions. See the beginning of config-extensions.def in the idlelib directory for further information. The default extensions @@ -615,19 +596,19 @@

  • 25.5.2.1. Automatic indentation
  • 25.5.2.2. Completions
  • 25.5.2.3. Python Shell window
  • +
  • 25.5.2.4. Text colors
  • -
  • 25.5.3. Syntax colors
  • -
  • 25.5.4. Startup and code execution
  • +
    +

    25.5.2.4. Text colors?

    +

    Idle defaults to black on white text, but colors text with special meanings. +For the shell, these are shell output, shell error, user output, and +user error. For Python code, at the shell prompt or in an editor, these are +keywords, builtin class and function names, names following class and +def, strings, and comments. For any text window, these are the cursor (when +present), found text (when possible), and selected text.

    +

    Text coloring is done in the background, so uncolorized text is occasionally +visible. To change the color scheme, use the Configure IDLE dialog +Highlighting tab. The marking of debugger breakpoint lines in the editor and +text in popups and dialogs is not user-configurable.

    -
    -

    25.5.3. Syntax colors?

    -

    The coloring is applied in a background “thread,” so you may occasionally see -uncolorized text. To change the color scheme, edit the [Colors] section in -config.txt.

    -
    -
    Python syntax colors:
    -
    -
    Keywords
    -
    orange
    -
    Strings
    -
    green
    -
    Comments
    -
    red
    -
    Definitions
    -
    blue
    -
    -
    -
    Shell colors:
    -
    -
    Console output
    -
    brown
    -
    stdout
    -
    blue
    -
    stderr
    -
    dark green
    -
    stdin
    -
    black
    -
    -
    -
    -

    25.5.4. Startup and code execution?

    +

    25.5.3. Startup and code execution?

    Upon startup with the -s option, IDLE will execute the file referenced by the environment variables IDLESTARTUP or PYTHONSTARTUP. IDLE first checks for IDLESTARTUP; if IDLESTARTUP is present the file @@ -487,7 +468,7 @@ executed in the Tk namespace, so this file is not useful for importing functions to be used from IDLE’s Python shell.

    -

    25.5.4.1. Command line usage?

    +

    25.5.3.1. Command line usage?

    idle.py [-c command] [-d] [-e] [-h] [-i] [-r file] [-s] [-t title] [-] [arg] ...
     
     -c command  run command in the shell window
    @@ -512,7 +493,7 @@
     
     
    -

    25.5.4.2. IDLE-console differences?

    +

    25.5.3.2. IDLE-console differences?

    As much as possible, the result of executing Python code with IDLE is the same as executing the same code in a console window. However, the different interface and operation occasionally affects results.

    @@ -526,7 +507,7 @@ Some consoles only work with a single physical line at a time.

    -

    25.5.4.3. Running without a subprocess?

    +

    25.5.3.3. Running without a subprocess?

    By default, IDLE executes user code in a separate subprocess via a socket, which uses the internal loopback interface. This connection is not externally visible and no data is sent to or received from the Internet. @@ -552,9 +533,9 @@

    -

    25.5.5. Help and preferences?

    +

    25.5.4. Help and preferences?

    -

    25.5.5.1. Additional help sources?

    +

    25.5.4.1. Additional help sources?

    IDLE includes a help menu entry called “Python Docs” that will open the extensive sources of help, including tutorials, available at docs.python.org. Selected URLs can be added or removed from the help menu at any time using the @@ -562,14 +543,14 @@ more information.

    -

    25.5.5.2. Setting preferences?

    +

    25.5.4.2. Setting preferences?

    The font preferences, highlighting, keys, and general preferences can be changed via Configure IDLE on the Option menu. Keys can be user defined; IDLE ships with four built in key sets. In addition a user can create a custom key set in the Configure IDLE dialog under the keys tab.

    -

    25.5.5.3. Extensions?

    +

    25.5.4.3. Extensions?

    IDLE contains an extension facility. Peferences for extensions can be changed with Configure Extensions. See the beginning of config-extensions.def in the idlelib directory for further information. The default extensions @@ -615,19 +596,19 @@

  • 25.5.2.1. Automatic indentation
  • 25.5.2.2. Completions
  • 25.5.2.3. Python Shell window
  • +
  • 25.5.2.4. Text colors
  • -
  • 25.5.3. Syntax colors
  • -
  • 25.5.4. Startup and code execution
      -
    • 25.5.4.1. Command line usage
    • -
    • 25.5.4.2. IDLE-console differences
    • -
    • 25.5.4.3. Running without a subprocess
    • +
    • 25.5.3. Startup and code execution
    • -
    • 25.5.5. Help and preferences -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 25 06:50:04 2015 From: python-checkins at python.org (terry.reedy) Date: Fri, 25 Sep 2015 04:50:04 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge_with_3=2E5?= Message-ID: <20150925045004.16573.27959@psf.io> https://hg.python.org/cpython/rev/089146b8ccc6 changeset: 98264:089146b8ccc6 parent: 98260:70cdc814dfef parent: 98263:37195cb81b8f user: Terry Jan Reedy date: Fri Sep 25 00:49:44 2015 -0400 summary: Merge with 3.5 files: Lib/idlelib/help.py | 6 +++++- 1 files changed, 5 insertions(+), 1 deletions(-) diff --git a/Lib/idlelib/help.py b/Lib/idlelib/help.py --- a/Lib/idlelib/help.py +++ b/Lib/idlelib/help.py @@ -28,6 +28,7 @@ from os.path import abspath, dirname, isdir, isfile, join from tkinter import Tk, Toplevel, Frame, Text, Scrollbar, Menu, Menubutton from tkinter import font as tkfont +from idlelib.configHandler import idleConf use_ttk = False # until available to import if use_ttk: @@ -158,8 +159,11 @@ "Display help.html." def __init__(self, parent, filename): "Configure tags and feed file to parser." + uwide = idleConf.GetOption('main', 'EditorWindow', 'width', type='int') + uhigh = idleConf.GetOption('main', 'EditorWindow', 'height', type='int') + uhigh = 3 * uhigh // 4 # lines average 4/3 of editor line height Text.__init__(self, parent, wrap='word', highlightthickness=0, - padx=5, borderwidth=0) + padx=5, borderwidth=0, width=uwide, height=uhigh) normalfont = self.findfont(['TkDefaultFont', 'arial', 'helvetica']) fixedfont = self.findfont(['TkFixedFont', 'monaco', 'courier']) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 25 06:50:05 2015 From: python-checkins at python.org (terry.reedy) Date: Fri, 25 Sep 2015 04:50:05 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Merge_with_3=2E4?= Message-ID: <20150925045004.81637.75402@psf.io> https://hg.python.org/cpython/rev/37195cb81b8f changeset: 98263:37195cb81b8f branch: 3.5 parent: 98259:e289a81c871e parent: 98262:1c119da20663 user: Terry Jan Reedy date: Fri Sep 25 00:49:31 2015 -0400 summary: Merge with 3.4 files: Lib/idlelib/help.py | 6 +++++- 1 files changed, 5 insertions(+), 1 deletions(-) diff --git a/Lib/idlelib/help.py b/Lib/idlelib/help.py --- a/Lib/idlelib/help.py +++ b/Lib/idlelib/help.py @@ -28,6 +28,7 @@ from os.path import abspath, dirname, isdir, isfile, join from tkinter import Tk, Toplevel, Frame, Text, Scrollbar, Menu, Menubutton from tkinter import font as tkfont +from idlelib.configHandler import idleConf use_ttk = False # until available to import if use_ttk: @@ -158,8 +159,11 @@ "Display help.html." def __init__(self, parent, filename): "Configure tags and feed file to parser." + uwide = idleConf.GetOption('main', 'EditorWindow', 'width', type='int') + uhigh = idleConf.GetOption('main', 'EditorWindow', 'height', type='int') + uhigh = 3 * uhigh // 4 # lines average 4/3 of editor line height Text.__init__(self, parent, wrap='word', highlightthickness=0, - padx=5, borderwidth=0) + padx=5, borderwidth=0, width=uwide, height=uhigh) normalfont = self.findfont(['TkDefaultFont', 'arial', 'helvetica']) fixedfont = self.findfont(['TkFixedFont', 'monaco', 'courier']) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 25 06:50:05 2015 From: python-checkins at python.org (terry.reedy) Date: Fri, 25 Sep 2015 04:50:05 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzI1MTk4?= =?utf-8?q?=3A_Idle_doc_viewer_now_uses_user_width_and_height_setting=2E?= Message-ID: <20150925045004.115214.82114@psf.io> https://hg.python.org/cpython/rev/c1eccae07977 changeset: 98261:c1eccae07977 branch: 2.7 parent: 98257:36095aa6dd5e user: Terry Jan Reedy date: Fri Sep 25 00:49:02 2015 -0400 summary: Issue #25198: Idle doc viewer now uses user width and height setting. The height is reduced by 3/4 to account for extra spacing between lines, relative to an Idle editor, and extra tall header lines. files: Lib/idlelib/help.py | 6 +++++- 1 files changed, 5 insertions(+), 1 deletions(-) diff --git a/Lib/idlelib/help.py b/Lib/idlelib/help.py --- a/Lib/idlelib/help.py +++ b/Lib/idlelib/help.py @@ -28,6 +28,7 @@ from os.path import abspath, dirname, isdir, isfile, join from Tkinter import Tk, Toplevel, Frame, Text, Scrollbar, Menu, Menubutton import tkFont as tkfont +from idlelib.configHandler import idleConf use_ttk = False # until available to import if use_ttk: @@ -161,8 +162,11 @@ "Display help.html." def __init__(self, parent, filename): "Configure tags and feed file to parser." + uwide = idleConf.GetOption('main', 'EditorWindow', 'width', type='int') + uhigh = idleConf.GetOption('main', 'EditorWindow', 'height', type='int') + uhigh = 3 * uhigh // 4 # lines average 4/3 of editor line height Text.__init__(self, parent, wrap='word', highlightthickness=0, - padx=5, borderwidth=0) + padx=5, borderwidth=0, width=uwide, height=uhigh) normalfont = self.findfont(['TkDefaultFont', 'arial', 'helvetica']) fixedfont = self.findfont(['TkFixedFont', 'monaco', 'courier']) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 25 06:50:05 2015 From: python-checkins at python.org (terry.reedy) Date: Fri, 25 Sep 2015 04:50:05 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzI1MTk4?= =?utf-8?q?=3A_Idle_doc_viewer_now_uses_user_width_and_height_setting=2E?= Message-ID: <20150925045004.3654.22832@psf.io> https://hg.python.org/cpython/rev/1c119da20663 changeset: 98262:1c119da20663 branch: 3.4 parent: 98258:a43d79b2434c user: Terry Jan Reedy date: Fri Sep 25 00:49:18 2015 -0400 summary: Issue #25198: Idle doc viewer now uses user width and height setting. The height is reduced by 3/4 to account for extra spacing between lines, relative to an Idle editor, and extra tall header lines. files: Lib/idlelib/help.py | 6 +++++- 1 files changed, 5 insertions(+), 1 deletions(-) diff --git a/Lib/idlelib/help.py b/Lib/idlelib/help.py --- a/Lib/idlelib/help.py +++ b/Lib/idlelib/help.py @@ -28,6 +28,7 @@ from os.path import abspath, dirname, isdir, isfile, join from tkinter import Tk, Toplevel, Frame, Text, Scrollbar, Menu, Menubutton from tkinter import font as tkfont +from idlelib.configHandler import idleConf use_ttk = False # until available to import if use_ttk: @@ -158,8 +159,11 @@ "Display help.html." def __init__(self, parent, filename): "Configure tags and feed file to parser." + uwide = idleConf.GetOption('main', 'EditorWindow', 'width', type='int') + uhigh = idleConf.GetOption('main', 'EditorWindow', 'height', type='int') + uhigh = 3 * uhigh // 4 # lines average 4/3 of editor line height Text.__init__(self, parent, wrap='word', highlightthickness=0, - padx=5, borderwidth=0) + padx=5, borderwidth=0, width=uwide, height=uhigh) normalfont = self.findfont(['TkDefaultFont', 'arial', 'helvetica']) fixedfont = self.findfont(['TkFixedFont', 'monaco', 'courier']) -- Repository URL: https://hg.python.org/cpython From solipsis at pitrou.net Fri Sep 25 10:48:01 2015 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Fri, 25 Sep 2015 08:48:01 +0000 Subject: [Python-checkins] Daily reference leaks (089146b8ccc6): sum=17869 Message-ID: <20150925084801.115399.91759@psf.io> results for 089146b8ccc6 on branch "default" -------------------------------------------- test_capi leaked [1598, 1598, 1598] references, sum=4794 test_capi leaked [387, 389, 389] memory blocks, sum=1165 test_collections leaked [-6, 0, 0] references, sum=-6 test_collections leaked [-3, 1, 0] memory blocks, sum=-2 test_functools leaked [0, 2, 2] memory blocks, sum=4 test_threading leaked [3196, 3196, 3196] references, sum=9588 test_threading leaked [774, 776, 776] memory blocks, sum=2326 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/psf-users/antoine/refleaks/reflog62myMf', '--timeout', '7200'] From lp_benchmark_robot at intel.com Fri Sep 25 12:43:25 2015 From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com) Date: Fri, 25 Sep 2015 11:43:25 +0100 Subject: [Python-checkins] Benchmark Results for Python 2.7 2015-09-25 Message-ID: <4dc71a6d-01b1-4ca7-8e74-9ca0f41b53ba@irsmsx106.ger.corp.intel.com> Results for project python_2.7-nightly, build date 2015-09-24 03:42:58 commit: e2f1f69d0618f70a72b9dc0d6cdbce162d72f11e revision date: 2015-09-24 00:19:42 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v2.7.10, with hash 15c95b7d81dcf821daade360741e00714667653f from 2015-05-23 16:02:14+00:00 ------------------------------------------------------------------------------------------ benchmark relative change since change since current rev with std_dev* last run v2.7.10 regrtest PGO ------------------------------------------------------------------------------------------ :-) django_v2 0.28025% -0.55377% 4.00355% 8.72489% :-) pybench 0.14593% 0.02415% 6.79149% 6.91599% :-| regex_v8 0.61847% 0.02126% -1.14722% 7.78544% :-) nbody 0.11601% 0.01084% 9.05576% 4.12003% :-) json_dump_v2 0.25821% 0.37153% 4.43383% 13.09859% :-| normal_startup 1.59986% 0.54183% -1.10534% 2.32355% :-) ssbench 0.45407% 0.64119% 2.53659% 1.46234% ------------------------------------------------------------------------------------------ Note: Benchmark results for ssbench are measured in requests/second while all other are measured in seconds. * Relative Standard Deviation (Standard Deviation/Average) Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. From lp_benchmark_robot at intel.com Fri Sep 25 12:43:07 2015 From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com) Date: Fri, 25 Sep 2015 11:43:07 +0100 Subject: [Python-checkins] Benchmark Results for Python Default 2015-09-25 Message-ID: Results for project python_default-nightly, build date 2015-09-25 03:02:16 commit: 7efb00bace60bf371cbf4151b97a4b6a132ac296 revision date: 2015-09-24 21:32:50 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v3.4.3, with hash b4cbecbc0781e89a309d03b60a1f75f8499250e6 from 2015-02-25 12:15:33+00:00 ------------------------------------------------------------------------------------------ benchmark relative change since change since current rev with std_dev* last run v3.4.3 regrtest PGO ------------------------------------------------------------------------------------------ :-) django_v2 0.19679% 1.67361% 9.11093% 15.69373% :-( pybench 0.09581% -0.00372% -2.32983% 8.50074% :-( regex_v8 2.84956% -0.18618% -5.60150% 6.73315% :-| nbody 0.10570% -0.02883% 0.30821% 9.26033% :-( json_dump_v2 0.26982% 0.65211% -3.05002% 10.47395% :-| normal_startup 0.65996% -0.07052% -0.00701% 5.35614% ------------------------------------------------------------------------------------------ Note: Benchmark results are measured in seconds. * Relative Standard Deviation (Standard Deviation/Average) Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. From python-checkins at python.org Fri Sep 25 22:05:18 2015 From: python-checkins at python.org (brett.cannon) Date: Fri, 25 Sep 2015 20:05:18 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2325186=3A_Remove_d?= =?utf-8?q?uplicated_function_from_importlib=2E=5Fbootstrap=5Fexternal?= Message-ID: <20150925200518.3666.99886@psf.io> https://hg.python.org/cpython/rev/1266e98fd04c changeset: 98265:1266e98fd04c user: Brett Cannon date: Fri Sep 25 13:05:13 2015 -0700 summary: Issue #25186: Remove duplicated function from importlib._bootstrap_external files: Lib/importlib/_bootstrap_external.py | 48 +- Python/importlib_external.h | 4983 ++++++------- 2 files changed, 2505 insertions(+), 2526 deletions(-) diff --git a/Lib/importlib/_bootstrap_external.py b/Lib/importlib/_bootstrap_external.py --- a/Lib/importlib/_bootstrap_external.py +++ b/Lib/importlib/_bootstrap_external.py @@ -360,14 +360,6 @@ return mode -def _verbose_message(message, *args, verbosity=1): - """Print the message to stderr if -v/PYTHONVERBOSE is turned on.""" - if sys.flags.verbose >= verbosity: - if not message.startswith(('#', 'import ')): - message = '# ' + message - print(message.format(*args), file=sys.stderr) - - def _check_name(method): """Decorator to verify that the module being requested matches the one the loader can handle. @@ -437,15 +429,15 @@ raw_size = data[8:12] if magic != MAGIC_NUMBER: message = 'bad magic number in {!r}: {!r}'.format(name, magic) - _verbose_message(message) + _bootstrap._verbose_message(message) raise ImportError(message, **exc_details) elif len(raw_timestamp) != 4: message = 'reached EOF while reading timestamp in {!r}'.format(name) - _verbose_message(message) + _bootstrap._verbose_message(message) raise EOFError(message) elif len(raw_size) != 4: message = 'reached EOF while reading size of source in {!r}'.format(name) - _verbose_message(message) + _bootstrap._verbose_message(message) raise EOFError(message) if source_stats is not None: try: @@ -455,7 +447,7 @@ else: if _r_long(raw_timestamp) != source_mtime: message = 'bytecode is stale for {!r}'.format(name) - _verbose_message(message) + _bootstrap._verbose_message(message) raise ImportError(message, **exc_details) try: source_size = source_stats['size'] & 0xFFFFFFFF @@ -472,7 +464,7 @@ """Compile bytecode as returned by _validate_bytecode_header().""" code = marshal.loads(data) if isinstance(code, _code_type): - _verbose_message('code object from {!r}', bytecode_path) + _bootstrap._verbose_message('code object from {!r}', bytecode_path) if source_path is not None: _imp._fix_co_filename(code, source_path) return code @@ -755,21 +747,21 @@ except (ImportError, EOFError): pass else: - _verbose_message('{} matches {}', bytecode_path, - source_path) + _bootstrap._verbose_message('{} matches {}', bytecode_path, + source_path) return _compile_bytecode(bytes_data, name=fullname, bytecode_path=bytecode_path, source_path=source_path) source_bytes = self.get_data(source_path) code_object = self.source_to_code(source_bytes, source_path) - _verbose_message('code object from {}', source_path) + _bootstrap._verbose_message('code object from {}', source_path) if (not sys.dont_write_bytecode and bytecode_path is not None and source_mtime is not None): data = _code_to_bytecode(code_object, source_mtime, len(source_bytes)) try: self._cache_bytecode(source_path, bytecode_path, data) - _verbose_message('wrote {!r}', bytecode_path) + _bootstrap._verbose_message('wrote {!r}', bytecode_path) except NotImplementedError: pass return code_object @@ -849,14 +841,16 @@ except OSError as exc: # Could be a permission error, read-only filesystem: just forget # about writing the data. - _verbose_message('could not create {!r}: {!r}', parent, exc) + _bootstrap._verbose_message('could not create {!r}: {!r}', + parent, exc) return try: _write_atomic(path, data, _mode) - _verbose_message('created {!r}', path) + _bootstrap._verbose_message('created {!r}', path) except OSError as exc: # Same as above: just don't write the bytecode. - _verbose_message('could not create {!r}: {!r}', path, exc) + _bootstrap._verbose_message('could not create {!r}: {!r}', path, + exc) class SourcelessFileLoader(FileLoader, _LoaderBasics): @@ -901,14 +895,14 @@ """Create an unitialized extension module""" module = _bootstrap._call_with_frames_removed( _imp.create_dynamic, spec) - _verbose_message('extension module {!r} loaded from {!r}', + _bootstrap._verbose_message('extension module {!r} loaded from {!r}', spec.name, self.path) return module def exec_module(self, module): """Initialize an extension module""" _bootstrap._call_with_frames_removed(_imp.exec_dynamic, module) - _verbose_message('extension module {!r} executed from {!r}', + _bootstrap._verbose_message('extension module {!r} executed from {!r}', self.name, self.path) def is_package(self, fullname): @@ -1023,7 +1017,8 @@ """ # The import system never calls this method. - _verbose_message('namespace module loaded with path {!r}', self._path) + _bootstrap._verbose_message('namespace module loaded with path {!r}', + self._path) return _bootstrap._load_module_shim(self, fullname) @@ -1243,12 +1238,13 @@ # Check for a file w/ a proper suffix exists. for suffix, loader_class in self._loaders: full_path = _path_join(self.path, tail_module + suffix) - _verbose_message('trying {}'.format(full_path), verbosity=2) + _bootstrap._verbose_message('trying {}', full_path, verbosity=2) if cache_module + suffix in cache: if _path_isfile(full_path): - return self._get_spec(loader_class, fullname, full_path, None, target) + return self._get_spec(loader_class, fullname, full_path, + None, target) if is_namespace: - _verbose_message('possible namespace for {}'.format(base_path)) + _bootstrap._verbose_message('possible namespace for {}', base_path) spec = _bootstrap.ModuleSpec(fullname, None) spec.submodule_search_locations = [base_path] return spec diff --git a/Python/importlib_external.h b/Python/importlib_external.h --- a/Python/importlib_external.h +++ b/Python/importlib_external.h @@ -1,8 +1,8 @@ /* Auto-generated by Programs/_freeze_importlib.c */ const unsigned char _Py_M__importlib_external[] = { 99,0,0,0,0,0,0,0,0,0,0,0,0,7,0,0, - 0,64,0,0,0,115,228,2,0,0,100,0,0,90,0,0, - 100,96,0,90,1,0,100,4,0,100,5,0,132,0,0,90, + 0,64,0,0,0,115,210,2,0,0,100,0,0,90,0,0, + 100,92,0,90,1,0,100,4,0,100,5,0,132,0,0,90, 2,0,100,6,0,100,7,0,132,0,0,90,3,0,100,8, 0,100,9,0,132,0,0,90,4,0,100,10,0,100,11,0, 132,0,0,90,5,0,100,12,0,100,13,0,132,0,0,90, @@ -20,563 +20,541 @@ 1,1,90,26,0,100,37,0,100,38,0,132,0,0,90,27, 0,100,39,0,100,40,0,132,0,0,90,28,0,100,41,0, 100,42,0,132,0,0,90,29,0,100,43,0,100,44,0,132, - 0,0,90,30,0,100,45,0,100,46,0,100,47,0,100,48, - 0,132,0,1,90,31,0,100,49,0,100,50,0,132,0,0, - 90,32,0,100,51,0,100,52,0,132,0,0,90,33,0,100, - 33,0,100,33,0,100,33,0,100,53,0,100,54,0,132,3, - 0,90,34,0,100,33,0,100,33,0,100,33,0,100,55,0, - 100,56,0,132,3,0,90,35,0,100,57,0,100,57,0,100, - 58,0,100,59,0,132,2,0,90,36,0,100,60,0,100,61, - 0,132,0,0,90,37,0,101,38,0,131,0,0,90,39,0, - 100,33,0,100,62,0,100,33,0,100,63,0,101,39,0,100, - 64,0,100,65,0,132,1,2,90,40,0,71,100,66,0,100, - 67,0,132,0,0,100,67,0,131,2,0,90,41,0,71,100, - 68,0,100,69,0,132,0,0,100,69,0,131,2,0,90,42, - 0,71,100,70,0,100,71,0,132,0,0,100,71,0,101,42, - 0,131,3,0,90,43,0,71,100,72,0,100,73,0,132,0, - 0,100,73,0,131,2,0,90,44,0,71,100,74,0,100,75, - 0,132,0,0,100,75,0,101,44,0,101,43,0,131,4,0, - 90,45,0,71,100,76,0,100,77,0,132,0,0,100,77,0, - 101,44,0,101,42,0,131,4,0,90,46,0,103,0,0,90, - 47,0,71,100,78,0,100,79,0,132,0,0,100,79,0,101, - 44,0,101,42,0,131,4,0,90,48,0,71,100,80,0,100, - 81,0,132,0,0,100,81,0,131,2,0,90,49,0,71,100, - 82,0,100,83,0,132,0,0,100,83,0,131,2,0,90,50, - 0,71,100,84,0,100,85,0,132,0,0,100,85,0,131,2, - 0,90,51,0,71,100,86,0,100,87,0,132,0,0,100,87, - 0,131,2,0,90,52,0,100,33,0,100,88,0,100,89,0, - 132,1,0,90,53,0,100,90,0,100,91,0,132,0,0,90, - 54,0,100,92,0,100,93,0,132,0,0,90,55,0,100,94, - 0,100,95,0,132,0,0,90,56,0,100,33,0,83,41,97, - 97,94,1,0,0,67,111,114,101,32,105,109,112,108,101,109, - 101,110,116,97,116,105,111,110,32,111,102,32,112,97,116,104, - 45,98,97,115,101,100,32,105,109,112,111,114,116,46,10,10, - 84,104,105,115,32,109,111,100,117,108,101,32,105,115,32,78, - 79,84,32,109,101,97,110,116,32,116,111,32,98,101,32,100, - 105,114,101,99,116,108,121,32,105,109,112,111,114,116,101,100, - 33,32,73,116,32,104,97,115,32,98,101,101,110,32,100,101, - 115,105,103,110,101,100,32,115,117,99,104,10,116,104,97,116, - 32,105,116,32,99,97,110,32,98,101,32,98,111,111,116,115, - 116,114,97,112,112,101,100,32,105,110,116,111,32,80,121,116, - 104,111,110,32,97,115,32,116,104,101,32,105,109,112,108,101, - 109,101,110,116,97,116,105,111,110,32,111,102,32,105,109,112, - 111,114,116,46,32,65,115,10,115,117,99,104,32,105,116,32, - 114,101,113,117,105,114,101,115,32,116,104,101,32,105,110,106, - 101,99,116,105,111,110,32,111,102,32,115,112,101,99,105,102, - 105,99,32,109,111,100,117,108,101,115,32,97,110,100,32,97, - 116,116,114,105,98,117,116,101,115,32,105,110,32,111,114,100, - 101,114,32,116,111,10,119,111,114,107,46,32,79,110,101,32, - 115,104,111,117,108,100,32,117,115,101,32,105,109,112,111,114, - 116,108,105,98,32,97,115,32,116,104,101,32,112,117,98,108, - 105,99,45,102,97,99,105,110,103,32,118,101,114,115,105,111, - 110,32,111,102,32,116,104,105,115,32,109,111,100,117,108,101, - 46,10,10,218,3,119,105,110,218,6,99,121,103,119,105,110, - 218,6,100,97,114,119,105,110,99,0,0,0,0,0,0,0, - 0,1,0,0,0,2,0,0,0,67,0,0,0,115,49,0, - 0,0,116,0,0,106,1,0,106,2,0,116,3,0,131,1, - 0,114,33,0,100,1,0,100,2,0,132,0,0,125,0,0, - 110,12,0,100,3,0,100,2,0,132,0,0,125,0,0,124, - 0,0,83,41,4,78,99,0,0,0,0,0,0,0,0,0, - 0,0,0,2,0,0,0,83,0,0,0,115,13,0,0,0, - 100,1,0,116,0,0,106,1,0,107,6,0,83,41,2,122, - 53,84,114,117,101,32,105,102,32,102,105,108,101,110,97,109, - 101,115,32,109,117,115,116,32,98,101,32,99,104,101,99,107, - 101,100,32,99,97,115,101,45,105,110,115,101,110,115,105,116, - 105,118,101,108,121,46,115,12,0,0,0,80,89,84,72,79, - 78,67,65,83,69,79,75,41,2,218,3,95,111,115,90,7, - 101,110,118,105,114,111,110,169,0,114,4,0,0,0,114,4, - 0,0,0,250,38,60,102,114,111,122,101,110,32,105,109,112, - 111,114,116,108,105,98,46,95,98,111,111,116,115,116,114,97, - 112,95,101,120,116,101,114,110,97,108,62,218,11,95,114,101, - 108,97,120,95,99,97,115,101,30,0,0,0,115,2,0,0, - 0,0,2,122,37,95,109,97,107,101,95,114,101,108,97,120, - 95,99,97,115,101,46,60,108,111,99,97,108,115,62,46,95, - 114,101,108,97,120,95,99,97,115,101,99,0,0,0,0,0, - 0,0,0,0,0,0,0,1,0,0,0,83,0,0,0,115, - 4,0,0,0,100,1,0,83,41,2,122,53,84,114,117,101, - 32,105,102,32,102,105,108,101,110,97,109,101,115,32,109,117, - 115,116,32,98,101,32,99,104,101,99,107,101,100,32,99,97, - 115,101,45,105,110,115,101,110,115,105,116,105,118,101,108,121, - 46,70,114,4,0,0,0,114,4,0,0,0,114,4,0,0, - 0,114,4,0,0,0,114,5,0,0,0,114,6,0,0,0, - 34,0,0,0,115,2,0,0,0,0,2,41,4,218,3,115, - 121,115,218,8,112,108,97,116,102,111,114,109,218,10,115,116, - 97,114,116,115,119,105,116,104,218,27,95,67,65,83,69,95, - 73,78,83,69,78,83,73,84,73,86,69,95,80,76,65,84, - 70,79,82,77,83,41,1,114,6,0,0,0,114,4,0,0, - 0,114,4,0,0,0,114,5,0,0,0,218,16,95,109,97, - 107,101,95,114,101,108,97,120,95,99,97,115,101,28,0,0, - 0,115,8,0,0,0,0,1,18,1,15,4,12,3,114,11, + 0,0,90,30,0,100,45,0,100,46,0,132,0,0,90,31, + 0,100,47,0,100,48,0,132,0,0,90,32,0,100,33,0, + 100,33,0,100,33,0,100,49,0,100,50,0,132,3,0,90, + 33,0,100,33,0,100,33,0,100,33,0,100,51,0,100,52, + 0,132,3,0,90,34,0,100,53,0,100,53,0,100,54,0, + 100,55,0,132,2,0,90,35,0,100,56,0,100,57,0,132, + 0,0,90,36,0,101,37,0,131,0,0,90,38,0,100,33, + 0,100,58,0,100,33,0,100,59,0,101,38,0,100,60,0, + 100,61,0,132,1,2,90,39,0,71,100,62,0,100,63,0, + 132,0,0,100,63,0,131,2,0,90,40,0,71,100,64,0, + 100,65,0,132,0,0,100,65,0,131,2,0,90,41,0,71, + 100,66,0,100,67,0,132,0,0,100,67,0,101,41,0,131, + 3,0,90,42,0,71,100,68,0,100,69,0,132,0,0,100, + 69,0,131,2,0,90,43,0,71,100,70,0,100,71,0,132, + 0,0,100,71,0,101,43,0,101,42,0,131,4,0,90,44, + 0,71,100,72,0,100,73,0,132,0,0,100,73,0,101,43, + 0,101,41,0,131,4,0,90,45,0,103,0,0,90,46,0, + 71,100,74,0,100,75,0,132,0,0,100,75,0,101,43,0, + 101,41,0,131,4,0,90,47,0,71,100,76,0,100,77,0, + 132,0,0,100,77,0,131,2,0,90,48,0,71,100,78,0, + 100,79,0,132,0,0,100,79,0,131,2,0,90,49,0,71, + 100,80,0,100,81,0,132,0,0,100,81,0,131,2,0,90, + 50,0,71,100,82,0,100,83,0,132,0,0,100,83,0,131, + 2,0,90,51,0,100,33,0,100,84,0,100,85,0,132,1, + 0,90,52,0,100,86,0,100,87,0,132,0,0,90,53,0, + 100,88,0,100,89,0,132,0,0,90,54,0,100,90,0,100, + 91,0,132,0,0,90,55,0,100,33,0,83,41,93,97,94, + 1,0,0,67,111,114,101,32,105,109,112,108,101,109,101,110, + 116,97,116,105,111,110,32,111,102,32,112,97,116,104,45,98, + 97,115,101,100,32,105,109,112,111,114,116,46,10,10,84,104, + 105,115,32,109,111,100,117,108,101,32,105,115,32,78,79,84, + 32,109,101,97,110,116,32,116,111,32,98,101,32,100,105,114, + 101,99,116,108,121,32,105,109,112,111,114,116,101,100,33,32, + 73,116,32,104,97,115,32,98,101,101,110,32,100,101,115,105, + 103,110,101,100,32,115,117,99,104,10,116,104,97,116,32,105, + 116,32,99,97,110,32,98,101,32,98,111,111,116,115,116,114, + 97,112,112,101,100,32,105,110,116,111,32,80,121,116,104,111, + 110,32,97,115,32,116,104,101,32,105,109,112,108,101,109,101, + 110,116,97,116,105,111,110,32,111,102,32,105,109,112,111,114, + 116,46,32,65,115,10,115,117,99,104,32,105,116,32,114,101, + 113,117,105,114,101,115,32,116,104,101,32,105,110,106,101,99, + 116,105,111,110,32,111,102,32,115,112,101,99,105,102,105,99, + 32,109,111,100,117,108,101,115,32,97,110,100,32,97,116,116, + 114,105,98,117,116,101,115,32,105,110,32,111,114,100,101,114, + 32,116,111,10,119,111,114,107,46,32,79,110,101,32,115,104, + 111,117,108,100,32,117,115,101,32,105,109,112,111,114,116,108, + 105,98,32,97,115,32,116,104,101,32,112,117,98,108,105,99, + 45,102,97,99,105,110,103,32,118,101,114,115,105,111,110,32, + 111,102,32,116,104,105,115,32,109,111,100,117,108,101,46,10, + 10,218,3,119,105,110,218,6,99,121,103,119,105,110,218,6, + 100,97,114,119,105,110,99,0,0,0,0,0,0,0,0,1, + 0,0,0,2,0,0,0,67,0,0,0,115,49,0,0,0, + 116,0,0,106,1,0,106,2,0,116,3,0,131,1,0,114, + 33,0,100,1,0,100,2,0,132,0,0,125,0,0,110,12, + 0,100,3,0,100,2,0,132,0,0,125,0,0,124,0,0, + 83,41,4,78,99,0,0,0,0,0,0,0,0,0,0,0, + 0,2,0,0,0,83,0,0,0,115,13,0,0,0,100,1, + 0,116,0,0,106,1,0,107,6,0,83,41,2,122,53,84, + 114,117,101,32,105,102,32,102,105,108,101,110,97,109,101,115, + 32,109,117,115,116,32,98,101,32,99,104,101,99,107,101,100, + 32,99,97,115,101,45,105,110,115,101,110,115,105,116,105,118, + 101,108,121,46,115,12,0,0,0,80,89,84,72,79,78,67, + 65,83,69,79,75,41,2,218,3,95,111,115,90,7,101,110, + 118,105,114,111,110,169,0,114,4,0,0,0,114,4,0,0, + 0,250,38,60,102,114,111,122,101,110,32,105,109,112,111,114, + 116,108,105,98,46,95,98,111,111,116,115,116,114,97,112,95, + 101,120,116,101,114,110,97,108,62,218,11,95,114,101,108,97, + 120,95,99,97,115,101,30,0,0,0,115,2,0,0,0,0, + 2,122,37,95,109,97,107,101,95,114,101,108,97,120,95,99, + 97,115,101,46,60,108,111,99,97,108,115,62,46,95,114,101, + 108,97,120,95,99,97,115,101,99,0,0,0,0,0,0,0, + 0,0,0,0,0,1,0,0,0,83,0,0,0,115,4,0, + 0,0,100,1,0,83,41,2,122,53,84,114,117,101,32,105, + 102,32,102,105,108,101,110,97,109,101,115,32,109,117,115,116, + 32,98,101,32,99,104,101,99,107,101,100,32,99,97,115,101, + 45,105,110,115,101,110,115,105,116,105,118,101,108,121,46,70, + 114,4,0,0,0,114,4,0,0,0,114,4,0,0,0,114, + 4,0,0,0,114,5,0,0,0,114,6,0,0,0,34,0, + 0,0,115,2,0,0,0,0,2,41,4,218,3,115,121,115, + 218,8,112,108,97,116,102,111,114,109,218,10,115,116,97,114, + 116,115,119,105,116,104,218,27,95,67,65,83,69,95,73,78, + 83,69,78,83,73,84,73,86,69,95,80,76,65,84,70,79, + 82,77,83,41,1,114,6,0,0,0,114,4,0,0,0,114, + 4,0,0,0,114,5,0,0,0,218,16,95,109,97,107,101, + 95,114,101,108,97,120,95,99,97,115,101,28,0,0,0,115, + 8,0,0,0,0,1,18,1,15,4,12,3,114,11,0,0, + 0,99,1,0,0,0,0,0,0,0,1,0,0,0,3,0, + 0,0,67,0,0,0,115,26,0,0,0,116,0,0,124,0, + 0,131,1,0,100,1,0,64,106,1,0,100,2,0,100,3, + 0,131,2,0,83,41,4,122,42,67,111,110,118,101,114,116, + 32,97,32,51,50,45,98,105,116,32,105,110,116,101,103,101, + 114,32,116,111,32,108,105,116,116,108,101,45,101,110,100,105, + 97,110,46,108,3,0,0,0,255,127,255,127,3,0,233,4, + 0,0,0,218,6,108,105,116,116,108,101,41,2,218,3,105, + 110,116,218,8,116,111,95,98,121,116,101,115,41,1,218,1, + 120,114,4,0,0,0,114,4,0,0,0,114,5,0,0,0, + 218,7,95,119,95,108,111,110,103,40,0,0,0,115,2,0, + 0,0,0,2,114,17,0,0,0,99,1,0,0,0,0,0, + 0,0,1,0,0,0,3,0,0,0,67,0,0,0,115,16, + 0,0,0,116,0,0,106,1,0,124,0,0,100,1,0,131, + 2,0,83,41,2,122,47,67,111,110,118,101,114,116,32,52, + 32,98,121,116,101,115,32,105,110,32,108,105,116,116,108,101, + 45,101,110,100,105,97,110,32,116,111,32,97,110,32,105,110, + 116,101,103,101,114,46,114,13,0,0,0,41,2,114,14,0, + 0,0,218,10,102,114,111,109,95,98,121,116,101,115,41,1, + 90,9,105,110,116,95,98,121,116,101,115,114,4,0,0,0, + 114,4,0,0,0,114,5,0,0,0,218,7,95,114,95,108, + 111,110,103,45,0,0,0,115,2,0,0,0,0,2,114,19, + 0,0,0,99,0,0,0,0,0,0,0,0,1,0,0,0, + 3,0,0,0,71,0,0,0,115,26,0,0,0,116,0,0, + 106,1,0,100,1,0,100,2,0,132,0,0,124,0,0,68, + 131,1,0,131,1,0,83,41,3,122,31,82,101,112,108,97, + 99,101,109,101,110,116,32,102,111,114,32,111,115,46,112,97, + 116,104,46,106,111,105,110,40,41,46,99,1,0,0,0,0, + 0,0,0,2,0,0,0,4,0,0,0,83,0,0,0,115, + 37,0,0,0,103,0,0,124,0,0,93,27,0,125,1,0, + 124,1,0,114,6,0,124,1,0,106,0,0,116,1,0,131, + 1,0,145,2,0,113,6,0,83,114,4,0,0,0,41,2, + 218,6,114,115,116,114,105,112,218,15,112,97,116,104,95,115, + 101,112,97,114,97,116,111,114,115,41,2,218,2,46,48,218, + 4,112,97,114,116,114,4,0,0,0,114,4,0,0,0,114, + 5,0,0,0,250,10,60,108,105,115,116,99,111,109,112,62, + 52,0,0,0,115,2,0,0,0,9,1,122,30,95,112,97, + 116,104,95,106,111,105,110,46,60,108,111,99,97,108,115,62, + 46,60,108,105,115,116,99,111,109,112,62,41,2,218,8,112, + 97,116,104,95,115,101,112,218,4,106,111,105,110,41,1,218, + 10,112,97,116,104,95,112,97,114,116,115,114,4,0,0,0, + 114,4,0,0,0,114,5,0,0,0,218,10,95,112,97,116, + 104,95,106,111,105,110,50,0,0,0,115,4,0,0,0,0, + 2,15,1,114,28,0,0,0,99,1,0,0,0,0,0,0, + 0,5,0,0,0,5,0,0,0,67,0,0,0,115,134,0, + 0,0,116,0,0,116,1,0,131,1,0,100,1,0,107,2, + 0,114,52,0,124,0,0,106,2,0,116,3,0,131,1,0, + 92,3,0,125,1,0,125,2,0,125,3,0,124,1,0,124, + 3,0,102,2,0,83,120,69,0,116,4,0,124,0,0,131, + 1,0,68,93,55,0,125,4,0,124,4,0,116,1,0,107, + 6,0,114,65,0,124,0,0,106,5,0,124,4,0,100,2, + 0,100,1,0,131,1,1,92,2,0,125,1,0,125,3,0, + 124,1,0,124,3,0,102,2,0,83,113,65,0,87,100,3, + 0,124,0,0,102,2,0,83,41,4,122,32,82,101,112,108, + 97,99,101,109,101,110,116,32,102,111,114,32,111,115,46,112, + 97,116,104,46,115,112,108,105,116,40,41,46,233,1,0,0, + 0,90,8,109,97,120,115,112,108,105,116,218,0,41,6,218, + 3,108,101,110,114,21,0,0,0,218,10,114,112,97,114,116, + 105,116,105,111,110,114,25,0,0,0,218,8,114,101,118,101, + 114,115,101,100,218,6,114,115,112,108,105,116,41,5,218,4, + 112,97,116,104,90,5,102,114,111,110,116,218,1,95,218,4, + 116,97,105,108,114,16,0,0,0,114,4,0,0,0,114,4, + 0,0,0,114,5,0,0,0,218,11,95,112,97,116,104,95, + 115,112,108,105,116,56,0,0,0,115,16,0,0,0,0,2, + 18,1,24,1,10,1,19,1,12,1,27,1,14,1,114,38, 0,0,0,99,1,0,0,0,0,0,0,0,1,0,0,0, - 3,0,0,0,67,0,0,0,115,26,0,0,0,116,0,0, - 124,0,0,131,1,0,100,1,0,64,106,1,0,100,2,0, - 100,3,0,131,2,0,83,41,4,122,42,67,111,110,118,101, - 114,116,32,97,32,51,50,45,98,105,116,32,105,110,116,101, - 103,101,114,32,116,111,32,108,105,116,116,108,101,45,101,110, - 100,105,97,110,46,108,3,0,0,0,255,127,255,127,3,0, - 233,4,0,0,0,218,6,108,105,116,116,108,101,41,2,218, - 3,105,110,116,218,8,116,111,95,98,121,116,101,115,41,1, - 218,1,120,114,4,0,0,0,114,4,0,0,0,114,5,0, - 0,0,218,7,95,119,95,108,111,110,103,40,0,0,0,115, - 2,0,0,0,0,2,114,17,0,0,0,99,1,0,0,0, - 0,0,0,0,1,0,0,0,3,0,0,0,67,0,0,0, - 115,16,0,0,0,116,0,0,106,1,0,124,0,0,100,1, - 0,131,2,0,83,41,2,122,47,67,111,110,118,101,114,116, - 32,52,32,98,121,116,101,115,32,105,110,32,108,105,116,116, - 108,101,45,101,110,100,105,97,110,32,116,111,32,97,110,32, - 105,110,116,101,103,101,114,46,114,13,0,0,0,41,2,114, - 14,0,0,0,218,10,102,114,111,109,95,98,121,116,101,115, - 41,1,90,9,105,110,116,95,98,121,116,101,115,114,4,0, - 0,0,114,4,0,0,0,114,5,0,0,0,218,7,95,114, - 95,108,111,110,103,45,0,0,0,115,2,0,0,0,0,2, - 114,19,0,0,0,99,0,0,0,0,0,0,0,0,1,0, - 0,0,3,0,0,0,71,0,0,0,115,26,0,0,0,116, - 0,0,106,1,0,100,1,0,100,2,0,132,0,0,124,0, - 0,68,131,1,0,131,1,0,83,41,3,122,31,82,101,112, + 2,0,0,0,67,0,0,0,115,13,0,0,0,116,0,0, + 106,1,0,124,0,0,131,1,0,83,41,1,122,126,83,116, + 97,116,32,116,104,101,32,112,97,116,104,46,10,10,32,32, + 32,32,77,97,100,101,32,97,32,115,101,112,97,114,97,116, + 101,32,102,117,110,99,116,105,111,110,32,116,111,32,109,97, + 107,101,32,105,116,32,101,97,115,105,101,114,32,116,111,32, + 111,118,101,114,114,105,100,101,32,105,110,32,101,120,112,101, + 114,105,109,101,110,116,115,10,32,32,32,32,40,101,46,103, + 46,32,99,97,99,104,101,32,115,116,97,116,32,114,101,115, + 117,108,116,115,41,46,10,10,32,32,32,32,41,2,114,3, + 0,0,0,90,4,115,116,97,116,41,1,114,35,0,0,0, + 114,4,0,0,0,114,4,0,0,0,114,5,0,0,0,218, + 10,95,112,97,116,104,95,115,116,97,116,68,0,0,0,115, + 2,0,0,0,0,7,114,39,0,0,0,99,2,0,0,0, + 0,0,0,0,3,0,0,0,11,0,0,0,67,0,0,0, + 115,58,0,0,0,121,16,0,116,0,0,124,0,0,131,1, + 0,125,2,0,87,110,22,0,4,116,1,0,107,10,0,114, + 40,0,1,1,1,100,1,0,83,89,110,1,0,88,124,2, + 0,106,2,0,100,2,0,64,124,1,0,107,2,0,83,41, + 3,122,49,84,101,115,116,32,119,104,101,116,104,101,114,32, + 116,104,101,32,112,97,116,104,32,105,115,32,116,104,101,32, + 115,112,101,99,105,102,105,101,100,32,109,111,100,101,32,116, + 121,112,101,46,70,105,0,240,0,0,41,3,114,39,0,0, + 0,218,7,79,83,69,114,114,111,114,218,7,115,116,95,109, + 111,100,101,41,3,114,35,0,0,0,218,4,109,111,100,101, + 90,9,115,116,97,116,95,105,110,102,111,114,4,0,0,0, + 114,4,0,0,0,114,5,0,0,0,218,18,95,112,97,116, + 104,95,105,115,95,109,111,100,101,95,116,121,112,101,78,0, + 0,0,115,10,0,0,0,0,2,3,1,16,1,13,1,9, + 1,114,43,0,0,0,99,1,0,0,0,0,0,0,0,1, + 0,0,0,3,0,0,0,67,0,0,0,115,13,0,0,0, + 116,0,0,124,0,0,100,1,0,131,2,0,83,41,2,122, + 31,82,101,112,108,97,99,101,109,101,110,116,32,102,111,114, + 32,111,115,46,112,97,116,104,46,105,115,102,105,108,101,46, + 105,0,128,0,0,41,1,114,43,0,0,0,41,1,114,35, + 0,0,0,114,4,0,0,0,114,4,0,0,0,114,5,0, + 0,0,218,12,95,112,97,116,104,95,105,115,102,105,108,101, + 87,0,0,0,115,2,0,0,0,0,2,114,44,0,0,0, + 99,1,0,0,0,0,0,0,0,1,0,0,0,3,0,0, + 0,67,0,0,0,115,31,0,0,0,124,0,0,115,18,0, + 116,0,0,106,1,0,131,0,0,125,0,0,116,2,0,124, + 0,0,100,1,0,131,2,0,83,41,2,122,30,82,101,112, 108,97,99,101,109,101,110,116,32,102,111,114,32,111,115,46, - 112,97,116,104,46,106,111,105,110,40,41,46,99,1,0,0, - 0,0,0,0,0,2,0,0,0,4,0,0,0,83,0,0, - 0,115,37,0,0,0,103,0,0,124,0,0,93,27,0,125, - 1,0,124,1,0,114,6,0,124,1,0,106,0,0,116,1, - 0,131,1,0,145,2,0,113,6,0,83,114,4,0,0,0, - 41,2,218,6,114,115,116,114,105,112,218,15,112,97,116,104, - 95,115,101,112,97,114,97,116,111,114,115,41,2,218,2,46, - 48,218,4,112,97,114,116,114,4,0,0,0,114,4,0,0, - 0,114,5,0,0,0,250,10,60,108,105,115,116,99,111,109, - 112,62,52,0,0,0,115,2,0,0,0,9,1,122,30,95, - 112,97,116,104,95,106,111,105,110,46,60,108,111,99,97,108, - 115,62,46,60,108,105,115,116,99,111,109,112,62,41,2,218, - 8,112,97,116,104,95,115,101,112,218,4,106,111,105,110,41, - 1,218,10,112,97,116,104,95,112,97,114,116,115,114,4,0, - 0,0,114,4,0,0,0,114,5,0,0,0,218,10,95,112, - 97,116,104,95,106,111,105,110,50,0,0,0,115,4,0,0, - 0,0,2,15,1,114,28,0,0,0,99,1,0,0,0,0, - 0,0,0,5,0,0,0,5,0,0,0,67,0,0,0,115, - 134,0,0,0,116,0,0,116,1,0,131,1,0,100,1,0, - 107,2,0,114,52,0,124,0,0,106,2,0,116,3,0,131, - 1,0,92,3,0,125,1,0,125,2,0,125,3,0,124,1, - 0,124,3,0,102,2,0,83,120,69,0,116,4,0,124,0, - 0,131,1,0,68,93,55,0,125,4,0,124,4,0,116,1, - 0,107,6,0,114,65,0,124,0,0,106,5,0,124,4,0, - 100,2,0,100,1,0,131,1,1,92,2,0,125,1,0,125, - 3,0,124,1,0,124,3,0,102,2,0,83,113,65,0,87, - 100,3,0,124,0,0,102,2,0,83,41,4,122,32,82,101, - 112,108,97,99,101,109,101,110,116,32,102,111,114,32,111,115, - 46,112,97,116,104,46,115,112,108,105,116,40,41,46,233,1, - 0,0,0,90,8,109,97,120,115,112,108,105,116,218,0,41, - 6,218,3,108,101,110,114,21,0,0,0,218,10,114,112,97, - 114,116,105,116,105,111,110,114,25,0,0,0,218,8,114,101, - 118,101,114,115,101,100,218,6,114,115,112,108,105,116,41,5, - 218,4,112,97,116,104,90,5,102,114,111,110,116,218,1,95, - 218,4,116,97,105,108,114,16,0,0,0,114,4,0,0,0, + 112,97,116,104,46,105,115,100,105,114,46,105,0,64,0,0, + 41,3,114,3,0,0,0,218,6,103,101,116,99,119,100,114, + 43,0,0,0,41,1,114,35,0,0,0,114,4,0,0,0, 114,4,0,0,0,114,5,0,0,0,218,11,95,112,97,116, - 104,95,115,112,108,105,116,56,0,0,0,115,16,0,0,0, - 0,2,18,1,24,1,10,1,19,1,12,1,27,1,14,1, - 114,38,0,0,0,99,1,0,0,0,0,0,0,0,1,0, - 0,0,2,0,0,0,67,0,0,0,115,13,0,0,0,116, - 0,0,106,1,0,124,0,0,131,1,0,83,41,1,122,126, - 83,116,97,116,32,116,104,101,32,112,97,116,104,46,10,10, - 32,32,32,32,77,97,100,101,32,97,32,115,101,112,97,114, - 97,116,101,32,102,117,110,99,116,105,111,110,32,116,111,32, - 109,97,107,101,32,105,116,32,101,97,115,105,101,114,32,116, - 111,32,111,118,101,114,114,105,100,101,32,105,110,32,101,120, - 112,101,114,105,109,101,110,116,115,10,32,32,32,32,40,101, - 46,103,46,32,99,97,99,104,101,32,115,116,97,116,32,114, - 101,115,117,108,116,115,41,46,10,10,32,32,32,32,41,2, - 114,3,0,0,0,90,4,115,116,97,116,41,1,114,35,0, - 0,0,114,4,0,0,0,114,4,0,0,0,114,5,0,0, - 0,218,10,95,112,97,116,104,95,115,116,97,116,68,0,0, - 0,115,2,0,0,0,0,7,114,39,0,0,0,99,2,0, - 0,0,0,0,0,0,3,0,0,0,11,0,0,0,67,0, - 0,0,115,58,0,0,0,121,16,0,116,0,0,124,0,0, - 131,1,0,125,2,0,87,110,22,0,4,116,1,0,107,10, - 0,114,40,0,1,1,1,100,1,0,83,89,110,1,0,88, - 124,2,0,106,2,0,100,2,0,64,124,1,0,107,2,0, - 83,41,3,122,49,84,101,115,116,32,119,104,101,116,104,101, - 114,32,116,104,101,32,112,97,116,104,32,105,115,32,116,104, - 101,32,115,112,101,99,105,102,105,101,100,32,109,111,100,101, - 32,116,121,112,101,46,70,105,0,240,0,0,41,3,114,39, - 0,0,0,218,7,79,83,69,114,114,111,114,218,7,115,116, - 95,109,111,100,101,41,3,114,35,0,0,0,218,4,109,111, - 100,101,90,9,115,116,97,116,95,105,110,102,111,114,4,0, - 0,0,114,4,0,0,0,114,5,0,0,0,218,18,95,112, - 97,116,104,95,105,115,95,109,111,100,101,95,116,121,112,101, - 78,0,0,0,115,10,0,0,0,0,2,3,1,16,1,13, - 1,9,1,114,43,0,0,0,99,1,0,0,0,0,0,0, - 0,1,0,0,0,3,0,0,0,67,0,0,0,115,13,0, - 0,0,116,0,0,124,0,0,100,1,0,131,2,0,83,41, - 2,122,31,82,101,112,108,97,99,101,109,101,110,116,32,102, - 111,114,32,111,115,46,112,97,116,104,46,105,115,102,105,108, - 101,46,105,0,128,0,0,41,1,114,43,0,0,0,41,1, - 114,35,0,0,0,114,4,0,0,0,114,4,0,0,0,114, - 5,0,0,0,218,12,95,112,97,116,104,95,105,115,102,105, - 108,101,87,0,0,0,115,2,0,0,0,0,2,114,44,0, - 0,0,99,1,0,0,0,0,0,0,0,1,0,0,0,3, - 0,0,0,67,0,0,0,115,31,0,0,0,124,0,0,115, - 18,0,116,0,0,106,1,0,131,0,0,125,0,0,116,2, - 0,124,0,0,100,1,0,131,2,0,83,41,2,122,30,82, - 101,112,108,97,99,101,109,101,110,116,32,102,111,114,32,111, - 115,46,112,97,116,104,46,105,115,100,105,114,46,105,0,64, - 0,0,41,3,114,3,0,0,0,218,6,103,101,116,99,119, - 100,114,43,0,0,0,41,1,114,35,0,0,0,114,4,0, - 0,0,114,4,0,0,0,114,5,0,0,0,218,11,95,112, - 97,116,104,95,105,115,100,105,114,92,0,0,0,115,6,0, - 0,0,0,2,6,1,12,1,114,46,0,0,0,105,182,1, - 0,0,99,3,0,0,0,0,0,0,0,6,0,0,0,17, - 0,0,0,67,0,0,0,115,193,0,0,0,100,1,0,106, - 0,0,124,0,0,116,1,0,124,0,0,131,1,0,131,2, - 0,125,3,0,116,2,0,106,3,0,124,3,0,116,2,0, - 106,4,0,116,2,0,106,5,0,66,116,2,0,106,6,0, - 66,124,2,0,100,2,0,64,131,3,0,125,4,0,121,61, - 0,116,7,0,106,8,0,124,4,0,100,3,0,131,2,0, - 143,20,0,125,5,0,124,5,0,106,9,0,124,1,0,131, - 1,0,1,87,100,4,0,81,82,88,116,2,0,106,10,0, - 124,3,0,124,0,0,131,2,0,1,87,110,59,0,4,116, - 11,0,107,10,0,114,188,0,1,1,1,121,17,0,116,2, - 0,106,12,0,124,3,0,131,1,0,1,87,110,18,0,4, - 116,11,0,107,10,0,114,180,0,1,1,1,89,110,1,0, - 88,130,0,0,89,110,1,0,88,100,4,0,83,41,5,122, - 162,66,101,115,116,45,101,102,102,111,114,116,32,102,117,110, - 99,116,105,111,110,32,116,111,32,119,114,105,116,101,32,100, - 97,116,97,32,116,111,32,97,32,112,97,116,104,32,97,116, - 111,109,105,99,97,108,108,121,46,10,32,32,32,32,66,101, - 32,112,114,101,112,97,114,101,100,32,116,111,32,104,97,110, - 100,108,101,32,97,32,70,105,108,101,69,120,105,115,116,115, - 69,114,114,111,114,32,105,102,32,99,111,110,99,117,114,114, - 101,110,116,32,119,114,105,116,105,110,103,32,111,102,32,116, - 104,101,10,32,32,32,32,116,101,109,112,111,114,97,114,121, - 32,102,105,108,101,32,105,115,32,97,116,116,101,109,112,116, - 101,100,46,122,5,123,125,46,123,125,105,182,1,0,0,90, - 2,119,98,78,41,13,218,6,102,111,114,109,97,116,218,2, - 105,100,114,3,0,0,0,90,4,111,112,101,110,90,6,79, - 95,69,88,67,76,90,7,79,95,67,82,69,65,84,90,8, - 79,95,87,82,79,78,76,89,218,3,95,105,111,218,6,70, - 105,108,101,73,79,218,5,119,114,105,116,101,218,7,114,101, - 112,108,97,99,101,114,40,0,0,0,90,6,117,110,108,105, - 110,107,41,6,114,35,0,0,0,218,4,100,97,116,97,114, - 42,0,0,0,90,8,112,97,116,104,95,116,109,112,90,2, - 102,100,218,4,102,105,108,101,114,4,0,0,0,114,4,0, - 0,0,114,5,0,0,0,218,13,95,119,114,105,116,101,95, - 97,116,111,109,105,99,99,0,0,0,115,26,0,0,0,0, - 5,24,1,9,1,33,1,3,3,21,1,20,1,20,1,13, - 1,3,1,17,1,13,1,5,1,114,55,0,0,0,105,22, - 13,0,0,233,2,0,0,0,114,13,0,0,0,115,2,0, - 0,0,13,10,90,11,95,95,112,121,99,97,99,104,101,95, - 95,122,4,111,112,116,45,122,3,46,112,121,122,4,46,112, - 121,99,78,218,12,111,112,116,105,109,105,122,97,116,105,111, - 110,99,2,0,0,0,1,0,0,0,11,0,0,0,6,0, - 0,0,67,0,0,0,115,87,1,0,0,124,1,0,100,1, - 0,107,9,0,114,76,0,116,0,0,106,1,0,100,2,0, - 116,2,0,131,2,0,1,124,2,0,100,1,0,107,9,0, - 114,58,0,100,3,0,125,3,0,116,3,0,124,3,0,131, - 1,0,130,1,0,124,1,0,114,70,0,100,4,0,110,3, - 0,100,5,0,125,2,0,116,4,0,124,0,0,131,1,0, - 92,2,0,125,4,0,125,5,0,124,5,0,106,5,0,100, - 6,0,131,1,0,92,3,0,125,6,0,125,7,0,125,8, - 0,116,6,0,106,7,0,106,8,0,125,9,0,124,9,0, - 100,1,0,107,8,0,114,154,0,116,9,0,100,7,0,131, - 1,0,130,1,0,100,4,0,106,10,0,124,6,0,114,172, - 0,124,6,0,110,3,0,124,8,0,124,7,0,124,9,0, - 103,3,0,131,1,0,125,10,0,124,2,0,100,1,0,107, - 8,0,114,241,0,116,6,0,106,11,0,106,12,0,100,8, - 0,107,2,0,114,229,0,100,4,0,125,2,0,110,12,0, - 116,6,0,106,11,0,106,12,0,125,2,0,116,13,0,124, - 2,0,131,1,0,125,2,0,124,2,0,100,4,0,107,3, - 0,114,63,1,124,2,0,106,14,0,131,0,0,115,42,1, - 116,15,0,100,9,0,106,16,0,124,2,0,131,1,0,131, - 1,0,130,1,0,100,10,0,106,16,0,124,10,0,116,17, - 0,124,2,0,131,3,0,125,10,0,116,18,0,124,4,0, - 116,19,0,124,10,0,116,20,0,100,8,0,25,23,131,3, - 0,83,41,11,97,254,2,0,0,71,105,118,101,110,32,116, - 104,101,32,112,97,116,104,32,116,111,32,97,32,46,112,121, - 32,102,105,108,101,44,32,114,101,116,117,114,110,32,116,104, - 101,32,112,97,116,104,32,116,111,32,105,116,115,32,46,112, - 121,99,32,102,105,108,101,46,10,10,32,32,32,32,84,104, - 101,32,46,112,121,32,102,105,108,101,32,100,111,101,115,32, - 110,111,116,32,110,101,101,100,32,116,111,32,101,120,105,115, - 116,59,32,116,104,105,115,32,115,105,109,112,108,121,32,114, - 101,116,117,114,110,115,32,116,104,101,32,112,97,116,104,32, - 116,111,32,116,104,101,10,32,32,32,32,46,112,121,99,32, - 102,105,108,101,32,99,97,108,99,117,108,97,116,101,100,32, - 97,115,32,105,102,32,116,104,101,32,46,112,121,32,102,105, - 108,101,32,119,101,114,101,32,105,109,112,111,114,116,101,100, - 46,10,10,32,32,32,32,84,104,101,32,39,111,112,116,105, - 109,105,122,97,116,105,111,110,39,32,112,97,114,97,109,101, - 116,101,114,32,99,111,110,116,114,111,108,115,32,116,104,101, - 32,112,114,101,115,117,109,101,100,32,111,112,116,105,109,105, - 122,97,116,105,111,110,32,108,101,118,101,108,32,111,102,10, - 32,32,32,32,116,104,101,32,98,121,116,101,99,111,100,101, - 32,102,105,108,101,46,32,73,102,32,39,111,112,116,105,109, - 105,122,97,116,105,111,110,39,32,105,115,32,110,111,116,32, - 78,111,110,101,44,32,116,104,101,32,115,116,114,105,110,103, - 32,114,101,112,114,101,115,101,110,116,97,116,105,111,110,10, - 32,32,32,32,111,102,32,116,104,101,32,97,114,103,117,109, - 101,110,116,32,105,115,32,116,97,107,101,110,32,97,110,100, - 32,118,101,114,105,102,105,101,100,32,116,111,32,98,101,32, - 97,108,112,104,97,110,117,109,101,114,105,99,32,40,101,108, - 115,101,32,86,97,108,117,101,69,114,114,111,114,10,32,32, - 32,32,105,115,32,114,97,105,115,101,100,41,46,10,10,32, - 32,32,32,84,104,101,32,100,101,98,117,103,95,111,118,101, - 114,114,105,100,101,32,112,97,114,97,109,101,116,101,114,32, - 105,115,32,100,101,112,114,101,99,97,116,101,100,46,32,73, - 102,32,100,101,98,117,103,95,111,118,101,114,114,105,100,101, - 32,105,115,32,110,111,116,32,78,111,110,101,44,10,32,32, - 32,32,97,32,84,114,117,101,32,118,97,108,117,101,32,105, - 115,32,116,104,101,32,115,97,109,101,32,97,115,32,115,101, - 116,116,105,110,103,32,39,111,112,116,105,109,105,122,97,116, - 105,111,110,39,32,116,111,32,116,104,101,32,101,109,112,116, - 121,32,115,116,114,105,110,103,10,32,32,32,32,119,104,105, - 108,101,32,97,32,70,97,108,115,101,32,118,97,108,117,101, - 32,105,115,32,101,113,117,105,118,97,108,101,110,116,32,116, - 111,32,115,101,116,116,105,110,103,32,39,111,112,116,105,109, - 105,122,97,116,105,111,110,39,32,116,111,32,39,49,39,46, - 10,10,32,32,32,32,73,102,32,115,121,115,46,105,109,112, - 108,101,109,101,110,116,97,116,105,111,110,46,99,97,99,104, - 101,95,116,97,103,32,105,115,32,78,111,110,101,32,116,104, - 101,110,32,78,111,116,73,109,112,108,101,109,101,110,116,101, - 100,69,114,114,111,114,32,105,115,32,114,97,105,115,101,100, - 46,10,10,32,32,32,32,78,122,70,116,104,101,32,100,101, - 98,117,103,95,111,118,101,114,114,105,100,101,32,112,97,114, - 97,109,101,116,101,114,32,105,115,32,100,101,112,114,101,99, - 97,116,101,100,59,32,117,115,101,32,39,111,112,116,105,109, - 105,122,97,116,105,111,110,39,32,105,110,115,116,101,97,100, - 122,50,100,101,98,117,103,95,111,118,101,114,114,105,100,101, - 32,111,114,32,111,112,116,105,109,105,122,97,116,105,111,110, - 32,109,117,115,116,32,98,101,32,115,101,116,32,116,111,32, - 78,111,110,101,114,30,0,0,0,114,29,0,0,0,218,1, - 46,122,36,115,121,115,46,105,109,112,108,101,109,101,110,116, - 97,116,105,111,110,46,99,97,99,104,101,95,116,97,103,32, - 105,115,32,78,111,110,101,233,0,0,0,0,122,24,123,33, - 114,125,32,105,115,32,110,111,116,32,97,108,112,104,97,110, - 117,109,101,114,105,99,122,7,123,125,46,123,125,123,125,41, - 21,218,9,95,119,97,114,110,105,110,103,115,218,4,119,97, - 114,110,218,18,68,101,112,114,101,99,97,116,105,111,110,87, - 97,114,110,105,110,103,218,9,84,121,112,101,69,114,114,111, - 114,114,38,0,0,0,114,32,0,0,0,114,7,0,0,0, - 218,14,105,109,112,108,101,109,101,110,116,97,116,105,111,110, - 218,9,99,97,99,104,101,95,116,97,103,218,19,78,111,116, - 73,109,112,108,101,109,101,110,116,101,100,69,114,114,111,114, - 114,26,0,0,0,218,5,102,108,97,103,115,218,8,111,112, - 116,105,109,105,122,101,218,3,115,116,114,218,7,105,115,97, - 108,110,117,109,218,10,86,97,108,117,101,69,114,114,111,114, - 114,47,0,0,0,218,4,95,79,80,84,114,28,0,0,0, - 218,8,95,80,89,67,65,67,72,69,218,17,66,89,84,69, - 67,79,68,69,95,83,85,70,70,73,88,69,83,41,11,114, - 35,0,0,0,90,14,100,101,98,117,103,95,111,118,101,114, - 114,105,100,101,114,57,0,0,0,218,7,109,101,115,115,97, - 103,101,218,4,104,101,97,100,114,37,0,0,0,90,4,98, - 97,115,101,218,3,115,101,112,218,4,114,101,115,116,90,3, - 116,97,103,90,15,97,108,109,111,115,116,95,102,105,108,101, - 110,97,109,101,114,4,0,0,0,114,4,0,0,0,114,5, - 0,0,0,218,17,99,97,99,104,101,95,102,114,111,109,95, - 115,111,117,114,99,101,243,0,0,0,115,46,0,0,0,0, - 18,12,1,9,1,7,1,12,1,6,1,12,1,18,1,18, - 1,24,1,12,1,12,1,12,1,36,1,12,1,18,1,9, - 2,12,1,12,1,12,1,12,1,21,1,21,1,114,79,0, - 0,0,99,1,0,0,0,0,0,0,0,8,0,0,0,5, - 0,0,0,67,0,0,0,115,62,1,0,0,116,0,0,106, - 1,0,106,2,0,100,1,0,107,8,0,114,30,0,116,3, - 0,100,2,0,131,1,0,130,1,0,116,4,0,124,0,0, - 131,1,0,92,2,0,125,1,0,125,2,0,116,4,0,124, - 1,0,131,1,0,92,2,0,125,1,0,125,3,0,124,3, - 0,116,5,0,107,3,0,114,102,0,116,6,0,100,3,0, - 106,7,0,116,5,0,124,0,0,131,2,0,131,1,0,130, - 1,0,124,2,0,106,8,0,100,4,0,131,1,0,125,4, - 0,124,4,0,100,11,0,107,7,0,114,153,0,116,6,0, - 100,7,0,106,7,0,124,2,0,131,1,0,131,1,0,130, - 1,0,110,125,0,124,4,0,100,6,0,107,2,0,114,22, - 1,124,2,0,106,9,0,100,4,0,100,5,0,131,2,0, - 100,12,0,25,125,5,0,124,5,0,106,10,0,116,11,0, - 131,1,0,115,223,0,116,6,0,100,8,0,106,7,0,116, - 11,0,131,1,0,131,1,0,130,1,0,124,5,0,116,12, - 0,116,11,0,131,1,0,100,1,0,133,2,0,25,125,6, - 0,124,6,0,106,13,0,131,0,0,115,22,1,116,6,0, - 100,9,0,106,7,0,124,5,0,131,1,0,131,1,0,130, - 1,0,124,2,0,106,14,0,100,4,0,131,1,0,100,10, - 0,25,125,7,0,116,15,0,124,1,0,124,7,0,116,16, - 0,100,10,0,25,23,131,2,0,83,41,13,97,110,1,0, - 0,71,105,118,101,110,32,116,104,101,32,112,97,116,104,32, - 116,111,32,97,32,46,112,121,99,46,32,102,105,108,101,44, - 32,114,101,116,117,114,110,32,116,104,101,32,112,97,116,104, - 32,116,111,32,105,116,115,32,46,112,121,32,102,105,108,101, - 46,10,10,32,32,32,32,84,104,101,32,46,112,121,99,32, - 102,105,108,101,32,100,111,101,115,32,110,111,116,32,110,101, - 101,100,32,116,111,32,101,120,105,115,116,59,32,116,104,105, - 115,32,115,105,109,112,108,121,32,114,101,116,117,114,110,115, - 32,116,104,101,32,112,97,116,104,32,116,111,10,32,32,32, - 32,116,104,101,32,46,112,121,32,102,105,108,101,32,99,97, - 108,99,117,108,97,116,101,100,32,116,111,32,99,111,114,114, - 101,115,112,111,110,100,32,116,111,32,116,104,101,32,46,112, - 121,99,32,102,105,108,101,46,32,32,73,102,32,112,97,116, - 104,32,100,111,101,115,10,32,32,32,32,110,111,116,32,99, - 111,110,102,111,114,109,32,116,111,32,80,69,80,32,51,49, - 52,55,47,52,56,56,32,102,111,114,109,97,116,44,32,86, - 97,108,117,101,69,114,114,111,114,32,119,105,108,108,32,98, - 101,32,114,97,105,115,101,100,46,32,73,102,10,32,32,32, - 32,115,121,115,46,105,109,112,108,101,109,101,110,116,97,116, + 104,95,105,115,100,105,114,92,0,0,0,115,6,0,0,0, + 0,2,6,1,12,1,114,46,0,0,0,105,182,1,0,0, + 99,3,0,0,0,0,0,0,0,6,0,0,0,17,0,0, + 0,67,0,0,0,115,193,0,0,0,100,1,0,106,0,0, + 124,0,0,116,1,0,124,0,0,131,1,0,131,2,0,125, + 3,0,116,2,0,106,3,0,124,3,0,116,2,0,106,4, + 0,116,2,0,106,5,0,66,116,2,0,106,6,0,66,124, + 2,0,100,2,0,64,131,3,0,125,4,0,121,61,0,116, + 7,0,106,8,0,124,4,0,100,3,0,131,2,0,143,20, + 0,125,5,0,124,5,0,106,9,0,124,1,0,131,1,0, + 1,87,100,4,0,81,82,88,116,2,0,106,10,0,124,3, + 0,124,0,0,131,2,0,1,87,110,59,0,4,116,11,0, + 107,10,0,114,188,0,1,1,1,121,17,0,116,2,0,106, + 12,0,124,3,0,131,1,0,1,87,110,18,0,4,116,11, + 0,107,10,0,114,180,0,1,1,1,89,110,1,0,88,130, + 0,0,89,110,1,0,88,100,4,0,83,41,5,122,162,66, + 101,115,116,45,101,102,102,111,114,116,32,102,117,110,99,116, + 105,111,110,32,116,111,32,119,114,105,116,101,32,100,97,116, + 97,32,116,111,32,97,32,112,97,116,104,32,97,116,111,109, + 105,99,97,108,108,121,46,10,32,32,32,32,66,101,32,112, + 114,101,112,97,114,101,100,32,116,111,32,104,97,110,100,108, + 101,32,97,32,70,105,108,101,69,120,105,115,116,115,69,114, + 114,111,114,32,105,102,32,99,111,110,99,117,114,114,101,110, + 116,32,119,114,105,116,105,110,103,32,111,102,32,116,104,101, + 10,32,32,32,32,116,101,109,112,111,114,97,114,121,32,102, + 105,108,101,32,105,115,32,97,116,116,101,109,112,116,101,100, + 46,122,5,123,125,46,123,125,105,182,1,0,0,90,2,119, + 98,78,41,13,218,6,102,111,114,109,97,116,218,2,105,100, + 114,3,0,0,0,90,4,111,112,101,110,90,6,79,95,69, + 88,67,76,90,7,79,95,67,82,69,65,84,90,8,79,95, + 87,82,79,78,76,89,218,3,95,105,111,218,6,70,105,108, + 101,73,79,218,5,119,114,105,116,101,218,7,114,101,112,108, + 97,99,101,114,40,0,0,0,90,6,117,110,108,105,110,107, + 41,6,114,35,0,0,0,218,4,100,97,116,97,114,42,0, + 0,0,90,8,112,97,116,104,95,116,109,112,90,2,102,100, + 218,4,102,105,108,101,114,4,0,0,0,114,4,0,0,0, + 114,5,0,0,0,218,13,95,119,114,105,116,101,95,97,116, + 111,109,105,99,99,0,0,0,115,26,0,0,0,0,5,24, + 1,9,1,33,1,3,3,21,1,20,1,20,1,13,1,3, + 1,17,1,13,1,5,1,114,55,0,0,0,105,22,13,0, + 0,233,2,0,0,0,114,13,0,0,0,115,2,0,0,0, + 13,10,90,11,95,95,112,121,99,97,99,104,101,95,95,122, + 4,111,112,116,45,122,3,46,112,121,122,4,46,112,121,99, + 78,218,12,111,112,116,105,109,105,122,97,116,105,111,110,99, + 2,0,0,0,1,0,0,0,11,0,0,0,6,0,0,0, + 67,0,0,0,115,87,1,0,0,124,1,0,100,1,0,107, + 9,0,114,76,0,116,0,0,106,1,0,100,2,0,116,2, + 0,131,2,0,1,124,2,0,100,1,0,107,9,0,114,58, + 0,100,3,0,125,3,0,116,3,0,124,3,0,131,1,0, + 130,1,0,124,1,0,114,70,0,100,4,0,110,3,0,100, + 5,0,125,2,0,116,4,0,124,0,0,131,1,0,92,2, + 0,125,4,0,125,5,0,124,5,0,106,5,0,100,6,0, + 131,1,0,92,3,0,125,6,0,125,7,0,125,8,0,116, + 6,0,106,7,0,106,8,0,125,9,0,124,9,0,100,1, + 0,107,8,0,114,154,0,116,9,0,100,7,0,131,1,0, + 130,1,0,100,4,0,106,10,0,124,6,0,114,172,0,124, + 6,0,110,3,0,124,8,0,124,7,0,124,9,0,103,3, + 0,131,1,0,125,10,0,124,2,0,100,1,0,107,8,0, + 114,241,0,116,6,0,106,11,0,106,12,0,100,8,0,107, + 2,0,114,229,0,100,4,0,125,2,0,110,12,0,116,6, + 0,106,11,0,106,12,0,125,2,0,116,13,0,124,2,0, + 131,1,0,125,2,0,124,2,0,100,4,0,107,3,0,114, + 63,1,124,2,0,106,14,0,131,0,0,115,42,1,116,15, + 0,100,9,0,106,16,0,124,2,0,131,1,0,131,1,0, + 130,1,0,100,10,0,106,16,0,124,10,0,116,17,0,124, + 2,0,131,3,0,125,10,0,116,18,0,124,4,0,116,19, + 0,124,10,0,116,20,0,100,8,0,25,23,131,3,0,83, + 41,11,97,254,2,0,0,71,105,118,101,110,32,116,104,101, + 32,112,97,116,104,32,116,111,32,97,32,46,112,121,32,102, + 105,108,101,44,32,114,101,116,117,114,110,32,116,104,101,32, + 112,97,116,104,32,116,111,32,105,116,115,32,46,112,121,99, + 32,102,105,108,101,46,10,10,32,32,32,32,84,104,101,32, + 46,112,121,32,102,105,108,101,32,100,111,101,115,32,110,111, + 116,32,110,101,101,100,32,116,111,32,101,120,105,115,116,59, + 32,116,104,105,115,32,115,105,109,112,108,121,32,114,101,116, + 117,114,110,115,32,116,104,101,32,112,97,116,104,32,116,111, + 32,116,104,101,10,32,32,32,32,46,112,121,99,32,102,105, + 108,101,32,99,97,108,99,117,108,97,116,101,100,32,97,115, + 32,105,102,32,116,104,101,32,46,112,121,32,102,105,108,101, + 32,119,101,114,101,32,105,109,112,111,114,116,101,100,46,10, + 10,32,32,32,32,84,104,101,32,39,111,112,116,105,109,105, + 122,97,116,105,111,110,39,32,112,97,114,97,109,101,116,101, + 114,32,99,111,110,116,114,111,108,115,32,116,104,101,32,112, + 114,101,115,117,109,101,100,32,111,112,116,105,109,105,122,97, + 116,105,111,110,32,108,101,118,101,108,32,111,102,10,32,32, + 32,32,116,104,101,32,98,121,116,101,99,111,100,101,32,102, + 105,108,101,46,32,73,102,32,39,111,112,116,105,109,105,122, + 97,116,105,111,110,39,32,105,115,32,110,111,116,32,78,111, + 110,101,44,32,116,104,101,32,115,116,114,105,110,103,32,114, + 101,112,114,101,115,101,110,116,97,116,105,111,110,10,32,32, + 32,32,111,102,32,116,104,101,32,97,114,103,117,109,101,110, + 116,32,105,115,32,116,97,107,101,110,32,97,110,100,32,118, + 101,114,105,102,105,101,100,32,116,111,32,98,101,32,97,108, + 112,104,97,110,117,109,101,114,105,99,32,40,101,108,115,101, + 32,86,97,108,117,101,69,114,114,111,114,10,32,32,32,32, + 105,115,32,114,97,105,115,101,100,41,46,10,10,32,32,32, + 32,84,104,101,32,100,101,98,117,103,95,111,118,101,114,114, + 105,100,101,32,112,97,114,97,109,101,116,101,114,32,105,115, + 32,100,101,112,114,101,99,97,116,101,100,46,32,73,102,32, + 100,101,98,117,103,95,111,118,101,114,114,105,100,101,32,105, + 115,32,110,111,116,32,78,111,110,101,44,10,32,32,32,32, + 97,32,84,114,117,101,32,118,97,108,117,101,32,105,115,32, + 116,104,101,32,115,97,109,101,32,97,115,32,115,101,116,116, + 105,110,103,32,39,111,112,116,105,109,105,122,97,116,105,111, + 110,39,32,116,111,32,116,104,101,32,101,109,112,116,121,32, + 115,116,114,105,110,103,10,32,32,32,32,119,104,105,108,101, + 32,97,32,70,97,108,115,101,32,118,97,108,117,101,32,105, + 115,32,101,113,117,105,118,97,108,101,110,116,32,116,111,32, + 115,101,116,116,105,110,103,32,39,111,112,116,105,109,105,122, + 97,116,105,111,110,39,32,116,111,32,39,49,39,46,10,10, + 32,32,32,32,73,102,32,115,121,115,46,105,109,112,108,101, + 109,101,110,116,97,116,105,111,110,46,99,97,99,104,101,95, + 116,97,103,32,105,115,32,78,111,110,101,32,116,104,101,110, + 32,78,111,116,73,109,112,108,101,109,101,110,116,101,100,69, + 114,114,111,114,32,105,115,32,114,97,105,115,101,100,46,10, + 10,32,32,32,32,78,122,70,116,104,101,32,100,101,98,117, + 103,95,111,118,101,114,114,105,100,101,32,112,97,114,97,109, + 101,116,101,114,32,105,115,32,100,101,112,114,101,99,97,116, + 101,100,59,32,117,115,101,32,39,111,112,116,105,109,105,122, + 97,116,105,111,110,39,32,105,110,115,116,101,97,100,122,50, + 100,101,98,117,103,95,111,118,101,114,114,105,100,101,32,111, + 114,32,111,112,116,105,109,105,122,97,116,105,111,110,32,109, + 117,115,116,32,98,101,32,115,101,116,32,116,111,32,78,111, + 110,101,114,30,0,0,0,114,29,0,0,0,218,1,46,122, + 36,115,121,115,46,105,109,112,108,101,109,101,110,116,97,116, 105,111,110,46,99,97,99,104,101,95,116,97,103,32,105,115, - 32,78,111,110,101,32,116,104,101,110,32,78,111,116,73,109, - 112,108,101,109,101,110,116,101,100,69,114,114,111,114,32,105, - 115,32,114,97,105,115,101,100,46,10,10,32,32,32,32,78, - 122,36,115,121,115,46,105,109,112,108,101,109,101,110,116,97, - 116,105,111,110,46,99,97,99,104,101,95,116,97,103,32,105, - 115,32,78,111,110,101,122,37,123,125,32,110,111,116,32,98, - 111,116,116,111,109,45,108,101,118,101,108,32,100,105,114,101, - 99,116,111,114,121,32,105,110,32,123,33,114,125,114,58,0, - 0,0,114,56,0,0,0,233,3,0,0,0,122,33,101,120, - 112,101,99,116,101,100,32,111,110,108,121,32,50,32,111,114, - 32,51,32,100,111,116,115,32,105,110,32,123,33,114,125,122, - 57,111,112,116,105,109,105,122,97,116,105,111,110,32,112,111, - 114,116,105,111,110,32,111,102,32,102,105,108,101,110,97,109, - 101,32,100,111,101,115,32,110,111,116,32,115,116,97,114,116, - 32,119,105,116,104,32,123,33,114,125,122,52,111,112,116,105, - 109,105,122,97,116,105,111,110,32,108,101,118,101,108,32,123, - 33,114,125,32,105,115,32,110,111,116,32,97,110,32,97,108, - 112,104,97,110,117,109,101,114,105,99,32,118,97,108,117,101, - 114,59,0,0,0,62,2,0,0,0,114,56,0,0,0,114, - 80,0,0,0,233,254,255,255,255,41,17,114,7,0,0,0, - 114,64,0,0,0,114,65,0,0,0,114,66,0,0,0,114, - 38,0,0,0,114,73,0,0,0,114,71,0,0,0,114,47, - 0,0,0,218,5,99,111,117,110,116,114,34,0,0,0,114, - 9,0,0,0,114,72,0,0,0,114,31,0,0,0,114,70, - 0,0,0,218,9,112,97,114,116,105,116,105,111,110,114,28, - 0,0,0,218,15,83,79,85,82,67,69,95,83,85,70,70, - 73,88,69,83,41,8,114,35,0,0,0,114,76,0,0,0, - 90,16,112,121,99,97,99,104,101,95,102,105,108,101,110,97, - 109,101,90,7,112,121,99,97,99,104,101,90,9,100,111,116, - 95,99,111,117,110,116,114,57,0,0,0,90,9,111,112,116, - 95,108,101,118,101,108,90,13,98,97,115,101,95,102,105,108, - 101,110,97,109,101,114,4,0,0,0,114,4,0,0,0,114, - 5,0,0,0,218,17,115,111,117,114,99,101,95,102,114,111, - 109,95,99,97,99,104,101,31,1,0,0,115,44,0,0,0, - 0,9,18,1,12,1,18,1,18,1,12,1,9,1,15,1, - 15,1,12,1,9,1,15,1,12,1,22,1,15,1,9,1, - 12,1,22,1,12,1,9,1,12,1,19,1,114,85,0,0, - 0,99,1,0,0,0,0,0,0,0,5,0,0,0,12,0, - 0,0,67,0,0,0,115,164,0,0,0,116,0,0,124,0, - 0,131,1,0,100,1,0,107,2,0,114,22,0,100,2,0, - 83,124,0,0,106,1,0,100,3,0,131,1,0,92,3,0, - 125,1,0,125,2,0,125,3,0,124,1,0,12,115,81,0, - 124,3,0,106,2,0,131,0,0,100,7,0,100,8,0,133, - 2,0,25,100,6,0,107,3,0,114,85,0,124,0,0,83, - 121,16,0,116,3,0,124,0,0,131,1,0,125,4,0,87, - 110,40,0,4,116,4,0,116,5,0,102,2,0,107,10,0, - 114,143,0,1,1,1,124,0,0,100,2,0,100,9,0,133, - 2,0,25,125,4,0,89,110,1,0,88,116,6,0,124,4, - 0,131,1,0,114,160,0,124,4,0,83,124,0,0,83,41, - 10,122,188,67,111,110,118,101,114,116,32,97,32,98,121,116, - 101,99,111,100,101,32,102,105,108,101,32,112,97,116,104,32, - 116,111,32,97,32,115,111,117,114,99,101,32,112,97,116,104, - 32,40,105,102,32,112,111,115,115,105,98,108,101,41,46,10, - 10,32,32,32,32,84,104,105,115,32,102,117,110,99,116,105, - 111,110,32,101,120,105,115,116,115,32,112,117,114,101,108,121, - 32,102,111,114,32,98,97,99,107,119,97,114,100,115,45,99, - 111,109,112,97,116,105,98,105,108,105,116,121,32,102,111,114, - 10,32,32,32,32,80,121,73,109,112,111,114,116,95,69,120, - 101,99,67,111,100,101,77,111,100,117,108,101,87,105,116,104, - 70,105,108,101,110,97,109,101,115,40,41,32,105,110,32,116, - 104,101,32,67,32,65,80,73,46,10,10,32,32,32,32,114, - 59,0,0,0,78,114,58,0,0,0,114,80,0,0,0,114, - 29,0,0,0,90,2,112,121,233,253,255,255,255,233,255,255, - 255,255,114,87,0,0,0,41,7,114,31,0,0,0,114,32, - 0,0,0,218,5,108,111,119,101,114,114,85,0,0,0,114, - 66,0,0,0,114,71,0,0,0,114,44,0,0,0,41,5, - 218,13,98,121,116,101,99,111,100,101,95,112,97,116,104,114, - 78,0,0,0,114,36,0,0,0,90,9,101,120,116,101,110, - 115,105,111,110,218,11,115,111,117,114,99,101,95,112,97,116, - 104,114,4,0,0,0,114,4,0,0,0,114,5,0,0,0, - 218,15,95,103,101,116,95,115,111,117,114,99,101,102,105,108, - 101,64,1,0,0,115,20,0,0,0,0,7,18,1,4,1, - 24,1,35,1,4,1,3,1,16,1,19,1,21,1,114,91, - 0,0,0,99,1,0,0,0,0,0,0,0,1,0,0,0, - 11,0,0,0,67,0,0,0,115,92,0,0,0,124,0,0, - 106,0,0,116,1,0,116,2,0,131,1,0,131,1,0,114, - 59,0,121,14,0,116,3,0,124,0,0,131,1,0,83,87, - 113,88,0,4,116,4,0,107,10,0,114,55,0,1,1,1, - 89,113,88,0,88,110,29,0,124,0,0,106,0,0,116,1, - 0,116,5,0,131,1,0,131,1,0,114,84,0,124,0,0, - 83,100,0,0,83,100,0,0,83,41,1,78,41,6,218,8, - 101,110,100,115,119,105,116,104,218,5,116,117,112,108,101,114, - 84,0,0,0,114,79,0,0,0,114,66,0,0,0,114,74, - 0,0,0,41,1,218,8,102,105,108,101,110,97,109,101,114, - 4,0,0,0,114,4,0,0,0,114,5,0,0,0,218,11, - 95,103,101,116,95,99,97,99,104,101,100,83,1,0,0,115, - 16,0,0,0,0,1,21,1,3,1,14,1,13,1,8,1, - 21,1,4,2,114,95,0,0,0,99,1,0,0,0,0,0, - 0,0,2,0,0,0,11,0,0,0,67,0,0,0,115,60, - 0,0,0,121,19,0,116,0,0,124,0,0,131,1,0,106, - 1,0,125,1,0,87,110,24,0,4,116,2,0,107,10,0, - 114,45,0,1,1,1,100,1,0,125,1,0,89,110,1,0, - 88,124,1,0,100,2,0,79,125,1,0,124,1,0,83,41, - 3,122,51,67,97,108,99,117,108,97,116,101,32,116,104,101, - 32,109,111,100,101,32,112,101,114,109,105,115,115,105,111,110, - 115,32,102,111,114,32,97,32,98,121,116,101,99,111,100,101, - 32,102,105,108,101,46,105,182,1,0,0,233,128,0,0,0, - 41,3,114,39,0,0,0,114,41,0,0,0,114,40,0,0, - 0,41,2,114,35,0,0,0,114,42,0,0,0,114,4,0, - 0,0,114,4,0,0,0,114,5,0,0,0,218,10,95,99, - 97,108,99,95,109,111,100,101,95,1,0,0,115,12,0,0, - 0,0,2,3,1,19,1,13,1,11,3,10,1,114,97,0, - 0,0,218,9,118,101,114,98,111,115,105,116,121,114,29,0, - 0,0,99,1,0,0,0,1,0,0,0,3,0,0,0,4, - 0,0,0,71,0,0,0,115,75,0,0,0,116,0,0,106, - 1,0,106,2,0,124,1,0,107,5,0,114,71,0,124,0, - 0,106,3,0,100,6,0,131,1,0,115,43,0,100,3,0, - 124,0,0,23,125,0,0,116,4,0,124,0,0,106,5,0, - 124,2,0,140,0,0,100,4,0,116,0,0,106,6,0,131, - 1,1,1,100,5,0,83,41,7,122,61,80,114,105,110,116, - 32,116,104,101,32,109,101,115,115,97,103,101,32,116,111,32, - 115,116,100,101,114,114,32,105,102,32,45,118,47,80,89,84, - 72,79,78,86,69,82,66,79,83,69,32,105,115,32,116,117, - 114,110,101,100,32,111,110,46,250,1,35,250,7,105,109,112, - 111,114,116,32,122,2,35,32,114,54,0,0,0,78,41,2, - 114,99,0,0,0,114,100,0,0,0,41,7,114,7,0,0, - 0,114,67,0,0,0,218,7,118,101,114,98,111,115,101,114, - 9,0,0,0,218,5,112,114,105,110,116,114,47,0,0,0, - 218,6,115,116,100,101,114,114,41,3,114,75,0,0,0,114, - 98,0,0,0,218,4,97,114,103,115,114,4,0,0,0,114, - 4,0,0,0,114,5,0,0,0,218,16,95,118,101,114,98, - 111,115,101,95,109,101,115,115,97,103,101,107,1,0,0,115, - 8,0,0,0,0,2,18,1,15,1,10,1,114,105,0,0, - 0,99,1,0,0,0,0,0,0,0,3,0,0,0,11,0, - 0,0,3,0,0,0,115,84,0,0,0,100,1,0,135,0, - 0,102,1,0,100,2,0,100,3,0,134,1,0,125,1,0, - 121,13,0,116,0,0,106,1,0,125,2,0,87,110,30,0, - 4,116,2,0,107,10,0,114,66,0,1,1,1,100,4,0, - 100,5,0,132,0,0,125,2,0,89,110,1,0,88,124,2, - 0,124,1,0,136,0,0,131,2,0,1,124,1,0,83,41, - 6,122,252,68,101,99,111,114,97,116,111,114,32,116,111,32, - 118,101,114,105,102,121,32,116,104,97,116,32,116,104,101,32, - 109,111,100,117,108,101,32,98,101,105,110,103,32,114,101,113, - 117,101,115,116,101,100,32,109,97,116,99,104,101,115,32,116, - 104,101,32,111,110,101,32,116,104,101,10,32,32,32,32,108, - 111,97,100,101,114,32,99,97,110,32,104,97,110,100,108,101, - 46,10,10,32,32,32,32,84,104,101,32,102,105,114,115,116, - 32,97,114,103,117,109,101,110,116,32,40,115,101,108,102,41, - 32,109,117,115,116,32,100,101,102,105,110,101,32,95,110,97, - 109,101,32,119,104,105,99,104,32,116,104,101,32,115,101,99, - 111,110,100,32,97,114,103,117,109,101,110,116,32,105,115,10, - 32,32,32,32,99,111,109,112,97,114,101,100,32,97,103,97, - 105,110,115,116,46,32,73,102,32,116,104,101,32,99,111,109, - 112,97,114,105,115,111,110,32,102,97,105,108,115,32,116,104, - 101,110,32,73,109,112,111,114,116,69,114,114,111,114,32,105, - 115,32,114,97,105,115,101,100,46,10,10,32,32,32,32,78, - 99,2,0,0,0,0,0,0,0,4,0,0,0,5,0,0, - 0,31,0,0,0,115,89,0,0,0,124,1,0,100,0,0, - 107,8,0,114,24,0,124,0,0,106,0,0,125,1,0,110, - 46,0,124,0,0,106,0,0,124,1,0,107,3,0,114,70, - 0,116,1,0,100,1,0,124,0,0,106,0,0,124,1,0, - 102,2,0,22,100,2,0,124,1,0,131,1,1,130,1,0, - 136,0,0,124,0,0,124,1,0,124,2,0,124,3,0,142, - 2,0,83,41,3,78,122,30,108,111,97,100,101,114,32,102, - 111,114,32,37,115,32,99,97,110,110,111,116,32,104,97,110, - 100,108,101,32,37,115,218,4,110,97,109,101,41,2,114,106, - 0,0,0,218,11,73,109,112,111,114,116,69,114,114,111,114, - 41,4,218,4,115,101,108,102,114,106,0,0,0,114,104,0, - 0,0,90,6,107,119,97,114,103,115,41,1,218,6,109,101, + 32,78,111,110,101,233,0,0,0,0,122,24,123,33,114,125, + 32,105,115,32,110,111,116,32,97,108,112,104,97,110,117,109, + 101,114,105,99,122,7,123,125,46,123,125,123,125,41,21,218, + 9,95,119,97,114,110,105,110,103,115,218,4,119,97,114,110, + 218,18,68,101,112,114,101,99,97,116,105,111,110,87,97,114, + 110,105,110,103,218,9,84,121,112,101,69,114,114,111,114,114, + 38,0,0,0,114,32,0,0,0,114,7,0,0,0,218,14, + 105,109,112,108,101,109,101,110,116,97,116,105,111,110,218,9, + 99,97,99,104,101,95,116,97,103,218,19,78,111,116,73,109, + 112,108,101,109,101,110,116,101,100,69,114,114,111,114,114,26, + 0,0,0,218,5,102,108,97,103,115,218,8,111,112,116,105, + 109,105,122,101,218,3,115,116,114,218,7,105,115,97,108,110, + 117,109,218,10,86,97,108,117,101,69,114,114,111,114,114,47, + 0,0,0,218,4,95,79,80,84,114,28,0,0,0,218,8, + 95,80,89,67,65,67,72,69,218,17,66,89,84,69,67,79, + 68,69,95,83,85,70,70,73,88,69,83,41,11,114,35,0, + 0,0,90,14,100,101,98,117,103,95,111,118,101,114,114,105, + 100,101,114,57,0,0,0,218,7,109,101,115,115,97,103,101, + 218,4,104,101,97,100,114,37,0,0,0,90,4,98,97,115, + 101,218,3,115,101,112,218,4,114,101,115,116,90,3,116,97, + 103,90,15,97,108,109,111,115,116,95,102,105,108,101,110,97, + 109,101,114,4,0,0,0,114,4,0,0,0,114,5,0,0, + 0,218,17,99,97,99,104,101,95,102,114,111,109,95,115,111, + 117,114,99,101,243,0,0,0,115,46,0,0,0,0,18,12, + 1,9,1,7,1,12,1,6,1,12,1,18,1,18,1,24, + 1,12,1,12,1,12,1,36,1,12,1,18,1,9,2,12, + 1,12,1,12,1,12,1,21,1,21,1,114,79,0,0,0, + 99,1,0,0,0,0,0,0,0,8,0,0,0,5,0,0, + 0,67,0,0,0,115,62,1,0,0,116,0,0,106,1,0, + 106,2,0,100,1,0,107,8,0,114,30,0,116,3,0,100, + 2,0,131,1,0,130,1,0,116,4,0,124,0,0,131,1, + 0,92,2,0,125,1,0,125,2,0,116,4,0,124,1,0, + 131,1,0,92,2,0,125,1,0,125,3,0,124,3,0,116, + 5,0,107,3,0,114,102,0,116,6,0,100,3,0,106,7, + 0,116,5,0,124,0,0,131,2,0,131,1,0,130,1,0, + 124,2,0,106,8,0,100,4,0,131,1,0,125,4,0,124, + 4,0,100,11,0,107,7,0,114,153,0,116,6,0,100,7, + 0,106,7,0,124,2,0,131,1,0,131,1,0,130,1,0, + 110,125,0,124,4,0,100,6,0,107,2,0,114,22,1,124, + 2,0,106,9,0,100,4,0,100,5,0,131,2,0,100,12, + 0,25,125,5,0,124,5,0,106,10,0,116,11,0,131,1, + 0,115,223,0,116,6,0,100,8,0,106,7,0,116,11,0, + 131,1,0,131,1,0,130,1,0,124,5,0,116,12,0,116, + 11,0,131,1,0,100,1,0,133,2,0,25,125,6,0,124, + 6,0,106,13,0,131,0,0,115,22,1,116,6,0,100,9, + 0,106,7,0,124,5,0,131,1,0,131,1,0,130,1,0, + 124,2,0,106,14,0,100,4,0,131,1,0,100,10,0,25, + 125,7,0,116,15,0,124,1,0,124,7,0,116,16,0,100, + 10,0,25,23,131,2,0,83,41,13,97,110,1,0,0,71, + 105,118,101,110,32,116,104,101,32,112,97,116,104,32,116,111, + 32,97,32,46,112,121,99,46,32,102,105,108,101,44,32,114, + 101,116,117,114,110,32,116,104,101,32,112,97,116,104,32,116, + 111,32,105,116,115,32,46,112,121,32,102,105,108,101,46,10, + 10,32,32,32,32,84,104,101,32,46,112,121,99,32,102,105, + 108,101,32,100,111,101,115,32,110,111,116,32,110,101,101,100, + 32,116,111,32,101,120,105,115,116,59,32,116,104,105,115,32, + 115,105,109,112,108,121,32,114,101,116,117,114,110,115,32,116, + 104,101,32,112,97,116,104,32,116,111,10,32,32,32,32,116, + 104,101,32,46,112,121,32,102,105,108,101,32,99,97,108,99, + 117,108,97,116,101,100,32,116,111,32,99,111,114,114,101,115, + 112,111,110,100,32,116,111,32,116,104,101,32,46,112,121,99, + 32,102,105,108,101,46,32,32,73,102,32,112,97,116,104,32, + 100,111,101,115,10,32,32,32,32,110,111,116,32,99,111,110, + 102,111,114,109,32,116,111,32,80,69,80,32,51,49,52,55, + 47,52,56,56,32,102,111,114,109,97,116,44,32,86,97,108, + 117,101,69,114,114,111,114,32,119,105,108,108,32,98,101,32, + 114,97,105,115,101,100,46,32,73,102,10,32,32,32,32,115, + 121,115,46,105,109,112,108,101,109,101,110,116,97,116,105,111, + 110,46,99,97,99,104,101,95,116,97,103,32,105,115,32,78, + 111,110,101,32,116,104,101,110,32,78,111,116,73,109,112,108, + 101,109,101,110,116,101,100,69,114,114,111,114,32,105,115,32, + 114,97,105,115,101,100,46,10,10,32,32,32,32,78,122,36, + 115,121,115,46,105,109,112,108,101,109,101,110,116,97,116,105, + 111,110,46,99,97,99,104,101,95,116,97,103,32,105,115,32, + 78,111,110,101,122,37,123,125,32,110,111,116,32,98,111,116, + 116,111,109,45,108,101,118,101,108,32,100,105,114,101,99,116, + 111,114,121,32,105,110,32,123,33,114,125,114,58,0,0,0, + 114,56,0,0,0,233,3,0,0,0,122,33,101,120,112,101, + 99,116,101,100,32,111,110,108,121,32,50,32,111,114,32,51, + 32,100,111,116,115,32,105,110,32,123,33,114,125,122,57,111, + 112,116,105,109,105,122,97,116,105,111,110,32,112,111,114,116, + 105,111,110,32,111,102,32,102,105,108,101,110,97,109,101,32, + 100,111,101,115,32,110,111,116,32,115,116,97,114,116,32,119, + 105,116,104,32,123,33,114,125,122,52,111,112,116,105,109,105, + 122,97,116,105,111,110,32,108,101,118,101,108,32,123,33,114, + 125,32,105,115,32,110,111,116,32,97,110,32,97,108,112,104, + 97,110,117,109,101,114,105,99,32,118,97,108,117,101,114,59, + 0,0,0,62,2,0,0,0,114,56,0,0,0,114,80,0, + 0,0,233,254,255,255,255,41,17,114,7,0,0,0,114,64, + 0,0,0,114,65,0,0,0,114,66,0,0,0,114,38,0, + 0,0,114,73,0,0,0,114,71,0,0,0,114,47,0,0, + 0,218,5,99,111,117,110,116,114,34,0,0,0,114,9,0, + 0,0,114,72,0,0,0,114,31,0,0,0,114,70,0,0, + 0,218,9,112,97,114,116,105,116,105,111,110,114,28,0,0, + 0,218,15,83,79,85,82,67,69,95,83,85,70,70,73,88, + 69,83,41,8,114,35,0,0,0,114,76,0,0,0,90,16, + 112,121,99,97,99,104,101,95,102,105,108,101,110,97,109,101, + 90,7,112,121,99,97,99,104,101,90,9,100,111,116,95,99, + 111,117,110,116,114,57,0,0,0,90,9,111,112,116,95,108, + 101,118,101,108,90,13,98,97,115,101,95,102,105,108,101,110, + 97,109,101,114,4,0,0,0,114,4,0,0,0,114,5,0, + 0,0,218,17,115,111,117,114,99,101,95,102,114,111,109,95, + 99,97,99,104,101,31,1,0,0,115,44,0,0,0,0,9, + 18,1,12,1,18,1,18,1,12,1,9,1,15,1,15,1, + 12,1,9,1,15,1,12,1,22,1,15,1,9,1,12,1, + 22,1,12,1,9,1,12,1,19,1,114,85,0,0,0,99, + 1,0,0,0,0,0,0,0,5,0,0,0,12,0,0,0, + 67,0,0,0,115,164,0,0,0,116,0,0,124,0,0,131, + 1,0,100,1,0,107,2,0,114,22,0,100,2,0,83,124, + 0,0,106,1,0,100,3,0,131,1,0,92,3,0,125,1, + 0,125,2,0,125,3,0,124,1,0,12,115,81,0,124,3, + 0,106,2,0,131,0,0,100,7,0,100,8,0,133,2,0, + 25,100,6,0,107,3,0,114,85,0,124,0,0,83,121,16, + 0,116,3,0,124,0,0,131,1,0,125,4,0,87,110,40, + 0,4,116,4,0,116,5,0,102,2,0,107,10,0,114,143, + 0,1,1,1,124,0,0,100,2,0,100,9,0,133,2,0, + 25,125,4,0,89,110,1,0,88,116,6,0,124,4,0,131, + 1,0,114,160,0,124,4,0,83,124,0,0,83,41,10,122, + 188,67,111,110,118,101,114,116,32,97,32,98,121,116,101,99, + 111,100,101,32,102,105,108,101,32,112,97,116,104,32,116,111, + 32,97,32,115,111,117,114,99,101,32,112,97,116,104,32,40, + 105,102,32,112,111,115,115,105,98,108,101,41,46,10,10,32, + 32,32,32,84,104,105,115,32,102,117,110,99,116,105,111,110, + 32,101,120,105,115,116,115,32,112,117,114,101,108,121,32,102, + 111,114,32,98,97,99,107,119,97,114,100,115,45,99,111,109, + 112,97,116,105,98,105,108,105,116,121,32,102,111,114,10,32, + 32,32,32,80,121,73,109,112,111,114,116,95,69,120,101,99, + 67,111,100,101,77,111,100,117,108,101,87,105,116,104,70,105, + 108,101,110,97,109,101,115,40,41,32,105,110,32,116,104,101, + 32,67,32,65,80,73,46,10,10,32,32,32,32,114,59,0, + 0,0,78,114,58,0,0,0,114,80,0,0,0,114,29,0, + 0,0,90,2,112,121,233,253,255,255,255,233,255,255,255,255, + 114,87,0,0,0,41,7,114,31,0,0,0,114,32,0,0, + 0,218,5,108,111,119,101,114,114,85,0,0,0,114,66,0, + 0,0,114,71,0,0,0,114,44,0,0,0,41,5,218,13, + 98,121,116,101,99,111,100,101,95,112,97,116,104,114,78,0, + 0,0,114,36,0,0,0,90,9,101,120,116,101,110,115,105, + 111,110,218,11,115,111,117,114,99,101,95,112,97,116,104,114, + 4,0,0,0,114,4,0,0,0,114,5,0,0,0,218,15, + 95,103,101,116,95,115,111,117,114,99,101,102,105,108,101,64, + 1,0,0,115,20,0,0,0,0,7,18,1,4,1,24,1, + 35,1,4,1,3,1,16,1,19,1,21,1,114,91,0,0, + 0,99,1,0,0,0,0,0,0,0,1,0,0,0,11,0, + 0,0,67,0,0,0,115,92,0,0,0,124,0,0,106,0, + 0,116,1,0,116,2,0,131,1,0,131,1,0,114,59,0, + 121,14,0,116,3,0,124,0,0,131,1,0,83,87,113,88, + 0,4,116,4,0,107,10,0,114,55,0,1,1,1,89,113, + 88,0,88,110,29,0,124,0,0,106,0,0,116,1,0,116, + 5,0,131,1,0,131,1,0,114,84,0,124,0,0,83,100, + 0,0,83,100,0,0,83,41,1,78,41,6,218,8,101,110, + 100,115,119,105,116,104,218,5,116,117,112,108,101,114,84,0, + 0,0,114,79,0,0,0,114,66,0,0,0,114,74,0,0, + 0,41,1,218,8,102,105,108,101,110,97,109,101,114,4,0, + 0,0,114,4,0,0,0,114,5,0,0,0,218,11,95,103, + 101,116,95,99,97,99,104,101,100,83,1,0,0,115,16,0, + 0,0,0,1,21,1,3,1,14,1,13,1,8,1,21,1, + 4,2,114,95,0,0,0,99,1,0,0,0,0,0,0,0, + 2,0,0,0,11,0,0,0,67,0,0,0,115,60,0,0, + 0,121,19,0,116,0,0,124,0,0,131,1,0,106,1,0, + 125,1,0,87,110,24,0,4,116,2,0,107,10,0,114,45, + 0,1,1,1,100,1,0,125,1,0,89,110,1,0,88,124, + 1,0,100,2,0,79,125,1,0,124,1,0,83,41,3,122, + 51,67,97,108,99,117,108,97,116,101,32,116,104,101,32,109, + 111,100,101,32,112,101,114,109,105,115,115,105,111,110,115,32, + 102,111,114,32,97,32,98,121,116,101,99,111,100,101,32,102, + 105,108,101,46,105,182,1,0,0,233,128,0,0,0,41,3, + 114,39,0,0,0,114,41,0,0,0,114,40,0,0,0,41, + 2,114,35,0,0,0,114,42,0,0,0,114,4,0,0,0, + 114,4,0,0,0,114,5,0,0,0,218,10,95,99,97,108, + 99,95,109,111,100,101,95,1,0,0,115,12,0,0,0,0, + 2,3,1,19,1,13,1,11,3,10,1,114,97,0,0,0, + 99,1,0,0,0,0,0,0,0,3,0,0,0,11,0,0, + 0,3,0,0,0,115,84,0,0,0,100,1,0,135,0,0, + 102,1,0,100,2,0,100,3,0,134,1,0,125,1,0,121, + 13,0,116,0,0,106,1,0,125,2,0,87,110,30,0,4, + 116,2,0,107,10,0,114,66,0,1,1,1,100,4,0,100, + 5,0,132,0,0,125,2,0,89,110,1,0,88,124,2,0, + 124,1,0,136,0,0,131,2,0,1,124,1,0,83,41,6, + 122,252,68,101,99,111,114,97,116,111,114,32,116,111,32,118, + 101,114,105,102,121,32,116,104,97,116,32,116,104,101,32,109, + 111,100,117,108,101,32,98,101,105,110,103,32,114,101,113,117, + 101,115,116,101,100,32,109,97,116,99,104,101,115,32,116,104, + 101,32,111,110,101,32,116,104,101,10,32,32,32,32,108,111, + 97,100,101,114,32,99,97,110,32,104,97,110,100,108,101,46, + 10,10,32,32,32,32,84,104,101,32,102,105,114,115,116,32, + 97,114,103,117,109,101,110,116,32,40,115,101,108,102,41,32, + 109,117,115,116,32,100,101,102,105,110,101,32,95,110,97,109, + 101,32,119,104,105,99,104,32,116,104,101,32,115,101,99,111, + 110,100,32,97,114,103,117,109,101,110,116,32,105,115,10,32, + 32,32,32,99,111,109,112,97,114,101,100,32,97,103,97,105, + 110,115,116,46,32,73,102,32,116,104,101,32,99,111,109,112, + 97,114,105,115,111,110,32,102,97,105,108,115,32,116,104,101, + 110,32,73,109,112,111,114,116,69,114,114,111,114,32,105,115, + 32,114,97,105,115,101,100,46,10,10,32,32,32,32,78,99, + 2,0,0,0,0,0,0,0,4,0,0,0,5,0,0,0, + 31,0,0,0,115,89,0,0,0,124,1,0,100,0,0,107, + 8,0,114,24,0,124,0,0,106,0,0,125,1,0,110,46, + 0,124,0,0,106,0,0,124,1,0,107,3,0,114,70,0, + 116,1,0,100,1,0,124,0,0,106,0,0,124,1,0,102, + 2,0,22,100,2,0,124,1,0,131,1,1,130,1,0,136, + 0,0,124,0,0,124,1,0,124,2,0,124,3,0,142,2, + 0,83,41,3,78,122,30,108,111,97,100,101,114,32,102,111, + 114,32,37,115,32,99,97,110,110,111,116,32,104,97,110,100, + 108,101,32,37,115,218,4,110,97,109,101,41,2,114,98,0, + 0,0,218,11,73,109,112,111,114,116,69,114,114,111,114,41, + 4,218,4,115,101,108,102,114,98,0,0,0,218,4,97,114, + 103,115,90,6,107,119,97,114,103,115,41,1,218,6,109,101, 116,104,111,100,114,4,0,0,0,114,5,0,0,0,218,19, 95,99,104,101,99,107,95,110,97,109,101,95,119,114,97,112, - 112,101,114,123,1,0,0,115,12,0,0,0,0,1,12,1, + 112,101,114,115,1,0,0,115,12,0,0,0,0,1,12,1, 12,1,15,1,6,1,25,1,122,40,95,99,104,101,99,107, 95,110,97,109,101,46,60,108,111,99,97,108,115,62,46,95, 99,104,101,99,107,95,110,97,109,101,95,119,114,97,112,112, @@ -595,17 +573,17 @@ 116,97,116,116,114,218,8,95,95,100,105,99,116,95,95,218, 6,117,112,100,97,116,101,41,3,90,3,110,101,119,90,3, 111,108,100,114,52,0,0,0,114,4,0,0,0,114,4,0, - 0,0,114,5,0,0,0,218,5,95,119,114,97,112,134,1, + 0,0,114,5,0,0,0,218,5,95,119,114,97,112,126,1, 0,0,115,8,0,0,0,0,1,25,1,15,1,29,1,122, 26,95,99,104,101,99,107,95,110,97,109,101,46,60,108,111, 99,97,108,115,62,46,95,119,114,97,112,41,3,218,10,95, - 98,111,111,116,115,116,114,97,112,114,120,0,0,0,218,9, - 78,97,109,101,69,114,114,111,114,41,3,114,109,0,0,0, - 114,110,0,0,0,114,120,0,0,0,114,4,0,0,0,41, - 1,114,109,0,0,0,114,5,0,0,0,218,11,95,99,104, - 101,99,107,95,110,97,109,101,115,1,0,0,115,14,0,0, + 98,111,111,116,115,116,114,97,112,114,113,0,0,0,218,9, + 78,97,109,101,69,114,114,111,114,41,3,114,102,0,0,0, + 114,103,0,0,0,114,113,0,0,0,114,4,0,0,0,41, + 1,114,102,0,0,0,114,5,0,0,0,218,11,95,99,104, + 101,99,107,95,110,97,109,101,107,1,0,0,115,14,0,0, 0,0,8,21,7,3,1,13,1,13,2,17,5,13,1,114, - 123,0,0,0,99,2,0,0,0,0,0,0,0,5,0,0, + 116,0,0,0,99,2,0,0,0,0,0,0,0,5,0,0, 0,4,0,0,0,67,0,0,0,115,84,0,0,0,124,0, 0,106,0,0,124,1,0,131,1,0,92,2,0,125,2,0, 125,3,0,124,2,0,100,1,0,107,8,0,114,80,0,116, @@ -628,14 +606,14 @@ 114,59,0,0,0,41,6,218,11,102,105,110,100,95,108,111, 97,100,101,114,114,31,0,0,0,114,60,0,0,0,114,61, 0,0,0,114,47,0,0,0,218,13,73,109,112,111,114,116, - 87,97,114,110,105,110,103,41,5,114,108,0,0,0,218,8, + 87,97,114,110,105,110,103,41,5,114,100,0,0,0,218,8, 102,117,108,108,110,97,109,101,218,6,108,111,97,100,101,114, 218,8,112,111,114,116,105,111,110,115,218,3,109,115,103,114, 4,0,0,0,114,4,0,0,0,114,5,0,0,0,218,17, 95,102,105,110,100,95,109,111,100,117,108,101,95,115,104,105, - 109,143,1,0,0,115,10,0,0,0,0,10,21,1,24,1, - 6,1,29,1,114,130,0,0,0,99,4,0,0,0,0,0, - 0,0,11,0,0,0,19,0,0,0,67,0,0,0,115,228, + 109,135,1,0,0,115,10,0,0,0,0,10,21,1,24,1, + 6,1,29,1,114,123,0,0,0,99,4,0,0,0,0,0, + 0,0,11,0,0,0,19,0,0,0,67,0,0,0,115,240, 1,0,0,105,0,0,125,4,0,124,2,0,100,1,0,107, 9,0,114,31,0,124,2,0,124,4,0,100,2,0,60,110, 6,0,100,3,0,125,2,0,124,3,0,100,1,0,107,9, @@ -643,1954 +621,1959 @@ 0,100,1,0,100,5,0,133,2,0,25,125,5,0,124,0, 0,100,5,0,100,6,0,133,2,0,25,125,6,0,124,0, 0,100,6,0,100,7,0,133,2,0,25,125,7,0,124,5, - 0,116,0,0,107,3,0,114,165,0,100,8,0,106,1,0, - 124,2,0,124,5,0,131,2,0,125,8,0,116,2,0,124, - 8,0,131,1,0,1,116,3,0,124,8,0,124,4,0,141, - 1,0,130,1,0,110,113,0,116,4,0,124,6,0,131,1, - 0,100,5,0,107,3,0,114,223,0,100,9,0,106,1,0, - 124,2,0,131,1,0,125,8,0,116,2,0,124,8,0,131, - 1,0,1,116,5,0,124,8,0,131,1,0,130,1,0,110, - 55,0,116,4,0,124,7,0,131,1,0,100,5,0,107,3, - 0,114,22,1,100,10,0,106,1,0,124,2,0,131,1,0, - 125,8,0,116,2,0,124,8,0,131,1,0,1,116,5,0, - 124,8,0,131,1,0,130,1,0,124,1,0,100,1,0,107, - 9,0,114,214,1,121,20,0,116,6,0,124,1,0,100,11, - 0,25,131,1,0,125,9,0,87,110,18,0,4,116,7,0, - 107,10,0,114,74,1,1,1,1,89,110,59,0,88,116,8, - 0,124,6,0,131,1,0,124,9,0,107,3,0,114,133,1, - 100,12,0,106,1,0,124,2,0,131,1,0,125,8,0,116, - 2,0,124,8,0,131,1,0,1,116,3,0,124,8,0,124, - 4,0,141,1,0,130,1,0,121,18,0,124,1,0,100,13, - 0,25,100,14,0,64,125,10,0,87,110,18,0,4,116,7, - 0,107,10,0,114,171,1,1,1,1,89,110,43,0,88,116, - 8,0,124,7,0,131,1,0,124,10,0,107,3,0,114,214, - 1,116,3,0,100,12,0,106,1,0,124,2,0,131,1,0, - 124,4,0,141,1,0,130,1,0,124,0,0,100,7,0,100, - 1,0,133,2,0,25,83,41,15,97,122,1,0,0,86,97, - 108,105,100,97,116,101,32,116,104,101,32,104,101,97,100,101, - 114,32,111,102,32,116,104,101,32,112,97,115,115,101,100,45, - 105,110,32,98,121,116,101,99,111,100,101,32,97,103,97,105, - 110,115,116,32,115,111,117,114,99,101,95,115,116,97,116,115, - 32,40,105,102,10,32,32,32,32,103,105,118,101,110,41,32, - 97,110,100,32,114,101,116,117,114,110,105,110,103,32,116,104, - 101,32,98,121,116,101,99,111,100,101,32,116,104,97,116,32, - 99,97,110,32,98,101,32,99,111,109,112,105,108,101,100,32, - 98,121,32,99,111,109,112,105,108,101,40,41,46,10,10,32, - 32,32,32,65,108,108,32,111,116,104,101,114,32,97,114,103, - 117,109,101,110,116,115,32,97,114,101,32,117,115,101,100,32, - 116,111,32,101,110,104,97,110,99,101,32,101,114,114,111,114, - 32,114,101,112,111,114,116,105,110,103,46,10,10,32,32,32, - 32,73,109,112,111,114,116,69,114,114,111,114,32,105,115,32, - 114,97,105,115,101,100,32,119,104,101,110,32,116,104,101,32, - 109,97,103,105,99,32,110,117,109,98,101,114,32,105,115,32, - 105,110,99,111,114,114,101,99,116,32,111,114,32,116,104,101, - 32,98,121,116,101,99,111,100,101,32,105,115,10,32,32,32, - 32,102,111,117,110,100,32,116,111,32,98,101,32,115,116,97, - 108,101,46,32,69,79,70,69,114,114,111,114,32,105,115,32, - 114,97,105,115,101,100,32,119,104,101,110,32,116,104,101,32, - 100,97,116,97,32,105,115,32,102,111,117,110,100,32,116,111, - 32,98,101,10,32,32,32,32,116,114,117,110,99,97,116,101, - 100,46,10,10,32,32,32,32,78,114,106,0,0,0,122,10, - 60,98,121,116,101,99,111,100,101,62,114,35,0,0,0,114, - 12,0,0,0,233,8,0,0,0,233,12,0,0,0,122,30, - 98,97,100,32,109,97,103,105,99,32,110,117,109,98,101,114, - 32,105,110,32,123,33,114,125,58,32,123,33,114,125,122,43, - 114,101,97,99,104,101,100,32,69,79,70,32,119,104,105,108, - 101,32,114,101,97,100,105,110,103,32,116,105,109,101,115,116, - 97,109,112,32,105,110,32,123,33,114,125,122,48,114,101,97, - 99,104,101,100,32,69,79,70,32,119,104,105,108,101,32,114, - 101,97,100,105,110,103,32,115,105,122,101,32,111,102,32,115, - 111,117,114,99,101,32,105,110,32,123,33,114,125,218,5,109, - 116,105,109,101,122,26,98,121,116,101,99,111,100,101,32,105, - 115,32,115,116,97,108,101,32,102,111,114,32,123,33,114,125, - 218,4,115,105,122,101,108,3,0,0,0,255,127,255,127,3, - 0,41,9,218,12,77,65,71,73,67,95,78,85,77,66,69, - 82,114,47,0,0,0,114,105,0,0,0,114,107,0,0,0, - 114,31,0,0,0,218,8,69,79,70,69,114,114,111,114,114, - 14,0,0,0,218,8,75,101,121,69,114,114,111,114,114,19, - 0,0,0,41,11,114,53,0,0,0,218,12,115,111,117,114, - 99,101,95,115,116,97,116,115,114,106,0,0,0,114,35,0, - 0,0,90,11,101,120,99,95,100,101,116,97,105,108,115,90, - 5,109,97,103,105,99,90,13,114,97,119,95,116,105,109,101, - 115,116,97,109,112,90,8,114,97,119,95,115,105,122,101,114, - 75,0,0,0,218,12,115,111,117,114,99,101,95,109,116,105, - 109,101,218,11,115,111,117,114,99,101,95,115,105,122,101,114, - 4,0,0,0,114,4,0,0,0,114,5,0,0,0,218,25, - 95,118,97,108,105,100,97,116,101,95,98,121,116,101,99,111, - 100,101,95,104,101,97,100,101,114,160,1,0,0,115,76,0, - 0,0,0,11,6,1,12,1,13,3,6,1,12,1,10,1, - 16,1,16,1,16,1,12,1,18,1,10,1,18,1,18,1, - 15,1,10,1,15,1,18,1,15,1,10,1,12,1,12,1, - 3,1,20,1,13,1,5,2,18,1,15,1,10,1,15,1, - 3,1,18,1,13,1,5,2,18,1,15,1,9,1,114,141, - 0,0,0,99,4,0,0,0,0,0,0,0,5,0,0,0, - 6,0,0,0,67,0,0,0,115,112,0,0,0,116,0,0, - 106,1,0,124,0,0,131,1,0,125,4,0,116,2,0,124, - 4,0,116,3,0,131,2,0,114,75,0,116,4,0,100,1, - 0,124,2,0,131,2,0,1,124,3,0,100,2,0,107,9, - 0,114,71,0,116,5,0,106,6,0,124,4,0,124,3,0, - 131,2,0,1,124,4,0,83,116,7,0,100,3,0,106,8, - 0,124,2,0,131,1,0,100,4,0,124,1,0,100,5,0, - 124,2,0,131,1,2,130,1,0,100,2,0,83,41,6,122, - 60,67,111,109,112,105,108,101,32,98,121,116,101,99,111,100, - 101,32,97,115,32,114,101,116,117,114,110,101,100,32,98,121, - 32,95,118,97,108,105,100,97,116,101,95,98,121,116,101,99, - 111,100,101,95,104,101,97,100,101,114,40,41,46,122,21,99, - 111,100,101,32,111,98,106,101,99,116,32,102,114,111,109,32, - 123,33,114,125,78,122,23,78,111,110,45,99,111,100,101,32, - 111,98,106,101,99,116,32,105,110,32,123,33,114,125,114,106, - 0,0,0,114,35,0,0,0,41,9,218,7,109,97,114,115, - 104,97,108,90,5,108,111,97,100,115,218,10,105,115,105,110, - 115,116,97,110,99,101,218,10,95,99,111,100,101,95,116,121, - 112,101,114,105,0,0,0,218,4,95,105,109,112,90,16,95, - 102,105,120,95,99,111,95,102,105,108,101,110,97,109,101,114, - 107,0,0,0,114,47,0,0,0,41,5,114,53,0,0,0, - 114,106,0,0,0,114,89,0,0,0,114,90,0,0,0,218, - 4,99,111,100,101,114,4,0,0,0,114,4,0,0,0,114, - 5,0,0,0,218,17,95,99,111,109,112,105,108,101,95,98, - 121,116,101,99,111,100,101,215,1,0,0,115,16,0,0,0, - 0,2,15,1,15,1,13,1,12,1,16,1,4,2,18,1, - 114,147,0,0,0,114,59,0,0,0,99,3,0,0,0,0, - 0,0,0,4,0,0,0,3,0,0,0,67,0,0,0,115, - 76,0,0,0,116,0,0,116,1,0,131,1,0,125,3,0, - 124,3,0,106,2,0,116,3,0,124,1,0,131,1,0,131, - 1,0,1,124,3,0,106,2,0,116,3,0,124,2,0,131, - 1,0,131,1,0,1,124,3,0,106,2,0,116,4,0,106, - 5,0,124,0,0,131,1,0,131,1,0,1,124,3,0,83, - 41,1,122,80,67,111,109,112,105,108,101,32,97,32,99,111, - 100,101,32,111,98,106,101,99,116,32,105,110,116,111,32,98, - 121,116,101,99,111,100,101,32,102,111,114,32,119,114,105,116, - 105,110,103,32,111,117,116,32,116,111,32,97,32,98,121,116, - 101,45,99,111,109,112,105,108,101,100,10,32,32,32,32,102, - 105,108,101,46,41,6,218,9,98,121,116,101,97,114,114,97, - 121,114,135,0,0,0,218,6,101,120,116,101,110,100,114,17, - 0,0,0,114,142,0,0,0,90,5,100,117,109,112,115,41, - 4,114,146,0,0,0,114,133,0,0,0,114,140,0,0,0, - 114,53,0,0,0,114,4,0,0,0,114,4,0,0,0,114, - 5,0,0,0,218,17,95,99,111,100,101,95,116,111,95,98, - 121,116,101,99,111,100,101,227,1,0,0,115,10,0,0,0, - 0,3,12,1,19,1,19,1,22,1,114,150,0,0,0,99, - 1,0,0,0,0,0,0,0,5,0,0,0,4,0,0,0, - 67,0,0,0,115,89,0,0,0,100,1,0,100,2,0,108, - 0,0,125,1,0,116,1,0,106,2,0,124,0,0,131,1, - 0,106,3,0,125,2,0,124,1,0,106,4,0,124,2,0, - 131,1,0,125,3,0,116,1,0,106,5,0,100,2,0,100, - 3,0,131,2,0,125,4,0,124,4,0,106,6,0,124,0, - 0,106,6,0,124,3,0,100,1,0,25,131,1,0,131,1, - 0,83,41,4,122,121,68,101,99,111,100,101,32,98,121,116, - 101,115,32,114,101,112,114,101,115,101,110,116,105,110,103,32, - 115,111,117,114,99,101,32,99,111,100,101,32,97,110,100,32, - 114,101,116,117,114,110,32,116,104,101,32,115,116,114,105,110, - 103,46,10,10,32,32,32,32,85,110,105,118,101,114,115,97, - 108,32,110,101,119,108,105,110,101,32,115,117,112,112,111,114, - 116,32,105,115,32,117,115,101,100,32,105,110,32,116,104,101, - 32,100,101,99,111,100,105,110,103,46,10,32,32,32,32,114, - 59,0,0,0,78,84,41,7,218,8,116,111,107,101,110,105, - 122,101,114,49,0,0,0,90,7,66,121,116,101,115,73,79, - 90,8,114,101,97,100,108,105,110,101,90,15,100,101,116,101, - 99,116,95,101,110,99,111,100,105,110,103,90,25,73,110,99, - 114,101,109,101,110,116,97,108,78,101,119,108,105,110,101,68, - 101,99,111,100,101,114,218,6,100,101,99,111,100,101,41,5, - 218,12,115,111,117,114,99,101,95,98,121,116,101,115,114,151, - 0,0,0,90,21,115,111,117,114,99,101,95,98,121,116,101, - 115,95,114,101,97,100,108,105,110,101,218,8,101,110,99,111, - 100,105,110,103,90,15,110,101,119,108,105,110,101,95,100,101, - 99,111,100,101,114,114,4,0,0,0,114,4,0,0,0,114, - 5,0,0,0,218,13,100,101,99,111,100,101,95,115,111,117, - 114,99,101,237,1,0,0,115,10,0,0,0,0,5,12,1, - 18,1,15,1,18,1,114,155,0,0,0,114,127,0,0,0, - 218,26,115,117,98,109,111,100,117,108,101,95,115,101,97,114, - 99,104,95,108,111,99,97,116,105,111,110,115,99,2,0,0, - 0,2,0,0,0,9,0,0,0,19,0,0,0,67,0,0, - 0,115,89,1,0,0,124,1,0,100,1,0,107,8,0,114, - 73,0,100,2,0,125,1,0,116,0,0,124,2,0,100,3, - 0,131,2,0,114,73,0,121,19,0,124,2,0,106,1,0, - 124,0,0,131,1,0,125,1,0,87,110,18,0,4,116,2, - 0,107,10,0,114,72,0,1,1,1,89,110,1,0,88,116, - 3,0,106,4,0,124,0,0,124,2,0,100,4,0,124,1, - 0,131,2,1,125,4,0,100,5,0,124,4,0,95,5,0, - 124,2,0,100,1,0,107,8,0,114,194,0,120,73,0,116, - 6,0,131,0,0,68,93,58,0,92,2,0,125,5,0,125, - 6,0,124,1,0,106,7,0,116,8,0,124,6,0,131,1, - 0,131,1,0,114,128,0,124,5,0,124,0,0,124,1,0, - 131,2,0,125,2,0,124,2,0,124,4,0,95,9,0,80, - 113,128,0,87,100,1,0,83,124,3,0,116,10,0,107,8, - 0,114,23,1,116,0,0,124,2,0,100,6,0,131,2,0, - 114,32,1,121,19,0,124,2,0,106,11,0,124,0,0,131, - 1,0,125,7,0,87,110,18,0,4,116,2,0,107,10,0, - 114,4,1,1,1,1,89,113,32,1,88,124,7,0,114,32, - 1,103,0,0,124,4,0,95,12,0,110,9,0,124,3,0, - 124,4,0,95,12,0,124,4,0,106,12,0,103,0,0,107, - 2,0,114,85,1,124,1,0,114,85,1,116,13,0,124,1, - 0,131,1,0,100,7,0,25,125,8,0,124,4,0,106,12, - 0,106,14,0,124,8,0,131,1,0,1,124,4,0,83,41, - 8,97,61,1,0,0,82,101,116,117,114,110,32,97,32,109, - 111,100,117,108,101,32,115,112,101,99,32,98,97,115,101,100, - 32,111,110,32,97,32,102,105,108,101,32,108,111,99,97,116, - 105,111,110,46,10,10,32,32,32,32,84,111,32,105,110,100, - 105,99,97,116,101,32,116,104,97,116,32,116,104,101,32,109, - 111,100,117,108,101,32,105,115,32,97,32,112,97,99,107,97, - 103,101,44,32,115,101,116,10,32,32,32,32,115,117,98,109, - 111,100,117,108,101,95,115,101,97,114,99,104,95,108,111,99, - 97,116,105,111,110,115,32,116,111,32,97,32,108,105,115,116, - 32,111,102,32,100,105,114,101,99,116,111,114,121,32,112,97, - 116,104,115,46,32,32,65,110,10,32,32,32,32,101,109,112, - 116,121,32,108,105,115,116,32,105,115,32,115,117,102,102,105, - 99,105,101,110,116,44,32,116,104,111,117,103,104,32,105,116, - 115,32,110,111,116,32,111,116,104,101,114,119,105,115,101,32, - 117,115,101,102,117,108,32,116,111,32,116,104,101,10,32,32, - 32,32,105,109,112,111,114,116,32,115,121,115,116,101,109,46, - 10,10,32,32,32,32,84,104,101,32,108,111,97,100,101,114, - 32,109,117,115,116,32,116,97,107,101,32,97,32,115,112,101, - 99,32,97,115,32,105,116,115,32,111,110,108,121,32,95,95, - 105,110,105,116,95,95,40,41,32,97,114,103,46,10,10,32, - 32,32,32,78,122,9,60,117,110,107,110,111,119,110,62,218, - 12,103,101,116,95,102,105,108,101,110,97,109,101,218,6,111, - 114,105,103,105,110,84,218,10,105,115,95,112,97,99,107,97, - 103,101,114,59,0,0,0,41,15,114,115,0,0,0,114,157, - 0,0,0,114,107,0,0,0,114,121,0,0,0,218,10,77, - 111,100,117,108,101,83,112,101,99,90,13,95,115,101,116,95, - 102,105,108,101,97,116,116,114,218,27,95,103,101,116,95,115, - 117,112,112,111,114,116,101,100,95,102,105,108,101,95,108,111, - 97,100,101,114,115,114,92,0,0,0,114,93,0,0,0,114, - 127,0,0,0,218,9,95,80,79,80,85,76,65,84,69,114, - 159,0,0,0,114,156,0,0,0,114,38,0,0,0,218,6, - 97,112,112,101,110,100,41,9,114,106,0,0,0,90,8,108, - 111,99,97,116,105,111,110,114,127,0,0,0,114,156,0,0, - 0,218,4,115,112,101,99,218,12,108,111,97,100,101,114,95, - 99,108,97,115,115,218,8,115,117,102,102,105,120,101,115,114, - 159,0,0,0,90,7,100,105,114,110,97,109,101,114,4,0, - 0,0,114,4,0,0,0,114,5,0,0,0,218,23,115,112, - 101,99,95,102,114,111,109,95,102,105,108,101,95,108,111,99, - 97,116,105,111,110,254,1,0,0,115,60,0,0,0,0,12, - 12,4,6,1,15,2,3,1,19,1,13,1,5,8,24,1, - 9,3,12,1,22,1,21,1,15,1,9,1,5,2,4,3, - 12,2,15,1,3,1,19,1,13,1,5,2,6,1,12,2, - 9,1,15,1,6,1,16,1,16,2,114,167,0,0,0,99, - 0,0,0,0,0,0,0,0,0,0,0,0,5,0,0,0, - 64,0,0,0,115,121,0,0,0,101,0,0,90,1,0,100, - 0,0,90,2,0,100,1,0,90,3,0,100,2,0,90,4, - 0,100,3,0,90,5,0,100,4,0,90,6,0,101,7,0, - 100,5,0,100,6,0,132,0,0,131,1,0,90,8,0,101, - 7,0,100,7,0,100,8,0,132,0,0,131,1,0,90,9, - 0,101,7,0,100,9,0,100,9,0,100,10,0,100,11,0, - 132,2,0,131,1,0,90,10,0,101,7,0,100,9,0,100, - 12,0,100,13,0,132,1,0,131,1,0,90,11,0,100,9, - 0,83,41,14,218,21,87,105,110,100,111,119,115,82,101,103, - 105,115,116,114,121,70,105,110,100,101,114,122,62,77,101,116, - 97,32,112,97,116,104,32,102,105,110,100,101,114,32,102,111, - 114,32,109,111,100,117,108,101,115,32,100,101,99,108,97,114, - 101,100,32,105,110,32,116,104,101,32,87,105,110,100,111,119, - 115,32,114,101,103,105,115,116,114,121,46,122,59,83,111,102, - 116,119,97,114,101,92,80,121,116,104,111,110,92,80,121,116, - 104,111,110,67,111,114,101,92,123,115,121,115,95,118,101,114, - 115,105,111,110,125,92,77,111,100,117,108,101,115,92,123,102, - 117,108,108,110,97,109,101,125,122,65,83,111,102,116,119,97, - 114,101,92,80,121,116,104,111,110,92,80,121,116,104,111,110, - 67,111,114,101,92,123,115,121,115,95,118,101,114,115,105,111, - 110,125,92,77,111,100,117,108,101,115,92,123,102,117,108,108, - 110,97,109,101,125,92,68,101,98,117,103,70,99,2,0,0, - 0,0,0,0,0,2,0,0,0,11,0,0,0,67,0,0, - 0,115,67,0,0,0,121,23,0,116,0,0,106,1,0,116, - 0,0,106,2,0,124,1,0,131,2,0,83,87,110,37,0, - 4,116,3,0,107,10,0,114,62,0,1,1,1,116,0,0, - 106,1,0,116,0,0,106,4,0,124,1,0,131,2,0,83, - 89,110,1,0,88,100,0,0,83,41,1,78,41,5,218,7, - 95,119,105,110,114,101,103,90,7,79,112,101,110,75,101,121, - 90,17,72,75,69,89,95,67,85,82,82,69,78,84,95,85, - 83,69,82,114,40,0,0,0,90,18,72,75,69,89,95,76, - 79,67,65,76,95,77,65,67,72,73,78,69,41,2,218,3, - 99,108,115,218,3,107,101,121,114,4,0,0,0,114,4,0, - 0,0,114,5,0,0,0,218,14,95,111,112,101,110,95,114, - 101,103,105,115,116,114,121,76,2,0,0,115,8,0,0,0, - 0,2,3,1,23,1,13,1,122,36,87,105,110,100,111,119, - 115,82,101,103,105,115,116,114,121,70,105,110,100,101,114,46, - 95,111,112,101,110,95,114,101,103,105,115,116,114,121,99,2, - 0,0,0,0,0,0,0,6,0,0,0,16,0,0,0,67, - 0,0,0,115,143,0,0,0,124,0,0,106,0,0,114,21, - 0,124,0,0,106,1,0,125,2,0,110,9,0,124,0,0, - 106,2,0,125,2,0,124,2,0,106,3,0,100,1,0,124, - 1,0,100,2,0,116,4,0,106,5,0,100,0,0,100,3, - 0,133,2,0,25,131,0,2,125,3,0,121,47,0,124,0, - 0,106,6,0,124,3,0,131,1,0,143,25,0,125,4,0, - 116,7,0,106,8,0,124,4,0,100,4,0,131,2,0,125, - 5,0,87,100,0,0,81,82,88,87,110,22,0,4,116,9, - 0,107,10,0,114,138,0,1,1,1,100,0,0,83,89,110, - 1,0,88,124,5,0,83,41,5,78,114,126,0,0,0,90, - 11,115,121,115,95,118,101,114,115,105,111,110,114,80,0,0, - 0,114,30,0,0,0,41,10,218,11,68,69,66,85,71,95, - 66,85,73,76,68,218,18,82,69,71,73,83,84,82,89,95, - 75,69,89,95,68,69,66,85,71,218,12,82,69,71,73,83, - 84,82,89,95,75,69,89,114,47,0,0,0,114,7,0,0, - 0,218,7,118,101,114,115,105,111,110,114,172,0,0,0,114, - 169,0,0,0,90,10,81,117,101,114,121,86,97,108,117,101, - 114,40,0,0,0,41,6,114,170,0,0,0,114,126,0,0, - 0,90,12,114,101,103,105,115,116,114,121,95,107,101,121,114, - 171,0,0,0,90,4,104,107,101,121,218,8,102,105,108,101, - 112,97,116,104,114,4,0,0,0,114,4,0,0,0,114,5, - 0,0,0,218,16,95,115,101,97,114,99,104,95,114,101,103, - 105,115,116,114,121,83,2,0,0,115,22,0,0,0,0,2, - 9,1,12,2,9,1,15,1,22,1,3,1,18,1,29,1, - 13,1,9,1,122,38,87,105,110,100,111,119,115,82,101,103, - 105,115,116,114,121,70,105,110,100,101,114,46,95,115,101,97, - 114,99,104,95,114,101,103,105,115,116,114,121,78,99,4,0, - 0,0,0,0,0,0,8,0,0,0,14,0,0,0,67,0, - 0,0,115,158,0,0,0,124,0,0,106,0,0,124,1,0, - 131,1,0,125,4,0,124,4,0,100,0,0,107,8,0,114, - 31,0,100,0,0,83,121,14,0,116,1,0,124,4,0,131, - 1,0,1,87,110,22,0,4,116,2,0,107,10,0,114,69, - 0,1,1,1,100,0,0,83,89,110,1,0,88,120,81,0, - 116,3,0,131,0,0,68,93,70,0,92,2,0,125,5,0, - 125,6,0,124,4,0,106,4,0,116,5,0,124,6,0,131, - 1,0,131,1,0,114,80,0,116,6,0,106,7,0,124,1, - 0,124,5,0,124,1,0,124,4,0,131,2,0,100,1,0, - 124,4,0,131,2,1,125,7,0,124,7,0,83,113,80,0, - 87,100,0,0,83,41,2,78,114,158,0,0,0,41,8,114, - 178,0,0,0,114,39,0,0,0,114,40,0,0,0,114,161, - 0,0,0,114,92,0,0,0,114,93,0,0,0,114,121,0, - 0,0,218,16,115,112,101,99,95,102,114,111,109,95,108,111, - 97,100,101,114,41,8,114,170,0,0,0,114,126,0,0,0, - 114,35,0,0,0,218,6,116,97,114,103,101,116,114,177,0, - 0,0,114,127,0,0,0,114,166,0,0,0,114,164,0,0, + 0,116,0,0,107,3,0,114,168,0,100,8,0,106,1,0, + 124,2,0,124,5,0,131,2,0,125,8,0,116,2,0,106, + 3,0,124,8,0,131,1,0,1,116,4,0,124,8,0,124, + 4,0,141,1,0,130,1,0,110,119,0,116,5,0,124,6, + 0,131,1,0,100,5,0,107,3,0,114,229,0,100,9,0, + 106,1,0,124,2,0,131,1,0,125,8,0,116,2,0,106, + 3,0,124,8,0,131,1,0,1,116,6,0,124,8,0,131, + 1,0,130,1,0,110,58,0,116,5,0,124,7,0,131,1, + 0,100,5,0,107,3,0,114,31,1,100,10,0,106,1,0, + 124,2,0,131,1,0,125,8,0,116,2,0,106,3,0,124, + 8,0,131,1,0,1,116,6,0,124,8,0,131,1,0,130, + 1,0,124,1,0,100,1,0,107,9,0,114,226,1,121,20, + 0,116,7,0,124,1,0,100,11,0,25,131,1,0,125,9, + 0,87,110,18,0,4,116,8,0,107,10,0,114,83,1,1, + 1,1,89,110,62,0,88,116,9,0,124,6,0,131,1,0, + 124,9,0,107,3,0,114,145,1,100,12,0,106,1,0,124, + 2,0,131,1,0,125,8,0,116,2,0,106,3,0,124,8, + 0,131,1,0,1,116,4,0,124,8,0,124,4,0,141,1, + 0,130,1,0,121,18,0,124,1,0,100,13,0,25,100,14, + 0,64,125,10,0,87,110,18,0,4,116,8,0,107,10,0, + 114,183,1,1,1,1,89,110,43,0,88,116,9,0,124,7, + 0,131,1,0,124,10,0,107,3,0,114,226,1,116,4,0, + 100,12,0,106,1,0,124,2,0,131,1,0,124,4,0,141, + 1,0,130,1,0,124,0,0,100,7,0,100,1,0,133,2, + 0,25,83,41,15,97,122,1,0,0,86,97,108,105,100,97, + 116,101,32,116,104,101,32,104,101,97,100,101,114,32,111,102, + 32,116,104,101,32,112,97,115,115,101,100,45,105,110,32,98, + 121,116,101,99,111,100,101,32,97,103,97,105,110,115,116,32, + 115,111,117,114,99,101,95,115,116,97,116,115,32,40,105,102, + 10,32,32,32,32,103,105,118,101,110,41,32,97,110,100,32, + 114,101,116,117,114,110,105,110,103,32,116,104,101,32,98,121, + 116,101,99,111,100,101,32,116,104,97,116,32,99,97,110,32, + 98,101,32,99,111,109,112,105,108,101,100,32,98,121,32,99, + 111,109,112,105,108,101,40,41,46,10,10,32,32,32,32,65, + 108,108,32,111,116,104,101,114,32,97,114,103,117,109,101,110, + 116,115,32,97,114,101,32,117,115,101,100,32,116,111,32,101, + 110,104,97,110,99,101,32,101,114,114,111,114,32,114,101,112, + 111,114,116,105,110,103,46,10,10,32,32,32,32,73,109,112, + 111,114,116,69,114,114,111,114,32,105,115,32,114,97,105,115, + 101,100,32,119,104,101,110,32,116,104,101,32,109,97,103,105, + 99,32,110,117,109,98,101,114,32,105,115,32,105,110,99,111, + 114,114,101,99,116,32,111,114,32,116,104,101,32,98,121,116, + 101,99,111,100,101,32,105,115,10,32,32,32,32,102,111,117, + 110,100,32,116,111,32,98,101,32,115,116,97,108,101,46,32, + 69,79,70,69,114,114,111,114,32,105,115,32,114,97,105,115, + 101,100,32,119,104,101,110,32,116,104,101,32,100,97,116,97, + 32,105,115,32,102,111,117,110,100,32,116,111,32,98,101,10, + 32,32,32,32,116,114,117,110,99,97,116,101,100,46,10,10, + 32,32,32,32,78,114,98,0,0,0,122,10,60,98,121,116, + 101,99,111,100,101,62,114,35,0,0,0,114,12,0,0,0, + 233,8,0,0,0,233,12,0,0,0,122,30,98,97,100,32, + 109,97,103,105,99,32,110,117,109,98,101,114,32,105,110,32, + 123,33,114,125,58,32,123,33,114,125,122,43,114,101,97,99, + 104,101,100,32,69,79,70,32,119,104,105,108,101,32,114,101, + 97,100,105,110,103,32,116,105,109,101,115,116,97,109,112,32, + 105,110,32,123,33,114,125,122,48,114,101,97,99,104,101,100, + 32,69,79,70,32,119,104,105,108,101,32,114,101,97,100,105, + 110,103,32,115,105,122,101,32,111,102,32,115,111,117,114,99, + 101,32,105,110,32,123,33,114,125,218,5,109,116,105,109,101, + 122,26,98,121,116,101,99,111,100,101,32,105,115,32,115,116, + 97,108,101,32,102,111,114,32,123,33,114,125,218,4,115,105, + 122,101,108,3,0,0,0,255,127,255,127,3,0,41,10,218, + 12,77,65,71,73,67,95,78,85,77,66,69,82,114,47,0, + 0,0,114,114,0,0,0,218,16,95,118,101,114,98,111,115, + 101,95,109,101,115,115,97,103,101,114,99,0,0,0,114,31, + 0,0,0,218,8,69,79,70,69,114,114,111,114,114,14,0, + 0,0,218,8,75,101,121,69,114,114,111,114,114,19,0,0, + 0,41,11,114,53,0,0,0,218,12,115,111,117,114,99,101, + 95,115,116,97,116,115,114,98,0,0,0,114,35,0,0,0, + 90,11,101,120,99,95,100,101,116,97,105,108,115,90,5,109, + 97,103,105,99,90,13,114,97,119,95,116,105,109,101,115,116, + 97,109,112,90,8,114,97,119,95,115,105,122,101,114,75,0, + 0,0,218,12,115,111,117,114,99,101,95,109,116,105,109,101, + 218,11,115,111,117,114,99,101,95,115,105,122,101,114,4,0, + 0,0,114,4,0,0,0,114,5,0,0,0,218,25,95,118, + 97,108,105,100,97,116,101,95,98,121,116,101,99,111,100,101, + 95,104,101,97,100,101,114,152,1,0,0,115,76,0,0,0, + 0,11,6,1,12,1,13,3,6,1,12,1,10,1,16,1, + 16,1,16,1,12,1,18,1,13,1,18,1,18,1,15,1, + 13,1,15,1,18,1,15,1,13,1,12,1,12,1,3,1, + 20,1,13,1,5,2,18,1,15,1,13,1,15,1,3,1, + 18,1,13,1,5,2,18,1,15,1,9,1,114,135,0,0, + 0,99,4,0,0,0,0,0,0,0,5,0,0,0,6,0, + 0,0,67,0,0,0,115,115,0,0,0,116,0,0,106,1, + 0,124,0,0,131,1,0,125,4,0,116,2,0,124,4,0, + 116,3,0,131,2,0,114,78,0,116,4,0,106,5,0,100, + 1,0,124,2,0,131,2,0,1,124,3,0,100,2,0,107, + 9,0,114,74,0,116,6,0,106,7,0,124,4,0,124,3, + 0,131,2,0,1,124,4,0,83,116,8,0,100,3,0,106, + 9,0,124,2,0,131,1,0,100,4,0,124,1,0,100,5, + 0,124,2,0,131,1,2,130,1,0,100,2,0,83,41,6, + 122,60,67,111,109,112,105,108,101,32,98,121,116,101,99,111, + 100,101,32,97,115,32,114,101,116,117,114,110,101,100,32,98, + 121,32,95,118,97,108,105,100,97,116,101,95,98,121,116,101, + 99,111,100,101,95,104,101,97,100,101,114,40,41,46,122,21, + 99,111,100,101,32,111,98,106,101,99,116,32,102,114,111,109, + 32,123,33,114,125,78,122,23,78,111,110,45,99,111,100,101, + 32,111,98,106,101,99,116,32,105,110,32,123,33,114,125,114, + 98,0,0,0,114,35,0,0,0,41,10,218,7,109,97,114, + 115,104,97,108,90,5,108,111,97,100,115,218,10,105,115,105, + 110,115,116,97,110,99,101,218,10,95,99,111,100,101,95,116, + 121,112,101,114,114,0,0,0,114,129,0,0,0,218,4,95, + 105,109,112,90,16,95,102,105,120,95,99,111,95,102,105,108, + 101,110,97,109,101,114,99,0,0,0,114,47,0,0,0,41, + 5,114,53,0,0,0,114,98,0,0,0,114,89,0,0,0, + 114,90,0,0,0,218,4,99,111,100,101,114,4,0,0,0, + 114,4,0,0,0,114,5,0,0,0,218,17,95,99,111,109, + 112,105,108,101,95,98,121,116,101,99,111,100,101,207,1,0, + 0,115,16,0,0,0,0,2,15,1,15,1,16,1,12,1, + 16,1,4,2,18,1,114,141,0,0,0,114,59,0,0,0, + 99,3,0,0,0,0,0,0,0,4,0,0,0,3,0,0, + 0,67,0,0,0,115,76,0,0,0,116,0,0,116,1,0, + 131,1,0,125,3,0,124,3,0,106,2,0,116,3,0,124, + 1,0,131,1,0,131,1,0,1,124,3,0,106,2,0,116, + 3,0,124,2,0,131,1,0,131,1,0,1,124,3,0,106, + 2,0,116,4,0,106,5,0,124,0,0,131,1,0,131,1, + 0,1,124,3,0,83,41,1,122,80,67,111,109,112,105,108, + 101,32,97,32,99,111,100,101,32,111,98,106,101,99,116,32, + 105,110,116,111,32,98,121,116,101,99,111,100,101,32,102,111, + 114,32,119,114,105,116,105,110,103,32,111,117,116,32,116,111, + 32,97,32,98,121,116,101,45,99,111,109,112,105,108,101,100, + 10,32,32,32,32,102,105,108,101,46,41,6,218,9,98,121, + 116,101,97,114,114,97,121,114,128,0,0,0,218,6,101,120, + 116,101,110,100,114,17,0,0,0,114,136,0,0,0,90,5, + 100,117,109,112,115,41,4,114,140,0,0,0,114,126,0,0, + 0,114,134,0,0,0,114,53,0,0,0,114,4,0,0,0, + 114,4,0,0,0,114,5,0,0,0,218,17,95,99,111,100, + 101,95,116,111,95,98,121,116,101,99,111,100,101,219,1,0, + 0,115,10,0,0,0,0,3,12,1,19,1,19,1,22,1, + 114,144,0,0,0,99,1,0,0,0,0,0,0,0,5,0, + 0,0,4,0,0,0,67,0,0,0,115,89,0,0,0,100, + 1,0,100,2,0,108,0,0,125,1,0,116,1,0,106,2, + 0,124,0,0,131,1,0,106,3,0,125,2,0,124,1,0, + 106,4,0,124,2,0,131,1,0,125,3,0,116,1,0,106, + 5,0,100,2,0,100,3,0,131,2,0,125,4,0,124,4, + 0,106,6,0,124,0,0,106,6,0,124,3,0,100,1,0, + 25,131,1,0,131,1,0,83,41,4,122,121,68,101,99,111, + 100,101,32,98,121,116,101,115,32,114,101,112,114,101,115,101, + 110,116,105,110,103,32,115,111,117,114,99,101,32,99,111,100, + 101,32,97,110,100,32,114,101,116,117,114,110,32,116,104,101, + 32,115,116,114,105,110,103,46,10,10,32,32,32,32,85,110, + 105,118,101,114,115,97,108,32,110,101,119,108,105,110,101,32, + 115,117,112,112,111,114,116,32,105,115,32,117,115,101,100,32, + 105,110,32,116,104,101,32,100,101,99,111,100,105,110,103,46, + 10,32,32,32,32,114,59,0,0,0,78,84,41,7,218,8, + 116,111,107,101,110,105,122,101,114,49,0,0,0,90,7,66, + 121,116,101,115,73,79,90,8,114,101,97,100,108,105,110,101, + 90,15,100,101,116,101,99,116,95,101,110,99,111,100,105,110, + 103,90,25,73,110,99,114,101,109,101,110,116,97,108,78,101, + 119,108,105,110,101,68,101,99,111,100,101,114,218,6,100,101, + 99,111,100,101,41,5,218,12,115,111,117,114,99,101,95,98, + 121,116,101,115,114,145,0,0,0,90,21,115,111,117,114,99, + 101,95,98,121,116,101,115,95,114,101,97,100,108,105,110,101, + 218,8,101,110,99,111,100,105,110,103,90,15,110,101,119,108, + 105,110,101,95,100,101,99,111,100,101,114,114,4,0,0,0, + 114,4,0,0,0,114,5,0,0,0,218,13,100,101,99,111, + 100,101,95,115,111,117,114,99,101,229,1,0,0,115,10,0, + 0,0,0,5,12,1,18,1,15,1,18,1,114,149,0,0, + 0,114,120,0,0,0,218,26,115,117,98,109,111,100,117,108, + 101,95,115,101,97,114,99,104,95,108,111,99,97,116,105,111, + 110,115,99,2,0,0,0,2,0,0,0,9,0,0,0,19, + 0,0,0,67,0,0,0,115,89,1,0,0,124,1,0,100, + 1,0,107,8,0,114,73,0,100,2,0,125,1,0,116,0, + 0,124,2,0,100,3,0,131,2,0,114,73,0,121,19,0, + 124,2,0,106,1,0,124,0,0,131,1,0,125,1,0,87, + 110,18,0,4,116,2,0,107,10,0,114,72,0,1,1,1, + 89,110,1,0,88,116,3,0,106,4,0,124,0,0,124,2, + 0,100,4,0,124,1,0,131,2,1,125,4,0,100,5,0, + 124,4,0,95,5,0,124,2,0,100,1,0,107,8,0,114, + 194,0,120,73,0,116,6,0,131,0,0,68,93,58,0,92, + 2,0,125,5,0,125,6,0,124,1,0,106,7,0,116,8, + 0,124,6,0,131,1,0,131,1,0,114,128,0,124,5,0, + 124,0,0,124,1,0,131,2,0,125,2,0,124,2,0,124, + 4,0,95,9,0,80,113,128,0,87,100,1,0,83,124,3, + 0,116,10,0,107,8,0,114,23,1,116,0,0,124,2,0, + 100,6,0,131,2,0,114,32,1,121,19,0,124,2,0,106, + 11,0,124,0,0,131,1,0,125,7,0,87,110,18,0,4, + 116,2,0,107,10,0,114,4,1,1,1,1,89,113,32,1, + 88,124,7,0,114,32,1,103,0,0,124,4,0,95,12,0, + 110,9,0,124,3,0,124,4,0,95,12,0,124,4,0,106, + 12,0,103,0,0,107,2,0,114,85,1,124,1,0,114,85, + 1,116,13,0,124,1,0,131,1,0,100,7,0,25,125,8, + 0,124,4,0,106,12,0,106,14,0,124,8,0,131,1,0, + 1,124,4,0,83,41,8,97,61,1,0,0,82,101,116,117, + 114,110,32,97,32,109,111,100,117,108,101,32,115,112,101,99, + 32,98,97,115,101,100,32,111,110,32,97,32,102,105,108,101, + 32,108,111,99,97,116,105,111,110,46,10,10,32,32,32,32, + 84,111,32,105,110,100,105,99,97,116,101,32,116,104,97,116, + 32,116,104,101,32,109,111,100,117,108,101,32,105,115,32,97, + 32,112,97,99,107,97,103,101,44,32,115,101,116,10,32,32, + 32,32,115,117,98,109,111,100,117,108,101,95,115,101,97,114, + 99,104,95,108,111,99,97,116,105,111,110,115,32,116,111,32, + 97,32,108,105,115,116,32,111,102,32,100,105,114,101,99,116, + 111,114,121,32,112,97,116,104,115,46,32,32,65,110,10,32, + 32,32,32,101,109,112,116,121,32,108,105,115,116,32,105,115, + 32,115,117,102,102,105,99,105,101,110,116,44,32,116,104,111, + 117,103,104,32,105,116,115,32,110,111,116,32,111,116,104,101, + 114,119,105,115,101,32,117,115,101,102,117,108,32,116,111,32, + 116,104,101,10,32,32,32,32,105,109,112,111,114,116,32,115, + 121,115,116,101,109,46,10,10,32,32,32,32,84,104,101,32, + 108,111,97,100,101,114,32,109,117,115,116,32,116,97,107,101, + 32,97,32,115,112,101,99,32,97,115,32,105,116,115,32,111, + 110,108,121,32,95,95,105,110,105,116,95,95,40,41,32,97, + 114,103,46,10,10,32,32,32,32,78,122,9,60,117,110,107, + 110,111,119,110,62,218,12,103,101,116,95,102,105,108,101,110, + 97,109,101,218,6,111,114,105,103,105,110,84,218,10,105,115, + 95,112,97,99,107,97,103,101,114,59,0,0,0,41,15,114, + 108,0,0,0,114,151,0,0,0,114,99,0,0,0,114,114, + 0,0,0,218,10,77,111,100,117,108,101,83,112,101,99,90, + 13,95,115,101,116,95,102,105,108,101,97,116,116,114,218,27, + 95,103,101,116,95,115,117,112,112,111,114,116,101,100,95,102, + 105,108,101,95,108,111,97,100,101,114,115,114,92,0,0,0, + 114,93,0,0,0,114,120,0,0,0,218,9,95,80,79,80, + 85,76,65,84,69,114,153,0,0,0,114,150,0,0,0,114, + 38,0,0,0,218,6,97,112,112,101,110,100,41,9,114,98, + 0,0,0,90,8,108,111,99,97,116,105,111,110,114,120,0, + 0,0,114,150,0,0,0,218,4,115,112,101,99,218,12,108, + 111,97,100,101,114,95,99,108,97,115,115,218,8,115,117,102, + 102,105,120,101,115,114,153,0,0,0,90,7,100,105,114,110, + 97,109,101,114,4,0,0,0,114,4,0,0,0,114,5,0, + 0,0,218,23,115,112,101,99,95,102,114,111,109,95,102,105, + 108,101,95,108,111,99,97,116,105,111,110,246,1,0,0,115, + 60,0,0,0,0,12,12,4,6,1,15,2,3,1,19,1, + 13,1,5,8,24,1,9,3,12,1,22,1,21,1,15,1, + 9,1,5,2,4,3,12,2,15,1,3,1,19,1,13,1, + 5,2,6,1,12,2,9,1,15,1,6,1,16,1,16,2, + 114,161,0,0,0,99,0,0,0,0,0,0,0,0,0,0, + 0,0,5,0,0,0,64,0,0,0,115,121,0,0,0,101, + 0,0,90,1,0,100,0,0,90,2,0,100,1,0,90,3, + 0,100,2,0,90,4,0,100,3,0,90,5,0,100,4,0, + 90,6,0,101,7,0,100,5,0,100,6,0,132,0,0,131, + 1,0,90,8,0,101,7,0,100,7,0,100,8,0,132,0, + 0,131,1,0,90,9,0,101,7,0,100,9,0,100,9,0, + 100,10,0,100,11,0,132,2,0,131,1,0,90,10,0,101, + 7,0,100,9,0,100,12,0,100,13,0,132,1,0,131,1, + 0,90,11,0,100,9,0,83,41,14,218,21,87,105,110,100, + 111,119,115,82,101,103,105,115,116,114,121,70,105,110,100,101, + 114,122,62,77,101,116,97,32,112,97,116,104,32,102,105,110, + 100,101,114,32,102,111,114,32,109,111,100,117,108,101,115,32, + 100,101,99,108,97,114,101,100,32,105,110,32,116,104,101,32, + 87,105,110,100,111,119,115,32,114,101,103,105,115,116,114,121, + 46,122,59,83,111,102,116,119,97,114,101,92,80,121,116,104, + 111,110,92,80,121,116,104,111,110,67,111,114,101,92,123,115, + 121,115,95,118,101,114,115,105,111,110,125,92,77,111,100,117, + 108,101,115,92,123,102,117,108,108,110,97,109,101,125,122,65, + 83,111,102,116,119,97,114,101,92,80,121,116,104,111,110,92, + 80,121,116,104,111,110,67,111,114,101,92,123,115,121,115,95, + 118,101,114,115,105,111,110,125,92,77,111,100,117,108,101,115, + 92,123,102,117,108,108,110,97,109,101,125,92,68,101,98,117, + 103,70,99,2,0,0,0,0,0,0,0,2,0,0,0,11, + 0,0,0,67,0,0,0,115,67,0,0,0,121,23,0,116, + 0,0,106,1,0,116,0,0,106,2,0,124,1,0,131,2, + 0,83,87,110,37,0,4,116,3,0,107,10,0,114,62,0, + 1,1,1,116,0,0,106,1,0,116,0,0,106,4,0,124, + 1,0,131,2,0,83,89,110,1,0,88,100,0,0,83,41, + 1,78,41,5,218,7,95,119,105,110,114,101,103,90,7,79, + 112,101,110,75,101,121,90,17,72,75,69,89,95,67,85,82, + 82,69,78,84,95,85,83,69,82,114,40,0,0,0,90,18, + 72,75,69,89,95,76,79,67,65,76,95,77,65,67,72,73, + 78,69,41,2,218,3,99,108,115,218,3,107,101,121,114,4, + 0,0,0,114,4,0,0,0,114,5,0,0,0,218,14,95, + 111,112,101,110,95,114,101,103,105,115,116,114,121,68,2,0, + 0,115,8,0,0,0,0,2,3,1,23,1,13,1,122,36, + 87,105,110,100,111,119,115,82,101,103,105,115,116,114,121,70, + 105,110,100,101,114,46,95,111,112,101,110,95,114,101,103,105, + 115,116,114,121,99,2,0,0,0,0,0,0,0,6,0,0, + 0,16,0,0,0,67,0,0,0,115,143,0,0,0,124,0, + 0,106,0,0,114,21,0,124,0,0,106,1,0,125,2,0, + 110,9,0,124,0,0,106,2,0,125,2,0,124,2,0,106, + 3,0,100,1,0,124,1,0,100,2,0,116,4,0,106,5, + 0,100,0,0,100,3,0,133,2,0,25,131,0,2,125,3, + 0,121,47,0,124,0,0,106,6,0,124,3,0,131,1,0, + 143,25,0,125,4,0,116,7,0,106,8,0,124,4,0,100, + 4,0,131,2,0,125,5,0,87,100,0,0,81,82,88,87, + 110,22,0,4,116,9,0,107,10,0,114,138,0,1,1,1, + 100,0,0,83,89,110,1,0,88,124,5,0,83,41,5,78, + 114,119,0,0,0,90,11,115,121,115,95,118,101,114,115,105, + 111,110,114,80,0,0,0,114,30,0,0,0,41,10,218,11, + 68,69,66,85,71,95,66,85,73,76,68,218,18,82,69,71, + 73,83,84,82,89,95,75,69,89,95,68,69,66,85,71,218, + 12,82,69,71,73,83,84,82,89,95,75,69,89,114,47,0, + 0,0,114,7,0,0,0,218,7,118,101,114,115,105,111,110, + 114,166,0,0,0,114,163,0,0,0,90,10,81,117,101,114, + 121,86,97,108,117,101,114,40,0,0,0,41,6,114,164,0, + 0,0,114,119,0,0,0,90,12,114,101,103,105,115,116,114, + 121,95,107,101,121,114,165,0,0,0,90,4,104,107,101,121, + 218,8,102,105,108,101,112,97,116,104,114,4,0,0,0,114, + 4,0,0,0,114,5,0,0,0,218,16,95,115,101,97,114, + 99,104,95,114,101,103,105,115,116,114,121,75,2,0,0,115, + 22,0,0,0,0,2,9,1,12,2,9,1,15,1,22,1, + 3,1,18,1,29,1,13,1,9,1,122,38,87,105,110,100, + 111,119,115,82,101,103,105,115,116,114,121,70,105,110,100,101, + 114,46,95,115,101,97,114,99,104,95,114,101,103,105,115,116, + 114,121,78,99,4,0,0,0,0,0,0,0,8,0,0,0, + 14,0,0,0,67,0,0,0,115,158,0,0,0,124,0,0, + 106,0,0,124,1,0,131,1,0,125,4,0,124,4,0,100, + 0,0,107,8,0,114,31,0,100,0,0,83,121,14,0,116, + 1,0,124,4,0,131,1,0,1,87,110,22,0,4,116,2, + 0,107,10,0,114,69,0,1,1,1,100,0,0,83,89,110, + 1,0,88,120,81,0,116,3,0,131,0,0,68,93,70,0, + 92,2,0,125,5,0,125,6,0,124,4,0,106,4,0,116, + 5,0,124,6,0,131,1,0,131,1,0,114,80,0,116,6, + 0,106,7,0,124,1,0,124,5,0,124,1,0,124,4,0, + 131,2,0,100,1,0,124,4,0,131,2,1,125,7,0,124, + 7,0,83,113,80,0,87,100,0,0,83,41,2,78,114,152, + 0,0,0,41,8,114,172,0,0,0,114,39,0,0,0,114, + 40,0,0,0,114,155,0,0,0,114,92,0,0,0,114,93, + 0,0,0,114,114,0,0,0,218,16,115,112,101,99,95,102, + 114,111,109,95,108,111,97,100,101,114,41,8,114,164,0,0, + 0,114,119,0,0,0,114,35,0,0,0,218,6,116,97,114, + 103,101,116,114,171,0,0,0,114,120,0,0,0,114,160,0, + 0,0,114,158,0,0,0,114,4,0,0,0,114,4,0,0, + 0,114,5,0,0,0,218,9,102,105,110,100,95,115,112,101, + 99,90,2,0,0,115,26,0,0,0,0,2,15,1,12,1, + 4,1,3,1,14,1,13,1,9,1,22,1,21,1,9,1, + 15,1,9,1,122,31,87,105,110,100,111,119,115,82,101,103, + 105,115,116,114,121,70,105,110,100,101,114,46,102,105,110,100, + 95,115,112,101,99,99,3,0,0,0,0,0,0,0,4,0, + 0,0,3,0,0,0,67,0,0,0,115,45,0,0,0,124, + 0,0,106,0,0,124,1,0,124,2,0,131,2,0,125,3, + 0,124,3,0,100,1,0,107,9,0,114,37,0,124,3,0, + 106,1,0,83,100,1,0,83,100,1,0,83,41,2,122,108, + 70,105,110,100,32,109,111,100,117,108,101,32,110,97,109,101, + 100,32,105,110,32,116,104,101,32,114,101,103,105,115,116,114, + 121,46,10,10,32,32,32,32,32,32,32,32,84,104,105,115, + 32,109,101,116,104,111,100,32,105,115,32,100,101,112,114,101, + 99,97,116,101,100,46,32,32,85,115,101,32,101,120,101,99, + 95,109,111,100,117,108,101,40,41,32,105,110,115,116,101,97, + 100,46,10,10,32,32,32,32,32,32,32,32,78,41,2,114, + 175,0,0,0,114,120,0,0,0,41,4,114,164,0,0,0, + 114,119,0,0,0,114,35,0,0,0,114,158,0,0,0,114, + 4,0,0,0,114,4,0,0,0,114,5,0,0,0,218,11, + 102,105,110,100,95,109,111,100,117,108,101,106,2,0,0,115, + 8,0,0,0,0,7,18,1,12,1,7,2,122,33,87,105, + 110,100,111,119,115,82,101,103,105,115,116,114,121,70,105,110, + 100,101,114,46,102,105,110,100,95,109,111,100,117,108,101,41, + 12,114,105,0,0,0,114,104,0,0,0,114,106,0,0,0, + 114,107,0,0,0,114,169,0,0,0,114,168,0,0,0,114, + 167,0,0,0,218,11,99,108,97,115,115,109,101,116,104,111, + 100,114,166,0,0,0,114,172,0,0,0,114,175,0,0,0, + 114,176,0,0,0,114,4,0,0,0,114,4,0,0,0,114, + 4,0,0,0,114,5,0,0,0,114,162,0,0,0,56,2, + 0,0,115,20,0,0,0,12,2,6,3,6,3,6,2,6, + 2,18,7,18,15,3,1,21,15,3,1,114,162,0,0,0, + 99,0,0,0,0,0,0,0,0,0,0,0,0,2,0,0, + 0,64,0,0,0,115,70,0,0,0,101,0,0,90,1,0, + 100,0,0,90,2,0,100,1,0,90,3,0,100,2,0,100, + 3,0,132,0,0,90,4,0,100,4,0,100,5,0,132,0, + 0,90,5,0,100,6,0,100,7,0,132,0,0,90,6,0, + 100,8,0,100,9,0,132,0,0,90,7,0,100,10,0,83, + 41,11,218,13,95,76,111,97,100,101,114,66,97,115,105,99, + 115,122,83,66,97,115,101,32,99,108,97,115,115,32,111,102, + 32,99,111,109,109,111,110,32,99,111,100,101,32,110,101,101, + 100,101,100,32,98,121,32,98,111,116,104,32,83,111,117,114, + 99,101,76,111,97,100,101,114,32,97,110,100,10,32,32,32, + 32,83,111,117,114,99,101,108,101,115,115,70,105,108,101,76, + 111,97,100,101,114,46,99,2,0,0,0,0,0,0,0,5, + 0,0,0,3,0,0,0,67,0,0,0,115,88,0,0,0, + 116,0,0,124,0,0,106,1,0,124,1,0,131,1,0,131, + 1,0,100,1,0,25,125,2,0,124,2,0,106,2,0,100, + 2,0,100,1,0,131,2,0,100,3,0,25,125,3,0,124, + 1,0,106,3,0,100,2,0,131,1,0,100,4,0,25,125, + 4,0,124,3,0,100,5,0,107,2,0,111,87,0,124,4, + 0,100,5,0,107,3,0,83,41,6,122,141,67,111,110,99, + 114,101,116,101,32,105,109,112,108,101,109,101,110,116,97,116, + 105,111,110,32,111,102,32,73,110,115,112,101,99,116,76,111, + 97,100,101,114,46,105,115,95,112,97,99,107,97,103,101,32, + 98,121,32,99,104,101,99,107,105,110,103,32,105,102,10,32, + 32,32,32,32,32,32,32,116,104,101,32,112,97,116,104,32, + 114,101,116,117,114,110,101,100,32,98,121,32,103,101,116,95, + 102,105,108,101,110,97,109,101,32,104,97,115,32,97,32,102, + 105,108,101,110,97,109,101,32,111,102,32,39,95,95,105,110, + 105,116,95,95,46,112,121,39,46,114,29,0,0,0,114,58, + 0,0,0,114,59,0,0,0,114,56,0,0,0,218,8,95, + 95,105,110,105,116,95,95,41,4,114,38,0,0,0,114,151, + 0,0,0,114,34,0,0,0,114,32,0,0,0,41,5,114, + 100,0,0,0,114,119,0,0,0,114,94,0,0,0,90,13, + 102,105,108,101,110,97,109,101,95,98,97,115,101,90,9,116, + 97,105,108,95,110,97,109,101,114,4,0,0,0,114,4,0, + 0,0,114,5,0,0,0,114,153,0,0,0,125,2,0,0, + 115,8,0,0,0,0,3,25,1,22,1,19,1,122,24,95, + 76,111,97,100,101,114,66,97,115,105,99,115,46,105,115,95, + 112,97,99,107,97,103,101,99,2,0,0,0,0,0,0,0, + 2,0,0,0,1,0,0,0,67,0,0,0,115,4,0,0, + 0,100,1,0,83,41,2,122,42,85,115,101,32,100,101,102, + 97,117,108,116,32,115,101,109,97,110,116,105,99,115,32,102, + 111,114,32,109,111,100,117,108,101,32,99,114,101,97,116,105, + 111,110,46,78,114,4,0,0,0,41,2,114,100,0,0,0, + 114,158,0,0,0,114,4,0,0,0,114,4,0,0,0,114, + 5,0,0,0,218,13,99,114,101,97,116,101,95,109,111,100, + 117,108,101,133,2,0,0,115,0,0,0,0,122,27,95,76, + 111,97,100,101,114,66,97,115,105,99,115,46,99,114,101,97, + 116,101,95,109,111,100,117,108,101,99,2,0,0,0,0,0, + 0,0,3,0,0,0,4,0,0,0,67,0,0,0,115,80, + 0,0,0,124,0,0,106,0,0,124,1,0,106,1,0,131, + 1,0,125,2,0,124,2,0,100,1,0,107,8,0,114,54, + 0,116,2,0,100,2,0,106,3,0,124,1,0,106,1,0, + 131,1,0,131,1,0,130,1,0,116,4,0,106,5,0,116, + 6,0,124,2,0,124,1,0,106,7,0,131,3,0,1,100, + 1,0,83,41,3,122,19,69,120,101,99,117,116,101,32,116, + 104,101,32,109,111,100,117,108,101,46,78,122,52,99,97,110, + 110,111,116,32,108,111,97,100,32,109,111,100,117,108,101,32, + 123,33,114,125,32,119,104,101,110,32,103,101,116,95,99,111, + 100,101,40,41,32,114,101,116,117,114,110,115,32,78,111,110, + 101,41,8,218,8,103,101,116,95,99,111,100,101,114,105,0, + 0,0,114,99,0,0,0,114,47,0,0,0,114,114,0,0, + 0,218,25,95,99,97,108,108,95,119,105,116,104,95,102,114, + 97,109,101,115,95,114,101,109,111,118,101,100,218,4,101,120, + 101,99,114,111,0,0,0,41,3,114,100,0,0,0,218,6, + 109,111,100,117,108,101,114,140,0,0,0,114,4,0,0,0, + 114,4,0,0,0,114,5,0,0,0,218,11,101,120,101,99, + 95,109,111,100,117,108,101,136,2,0,0,115,10,0,0,0, + 0,2,18,1,12,1,9,1,15,1,122,25,95,76,111,97, + 100,101,114,66,97,115,105,99,115,46,101,120,101,99,95,109, + 111,100,117,108,101,99,2,0,0,0,0,0,0,0,2,0, + 0,0,3,0,0,0,67,0,0,0,115,16,0,0,0,116, + 0,0,106,1,0,124,0,0,124,1,0,131,2,0,83,41, + 1,78,41,2,114,114,0,0,0,218,17,95,108,111,97,100, + 95,109,111,100,117,108,101,95,115,104,105,109,41,2,114,100, + 0,0,0,114,119,0,0,0,114,4,0,0,0,114,4,0, + 0,0,114,5,0,0,0,218,11,108,111,97,100,95,109,111, + 100,117,108,101,144,2,0,0,115,2,0,0,0,0,1,122, + 25,95,76,111,97,100,101,114,66,97,115,105,99,115,46,108, + 111,97,100,95,109,111,100,117,108,101,78,41,8,114,105,0, + 0,0,114,104,0,0,0,114,106,0,0,0,114,107,0,0, + 0,114,153,0,0,0,114,180,0,0,0,114,185,0,0,0, + 114,187,0,0,0,114,4,0,0,0,114,4,0,0,0,114, + 4,0,0,0,114,5,0,0,0,114,178,0,0,0,120,2, + 0,0,115,10,0,0,0,12,3,6,2,12,8,12,3,12, + 8,114,178,0,0,0,99,0,0,0,0,0,0,0,0,0, + 0,0,0,4,0,0,0,64,0,0,0,115,106,0,0,0, + 101,0,0,90,1,0,100,0,0,90,2,0,100,1,0,100, + 2,0,132,0,0,90,3,0,100,3,0,100,4,0,132,0, + 0,90,4,0,100,5,0,100,6,0,132,0,0,90,5,0, + 100,7,0,100,8,0,132,0,0,90,6,0,100,9,0,100, + 10,0,132,0,0,90,7,0,100,11,0,100,18,0,100,13, + 0,100,14,0,132,0,1,90,8,0,100,15,0,100,16,0, + 132,0,0,90,9,0,100,17,0,83,41,19,218,12,83,111, + 117,114,99,101,76,111,97,100,101,114,99,2,0,0,0,0, + 0,0,0,2,0,0,0,1,0,0,0,67,0,0,0,115, + 10,0,0,0,116,0,0,130,1,0,100,1,0,83,41,2, + 122,178,79,112,116,105,111,110,97,108,32,109,101,116,104,111, + 100,32,116,104,97,116,32,114,101,116,117,114,110,115,32,116, + 104,101,32,109,111,100,105,102,105,99,97,116,105,111,110,32, + 116,105,109,101,32,40,97,110,32,105,110,116,41,32,102,111, + 114,32,116,104,101,10,32,32,32,32,32,32,32,32,115,112, + 101,99,105,102,105,101,100,32,112,97,116,104,44,32,119,104, + 101,114,101,32,112,97,116,104,32,105,115,32,97,32,115,116, + 114,46,10,10,32,32,32,32,32,32,32,32,82,97,105,115, + 101,115,32,73,79,69,114,114,111,114,32,119,104,101,110,32, + 116,104,101,32,112,97,116,104,32,99,97,110,110,111,116,32, + 98,101,32,104,97,110,100,108,101,100,46,10,32,32,32,32, + 32,32,32,32,78,41,1,218,7,73,79,69,114,114,111,114, + 41,2,114,100,0,0,0,114,35,0,0,0,114,4,0,0, + 0,114,4,0,0,0,114,5,0,0,0,218,10,112,97,116, + 104,95,109,116,105,109,101,150,2,0,0,115,2,0,0,0, + 0,6,122,23,83,111,117,114,99,101,76,111,97,100,101,114, + 46,112,97,116,104,95,109,116,105,109,101,99,2,0,0,0, + 0,0,0,0,2,0,0,0,3,0,0,0,67,0,0,0, + 115,19,0,0,0,100,1,0,124,0,0,106,0,0,124,1, + 0,131,1,0,105,1,0,83,41,2,97,170,1,0,0,79, + 112,116,105,111,110,97,108,32,109,101,116,104,111,100,32,114, + 101,116,117,114,110,105,110,103,32,97,32,109,101,116,97,100, + 97,116,97,32,100,105,99,116,32,102,111,114,32,116,104,101, + 32,115,112,101,99,105,102,105,101,100,32,112,97,116,104,10, + 32,32,32,32,32,32,32,32,116,111,32,98,121,32,116,104, + 101,32,112,97,116,104,32,40,115,116,114,41,46,10,32,32, + 32,32,32,32,32,32,80,111,115,115,105,98,108,101,32,107, + 101,121,115,58,10,32,32,32,32,32,32,32,32,45,32,39, + 109,116,105,109,101,39,32,40,109,97,110,100,97,116,111,114, + 121,41,32,105,115,32,116,104,101,32,110,117,109,101,114,105, + 99,32,116,105,109,101,115,116,97,109,112,32,111,102,32,108, + 97,115,116,32,115,111,117,114,99,101,10,32,32,32,32,32, + 32,32,32,32,32,99,111,100,101,32,109,111,100,105,102,105, + 99,97,116,105,111,110,59,10,32,32,32,32,32,32,32,32, + 45,32,39,115,105,122,101,39,32,40,111,112,116,105,111,110, + 97,108,41,32,105,115,32,116,104,101,32,115,105,122,101,32, + 105,110,32,98,121,116,101,115,32,111,102,32,116,104,101,32, + 115,111,117,114,99,101,32,99,111,100,101,46,10,10,32,32, + 32,32,32,32,32,32,73,109,112,108,101,109,101,110,116,105, + 110,103,32,116,104,105,115,32,109,101,116,104,111,100,32,97, + 108,108,111,119,115,32,116,104,101,32,108,111,97,100,101,114, + 32,116,111,32,114,101,97,100,32,98,121,116,101,99,111,100, + 101,32,102,105,108,101,115,46,10,32,32,32,32,32,32,32, + 32,82,97,105,115,101,115,32,73,79,69,114,114,111,114,32, + 119,104,101,110,32,116,104,101,32,112,97,116,104,32,99,97, + 110,110,111,116,32,98,101,32,104,97,110,100,108,101,100,46, + 10,32,32,32,32,32,32,32,32,114,126,0,0,0,41,1, + 114,190,0,0,0,41,2,114,100,0,0,0,114,35,0,0, 0,114,4,0,0,0,114,4,0,0,0,114,5,0,0,0, - 218,9,102,105,110,100,95,115,112,101,99,98,2,0,0,115, - 26,0,0,0,0,2,15,1,12,1,4,1,3,1,14,1, - 13,1,9,1,22,1,21,1,9,1,15,1,9,1,122,31, - 87,105,110,100,111,119,115,82,101,103,105,115,116,114,121,70, - 105,110,100,101,114,46,102,105,110,100,95,115,112,101,99,99, - 3,0,0,0,0,0,0,0,4,0,0,0,3,0,0,0, - 67,0,0,0,115,45,0,0,0,124,0,0,106,0,0,124, - 1,0,124,2,0,131,2,0,125,3,0,124,3,0,100,1, - 0,107,9,0,114,37,0,124,3,0,106,1,0,83,100,1, - 0,83,100,1,0,83,41,2,122,108,70,105,110,100,32,109, - 111,100,117,108,101,32,110,97,109,101,100,32,105,110,32,116, - 104,101,32,114,101,103,105,115,116,114,121,46,10,10,32,32, - 32,32,32,32,32,32,84,104,105,115,32,109,101,116,104,111, - 100,32,105,115,32,100,101,112,114,101,99,97,116,101,100,46, - 32,32,85,115,101,32,101,120,101,99,95,109,111,100,117,108, - 101,40,41,32,105,110,115,116,101,97,100,46,10,10,32,32, - 32,32,32,32,32,32,78,41,2,114,181,0,0,0,114,127, - 0,0,0,41,4,114,170,0,0,0,114,126,0,0,0,114, - 35,0,0,0,114,164,0,0,0,114,4,0,0,0,114,4, - 0,0,0,114,5,0,0,0,218,11,102,105,110,100,95,109, - 111,100,117,108,101,114,2,0,0,115,8,0,0,0,0,7, - 18,1,12,1,7,2,122,33,87,105,110,100,111,119,115,82, - 101,103,105,115,116,114,121,70,105,110,100,101,114,46,102,105, - 110,100,95,109,111,100,117,108,101,41,12,114,112,0,0,0, - 114,111,0,0,0,114,113,0,0,0,114,114,0,0,0,114, - 175,0,0,0,114,174,0,0,0,114,173,0,0,0,218,11, - 99,108,97,115,115,109,101,116,104,111,100,114,172,0,0,0, - 114,178,0,0,0,114,181,0,0,0,114,182,0,0,0,114, - 4,0,0,0,114,4,0,0,0,114,4,0,0,0,114,5, - 0,0,0,114,168,0,0,0,64,2,0,0,115,20,0,0, - 0,12,2,6,3,6,3,6,2,6,2,18,7,18,15,3, - 1,21,15,3,1,114,168,0,0,0,99,0,0,0,0,0, - 0,0,0,0,0,0,0,2,0,0,0,64,0,0,0,115, - 70,0,0,0,101,0,0,90,1,0,100,0,0,90,2,0, + 218,10,112,97,116,104,95,115,116,97,116,115,158,2,0,0, + 115,2,0,0,0,0,11,122,23,83,111,117,114,99,101,76, + 111,97,100,101,114,46,112,97,116,104,95,115,116,97,116,115, + 99,4,0,0,0,0,0,0,0,4,0,0,0,3,0,0, + 0,67,0,0,0,115,16,0,0,0,124,0,0,106,0,0, + 124,2,0,124,3,0,131,2,0,83,41,1,122,228,79,112, + 116,105,111,110,97,108,32,109,101,116,104,111,100,32,119,104, + 105,99,104,32,119,114,105,116,101,115,32,100,97,116,97,32, + 40,98,121,116,101,115,41,32,116,111,32,97,32,102,105,108, + 101,32,112,97,116,104,32,40,97,32,115,116,114,41,46,10, + 10,32,32,32,32,32,32,32,32,73,109,112,108,101,109,101, + 110,116,105,110,103,32,116,104,105,115,32,109,101,116,104,111, + 100,32,97,108,108,111,119,115,32,102,111,114,32,116,104,101, + 32,119,114,105,116,105,110,103,32,111,102,32,98,121,116,101, + 99,111,100,101,32,102,105,108,101,115,46,10,10,32,32,32, + 32,32,32,32,32,84,104,101,32,115,111,117,114,99,101,32, + 112,97,116,104,32,105,115,32,110,101,101,100,101,100,32,105, + 110,32,111,114,100,101,114,32,116,111,32,99,111,114,114,101, + 99,116,108,121,32,116,114,97,110,115,102,101,114,32,112,101, + 114,109,105,115,115,105,111,110,115,10,32,32,32,32,32,32, + 32,32,41,1,218,8,115,101,116,95,100,97,116,97,41,4, + 114,100,0,0,0,114,90,0,0,0,90,10,99,97,99,104, + 101,95,112,97,116,104,114,53,0,0,0,114,4,0,0,0, + 114,4,0,0,0,114,5,0,0,0,218,15,95,99,97,99, + 104,101,95,98,121,116,101,99,111,100,101,171,2,0,0,115, + 2,0,0,0,0,8,122,28,83,111,117,114,99,101,76,111, + 97,100,101,114,46,95,99,97,99,104,101,95,98,121,116,101, + 99,111,100,101,99,3,0,0,0,0,0,0,0,3,0,0, + 0,1,0,0,0,67,0,0,0,115,4,0,0,0,100,1, + 0,83,41,2,122,150,79,112,116,105,111,110,97,108,32,109, + 101,116,104,111,100,32,119,104,105,99,104,32,119,114,105,116, + 101,115,32,100,97,116,97,32,40,98,121,116,101,115,41,32, + 116,111,32,97,32,102,105,108,101,32,112,97,116,104,32,40, + 97,32,115,116,114,41,46,10,10,32,32,32,32,32,32,32, + 32,73,109,112,108,101,109,101,110,116,105,110,103,32,116,104, + 105,115,32,109,101,116,104,111,100,32,97,108,108,111,119,115, + 32,102,111,114,32,116,104,101,32,119,114,105,116,105,110,103, + 32,111,102,32,98,121,116,101,99,111,100,101,32,102,105,108, + 101,115,46,10,32,32,32,32,32,32,32,32,78,114,4,0, + 0,0,41,3,114,100,0,0,0,114,35,0,0,0,114,53, + 0,0,0,114,4,0,0,0,114,4,0,0,0,114,5,0, + 0,0,114,192,0,0,0,181,2,0,0,115,0,0,0,0, + 122,21,83,111,117,114,99,101,76,111,97,100,101,114,46,115, + 101,116,95,100,97,116,97,99,2,0,0,0,0,0,0,0, + 5,0,0,0,16,0,0,0,67,0,0,0,115,105,0,0, + 0,124,0,0,106,0,0,124,1,0,131,1,0,125,2,0, + 121,19,0,124,0,0,106,1,0,124,2,0,131,1,0,125, + 3,0,87,110,58,0,4,116,2,0,107,10,0,114,94,0, + 1,125,4,0,1,122,26,0,116,3,0,100,1,0,100,2, + 0,124,1,0,131,1,1,124,4,0,130,2,0,87,89,100, + 3,0,100,3,0,125,4,0,126,4,0,88,110,1,0,88, + 116,4,0,124,3,0,131,1,0,83,41,4,122,52,67,111, + 110,99,114,101,116,101,32,105,109,112,108,101,109,101,110,116, + 97,116,105,111,110,32,111,102,32,73,110,115,112,101,99,116, + 76,111,97,100,101,114,46,103,101,116,95,115,111,117,114,99, + 101,46,122,39,115,111,117,114,99,101,32,110,111,116,32,97, + 118,97,105,108,97,98,108,101,32,116,104,114,111,117,103,104, + 32,103,101,116,95,100,97,116,97,40,41,114,98,0,0,0, + 78,41,5,114,151,0,0,0,218,8,103,101,116,95,100,97, + 116,97,114,40,0,0,0,114,99,0,0,0,114,149,0,0, + 0,41,5,114,100,0,0,0,114,119,0,0,0,114,35,0, + 0,0,114,147,0,0,0,218,3,101,120,99,114,4,0,0, + 0,114,4,0,0,0,114,5,0,0,0,218,10,103,101,116, + 95,115,111,117,114,99,101,188,2,0,0,115,14,0,0,0, + 0,2,15,1,3,1,19,1,18,1,9,1,31,1,122,23, + 83,111,117,114,99,101,76,111,97,100,101,114,46,103,101,116, + 95,115,111,117,114,99,101,218,9,95,111,112,116,105,109,105, + 122,101,114,29,0,0,0,99,3,0,0,0,1,0,0,0, + 4,0,0,0,9,0,0,0,67,0,0,0,115,34,0,0, + 0,116,0,0,106,1,0,116,2,0,124,1,0,124,2,0, + 100,1,0,100,2,0,100,3,0,100,4,0,124,3,0,131, + 4,2,83,41,5,122,130,82,101,116,117,114,110,32,116,104, + 101,32,99,111,100,101,32,111,98,106,101,99,116,32,99,111, + 109,112,105,108,101,100,32,102,114,111,109,32,115,111,117,114, + 99,101,46,10,10,32,32,32,32,32,32,32,32,84,104,101, + 32,39,100,97,116,97,39,32,97,114,103,117,109,101,110,116, + 32,99,97,110,32,98,101,32,97,110,121,32,111,98,106,101, + 99,116,32,116,121,112,101,32,116,104,97,116,32,99,111,109, + 112,105,108,101,40,41,32,115,117,112,112,111,114,116,115,46, + 10,32,32,32,32,32,32,32,32,114,183,0,0,0,218,12, + 100,111,110,116,95,105,110,104,101,114,105,116,84,114,68,0, + 0,0,41,3,114,114,0,0,0,114,182,0,0,0,218,7, + 99,111,109,112,105,108,101,41,4,114,100,0,0,0,114,53, + 0,0,0,114,35,0,0,0,114,197,0,0,0,114,4,0, + 0,0,114,4,0,0,0,114,5,0,0,0,218,14,115,111, + 117,114,99,101,95,116,111,95,99,111,100,101,198,2,0,0, + 115,4,0,0,0,0,5,21,1,122,27,83,111,117,114,99, + 101,76,111,97,100,101,114,46,115,111,117,114,99,101,95,116, + 111,95,99,111,100,101,99,2,0,0,0,0,0,0,0,10, + 0,0,0,43,0,0,0,67,0,0,0,115,183,1,0,0, + 124,0,0,106,0,0,124,1,0,131,1,0,125,2,0,100, + 1,0,125,3,0,121,16,0,116,1,0,124,2,0,131,1, + 0,125,4,0,87,110,24,0,4,116,2,0,107,10,0,114, + 63,0,1,1,1,100,1,0,125,4,0,89,110,205,0,88, + 121,19,0,124,0,0,106,3,0,124,2,0,131,1,0,125, + 5,0,87,110,18,0,4,116,4,0,107,10,0,114,103,0, + 1,1,1,89,110,165,0,88,116,5,0,124,5,0,100,2, + 0,25,131,1,0,125,3,0,121,19,0,124,0,0,106,6, + 0,124,4,0,131,1,0,125,6,0,87,110,18,0,4,116, + 7,0,107,10,0,114,159,0,1,1,1,89,110,109,0,88, + 121,34,0,116,8,0,124,6,0,100,3,0,124,5,0,100, + 4,0,124,1,0,100,5,0,124,4,0,131,1,3,125,7, + 0,87,110,24,0,4,116,9,0,116,10,0,102,2,0,107, + 10,0,114,220,0,1,1,1,89,110,48,0,88,116,11,0, + 106,12,0,100,6,0,124,4,0,124,2,0,131,3,0,1, + 116,13,0,124,7,0,100,4,0,124,1,0,100,7,0,124, + 4,0,100,8,0,124,2,0,131,1,3,83,124,0,0,106, + 6,0,124,2,0,131,1,0,125,8,0,124,0,0,106,14, + 0,124,8,0,124,2,0,131,2,0,125,9,0,116,11,0, + 106,12,0,100,9,0,124,2,0,131,2,0,1,116,15,0, + 106,16,0,12,114,179,1,124,4,0,100,1,0,107,9,0, + 114,179,1,124,3,0,100,1,0,107,9,0,114,179,1,116, + 17,0,124,9,0,124,3,0,116,18,0,124,8,0,131,1, + 0,131,3,0,125,6,0,121,39,0,124,0,0,106,19,0, + 124,2,0,124,4,0,124,6,0,131,3,0,1,116,11,0, + 106,12,0,100,10,0,124,4,0,131,2,0,1,87,110,18, + 0,4,116,2,0,107,10,0,114,178,1,1,1,1,89,110, + 1,0,88,124,9,0,83,41,11,122,190,67,111,110,99,114, + 101,116,101,32,105,109,112,108,101,109,101,110,116,97,116,105, + 111,110,32,111,102,32,73,110,115,112,101,99,116,76,111,97, + 100,101,114,46,103,101,116,95,99,111,100,101,46,10,10,32, + 32,32,32,32,32,32,32,82,101,97,100,105,110,103,32,111, + 102,32,98,121,116,101,99,111,100,101,32,114,101,113,117,105, + 114,101,115,32,112,97,116,104,95,115,116,97,116,115,32,116, + 111,32,98,101,32,105,109,112,108,101,109,101,110,116,101,100, + 46,32,84,111,32,119,114,105,116,101,10,32,32,32,32,32, + 32,32,32,98,121,116,101,99,111,100,101,44,32,115,101,116, + 95,100,97,116,97,32,109,117,115,116,32,97,108,115,111,32, + 98,101,32,105,109,112,108,101,109,101,110,116,101,100,46,10, + 10,32,32,32,32,32,32,32,32,78,114,126,0,0,0,114, + 132,0,0,0,114,98,0,0,0,114,35,0,0,0,122,13, + 123,125,32,109,97,116,99,104,101,115,32,123,125,114,89,0, + 0,0,114,90,0,0,0,122,19,99,111,100,101,32,111,98, + 106,101,99,116,32,102,114,111,109,32,123,125,122,10,119,114, + 111,116,101,32,123,33,114,125,41,20,114,151,0,0,0,114, + 79,0,0,0,114,66,0,0,0,114,191,0,0,0,114,189, + 0,0,0,114,14,0,0,0,114,194,0,0,0,114,40,0, + 0,0,114,135,0,0,0,114,99,0,0,0,114,130,0,0, + 0,114,114,0,0,0,114,129,0,0,0,114,141,0,0,0, + 114,200,0,0,0,114,7,0,0,0,218,19,100,111,110,116, + 95,119,114,105,116,101,95,98,121,116,101,99,111,100,101,114, + 144,0,0,0,114,31,0,0,0,114,193,0,0,0,41,10, + 114,100,0,0,0,114,119,0,0,0,114,90,0,0,0,114, + 133,0,0,0,114,89,0,0,0,218,2,115,116,114,53,0, + 0,0,218,10,98,121,116,101,115,95,100,97,116,97,114,147, + 0,0,0,90,11,99,111,100,101,95,111,98,106,101,99,116, + 114,4,0,0,0,114,4,0,0,0,114,5,0,0,0,114, + 181,0,0,0,206,2,0,0,115,78,0,0,0,0,7,15, + 1,6,1,3,1,16,1,13,1,11,2,3,1,19,1,13, + 1,5,2,16,1,3,1,19,1,13,1,5,2,3,1,9, + 1,12,1,13,1,19,1,5,2,12,1,7,1,15,1,6, + 1,7,1,15,1,18,1,16,1,22,1,12,1,9,1,15, + 1,3,1,19,1,20,1,13,1,5,1,122,21,83,111,117, + 114,99,101,76,111,97,100,101,114,46,103,101,116,95,99,111, + 100,101,78,114,87,0,0,0,41,10,114,105,0,0,0,114, + 104,0,0,0,114,106,0,0,0,114,190,0,0,0,114,191, + 0,0,0,114,193,0,0,0,114,192,0,0,0,114,196,0, + 0,0,114,200,0,0,0,114,181,0,0,0,114,4,0,0, + 0,114,4,0,0,0,114,4,0,0,0,114,5,0,0,0, + 114,188,0,0,0,148,2,0,0,115,14,0,0,0,12,2, + 12,8,12,13,12,10,12,7,12,10,18,8,114,188,0,0, + 0,99,0,0,0,0,0,0,0,0,0,0,0,0,4,0, + 0,0,0,0,0,0,115,112,0,0,0,101,0,0,90,1, + 0,100,0,0,90,2,0,100,1,0,90,3,0,100,2,0, + 100,3,0,132,0,0,90,4,0,100,4,0,100,5,0,132, + 0,0,90,5,0,100,6,0,100,7,0,132,0,0,90,6, + 0,101,7,0,135,0,0,102,1,0,100,8,0,100,9,0, + 134,0,0,131,1,0,90,8,0,101,7,0,100,10,0,100, + 11,0,132,0,0,131,1,0,90,9,0,100,12,0,100,13, + 0,132,0,0,90,10,0,135,0,0,83,41,14,218,10,70, + 105,108,101,76,111,97,100,101,114,122,103,66,97,115,101,32, + 102,105,108,101,32,108,111,97,100,101,114,32,99,108,97,115, + 115,32,119,104,105,99,104,32,105,109,112,108,101,109,101,110, + 116,115,32,116,104,101,32,108,111,97,100,101,114,32,112,114, + 111,116,111,99,111,108,32,109,101,116,104,111,100,115,32,116, + 104,97,116,10,32,32,32,32,114,101,113,117,105,114,101,32, + 102,105,108,101,32,115,121,115,116,101,109,32,117,115,97,103, + 101,46,99,3,0,0,0,0,0,0,0,3,0,0,0,2, + 0,0,0,67,0,0,0,115,22,0,0,0,124,1,0,124, + 0,0,95,0,0,124,2,0,124,0,0,95,1,0,100,1, + 0,83,41,2,122,75,67,97,99,104,101,32,116,104,101,32, + 109,111,100,117,108,101,32,110,97,109,101,32,97,110,100,32, + 116,104,101,32,112,97,116,104,32,116,111,32,116,104,101,32, + 102,105,108,101,32,102,111,117,110,100,32,98,121,32,116,104, + 101,10,32,32,32,32,32,32,32,32,102,105,110,100,101,114, + 46,78,41,2,114,98,0,0,0,114,35,0,0,0,41,3, + 114,100,0,0,0,114,119,0,0,0,114,35,0,0,0,114, + 4,0,0,0,114,4,0,0,0,114,5,0,0,0,114,179, + 0,0,0,7,3,0,0,115,4,0,0,0,0,3,9,1, + 122,19,70,105,108,101,76,111,97,100,101,114,46,95,95,105, + 110,105,116,95,95,99,2,0,0,0,0,0,0,0,2,0, + 0,0,2,0,0,0,67,0,0,0,115,34,0,0,0,124, + 0,0,106,0,0,124,1,0,106,0,0,107,2,0,111,33, + 0,124,0,0,106,1,0,124,1,0,106,1,0,107,2,0, + 83,41,1,78,41,2,218,9,95,95,99,108,97,115,115,95, + 95,114,111,0,0,0,41,2,114,100,0,0,0,218,5,111, + 116,104,101,114,114,4,0,0,0,114,4,0,0,0,114,5, + 0,0,0,218,6,95,95,101,113,95,95,13,3,0,0,115, + 4,0,0,0,0,1,18,1,122,17,70,105,108,101,76,111, + 97,100,101,114,46,95,95,101,113,95,95,99,1,0,0,0, + 0,0,0,0,1,0,0,0,3,0,0,0,67,0,0,0, + 115,26,0,0,0,116,0,0,124,0,0,106,1,0,131,1, + 0,116,0,0,124,0,0,106,2,0,131,1,0,65,83,41, + 1,78,41,3,218,4,104,97,115,104,114,98,0,0,0,114, + 35,0,0,0,41,1,114,100,0,0,0,114,4,0,0,0, + 114,4,0,0,0,114,5,0,0,0,218,8,95,95,104,97, + 115,104,95,95,17,3,0,0,115,2,0,0,0,0,1,122, + 19,70,105,108,101,76,111,97,100,101,114,46,95,95,104,97, + 115,104,95,95,99,2,0,0,0,0,0,0,0,2,0,0, + 0,3,0,0,0,3,0,0,0,115,22,0,0,0,116,0, + 0,116,1,0,124,0,0,131,2,0,106,2,0,124,1,0, + 131,1,0,83,41,1,122,100,76,111,97,100,32,97,32,109, + 111,100,117,108,101,32,102,114,111,109,32,97,32,102,105,108, + 101,46,10,10,32,32,32,32,32,32,32,32,84,104,105,115, + 32,109,101,116,104,111,100,32,105,115,32,100,101,112,114,101, + 99,97,116,101,100,46,32,32,85,115,101,32,101,120,101,99, + 95,109,111,100,117,108,101,40,41,32,105,110,115,116,101,97, + 100,46,10,10,32,32,32,32,32,32,32,32,41,3,218,5, + 115,117,112,101,114,114,204,0,0,0,114,187,0,0,0,41, + 2,114,100,0,0,0,114,119,0,0,0,41,1,114,205,0, + 0,0,114,4,0,0,0,114,5,0,0,0,114,187,0,0, + 0,20,3,0,0,115,2,0,0,0,0,10,122,22,70,105, + 108,101,76,111,97,100,101,114,46,108,111,97,100,95,109,111, + 100,117,108,101,99,2,0,0,0,0,0,0,0,2,0,0, + 0,1,0,0,0,67,0,0,0,115,7,0,0,0,124,0, + 0,106,0,0,83,41,1,122,58,82,101,116,117,114,110,32, + 116,104,101,32,112,97,116,104,32,116,111,32,116,104,101,32, + 115,111,117,114,99,101,32,102,105,108,101,32,97,115,32,102, + 111,117,110,100,32,98,121,32,116,104,101,32,102,105,110,100, + 101,114,46,41,1,114,35,0,0,0,41,2,114,100,0,0, + 0,114,119,0,0,0,114,4,0,0,0,114,4,0,0,0, + 114,5,0,0,0,114,151,0,0,0,32,3,0,0,115,2, + 0,0,0,0,3,122,23,70,105,108,101,76,111,97,100,101, + 114,46,103,101,116,95,102,105,108,101,110,97,109,101,99,2, + 0,0,0,0,0,0,0,3,0,0,0,9,0,0,0,67, + 0,0,0,115,42,0,0,0,116,0,0,106,1,0,124,1, + 0,100,1,0,131,2,0,143,17,0,125,2,0,124,2,0, + 106,2,0,131,0,0,83,87,100,2,0,81,82,88,100,2, + 0,83,41,3,122,39,82,101,116,117,114,110,32,116,104,101, + 32,100,97,116,97,32,102,114,111,109,32,112,97,116,104,32, + 97,115,32,114,97,119,32,98,121,116,101,115,46,218,1,114, + 78,41,3,114,49,0,0,0,114,50,0,0,0,90,4,114, + 101,97,100,41,3,114,100,0,0,0,114,35,0,0,0,114, + 54,0,0,0,114,4,0,0,0,114,4,0,0,0,114,5, + 0,0,0,114,194,0,0,0,37,3,0,0,115,4,0,0, + 0,0,2,21,1,122,19,70,105,108,101,76,111,97,100,101, + 114,46,103,101,116,95,100,97,116,97,41,11,114,105,0,0, + 0,114,104,0,0,0,114,106,0,0,0,114,107,0,0,0, + 114,179,0,0,0,114,207,0,0,0,114,209,0,0,0,114, + 116,0,0,0,114,187,0,0,0,114,151,0,0,0,114,194, + 0,0,0,114,4,0,0,0,114,4,0,0,0,41,1,114, + 205,0,0,0,114,5,0,0,0,114,204,0,0,0,2,3, + 0,0,115,14,0,0,0,12,3,6,2,12,6,12,4,12, + 3,24,12,18,5,114,204,0,0,0,99,0,0,0,0,0, + 0,0,0,0,0,0,0,4,0,0,0,64,0,0,0,115, + 64,0,0,0,101,0,0,90,1,0,100,0,0,90,2,0, 100,1,0,90,3,0,100,2,0,100,3,0,132,0,0,90, 4,0,100,4,0,100,5,0,132,0,0,90,5,0,100,6, - 0,100,7,0,132,0,0,90,6,0,100,8,0,100,9,0, - 132,0,0,90,7,0,100,10,0,83,41,11,218,13,95,76, - 111,97,100,101,114,66,97,115,105,99,115,122,83,66,97,115, - 101,32,99,108,97,115,115,32,111,102,32,99,111,109,109,111, - 110,32,99,111,100,101,32,110,101,101,100,101,100,32,98,121, - 32,98,111,116,104,32,83,111,117,114,99,101,76,111,97,100, - 101,114,32,97,110,100,10,32,32,32,32,83,111,117,114,99, - 101,108,101,115,115,70,105,108,101,76,111,97,100,101,114,46, - 99,2,0,0,0,0,0,0,0,5,0,0,0,3,0,0, - 0,67,0,0,0,115,88,0,0,0,116,0,0,124,0,0, - 106,1,0,124,1,0,131,1,0,131,1,0,100,1,0,25, - 125,2,0,124,2,0,106,2,0,100,2,0,100,1,0,131, - 2,0,100,3,0,25,125,3,0,124,1,0,106,3,0,100, - 2,0,131,1,0,100,4,0,25,125,4,0,124,3,0,100, - 5,0,107,2,0,111,87,0,124,4,0,100,5,0,107,3, - 0,83,41,6,122,141,67,111,110,99,114,101,116,101,32,105, - 109,112,108,101,109,101,110,116,97,116,105,111,110,32,111,102, - 32,73,110,115,112,101,99,116,76,111,97,100,101,114,46,105, - 115,95,112,97,99,107,97,103,101,32,98,121,32,99,104,101, - 99,107,105,110,103,32,105,102,10,32,32,32,32,32,32,32, - 32,116,104,101,32,112,97,116,104,32,114,101,116,117,114,110, - 101,100,32,98,121,32,103,101,116,95,102,105,108,101,110,97, - 109,101,32,104,97,115,32,97,32,102,105,108,101,110,97,109, - 101,32,111,102,32,39,95,95,105,110,105,116,95,95,46,112, - 121,39,46,114,29,0,0,0,114,58,0,0,0,114,59,0, - 0,0,114,56,0,0,0,218,8,95,95,105,110,105,116,95, - 95,41,4,114,38,0,0,0,114,157,0,0,0,114,34,0, - 0,0,114,32,0,0,0,41,5,114,108,0,0,0,114,126, - 0,0,0,114,94,0,0,0,90,13,102,105,108,101,110,97, - 109,101,95,98,97,115,101,90,9,116,97,105,108,95,110,97, - 109,101,114,4,0,0,0,114,4,0,0,0,114,5,0,0, - 0,114,159,0,0,0,133,2,0,0,115,8,0,0,0,0, - 3,25,1,22,1,19,1,122,24,95,76,111,97,100,101,114, - 66,97,115,105,99,115,46,105,115,95,112,97,99,107,97,103, - 101,99,2,0,0,0,0,0,0,0,2,0,0,0,1,0, - 0,0,67,0,0,0,115,4,0,0,0,100,1,0,83,41, - 2,122,42,85,115,101,32,100,101,102,97,117,108,116,32,115, - 101,109,97,110,116,105,99,115,32,102,111,114,32,109,111,100, - 117,108,101,32,99,114,101,97,116,105,111,110,46,78,114,4, - 0,0,0,41,2,114,108,0,0,0,114,164,0,0,0,114, - 4,0,0,0,114,4,0,0,0,114,5,0,0,0,218,13, - 99,114,101,97,116,101,95,109,111,100,117,108,101,141,2,0, - 0,115,0,0,0,0,122,27,95,76,111,97,100,101,114,66, - 97,115,105,99,115,46,99,114,101,97,116,101,95,109,111,100, - 117,108,101,99,2,0,0,0,0,0,0,0,3,0,0,0, - 4,0,0,0,67,0,0,0,115,80,0,0,0,124,0,0, - 106,0,0,124,1,0,106,1,0,131,1,0,125,2,0,124, - 2,0,100,1,0,107,8,0,114,54,0,116,2,0,100,2, - 0,106,3,0,124,1,0,106,1,0,131,1,0,131,1,0, - 130,1,0,116,4,0,106,5,0,116,6,0,124,2,0,124, - 1,0,106,7,0,131,3,0,1,100,1,0,83,41,3,122, - 19,69,120,101,99,117,116,101,32,116,104,101,32,109,111,100, - 117,108,101,46,78,122,52,99,97,110,110,111,116,32,108,111, - 97,100,32,109,111,100,117,108,101,32,123,33,114,125,32,119, - 104,101,110,32,103,101,116,95,99,111,100,101,40,41,32,114, - 101,116,117,114,110,115,32,78,111,110,101,41,8,218,8,103, - 101,116,95,99,111,100,101,114,112,0,0,0,114,107,0,0, - 0,114,47,0,0,0,114,121,0,0,0,218,25,95,99,97, - 108,108,95,119,105,116,104,95,102,114,97,109,101,115,95,114, - 101,109,111,118,101,100,218,4,101,120,101,99,114,118,0,0, - 0,41,3,114,108,0,0,0,218,6,109,111,100,117,108,101, - 114,146,0,0,0,114,4,0,0,0,114,4,0,0,0,114, - 5,0,0,0,218,11,101,120,101,99,95,109,111,100,117,108, - 101,144,2,0,0,115,10,0,0,0,0,2,18,1,12,1, - 9,1,15,1,122,25,95,76,111,97,100,101,114,66,97,115, - 105,99,115,46,101,120,101,99,95,109,111,100,117,108,101,99, - 2,0,0,0,0,0,0,0,2,0,0,0,3,0,0,0, - 67,0,0,0,115,16,0,0,0,116,0,0,106,1,0,124, - 0,0,124,1,0,131,2,0,83,41,1,78,41,2,114,121, - 0,0,0,218,17,95,108,111,97,100,95,109,111,100,117,108, - 101,95,115,104,105,109,41,2,114,108,0,0,0,114,126,0, - 0,0,114,4,0,0,0,114,4,0,0,0,114,5,0,0, - 0,218,11,108,111,97,100,95,109,111,100,117,108,101,152,2, - 0,0,115,2,0,0,0,0,1,122,25,95,76,111,97,100, - 101,114,66,97,115,105,99,115,46,108,111,97,100,95,109,111, - 100,117,108,101,78,41,8,114,112,0,0,0,114,111,0,0, - 0,114,113,0,0,0,114,114,0,0,0,114,159,0,0,0, - 114,186,0,0,0,114,191,0,0,0,114,193,0,0,0,114, + 0,100,7,0,100,8,0,100,9,0,132,0,1,90,6,0, + 100,10,0,83,41,11,218,16,83,111,117,114,99,101,70,105, + 108,101,76,111,97,100,101,114,122,62,67,111,110,99,114,101, + 116,101,32,105,109,112,108,101,109,101,110,116,97,116,105,111, + 110,32,111,102,32,83,111,117,114,99,101,76,111,97,100,101, + 114,32,117,115,105,110,103,32,116,104,101,32,102,105,108,101, + 32,115,121,115,116,101,109,46,99,2,0,0,0,0,0,0, + 0,3,0,0,0,4,0,0,0,67,0,0,0,115,34,0, + 0,0,116,0,0,124,1,0,131,1,0,125,2,0,100,1, + 0,124,2,0,106,1,0,100,2,0,124,2,0,106,2,0, + 105,2,0,83,41,3,122,33,82,101,116,117,114,110,32,116, + 104,101,32,109,101,116,97,100,97,116,97,32,102,111,114,32, + 116,104,101,32,112,97,116,104,46,114,126,0,0,0,114,127, + 0,0,0,41,3,114,39,0,0,0,218,8,115,116,95,109, + 116,105,109,101,90,7,115,116,95,115,105,122,101,41,3,114, + 100,0,0,0,114,35,0,0,0,114,202,0,0,0,114,4, + 0,0,0,114,4,0,0,0,114,5,0,0,0,114,191,0, + 0,0,47,3,0,0,115,4,0,0,0,0,2,12,1,122, + 27,83,111,117,114,99,101,70,105,108,101,76,111,97,100,101, + 114,46,112,97,116,104,95,115,116,97,116,115,99,4,0,0, + 0,0,0,0,0,5,0,0,0,5,0,0,0,67,0,0, + 0,115,34,0,0,0,116,0,0,124,1,0,131,1,0,125, + 4,0,124,0,0,106,1,0,124,2,0,124,3,0,100,1, + 0,124,4,0,131,2,1,83,41,2,78,218,5,95,109,111, + 100,101,41,2,114,97,0,0,0,114,192,0,0,0,41,5, + 114,100,0,0,0,114,90,0,0,0,114,89,0,0,0,114, + 53,0,0,0,114,42,0,0,0,114,4,0,0,0,114,4, + 0,0,0,114,5,0,0,0,114,193,0,0,0,52,3,0, + 0,115,4,0,0,0,0,2,12,1,122,32,83,111,117,114, + 99,101,70,105,108,101,76,111,97,100,101,114,46,95,99,97, + 99,104,101,95,98,121,116,101,99,111,100,101,114,214,0,0, + 0,105,182,1,0,0,99,3,0,0,0,1,0,0,0,9, + 0,0,0,17,0,0,0,67,0,0,0,115,62,1,0,0, + 116,0,0,124,1,0,131,1,0,92,2,0,125,4,0,125, + 5,0,103,0,0,125,6,0,120,54,0,124,4,0,114,80, + 0,116,1,0,124,4,0,131,1,0,12,114,80,0,116,0, + 0,124,4,0,131,1,0,92,2,0,125,4,0,125,7,0, + 124,6,0,106,2,0,124,7,0,131,1,0,1,113,27,0, + 87,120,135,0,116,3,0,124,6,0,131,1,0,68,93,121, + 0,125,7,0,116,4,0,124,4,0,124,7,0,131,2,0, + 125,4,0,121,17,0,116,5,0,106,6,0,124,4,0,131, + 1,0,1,87,113,94,0,4,116,7,0,107,10,0,114,155, + 0,1,1,1,119,94,0,89,113,94,0,4,116,8,0,107, + 10,0,114,214,0,1,125,8,0,1,122,28,0,116,9,0, + 106,10,0,100,1,0,124,4,0,124,8,0,131,3,0,1, + 100,2,0,83,87,89,100,2,0,100,2,0,125,8,0,126, + 8,0,88,113,94,0,88,113,94,0,87,121,36,0,116,11, + 0,124,1,0,124,2,0,124,3,0,131,3,0,1,116,9, + 0,106,10,0,100,3,0,124,1,0,131,2,0,1,87,110, + 56,0,4,116,8,0,107,10,0,114,57,1,1,125,8,0, + 1,122,24,0,116,9,0,106,10,0,100,1,0,124,1,0, + 124,8,0,131,3,0,1,87,89,100,2,0,100,2,0,125, + 8,0,126,8,0,88,110,1,0,88,100,2,0,83,41,4, + 122,27,87,114,105,116,101,32,98,121,116,101,115,32,100,97, + 116,97,32,116,111,32,97,32,102,105,108,101,46,122,27,99, + 111,117,108,100,32,110,111,116,32,99,114,101,97,116,101,32, + 123,33,114,125,58,32,123,33,114,125,78,122,12,99,114,101, + 97,116,101,100,32,123,33,114,125,41,12,114,38,0,0,0, + 114,46,0,0,0,114,157,0,0,0,114,33,0,0,0,114, + 28,0,0,0,114,3,0,0,0,90,5,109,107,100,105,114, + 218,15,70,105,108,101,69,120,105,115,116,115,69,114,114,111, + 114,114,40,0,0,0,114,114,0,0,0,114,129,0,0,0, + 114,55,0,0,0,41,9,114,100,0,0,0,114,35,0,0, + 0,114,53,0,0,0,114,214,0,0,0,218,6,112,97,114, + 101,110,116,114,94,0,0,0,114,27,0,0,0,114,23,0, + 0,0,114,195,0,0,0,114,4,0,0,0,114,4,0,0, + 0,114,5,0,0,0,114,192,0,0,0,57,3,0,0,115, + 42,0,0,0,0,2,18,1,6,2,22,1,18,1,17,2, + 19,1,15,1,3,1,17,1,13,2,7,1,18,3,9,1, + 10,1,27,1,3,1,16,1,20,1,18,2,12,1,122,25, + 83,111,117,114,99,101,70,105,108,101,76,111,97,100,101,114, + 46,115,101,116,95,100,97,116,97,78,41,7,114,105,0,0, + 0,114,104,0,0,0,114,106,0,0,0,114,107,0,0,0, + 114,191,0,0,0,114,193,0,0,0,114,192,0,0,0,114, 4,0,0,0,114,4,0,0,0,114,4,0,0,0,114,5, - 0,0,0,114,184,0,0,0,128,2,0,0,115,10,0,0, - 0,12,3,6,2,12,8,12,3,12,8,114,184,0,0,0, - 99,0,0,0,0,0,0,0,0,0,0,0,0,4,0,0, - 0,64,0,0,0,115,106,0,0,0,101,0,0,90,1,0, - 100,0,0,90,2,0,100,1,0,100,2,0,132,0,0,90, - 3,0,100,3,0,100,4,0,132,0,0,90,4,0,100,5, - 0,100,6,0,132,0,0,90,5,0,100,7,0,100,8,0, - 132,0,0,90,6,0,100,9,0,100,10,0,132,0,0,90, - 7,0,100,11,0,100,18,0,100,13,0,100,14,0,132,0, - 1,90,8,0,100,15,0,100,16,0,132,0,0,90,9,0, - 100,17,0,83,41,19,218,12,83,111,117,114,99,101,76,111, - 97,100,101,114,99,2,0,0,0,0,0,0,0,2,0,0, - 0,1,0,0,0,67,0,0,0,115,10,0,0,0,116,0, - 0,130,1,0,100,1,0,83,41,2,122,178,79,112,116,105, - 111,110,97,108,32,109,101,116,104,111,100,32,116,104,97,116, - 32,114,101,116,117,114,110,115,32,116,104,101,32,109,111,100, - 105,102,105,99,97,116,105,111,110,32,116,105,109,101,32,40, - 97,110,32,105,110,116,41,32,102,111,114,32,116,104,101,10, - 32,32,32,32,32,32,32,32,115,112,101,99,105,102,105,101, - 100,32,112,97,116,104,44,32,119,104,101,114,101,32,112,97, - 116,104,32,105,115,32,97,32,115,116,114,46,10,10,32,32, - 32,32,32,32,32,32,82,97,105,115,101,115,32,73,79,69, - 114,114,111,114,32,119,104,101,110,32,116,104,101,32,112,97, - 116,104,32,99,97,110,110,111,116,32,98,101,32,104,97,110, - 100,108,101,100,46,10,32,32,32,32,32,32,32,32,78,41, - 1,218,7,73,79,69,114,114,111,114,41,2,114,108,0,0, - 0,114,35,0,0,0,114,4,0,0,0,114,4,0,0,0, - 114,5,0,0,0,218,10,112,97,116,104,95,109,116,105,109, - 101,158,2,0,0,115,2,0,0,0,0,6,122,23,83,111, - 117,114,99,101,76,111,97,100,101,114,46,112,97,116,104,95, - 109,116,105,109,101,99,2,0,0,0,0,0,0,0,2,0, - 0,0,3,0,0,0,67,0,0,0,115,19,0,0,0,100, - 1,0,124,0,0,106,0,0,124,1,0,131,1,0,105,1, - 0,83,41,2,97,170,1,0,0,79,112,116,105,111,110,97, - 108,32,109,101,116,104,111,100,32,114,101,116,117,114,110,105, - 110,103,32,97,32,109,101,116,97,100,97,116,97,32,100,105, - 99,116,32,102,111,114,32,116,104,101,32,115,112,101,99,105, - 102,105,101,100,32,112,97,116,104,10,32,32,32,32,32,32, - 32,32,116,111,32,98,121,32,116,104,101,32,112,97,116,104, - 32,40,115,116,114,41,46,10,32,32,32,32,32,32,32,32, - 80,111,115,115,105,98,108,101,32,107,101,121,115,58,10,32, - 32,32,32,32,32,32,32,45,32,39,109,116,105,109,101,39, - 32,40,109,97,110,100,97,116,111,114,121,41,32,105,115,32, - 116,104,101,32,110,117,109,101,114,105,99,32,116,105,109,101, - 115,116,97,109,112,32,111,102,32,108,97,115,116,32,115,111, - 117,114,99,101,10,32,32,32,32,32,32,32,32,32,32,99, - 111,100,101,32,109,111,100,105,102,105,99,97,116,105,111,110, - 59,10,32,32,32,32,32,32,32,32,45,32,39,115,105,122, - 101,39,32,40,111,112,116,105,111,110,97,108,41,32,105,115, - 32,116,104,101,32,115,105,122,101,32,105,110,32,98,121,116, - 101,115,32,111,102,32,116,104,101,32,115,111,117,114,99,101, - 32,99,111,100,101,46,10,10,32,32,32,32,32,32,32,32, - 73,109,112,108,101,109,101,110,116,105,110,103,32,116,104,105, - 115,32,109,101,116,104,111,100,32,97,108,108,111,119,115,32, - 116,104,101,32,108,111,97,100,101,114,32,116,111,32,114,101, - 97,100,32,98,121,116,101,99,111,100,101,32,102,105,108,101, - 115,46,10,32,32,32,32,32,32,32,32,82,97,105,115,101, - 115,32,73,79,69,114,114,111,114,32,119,104,101,110,32,116, - 104,101,32,112,97,116,104,32,99,97,110,110,111,116,32,98, - 101,32,104,97,110,100,108,101,100,46,10,32,32,32,32,32, - 32,32,32,114,133,0,0,0,41,1,114,196,0,0,0,41, - 2,114,108,0,0,0,114,35,0,0,0,114,4,0,0,0, - 114,4,0,0,0,114,5,0,0,0,218,10,112,97,116,104, - 95,115,116,97,116,115,166,2,0,0,115,2,0,0,0,0, - 11,122,23,83,111,117,114,99,101,76,111,97,100,101,114,46, - 112,97,116,104,95,115,116,97,116,115,99,4,0,0,0,0, - 0,0,0,4,0,0,0,3,0,0,0,67,0,0,0,115, - 16,0,0,0,124,0,0,106,0,0,124,2,0,124,3,0, - 131,2,0,83,41,1,122,228,79,112,116,105,111,110,97,108, - 32,109,101,116,104,111,100,32,119,104,105,99,104,32,119,114, - 105,116,101,115,32,100,97,116,97,32,40,98,121,116,101,115, - 41,32,116,111,32,97,32,102,105,108,101,32,112,97,116,104, - 32,40,97,32,115,116,114,41,46,10,10,32,32,32,32,32, - 32,32,32,73,109,112,108,101,109,101,110,116,105,110,103,32, - 116,104,105,115,32,109,101,116,104,111,100,32,97,108,108,111, - 119,115,32,102,111,114,32,116,104,101,32,119,114,105,116,105, - 110,103,32,111,102,32,98,121,116,101,99,111,100,101,32,102, - 105,108,101,115,46,10,10,32,32,32,32,32,32,32,32,84, - 104,101,32,115,111,117,114,99,101,32,112,97,116,104,32,105, - 115,32,110,101,101,100,101,100,32,105,110,32,111,114,100,101, - 114,32,116,111,32,99,111,114,114,101,99,116,108,121,32,116, - 114,97,110,115,102,101,114,32,112,101,114,109,105,115,115,105, - 111,110,115,10,32,32,32,32,32,32,32,32,41,1,218,8, - 115,101,116,95,100,97,116,97,41,4,114,108,0,0,0,114, - 90,0,0,0,90,10,99,97,99,104,101,95,112,97,116,104, - 114,53,0,0,0,114,4,0,0,0,114,4,0,0,0,114, - 5,0,0,0,218,15,95,99,97,99,104,101,95,98,121,116, - 101,99,111,100,101,179,2,0,0,115,2,0,0,0,0,8, - 122,28,83,111,117,114,99,101,76,111,97,100,101,114,46,95, - 99,97,99,104,101,95,98,121,116,101,99,111,100,101,99,3, - 0,0,0,0,0,0,0,3,0,0,0,1,0,0,0,67, - 0,0,0,115,4,0,0,0,100,1,0,83,41,2,122,150, - 79,112,116,105,111,110,97,108,32,109,101,116,104,111,100,32, - 119,104,105,99,104,32,119,114,105,116,101,115,32,100,97,116, - 97,32,40,98,121,116,101,115,41,32,116,111,32,97,32,102, - 105,108,101,32,112,97,116,104,32,40,97,32,115,116,114,41, - 46,10,10,32,32,32,32,32,32,32,32,73,109,112,108,101, - 109,101,110,116,105,110,103,32,116,104,105,115,32,109,101,116, - 104,111,100,32,97,108,108,111,119,115,32,102,111,114,32,116, - 104,101,32,119,114,105,116,105,110,103,32,111,102,32,98,121, - 116,101,99,111,100,101,32,102,105,108,101,115,46,10,32,32, - 32,32,32,32,32,32,78,114,4,0,0,0,41,3,114,108, - 0,0,0,114,35,0,0,0,114,53,0,0,0,114,4,0, - 0,0,114,4,0,0,0,114,5,0,0,0,114,198,0,0, - 0,189,2,0,0,115,0,0,0,0,122,21,83,111,117,114, - 99,101,76,111,97,100,101,114,46,115,101,116,95,100,97,116, - 97,99,2,0,0,0,0,0,0,0,5,0,0,0,16,0, - 0,0,67,0,0,0,115,105,0,0,0,124,0,0,106,0, - 0,124,1,0,131,1,0,125,2,0,121,19,0,124,0,0, - 106,1,0,124,2,0,131,1,0,125,3,0,87,110,58,0, - 4,116,2,0,107,10,0,114,94,0,1,125,4,0,1,122, - 26,0,116,3,0,100,1,0,100,2,0,124,1,0,131,1, - 1,124,4,0,130,2,0,87,89,100,3,0,100,3,0,125, - 4,0,126,4,0,88,110,1,0,88,116,4,0,124,3,0, - 131,1,0,83,41,4,122,52,67,111,110,99,114,101,116,101, - 32,105,109,112,108,101,109,101,110,116,97,116,105,111,110,32, - 111,102,32,73,110,115,112,101,99,116,76,111,97,100,101,114, - 46,103,101,116,95,115,111,117,114,99,101,46,122,39,115,111, - 117,114,99,101,32,110,111,116,32,97,118,97,105,108,97,98, - 108,101,32,116,104,114,111,117,103,104,32,103,101,116,95,100, - 97,116,97,40,41,114,106,0,0,0,78,41,5,114,157,0, - 0,0,218,8,103,101,116,95,100,97,116,97,114,40,0,0, - 0,114,107,0,0,0,114,155,0,0,0,41,5,114,108,0, - 0,0,114,126,0,0,0,114,35,0,0,0,114,153,0,0, - 0,218,3,101,120,99,114,4,0,0,0,114,4,0,0,0, - 114,5,0,0,0,218,10,103,101,116,95,115,111,117,114,99, - 101,196,2,0,0,115,14,0,0,0,0,2,15,1,3,1, - 19,1,18,1,9,1,31,1,122,23,83,111,117,114,99,101, - 76,111,97,100,101,114,46,103,101,116,95,115,111,117,114,99, - 101,218,9,95,111,112,116,105,109,105,122,101,114,29,0,0, - 0,99,3,0,0,0,1,0,0,0,4,0,0,0,9,0, - 0,0,67,0,0,0,115,34,0,0,0,116,0,0,106,1, - 0,116,2,0,124,1,0,124,2,0,100,1,0,100,2,0, - 100,3,0,100,4,0,124,3,0,131,4,2,83,41,5,122, - 130,82,101,116,117,114,110,32,116,104,101,32,99,111,100,101, - 32,111,98,106,101,99,116,32,99,111,109,112,105,108,101,100, - 32,102,114,111,109,32,115,111,117,114,99,101,46,10,10,32, - 32,32,32,32,32,32,32,84,104,101,32,39,100,97,116,97, - 39,32,97,114,103,117,109,101,110,116,32,99,97,110,32,98, - 101,32,97,110,121,32,111,98,106,101,99,116,32,116,121,112, - 101,32,116,104,97,116,32,99,111,109,112,105,108,101,40,41, - 32,115,117,112,112,111,114,116,115,46,10,32,32,32,32,32, - 32,32,32,114,189,0,0,0,218,12,100,111,110,116,95,105, - 110,104,101,114,105,116,84,114,68,0,0,0,41,3,114,121, - 0,0,0,114,188,0,0,0,218,7,99,111,109,112,105,108, - 101,41,4,114,108,0,0,0,114,53,0,0,0,114,35,0, - 0,0,114,203,0,0,0,114,4,0,0,0,114,4,0,0, - 0,114,5,0,0,0,218,14,115,111,117,114,99,101,95,116, - 111,95,99,111,100,101,206,2,0,0,115,4,0,0,0,0, - 5,21,1,122,27,83,111,117,114,99,101,76,111,97,100,101, - 114,46,115,111,117,114,99,101,95,116,111,95,99,111,100,101, - 99,2,0,0,0,0,0,0,0,10,0,0,0,43,0,0, - 0,67,0,0,0,115,174,1,0,0,124,0,0,106,0,0, - 124,1,0,131,1,0,125,2,0,100,1,0,125,3,0,121, - 16,0,116,1,0,124,2,0,131,1,0,125,4,0,87,110, - 24,0,4,116,2,0,107,10,0,114,63,0,1,1,1,100, - 1,0,125,4,0,89,110,202,0,88,121,19,0,124,0,0, - 106,3,0,124,2,0,131,1,0,125,5,0,87,110,18,0, - 4,116,4,0,107,10,0,114,103,0,1,1,1,89,110,162, - 0,88,116,5,0,124,5,0,100,2,0,25,131,1,0,125, - 3,0,121,19,0,124,0,0,106,6,0,124,4,0,131,1, - 0,125,6,0,87,110,18,0,4,116,7,0,107,10,0,114, - 159,0,1,1,1,89,110,106,0,88,121,34,0,116,8,0, - 124,6,0,100,3,0,124,5,0,100,4,0,124,1,0,100, - 5,0,124,4,0,131,1,3,125,7,0,87,110,24,0,4, - 116,9,0,116,10,0,102,2,0,107,10,0,114,220,0,1, - 1,1,89,110,45,0,88,116,11,0,100,6,0,124,4,0, - 124,2,0,131,3,0,1,116,12,0,124,7,0,100,4,0, - 124,1,0,100,7,0,124,4,0,100,8,0,124,2,0,131, - 1,3,83,124,0,0,106,6,0,124,2,0,131,1,0,125, - 8,0,124,0,0,106,13,0,124,8,0,124,2,0,131,2, - 0,125,9,0,116,11,0,100,9,0,124,2,0,131,2,0, - 1,116,14,0,106,15,0,12,114,170,1,124,4,0,100,1, - 0,107,9,0,114,170,1,124,3,0,100,1,0,107,9,0, - 114,170,1,116,16,0,124,9,0,124,3,0,116,17,0,124, - 8,0,131,1,0,131,3,0,125,6,0,121,36,0,124,0, - 0,106,18,0,124,2,0,124,4,0,124,6,0,131,3,0, - 1,116,11,0,100,10,0,124,4,0,131,2,0,1,87,110, - 18,0,4,116,2,0,107,10,0,114,169,1,1,1,1,89, - 110,1,0,88,124,9,0,83,41,11,122,190,67,111,110,99, - 114,101,116,101,32,105,109,112,108,101,109,101,110,116,97,116, - 105,111,110,32,111,102,32,73,110,115,112,101,99,116,76,111, - 97,100,101,114,46,103,101,116,95,99,111,100,101,46,10,10, - 32,32,32,32,32,32,32,32,82,101,97,100,105,110,103,32, - 111,102,32,98,121,116,101,99,111,100,101,32,114,101,113,117, - 105,114,101,115,32,112,97,116,104,95,115,116,97,116,115,32, - 116,111,32,98,101,32,105,109,112,108,101,109,101,110,116,101, - 100,46,32,84,111,32,119,114,105,116,101,10,32,32,32,32, - 32,32,32,32,98,121,116,101,99,111,100,101,44,32,115,101, - 116,95,100,97,116,97,32,109,117,115,116,32,97,108,115,111, - 32,98,101,32,105,109,112,108,101,109,101,110,116,101,100,46, - 10,10,32,32,32,32,32,32,32,32,78,114,133,0,0,0, - 114,138,0,0,0,114,106,0,0,0,114,35,0,0,0,122, - 13,123,125,32,109,97,116,99,104,101,115,32,123,125,114,89, - 0,0,0,114,90,0,0,0,122,19,99,111,100,101,32,111, - 98,106,101,99,116,32,102,114,111,109,32,123,125,122,10,119, - 114,111,116,101,32,123,33,114,125,41,19,114,157,0,0,0, - 114,79,0,0,0,114,66,0,0,0,114,197,0,0,0,114, - 195,0,0,0,114,14,0,0,0,114,200,0,0,0,114,40, - 0,0,0,114,141,0,0,0,114,107,0,0,0,114,136,0, - 0,0,114,105,0,0,0,114,147,0,0,0,114,206,0,0, - 0,114,7,0,0,0,218,19,100,111,110,116,95,119,114,105, - 116,101,95,98,121,116,101,99,111,100,101,114,150,0,0,0, - 114,31,0,0,0,114,199,0,0,0,41,10,114,108,0,0, - 0,114,126,0,0,0,114,90,0,0,0,114,139,0,0,0, - 114,89,0,0,0,218,2,115,116,114,53,0,0,0,218,10, - 98,121,116,101,115,95,100,97,116,97,114,153,0,0,0,90, - 11,99,111,100,101,95,111,98,106,101,99,116,114,4,0,0, - 0,114,4,0,0,0,114,5,0,0,0,114,187,0,0,0, - 214,2,0,0,115,78,0,0,0,0,7,15,1,6,1,3, - 1,16,1,13,1,11,2,3,1,19,1,13,1,5,2,16, - 1,3,1,19,1,13,1,5,2,3,1,9,1,12,1,13, - 1,19,1,5,2,9,1,7,1,15,1,6,1,7,1,15, - 1,18,1,13,1,22,1,12,1,9,1,15,1,3,1,19, - 1,17,1,13,1,5,1,122,21,83,111,117,114,99,101,76, - 111,97,100,101,114,46,103,101,116,95,99,111,100,101,78,114, - 87,0,0,0,41,10,114,112,0,0,0,114,111,0,0,0, - 114,113,0,0,0,114,196,0,0,0,114,197,0,0,0,114, - 199,0,0,0,114,198,0,0,0,114,202,0,0,0,114,206, - 0,0,0,114,187,0,0,0,114,4,0,0,0,114,4,0, - 0,0,114,4,0,0,0,114,5,0,0,0,114,194,0,0, - 0,156,2,0,0,115,14,0,0,0,12,2,12,8,12,13, - 12,10,12,7,12,10,18,8,114,194,0,0,0,99,0,0, - 0,0,0,0,0,0,0,0,0,0,4,0,0,0,0,0, - 0,0,115,112,0,0,0,101,0,0,90,1,0,100,0,0, - 90,2,0,100,1,0,90,3,0,100,2,0,100,3,0,132, - 0,0,90,4,0,100,4,0,100,5,0,132,0,0,90,5, - 0,100,6,0,100,7,0,132,0,0,90,6,0,101,7,0, - 135,0,0,102,1,0,100,8,0,100,9,0,134,0,0,131, - 1,0,90,8,0,101,7,0,100,10,0,100,11,0,132,0, - 0,131,1,0,90,9,0,100,12,0,100,13,0,132,0,0, - 90,10,0,135,0,0,83,41,14,218,10,70,105,108,101,76, - 111,97,100,101,114,122,103,66,97,115,101,32,102,105,108,101, - 32,108,111,97,100,101,114,32,99,108,97,115,115,32,119,104, - 105,99,104,32,105,109,112,108,101,109,101,110,116,115,32,116, - 104,101,32,108,111,97,100,101,114,32,112,114,111,116,111,99, - 111,108,32,109,101,116,104,111,100,115,32,116,104,97,116,10, - 32,32,32,32,114,101,113,117,105,114,101,32,102,105,108,101, - 32,115,121,115,116,101,109,32,117,115,97,103,101,46,99,3, - 0,0,0,0,0,0,0,3,0,0,0,2,0,0,0,67, - 0,0,0,115,22,0,0,0,124,1,0,124,0,0,95,0, - 0,124,2,0,124,0,0,95,1,0,100,1,0,83,41,2, - 122,75,67,97,99,104,101,32,116,104,101,32,109,111,100,117, - 108,101,32,110,97,109,101,32,97,110,100,32,116,104,101,32, - 112,97,116,104,32,116,111,32,116,104,101,32,102,105,108,101, - 32,102,111,117,110,100,32,98,121,32,116,104,101,10,32,32, - 32,32,32,32,32,32,102,105,110,100,101,114,46,78,41,2, - 114,106,0,0,0,114,35,0,0,0,41,3,114,108,0,0, - 0,114,126,0,0,0,114,35,0,0,0,114,4,0,0,0, - 114,4,0,0,0,114,5,0,0,0,114,185,0,0,0,15, - 3,0,0,115,4,0,0,0,0,3,9,1,122,19,70,105, - 108,101,76,111,97,100,101,114,46,95,95,105,110,105,116,95, - 95,99,2,0,0,0,0,0,0,0,2,0,0,0,2,0, - 0,0,67,0,0,0,115,34,0,0,0,124,0,0,106,0, - 0,124,1,0,106,0,0,107,2,0,111,33,0,124,0,0, - 106,1,0,124,1,0,106,1,0,107,2,0,83,41,1,78, - 41,2,218,9,95,95,99,108,97,115,115,95,95,114,118,0, - 0,0,41,2,114,108,0,0,0,218,5,111,116,104,101,114, - 114,4,0,0,0,114,4,0,0,0,114,5,0,0,0,218, - 6,95,95,101,113,95,95,21,3,0,0,115,4,0,0,0, - 0,1,18,1,122,17,70,105,108,101,76,111,97,100,101,114, - 46,95,95,101,113,95,95,99,1,0,0,0,0,0,0,0, - 1,0,0,0,3,0,0,0,67,0,0,0,115,26,0,0, - 0,116,0,0,124,0,0,106,1,0,131,1,0,116,0,0, - 124,0,0,106,2,0,131,1,0,65,83,41,1,78,41,3, - 218,4,104,97,115,104,114,106,0,0,0,114,35,0,0,0, - 41,1,114,108,0,0,0,114,4,0,0,0,114,4,0,0, - 0,114,5,0,0,0,218,8,95,95,104,97,115,104,95,95, - 25,3,0,0,115,2,0,0,0,0,1,122,19,70,105,108, - 101,76,111,97,100,101,114,46,95,95,104,97,115,104,95,95, - 99,2,0,0,0,0,0,0,0,2,0,0,0,3,0,0, - 0,3,0,0,0,115,22,0,0,0,116,0,0,116,1,0, - 124,0,0,131,2,0,106,2,0,124,1,0,131,1,0,83, - 41,1,122,100,76,111,97,100,32,97,32,109,111,100,117,108, - 101,32,102,114,111,109,32,97,32,102,105,108,101,46,10,10, - 32,32,32,32,32,32,32,32,84,104,105,115,32,109,101,116, - 104,111,100,32,105,115,32,100,101,112,114,101,99,97,116,101, - 100,46,32,32,85,115,101,32,101,120,101,99,95,109,111,100, - 117,108,101,40,41,32,105,110,115,116,101,97,100,46,10,10, - 32,32,32,32,32,32,32,32,41,3,218,5,115,117,112,101, - 114,114,210,0,0,0,114,193,0,0,0,41,2,114,108,0, - 0,0,114,126,0,0,0,41,1,114,211,0,0,0,114,4, - 0,0,0,114,5,0,0,0,114,193,0,0,0,28,3,0, - 0,115,2,0,0,0,0,10,122,22,70,105,108,101,76,111, - 97,100,101,114,46,108,111,97,100,95,109,111,100,117,108,101, - 99,2,0,0,0,0,0,0,0,2,0,0,0,1,0,0, - 0,67,0,0,0,115,7,0,0,0,124,0,0,106,0,0, - 83,41,1,122,58,82,101,116,117,114,110,32,116,104,101,32, - 112,97,116,104,32,116,111,32,116,104,101,32,115,111,117,114, - 99,101,32,102,105,108,101,32,97,115,32,102,111,117,110,100, - 32,98,121,32,116,104,101,32,102,105,110,100,101,114,46,41, - 1,114,35,0,0,0,41,2,114,108,0,0,0,114,126,0, - 0,0,114,4,0,0,0,114,4,0,0,0,114,5,0,0, - 0,114,157,0,0,0,40,3,0,0,115,2,0,0,0,0, - 3,122,23,70,105,108,101,76,111,97,100,101,114,46,103,101, - 116,95,102,105,108,101,110,97,109,101,99,2,0,0,0,0, - 0,0,0,3,0,0,0,9,0,0,0,67,0,0,0,115, - 42,0,0,0,116,0,0,106,1,0,124,1,0,100,1,0, - 131,2,0,143,17,0,125,2,0,124,2,0,106,2,0,131, - 0,0,83,87,100,2,0,81,82,88,100,2,0,83,41,3, - 122,39,82,101,116,117,114,110,32,116,104,101,32,100,97,116, - 97,32,102,114,111,109,32,112,97,116,104,32,97,115,32,114, - 97,119,32,98,121,116,101,115,46,218,1,114,78,41,3,114, - 49,0,0,0,114,50,0,0,0,90,4,114,101,97,100,41, - 3,114,108,0,0,0,114,35,0,0,0,114,54,0,0,0, + 0,0,0,114,212,0,0,0,43,3,0,0,115,8,0,0, + 0,12,2,6,2,12,5,12,5,114,212,0,0,0,99,0, + 0,0,0,0,0,0,0,0,0,0,0,2,0,0,0,64, + 0,0,0,115,46,0,0,0,101,0,0,90,1,0,100,0, + 0,90,2,0,100,1,0,90,3,0,100,2,0,100,3,0, + 132,0,0,90,4,0,100,4,0,100,5,0,132,0,0,90, + 5,0,100,6,0,83,41,7,218,20,83,111,117,114,99,101, + 108,101,115,115,70,105,108,101,76,111,97,100,101,114,122,45, + 76,111,97,100,101,114,32,119,104,105,99,104,32,104,97,110, + 100,108,101,115,32,115,111,117,114,99,101,108,101,115,115,32, + 102,105,108,101,32,105,109,112,111,114,116,115,46,99,2,0, + 0,0,0,0,0,0,5,0,0,0,6,0,0,0,67,0, + 0,0,115,76,0,0,0,124,0,0,106,0,0,124,1,0, + 131,1,0,125,2,0,124,0,0,106,1,0,124,2,0,131, + 1,0,125,3,0,116,2,0,124,3,0,100,1,0,124,1, + 0,100,2,0,124,2,0,131,1,2,125,4,0,116,3,0, + 124,4,0,100,1,0,124,1,0,100,3,0,124,2,0,131, + 1,2,83,41,4,78,114,98,0,0,0,114,35,0,0,0, + 114,89,0,0,0,41,4,114,151,0,0,0,114,194,0,0, + 0,114,135,0,0,0,114,141,0,0,0,41,5,114,100,0, + 0,0,114,119,0,0,0,114,35,0,0,0,114,53,0,0, + 0,114,203,0,0,0,114,4,0,0,0,114,4,0,0,0, + 114,5,0,0,0,114,181,0,0,0,92,3,0,0,115,8, + 0,0,0,0,1,15,1,15,1,24,1,122,29,83,111,117, + 114,99,101,108,101,115,115,70,105,108,101,76,111,97,100,101, + 114,46,103,101,116,95,99,111,100,101,99,2,0,0,0,0, + 0,0,0,2,0,0,0,1,0,0,0,67,0,0,0,115, + 4,0,0,0,100,1,0,83,41,2,122,39,82,101,116,117, + 114,110,32,78,111,110,101,32,97,115,32,116,104,101,114,101, + 32,105,115,32,110,111,32,115,111,117,114,99,101,32,99,111, + 100,101,46,78,114,4,0,0,0,41,2,114,100,0,0,0, + 114,119,0,0,0,114,4,0,0,0,114,4,0,0,0,114, + 5,0,0,0,114,196,0,0,0,98,3,0,0,115,2,0, + 0,0,0,2,122,31,83,111,117,114,99,101,108,101,115,115, + 70,105,108,101,76,111,97,100,101,114,46,103,101,116,95,115, + 111,117,114,99,101,78,41,6,114,105,0,0,0,114,104,0, + 0,0,114,106,0,0,0,114,107,0,0,0,114,181,0,0, + 0,114,196,0,0,0,114,4,0,0,0,114,4,0,0,0, + 114,4,0,0,0,114,5,0,0,0,114,217,0,0,0,88, + 3,0,0,115,6,0,0,0,12,2,6,2,12,6,114,217, + 0,0,0,99,0,0,0,0,0,0,0,0,0,0,0,0, + 3,0,0,0,64,0,0,0,115,136,0,0,0,101,0,0, + 90,1,0,100,0,0,90,2,0,100,1,0,90,3,0,100, + 2,0,100,3,0,132,0,0,90,4,0,100,4,0,100,5, + 0,132,0,0,90,5,0,100,6,0,100,7,0,132,0,0, + 90,6,0,100,8,0,100,9,0,132,0,0,90,7,0,100, + 10,0,100,11,0,132,0,0,90,8,0,100,12,0,100,13, + 0,132,0,0,90,9,0,100,14,0,100,15,0,132,0,0, + 90,10,0,100,16,0,100,17,0,132,0,0,90,11,0,101, + 12,0,100,18,0,100,19,0,132,0,0,131,1,0,90,13, + 0,100,20,0,83,41,21,218,19,69,120,116,101,110,115,105, + 111,110,70,105,108,101,76,111,97,100,101,114,122,93,76,111, + 97,100,101,114,32,102,111,114,32,101,120,116,101,110,115,105, + 111,110,32,109,111,100,117,108,101,115,46,10,10,32,32,32, + 32,84,104,101,32,99,111,110,115,116,114,117,99,116,111,114, + 32,105,115,32,100,101,115,105,103,110,101,100,32,116,111,32, + 119,111,114,107,32,119,105,116,104,32,70,105,108,101,70,105, + 110,100,101,114,46,10,10,32,32,32,32,99,3,0,0,0, + 0,0,0,0,3,0,0,0,2,0,0,0,67,0,0,0, + 115,22,0,0,0,124,1,0,124,0,0,95,0,0,124,2, + 0,124,0,0,95,1,0,100,0,0,83,41,1,78,41,2, + 114,98,0,0,0,114,35,0,0,0,41,3,114,100,0,0, + 0,114,98,0,0,0,114,35,0,0,0,114,4,0,0,0, + 114,4,0,0,0,114,5,0,0,0,114,179,0,0,0,115, + 3,0,0,115,4,0,0,0,0,1,9,1,122,28,69,120, + 116,101,110,115,105,111,110,70,105,108,101,76,111,97,100,101, + 114,46,95,95,105,110,105,116,95,95,99,2,0,0,0,0, + 0,0,0,2,0,0,0,2,0,0,0,67,0,0,0,115, + 34,0,0,0,124,0,0,106,0,0,124,1,0,106,0,0, + 107,2,0,111,33,0,124,0,0,106,1,0,124,1,0,106, + 1,0,107,2,0,83,41,1,78,41,2,114,205,0,0,0, + 114,111,0,0,0,41,2,114,100,0,0,0,114,206,0,0, + 0,114,4,0,0,0,114,4,0,0,0,114,5,0,0,0, + 114,207,0,0,0,119,3,0,0,115,4,0,0,0,0,1, + 18,1,122,26,69,120,116,101,110,115,105,111,110,70,105,108, + 101,76,111,97,100,101,114,46,95,95,101,113,95,95,99,1, + 0,0,0,0,0,0,0,1,0,0,0,3,0,0,0,67, + 0,0,0,115,26,0,0,0,116,0,0,124,0,0,106,1, + 0,131,1,0,116,0,0,124,0,0,106,2,0,131,1,0, + 65,83,41,1,78,41,3,114,208,0,0,0,114,98,0,0, + 0,114,35,0,0,0,41,1,114,100,0,0,0,114,4,0, + 0,0,114,4,0,0,0,114,5,0,0,0,114,209,0,0, + 0,123,3,0,0,115,2,0,0,0,0,1,122,28,69,120, + 116,101,110,115,105,111,110,70,105,108,101,76,111,97,100,101, + 114,46,95,95,104,97,115,104,95,95,99,2,0,0,0,0, + 0,0,0,3,0,0,0,4,0,0,0,67,0,0,0,115, + 50,0,0,0,116,0,0,106,1,0,116,2,0,106,3,0, + 124,1,0,131,2,0,125,2,0,116,0,0,106,4,0,100, + 1,0,124,1,0,106,5,0,124,0,0,106,6,0,131,3, + 0,1,124,2,0,83,41,2,122,38,67,114,101,97,116,101, + 32,97,110,32,117,110,105,116,105,97,108,105,122,101,100,32, + 101,120,116,101,110,115,105,111,110,32,109,111,100,117,108,101, + 122,38,101,120,116,101,110,115,105,111,110,32,109,111,100,117, + 108,101,32,123,33,114,125,32,108,111,97,100,101,100,32,102, + 114,111,109,32,123,33,114,125,41,7,114,114,0,0,0,114, + 182,0,0,0,114,139,0,0,0,90,14,99,114,101,97,116, + 101,95,100,121,110,97,109,105,99,114,129,0,0,0,114,98, + 0,0,0,114,35,0,0,0,41,3,114,100,0,0,0,114, + 158,0,0,0,114,184,0,0,0,114,4,0,0,0,114,4, + 0,0,0,114,5,0,0,0,114,180,0,0,0,126,3,0, + 0,115,10,0,0,0,0,2,6,1,15,1,9,1,16,1, + 122,33,69,120,116,101,110,115,105,111,110,70,105,108,101,76, + 111,97,100,101,114,46,99,114,101,97,116,101,95,109,111,100, + 117,108,101,99,2,0,0,0,0,0,0,0,2,0,0,0, + 4,0,0,0,67,0,0,0,115,48,0,0,0,116,0,0, + 106,1,0,116,2,0,106,3,0,124,1,0,131,2,0,1, + 116,0,0,106,4,0,100,1,0,124,0,0,106,5,0,124, + 0,0,106,6,0,131,3,0,1,100,2,0,83,41,3,122, + 30,73,110,105,116,105,97,108,105,122,101,32,97,110,32,101, + 120,116,101,110,115,105,111,110,32,109,111,100,117,108,101,122, + 40,101,120,116,101,110,115,105,111,110,32,109,111,100,117,108, + 101,32,123,33,114,125,32,101,120,101,99,117,116,101,100,32, + 102,114,111,109,32,123,33,114,125,78,41,7,114,114,0,0, + 0,114,182,0,0,0,114,139,0,0,0,90,12,101,120,101, + 99,95,100,121,110,97,109,105,99,114,129,0,0,0,114,98, + 0,0,0,114,35,0,0,0,41,2,114,100,0,0,0,114, + 184,0,0,0,114,4,0,0,0,114,4,0,0,0,114,5, + 0,0,0,114,185,0,0,0,134,3,0,0,115,6,0,0, + 0,0,2,19,1,9,1,122,31,69,120,116,101,110,115,105, + 111,110,70,105,108,101,76,111,97,100,101,114,46,101,120,101, + 99,95,109,111,100,117,108,101,99,2,0,0,0,0,0,0, + 0,2,0,0,0,4,0,0,0,3,0,0,0,115,48,0, + 0,0,116,0,0,124,0,0,106,1,0,131,1,0,100,1, + 0,25,137,0,0,116,2,0,135,0,0,102,1,0,100,2, + 0,100,3,0,134,0,0,116,3,0,68,131,1,0,131,1, + 0,83,41,4,122,49,82,101,116,117,114,110,32,84,114,117, + 101,32,105,102,32,116,104,101,32,101,120,116,101,110,115,105, + 111,110,32,109,111,100,117,108,101,32,105,115,32,97,32,112, + 97,99,107,97,103,101,46,114,29,0,0,0,99,1,0,0, + 0,0,0,0,0,2,0,0,0,4,0,0,0,51,0,0, + 0,115,31,0,0,0,124,0,0,93,21,0,125,1,0,136, + 0,0,100,0,0,124,1,0,23,107,2,0,86,1,113,3, + 0,100,1,0,83,41,2,114,179,0,0,0,78,114,4,0, + 0,0,41,2,114,22,0,0,0,218,6,115,117,102,102,105, + 120,41,1,218,9,102,105,108,101,95,110,97,109,101,114,4, + 0,0,0,114,5,0,0,0,250,9,60,103,101,110,101,120, + 112,114,62,143,3,0,0,115,2,0,0,0,6,1,122,49, + 69,120,116,101,110,115,105,111,110,70,105,108,101,76,111,97, + 100,101,114,46,105,115,95,112,97,99,107,97,103,101,46,60, + 108,111,99,97,108,115,62,46,60,103,101,110,101,120,112,114, + 62,41,4,114,38,0,0,0,114,35,0,0,0,218,3,97, + 110,121,218,18,69,88,84,69,78,83,73,79,78,95,83,85, + 70,70,73,88,69,83,41,2,114,100,0,0,0,114,119,0, + 0,0,114,4,0,0,0,41,1,114,220,0,0,0,114,5, + 0,0,0,114,153,0,0,0,140,3,0,0,115,6,0,0, + 0,0,2,19,1,18,1,122,30,69,120,116,101,110,115,105, + 111,110,70,105,108,101,76,111,97,100,101,114,46,105,115,95, + 112,97,99,107,97,103,101,99,2,0,0,0,0,0,0,0, + 2,0,0,0,1,0,0,0,67,0,0,0,115,4,0,0, + 0,100,1,0,83,41,2,122,63,82,101,116,117,114,110,32, + 78,111,110,101,32,97,115,32,97,110,32,101,120,116,101,110, + 115,105,111,110,32,109,111,100,117,108,101,32,99,97,110,110, + 111,116,32,99,114,101,97,116,101,32,97,32,99,111,100,101, + 32,111,98,106,101,99,116,46,78,114,4,0,0,0,41,2, + 114,100,0,0,0,114,119,0,0,0,114,4,0,0,0,114, + 4,0,0,0,114,5,0,0,0,114,181,0,0,0,146,3, + 0,0,115,2,0,0,0,0,2,122,28,69,120,116,101,110, + 115,105,111,110,70,105,108,101,76,111,97,100,101,114,46,103, + 101,116,95,99,111,100,101,99,2,0,0,0,0,0,0,0, + 2,0,0,0,1,0,0,0,67,0,0,0,115,4,0,0, + 0,100,1,0,83,41,2,122,53,82,101,116,117,114,110,32, + 78,111,110,101,32,97,115,32,101,120,116,101,110,115,105,111, + 110,32,109,111,100,117,108,101,115,32,104,97,118,101,32,110, + 111,32,115,111,117,114,99,101,32,99,111,100,101,46,78,114, + 4,0,0,0,41,2,114,100,0,0,0,114,119,0,0,0, 114,4,0,0,0,114,4,0,0,0,114,5,0,0,0,114, - 200,0,0,0,45,3,0,0,115,4,0,0,0,0,2,21, - 1,122,19,70,105,108,101,76,111,97,100,101,114,46,103,101, - 116,95,100,97,116,97,41,11,114,112,0,0,0,114,111,0, - 0,0,114,113,0,0,0,114,114,0,0,0,114,185,0,0, - 0,114,213,0,0,0,114,215,0,0,0,114,123,0,0,0, - 114,193,0,0,0,114,157,0,0,0,114,200,0,0,0,114, - 4,0,0,0,114,4,0,0,0,41,1,114,211,0,0,0, - 114,5,0,0,0,114,210,0,0,0,10,3,0,0,115,14, - 0,0,0,12,3,6,2,12,6,12,4,12,3,24,12,18, - 5,114,210,0,0,0,99,0,0,0,0,0,0,0,0,0, - 0,0,0,4,0,0,0,64,0,0,0,115,64,0,0,0, + 196,0,0,0,150,3,0,0,115,2,0,0,0,0,2,122, + 30,69,120,116,101,110,115,105,111,110,70,105,108,101,76,111, + 97,100,101,114,46,103,101,116,95,115,111,117,114,99,101,99, + 2,0,0,0,0,0,0,0,2,0,0,0,1,0,0,0, + 67,0,0,0,115,7,0,0,0,124,0,0,106,0,0,83, + 41,1,122,58,82,101,116,117,114,110,32,116,104,101,32,112, + 97,116,104,32,116,111,32,116,104,101,32,115,111,117,114,99, + 101,32,102,105,108,101,32,97,115,32,102,111,117,110,100,32, + 98,121,32,116,104,101,32,102,105,110,100,101,114,46,41,1, + 114,35,0,0,0,41,2,114,100,0,0,0,114,119,0,0, + 0,114,4,0,0,0,114,4,0,0,0,114,5,0,0,0, + 114,151,0,0,0,154,3,0,0,115,2,0,0,0,0,3, + 122,32,69,120,116,101,110,115,105,111,110,70,105,108,101,76, + 111,97,100,101,114,46,103,101,116,95,102,105,108,101,110,97, + 109,101,78,41,14,114,105,0,0,0,114,104,0,0,0,114, + 106,0,0,0,114,107,0,0,0,114,179,0,0,0,114,207, + 0,0,0,114,209,0,0,0,114,180,0,0,0,114,185,0, + 0,0,114,153,0,0,0,114,181,0,0,0,114,196,0,0, + 0,114,116,0,0,0,114,151,0,0,0,114,4,0,0,0, + 114,4,0,0,0,114,4,0,0,0,114,5,0,0,0,114, + 218,0,0,0,107,3,0,0,115,20,0,0,0,12,6,6, + 2,12,4,12,4,12,3,12,8,12,6,12,6,12,4,12, + 4,114,218,0,0,0,99,0,0,0,0,0,0,0,0,0, + 0,0,0,2,0,0,0,64,0,0,0,115,130,0,0,0, 101,0,0,90,1,0,100,0,0,90,2,0,100,1,0,90, 3,0,100,2,0,100,3,0,132,0,0,90,4,0,100,4, 0,100,5,0,132,0,0,90,5,0,100,6,0,100,7,0, - 100,8,0,100,9,0,132,0,1,90,6,0,100,10,0,83, - 41,11,218,16,83,111,117,114,99,101,70,105,108,101,76,111, - 97,100,101,114,122,62,67,111,110,99,114,101,116,101,32,105, - 109,112,108,101,109,101,110,116,97,116,105,111,110,32,111,102, - 32,83,111,117,114,99,101,76,111,97,100,101,114,32,117,115, - 105,110,103,32,116,104,101,32,102,105,108,101,32,115,121,115, - 116,101,109,46,99,2,0,0,0,0,0,0,0,3,0,0, - 0,4,0,0,0,67,0,0,0,115,34,0,0,0,116,0, - 0,124,1,0,131,1,0,125,2,0,100,1,0,124,2,0, - 106,1,0,100,2,0,124,2,0,106,2,0,105,2,0,83, - 41,3,122,33,82,101,116,117,114,110,32,116,104,101,32,109, - 101,116,97,100,97,116,97,32,102,111,114,32,116,104,101,32, - 112,97,116,104,46,114,133,0,0,0,114,134,0,0,0,41, - 3,114,39,0,0,0,218,8,115,116,95,109,116,105,109,101, - 90,7,115,116,95,115,105,122,101,41,3,114,108,0,0,0, - 114,35,0,0,0,114,208,0,0,0,114,4,0,0,0,114, - 4,0,0,0,114,5,0,0,0,114,197,0,0,0,55,3, - 0,0,115,4,0,0,0,0,2,12,1,122,27,83,111,117, - 114,99,101,70,105,108,101,76,111,97,100,101,114,46,112,97, - 116,104,95,115,116,97,116,115,99,4,0,0,0,0,0,0, - 0,5,0,0,0,5,0,0,0,67,0,0,0,115,34,0, - 0,0,116,0,0,124,1,0,131,1,0,125,4,0,124,0, - 0,106,1,0,124,2,0,124,3,0,100,1,0,124,4,0, - 131,2,1,83,41,2,78,218,5,95,109,111,100,101,41,2, - 114,97,0,0,0,114,198,0,0,0,41,5,114,108,0,0, - 0,114,90,0,0,0,114,89,0,0,0,114,53,0,0,0, - 114,42,0,0,0,114,4,0,0,0,114,4,0,0,0,114, - 5,0,0,0,114,199,0,0,0,60,3,0,0,115,4,0, - 0,0,0,2,12,1,122,32,83,111,117,114,99,101,70,105, - 108,101,76,111,97,100,101,114,46,95,99,97,99,104,101,95, - 98,121,116,101,99,111,100,101,114,220,0,0,0,105,182,1, - 0,0,99,3,0,0,0,1,0,0,0,9,0,0,0,17, - 0,0,0,67,0,0,0,115,53,1,0,0,116,0,0,124, - 1,0,131,1,0,92,2,0,125,4,0,125,5,0,103,0, - 0,125,6,0,120,54,0,124,4,0,114,80,0,116,1,0, - 124,4,0,131,1,0,12,114,80,0,116,0,0,124,4,0, - 131,1,0,92,2,0,125,4,0,125,7,0,124,6,0,106, - 2,0,124,7,0,131,1,0,1,113,27,0,87,120,132,0, - 116,3,0,124,6,0,131,1,0,68,93,118,0,125,7,0, - 116,4,0,124,4,0,124,7,0,131,2,0,125,4,0,121, - 17,0,116,5,0,106,6,0,124,4,0,131,1,0,1,87, - 113,94,0,4,116,7,0,107,10,0,114,155,0,1,1,1, - 119,94,0,89,113,94,0,4,116,8,0,107,10,0,114,211, - 0,1,125,8,0,1,122,25,0,116,9,0,100,1,0,124, - 4,0,124,8,0,131,3,0,1,100,2,0,83,87,89,100, - 2,0,100,2,0,125,8,0,126,8,0,88,113,94,0,88, - 113,94,0,87,121,33,0,116,10,0,124,1,0,124,2,0, - 124,3,0,131,3,0,1,116,9,0,100,3,0,124,1,0, - 131,2,0,1,87,110,53,0,4,116,8,0,107,10,0,114, - 48,1,1,125,8,0,1,122,21,0,116,9,0,100,1,0, - 124,1,0,124,8,0,131,3,0,1,87,89,100,2,0,100, - 2,0,125,8,0,126,8,0,88,110,1,0,88,100,2,0, - 83,41,4,122,27,87,114,105,116,101,32,98,121,116,101,115, - 32,100,97,116,97,32,116,111,32,97,32,102,105,108,101,46, - 122,27,99,111,117,108,100,32,110,111,116,32,99,114,101,97, - 116,101,32,123,33,114,125,58,32,123,33,114,125,78,122,12, - 99,114,101,97,116,101,100,32,123,33,114,125,41,11,114,38, - 0,0,0,114,46,0,0,0,114,163,0,0,0,114,33,0, - 0,0,114,28,0,0,0,114,3,0,0,0,90,5,109,107, - 100,105,114,218,15,70,105,108,101,69,120,105,115,116,115,69, - 114,114,111,114,114,40,0,0,0,114,105,0,0,0,114,55, - 0,0,0,41,9,114,108,0,0,0,114,35,0,0,0,114, - 53,0,0,0,114,220,0,0,0,218,6,112,97,114,101,110, - 116,114,94,0,0,0,114,27,0,0,0,114,23,0,0,0, - 114,201,0,0,0,114,4,0,0,0,114,4,0,0,0,114, - 5,0,0,0,114,198,0,0,0,65,3,0,0,115,38,0, - 0,0,0,2,18,1,6,2,22,1,18,1,17,2,19,1, - 15,1,3,1,17,1,13,2,7,1,18,3,16,1,27,1, - 3,1,16,1,17,1,18,2,122,25,83,111,117,114,99,101, - 70,105,108,101,76,111,97,100,101,114,46,115,101,116,95,100, - 97,116,97,78,41,7,114,112,0,0,0,114,111,0,0,0, - 114,113,0,0,0,114,114,0,0,0,114,197,0,0,0,114, - 199,0,0,0,114,198,0,0,0,114,4,0,0,0,114,4, - 0,0,0,114,4,0,0,0,114,5,0,0,0,114,218,0, - 0,0,51,3,0,0,115,8,0,0,0,12,2,6,2,12, - 5,12,5,114,218,0,0,0,99,0,0,0,0,0,0,0, - 0,0,0,0,0,2,0,0,0,64,0,0,0,115,46,0, - 0,0,101,0,0,90,1,0,100,0,0,90,2,0,100,1, - 0,90,3,0,100,2,0,100,3,0,132,0,0,90,4,0, - 100,4,0,100,5,0,132,0,0,90,5,0,100,6,0,83, - 41,7,218,20,83,111,117,114,99,101,108,101,115,115,70,105, - 108,101,76,111,97,100,101,114,122,45,76,111,97,100,101,114, - 32,119,104,105,99,104,32,104,97,110,100,108,101,115,32,115, - 111,117,114,99,101,108,101,115,115,32,102,105,108,101,32,105, - 109,112,111,114,116,115,46,99,2,0,0,0,0,0,0,0, - 5,0,0,0,6,0,0,0,67,0,0,0,115,76,0,0, - 0,124,0,0,106,0,0,124,1,0,131,1,0,125,2,0, - 124,0,0,106,1,0,124,2,0,131,1,0,125,3,0,116, - 2,0,124,3,0,100,1,0,124,1,0,100,2,0,124,2, - 0,131,1,2,125,4,0,116,3,0,124,4,0,100,1,0, - 124,1,0,100,3,0,124,2,0,131,1,2,83,41,4,78, - 114,106,0,0,0,114,35,0,0,0,114,89,0,0,0,41, - 4,114,157,0,0,0,114,200,0,0,0,114,141,0,0,0, - 114,147,0,0,0,41,5,114,108,0,0,0,114,126,0,0, - 0,114,35,0,0,0,114,53,0,0,0,114,209,0,0,0, + 132,0,0,90,6,0,100,8,0,100,9,0,132,0,0,90, + 7,0,100,10,0,100,11,0,132,0,0,90,8,0,100,12, + 0,100,13,0,132,0,0,90,9,0,100,14,0,100,15,0, + 132,0,0,90,10,0,100,16,0,100,17,0,132,0,0,90, + 11,0,100,18,0,100,19,0,132,0,0,90,12,0,100,20, + 0,83,41,21,218,14,95,78,97,109,101,115,112,97,99,101, + 80,97,116,104,97,38,1,0,0,82,101,112,114,101,115,101, + 110,116,115,32,97,32,110,97,109,101,115,112,97,99,101,32, + 112,97,99,107,97,103,101,39,115,32,112,97,116,104,46,32, + 32,73,116,32,117,115,101,115,32,116,104,101,32,109,111,100, + 117,108,101,32,110,97,109,101,10,32,32,32,32,116,111,32, + 102,105,110,100,32,105,116,115,32,112,97,114,101,110,116,32, + 109,111,100,117,108,101,44,32,97,110,100,32,102,114,111,109, + 32,116,104,101,114,101,32,105,116,32,108,111,111,107,115,32, + 117,112,32,116,104,101,32,112,97,114,101,110,116,39,115,10, + 32,32,32,32,95,95,112,97,116,104,95,95,46,32,32,87, + 104,101,110,32,116,104,105,115,32,99,104,97,110,103,101,115, + 44,32,116,104,101,32,109,111,100,117,108,101,39,115,32,111, + 119,110,32,112,97,116,104,32,105,115,32,114,101,99,111,109, + 112,117,116,101,100,44,10,32,32,32,32,117,115,105,110,103, + 32,112,97,116,104,95,102,105,110,100,101,114,46,32,32,70, + 111,114,32,116,111,112,45,108,101,118,101,108,32,109,111,100, + 117,108,101,115,44,32,116,104,101,32,112,97,114,101,110,116, + 32,109,111,100,117,108,101,39,115,32,112,97,116,104,10,32, + 32,32,32,105,115,32,115,121,115,46,112,97,116,104,46,99, + 4,0,0,0,0,0,0,0,4,0,0,0,2,0,0,0, + 67,0,0,0,115,52,0,0,0,124,1,0,124,0,0,95, + 0,0,124,2,0,124,0,0,95,1,0,116,2,0,124,0, + 0,106,3,0,131,0,0,131,1,0,124,0,0,95,4,0, + 124,3,0,124,0,0,95,5,0,100,0,0,83,41,1,78, + 41,6,218,5,95,110,97,109,101,218,5,95,112,97,116,104, + 114,93,0,0,0,218,16,95,103,101,116,95,112,97,114,101, + 110,116,95,112,97,116,104,218,17,95,108,97,115,116,95,112, + 97,114,101,110,116,95,112,97,116,104,218,12,95,112,97,116, + 104,95,102,105,110,100,101,114,41,4,114,100,0,0,0,114, + 98,0,0,0,114,35,0,0,0,218,11,112,97,116,104,95, + 102,105,110,100,101,114,114,4,0,0,0,114,4,0,0,0, + 114,5,0,0,0,114,179,0,0,0,167,3,0,0,115,8, + 0,0,0,0,1,9,1,9,1,21,1,122,23,95,78,97, + 109,101,115,112,97,99,101,80,97,116,104,46,95,95,105,110, + 105,116,95,95,99,1,0,0,0,0,0,0,0,4,0,0, + 0,3,0,0,0,67,0,0,0,115,53,0,0,0,124,0, + 0,106,0,0,106,1,0,100,1,0,131,1,0,92,3,0, + 125,1,0,125,2,0,125,3,0,124,2,0,100,2,0,107, + 2,0,114,43,0,100,6,0,83,124,1,0,100,5,0,102, + 2,0,83,41,7,122,62,82,101,116,117,114,110,115,32,97, + 32,116,117,112,108,101,32,111,102,32,40,112,97,114,101,110, + 116,45,109,111,100,117,108,101,45,110,97,109,101,44,32,112, + 97,114,101,110,116,45,112,97,116,104,45,97,116,116,114,45, + 110,97,109,101,41,114,58,0,0,0,114,30,0,0,0,114, + 7,0,0,0,114,35,0,0,0,90,8,95,95,112,97,116, + 104,95,95,41,2,122,3,115,121,115,122,4,112,97,116,104, + 41,2,114,225,0,0,0,114,32,0,0,0,41,4,114,100, + 0,0,0,114,216,0,0,0,218,3,100,111,116,90,2,109, + 101,114,4,0,0,0,114,4,0,0,0,114,5,0,0,0, + 218,23,95,102,105,110,100,95,112,97,114,101,110,116,95,112, + 97,116,104,95,110,97,109,101,115,173,3,0,0,115,8,0, + 0,0,0,2,27,1,12,2,4,3,122,38,95,78,97,109, + 101,115,112,97,99,101,80,97,116,104,46,95,102,105,110,100, + 95,112,97,114,101,110,116,95,112,97,116,104,95,110,97,109, + 101,115,99,1,0,0,0,0,0,0,0,3,0,0,0,3, + 0,0,0,67,0,0,0,115,38,0,0,0,124,0,0,106, + 0,0,131,0,0,92,2,0,125,1,0,125,2,0,116,1, + 0,116,2,0,106,3,0,124,1,0,25,124,2,0,131,2, + 0,83,41,1,78,41,4,114,232,0,0,0,114,110,0,0, + 0,114,7,0,0,0,218,7,109,111,100,117,108,101,115,41, + 3,114,100,0,0,0,90,18,112,97,114,101,110,116,95,109, + 111,100,117,108,101,95,110,97,109,101,90,14,112,97,116,104, + 95,97,116,116,114,95,110,97,109,101,114,4,0,0,0,114, + 4,0,0,0,114,5,0,0,0,114,227,0,0,0,183,3, + 0,0,115,4,0,0,0,0,1,18,1,122,31,95,78,97, + 109,101,115,112,97,99,101,80,97,116,104,46,95,103,101,116, + 95,112,97,114,101,110,116,95,112,97,116,104,99,1,0,0, + 0,0,0,0,0,3,0,0,0,3,0,0,0,67,0,0, + 0,115,118,0,0,0,116,0,0,124,0,0,106,1,0,131, + 0,0,131,1,0,125,1,0,124,1,0,124,0,0,106,2, + 0,107,3,0,114,111,0,124,0,0,106,3,0,124,0,0, + 106,4,0,124,1,0,131,2,0,125,2,0,124,2,0,100, + 0,0,107,9,0,114,102,0,124,2,0,106,5,0,100,0, + 0,107,8,0,114,102,0,124,2,0,106,6,0,114,102,0, + 124,2,0,106,6,0,124,0,0,95,7,0,124,1,0,124, + 0,0,95,2,0,124,0,0,106,7,0,83,41,1,78,41, + 8,114,93,0,0,0,114,227,0,0,0,114,228,0,0,0, + 114,229,0,0,0,114,225,0,0,0,114,120,0,0,0,114, + 150,0,0,0,114,226,0,0,0,41,3,114,100,0,0,0, + 90,11,112,97,114,101,110,116,95,112,97,116,104,114,158,0, + 0,0,114,4,0,0,0,114,4,0,0,0,114,5,0,0, + 0,218,12,95,114,101,99,97,108,99,117,108,97,116,101,187, + 3,0,0,115,16,0,0,0,0,2,18,1,15,1,21,3, + 27,1,9,1,12,1,9,1,122,27,95,78,97,109,101,115, + 112,97,99,101,80,97,116,104,46,95,114,101,99,97,108,99, + 117,108,97,116,101,99,1,0,0,0,0,0,0,0,1,0, + 0,0,2,0,0,0,67,0,0,0,115,16,0,0,0,116, + 0,0,124,0,0,106,1,0,131,0,0,131,1,0,83,41, + 1,78,41,2,218,4,105,116,101,114,114,234,0,0,0,41, + 1,114,100,0,0,0,114,4,0,0,0,114,4,0,0,0, + 114,5,0,0,0,218,8,95,95,105,116,101,114,95,95,200, + 3,0,0,115,2,0,0,0,0,1,122,23,95,78,97,109, + 101,115,112,97,99,101,80,97,116,104,46,95,95,105,116,101, + 114,95,95,99,1,0,0,0,0,0,0,0,1,0,0,0, + 2,0,0,0,67,0,0,0,115,16,0,0,0,116,0,0, + 124,0,0,106,1,0,131,0,0,131,1,0,83,41,1,78, + 41,2,114,31,0,0,0,114,234,0,0,0,41,1,114,100, + 0,0,0,114,4,0,0,0,114,4,0,0,0,114,5,0, + 0,0,218,7,95,95,108,101,110,95,95,203,3,0,0,115, + 2,0,0,0,0,1,122,22,95,78,97,109,101,115,112,97, + 99,101,80,97,116,104,46,95,95,108,101,110,95,95,99,1, + 0,0,0,0,0,0,0,1,0,0,0,2,0,0,0,67, + 0,0,0,115,16,0,0,0,100,1,0,106,0,0,124,0, + 0,106,1,0,131,1,0,83,41,2,78,122,20,95,78,97, + 109,101,115,112,97,99,101,80,97,116,104,40,123,33,114,125, + 41,41,2,114,47,0,0,0,114,226,0,0,0,41,1,114, + 100,0,0,0,114,4,0,0,0,114,4,0,0,0,114,5, + 0,0,0,218,8,95,95,114,101,112,114,95,95,206,3,0, + 0,115,2,0,0,0,0,1,122,23,95,78,97,109,101,115, + 112,97,99,101,80,97,116,104,46,95,95,114,101,112,114,95, + 95,99,2,0,0,0,0,0,0,0,2,0,0,0,2,0, + 0,0,67,0,0,0,115,16,0,0,0,124,1,0,124,0, + 0,106,0,0,131,0,0,107,6,0,83,41,1,78,41,1, + 114,234,0,0,0,41,2,114,100,0,0,0,218,4,105,116, + 101,109,114,4,0,0,0,114,4,0,0,0,114,5,0,0, + 0,218,12,95,95,99,111,110,116,97,105,110,115,95,95,209, + 3,0,0,115,2,0,0,0,0,1,122,27,95,78,97,109, + 101,115,112,97,99,101,80,97,116,104,46,95,95,99,111,110, + 116,97,105,110,115,95,95,99,2,0,0,0,0,0,0,0, + 2,0,0,0,2,0,0,0,67,0,0,0,115,20,0,0, + 0,124,0,0,106,0,0,106,1,0,124,1,0,131,1,0, + 1,100,0,0,83,41,1,78,41,2,114,226,0,0,0,114, + 157,0,0,0,41,2,114,100,0,0,0,114,239,0,0,0, 114,4,0,0,0,114,4,0,0,0,114,5,0,0,0,114, - 187,0,0,0,98,3,0,0,115,8,0,0,0,0,1,15, - 1,15,1,24,1,122,29,83,111,117,114,99,101,108,101,115, - 115,70,105,108,101,76,111,97,100,101,114,46,103,101,116,95, - 99,111,100,101,99,2,0,0,0,0,0,0,0,2,0,0, + 157,0,0,0,212,3,0,0,115,2,0,0,0,0,1,122, + 21,95,78,97,109,101,115,112,97,99,101,80,97,116,104,46, + 97,112,112,101,110,100,78,41,13,114,105,0,0,0,114,104, + 0,0,0,114,106,0,0,0,114,107,0,0,0,114,179,0, + 0,0,114,232,0,0,0,114,227,0,0,0,114,234,0,0, + 0,114,236,0,0,0,114,237,0,0,0,114,238,0,0,0, + 114,240,0,0,0,114,157,0,0,0,114,4,0,0,0,114, + 4,0,0,0,114,4,0,0,0,114,5,0,0,0,114,224, + 0,0,0,160,3,0,0,115,20,0,0,0,12,5,6,2, + 12,6,12,10,12,4,12,13,12,3,12,3,12,3,12,3, + 114,224,0,0,0,99,0,0,0,0,0,0,0,0,0,0, + 0,0,3,0,0,0,64,0,0,0,115,118,0,0,0,101, + 0,0,90,1,0,100,0,0,90,2,0,100,1,0,100,2, + 0,132,0,0,90,3,0,101,4,0,100,3,0,100,4,0, + 132,0,0,131,1,0,90,5,0,100,5,0,100,6,0,132, + 0,0,90,6,0,100,7,0,100,8,0,132,0,0,90,7, + 0,100,9,0,100,10,0,132,0,0,90,8,0,100,11,0, + 100,12,0,132,0,0,90,9,0,100,13,0,100,14,0,132, + 0,0,90,10,0,100,15,0,100,16,0,132,0,0,90,11, + 0,100,17,0,83,41,18,218,16,95,78,97,109,101,115,112, + 97,99,101,76,111,97,100,101,114,99,4,0,0,0,0,0, + 0,0,4,0,0,0,4,0,0,0,67,0,0,0,115,25, + 0,0,0,116,0,0,124,1,0,124,2,0,124,3,0,131, + 3,0,124,0,0,95,1,0,100,0,0,83,41,1,78,41, + 2,114,224,0,0,0,114,226,0,0,0,41,4,114,100,0, + 0,0,114,98,0,0,0,114,35,0,0,0,114,230,0,0, + 0,114,4,0,0,0,114,4,0,0,0,114,5,0,0,0, + 114,179,0,0,0,218,3,0,0,115,2,0,0,0,0,1, + 122,25,95,78,97,109,101,115,112,97,99,101,76,111,97,100, + 101,114,46,95,95,105,110,105,116,95,95,99,2,0,0,0, + 0,0,0,0,2,0,0,0,2,0,0,0,67,0,0,0, + 115,16,0,0,0,100,1,0,106,0,0,124,1,0,106,1, + 0,131,1,0,83,41,2,122,115,82,101,116,117,114,110,32, + 114,101,112,114,32,102,111,114,32,116,104,101,32,109,111,100, + 117,108,101,46,10,10,32,32,32,32,32,32,32,32,84,104, + 101,32,109,101,116,104,111,100,32,105,115,32,100,101,112,114, + 101,99,97,116,101,100,46,32,32,84,104,101,32,105,109,112, + 111,114,116,32,109,97,99,104,105,110,101,114,121,32,100,111, + 101,115,32,116,104,101,32,106,111,98,32,105,116,115,101,108, + 102,46,10,10,32,32,32,32,32,32,32,32,122,25,60,109, + 111,100,117,108,101,32,123,33,114,125,32,40,110,97,109,101, + 115,112,97,99,101,41,62,41,2,114,47,0,0,0,114,105, + 0,0,0,41,2,114,164,0,0,0,114,184,0,0,0,114, + 4,0,0,0,114,4,0,0,0,114,5,0,0,0,218,11, + 109,111,100,117,108,101,95,114,101,112,114,221,3,0,0,115, + 2,0,0,0,0,7,122,28,95,78,97,109,101,115,112,97, + 99,101,76,111,97,100,101,114,46,109,111,100,117,108,101,95, + 114,101,112,114,99,2,0,0,0,0,0,0,0,2,0,0, 0,1,0,0,0,67,0,0,0,115,4,0,0,0,100,1, - 0,83,41,2,122,39,82,101,116,117,114,110,32,78,111,110, - 101,32,97,115,32,116,104,101,114,101,32,105,115,32,110,111, - 32,115,111,117,114,99,101,32,99,111,100,101,46,78,114,4, - 0,0,0,41,2,114,108,0,0,0,114,126,0,0,0,114, - 4,0,0,0,114,4,0,0,0,114,5,0,0,0,114,202, - 0,0,0,104,3,0,0,115,2,0,0,0,0,2,122,31, - 83,111,117,114,99,101,108,101,115,115,70,105,108,101,76,111, - 97,100,101,114,46,103,101,116,95,115,111,117,114,99,101,78, - 41,6,114,112,0,0,0,114,111,0,0,0,114,113,0,0, - 0,114,114,0,0,0,114,187,0,0,0,114,202,0,0,0, - 114,4,0,0,0,114,4,0,0,0,114,4,0,0,0,114, - 5,0,0,0,114,223,0,0,0,94,3,0,0,115,6,0, - 0,0,12,2,6,2,12,6,114,223,0,0,0,99,0,0, - 0,0,0,0,0,0,0,0,0,0,3,0,0,0,64,0, - 0,0,115,136,0,0,0,101,0,0,90,1,0,100,0,0, - 90,2,0,100,1,0,90,3,0,100,2,0,100,3,0,132, - 0,0,90,4,0,100,4,0,100,5,0,132,0,0,90,5, - 0,100,6,0,100,7,0,132,0,0,90,6,0,100,8,0, - 100,9,0,132,0,0,90,7,0,100,10,0,100,11,0,132, - 0,0,90,8,0,100,12,0,100,13,0,132,0,0,90,9, - 0,100,14,0,100,15,0,132,0,0,90,10,0,100,16,0, - 100,17,0,132,0,0,90,11,0,101,12,0,100,18,0,100, - 19,0,132,0,0,131,1,0,90,13,0,100,20,0,83,41, - 21,218,19,69,120,116,101,110,115,105,111,110,70,105,108,101, - 76,111,97,100,101,114,122,93,76,111,97,100,101,114,32,102, - 111,114,32,101,120,116,101,110,115,105,111,110,32,109,111,100, - 117,108,101,115,46,10,10,32,32,32,32,84,104,101,32,99, - 111,110,115,116,114,117,99,116,111,114,32,105,115,32,100,101, - 115,105,103,110,101,100,32,116,111,32,119,111,114,107,32,119, - 105,116,104,32,70,105,108,101,70,105,110,100,101,114,46,10, - 10,32,32,32,32,99,3,0,0,0,0,0,0,0,3,0, - 0,0,2,0,0,0,67,0,0,0,115,22,0,0,0,124, - 1,0,124,0,0,95,0,0,124,2,0,124,0,0,95,1, - 0,100,0,0,83,41,1,78,41,2,114,106,0,0,0,114, - 35,0,0,0,41,3,114,108,0,0,0,114,106,0,0,0, - 114,35,0,0,0,114,4,0,0,0,114,4,0,0,0,114, - 5,0,0,0,114,185,0,0,0,121,3,0,0,115,4,0, - 0,0,0,1,9,1,122,28,69,120,116,101,110,115,105,111, - 110,70,105,108,101,76,111,97,100,101,114,46,95,95,105,110, - 105,116,95,95,99,2,0,0,0,0,0,0,0,2,0,0, - 0,2,0,0,0,67,0,0,0,115,34,0,0,0,124,0, - 0,106,0,0,124,1,0,106,0,0,107,2,0,111,33,0, - 124,0,0,106,1,0,124,1,0,106,1,0,107,2,0,83, - 41,1,78,41,2,114,211,0,0,0,114,118,0,0,0,41, - 2,114,108,0,0,0,114,212,0,0,0,114,4,0,0,0, - 114,4,0,0,0,114,5,0,0,0,114,213,0,0,0,125, - 3,0,0,115,4,0,0,0,0,1,18,1,122,26,69,120, - 116,101,110,115,105,111,110,70,105,108,101,76,111,97,100,101, - 114,46,95,95,101,113,95,95,99,1,0,0,0,0,0,0, - 0,1,0,0,0,3,0,0,0,67,0,0,0,115,26,0, - 0,0,116,0,0,124,0,0,106,1,0,131,1,0,116,0, - 0,124,0,0,106,2,0,131,1,0,65,83,41,1,78,41, - 3,114,214,0,0,0,114,106,0,0,0,114,35,0,0,0, - 41,1,114,108,0,0,0,114,4,0,0,0,114,4,0,0, - 0,114,5,0,0,0,114,215,0,0,0,129,3,0,0,115, - 2,0,0,0,0,1,122,28,69,120,116,101,110,115,105,111, - 110,70,105,108,101,76,111,97,100,101,114,46,95,95,104,97, - 115,104,95,95,99,2,0,0,0,0,0,0,0,3,0,0, - 0,4,0,0,0,67,0,0,0,115,47,0,0,0,116,0, - 0,106,1,0,116,2,0,106,3,0,124,1,0,131,2,0, - 125,2,0,116,4,0,100,1,0,124,1,0,106,5,0,124, - 0,0,106,6,0,131,3,0,1,124,2,0,83,41,2,122, - 38,67,114,101,97,116,101,32,97,110,32,117,110,105,116,105, - 97,108,105,122,101,100,32,101,120,116,101,110,115,105,111,110, - 32,109,111,100,117,108,101,122,38,101,120,116,101,110,115,105, - 111,110,32,109,111,100,117,108,101,32,123,33,114,125,32,108, - 111,97,100,101,100,32,102,114,111,109,32,123,33,114,125,41, - 7,114,121,0,0,0,114,188,0,0,0,114,145,0,0,0, - 90,14,99,114,101,97,116,101,95,100,121,110,97,109,105,99, - 114,105,0,0,0,114,106,0,0,0,114,35,0,0,0,41, - 3,114,108,0,0,0,114,164,0,0,0,114,190,0,0,0, + 0,83,41,2,78,84,114,4,0,0,0,41,2,114,100,0, + 0,0,114,119,0,0,0,114,4,0,0,0,114,4,0,0, + 0,114,5,0,0,0,114,153,0,0,0,230,3,0,0,115, + 2,0,0,0,0,1,122,27,95,78,97,109,101,115,112,97, + 99,101,76,111,97,100,101,114,46,105,115,95,112,97,99,107, + 97,103,101,99,2,0,0,0,0,0,0,0,2,0,0,0, + 1,0,0,0,67,0,0,0,115,4,0,0,0,100,1,0, + 83,41,2,78,114,30,0,0,0,114,4,0,0,0,41,2, + 114,100,0,0,0,114,119,0,0,0,114,4,0,0,0,114, + 4,0,0,0,114,5,0,0,0,114,196,0,0,0,233,3, + 0,0,115,2,0,0,0,0,1,122,27,95,78,97,109,101, + 115,112,97,99,101,76,111,97,100,101,114,46,103,101,116,95, + 115,111,117,114,99,101,99,2,0,0,0,0,0,0,0,2, + 0,0,0,6,0,0,0,67,0,0,0,115,22,0,0,0, + 116,0,0,100,1,0,100,2,0,100,3,0,100,4,0,100, + 5,0,131,3,1,83,41,6,78,114,30,0,0,0,122,8, + 60,115,116,114,105,110,103,62,114,183,0,0,0,114,198,0, + 0,0,84,41,1,114,199,0,0,0,41,2,114,100,0,0, + 0,114,119,0,0,0,114,4,0,0,0,114,4,0,0,0, + 114,5,0,0,0,114,181,0,0,0,236,3,0,0,115,2, + 0,0,0,0,1,122,25,95,78,97,109,101,115,112,97,99, + 101,76,111,97,100,101,114,46,103,101,116,95,99,111,100,101, + 99,2,0,0,0,0,0,0,0,2,0,0,0,1,0,0, + 0,67,0,0,0,115,4,0,0,0,100,1,0,83,41,2, + 122,42,85,115,101,32,100,101,102,97,117,108,116,32,115,101, + 109,97,110,116,105,99,115,32,102,111,114,32,109,111,100,117, + 108,101,32,99,114,101,97,116,105,111,110,46,78,114,4,0, + 0,0,41,2,114,100,0,0,0,114,158,0,0,0,114,4, + 0,0,0,114,4,0,0,0,114,5,0,0,0,114,180,0, + 0,0,239,3,0,0,115,0,0,0,0,122,30,95,78,97, + 109,101,115,112,97,99,101,76,111,97,100,101,114,46,99,114, + 101,97,116,101,95,109,111,100,117,108,101,99,2,0,0,0, + 0,0,0,0,2,0,0,0,1,0,0,0,67,0,0,0, + 115,4,0,0,0,100,0,0,83,41,1,78,114,4,0,0, + 0,41,2,114,100,0,0,0,114,184,0,0,0,114,4,0, + 0,0,114,4,0,0,0,114,5,0,0,0,114,185,0,0, + 0,242,3,0,0,115,2,0,0,0,0,1,122,28,95,78, + 97,109,101,115,112,97,99,101,76,111,97,100,101,114,46,101, + 120,101,99,95,109,111,100,117,108,101,99,2,0,0,0,0, + 0,0,0,2,0,0,0,3,0,0,0,67,0,0,0,115, + 35,0,0,0,116,0,0,106,1,0,100,1,0,124,0,0, + 106,2,0,131,2,0,1,116,0,0,106,3,0,124,0,0, + 124,1,0,131,2,0,83,41,2,122,98,76,111,97,100,32, + 97,32,110,97,109,101,115,112,97,99,101,32,109,111,100,117, + 108,101,46,10,10,32,32,32,32,32,32,32,32,84,104,105, + 115,32,109,101,116,104,111,100,32,105,115,32,100,101,112,114, + 101,99,97,116,101,100,46,32,32,85,115,101,32,101,120,101, + 99,95,109,111,100,117,108,101,40,41,32,105,110,115,116,101, + 97,100,46,10,10,32,32,32,32,32,32,32,32,122,38,110, + 97,109,101,115,112,97,99,101,32,109,111,100,117,108,101,32, + 108,111,97,100,101,100,32,119,105,116,104,32,112,97,116,104, + 32,123,33,114,125,41,4,114,114,0,0,0,114,129,0,0, + 0,114,226,0,0,0,114,186,0,0,0,41,2,114,100,0, + 0,0,114,119,0,0,0,114,4,0,0,0,114,4,0,0, + 0,114,5,0,0,0,114,187,0,0,0,245,3,0,0,115, + 6,0,0,0,0,7,9,1,10,1,122,28,95,78,97,109, + 101,115,112,97,99,101,76,111,97,100,101,114,46,108,111,97, + 100,95,109,111,100,117,108,101,78,41,12,114,105,0,0,0, + 114,104,0,0,0,114,106,0,0,0,114,179,0,0,0,114, + 177,0,0,0,114,242,0,0,0,114,153,0,0,0,114,196, + 0,0,0,114,181,0,0,0,114,180,0,0,0,114,185,0, + 0,0,114,187,0,0,0,114,4,0,0,0,114,4,0,0, + 0,114,4,0,0,0,114,5,0,0,0,114,241,0,0,0, + 217,3,0,0,115,16,0,0,0,12,1,12,3,18,9,12, + 3,12,3,12,3,12,3,12,3,114,241,0,0,0,99,0, + 0,0,0,0,0,0,0,0,0,0,0,5,0,0,0,64, + 0,0,0,115,160,0,0,0,101,0,0,90,1,0,100,0, + 0,90,2,0,100,1,0,90,3,0,101,4,0,100,2,0, + 100,3,0,132,0,0,131,1,0,90,5,0,101,4,0,100, + 4,0,100,5,0,132,0,0,131,1,0,90,6,0,101,4, + 0,100,6,0,100,7,0,132,0,0,131,1,0,90,7,0, + 101,4,0,100,8,0,100,9,0,132,0,0,131,1,0,90, + 8,0,101,4,0,100,10,0,100,11,0,100,12,0,132,1, + 0,131,1,0,90,9,0,101,4,0,100,10,0,100,10,0, + 100,13,0,100,14,0,132,2,0,131,1,0,90,10,0,101, + 4,0,100,10,0,100,15,0,100,16,0,132,1,0,131,1, + 0,90,11,0,100,10,0,83,41,17,218,10,80,97,116,104, + 70,105,110,100,101,114,122,62,77,101,116,97,32,112,97,116, + 104,32,102,105,110,100,101,114,32,102,111,114,32,115,121,115, + 46,112,97,116,104,32,97,110,100,32,112,97,99,107,97,103, + 101,32,95,95,112,97,116,104,95,95,32,97,116,116,114,105, + 98,117,116,101,115,46,99,1,0,0,0,0,0,0,0,2, + 0,0,0,4,0,0,0,67,0,0,0,115,55,0,0,0, + 120,48,0,116,0,0,106,1,0,106,2,0,131,0,0,68, + 93,31,0,125,1,0,116,3,0,124,1,0,100,1,0,131, + 2,0,114,16,0,124,1,0,106,4,0,131,0,0,1,113, + 16,0,87,100,2,0,83,41,3,122,125,67,97,108,108,32, + 116,104,101,32,105,110,118,97,108,105,100,97,116,101,95,99, + 97,99,104,101,115,40,41,32,109,101,116,104,111,100,32,111, + 110,32,97,108,108,32,112,97,116,104,32,101,110,116,114,121, + 32,102,105,110,100,101,114,115,10,32,32,32,32,32,32,32, + 32,115,116,111,114,101,100,32,105,110,32,115,121,115,46,112, + 97,116,104,95,105,109,112,111,114,116,101,114,95,99,97,99, + 104,101,115,32,40,119,104,101,114,101,32,105,109,112,108,101, + 109,101,110,116,101,100,41,46,218,17,105,110,118,97,108,105, + 100,97,116,101,95,99,97,99,104,101,115,78,41,5,114,7, + 0,0,0,218,19,112,97,116,104,95,105,109,112,111,114,116, + 101,114,95,99,97,99,104,101,218,6,118,97,108,117,101,115, + 114,108,0,0,0,114,244,0,0,0,41,2,114,164,0,0, + 0,218,6,102,105,110,100,101,114,114,4,0,0,0,114,4, + 0,0,0,114,5,0,0,0,114,244,0,0,0,7,4,0, + 0,115,6,0,0,0,0,4,22,1,15,1,122,28,80,97, + 116,104,70,105,110,100,101,114,46,105,110,118,97,108,105,100, + 97,116,101,95,99,97,99,104,101,115,99,2,0,0,0,0, + 0,0,0,3,0,0,0,12,0,0,0,67,0,0,0,115, + 107,0,0,0,116,0,0,106,1,0,100,1,0,107,9,0, + 114,41,0,116,0,0,106,1,0,12,114,41,0,116,2,0, + 106,3,0,100,2,0,116,4,0,131,2,0,1,120,59,0, + 116,0,0,106,1,0,68,93,44,0,125,2,0,121,14,0, + 124,2,0,124,1,0,131,1,0,83,87,113,51,0,4,116, + 5,0,107,10,0,114,94,0,1,1,1,119,51,0,89,113, + 51,0,88,113,51,0,87,100,1,0,83,100,1,0,83,41, + 3,122,113,83,101,97,114,99,104,32,115,101,113,117,101,110, + 99,101,32,111,102,32,104,111,111,107,115,32,102,111,114,32, + 97,32,102,105,110,100,101,114,32,102,111,114,32,39,112,97, + 116,104,39,46,10,10,32,32,32,32,32,32,32,32,73,102, + 32,39,104,111,111,107,115,39,32,105,115,32,102,97,108,115, + 101,32,116,104,101,110,32,117,115,101,32,115,121,115,46,112, + 97,116,104,95,104,111,111,107,115,46,10,10,32,32,32,32, + 32,32,32,32,78,122,23,115,121,115,46,112,97,116,104,95, + 104,111,111,107,115,32,105,115,32,101,109,112,116,121,41,6, + 114,7,0,0,0,218,10,112,97,116,104,95,104,111,111,107, + 115,114,60,0,0,0,114,61,0,0,0,114,118,0,0,0, + 114,99,0,0,0,41,3,114,164,0,0,0,114,35,0,0, + 0,90,4,104,111,111,107,114,4,0,0,0,114,4,0,0, + 0,114,5,0,0,0,218,11,95,112,97,116,104,95,104,111, + 111,107,115,15,4,0,0,115,16,0,0,0,0,7,25,1, + 16,1,16,1,3,1,14,1,13,1,12,2,122,22,80,97, + 116,104,70,105,110,100,101,114,46,95,112,97,116,104,95,104, + 111,111,107,115,99,2,0,0,0,0,0,0,0,3,0,0, + 0,19,0,0,0,67,0,0,0,115,123,0,0,0,124,1, + 0,100,1,0,107,2,0,114,53,0,121,16,0,116,0,0, + 106,1,0,131,0,0,125,1,0,87,110,22,0,4,116,2, + 0,107,10,0,114,52,0,1,1,1,100,2,0,83,89,110, + 1,0,88,121,17,0,116,3,0,106,4,0,124,1,0,25, + 125,2,0,87,110,46,0,4,116,5,0,107,10,0,114,118, + 0,1,1,1,124,0,0,106,6,0,124,1,0,131,1,0, + 125,2,0,124,2,0,116,3,0,106,4,0,124,1,0,60, + 89,110,1,0,88,124,2,0,83,41,3,122,210,71,101,116, + 32,116,104,101,32,102,105,110,100,101,114,32,102,111,114,32, + 116,104,101,32,112,97,116,104,32,101,110,116,114,121,32,102, + 114,111,109,32,115,121,115,46,112,97,116,104,95,105,109,112, + 111,114,116,101,114,95,99,97,99,104,101,46,10,10,32,32, + 32,32,32,32,32,32,73,102,32,116,104,101,32,112,97,116, + 104,32,101,110,116,114,121,32,105,115,32,110,111,116,32,105, + 110,32,116,104,101,32,99,97,99,104,101,44,32,102,105,110, + 100,32,116,104,101,32,97,112,112,114,111,112,114,105,97,116, + 101,32,102,105,110,100,101,114,10,32,32,32,32,32,32,32, + 32,97,110,100,32,99,97,99,104,101,32,105,116,46,32,73, + 102,32,110,111,32,102,105,110,100,101,114,32,105,115,32,97, + 118,97,105,108,97,98,108,101,44,32,115,116,111,114,101,32, + 78,111,110,101,46,10,10,32,32,32,32,32,32,32,32,114, + 30,0,0,0,78,41,7,114,3,0,0,0,114,45,0,0, + 0,218,17,70,105,108,101,78,111,116,70,111,117,110,100,69, + 114,114,111,114,114,7,0,0,0,114,245,0,0,0,114,131, + 0,0,0,114,249,0,0,0,41,3,114,164,0,0,0,114, + 35,0,0,0,114,247,0,0,0,114,4,0,0,0,114,4, + 0,0,0,114,5,0,0,0,218,20,95,112,97,116,104,95, + 105,109,112,111,114,116,101,114,95,99,97,99,104,101,32,4, + 0,0,115,22,0,0,0,0,8,12,1,3,1,16,1,13, + 3,9,1,3,1,17,1,13,1,15,1,18,1,122,31,80, + 97,116,104,70,105,110,100,101,114,46,95,112,97,116,104,95, + 105,109,112,111,114,116,101,114,95,99,97,99,104,101,99,3, + 0,0,0,0,0,0,0,6,0,0,0,3,0,0,0,67, + 0,0,0,115,119,0,0,0,116,0,0,124,2,0,100,1, + 0,131,2,0,114,39,0,124,2,0,106,1,0,124,1,0, + 131,1,0,92,2,0,125,3,0,125,4,0,110,21,0,124, + 2,0,106,2,0,124,1,0,131,1,0,125,3,0,103,0, + 0,125,4,0,124,3,0,100,0,0,107,9,0,114,88,0, + 116,3,0,106,4,0,124,1,0,124,3,0,131,2,0,83, + 116,3,0,106,5,0,124,1,0,100,0,0,131,2,0,125, + 5,0,124,4,0,124,5,0,95,6,0,124,5,0,83,41, + 2,78,114,117,0,0,0,41,7,114,108,0,0,0,114,117, + 0,0,0,114,176,0,0,0,114,114,0,0,0,114,173,0, + 0,0,114,154,0,0,0,114,150,0,0,0,41,6,114,164, + 0,0,0,114,119,0,0,0,114,247,0,0,0,114,120,0, + 0,0,114,121,0,0,0,114,158,0,0,0,114,4,0,0, + 0,114,4,0,0,0,114,5,0,0,0,218,16,95,108,101, + 103,97,99,121,95,103,101,116,95,115,112,101,99,54,4,0, + 0,115,18,0,0,0,0,4,15,1,24,2,15,1,6,1, + 12,1,16,1,18,1,9,1,122,27,80,97,116,104,70,105, + 110,100,101,114,46,95,108,101,103,97,99,121,95,103,101,116, + 95,115,112,101,99,78,99,4,0,0,0,0,0,0,0,9, + 0,0,0,5,0,0,0,67,0,0,0,115,243,0,0,0, + 103,0,0,125,4,0,120,230,0,124,2,0,68,93,191,0, + 125,5,0,116,0,0,124,5,0,116,1,0,116,2,0,102, + 2,0,131,2,0,115,43,0,113,13,0,124,0,0,106,3, + 0,124,5,0,131,1,0,125,6,0,124,6,0,100,1,0, + 107,9,0,114,13,0,116,4,0,124,6,0,100,2,0,131, + 2,0,114,106,0,124,6,0,106,5,0,124,1,0,124,3, + 0,131,2,0,125,7,0,110,18,0,124,0,0,106,6,0, + 124,1,0,124,6,0,131,2,0,125,7,0,124,7,0,100, + 1,0,107,8,0,114,139,0,113,13,0,124,7,0,106,7, + 0,100,1,0,107,9,0,114,158,0,124,7,0,83,124,7, + 0,106,8,0,125,8,0,124,8,0,100,1,0,107,8,0, + 114,191,0,116,9,0,100,3,0,131,1,0,130,1,0,124, + 4,0,106,10,0,124,8,0,131,1,0,1,113,13,0,87, + 116,11,0,106,12,0,124,1,0,100,1,0,131,2,0,125, + 7,0,124,4,0,124,7,0,95,8,0,124,7,0,83,100, + 1,0,83,41,4,122,63,70,105,110,100,32,116,104,101,32, + 108,111,97,100,101,114,32,111,114,32,110,97,109,101,115,112, + 97,99,101,95,112,97,116,104,32,102,111,114,32,116,104,105, + 115,32,109,111,100,117,108,101,47,112,97,99,107,97,103,101, + 32,110,97,109,101,46,78,114,175,0,0,0,122,19,115,112, + 101,99,32,109,105,115,115,105,110,103,32,108,111,97,100,101, + 114,41,13,114,137,0,0,0,114,69,0,0,0,218,5,98, + 121,116,101,115,114,251,0,0,0,114,108,0,0,0,114,175, + 0,0,0,114,252,0,0,0,114,120,0,0,0,114,150,0, + 0,0,114,99,0,0,0,114,143,0,0,0,114,114,0,0, + 0,114,154,0,0,0,41,9,114,164,0,0,0,114,119,0, + 0,0,114,35,0,0,0,114,174,0,0,0,218,14,110,97, + 109,101,115,112,97,99,101,95,112,97,116,104,90,5,101,110, + 116,114,121,114,247,0,0,0,114,158,0,0,0,114,121,0, + 0,0,114,4,0,0,0,114,4,0,0,0,114,5,0,0, + 0,218,9,95,103,101,116,95,115,112,101,99,69,4,0,0, + 115,40,0,0,0,0,5,6,1,13,1,21,1,3,1,15, + 1,12,1,15,1,21,2,18,1,12,1,3,1,15,1,4, + 1,9,1,12,1,12,5,17,2,18,1,9,1,122,20,80, + 97,116,104,70,105,110,100,101,114,46,95,103,101,116,95,115, + 112,101,99,99,4,0,0,0,0,0,0,0,6,0,0,0, + 4,0,0,0,67,0,0,0,115,140,0,0,0,124,2,0, + 100,1,0,107,8,0,114,21,0,116,0,0,106,1,0,125, + 2,0,124,0,0,106,2,0,124,1,0,124,2,0,124,3, + 0,131,3,0,125,4,0,124,4,0,100,1,0,107,8,0, + 114,58,0,100,1,0,83,124,4,0,106,3,0,100,1,0, + 107,8,0,114,132,0,124,4,0,106,4,0,125,5,0,124, + 5,0,114,125,0,100,2,0,124,4,0,95,5,0,116,6, + 0,124,1,0,124,5,0,124,0,0,106,2,0,131,3,0, + 124,4,0,95,4,0,124,4,0,83,100,1,0,83,110,4, + 0,124,4,0,83,100,1,0,83,41,3,122,98,102,105,110, + 100,32,116,104,101,32,109,111,100,117,108,101,32,111,110,32, + 115,121,115,46,112,97,116,104,32,111,114,32,39,112,97,116, + 104,39,32,98,97,115,101,100,32,111,110,32,115,121,115,46, + 112,97,116,104,95,104,111,111,107,115,32,97,110,100,10,32, + 32,32,32,32,32,32,32,115,121,115,46,112,97,116,104,95, + 105,109,112,111,114,116,101,114,95,99,97,99,104,101,46,78, + 90,9,110,97,109,101,115,112,97,99,101,41,7,114,7,0, + 0,0,114,35,0,0,0,114,255,0,0,0,114,120,0,0, + 0,114,150,0,0,0,114,152,0,0,0,114,224,0,0,0, + 41,6,114,164,0,0,0,114,119,0,0,0,114,35,0,0, + 0,114,174,0,0,0,114,158,0,0,0,114,254,0,0,0, 114,4,0,0,0,114,4,0,0,0,114,5,0,0,0,114, - 186,0,0,0,132,3,0,0,115,10,0,0,0,0,2,6, - 1,15,1,6,1,16,1,122,33,69,120,116,101,110,115,105, - 111,110,70,105,108,101,76,111,97,100,101,114,46,99,114,101, - 97,116,101,95,109,111,100,117,108,101,99,2,0,0,0,0, - 0,0,0,2,0,0,0,4,0,0,0,67,0,0,0,115, - 45,0,0,0,116,0,0,106,1,0,116,2,0,106,3,0, - 124,1,0,131,2,0,1,116,4,0,100,1,0,124,0,0, - 106,5,0,124,0,0,106,6,0,131,3,0,1,100,2,0, - 83,41,3,122,30,73,110,105,116,105,97,108,105,122,101,32, - 97,110,32,101,120,116,101,110,115,105,111,110,32,109,111,100, - 117,108,101,122,40,101,120,116,101,110,115,105,111,110,32,109, - 111,100,117,108,101,32,123,33,114,125,32,101,120,101,99,117, - 116,101,100,32,102,114,111,109,32,123,33,114,125,78,41,7, - 114,121,0,0,0,114,188,0,0,0,114,145,0,0,0,90, - 12,101,120,101,99,95,100,121,110,97,109,105,99,114,105,0, - 0,0,114,106,0,0,0,114,35,0,0,0,41,2,114,108, - 0,0,0,114,190,0,0,0,114,4,0,0,0,114,4,0, - 0,0,114,5,0,0,0,114,191,0,0,0,140,3,0,0, - 115,6,0,0,0,0,2,19,1,6,1,122,31,69,120,116, - 101,110,115,105,111,110,70,105,108,101,76,111,97,100,101,114, - 46,101,120,101,99,95,109,111,100,117,108,101,99,2,0,0, - 0,0,0,0,0,2,0,0,0,4,0,0,0,3,0,0, - 0,115,48,0,0,0,116,0,0,124,0,0,106,1,0,131, - 1,0,100,1,0,25,137,0,0,116,2,0,135,0,0,102, - 1,0,100,2,0,100,3,0,134,0,0,116,3,0,68,131, - 1,0,131,1,0,83,41,4,122,49,82,101,116,117,114,110, - 32,84,114,117,101,32,105,102,32,116,104,101,32,101,120,116, - 101,110,115,105,111,110,32,109,111,100,117,108,101,32,105,115, - 32,97,32,112,97,99,107,97,103,101,46,114,29,0,0,0, - 99,1,0,0,0,0,0,0,0,2,0,0,0,4,0,0, - 0,51,0,0,0,115,31,0,0,0,124,0,0,93,21,0, - 125,1,0,136,0,0,100,0,0,124,1,0,23,107,2,0, - 86,1,113,3,0,100,1,0,83,41,2,114,185,0,0,0, - 78,114,4,0,0,0,41,2,114,22,0,0,0,218,6,115, - 117,102,102,105,120,41,1,218,9,102,105,108,101,95,110,97, - 109,101,114,4,0,0,0,114,5,0,0,0,250,9,60,103, - 101,110,101,120,112,114,62,149,3,0,0,115,2,0,0,0, - 6,1,122,49,69,120,116,101,110,115,105,111,110,70,105,108, - 101,76,111,97,100,101,114,46,105,115,95,112,97,99,107,97, - 103,101,46,60,108,111,99,97,108,115,62,46,60,103,101,110, - 101,120,112,114,62,41,4,114,38,0,0,0,114,35,0,0, - 0,218,3,97,110,121,218,18,69,88,84,69,78,83,73,79, - 78,95,83,85,70,70,73,88,69,83,41,2,114,108,0,0, - 0,114,126,0,0,0,114,4,0,0,0,41,1,114,226,0, - 0,0,114,5,0,0,0,114,159,0,0,0,146,3,0,0, - 115,6,0,0,0,0,2,19,1,18,1,122,30,69,120,116, - 101,110,115,105,111,110,70,105,108,101,76,111,97,100,101,114, - 46,105,115,95,112,97,99,107,97,103,101,99,2,0,0,0, - 0,0,0,0,2,0,0,0,1,0,0,0,67,0,0,0, - 115,4,0,0,0,100,1,0,83,41,2,122,63,82,101,116, - 117,114,110,32,78,111,110,101,32,97,115,32,97,110,32,101, - 120,116,101,110,115,105,111,110,32,109,111,100,117,108,101,32, - 99,97,110,110,111,116,32,99,114,101,97,116,101,32,97,32, - 99,111,100,101,32,111,98,106,101,99,116,46,78,114,4,0, - 0,0,41,2,114,108,0,0,0,114,126,0,0,0,114,4, - 0,0,0,114,4,0,0,0,114,5,0,0,0,114,187,0, - 0,0,152,3,0,0,115,2,0,0,0,0,2,122,28,69, - 120,116,101,110,115,105,111,110,70,105,108,101,76,111,97,100, - 101,114,46,103,101,116,95,99,111,100,101,99,2,0,0,0, - 0,0,0,0,2,0,0,0,1,0,0,0,67,0,0,0, - 115,4,0,0,0,100,1,0,83,41,2,122,53,82,101,116, - 117,114,110,32,78,111,110,101,32,97,115,32,101,120,116,101, - 110,115,105,111,110,32,109,111,100,117,108,101,115,32,104,97, - 118,101,32,110,111,32,115,111,117,114,99,101,32,99,111,100, - 101,46,78,114,4,0,0,0,41,2,114,108,0,0,0,114, - 126,0,0,0,114,4,0,0,0,114,4,0,0,0,114,5, - 0,0,0,114,202,0,0,0,156,3,0,0,115,2,0,0, - 0,0,2,122,30,69,120,116,101,110,115,105,111,110,70,105, - 108,101,76,111,97,100,101,114,46,103,101,116,95,115,111,117, - 114,99,101,99,2,0,0,0,0,0,0,0,2,0,0,0, - 1,0,0,0,67,0,0,0,115,7,0,0,0,124,0,0, - 106,0,0,83,41,1,122,58,82,101,116,117,114,110,32,116, - 104,101,32,112,97,116,104,32,116,111,32,116,104,101,32,115, - 111,117,114,99,101,32,102,105,108,101,32,97,115,32,102,111, - 117,110,100,32,98,121,32,116,104,101,32,102,105,110,100,101, - 114,46,41,1,114,35,0,0,0,41,2,114,108,0,0,0, - 114,126,0,0,0,114,4,0,0,0,114,4,0,0,0,114, - 5,0,0,0,114,157,0,0,0,160,3,0,0,115,2,0, - 0,0,0,3,122,32,69,120,116,101,110,115,105,111,110,70, - 105,108,101,76,111,97,100,101,114,46,103,101,116,95,102,105, - 108,101,110,97,109,101,78,41,14,114,112,0,0,0,114,111, - 0,0,0,114,113,0,0,0,114,114,0,0,0,114,185,0, - 0,0,114,213,0,0,0,114,215,0,0,0,114,186,0,0, - 0,114,191,0,0,0,114,159,0,0,0,114,187,0,0,0, - 114,202,0,0,0,114,123,0,0,0,114,157,0,0,0,114, - 4,0,0,0,114,4,0,0,0,114,4,0,0,0,114,5, - 0,0,0,114,224,0,0,0,113,3,0,0,115,20,0,0, - 0,12,6,6,2,12,4,12,4,12,3,12,8,12,6,12, - 6,12,4,12,4,114,224,0,0,0,99,0,0,0,0,0, - 0,0,0,0,0,0,0,2,0,0,0,64,0,0,0,115, - 130,0,0,0,101,0,0,90,1,0,100,0,0,90,2,0, - 100,1,0,90,3,0,100,2,0,100,3,0,132,0,0,90, - 4,0,100,4,0,100,5,0,132,0,0,90,5,0,100,6, - 0,100,7,0,132,0,0,90,6,0,100,8,0,100,9,0, - 132,0,0,90,7,0,100,10,0,100,11,0,132,0,0,90, - 8,0,100,12,0,100,13,0,132,0,0,90,9,0,100,14, - 0,100,15,0,132,0,0,90,10,0,100,16,0,100,17,0, - 132,0,0,90,11,0,100,18,0,100,19,0,132,0,0,90, - 12,0,100,20,0,83,41,21,218,14,95,78,97,109,101,115, - 112,97,99,101,80,97,116,104,97,38,1,0,0,82,101,112, - 114,101,115,101,110,116,115,32,97,32,110,97,109,101,115,112, - 97,99,101,32,112,97,99,107,97,103,101,39,115,32,112,97, - 116,104,46,32,32,73,116,32,117,115,101,115,32,116,104,101, - 32,109,111,100,117,108,101,32,110,97,109,101,10,32,32,32, - 32,116,111,32,102,105,110,100,32,105,116,115,32,112,97,114, - 101,110,116,32,109,111,100,117,108,101,44,32,97,110,100,32, - 102,114,111,109,32,116,104,101,114,101,32,105,116,32,108,111, - 111,107,115,32,117,112,32,116,104,101,32,112,97,114,101,110, - 116,39,115,10,32,32,32,32,95,95,112,97,116,104,95,95, - 46,32,32,87,104,101,110,32,116,104,105,115,32,99,104,97, - 110,103,101,115,44,32,116,104,101,32,109,111,100,117,108,101, - 39,115,32,111,119,110,32,112,97,116,104,32,105,115,32,114, - 101,99,111,109,112,117,116,101,100,44,10,32,32,32,32,117, - 115,105,110,103,32,112,97,116,104,95,102,105,110,100,101,114, - 46,32,32,70,111,114,32,116,111,112,45,108,101,118,101,108, - 32,109,111,100,117,108,101,115,44,32,116,104,101,32,112,97, - 114,101,110,116,32,109,111,100,117,108,101,39,115,32,112,97, - 116,104,10,32,32,32,32,105,115,32,115,121,115,46,112,97, - 116,104,46,99,4,0,0,0,0,0,0,0,4,0,0,0, - 2,0,0,0,67,0,0,0,115,52,0,0,0,124,1,0, - 124,0,0,95,0,0,124,2,0,124,0,0,95,1,0,116, - 2,0,124,0,0,106,3,0,131,0,0,131,1,0,124,0, - 0,95,4,0,124,3,0,124,0,0,95,5,0,100,0,0, - 83,41,1,78,41,6,218,5,95,110,97,109,101,218,5,95, - 112,97,116,104,114,93,0,0,0,218,16,95,103,101,116,95, - 112,97,114,101,110,116,95,112,97,116,104,218,17,95,108,97, - 115,116,95,112,97,114,101,110,116,95,112,97,116,104,218,12, - 95,112,97,116,104,95,102,105,110,100,101,114,41,4,114,108, - 0,0,0,114,106,0,0,0,114,35,0,0,0,218,11,112, - 97,116,104,95,102,105,110,100,101,114,114,4,0,0,0,114, - 4,0,0,0,114,5,0,0,0,114,185,0,0,0,173,3, - 0,0,115,8,0,0,0,0,1,9,1,9,1,21,1,122, - 23,95,78,97,109,101,115,112,97,99,101,80,97,116,104,46, - 95,95,105,110,105,116,95,95,99,1,0,0,0,0,0,0, - 0,4,0,0,0,3,0,0,0,67,0,0,0,115,53,0, - 0,0,124,0,0,106,0,0,106,1,0,100,1,0,131,1, - 0,92,3,0,125,1,0,125,2,0,125,3,0,124,2,0, - 100,2,0,107,2,0,114,43,0,100,6,0,83,124,1,0, - 100,5,0,102,2,0,83,41,7,122,62,82,101,116,117,114, - 110,115,32,97,32,116,117,112,108,101,32,111,102,32,40,112, - 97,114,101,110,116,45,109,111,100,117,108,101,45,110,97,109, - 101,44,32,112,97,114,101,110,116,45,112,97,116,104,45,97, - 116,116,114,45,110,97,109,101,41,114,58,0,0,0,114,30, - 0,0,0,114,7,0,0,0,114,35,0,0,0,90,8,95, - 95,112,97,116,104,95,95,41,2,122,3,115,121,115,122,4, - 112,97,116,104,41,2,114,231,0,0,0,114,32,0,0,0, - 41,4,114,108,0,0,0,114,222,0,0,0,218,3,100,111, - 116,90,2,109,101,114,4,0,0,0,114,4,0,0,0,114, - 5,0,0,0,218,23,95,102,105,110,100,95,112,97,114,101, - 110,116,95,112,97,116,104,95,110,97,109,101,115,179,3,0, - 0,115,8,0,0,0,0,2,27,1,12,2,4,3,122,38, - 95,78,97,109,101,115,112,97,99,101,80,97,116,104,46,95, - 102,105,110,100,95,112,97,114,101,110,116,95,112,97,116,104, - 95,110,97,109,101,115,99,1,0,0,0,0,0,0,0,3, - 0,0,0,3,0,0,0,67,0,0,0,115,38,0,0,0, - 124,0,0,106,0,0,131,0,0,92,2,0,125,1,0,125, - 2,0,116,1,0,116,2,0,106,3,0,124,1,0,25,124, - 2,0,131,2,0,83,41,1,78,41,4,114,238,0,0,0, - 114,117,0,0,0,114,7,0,0,0,218,7,109,111,100,117, - 108,101,115,41,3,114,108,0,0,0,90,18,112,97,114,101, - 110,116,95,109,111,100,117,108,101,95,110,97,109,101,90,14, - 112,97,116,104,95,97,116,116,114,95,110,97,109,101,114,4, - 0,0,0,114,4,0,0,0,114,5,0,0,0,114,233,0, - 0,0,189,3,0,0,115,4,0,0,0,0,1,18,1,122, - 31,95,78,97,109,101,115,112,97,99,101,80,97,116,104,46, - 95,103,101,116,95,112,97,114,101,110,116,95,112,97,116,104, - 99,1,0,0,0,0,0,0,0,3,0,0,0,3,0,0, - 0,67,0,0,0,115,118,0,0,0,116,0,0,124,0,0, - 106,1,0,131,0,0,131,1,0,125,1,0,124,1,0,124, - 0,0,106,2,0,107,3,0,114,111,0,124,0,0,106,3, - 0,124,0,0,106,4,0,124,1,0,131,2,0,125,2,0, - 124,2,0,100,0,0,107,9,0,114,102,0,124,2,0,106, - 5,0,100,0,0,107,8,0,114,102,0,124,2,0,106,6, - 0,114,102,0,124,2,0,106,6,0,124,0,0,95,7,0, - 124,1,0,124,0,0,95,2,0,124,0,0,106,7,0,83, - 41,1,78,41,8,114,93,0,0,0,114,233,0,0,0,114, - 234,0,0,0,114,235,0,0,0,114,231,0,0,0,114,127, - 0,0,0,114,156,0,0,0,114,232,0,0,0,41,3,114, - 108,0,0,0,90,11,112,97,114,101,110,116,95,112,97,116, - 104,114,164,0,0,0,114,4,0,0,0,114,4,0,0,0, - 114,5,0,0,0,218,12,95,114,101,99,97,108,99,117,108, - 97,116,101,193,3,0,0,115,16,0,0,0,0,2,18,1, - 15,1,21,3,27,1,9,1,12,1,9,1,122,27,95,78, - 97,109,101,115,112,97,99,101,80,97,116,104,46,95,114,101, - 99,97,108,99,117,108,97,116,101,99,1,0,0,0,0,0, - 0,0,1,0,0,0,2,0,0,0,67,0,0,0,115,16, - 0,0,0,116,0,0,124,0,0,106,1,0,131,0,0,131, - 1,0,83,41,1,78,41,2,218,4,105,116,101,114,114,240, - 0,0,0,41,1,114,108,0,0,0,114,4,0,0,0,114, - 4,0,0,0,114,5,0,0,0,218,8,95,95,105,116,101, - 114,95,95,206,3,0,0,115,2,0,0,0,0,1,122,23, - 95,78,97,109,101,115,112,97,99,101,80,97,116,104,46,95, - 95,105,116,101,114,95,95,99,1,0,0,0,0,0,0,0, - 1,0,0,0,2,0,0,0,67,0,0,0,115,16,0,0, - 0,116,0,0,124,0,0,106,1,0,131,0,0,131,1,0, - 83,41,1,78,41,2,114,31,0,0,0,114,240,0,0,0, - 41,1,114,108,0,0,0,114,4,0,0,0,114,4,0,0, - 0,114,5,0,0,0,218,7,95,95,108,101,110,95,95,209, - 3,0,0,115,2,0,0,0,0,1,122,22,95,78,97,109, - 101,115,112,97,99,101,80,97,116,104,46,95,95,108,101,110, - 95,95,99,1,0,0,0,0,0,0,0,1,0,0,0,2, - 0,0,0,67,0,0,0,115,16,0,0,0,100,1,0,106, - 0,0,124,0,0,106,1,0,131,1,0,83,41,2,78,122, - 20,95,78,97,109,101,115,112,97,99,101,80,97,116,104,40, - 123,33,114,125,41,41,2,114,47,0,0,0,114,232,0,0, - 0,41,1,114,108,0,0,0,114,4,0,0,0,114,4,0, - 0,0,114,5,0,0,0,218,8,95,95,114,101,112,114,95, - 95,212,3,0,0,115,2,0,0,0,0,1,122,23,95,78, - 97,109,101,115,112,97,99,101,80,97,116,104,46,95,95,114, - 101,112,114,95,95,99,2,0,0,0,0,0,0,0,2,0, - 0,0,2,0,0,0,67,0,0,0,115,16,0,0,0,124, - 1,0,124,0,0,106,0,0,131,0,0,107,6,0,83,41, - 1,78,41,1,114,240,0,0,0,41,2,114,108,0,0,0, - 218,4,105,116,101,109,114,4,0,0,0,114,4,0,0,0, - 114,5,0,0,0,218,12,95,95,99,111,110,116,97,105,110, - 115,95,95,215,3,0,0,115,2,0,0,0,0,1,122,27, - 95,78,97,109,101,115,112,97,99,101,80,97,116,104,46,95, - 95,99,111,110,116,97,105,110,115,95,95,99,2,0,0,0, - 0,0,0,0,2,0,0,0,2,0,0,0,67,0,0,0, - 115,20,0,0,0,124,0,0,106,0,0,106,1,0,124,1, - 0,131,1,0,1,100,0,0,83,41,1,78,41,2,114,232, - 0,0,0,114,163,0,0,0,41,2,114,108,0,0,0,114, - 245,0,0,0,114,4,0,0,0,114,4,0,0,0,114,5, - 0,0,0,114,163,0,0,0,218,3,0,0,115,2,0,0, - 0,0,1,122,21,95,78,97,109,101,115,112,97,99,101,80, - 97,116,104,46,97,112,112,101,110,100,78,41,13,114,112,0, - 0,0,114,111,0,0,0,114,113,0,0,0,114,114,0,0, - 0,114,185,0,0,0,114,238,0,0,0,114,233,0,0,0, - 114,240,0,0,0,114,242,0,0,0,114,243,0,0,0,114, - 244,0,0,0,114,246,0,0,0,114,163,0,0,0,114,4, + 175,0,0,0,101,4,0,0,115,26,0,0,0,0,4,12, + 1,9,1,21,1,12,1,4,1,15,1,9,1,6,3,9, + 1,24,1,4,2,7,2,122,20,80,97,116,104,70,105,110, + 100,101,114,46,102,105,110,100,95,115,112,101,99,99,3,0, + 0,0,0,0,0,0,4,0,0,0,3,0,0,0,67,0, + 0,0,115,41,0,0,0,124,0,0,106,0,0,124,1,0, + 124,2,0,131,2,0,125,3,0,124,3,0,100,1,0,107, + 8,0,114,34,0,100,1,0,83,124,3,0,106,1,0,83, + 41,2,122,170,102,105,110,100,32,116,104,101,32,109,111,100, + 117,108,101,32,111,110,32,115,121,115,46,112,97,116,104,32, + 111,114,32,39,112,97,116,104,39,32,98,97,115,101,100,32, + 111,110,32,115,121,115,46,112,97,116,104,95,104,111,111,107, + 115,32,97,110,100,10,32,32,32,32,32,32,32,32,115,121, + 115,46,112,97,116,104,95,105,109,112,111,114,116,101,114,95, + 99,97,99,104,101,46,10,10,32,32,32,32,32,32,32,32, + 84,104,105,115,32,109,101,116,104,111,100,32,105,115,32,100, + 101,112,114,101,99,97,116,101,100,46,32,32,85,115,101,32, + 102,105,110,100,95,115,112,101,99,40,41,32,105,110,115,116, + 101,97,100,46,10,10,32,32,32,32,32,32,32,32,78,41, + 2,114,175,0,0,0,114,120,0,0,0,41,4,114,164,0, + 0,0,114,119,0,0,0,114,35,0,0,0,114,158,0,0, + 0,114,4,0,0,0,114,4,0,0,0,114,5,0,0,0, + 114,176,0,0,0,123,4,0,0,115,8,0,0,0,0,8, + 18,1,12,1,4,1,122,22,80,97,116,104,70,105,110,100, + 101,114,46,102,105,110,100,95,109,111,100,117,108,101,41,12, + 114,105,0,0,0,114,104,0,0,0,114,106,0,0,0,114, + 107,0,0,0,114,177,0,0,0,114,244,0,0,0,114,249, + 0,0,0,114,251,0,0,0,114,252,0,0,0,114,255,0, + 0,0,114,175,0,0,0,114,176,0,0,0,114,4,0,0, + 0,114,4,0,0,0,114,4,0,0,0,114,5,0,0,0, + 114,243,0,0,0,3,4,0,0,115,22,0,0,0,12,2, + 6,2,18,8,18,17,18,22,18,15,3,1,18,31,3,1, + 21,21,3,1,114,243,0,0,0,99,0,0,0,0,0,0, + 0,0,0,0,0,0,3,0,0,0,64,0,0,0,115,133, + 0,0,0,101,0,0,90,1,0,100,0,0,90,2,0,100, + 1,0,90,3,0,100,2,0,100,3,0,132,0,0,90,4, + 0,100,4,0,100,5,0,132,0,0,90,5,0,101,6,0, + 90,7,0,100,6,0,100,7,0,132,0,0,90,8,0,100, + 8,0,100,9,0,132,0,0,90,9,0,100,10,0,100,11, + 0,100,12,0,132,1,0,90,10,0,100,13,0,100,14,0, + 132,0,0,90,11,0,101,12,0,100,15,0,100,16,0,132, + 0,0,131,1,0,90,13,0,100,17,0,100,18,0,132,0, + 0,90,14,0,100,10,0,83,41,19,218,10,70,105,108,101, + 70,105,110,100,101,114,122,172,70,105,108,101,45,98,97,115, + 101,100,32,102,105,110,100,101,114,46,10,10,32,32,32,32, + 73,110,116,101,114,97,99,116,105,111,110,115,32,119,105,116, + 104,32,116,104,101,32,102,105,108,101,32,115,121,115,116,101, + 109,32,97,114,101,32,99,97,99,104,101,100,32,102,111,114, + 32,112,101,114,102,111,114,109,97,110,99,101,44,32,98,101, + 105,110,103,10,32,32,32,32,114,101,102,114,101,115,104,101, + 100,32,119,104,101,110,32,116,104,101,32,100,105,114,101,99, + 116,111,114,121,32,116,104,101,32,102,105,110,100,101,114,32, + 105,115,32,104,97,110,100,108,105,110,103,32,104,97,115,32, + 98,101,101,110,32,109,111,100,105,102,105,101,100,46,10,10, + 32,32,32,32,99,2,0,0,0,0,0,0,0,5,0,0, + 0,5,0,0,0,7,0,0,0,115,122,0,0,0,103,0, + 0,125,3,0,120,52,0,124,2,0,68,93,44,0,92,2, + 0,137,0,0,125,4,0,124,3,0,106,0,0,135,0,0, + 102,1,0,100,1,0,100,2,0,134,0,0,124,4,0,68, + 131,1,0,131,1,0,1,113,13,0,87,124,3,0,124,0, + 0,95,1,0,124,1,0,112,79,0,100,3,0,124,0,0, + 95,2,0,100,6,0,124,0,0,95,3,0,116,4,0,131, + 0,0,124,0,0,95,5,0,116,4,0,131,0,0,124,0, + 0,95,6,0,100,5,0,83,41,7,122,154,73,110,105,116, + 105,97,108,105,122,101,32,119,105,116,104,32,116,104,101,32, + 112,97,116,104,32,116,111,32,115,101,97,114,99,104,32,111, + 110,32,97,110,100,32,97,32,118,97,114,105,97,98,108,101, + 32,110,117,109,98,101,114,32,111,102,10,32,32,32,32,32, + 32,32,32,50,45,116,117,112,108,101,115,32,99,111,110,116, + 97,105,110,105,110,103,32,116,104,101,32,108,111,97,100,101, + 114,32,97,110,100,32,116,104,101,32,102,105,108,101,32,115, + 117,102,102,105,120,101,115,32,116,104,101,32,108,111,97,100, + 101,114,10,32,32,32,32,32,32,32,32,114,101,99,111,103, + 110,105,122,101,115,46,99,1,0,0,0,0,0,0,0,2, + 0,0,0,3,0,0,0,51,0,0,0,115,27,0,0,0, + 124,0,0,93,17,0,125,1,0,124,1,0,136,0,0,102, + 2,0,86,1,113,3,0,100,0,0,83,41,1,78,114,4, + 0,0,0,41,2,114,22,0,0,0,114,219,0,0,0,41, + 1,114,120,0,0,0,114,4,0,0,0,114,5,0,0,0, + 114,221,0,0,0,152,4,0,0,115,2,0,0,0,6,0, + 122,38,70,105,108,101,70,105,110,100,101,114,46,95,95,105, + 110,105,116,95,95,46,60,108,111,99,97,108,115,62,46,60, + 103,101,110,101,120,112,114,62,114,58,0,0,0,114,29,0, + 0,0,78,114,87,0,0,0,41,7,114,143,0,0,0,218, + 8,95,108,111,97,100,101,114,115,114,35,0,0,0,218,11, + 95,112,97,116,104,95,109,116,105,109,101,218,3,115,101,116, + 218,11,95,112,97,116,104,95,99,97,99,104,101,218,19,95, + 114,101,108,97,120,101,100,95,112,97,116,104,95,99,97,99, + 104,101,41,5,114,100,0,0,0,114,35,0,0,0,218,14, + 108,111,97,100,101,114,95,100,101,116,97,105,108,115,90,7, + 108,111,97,100,101,114,115,114,160,0,0,0,114,4,0,0, + 0,41,1,114,120,0,0,0,114,5,0,0,0,114,179,0, + 0,0,146,4,0,0,115,16,0,0,0,0,4,6,1,19, + 1,36,1,9,2,15,1,9,1,12,1,122,19,70,105,108, + 101,70,105,110,100,101,114,46,95,95,105,110,105,116,95,95, + 99,1,0,0,0,0,0,0,0,1,0,0,0,2,0,0, + 0,67,0,0,0,115,13,0,0,0,100,3,0,124,0,0, + 95,0,0,100,2,0,83,41,4,122,31,73,110,118,97,108, + 105,100,97,116,101,32,116,104,101,32,100,105,114,101,99,116, + 111,114,121,32,109,116,105,109,101,46,114,29,0,0,0,78, + 114,87,0,0,0,41,1,114,2,1,0,0,41,1,114,100, 0,0,0,114,4,0,0,0,114,4,0,0,0,114,5,0, - 0,0,114,230,0,0,0,166,3,0,0,115,20,0,0,0, - 12,5,6,2,12,6,12,10,12,4,12,13,12,3,12,3, - 12,3,12,3,114,230,0,0,0,99,0,0,0,0,0,0, - 0,0,0,0,0,0,3,0,0,0,64,0,0,0,115,118, - 0,0,0,101,0,0,90,1,0,100,0,0,90,2,0,100, - 1,0,100,2,0,132,0,0,90,3,0,101,4,0,100,3, - 0,100,4,0,132,0,0,131,1,0,90,5,0,100,5,0, - 100,6,0,132,0,0,90,6,0,100,7,0,100,8,0,132, - 0,0,90,7,0,100,9,0,100,10,0,132,0,0,90,8, - 0,100,11,0,100,12,0,132,0,0,90,9,0,100,13,0, - 100,14,0,132,0,0,90,10,0,100,15,0,100,16,0,132, - 0,0,90,11,0,100,17,0,83,41,18,218,16,95,78,97, - 109,101,115,112,97,99,101,76,111,97,100,101,114,99,4,0, - 0,0,0,0,0,0,4,0,0,0,4,0,0,0,67,0, - 0,0,115,25,0,0,0,116,0,0,124,1,0,124,2,0, - 124,3,0,131,3,0,124,0,0,95,1,0,100,0,0,83, - 41,1,78,41,2,114,230,0,0,0,114,232,0,0,0,41, - 4,114,108,0,0,0,114,106,0,0,0,114,35,0,0,0, - 114,236,0,0,0,114,4,0,0,0,114,4,0,0,0,114, - 5,0,0,0,114,185,0,0,0,224,3,0,0,115,2,0, - 0,0,0,1,122,25,95,78,97,109,101,115,112,97,99,101, - 76,111,97,100,101,114,46,95,95,105,110,105,116,95,95,99, - 2,0,0,0,0,0,0,0,2,0,0,0,2,0,0,0, - 67,0,0,0,115,16,0,0,0,100,1,0,106,0,0,124, - 1,0,106,1,0,131,1,0,83,41,2,122,115,82,101,116, - 117,114,110,32,114,101,112,114,32,102,111,114,32,116,104,101, - 32,109,111,100,117,108,101,46,10,10,32,32,32,32,32,32, - 32,32,84,104,101,32,109,101,116,104,111,100,32,105,115,32, - 100,101,112,114,101,99,97,116,101,100,46,32,32,84,104,101, - 32,105,109,112,111,114,116,32,109,97,99,104,105,110,101,114, - 121,32,100,111,101,115,32,116,104,101,32,106,111,98,32,105, - 116,115,101,108,102,46,10,10,32,32,32,32,32,32,32,32, - 122,25,60,109,111,100,117,108,101,32,123,33,114,125,32,40, - 110,97,109,101,115,112,97,99,101,41,62,41,2,114,47,0, - 0,0,114,112,0,0,0,41,2,114,170,0,0,0,114,190, - 0,0,0,114,4,0,0,0,114,4,0,0,0,114,5,0, - 0,0,218,11,109,111,100,117,108,101,95,114,101,112,114,227, - 3,0,0,115,2,0,0,0,0,7,122,28,95,78,97,109, - 101,115,112,97,99,101,76,111,97,100,101,114,46,109,111,100, - 117,108,101,95,114,101,112,114,99,2,0,0,0,0,0,0, - 0,2,0,0,0,1,0,0,0,67,0,0,0,115,4,0, - 0,0,100,1,0,83,41,2,78,84,114,4,0,0,0,41, - 2,114,108,0,0,0,114,126,0,0,0,114,4,0,0,0, - 114,4,0,0,0,114,5,0,0,0,114,159,0,0,0,236, - 3,0,0,115,2,0,0,0,0,1,122,27,95,78,97,109, - 101,115,112,97,99,101,76,111,97,100,101,114,46,105,115,95, - 112,97,99,107,97,103,101,99,2,0,0,0,0,0,0,0, - 2,0,0,0,1,0,0,0,67,0,0,0,115,4,0,0, - 0,100,1,0,83,41,2,78,114,30,0,0,0,114,4,0, - 0,0,41,2,114,108,0,0,0,114,126,0,0,0,114,4, - 0,0,0,114,4,0,0,0,114,5,0,0,0,114,202,0, - 0,0,239,3,0,0,115,2,0,0,0,0,1,122,27,95, - 78,97,109,101,115,112,97,99,101,76,111,97,100,101,114,46, - 103,101,116,95,115,111,117,114,99,101,99,2,0,0,0,0, - 0,0,0,2,0,0,0,6,0,0,0,67,0,0,0,115, - 22,0,0,0,116,0,0,100,1,0,100,2,0,100,3,0, - 100,4,0,100,5,0,131,3,1,83,41,6,78,114,30,0, - 0,0,122,8,60,115,116,114,105,110,103,62,114,189,0,0, - 0,114,204,0,0,0,84,41,1,114,205,0,0,0,41,2, - 114,108,0,0,0,114,126,0,0,0,114,4,0,0,0,114, - 4,0,0,0,114,5,0,0,0,114,187,0,0,0,242,3, - 0,0,115,2,0,0,0,0,1,122,25,95,78,97,109,101, - 115,112,97,99,101,76,111,97,100,101,114,46,103,101,116,95, - 99,111,100,101,99,2,0,0,0,0,0,0,0,2,0,0, - 0,1,0,0,0,67,0,0,0,115,4,0,0,0,100,1, - 0,83,41,2,122,42,85,115,101,32,100,101,102,97,117,108, - 116,32,115,101,109,97,110,116,105,99,115,32,102,111,114,32, - 109,111,100,117,108,101,32,99,114,101,97,116,105,111,110,46, - 78,114,4,0,0,0,41,2,114,108,0,0,0,114,164,0, + 0,0,114,244,0,0,0,160,4,0,0,115,2,0,0,0, + 0,2,122,28,70,105,108,101,70,105,110,100,101,114,46,105, + 110,118,97,108,105,100,97,116,101,95,99,97,99,104,101,115, + 99,2,0,0,0,0,0,0,0,3,0,0,0,2,0,0, + 0,67,0,0,0,115,59,0,0,0,124,0,0,106,0,0, + 124,1,0,131,1,0,125,2,0,124,2,0,100,1,0,107, + 8,0,114,37,0,100,1,0,103,0,0,102,2,0,83,124, + 2,0,106,1,0,124,2,0,106,2,0,112,55,0,103,0, + 0,102,2,0,83,41,2,122,197,84,114,121,32,116,111,32, + 102,105,110,100,32,97,32,108,111,97,100,101,114,32,102,111, + 114,32,116,104,101,32,115,112,101,99,105,102,105,101,100,32, + 109,111,100,117,108,101,44,32,111,114,32,116,104,101,32,110, + 97,109,101,115,112,97,99,101,10,32,32,32,32,32,32,32, + 32,112,97,99,107,97,103,101,32,112,111,114,116,105,111,110, + 115,46,32,82,101,116,117,114,110,115,32,40,108,111,97,100, + 101,114,44,32,108,105,115,116,45,111,102,45,112,111,114,116, + 105,111,110,115,41,46,10,10,32,32,32,32,32,32,32,32, + 84,104,105,115,32,109,101,116,104,111,100,32,105,115,32,100, + 101,112,114,101,99,97,116,101,100,46,32,32,85,115,101,32, + 102,105,110,100,95,115,112,101,99,40,41,32,105,110,115,116, + 101,97,100,46,10,10,32,32,32,32,32,32,32,32,78,41, + 3,114,175,0,0,0,114,120,0,0,0,114,150,0,0,0, + 41,3,114,100,0,0,0,114,119,0,0,0,114,158,0,0, + 0,114,4,0,0,0,114,4,0,0,0,114,5,0,0,0, + 114,117,0,0,0,166,4,0,0,115,8,0,0,0,0,7, + 15,1,12,1,10,1,122,22,70,105,108,101,70,105,110,100, + 101,114,46,102,105,110,100,95,108,111,97,100,101,114,99,6, + 0,0,0,0,0,0,0,7,0,0,0,7,0,0,0,67, + 0,0,0,115,40,0,0,0,124,1,0,124,2,0,124,3, + 0,131,2,0,125,6,0,116,0,0,124,2,0,124,3,0, + 100,1,0,124,6,0,100,2,0,124,4,0,131,2,2,83, + 41,3,78,114,120,0,0,0,114,150,0,0,0,41,1,114, + 161,0,0,0,41,7,114,100,0,0,0,114,159,0,0,0, + 114,119,0,0,0,114,35,0,0,0,90,4,115,109,115,108, + 114,174,0,0,0,114,120,0,0,0,114,4,0,0,0,114, + 4,0,0,0,114,5,0,0,0,114,255,0,0,0,178,4, + 0,0,115,6,0,0,0,0,1,15,1,18,1,122,20,70, + 105,108,101,70,105,110,100,101,114,46,95,103,101,116,95,115, + 112,101,99,78,99,3,0,0,0,0,0,0,0,14,0,0, + 0,15,0,0,0,67,0,0,0,115,228,1,0,0,100,1, + 0,125,3,0,124,1,0,106,0,0,100,2,0,131,1,0, + 100,3,0,25,125,4,0,121,34,0,116,1,0,124,0,0, + 106,2,0,112,49,0,116,3,0,106,4,0,131,0,0,131, + 1,0,106,5,0,125,5,0,87,110,24,0,4,116,6,0, + 107,10,0,114,85,0,1,1,1,100,10,0,125,5,0,89, + 110,1,0,88,124,5,0,124,0,0,106,7,0,107,3,0, + 114,120,0,124,0,0,106,8,0,131,0,0,1,124,5,0, + 124,0,0,95,7,0,116,9,0,131,0,0,114,153,0,124, + 0,0,106,10,0,125,6,0,124,4,0,106,11,0,131,0, + 0,125,7,0,110,15,0,124,0,0,106,12,0,125,6,0, + 124,4,0,125,7,0,124,7,0,124,6,0,107,6,0,114, + 45,1,116,13,0,124,0,0,106,2,0,124,4,0,131,2, + 0,125,8,0,120,100,0,124,0,0,106,14,0,68,93,77, + 0,92,2,0,125,9,0,125,10,0,100,5,0,124,9,0, + 23,125,11,0,116,13,0,124,8,0,124,11,0,131,2,0, + 125,12,0,116,15,0,124,12,0,131,1,0,114,208,0,124, + 0,0,106,16,0,124,10,0,124,1,0,124,12,0,124,8, + 0,103,1,0,124,2,0,131,5,0,83,113,208,0,87,116, + 17,0,124,8,0,131,1,0,125,3,0,120,120,0,124,0, + 0,106,14,0,68,93,109,0,92,2,0,125,9,0,125,10, + 0,116,13,0,124,0,0,106,2,0,124,4,0,124,9,0, + 23,131,2,0,125,12,0,116,18,0,106,19,0,100,6,0, + 124,12,0,100,7,0,100,3,0,131,2,1,1,124,7,0, + 124,9,0,23,124,6,0,107,6,0,114,55,1,116,15,0, + 124,12,0,131,1,0,114,55,1,124,0,0,106,16,0,124, + 10,0,124,1,0,124,12,0,100,8,0,124,2,0,131,5, + 0,83,113,55,1,87,124,3,0,114,224,1,116,18,0,106, + 19,0,100,9,0,124,8,0,131,2,0,1,116,18,0,106, + 20,0,124,1,0,100,8,0,131,2,0,125,13,0,124,8, + 0,103,1,0,124,13,0,95,21,0,124,13,0,83,100,8, + 0,83,41,11,122,125,84,114,121,32,116,111,32,102,105,110, + 100,32,97,32,108,111,97,100,101,114,32,102,111,114,32,116, + 104,101,32,115,112,101,99,105,102,105,101,100,32,109,111,100, + 117,108,101,44,32,111,114,32,116,104,101,32,110,97,109,101, + 115,112,97,99,101,10,32,32,32,32,32,32,32,32,112,97, + 99,107,97,103,101,32,112,111,114,116,105,111,110,115,46,32, + 82,101,116,117,114,110,115,32,40,108,111,97,100,101,114,44, + 32,108,105,115,116,45,111,102,45,112,111,114,116,105,111,110, + 115,41,46,70,114,58,0,0,0,114,56,0,0,0,114,29, + 0,0,0,114,179,0,0,0,122,9,116,114,121,105,110,103, + 32,123,125,90,9,118,101,114,98,111,115,105,116,121,78,122, + 25,112,111,115,115,105,98,108,101,32,110,97,109,101,115,112, + 97,99,101,32,102,111,114,32,123,125,114,87,0,0,0,41, + 22,114,32,0,0,0,114,39,0,0,0,114,35,0,0,0, + 114,3,0,0,0,114,45,0,0,0,114,213,0,0,0,114, + 40,0,0,0,114,2,1,0,0,218,11,95,102,105,108,108, + 95,99,97,99,104,101,114,6,0,0,0,114,5,1,0,0, + 114,88,0,0,0,114,4,1,0,0,114,28,0,0,0,114, + 1,1,0,0,114,44,0,0,0,114,255,0,0,0,114,46, + 0,0,0,114,114,0,0,0,114,129,0,0,0,114,154,0, + 0,0,114,150,0,0,0,41,14,114,100,0,0,0,114,119, + 0,0,0,114,174,0,0,0,90,12,105,115,95,110,97,109, + 101,115,112,97,99,101,90,11,116,97,105,108,95,109,111,100, + 117,108,101,114,126,0,0,0,90,5,99,97,99,104,101,90, + 12,99,97,99,104,101,95,109,111,100,117,108,101,90,9,98, + 97,115,101,95,112,97,116,104,114,219,0,0,0,114,159,0, + 0,0,90,13,105,110,105,116,95,102,105,108,101,110,97,109, + 101,90,9,102,117,108,108,95,112,97,116,104,114,158,0,0, + 0,114,4,0,0,0,114,4,0,0,0,114,5,0,0,0, + 114,175,0,0,0,183,4,0,0,115,70,0,0,0,0,3, + 6,1,19,1,3,1,34,1,13,1,11,1,15,1,10,1, + 9,2,9,1,9,1,15,2,9,1,6,2,12,1,18,1, + 22,1,10,1,15,1,12,1,32,4,12,2,22,1,22,1, + 22,1,16,1,12,1,15,1,14,1,6,1,16,1,18,1, + 12,1,4,1,122,20,70,105,108,101,70,105,110,100,101,114, + 46,102,105,110,100,95,115,112,101,99,99,1,0,0,0,0, + 0,0,0,9,0,0,0,13,0,0,0,67,0,0,0,115, + 11,1,0,0,124,0,0,106,0,0,125,1,0,121,31,0, + 116,1,0,106,2,0,124,1,0,112,33,0,116,1,0,106, + 3,0,131,0,0,131,1,0,125,2,0,87,110,33,0,4, + 116,4,0,116,5,0,116,6,0,102,3,0,107,10,0,114, + 75,0,1,1,1,103,0,0,125,2,0,89,110,1,0,88, + 116,7,0,106,8,0,106,9,0,100,1,0,131,1,0,115, + 112,0,116,10,0,124,2,0,131,1,0,124,0,0,95,11, + 0,110,111,0,116,10,0,131,0,0,125,3,0,120,90,0, + 124,2,0,68,93,82,0,125,4,0,124,4,0,106,12,0, + 100,2,0,131,1,0,92,3,0,125,5,0,125,6,0,125, + 7,0,124,6,0,114,191,0,100,3,0,106,13,0,124,5, + 0,124,7,0,106,14,0,131,0,0,131,2,0,125,8,0, + 110,6,0,124,5,0,125,8,0,124,3,0,106,15,0,124, + 8,0,131,1,0,1,113,128,0,87,124,3,0,124,0,0, + 95,11,0,116,7,0,106,8,0,106,9,0,116,16,0,131, + 1,0,114,7,1,100,4,0,100,5,0,132,0,0,124,2, + 0,68,131,1,0,124,0,0,95,17,0,100,6,0,83,41, + 7,122,68,70,105,108,108,32,116,104,101,32,99,97,99,104, + 101,32,111,102,32,112,111,116,101,110,116,105,97,108,32,109, + 111,100,117,108,101,115,32,97,110,100,32,112,97,99,107,97, + 103,101,115,32,102,111,114,32,116,104,105,115,32,100,105,114, + 101,99,116,111,114,121,46,114,0,0,0,0,114,58,0,0, + 0,122,5,123,125,46,123,125,99,1,0,0,0,0,0,0, + 0,2,0,0,0,3,0,0,0,83,0,0,0,115,28,0, + 0,0,104,0,0,124,0,0,93,18,0,125,1,0,124,1, + 0,106,0,0,131,0,0,146,2,0,113,6,0,83,114,4, + 0,0,0,41,1,114,88,0,0,0,41,2,114,22,0,0, + 0,90,2,102,110,114,4,0,0,0,114,4,0,0,0,114, + 5,0,0,0,250,9,60,115,101,116,99,111,109,112,62,2, + 5,0,0,115,2,0,0,0,9,0,122,41,70,105,108,101, + 70,105,110,100,101,114,46,95,102,105,108,108,95,99,97,99, + 104,101,46,60,108,111,99,97,108,115,62,46,60,115,101,116, + 99,111,109,112,62,78,41,18,114,35,0,0,0,114,3,0, + 0,0,90,7,108,105,115,116,100,105,114,114,45,0,0,0, + 114,250,0,0,0,218,15,80,101,114,109,105,115,115,105,111, + 110,69,114,114,111,114,218,18,78,111,116,65,68,105,114,101, + 99,116,111,114,121,69,114,114,111,114,114,7,0,0,0,114, + 8,0,0,0,114,9,0,0,0,114,3,1,0,0,114,4, + 1,0,0,114,83,0,0,0,114,47,0,0,0,114,88,0, + 0,0,218,3,97,100,100,114,10,0,0,0,114,5,1,0, + 0,41,9,114,100,0,0,0,114,35,0,0,0,90,8,99, + 111,110,116,101,110,116,115,90,21,108,111,119,101,114,95,115, + 117,102,102,105,120,95,99,111,110,116,101,110,116,115,114,239, + 0,0,0,114,98,0,0,0,114,231,0,0,0,114,219,0, + 0,0,90,8,110,101,119,95,110,97,109,101,114,4,0,0, + 0,114,4,0,0,0,114,5,0,0,0,114,7,1,0,0, + 229,4,0,0,115,34,0,0,0,0,2,9,1,3,1,31, + 1,22,3,11,3,18,1,18,7,9,1,13,1,24,1,6, + 1,27,2,6,1,17,1,9,1,18,1,122,22,70,105,108, + 101,70,105,110,100,101,114,46,95,102,105,108,108,95,99,97, + 99,104,101,99,1,0,0,0,0,0,0,0,3,0,0,0, + 3,0,0,0,7,0,0,0,115,25,0,0,0,135,0,0, + 135,1,0,102,2,0,100,1,0,100,2,0,134,0,0,125, + 2,0,124,2,0,83,41,3,97,20,1,0,0,65,32,99, + 108,97,115,115,32,109,101,116,104,111,100,32,119,104,105,99, + 104,32,114,101,116,117,114,110,115,32,97,32,99,108,111,115, + 117,114,101,32,116,111,32,117,115,101,32,111,110,32,115,121, + 115,46,112,97,116,104,95,104,111,111,107,10,32,32,32,32, + 32,32,32,32,119,104,105,99,104,32,119,105,108,108,32,114, + 101,116,117,114,110,32,97,110,32,105,110,115,116,97,110,99, + 101,32,117,115,105,110,103,32,116,104,101,32,115,112,101,99, + 105,102,105,101,100,32,108,111,97,100,101,114,115,32,97,110, + 100,32,116,104,101,32,112,97,116,104,10,32,32,32,32,32, + 32,32,32,99,97,108,108,101,100,32,111,110,32,116,104,101, + 32,99,108,111,115,117,114,101,46,10,10,32,32,32,32,32, + 32,32,32,73,102,32,116,104,101,32,112,97,116,104,32,99, + 97,108,108,101,100,32,111,110,32,116,104,101,32,99,108,111, + 115,117,114,101,32,105,115,32,110,111,116,32,97,32,100,105, + 114,101,99,116,111,114,121,44,32,73,109,112,111,114,116,69, + 114,114,111,114,32,105,115,10,32,32,32,32,32,32,32,32, + 114,97,105,115,101,100,46,10,10,32,32,32,32,32,32,32, + 32,99,1,0,0,0,0,0,0,0,1,0,0,0,4,0, + 0,0,19,0,0,0,115,43,0,0,0,116,0,0,124,0, + 0,131,1,0,115,30,0,116,1,0,100,1,0,100,2,0, + 124,0,0,131,1,1,130,1,0,136,0,0,124,0,0,136, + 1,0,140,1,0,83,41,3,122,45,80,97,116,104,32,104, + 111,111,107,32,102,111,114,32,105,109,112,111,114,116,108,105, + 98,46,109,97,99,104,105,110,101,114,121,46,70,105,108,101, + 70,105,110,100,101,114,46,122,30,111,110,108,121,32,100,105, + 114,101,99,116,111,114,105,101,115,32,97,114,101,32,115,117, + 112,112,111,114,116,101,100,114,35,0,0,0,41,2,114,46, + 0,0,0,114,99,0,0,0,41,1,114,35,0,0,0,41, + 2,114,164,0,0,0,114,6,1,0,0,114,4,0,0,0, + 114,5,0,0,0,218,24,112,97,116,104,95,104,111,111,107, + 95,102,111,114,95,70,105,108,101,70,105,110,100,101,114,14, + 5,0,0,115,6,0,0,0,0,2,12,1,18,1,122,54, + 70,105,108,101,70,105,110,100,101,114,46,112,97,116,104,95, + 104,111,111,107,46,60,108,111,99,97,108,115,62,46,112,97, + 116,104,95,104,111,111,107,95,102,111,114,95,70,105,108,101, + 70,105,110,100,101,114,114,4,0,0,0,41,3,114,164,0, + 0,0,114,6,1,0,0,114,12,1,0,0,114,4,0,0, + 0,41,2,114,164,0,0,0,114,6,1,0,0,114,5,0, + 0,0,218,9,112,97,116,104,95,104,111,111,107,4,5,0, + 0,115,4,0,0,0,0,10,21,6,122,20,70,105,108,101, + 70,105,110,100,101,114,46,112,97,116,104,95,104,111,111,107, + 99,1,0,0,0,0,0,0,0,1,0,0,0,2,0,0, + 0,67,0,0,0,115,16,0,0,0,100,1,0,106,0,0, + 124,0,0,106,1,0,131,1,0,83,41,2,78,122,16,70, + 105,108,101,70,105,110,100,101,114,40,123,33,114,125,41,41, + 2,114,47,0,0,0,114,35,0,0,0,41,1,114,100,0, 0,0,114,4,0,0,0,114,4,0,0,0,114,5,0,0, - 0,114,186,0,0,0,245,3,0,0,115,0,0,0,0,122, - 30,95,78,97,109,101,115,112,97,99,101,76,111,97,100,101, - 114,46,99,114,101,97,116,101,95,109,111,100,117,108,101,99, - 2,0,0,0,0,0,0,0,2,0,0,0,1,0,0,0, - 67,0,0,0,115,4,0,0,0,100,0,0,83,41,1,78, - 114,4,0,0,0,41,2,114,108,0,0,0,114,190,0,0, + 0,114,238,0,0,0,22,5,0,0,115,2,0,0,0,0, + 1,122,19,70,105,108,101,70,105,110,100,101,114,46,95,95, + 114,101,112,114,95,95,41,15,114,105,0,0,0,114,104,0, + 0,0,114,106,0,0,0,114,107,0,0,0,114,179,0,0, + 0,114,244,0,0,0,114,123,0,0,0,114,176,0,0,0, + 114,117,0,0,0,114,255,0,0,0,114,175,0,0,0,114, + 7,1,0,0,114,177,0,0,0,114,13,1,0,0,114,238, + 0,0,0,114,4,0,0,0,114,4,0,0,0,114,4,0, + 0,0,114,5,0,0,0,114,0,1,0,0,137,4,0,0, + 115,20,0,0,0,12,7,6,2,12,14,12,4,6,2,12, + 12,12,5,15,46,12,31,18,18,114,0,1,0,0,99,4, + 0,0,0,0,0,0,0,6,0,0,0,11,0,0,0,67, + 0,0,0,115,195,0,0,0,124,0,0,106,0,0,100,1, + 0,131,1,0,125,4,0,124,0,0,106,0,0,100,2,0, + 131,1,0,125,5,0,124,4,0,115,99,0,124,5,0,114, + 54,0,124,5,0,106,1,0,125,4,0,110,45,0,124,2, + 0,124,3,0,107,2,0,114,84,0,116,2,0,124,1,0, + 124,2,0,131,2,0,125,4,0,110,15,0,116,3,0,124, + 1,0,124,2,0,131,2,0,125,4,0,124,5,0,115,126, + 0,116,4,0,124,1,0,124,2,0,100,3,0,124,4,0, + 131,2,1,125,5,0,121,44,0,124,5,0,124,0,0,100, + 2,0,60,124,4,0,124,0,0,100,1,0,60,124,2,0, + 124,0,0,100,4,0,60,124,3,0,124,0,0,100,5,0, + 60,87,110,18,0,4,116,5,0,107,10,0,114,190,0,1, + 1,1,89,110,1,0,88,100,0,0,83,41,6,78,218,10, + 95,95,108,111,97,100,101,114,95,95,218,8,95,95,115,112, + 101,99,95,95,114,120,0,0,0,90,8,95,95,102,105,108, + 101,95,95,90,10,95,95,99,97,99,104,101,100,95,95,41, + 6,218,3,103,101,116,114,120,0,0,0,114,217,0,0,0, + 114,212,0,0,0,114,161,0,0,0,218,9,69,120,99,101, + 112,116,105,111,110,41,6,90,2,110,115,114,98,0,0,0, + 90,8,112,97,116,104,110,97,109,101,90,9,99,112,97,116, + 104,110,97,109,101,114,120,0,0,0,114,158,0,0,0,114, + 4,0,0,0,114,4,0,0,0,114,5,0,0,0,218,14, + 95,102,105,120,95,117,112,95,109,111,100,117,108,101,28,5, + 0,0,115,34,0,0,0,0,2,15,1,15,1,6,1,6, + 1,12,1,12,1,18,2,15,1,6,1,21,1,3,1,10, + 1,10,1,10,1,14,1,13,2,114,18,1,0,0,99,0, + 0,0,0,0,0,0,0,3,0,0,0,3,0,0,0,67, + 0,0,0,115,55,0,0,0,116,0,0,116,1,0,106,2, + 0,131,0,0,102,2,0,125,0,0,116,3,0,116,4,0, + 102,2,0,125,1,0,116,5,0,116,6,0,102,2,0,125, + 2,0,124,0,0,124,1,0,124,2,0,103,3,0,83,41, + 1,122,95,82,101,116,117,114,110,115,32,97,32,108,105,115, + 116,32,111,102,32,102,105,108,101,45,98,97,115,101,100,32, + 109,111,100,117,108,101,32,108,111,97,100,101,114,115,46,10, + 10,32,32,32,32,69,97,99,104,32,105,116,101,109,32,105, + 115,32,97,32,116,117,112,108,101,32,40,108,111,97,100,101, + 114,44,32,115,117,102,102,105,120,101,115,41,46,10,32,32, + 32,32,41,7,114,218,0,0,0,114,139,0,0,0,218,18, + 101,120,116,101,110,115,105,111,110,95,115,117,102,102,105,120, + 101,115,114,212,0,0,0,114,84,0,0,0,114,217,0,0, + 0,114,74,0,0,0,41,3,90,10,101,120,116,101,110,115, + 105,111,110,115,90,6,115,111,117,114,99,101,90,8,98,121, + 116,101,99,111,100,101,114,4,0,0,0,114,4,0,0,0, + 114,5,0,0,0,114,155,0,0,0,51,5,0,0,115,8, + 0,0,0,0,5,18,1,12,1,12,1,114,155,0,0,0, + 99,1,0,0,0,0,0,0,0,12,0,0,0,12,0,0, + 0,67,0,0,0,115,70,2,0,0,124,0,0,97,0,0, + 116,0,0,106,1,0,97,1,0,116,0,0,106,2,0,97, + 2,0,116,1,0,106,3,0,116,4,0,25,125,1,0,120, + 76,0,100,26,0,68,93,68,0,125,2,0,124,2,0,116, + 1,0,106,3,0,107,7,0,114,83,0,116,0,0,106,5, + 0,124,2,0,131,1,0,125,3,0,110,13,0,116,1,0, + 106,3,0,124,2,0,25,125,3,0,116,6,0,124,1,0, + 124,2,0,124,3,0,131,3,0,1,113,44,0,87,100,5, + 0,100,6,0,103,1,0,102,2,0,100,7,0,100,8,0, + 100,6,0,103,2,0,102,2,0,102,2,0,125,4,0,120, + 149,0,124,4,0,68,93,129,0,92,2,0,125,5,0,125, + 6,0,116,7,0,100,9,0,100,10,0,132,0,0,124,6, + 0,68,131,1,0,131,1,0,115,199,0,116,8,0,130,1, + 0,124,6,0,100,11,0,25,125,7,0,124,5,0,116,1, + 0,106,3,0,107,6,0,114,241,0,116,1,0,106,3,0, + 124,5,0,25,125,8,0,80,113,156,0,121,20,0,116,0, + 0,106,5,0,124,5,0,131,1,0,125,8,0,80,87,113, + 156,0,4,116,9,0,107,10,0,114,28,1,1,1,1,119, + 156,0,89,113,156,0,88,113,156,0,87,116,9,0,100,12, + 0,131,1,0,130,1,0,116,6,0,124,1,0,100,13,0, + 124,8,0,131,3,0,1,116,6,0,124,1,0,100,14,0, + 124,7,0,131,3,0,1,116,6,0,124,1,0,100,15,0, + 100,16,0,106,10,0,124,6,0,131,1,0,131,3,0,1, + 121,19,0,116,0,0,106,5,0,100,17,0,131,1,0,125, + 9,0,87,110,24,0,4,116,9,0,107,10,0,114,147,1, + 1,1,1,100,18,0,125,9,0,89,110,1,0,88,116,6, + 0,124,1,0,100,17,0,124,9,0,131,3,0,1,116,0, + 0,106,5,0,100,19,0,131,1,0,125,10,0,116,6,0, + 124,1,0,100,19,0,124,10,0,131,3,0,1,124,5,0, + 100,7,0,107,2,0,114,238,1,116,0,0,106,5,0,100, + 20,0,131,1,0,125,11,0,116,6,0,124,1,0,100,21, + 0,124,11,0,131,3,0,1,116,6,0,124,1,0,100,22, + 0,116,11,0,131,0,0,131,3,0,1,116,12,0,106,13, + 0,116,2,0,106,14,0,131,0,0,131,1,0,1,124,5, + 0,100,7,0,107,2,0,114,66,2,116,15,0,106,16,0, + 100,23,0,131,1,0,1,100,24,0,116,12,0,107,6,0, + 114,66,2,100,25,0,116,17,0,95,18,0,100,18,0,83, + 41,27,122,205,83,101,116,117,112,32,116,104,101,32,112,97, + 116,104,45,98,97,115,101,100,32,105,109,112,111,114,116,101, + 114,115,32,102,111,114,32,105,109,112,111,114,116,108,105,98, + 32,98,121,32,105,109,112,111,114,116,105,110,103,32,110,101, + 101,100,101,100,10,32,32,32,32,98,117,105,108,116,45,105, + 110,32,109,111,100,117,108,101,115,32,97,110,100,32,105,110, + 106,101,99,116,105,110,103,32,116,104,101,109,32,105,110,116, + 111,32,116,104,101,32,103,108,111,98,97,108,32,110,97,109, + 101,115,112,97,99,101,46,10,10,32,32,32,32,79,116,104, + 101,114,32,99,111,109,112,111,110,101,110,116,115,32,97,114, + 101,32,101,120,116,114,97,99,116,101,100,32,102,114,111,109, + 32,116,104,101,32,99,111,114,101,32,98,111,111,116,115,116, + 114,97,112,32,109,111,100,117,108,101,46,10,10,32,32,32, + 32,114,49,0,0,0,114,60,0,0,0,218,8,98,117,105, + 108,116,105,110,115,114,136,0,0,0,90,5,112,111,115,105, + 120,250,1,47,218,2,110,116,250,1,92,99,1,0,0,0, + 0,0,0,0,2,0,0,0,3,0,0,0,115,0,0,0, + 115,33,0,0,0,124,0,0,93,23,0,125,1,0,116,0, + 0,124,1,0,131,1,0,100,0,0,107,2,0,86,1,113, + 3,0,100,1,0,83,41,2,114,29,0,0,0,78,41,1, + 114,31,0,0,0,41,2,114,22,0,0,0,114,77,0,0, 0,114,4,0,0,0,114,4,0,0,0,114,5,0,0,0, - 114,191,0,0,0,248,3,0,0,115,2,0,0,0,0,1, - 122,28,95,78,97,109,101,115,112,97,99,101,76,111,97,100, - 101,114,46,101,120,101,99,95,109,111,100,117,108,101,99,2, - 0,0,0,0,0,0,0,2,0,0,0,3,0,0,0,67, - 0,0,0,115,32,0,0,0,116,0,0,100,1,0,124,0, - 0,106,1,0,131,2,0,1,116,2,0,106,3,0,124,0, - 0,124,1,0,131,2,0,83,41,2,122,98,76,111,97,100, - 32,97,32,110,97,109,101,115,112,97,99,101,32,109,111,100, - 117,108,101,46,10,10,32,32,32,32,32,32,32,32,84,104, - 105,115,32,109,101,116,104,111,100,32,105,115,32,100,101,112, - 114,101,99,97,116,101,100,46,32,32,85,115,101,32,101,120, - 101,99,95,109,111,100,117,108,101,40,41,32,105,110,115,116, - 101,97,100,46,10,10,32,32,32,32,32,32,32,32,122,38, - 110,97,109,101,115,112,97,99,101,32,109,111,100,117,108,101, - 32,108,111,97,100,101,100,32,119,105,116,104,32,112,97,116, - 104,32,123,33,114,125,41,4,114,105,0,0,0,114,232,0, - 0,0,114,121,0,0,0,114,192,0,0,0,41,2,114,108, - 0,0,0,114,126,0,0,0,114,4,0,0,0,114,4,0, - 0,0,114,5,0,0,0,114,193,0,0,0,251,3,0,0, - 115,4,0,0,0,0,7,16,1,122,28,95,78,97,109,101, - 115,112,97,99,101,76,111,97,100,101,114,46,108,111,97,100, - 95,109,111,100,117,108,101,78,41,12,114,112,0,0,0,114, - 111,0,0,0,114,113,0,0,0,114,185,0,0,0,114,183, - 0,0,0,114,248,0,0,0,114,159,0,0,0,114,202,0, - 0,0,114,187,0,0,0,114,186,0,0,0,114,191,0,0, - 0,114,193,0,0,0,114,4,0,0,0,114,4,0,0,0, - 114,4,0,0,0,114,5,0,0,0,114,247,0,0,0,223, - 3,0,0,115,16,0,0,0,12,1,12,3,18,9,12,3, - 12,3,12,3,12,3,12,3,114,247,0,0,0,99,0,0, - 0,0,0,0,0,0,0,0,0,0,5,0,0,0,64,0, - 0,0,115,160,0,0,0,101,0,0,90,1,0,100,0,0, - 90,2,0,100,1,0,90,3,0,101,4,0,100,2,0,100, - 3,0,132,0,0,131,1,0,90,5,0,101,4,0,100,4, - 0,100,5,0,132,0,0,131,1,0,90,6,0,101,4,0, - 100,6,0,100,7,0,132,0,0,131,1,0,90,7,0,101, - 4,0,100,8,0,100,9,0,132,0,0,131,1,0,90,8, - 0,101,4,0,100,10,0,100,11,0,100,12,0,132,1,0, - 131,1,0,90,9,0,101,4,0,100,10,0,100,10,0,100, - 13,0,100,14,0,132,2,0,131,1,0,90,10,0,101,4, - 0,100,10,0,100,15,0,100,16,0,132,1,0,131,1,0, - 90,11,0,100,10,0,83,41,17,218,10,80,97,116,104,70, - 105,110,100,101,114,122,62,77,101,116,97,32,112,97,116,104, - 32,102,105,110,100,101,114,32,102,111,114,32,115,121,115,46, - 112,97,116,104,32,97,110,100,32,112,97,99,107,97,103,101, - 32,95,95,112,97,116,104,95,95,32,97,116,116,114,105,98, - 117,116,101,115,46,99,1,0,0,0,0,0,0,0,2,0, - 0,0,4,0,0,0,67,0,0,0,115,55,0,0,0,120, - 48,0,116,0,0,106,1,0,106,2,0,131,0,0,68,93, - 31,0,125,1,0,116,3,0,124,1,0,100,1,0,131,2, - 0,114,16,0,124,1,0,106,4,0,131,0,0,1,113,16, - 0,87,100,2,0,83,41,3,122,125,67,97,108,108,32,116, - 104,101,32,105,110,118,97,108,105,100,97,116,101,95,99,97, - 99,104,101,115,40,41,32,109,101,116,104,111,100,32,111,110, - 32,97,108,108,32,112,97,116,104,32,101,110,116,114,121,32, - 102,105,110,100,101,114,115,10,32,32,32,32,32,32,32,32, - 115,116,111,114,101,100,32,105,110,32,115,121,115,46,112,97, - 116,104,95,105,109,112,111,114,116,101,114,95,99,97,99,104, - 101,115,32,40,119,104,101,114,101,32,105,109,112,108,101,109, - 101,110,116,101,100,41,46,218,17,105,110,118,97,108,105,100, - 97,116,101,95,99,97,99,104,101,115,78,41,5,114,7,0, - 0,0,218,19,112,97,116,104,95,105,109,112,111,114,116,101, - 114,95,99,97,99,104,101,218,6,118,97,108,117,101,115,114, - 115,0,0,0,114,250,0,0,0,41,2,114,170,0,0,0, - 218,6,102,105,110,100,101,114,114,4,0,0,0,114,4,0, - 0,0,114,5,0,0,0,114,250,0,0,0,12,4,0,0, - 115,6,0,0,0,0,4,22,1,15,1,122,28,80,97,116, - 104,70,105,110,100,101,114,46,105,110,118,97,108,105,100,97, - 116,101,95,99,97,99,104,101,115,99,2,0,0,0,0,0, - 0,0,3,0,0,0,12,0,0,0,67,0,0,0,115,107, - 0,0,0,116,0,0,106,1,0,100,1,0,107,9,0,114, - 41,0,116,0,0,106,1,0,12,114,41,0,116,2,0,106, - 3,0,100,2,0,116,4,0,131,2,0,1,120,59,0,116, - 0,0,106,1,0,68,93,44,0,125,2,0,121,14,0,124, - 2,0,124,1,0,131,1,0,83,87,113,51,0,4,116,5, - 0,107,10,0,114,94,0,1,1,1,119,51,0,89,113,51, - 0,88,113,51,0,87,100,1,0,83,100,1,0,83,41,3, - 122,113,83,101,97,114,99,104,32,115,101,113,117,101,110,99, - 101,32,111,102,32,104,111,111,107,115,32,102,111,114,32,97, - 32,102,105,110,100,101,114,32,102,111,114,32,39,112,97,116, - 104,39,46,10,10,32,32,32,32,32,32,32,32,73,102,32, - 39,104,111,111,107,115,39,32,105,115,32,102,97,108,115,101, - 32,116,104,101,110,32,117,115,101,32,115,121,115,46,112,97, - 116,104,95,104,111,111,107,115,46,10,10,32,32,32,32,32, - 32,32,32,78,122,23,115,121,115,46,112,97,116,104,95,104, - 111,111,107,115,32,105,115,32,101,109,112,116,121,41,6,114, - 7,0,0,0,218,10,112,97,116,104,95,104,111,111,107,115, - 114,60,0,0,0,114,61,0,0,0,114,125,0,0,0,114, - 107,0,0,0,41,3,114,170,0,0,0,114,35,0,0,0, - 90,4,104,111,111,107,114,4,0,0,0,114,4,0,0,0, - 114,5,0,0,0,218,11,95,112,97,116,104,95,104,111,111, - 107,115,20,4,0,0,115,16,0,0,0,0,7,25,1,16, - 1,16,1,3,1,14,1,13,1,12,2,122,22,80,97,116, - 104,70,105,110,100,101,114,46,95,112,97,116,104,95,104,111, - 111,107,115,99,2,0,0,0,0,0,0,0,3,0,0,0, - 19,0,0,0,67,0,0,0,115,123,0,0,0,124,1,0, - 100,1,0,107,2,0,114,53,0,121,16,0,116,0,0,106, - 1,0,131,0,0,125,1,0,87,110,22,0,4,116,2,0, - 107,10,0,114,52,0,1,1,1,100,2,0,83,89,110,1, - 0,88,121,17,0,116,3,0,106,4,0,124,1,0,25,125, - 2,0,87,110,46,0,4,116,5,0,107,10,0,114,118,0, - 1,1,1,124,0,0,106,6,0,124,1,0,131,1,0,125, - 2,0,124,2,0,116,3,0,106,4,0,124,1,0,60,89, - 110,1,0,88,124,2,0,83,41,3,122,210,71,101,116,32, - 116,104,101,32,102,105,110,100,101,114,32,102,111,114,32,116, - 104,101,32,112,97,116,104,32,101,110,116,114,121,32,102,114, - 111,109,32,115,121,115,46,112,97,116,104,95,105,109,112,111, - 114,116,101,114,95,99,97,99,104,101,46,10,10,32,32,32, - 32,32,32,32,32,73,102,32,116,104,101,32,112,97,116,104, - 32,101,110,116,114,121,32,105,115,32,110,111,116,32,105,110, - 32,116,104,101,32,99,97,99,104,101,44,32,102,105,110,100, - 32,116,104,101,32,97,112,112,114,111,112,114,105,97,116,101, - 32,102,105,110,100,101,114,10,32,32,32,32,32,32,32,32, - 97,110,100,32,99,97,99,104,101,32,105,116,46,32,73,102, - 32,110,111,32,102,105,110,100,101,114,32,105,115,32,97,118, - 97,105,108,97,98,108,101,44,32,115,116,111,114,101,32,78, - 111,110,101,46,10,10,32,32,32,32,32,32,32,32,114,30, - 0,0,0,78,41,7,114,3,0,0,0,114,45,0,0,0, - 218,17,70,105,108,101,78,111,116,70,111,117,110,100,69,114, - 114,111,114,114,7,0,0,0,114,251,0,0,0,114,137,0, - 0,0,114,255,0,0,0,41,3,114,170,0,0,0,114,35, - 0,0,0,114,253,0,0,0,114,4,0,0,0,114,4,0, - 0,0,114,5,0,0,0,218,20,95,112,97,116,104,95,105, - 109,112,111,114,116,101,114,95,99,97,99,104,101,37,4,0, - 0,115,22,0,0,0,0,8,12,1,3,1,16,1,13,3, - 9,1,3,1,17,1,13,1,15,1,18,1,122,31,80,97, - 116,104,70,105,110,100,101,114,46,95,112,97,116,104,95,105, - 109,112,111,114,116,101,114,95,99,97,99,104,101,99,3,0, - 0,0,0,0,0,0,6,0,0,0,3,0,0,0,67,0, - 0,0,115,119,0,0,0,116,0,0,124,2,0,100,1,0, - 131,2,0,114,39,0,124,2,0,106,1,0,124,1,0,131, - 1,0,92,2,0,125,3,0,125,4,0,110,21,0,124,2, - 0,106,2,0,124,1,0,131,1,0,125,3,0,103,0,0, - 125,4,0,124,3,0,100,0,0,107,9,0,114,88,0,116, - 3,0,106,4,0,124,1,0,124,3,0,131,2,0,83,116, - 3,0,106,5,0,124,1,0,100,0,0,131,2,0,125,5, - 0,124,4,0,124,5,0,95,6,0,124,5,0,83,41,2, - 78,114,124,0,0,0,41,7,114,115,0,0,0,114,124,0, - 0,0,114,182,0,0,0,114,121,0,0,0,114,179,0,0, - 0,114,160,0,0,0,114,156,0,0,0,41,6,114,170,0, - 0,0,114,126,0,0,0,114,253,0,0,0,114,127,0,0, - 0,114,128,0,0,0,114,164,0,0,0,114,4,0,0,0, - 114,4,0,0,0,114,5,0,0,0,218,16,95,108,101,103, - 97,99,121,95,103,101,116,95,115,112,101,99,59,4,0,0, - 115,18,0,0,0,0,4,15,1,24,2,15,1,6,1,12, - 1,16,1,18,1,9,1,122,27,80,97,116,104,70,105,110, - 100,101,114,46,95,108,101,103,97,99,121,95,103,101,116,95, - 115,112,101,99,78,99,4,0,0,0,0,0,0,0,9,0, - 0,0,5,0,0,0,67,0,0,0,115,243,0,0,0,103, - 0,0,125,4,0,120,230,0,124,2,0,68,93,191,0,125, - 5,0,116,0,0,124,5,0,116,1,0,116,2,0,102,2, - 0,131,2,0,115,43,0,113,13,0,124,0,0,106,3,0, - 124,5,0,131,1,0,125,6,0,124,6,0,100,1,0,107, - 9,0,114,13,0,116,4,0,124,6,0,100,2,0,131,2, - 0,114,106,0,124,6,0,106,5,0,124,1,0,124,3,0, - 131,2,0,125,7,0,110,18,0,124,0,0,106,6,0,124, - 1,0,124,6,0,131,2,0,125,7,0,124,7,0,100,1, - 0,107,8,0,114,139,0,113,13,0,124,7,0,106,7,0, - 100,1,0,107,9,0,114,158,0,124,7,0,83,124,7,0, - 106,8,0,125,8,0,124,8,0,100,1,0,107,8,0,114, - 191,0,116,9,0,100,3,0,131,1,0,130,1,0,124,4, - 0,106,10,0,124,8,0,131,1,0,1,113,13,0,87,116, - 11,0,106,12,0,124,1,0,100,1,0,131,2,0,125,7, - 0,124,4,0,124,7,0,95,8,0,124,7,0,83,100,1, - 0,83,41,4,122,63,70,105,110,100,32,116,104,101,32,108, - 111,97,100,101,114,32,111,114,32,110,97,109,101,115,112,97, - 99,101,95,112,97,116,104,32,102,111,114,32,116,104,105,115, - 32,109,111,100,117,108,101,47,112,97,99,107,97,103,101,32, - 110,97,109,101,46,78,114,181,0,0,0,122,19,115,112,101, - 99,32,109,105,115,115,105,110,103,32,108,111,97,100,101,114, - 41,13,114,143,0,0,0,114,69,0,0,0,218,5,98,121, - 116,101,115,114,1,1,0,0,114,115,0,0,0,114,181,0, - 0,0,114,2,1,0,0,114,127,0,0,0,114,156,0,0, - 0,114,107,0,0,0,114,149,0,0,0,114,121,0,0,0, - 114,160,0,0,0,41,9,114,170,0,0,0,114,126,0,0, - 0,114,35,0,0,0,114,180,0,0,0,218,14,110,97,109, - 101,115,112,97,99,101,95,112,97,116,104,90,5,101,110,116, - 114,121,114,253,0,0,0,114,164,0,0,0,114,128,0,0, - 0,114,4,0,0,0,114,4,0,0,0,114,5,0,0,0, - 218,9,95,103,101,116,95,115,112,101,99,74,4,0,0,115, - 40,0,0,0,0,5,6,1,13,1,21,1,3,1,15,1, - 12,1,15,1,21,2,18,1,12,1,3,1,15,1,4,1, - 9,1,12,1,12,5,17,2,18,1,9,1,122,20,80,97, - 116,104,70,105,110,100,101,114,46,95,103,101,116,95,115,112, - 101,99,99,4,0,0,0,0,0,0,0,6,0,0,0,4, - 0,0,0,67,0,0,0,115,140,0,0,0,124,2,0,100, - 1,0,107,8,0,114,21,0,116,0,0,106,1,0,125,2, - 0,124,0,0,106,2,0,124,1,0,124,2,0,124,3,0, - 131,3,0,125,4,0,124,4,0,100,1,0,107,8,0,114, - 58,0,100,1,0,83,124,4,0,106,3,0,100,1,0,107, - 8,0,114,132,0,124,4,0,106,4,0,125,5,0,124,5, - 0,114,125,0,100,2,0,124,4,0,95,5,0,116,6,0, - 124,1,0,124,5,0,124,0,0,106,2,0,131,3,0,124, - 4,0,95,4,0,124,4,0,83,100,1,0,83,110,4,0, - 124,4,0,83,100,1,0,83,41,3,122,98,102,105,110,100, - 32,116,104,101,32,109,111,100,117,108,101,32,111,110,32,115, - 121,115,46,112,97,116,104,32,111,114,32,39,112,97,116,104, - 39,32,98,97,115,101,100,32,111,110,32,115,121,115,46,112, - 97,116,104,95,104,111,111,107,115,32,97,110,100,10,32,32, - 32,32,32,32,32,32,115,121,115,46,112,97,116,104,95,105, - 109,112,111,114,116,101,114,95,99,97,99,104,101,46,78,90, - 9,110,97,109,101,115,112,97,99,101,41,7,114,7,0,0, - 0,114,35,0,0,0,114,5,1,0,0,114,127,0,0,0, - 114,156,0,0,0,114,158,0,0,0,114,230,0,0,0,41, - 6,114,170,0,0,0,114,126,0,0,0,114,35,0,0,0, - 114,180,0,0,0,114,164,0,0,0,114,4,1,0,0,114, - 4,0,0,0,114,4,0,0,0,114,5,0,0,0,114,181, - 0,0,0,106,4,0,0,115,26,0,0,0,0,4,12,1, - 9,1,21,1,12,1,4,1,15,1,9,1,6,3,9,1, - 24,1,4,2,7,2,122,20,80,97,116,104,70,105,110,100, - 101,114,46,102,105,110,100,95,115,112,101,99,99,3,0,0, - 0,0,0,0,0,4,0,0,0,3,0,0,0,67,0,0, - 0,115,41,0,0,0,124,0,0,106,0,0,124,1,0,124, - 2,0,131,2,0,125,3,0,124,3,0,100,1,0,107,8, - 0,114,34,0,100,1,0,83,124,3,0,106,1,0,83,41, - 2,122,170,102,105,110,100,32,116,104,101,32,109,111,100,117, - 108,101,32,111,110,32,115,121,115,46,112,97,116,104,32,111, - 114,32,39,112,97,116,104,39,32,98,97,115,101,100,32,111, - 110,32,115,121,115,46,112,97,116,104,95,104,111,111,107,115, - 32,97,110,100,10,32,32,32,32,32,32,32,32,115,121,115, - 46,112,97,116,104,95,105,109,112,111,114,116,101,114,95,99, - 97,99,104,101,46,10,10,32,32,32,32,32,32,32,32,84, - 104,105,115,32,109,101,116,104,111,100,32,105,115,32,100,101, - 112,114,101,99,97,116,101,100,46,32,32,85,115,101,32,102, - 105,110,100,95,115,112,101,99,40,41,32,105,110,115,116,101, - 97,100,46,10,10,32,32,32,32,32,32,32,32,78,41,2, - 114,181,0,0,0,114,127,0,0,0,41,4,114,170,0,0, - 0,114,126,0,0,0,114,35,0,0,0,114,164,0,0,0, - 114,4,0,0,0,114,4,0,0,0,114,5,0,0,0,114, - 182,0,0,0,128,4,0,0,115,8,0,0,0,0,8,18, - 1,12,1,4,1,122,22,80,97,116,104,70,105,110,100,101, - 114,46,102,105,110,100,95,109,111,100,117,108,101,41,12,114, - 112,0,0,0,114,111,0,0,0,114,113,0,0,0,114,114, - 0,0,0,114,183,0,0,0,114,250,0,0,0,114,255,0, - 0,0,114,1,1,0,0,114,2,1,0,0,114,5,1,0, - 0,114,181,0,0,0,114,182,0,0,0,114,4,0,0,0, - 114,4,0,0,0,114,4,0,0,0,114,5,0,0,0,114, - 249,0,0,0,8,4,0,0,115,22,0,0,0,12,2,6, - 2,18,8,18,17,18,22,18,15,3,1,18,31,3,1,21, - 21,3,1,114,249,0,0,0,99,0,0,0,0,0,0,0, - 0,0,0,0,0,3,0,0,0,64,0,0,0,115,133,0, - 0,0,101,0,0,90,1,0,100,0,0,90,2,0,100,1, - 0,90,3,0,100,2,0,100,3,0,132,0,0,90,4,0, - 100,4,0,100,5,0,132,0,0,90,5,0,101,6,0,90, - 7,0,100,6,0,100,7,0,132,0,0,90,8,0,100,8, - 0,100,9,0,132,0,0,90,9,0,100,10,0,100,11,0, - 100,12,0,132,1,0,90,10,0,100,13,0,100,14,0,132, - 0,0,90,11,0,101,12,0,100,15,0,100,16,0,132,0, - 0,131,1,0,90,13,0,100,17,0,100,18,0,132,0,0, - 90,14,0,100,10,0,83,41,19,218,10,70,105,108,101,70, - 105,110,100,101,114,122,172,70,105,108,101,45,98,97,115,101, - 100,32,102,105,110,100,101,114,46,10,10,32,32,32,32,73, - 110,116,101,114,97,99,116,105,111,110,115,32,119,105,116,104, - 32,116,104,101,32,102,105,108,101,32,115,121,115,116,101,109, - 32,97,114,101,32,99,97,99,104,101,100,32,102,111,114,32, - 112,101,114,102,111,114,109,97,110,99,101,44,32,98,101,105, - 110,103,10,32,32,32,32,114,101,102,114,101,115,104,101,100, - 32,119,104,101,110,32,116,104,101,32,100,105,114,101,99,116, - 111,114,121,32,116,104,101,32,102,105,110,100,101,114,32,105, - 115,32,104,97,110,100,108,105,110,103,32,104,97,115,32,98, - 101,101,110,32,109,111,100,105,102,105,101,100,46,10,10,32, - 32,32,32,99,2,0,0,0,0,0,0,0,5,0,0,0, - 5,0,0,0,7,0,0,0,115,122,0,0,0,103,0,0, - 125,3,0,120,52,0,124,2,0,68,93,44,0,92,2,0, - 137,0,0,125,4,0,124,3,0,106,0,0,135,0,0,102, - 1,0,100,1,0,100,2,0,134,0,0,124,4,0,68,131, - 1,0,131,1,0,1,113,13,0,87,124,3,0,124,0,0, - 95,1,0,124,1,0,112,79,0,100,3,0,124,0,0,95, - 2,0,100,6,0,124,0,0,95,3,0,116,4,0,131,0, - 0,124,0,0,95,5,0,116,4,0,131,0,0,124,0,0, - 95,6,0,100,5,0,83,41,7,122,154,73,110,105,116,105, - 97,108,105,122,101,32,119,105,116,104,32,116,104,101,32,112, - 97,116,104,32,116,111,32,115,101,97,114,99,104,32,111,110, - 32,97,110,100,32,97,32,118,97,114,105,97,98,108,101,32, - 110,117,109,98,101,114,32,111,102,10,32,32,32,32,32,32, - 32,32,50,45,116,117,112,108,101,115,32,99,111,110,116,97, - 105,110,105,110,103,32,116,104,101,32,108,111,97,100,101,114, - 32,97,110,100,32,116,104,101,32,102,105,108,101,32,115,117, - 102,102,105,120,101,115,32,116,104,101,32,108,111,97,100,101, - 114,10,32,32,32,32,32,32,32,32,114,101,99,111,103,110, - 105,122,101,115,46,99,1,0,0,0,0,0,0,0,2,0, - 0,0,3,0,0,0,51,0,0,0,115,27,0,0,0,124, - 0,0,93,17,0,125,1,0,124,1,0,136,0,0,102,2, - 0,86,1,113,3,0,100,0,0,83,41,1,78,114,4,0, - 0,0,41,2,114,22,0,0,0,114,225,0,0,0,41,1, - 114,127,0,0,0,114,4,0,0,0,114,5,0,0,0,114, - 227,0,0,0,157,4,0,0,115,2,0,0,0,6,0,122, - 38,70,105,108,101,70,105,110,100,101,114,46,95,95,105,110, - 105,116,95,95,46,60,108,111,99,97,108,115,62,46,60,103, - 101,110,101,120,112,114,62,114,58,0,0,0,114,29,0,0, - 0,78,114,87,0,0,0,41,7,114,149,0,0,0,218,8, - 95,108,111,97,100,101,114,115,114,35,0,0,0,218,11,95, - 112,97,116,104,95,109,116,105,109,101,218,3,115,101,116,218, - 11,95,112,97,116,104,95,99,97,99,104,101,218,19,95,114, - 101,108,97,120,101,100,95,112,97,116,104,95,99,97,99,104, - 101,41,5,114,108,0,0,0,114,35,0,0,0,218,14,108, - 111,97,100,101,114,95,100,101,116,97,105,108,115,90,7,108, - 111,97,100,101,114,115,114,166,0,0,0,114,4,0,0,0, - 41,1,114,127,0,0,0,114,5,0,0,0,114,185,0,0, - 0,151,4,0,0,115,16,0,0,0,0,4,6,1,19,1, - 36,1,9,2,15,1,9,1,12,1,122,19,70,105,108,101, - 70,105,110,100,101,114,46,95,95,105,110,105,116,95,95,99, - 1,0,0,0,0,0,0,0,1,0,0,0,2,0,0,0, - 67,0,0,0,115,13,0,0,0,100,3,0,124,0,0,95, - 0,0,100,2,0,83,41,4,122,31,73,110,118,97,108,105, - 100,97,116,101,32,116,104,101,32,100,105,114,101,99,116,111, - 114,121,32,109,116,105,109,101,46,114,29,0,0,0,78,114, - 87,0,0,0,41,1,114,8,1,0,0,41,1,114,108,0, - 0,0,114,4,0,0,0,114,4,0,0,0,114,5,0,0, - 0,114,250,0,0,0,165,4,0,0,115,2,0,0,0,0, - 2,122,28,70,105,108,101,70,105,110,100,101,114,46,105,110, - 118,97,108,105,100,97,116,101,95,99,97,99,104,101,115,99, - 2,0,0,0,0,0,0,0,3,0,0,0,2,0,0,0, - 67,0,0,0,115,59,0,0,0,124,0,0,106,0,0,124, - 1,0,131,1,0,125,2,0,124,2,0,100,1,0,107,8, - 0,114,37,0,100,1,0,103,0,0,102,2,0,83,124,2, - 0,106,1,0,124,2,0,106,2,0,112,55,0,103,0,0, - 102,2,0,83,41,2,122,197,84,114,121,32,116,111,32,102, - 105,110,100,32,97,32,108,111,97,100,101,114,32,102,111,114, - 32,116,104,101,32,115,112,101,99,105,102,105,101,100,32,109, - 111,100,117,108,101,44,32,111,114,32,116,104,101,32,110,97, - 109,101,115,112,97,99,101,10,32,32,32,32,32,32,32,32, - 112,97,99,107,97,103,101,32,112,111,114,116,105,111,110,115, - 46,32,82,101,116,117,114,110,115,32,40,108,111,97,100,101, - 114,44,32,108,105,115,116,45,111,102,45,112,111,114,116,105, - 111,110,115,41,46,10,10,32,32,32,32,32,32,32,32,84, - 104,105,115,32,109,101,116,104,111,100,32,105,115,32,100,101, - 112,114,101,99,97,116,101,100,46,32,32,85,115,101,32,102, - 105,110,100,95,115,112,101,99,40,41,32,105,110,115,116,101, - 97,100,46,10,10,32,32,32,32,32,32,32,32,78,41,3, - 114,181,0,0,0,114,127,0,0,0,114,156,0,0,0,41, - 3,114,108,0,0,0,114,126,0,0,0,114,164,0,0,0, - 114,4,0,0,0,114,4,0,0,0,114,5,0,0,0,114, - 124,0,0,0,171,4,0,0,115,8,0,0,0,0,7,15, - 1,12,1,10,1,122,22,70,105,108,101,70,105,110,100,101, - 114,46,102,105,110,100,95,108,111,97,100,101,114,99,6,0, - 0,0,0,0,0,0,7,0,0,0,7,0,0,0,67,0, - 0,0,115,40,0,0,0,124,1,0,124,2,0,124,3,0, - 131,2,0,125,6,0,116,0,0,124,2,0,124,3,0,100, - 1,0,124,6,0,100,2,0,124,4,0,131,2,2,83,41, - 3,78,114,127,0,0,0,114,156,0,0,0,41,1,114,167, - 0,0,0,41,7,114,108,0,0,0,114,165,0,0,0,114, - 126,0,0,0,114,35,0,0,0,90,4,115,109,115,108,114, - 180,0,0,0,114,127,0,0,0,114,4,0,0,0,114,4, - 0,0,0,114,5,0,0,0,114,5,1,0,0,183,4,0, - 0,115,6,0,0,0,0,1,15,1,18,1,122,20,70,105, - 108,101,70,105,110,100,101,114,46,95,103,101,116,95,115,112, - 101,99,78,99,3,0,0,0,0,0,0,0,14,0,0,0, - 15,0,0,0,67,0,0,0,115,234,1,0,0,100,1,0, - 125,3,0,124,1,0,106,0,0,100,2,0,131,1,0,100, - 3,0,25,125,4,0,121,34,0,116,1,0,124,0,0,106, - 2,0,112,49,0,116,3,0,106,4,0,131,0,0,131,1, - 0,106,5,0,125,5,0,87,110,24,0,4,116,6,0,107, - 10,0,114,85,0,1,1,1,100,10,0,125,5,0,89,110, - 1,0,88,124,5,0,124,0,0,106,7,0,107,3,0,114, - 120,0,124,0,0,106,8,0,131,0,0,1,124,5,0,124, - 0,0,95,7,0,116,9,0,131,0,0,114,153,0,124,0, - 0,106,10,0,125,6,0,124,4,0,106,11,0,131,0,0, - 125,7,0,110,15,0,124,0,0,106,12,0,125,6,0,124, - 4,0,125,7,0,124,7,0,124,6,0,107,6,0,114,45, - 1,116,13,0,124,0,0,106,2,0,124,4,0,131,2,0, - 125,8,0,120,100,0,124,0,0,106,14,0,68,93,77,0, - 92,2,0,125,9,0,125,10,0,100,5,0,124,9,0,23, - 125,11,0,116,13,0,124,8,0,124,11,0,131,2,0,125, - 12,0,116,15,0,124,12,0,131,1,0,114,208,0,124,0, - 0,106,16,0,124,10,0,124,1,0,124,12,0,124,8,0, - 103,1,0,124,2,0,131,5,0,83,113,208,0,87,116,17, - 0,124,8,0,131,1,0,125,3,0,120,123,0,124,0,0, - 106,14,0,68,93,112,0,92,2,0,125,9,0,125,10,0, - 116,13,0,124,0,0,106,2,0,124,4,0,124,9,0,23, - 131,2,0,125,12,0,116,18,0,100,6,0,106,19,0,124, - 12,0,131,1,0,100,7,0,100,3,0,131,1,1,1,124, - 7,0,124,9,0,23,124,6,0,107,6,0,114,55,1,116, - 15,0,124,12,0,131,1,0,114,55,1,124,0,0,106,16, - 0,124,10,0,124,1,0,124,12,0,100,8,0,124,2,0, - 131,5,0,83,113,55,1,87,124,3,0,114,230,1,116,18, - 0,100,9,0,106,19,0,124,8,0,131,1,0,131,1,0, - 1,116,20,0,106,21,0,124,1,0,100,8,0,131,2,0, - 125,13,0,124,8,0,103,1,0,124,13,0,95,22,0,124, - 13,0,83,100,8,0,83,41,11,122,125,84,114,121,32,116, - 111,32,102,105,110,100,32,97,32,108,111,97,100,101,114,32, - 102,111,114,32,116,104,101,32,115,112,101,99,105,102,105,101, - 100,32,109,111,100,117,108,101,44,32,111,114,32,116,104,101, - 32,110,97,109,101,115,112,97,99,101,10,32,32,32,32,32, - 32,32,32,112,97,99,107,97,103,101,32,112,111,114,116,105, - 111,110,115,46,32,82,101,116,117,114,110,115,32,40,108,111, - 97,100,101,114,44,32,108,105,115,116,45,111,102,45,112,111, - 114,116,105,111,110,115,41,46,70,114,58,0,0,0,114,56, - 0,0,0,114,29,0,0,0,114,185,0,0,0,122,9,116, - 114,121,105,110,103,32,123,125,114,98,0,0,0,78,122,25, - 112,111,115,115,105,98,108,101,32,110,97,109,101,115,112,97, - 99,101,32,102,111,114,32,123,125,114,87,0,0,0,41,23, - 114,32,0,0,0,114,39,0,0,0,114,35,0,0,0,114, - 3,0,0,0,114,45,0,0,0,114,219,0,0,0,114,40, - 0,0,0,114,8,1,0,0,218,11,95,102,105,108,108,95, - 99,97,99,104,101,114,6,0,0,0,114,11,1,0,0,114, - 88,0,0,0,114,10,1,0,0,114,28,0,0,0,114,7, - 1,0,0,114,44,0,0,0,114,5,1,0,0,114,46,0, - 0,0,114,105,0,0,0,114,47,0,0,0,114,121,0,0, - 0,114,160,0,0,0,114,156,0,0,0,41,14,114,108,0, - 0,0,114,126,0,0,0,114,180,0,0,0,90,12,105,115, - 95,110,97,109,101,115,112,97,99,101,90,11,116,97,105,108, - 95,109,111,100,117,108,101,114,133,0,0,0,90,5,99,97, - 99,104,101,90,12,99,97,99,104,101,95,109,111,100,117,108, - 101,90,9,98,97,115,101,95,112,97,116,104,114,225,0,0, - 0,114,165,0,0,0,90,13,105,110,105,116,95,102,105,108, - 101,110,97,109,101,90,9,102,117,108,108,95,112,97,116,104, - 114,164,0,0,0,114,4,0,0,0,114,4,0,0,0,114, - 5,0,0,0,114,181,0,0,0,188,4,0,0,115,68,0, - 0,0,0,3,6,1,19,1,3,1,34,1,13,1,11,1, - 15,1,10,1,9,2,9,1,9,1,15,2,9,1,6,2, - 12,1,18,1,22,1,10,1,15,1,12,1,32,4,12,2, - 22,1,22,1,25,1,16,1,12,1,29,1,6,1,19,1, - 18,1,12,1,4,1,122,20,70,105,108,101,70,105,110,100, - 101,114,46,102,105,110,100,95,115,112,101,99,99,1,0,0, - 0,0,0,0,0,9,0,0,0,13,0,0,0,67,0,0, - 0,115,11,1,0,0,124,0,0,106,0,0,125,1,0,121, - 31,0,116,1,0,106,2,0,124,1,0,112,33,0,116,1, - 0,106,3,0,131,0,0,131,1,0,125,2,0,87,110,33, - 0,4,116,4,0,116,5,0,116,6,0,102,3,0,107,10, - 0,114,75,0,1,1,1,103,0,0,125,2,0,89,110,1, - 0,88,116,7,0,106,8,0,106,9,0,100,1,0,131,1, - 0,115,112,0,116,10,0,124,2,0,131,1,0,124,0,0, - 95,11,0,110,111,0,116,10,0,131,0,0,125,3,0,120, - 90,0,124,2,0,68,93,82,0,125,4,0,124,4,0,106, - 12,0,100,2,0,131,1,0,92,3,0,125,5,0,125,6, - 0,125,7,0,124,6,0,114,191,0,100,3,0,106,13,0, - 124,5,0,124,7,0,106,14,0,131,0,0,131,2,0,125, - 8,0,110,6,0,124,5,0,125,8,0,124,3,0,106,15, - 0,124,8,0,131,1,0,1,113,128,0,87,124,3,0,124, - 0,0,95,11,0,116,7,0,106,8,0,106,9,0,116,16, - 0,131,1,0,114,7,1,100,4,0,100,5,0,132,0,0, - 124,2,0,68,131,1,0,124,0,0,95,17,0,100,6,0, - 83,41,7,122,68,70,105,108,108,32,116,104,101,32,99,97, - 99,104,101,32,111,102,32,112,111,116,101,110,116,105,97,108, - 32,109,111,100,117,108,101,115,32,97,110,100,32,112,97,99, - 107,97,103,101,115,32,102,111,114,32,116,104,105,115,32,100, - 105,114,101,99,116,111,114,121,46,114,0,0,0,0,114,58, - 0,0,0,122,5,123,125,46,123,125,99,1,0,0,0,0, - 0,0,0,2,0,0,0,3,0,0,0,83,0,0,0,115, - 28,0,0,0,104,0,0,124,0,0,93,18,0,125,1,0, - 124,1,0,106,0,0,131,0,0,146,2,0,113,6,0,83, - 114,4,0,0,0,41,1,114,88,0,0,0,41,2,114,22, - 0,0,0,90,2,102,110,114,4,0,0,0,114,4,0,0, - 0,114,5,0,0,0,250,9,60,115,101,116,99,111,109,112, - 62,6,5,0,0,115,2,0,0,0,9,0,122,41,70,105, - 108,101,70,105,110,100,101,114,46,95,102,105,108,108,95,99, - 97,99,104,101,46,60,108,111,99,97,108,115,62,46,60,115, - 101,116,99,111,109,112,62,78,41,18,114,35,0,0,0,114, - 3,0,0,0,90,7,108,105,115,116,100,105,114,114,45,0, - 0,0,114,0,1,0,0,218,15,80,101,114,109,105,115,115, - 105,111,110,69,114,114,111,114,218,18,78,111,116,65,68,105, - 114,101,99,116,111,114,121,69,114,114,111,114,114,7,0,0, - 0,114,8,0,0,0,114,9,0,0,0,114,9,1,0,0, - 114,10,1,0,0,114,83,0,0,0,114,47,0,0,0,114, - 88,0,0,0,218,3,97,100,100,114,10,0,0,0,114,11, - 1,0,0,41,9,114,108,0,0,0,114,35,0,0,0,90, - 8,99,111,110,116,101,110,116,115,90,21,108,111,119,101,114, - 95,115,117,102,102,105,120,95,99,111,110,116,101,110,116,115, - 114,245,0,0,0,114,106,0,0,0,114,237,0,0,0,114, - 225,0,0,0,90,8,110,101,119,95,110,97,109,101,114,4, - 0,0,0,114,4,0,0,0,114,5,0,0,0,114,13,1, - 0,0,233,4,0,0,115,34,0,0,0,0,2,9,1,3, - 1,31,1,22,3,11,3,18,1,18,7,9,1,13,1,24, - 1,6,1,27,2,6,1,17,1,9,1,18,1,122,22,70, - 105,108,101,70,105,110,100,101,114,46,95,102,105,108,108,95, - 99,97,99,104,101,99,1,0,0,0,0,0,0,0,3,0, - 0,0,3,0,0,0,7,0,0,0,115,25,0,0,0,135, - 0,0,135,1,0,102,2,0,100,1,0,100,2,0,134,0, - 0,125,2,0,124,2,0,83,41,3,97,20,1,0,0,65, - 32,99,108,97,115,115,32,109,101,116,104,111,100,32,119,104, - 105,99,104,32,114,101,116,117,114,110,115,32,97,32,99,108, - 111,115,117,114,101,32,116,111,32,117,115,101,32,111,110,32, - 115,121,115,46,112,97,116,104,95,104,111,111,107,10,32,32, - 32,32,32,32,32,32,119,104,105,99,104,32,119,105,108,108, - 32,114,101,116,117,114,110,32,97,110,32,105,110,115,116,97, - 110,99,101,32,117,115,105,110,103,32,116,104,101,32,115,112, - 101,99,105,102,105,101,100,32,108,111,97,100,101,114,115,32, - 97,110,100,32,116,104,101,32,112,97,116,104,10,32,32,32, - 32,32,32,32,32,99,97,108,108,101,100,32,111,110,32,116, - 104,101,32,99,108,111,115,117,114,101,46,10,10,32,32,32, - 32,32,32,32,32,73,102,32,116,104,101,32,112,97,116,104, - 32,99,97,108,108,101,100,32,111,110,32,116,104,101,32,99, - 108,111,115,117,114,101,32,105,115,32,110,111,116,32,97,32, - 100,105,114,101,99,116,111,114,121,44,32,73,109,112,111,114, - 116,69,114,114,111,114,32,105,115,10,32,32,32,32,32,32, - 32,32,114,97,105,115,101,100,46,10,10,32,32,32,32,32, - 32,32,32,99,1,0,0,0,0,0,0,0,1,0,0,0, - 4,0,0,0,19,0,0,0,115,43,0,0,0,116,0,0, - 124,0,0,131,1,0,115,30,0,116,1,0,100,1,0,100, - 2,0,124,0,0,131,1,1,130,1,0,136,0,0,124,0, - 0,136,1,0,140,1,0,83,41,3,122,45,80,97,116,104, - 32,104,111,111,107,32,102,111,114,32,105,109,112,111,114,116, - 108,105,98,46,109,97,99,104,105,110,101,114,121,46,70,105, - 108,101,70,105,110,100,101,114,46,122,30,111,110,108,121,32, - 100,105,114,101,99,116,111,114,105,101,115,32,97,114,101,32, - 115,117,112,112,111,114,116,101,100,114,35,0,0,0,41,2, - 114,46,0,0,0,114,107,0,0,0,41,1,114,35,0,0, - 0,41,2,114,170,0,0,0,114,12,1,0,0,114,4,0, - 0,0,114,5,0,0,0,218,24,112,97,116,104,95,104,111, - 111,107,95,102,111,114,95,70,105,108,101,70,105,110,100,101, - 114,18,5,0,0,115,6,0,0,0,0,2,12,1,18,1, - 122,54,70,105,108,101,70,105,110,100,101,114,46,112,97,116, - 104,95,104,111,111,107,46,60,108,111,99,97,108,115,62,46, - 112,97,116,104,95,104,111,111,107,95,102,111,114,95,70,105, - 108,101,70,105,110,100,101,114,114,4,0,0,0,41,3,114, - 170,0,0,0,114,12,1,0,0,114,18,1,0,0,114,4, - 0,0,0,41,2,114,170,0,0,0,114,12,1,0,0,114, - 5,0,0,0,218,9,112,97,116,104,95,104,111,111,107,8, - 5,0,0,115,4,0,0,0,0,10,21,6,122,20,70,105, - 108,101,70,105,110,100,101,114,46,112,97,116,104,95,104,111, - 111,107,99,1,0,0,0,0,0,0,0,1,0,0,0,2, - 0,0,0,67,0,0,0,115,16,0,0,0,100,1,0,106, - 0,0,124,0,0,106,1,0,131,1,0,83,41,2,78,122, - 16,70,105,108,101,70,105,110,100,101,114,40,123,33,114,125, - 41,41,2,114,47,0,0,0,114,35,0,0,0,41,1,114, - 108,0,0,0,114,4,0,0,0,114,4,0,0,0,114,5, - 0,0,0,114,244,0,0,0,26,5,0,0,115,2,0,0, - 0,0,1,122,19,70,105,108,101,70,105,110,100,101,114,46, - 95,95,114,101,112,114,95,95,41,15,114,112,0,0,0,114, - 111,0,0,0,114,113,0,0,0,114,114,0,0,0,114,185, - 0,0,0,114,250,0,0,0,114,130,0,0,0,114,182,0, - 0,0,114,124,0,0,0,114,5,1,0,0,114,181,0,0, - 0,114,13,1,0,0,114,183,0,0,0,114,19,1,0,0, - 114,244,0,0,0,114,4,0,0,0,114,4,0,0,0,114, - 4,0,0,0,114,5,0,0,0,114,6,1,0,0,142,4, - 0,0,115,20,0,0,0,12,7,6,2,12,14,12,4,6, - 2,12,12,12,5,15,45,12,31,18,18,114,6,1,0,0, - 99,4,0,0,0,0,0,0,0,6,0,0,0,11,0,0, - 0,67,0,0,0,115,195,0,0,0,124,0,0,106,0,0, - 100,1,0,131,1,0,125,4,0,124,0,0,106,0,0,100, - 2,0,131,1,0,125,5,0,124,4,0,115,99,0,124,5, - 0,114,54,0,124,5,0,106,1,0,125,4,0,110,45,0, - 124,2,0,124,3,0,107,2,0,114,84,0,116,2,0,124, - 1,0,124,2,0,131,2,0,125,4,0,110,15,0,116,3, - 0,124,1,0,124,2,0,131,2,0,125,4,0,124,5,0, - 115,126,0,116,4,0,124,1,0,124,2,0,100,3,0,124, - 4,0,131,2,1,125,5,0,121,44,0,124,5,0,124,0, - 0,100,2,0,60,124,4,0,124,0,0,100,1,0,60,124, - 2,0,124,0,0,100,4,0,60,124,3,0,124,0,0,100, - 5,0,60,87,110,18,0,4,116,5,0,107,10,0,114,190, - 0,1,1,1,89,110,1,0,88,100,0,0,83,41,6,78, - 218,10,95,95,108,111,97,100,101,114,95,95,218,8,95,95, - 115,112,101,99,95,95,114,127,0,0,0,90,8,95,95,102, - 105,108,101,95,95,90,10,95,95,99,97,99,104,101,100,95, - 95,41,6,218,3,103,101,116,114,127,0,0,0,114,223,0, - 0,0,114,218,0,0,0,114,167,0,0,0,218,9,69,120, - 99,101,112,116,105,111,110,41,6,90,2,110,115,114,106,0, - 0,0,90,8,112,97,116,104,110,97,109,101,90,9,99,112, - 97,116,104,110,97,109,101,114,127,0,0,0,114,164,0,0, - 0,114,4,0,0,0,114,4,0,0,0,114,5,0,0,0, - 218,14,95,102,105,120,95,117,112,95,109,111,100,117,108,101, - 32,5,0,0,115,34,0,0,0,0,2,15,1,15,1,6, - 1,6,1,12,1,12,1,18,2,15,1,6,1,21,1,3, - 1,10,1,10,1,10,1,14,1,13,2,114,24,1,0,0, - 99,0,0,0,0,0,0,0,0,3,0,0,0,3,0,0, - 0,67,0,0,0,115,55,0,0,0,116,0,0,116,1,0, - 106,2,0,131,0,0,102,2,0,125,0,0,116,3,0,116, - 4,0,102,2,0,125,1,0,116,5,0,116,6,0,102,2, - 0,125,2,0,124,0,0,124,1,0,124,2,0,103,3,0, - 83,41,1,122,95,82,101,116,117,114,110,115,32,97,32,108, - 105,115,116,32,111,102,32,102,105,108,101,45,98,97,115,101, - 100,32,109,111,100,117,108,101,32,108,111,97,100,101,114,115, - 46,10,10,32,32,32,32,69,97,99,104,32,105,116,101,109, - 32,105,115,32,97,32,116,117,112,108,101,32,40,108,111,97, - 100,101,114,44,32,115,117,102,102,105,120,101,115,41,46,10, - 32,32,32,32,41,7,114,224,0,0,0,114,145,0,0,0, - 218,18,101,120,116,101,110,115,105,111,110,95,115,117,102,102, - 105,120,101,115,114,218,0,0,0,114,84,0,0,0,114,223, - 0,0,0,114,74,0,0,0,41,3,90,10,101,120,116,101, - 110,115,105,111,110,115,90,6,115,111,117,114,99,101,90,8, - 98,121,116,101,99,111,100,101,114,4,0,0,0,114,4,0, - 0,0,114,5,0,0,0,114,161,0,0,0,55,5,0,0, - 115,8,0,0,0,0,5,18,1,12,1,12,1,114,161,0, - 0,0,99,1,0,0,0,0,0,0,0,12,0,0,0,12, - 0,0,0,67,0,0,0,115,70,2,0,0,124,0,0,97, - 0,0,116,0,0,106,1,0,97,1,0,116,0,0,106,2, - 0,97,2,0,116,1,0,106,3,0,116,4,0,25,125,1, - 0,120,76,0,100,26,0,68,93,68,0,125,2,0,124,2, - 0,116,1,0,106,3,0,107,7,0,114,83,0,116,0,0, - 106,5,0,124,2,0,131,1,0,125,3,0,110,13,0,116, - 1,0,106,3,0,124,2,0,25,125,3,0,116,6,0,124, - 1,0,124,2,0,124,3,0,131,3,0,1,113,44,0,87, - 100,5,0,100,6,0,103,1,0,102,2,0,100,7,0,100, - 8,0,100,6,0,103,2,0,102,2,0,102,2,0,125,4, - 0,120,149,0,124,4,0,68,93,129,0,92,2,0,125,5, - 0,125,6,0,116,7,0,100,9,0,100,10,0,132,0,0, - 124,6,0,68,131,1,0,131,1,0,115,199,0,116,8,0, - 130,1,0,124,6,0,100,11,0,25,125,7,0,124,5,0, - 116,1,0,106,3,0,107,6,0,114,241,0,116,1,0,106, - 3,0,124,5,0,25,125,8,0,80,113,156,0,121,20,0, - 116,0,0,106,5,0,124,5,0,131,1,0,125,8,0,80, - 87,113,156,0,4,116,9,0,107,10,0,114,28,1,1,1, - 1,119,156,0,89,113,156,0,88,113,156,0,87,116,9,0, - 100,12,0,131,1,0,130,1,0,116,6,0,124,1,0,100, - 13,0,124,8,0,131,3,0,1,116,6,0,124,1,0,100, - 14,0,124,7,0,131,3,0,1,116,6,0,124,1,0,100, - 15,0,100,16,0,106,10,0,124,6,0,131,1,0,131,3, - 0,1,121,19,0,116,0,0,106,5,0,100,17,0,131,1, - 0,125,9,0,87,110,24,0,4,116,9,0,107,10,0,114, - 147,1,1,1,1,100,18,0,125,9,0,89,110,1,0,88, - 116,6,0,124,1,0,100,17,0,124,9,0,131,3,0,1, - 116,0,0,106,5,0,100,19,0,131,1,0,125,10,0,116, - 6,0,124,1,0,100,19,0,124,10,0,131,3,0,1,124, - 5,0,100,7,0,107,2,0,114,238,1,116,0,0,106,5, - 0,100,20,0,131,1,0,125,11,0,116,6,0,124,1,0, - 100,21,0,124,11,0,131,3,0,1,116,6,0,124,1,0, - 100,22,0,116,11,0,131,0,0,131,3,0,1,116,12,0, - 106,13,0,116,2,0,106,14,0,131,0,0,131,1,0,1, - 124,5,0,100,7,0,107,2,0,114,66,2,116,15,0,106, - 16,0,100,23,0,131,1,0,1,100,24,0,116,12,0,107, - 6,0,114,66,2,100,25,0,116,17,0,95,18,0,100,18, - 0,83,41,27,122,205,83,101,116,117,112,32,116,104,101,32, - 112,97,116,104,45,98,97,115,101,100,32,105,109,112,111,114, - 116,101,114,115,32,102,111,114,32,105,109,112,111,114,116,108, - 105,98,32,98,121,32,105,109,112,111,114,116,105,110,103,32, - 110,101,101,100,101,100,10,32,32,32,32,98,117,105,108,116, - 45,105,110,32,109,111,100,117,108,101,115,32,97,110,100,32, - 105,110,106,101,99,116,105,110,103,32,116,104,101,109,32,105, - 110,116,111,32,116,104,101,32,103,108,111,98,97,108,32,110, - 97,109,101,115,112,97,99,101,46,10,10,32,32,32,32,79, - 116,104,101,114,32,99,111,109,112,111,110,101,110,116,115,32, - 97,114,101,32,101,120,116,114,97,99,116,101,100,32,102,114, - 111,109,32,116,104,101,32,99,111,114,101,32,98,111,111,116, - 115,116,114,97,112,32,109,111,100,117,108,101,46,10,10,32, - 32,32,32,114,49,0,0,0,114,60,0,0,0,218,8,98, - 117,105,108,116,105,110,115,114,142,0,0,0,90,5,112,111, - 115,105,120,250,1,47,218,2,110,116,250,1,92,99,1,0, - 0,0,0,0,0,0,2,0,0,0,3,0,0,0,115,0, - 0,0,115,33,0,0,0,124,0,0,93,23,0,125,1,0, - 116,0,0,124,1,0,131,1,0,100,0,0,107,2,0,86, - 1,113,3,0,100,1,0,83,41,2,114,29,0,0,0,78, - 41,1,114,31,0,0,0,41,2,114,22,0,0,0,114,77, - 0,0,0,114,4,0,0,0,114,4,0,0,0,114,5,0, - 0,0,114,227,0,0,0,91,5,0,0,115,2,0,0,0, - 6,0,122,25,95,115,101,116,117,112,46,60,108,111,99,97, - 108,115,62,46,60,103,101,110,101,120,112,114,62,114,59,0, - 0,0,122,30,105,109,112,111,114,116,108,105,98,32,114,101, - 113,117,105,114,101,115,32,112,111,115,105,120,32,111,114,32, - 110,116,114,3,0,0,0,114,25,0,0,0,114,21,0,0, - 0,114,30,0,0,0,90,7,95,116,104,114,101,97,100,78, - 90,8,95,119,101,97,107,114,101,102,90,6,119,105,110,114, - 101,103,114,169,0,0,0,114,6,0,0,0,122,4,46,112, - 121,119,122,6,95,100,46,112,121,100,84,41,4,122,3,95, - 105,111,122,9,95,119,97,114,110,105,110,103,115,122,8,98, - 117,105,108,116,105,110,115,122,7,109,97,114,115,104,97,108, - 41,19,114,121,0,0,0,114,7,0,0,0,114,145,0,0, - 0,114,239,0,0,0,114,112,0,0,0,90,18,95,98,117, - 105,108,116,105,110,95,102,114,111,109,95,110,97,109,101,114, - 116,0,0,0,218,3,97,108,108,218,14,65,115,115,101,114, - 116,105,111,110,69,114,114,111,114,114,107,0,0,0,114,26, - 0,0,0,114,11,0,0,0,114,229,0,0,0,114,149,0, - 0,0,114,25,1,0,0,114,84,0,0,0,114,163,0,0, - 0,114,168,0,0,0,114,173,0,0,0,41,12,218,17,95, - 98,111,111,116,115,116,114,97,112,95,109,111,100,117,108,101, - 90,11,115,101,108,102,95,109,111,100,117,108,101,90,12,98, - 117,105,108,116,105,110,95,110,97,109,101,90,14,98,117,105, - 108,116,105,110,95,109,111,100,117,108,101,90,10,111,115,95, - 100,101,116,97,105,108,115,90,10,98,117,105,108,116,105,110, - 95,111,115,114,21,0,0,0,114,25,0,0,0,90,9,111, - 115,95,109,111,100,117,108,101,90,13,116,104,114,101,97,100, - 95,109,111,100,117,108,101,90,14,119,101,97,107,114,101,102, - 95,109,111,100,117,108,101,90,13,119,105,110,114,101,103,95, - 109,111,100,117,108,101,114,4,0,0,0,114,4,0,0,0, - 114,5,0,0,0,218,6,95,115,101,116,117,112,66,5,0, - 0,115,82,0,0,0,0,8,6,1,9,1,9,3,13,1, - 13,1,15,1,18,2,13,1,20,3,33,1,19,2,31,1, - 10,1,15,1,13,1,4,2,3,1,15,1,5,1,13,1, - 12,2,12,1,16,1,16,1,25,3,3,1,19,1,13,2, - 11,1,16,3,15,1,16,3,12,1,15,1,16,3,19,1, - 19,1,12,1,13,1,12,1,114,33,1,0,0,99,1,0, - 0,0,0,0,0,0,2,0,0,0,3,0,0,0,67,0, - 0,0,115,116,0,0,0,116,0,0,124,0,0,131,1,0, - 1,116,1,0,131,0,0,125,1,0,116,2,0,106,3,0, - 106,4,0,116,5,0,106,6,0,124,1,0,140,0,0,103, - 1,0,131,1,0,1,116,7,0,106,8,0,100,1,0,107, - 2,0,114,78,0,116,2,0,106,9,0,106,10,0,116,11, - 0,131,1,0,1,116,2,0,106,9,0,106,10,0,116,12, - 0,131,1,0,1,116,5,0,124,0,0,95,5,0,116,13, - 0,124,0,0,95,13,0,100,2,0,83,41,3,122,41,73, - 110,115,116,97,108,108,32,116,104,101,32,112,97,116,104,45, - 98,97,115,101,100,32,105,109,112,111,114,116,32,99,111,109, - 112,111,110,101,110,116,115,46,114,28,1,0,0,78,41,14, - 114,33,1,0,0,114,161,0,0,0,114,7,0,0,0,114, - 254,0,0,0,114,149,0,0,0,114,6,1,0,0,114,19, - 1,0,0,114,3,0,0,0,114,112,0,0,0,218,9,109, - 101,116,97,95,112,97,116,104,114,163,0,0,0,114,168,0, - 0,0,114,249,0,0,0,114,218,0,0,0,41,2,114,32, - 1,0,0,90,17,115,117,112,112,111,114,116,101,100,95,108, - 111,97,100,101,114,115,114,4,0,0,0,114,4,0,0,0, - 114,5,0,0,0,218,8,95,105,110,115,116,97,108,108,134, - 5,0,0,115,16,0,0,0,0,2,10,1,9,1,28,1, - 15,1,16,1,16,4,9,1,114,35,1,0,0,41,3,122, - 3,119,105,110,114,1,0,0,0,114,2,0,0,0,41,57, - 114,114,0,0,0,114,10,0,0,0,114,11,0,0,0,114, - 17,0,0,0,114,19,0,0,0,114,28,0,0,0,114,38, - 0,0,0,114,39,0,0,0,114,43,0,0,0,114,44,0, - 0,0,114,46,0,0,0,114,55,0,0,0,218,4,116,121, - 112,101,218,8,95,95,99,111,100,101,95,95,114,144,0,0, - 0,114,15,0,0,0,114,135,0,0,0,114,14,0,0,0, - 114,18,0,0,0,90,17,95,82,65,87,95,77,65,71,73, - 67,95,78,85,77,66,69,82,114,73,0,0,0,114,72,0, - 0,0,114,84,0,0,0,114,74,0,0,0,90,23,68,69, - 66,85,71,95,66,89,84,69,67,79,68,69,95,83,85,70, - 70,73,88,69,83,90,27,79,80,84,73,77,73,90,69,68, - 95,66,89,84,69,67,79,68,69,95,83,85,70,70,73,88, - 69,83,114,79,0,0,0,114,85,0,0,0,114,91,0,0, - 0,114,95,0,0,0,114,97,0,0,0,114,105,0,0,0, - 114,123,0,0,0,114,130,0,0,0,114,141,0,0,0,114, - 147,0,0,0,114,150,0,0,0,114,155,0,0,0,218,6, - 111,98,106,101,99,116,114,162,0,0,0,114,167,0,0,0, - 114,168,0,0,0,114,184,0,0,0,114,194,0,0,0,114, - 210,0,0,0,114,218,0,0,0,114,223,0,0,0,114,229, - 0,0,0,114,224,0,0,0,114,230,0,0,0,114,247,0, - 0,0,114,249,0,0,0,114,6,1,0,0,114,24,1,0, - 0,114,161,0,0,0,114,33,1,0,0,114,35,1,0,0, - 114,4,0,0,0,114,4,0,0,0,114,4,0,0,0,114, - 5,0,0,0,218,8,60,109,111,100,117,108,101,62,8,0, - 0,0,115,100,0,0,0,6,17,6,3,12,12,12,5,12, - 5,12,6,12,12,12,10,12,9,12,5,12,7,15,22,15, - 110,22,1,18,2,6,1,6,2,9,2,9,2,10,2,21, - 44,12,33,12,19,12,12,12,12,18,8,12,28,12,17,21, - 55,21,12,18,10,12,14,9,3,12,1,15,65,19,64,19, - 28,22,110,19,41,25,43,25,16,6,3,25,53,19,57,19, - 41,19,134,19,146,15,23,12,11,12,68, + 114,221,0,0,0,87,5,0,0,115,2,0,0,0,6,0, + 122,25,95,115,101,116,117,112,46,60,108,111,99,97,108,115, + 62,46,60,103,101,110,101,120,112,114,62,114,59,0,0,0, + 122,30,105,109,112,111,114,116,108,105,98,32,114,101,113,117, + 105,114,101,115,32,112,111,115,105,120,32,111,114,32,110,116, + 114,3,0,0,0,114,25,0,0,0,114,21,0,0,0,114, + 30,0,0,0,90,7,95,116,104,114,101,97,100,78,90,8, + 95,119,101,97,107,114,101,102,90,6,119,105,110,114,101,103, + 114,163,0,0,0,114,6,0,0,0,122,4,46,112,121,119, + 122,6,95,100,46,112,121,100,84,41,4,122,3,95,105,111, + 122,9,95,119,97,114,110,105,110,103,115,122,8,98,117,105, + 108,116,105,110,115,122,7,109,97,114,115,104,97,108,41,19, + 114,114,0,0,0,114,7,0,0,0,114,139,0,0,0,114, + 233,0,0,0,114,105,0,0,0,90,18,95,98,117,105,108, + 116,105,110,95,102,114,111,109,95,110,97,109,101,114,109,0, + 0,0,218,3,97,108,108,218,14,65,115,115,101,114,116,105, + 111,110,69,114,114,111,114,114,99,0,0,0,114,26,0,0, + 0,114,11,0,0,0,114,223,0,0,0,114,143,0,0,0, + 114,19,1,0,0,114,84,0,0,0,114,157,0,0,0,114, + 162,0,0,0,114,167,0,0,0,41,12,218,17,95,98,111, + 111,116,115,116,114,97,112,95,109,111,100,117,108,101,90,11, + 115,101,108,102,95,109,111,100,117,108,101,90,12,98,117,105, + 108,116,105,110,95,110,97,109,101,90,14,98,117,105,108,116, + 105,110,95,109,111,100,117,108,101,90,10,111,115,95,100,101, + 116,97,105,108,115,90,10,98,117,105,108,116,105,110,95,111, + 115,114,21,0,0,0,114,25,0,0,0,90,9,111,115,95, + 109,111,100,117,108,101,90,13,116,104,114,101,97,100,95,109, + 111,100,117,108,101,90,14,119,101,97,107,114,101,102,95,109, + 111,100,117,108,101,90,13,119,105,110,114,101,103,95,109,111, + 100,117,108,101,114,4,0,0,0,114,4,0,0,0,114,5, + 0,0,0,218,6,95,115,101,116,117,112,62,5,0,0,115, + 82,0,0,0,0,8,6,1,9,1,9,3,13,1,13,1, + 15,1,18,2,13,1,20,3,33,1,19,2,31,1,10,1, + 15,1,13,1,4,2,3,1,15,1,5,1,13,1,12,2, + 12,1,16,1,16,1,25,3,3,1,19,1,13,2,11,1, + 16,3,15,1,16,3,12,1,15,1,16,3,19,1,19,1, + 12,1,13,1,12,1,114,27,1,0,0,99,1,0,0,0, + 0,0,0,0,2,0,0,0,3,0,0,0,67,0,0,0, + 115,116,0,0,0,116,0,0,124,0,0,131,1,0,1,116, + 1,0,131,0,0,125,1,0,116,2,0,106,3,0,106,4, + 0,116,5,0,106,6,0,124,1,0,140,0,0,103,1,0, + 131,1,0,1,116,7,0,106,8,0,100,1,0,107,2,0, + 114,78,0,116,2,0,106,9,0,106,10,0,116,11,0,131, + 1,0,1,116,2,0,106,9,0,106,10,0,116,12,0,131, + 1,0,1,116,5,0,124,0,0,95,5,0,116,13,0,124, + 0,0,95,13,0,100,2,0,83,41,3,122,41,73,110,115, + 116,97,108,108,32,116,104,101,32,112,97,116,104,45,98,97, + 115,101,100,32,105,109,112,111,114,116,32,99,111,109,112,111, + 110,101,110,116,115,46,114,22,1,0,0,78,41,14,114,27, + 1,0,0,114,155,0,0,0,114,7,0,0,0,114,248,0, + 0,0,114,143,0,0,0,114,0,1,0,0,114,13,1,0, + 0,114,3,0,0,0,114,105,0,0,0,218,9,109,101,116, + 97,95,112,97,116,104,114,157,0,0,0,114,162,0,0,0, + 114,243,0,0,0,114,212,0,0,0,41,2,114,26,1,0, + 0,90,17,115,117,112,112,111,114,116,101,100,95,108,111,97, + 100,101,114,115,114,4,0,0,0,114,4,0,0,0,114,5, + 0,0,0,218,8,95,105,110,115,116,97,108,108,130,5,0, + 0,115,16,0,0,0,0,2,10,1,9,1,28,1,15,1, + 16,1,16,4,9,1,114,29,1,0,0,41,3,122,3,119, + 105,110,114,1,0,0,0,114,2,0,0,0,41,56,114,107, + 0,0,0,114,10,0,0,0,114,11,0,0,0,114,17,0, + 0,0,114,19,0,0,0,114,28,0,0,0,114,38,0,0, + 0,114,39,0,0,0,114,43,0,0,0,114,44,0,0,0, + 114,46,0,0,0,114,55,0,0,0,218,4,116,121,112,101, + 218,8,95,95,99,111,100,101,95,95,114,138,0,0,0,114, + 15,0,0,0,114,128,0,0,0,114,14,0,0,0,114,18, + 0,0,0,90,17,95,82,65,87,95,77,65,71,73,67,95, + 78,85,77,66,69,82,114,73,0,0,0,114,72,0,0,0, + 114,84,0,0,0,114,74,0,0,0,90,23,68,69,66,85, + 71,95,66,89,84,69,67,79,68,69,95,83,85,70,70,73, + 88,69,83,90,27,79,80,84,73,77,73,90,69,68,95,66, + 89,84,69,67,79,68,69,95,83,85,70,70,73,88,69,83, + 114,79,0,0,0,114,85,0,0,0,114,91,0,0,0,114, + 95,0,0,0,114,97,0,0,0,114,116,0,0,0,114,123, + 0,0,0,114,135,0,0,0,114,141,0,0,0,114,144,0, + 0,0,114,149,0,0,0,218,6,111,98,106,101,99,116,114, + 156,0,0,0,114,161,0,0,0,114,162,0,0,0,114,178, + 0,0,0,114,188,0,0,0,114,204,0,0,0,114,212,0, + 0,0,114,217,0,0,0,114,223,0,0,0,114,218,0,0, + 0,114,224,0,0,0,114,241,0,0,0,114,243,0,0,0, + 114,0,1,0,0,114,18,1,0,0,114,155,0,0,0,114, + 27,1,0,0,114,29,1,0,0,114,4,0,0,0,114,4, + 0,0,0,114,4,0,0,0,114,5,0,0,0,218,8,60, + 109,111,100,117,108,101,62,8,0,0,0,115,98,0,0,0, + 6,17,6,3,12,12,12,5,12,5,12,6,12,12,12,10, + 12,9,12,5,12,7,15,22,15,110,22,1,18,2,6,1, + 6,2,9,2,9,2,10,2,21,44,12,33,12,19,12,12, + 12,12,12,28,12,17,21,55,21,12,18,10,12,14,9,3, + 12,1,15,65,19,64,19,28,22,110,19,41,25,45,25,16, + 6,3,25,53,19,57,19,42,19,134,19,147,15,23,12,11, + 12,68, }; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 26 02:12:57 2015 From: python-checkins at python.org (martin.panter) Date: Sat, 26 Sep 2015 00:12:57 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzI1MjEx?= =?utf-8?q?=3A_Eliminate_lazy_error_message_class_by_using_subTest?= Message-ID: <20150926001257.11700.60375@psf.io> https://hg.python.org/cpython/rev/697781ff3b49 changeset: 98266:697781ff3b49 branch: 3.4 parent: 98262:1c119da20663 user: Martin Panter date: Fri Sep 25 23:50:47 2015 +0000 summary: Issue #25211: Eliminate lazy error message class by using subTest Some of the calls to the Frm class were buggy anyway. files: Lib/test/test_long.py | 160 +++++++++++++---------------- 1 files changed, 72 insertions(+), 88 deletions(-) diff --git a/Lib/test/test_long.py b/Lib/test/test_long.py --- a/Lib/test/test_long.py +++ b/Lib/test/test_long.py @@ -7,15 +7,6 @@ import math import array -# Used for lazy formatting of failure messages -class Frm(object): - def __init__(self, format, *args): - self.format = format - self.args = args - - def __str__(self): - return self.format % self.args - # SHIFT should match the value in longintrepr.h for best testing. SHIFT = sys.int_info.bits_per_digit BASE = 2 ** SHIFT @@ -163,17 +154,18 @@ def check_division(self, x, y): eq = self.assertEqual - q, r = divmod(x, y) - q2, r2 = x//y, x%y - pab, pba = x*y, y*x - eq(pab, pba, Frm("multiplication does not commute for %r and %r", x, y)) - eq(q, q2, Frm("divmod returns different quotient than / for %r and %r", x, y)) - eq(r, r2, Frm("divmod returns different mod than %% for %r and %r", x, y)) - eq(x, q*y + r, Frm("x != q*y + r after divmod on x=%r, y=%r", x, y)) - if y > 0: - self.assertTrue(0 <= r < y, Frm("bad mod from divmod on %r and %r", x, y)) - else: - self.assertTrue(y < r <= 0, Frm("bad mod from divmod on %r and %r", x, y)) + with self.subTest(x=x, y=y): + q, r = divmod(x, y) + q2, r2 = x//y, x%y + pab, pba = x*y, y*x + eq(pab, pba, "multiplication does not commute") + eq(q, q2, "divmod returns different quotient than /") + eq(r, r2, "divmod returns different mod than %") + eq(x, q*y + r, "x != q*y + r after divmod") + if y > 0: + self.assertTrue(0 <= r < y, "bad mod from divmod") + else: + self.assertTrue(y < r <= 0, "bad mod from divmod") def test_division(self): digits = list(range(1, MAXDIGITS+1)) + list(range(KARATSUBA_CUTOFF, @@ -228,72 +220,63 @@ for bbits in bits: if bbits < abits: continue - b = (1 << bbits) - 1 - x = a * b - y = ((1 << (abits + bbits)) - - (1 << abits) - - (1 << bbits) + - 1) - self.assertEqual(x, y, - Frm("bad result for a*b: a=%r, b=%r, x=%r, y=%r", a, b, x, y)) + with self.subTest(abits=abits, bbits=bbits): + b = (1 << bbits) - 1 + x = a * b + y = ((1 << (abits + bbits)) - + (1 << abits) - + (1 << bbits) + + 1) + self.assertEqual(x, y) def check_bitop_identities_1(self, x): eq = self.assertEqual - eq(x & 0, 0, Frm("x & 0 != 0 for x=%r", x)) - eq(x | 0, x, Frm("x | 0 != x for x=%r", x)) - eq(x ^ 0, x, Frm("x ^ 0 != x for x=%r", x)) - eq(x & -1, x, Frm("x & -1 != x for x=%r", x)) - eq(x | -1, -1, Frm("x | -1 != -1 for x=%r", x)) - eq(x ^ -1, ~x, Frm("x ^ -1 != ~x for x=%r", x)) - eq(x, ~~x, Frm("x != ~~x for x=%r", x)) - eq(x & x, x, Frm("x & x != x for x=%r", x)) - eq(x | x, x, Frm("x | x != x for x=%r", x)) - eq(x ^ x, 0, Frm("x ^ x != 0 for x=%r", x)) - eq(x & ~x, 0, Frm("x & ~x != 0 for x=%r", x)) - eq(x | ~x, -1, Frm("x | ~x != -1 for x=%r", x)) - eq(x ^ ~x, -1, Frm("x ^ ~x != -1 for x=%r", x)) - eq(-x, 1 + ~x, Frm("not -x == 1 + ~x for x=%r", x)) - eq(-x, ~(x-1), Frm("not -x == ~(x-1) forx =%r", x)) + with self.subTest(x=x): + eq(x & 0, 0) + eq(x | 0, x) + eq(x ^ 0, x) + eq(x & -1, x) + eq(x | -1, -1) + eq(x ^ -1, ~x) + eq(x, ~~x) + eq(x & x, x) + eq(x | x, x) + eq(x ^ x, 0) + eq(x & ~x, 0) + eq(x | ~x, -1) + eq(x ^ ~x, -1) + eq(-x, 1 + ~x) + eq(-x, ~(x-1)) for n in range(2*SHIFT): p2 = 2 ** n - eq(x << n >> n, x, - Frm("x << n >> n != x for x=%r, n=%r", (x, n))) - eq(x // p2, x >> n, - Frm("x // p2 != x >> n for x=%r n=%r p2=%r", (x, n, p2))) - eq(x * p2, x << n, - Frm("x * p2 != x << n for x=%r n=%r p2=%r", (x, n, p2))) - eq(x & -p2, x >> n << n, - Frm("not x & -p2 == x >> n << n for x=%r n=%r p2=%r", (x, n, p2))) - eq(x & -p2, x & ~(p2 - 1), - Frm("not x & -p2 == x & ~(p2 - 1) for x=%r n=%r p2=%r", (x, n, p2))) + with self.subTest(x=x, n=n, p2=p2): + eq(x << n >> n, x) + eq(x // p2, x >> n) + eq(x * p2, x << n) + eq(x & -p2, x >> n << n) + eq(x & -p2, x & ~(p2 - 1)) def check_bitop_identities_2(self, x, y): eq = self.assertEqual - eq(x & y, y & x, Frm("x & y != y & x for x=%r, y=%r", (x, y))) - eq(x | y, y | x, Frm("x | y != y | x for x=%r, y=%r", (x, y))) - eq(x ^ y, y ^ x, Frm("x ^ y != y ^ x for x=%r, y=%r", (x, y))) - eq(x ^ y ^ x, y, Frm("x ^ y ^ x != y for x=%r, y=%r", (x, y))) - eq(x & y, ~(~x | ~y), Frm("x & y != ~(~x | ~y) for x=%r, y=%r", (x, y))) - eq(x | y, ~(~x & ~y), Frm("x | y != ~(~x & ~y) for x=%r, y=%r", (x, y))) - eq(x ^ y, (x | y) & ~(x & y), - Frm("x ^ y != (x | y) & ~(x & y) for x=%r, y=%r", (x, y))) - eq(x ^ y, (x & ~y) | (~x & y), - Frm("x ^ y == (x & ~y) | (~x & y) for x=%r, y=%r", (x, y))) - eq(x ^ y, (x | y) & (~x | ~y), - Frm("x ^ y == (x | y) & (~x | ~y) for x=%r, y=%r", (x, y))) + with self.subTest(x=x, y=y): + eq(x & y, y & x) + eq(x | y, y | x) + eq(x ^ y, y ^ x) + eq(x ^ y ^ x, y) + eq(x & y, ~(~x | ~y)) + eq(x | y, ~(~x & ~y)) + eq(x ^ y, (x | y) & ~(x & y)) + eq(x ^ y, (x & ~y) | (~x & y)) + eq(x ^ y, (x | y) & (~x | ~y)) def check_bitop_identities_3(self, x, y, z): eq = self.assertEqual - eq((x & y) & z, x & (y & z), - Frm("(x & y) & z != x & (y & z) for x=%r, y=%r, z=%r", (x, y, z))) - eq((x | y) | z, x | (y | z), - Frm("(x | y) | z != x | (y | z) for x=%r, y=%r, z=%r", (x, y, z))) - eq((x ^ y) ^ z, x ^ (y ^ z), - Frm("(x ^ y) ^ z != x ^ (y ^ z) for x=%r, y=%r, z=%r", (x, y, z))) - eq(x & (y | z), (x & y) | (x & z), - Frm("x & (y | z) != (x & y) | (x & z) for x=%r, y=%r, z=%r", (x, y, z))) - eq(x | (y & z), (x | y) & (x | z), - Frm("x | (y & z) != (x | y) & (x | z) for x=%r, y=%r, z=%r", (x, y, z))) + with self.subTest(x=x, y=y, z=z): + eq((x & y) & z, x & (y & z)) + eq((x | y) | z, x | (y | z)) + eq((x ^ y) ^ z, x ^ (y ^ z)) + eq(x & (y | z), (x & y) | (x & z)) + eq(x | (y & z), (x | y) & (x | z)) def test_bitop_identities(self): for x in special: @@ -324,11 +307,11 @@ def check_format_1(self, x): for base, mapper in (2, bin), (8, oct), (10, str), (10, repr), (16, hex): got = mapper(x) - expected = self.slow_format(x, base) - msg = Frm("%s returned %r but expected %r for %r", - mapper.__name__, got, expected, x) - self.assertEqual(got, expected, msg) - self.assertEqual(int(got, 0), x, Frm('int("%s", 0) != %r', got, x)) + with self.subTest(x=x, mapper=mapper.__name__): + expected = self.slow_format(x, base) + self.assertEqual(got, expected) + with self.subTest(got=got): + self.assertEqual(int(got, 0), x) def test_format(self): for x in special: @@ -627,14 +610,15 @@ for y in cases: Ry = Rat(y) Rcmp = (Rx > Ry) - (Rx < Ry) - xycmp = (x > y) - (x < y) - eq(Rcmp, xycmp, Frm("%r %r %d %d", x, y, Rcmp, xycmp)) - eq(x == y, Rcmp == 0, Frm("%r == %r %d", x, y, Rcmp)) - eq(x != y, Rcmp != 0, Frm("%r != %r %d", x, y, Rcmp)) - eq(x < y, Rcmp < 0, Frm("%r < %r %d", x, y, Rcmp)) - eq(x <= y, Rcmp <= 0, Frm("%r <= %r %d", x, y, Rcmp)) - eq(x > y, Rcmp > 0, Frm("%r > %r %d", x, y, Rcmp)) - eq(x >= y, Rcmp >= 0, Frm("%r >= %r %d", x, y, Rcmp)) + with self.subTest(x=x, y=y, Rcmp=Rcmp): + xycmp = (x > y) - (x < y) + eq(Rcmp, xycmp) + eq(x == y, Rcmp == 0) + eq(x != y, Rcmp != 0) + eq(x < y, Rcmp < 0) + eq(x <= y, Rcmp <= 0) + eq(x > y, Rcmp > 0) + eq(x >= y, Rcmp >= 0) def test__format__(self): self.assertEqual(format(123456789, 'd'), '123456789') -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 26 02:12:58 2015 From: python-checkins at python.org (martin.panter) Date: Sat, 26 Sep 2015 00:12:58 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Issue_=2325211=3A_Merge_test=5Flong_from_3=2E4_into_3=2E5?= Message-ID: <20150926001257.81647.57291@psf.io> https://hg.python.org/cpython/rev/6e11708dcb3b changeset: 98267:6e11708dcb3b branch: 3.5 parent: 98263:37195cb81b8f parent: 98266:697781ff3b49 user: Martin Panter date: Sat Sep 26 00:07:29 2015 +0000 summary: Issue #25211: Merge test_long from 3.4 into 3.5 files: Lib/test/test_long.py | 160 +++++++++++++---------------- 1 files changed, 72 insertions(+), 88 deletions(-) diff --git a/Lib/test/test_long.py b/Lib/test/test_long.py --- a/Lib/test/test_long.py +++ b/Lib/test/test_long.py @@ -7,15 +7,6 @@ import math import array -# Used for lazy formatting of failure messages -class Frm(object): - def __init__(self, format, *args): - self.format = format - self.args = args - - def __str__(self): - return self.format % self.args - # SHIFT should match the value in longintrepr.h for best testing. SHIFT = sys.int_info.bits_per_digit BASE = 2 ** SHIFT @@ -163,17 +154,18 @@ def check_division(self, x, y): eq = self.assertEqual - q, r = divmod(x, y) - q2, r2 = x//y, x%y - pab, pba = x*y, y*x - eq(pab, pba, Frm("multiplication does not commute for %r and %r", x, y)) - eq(q, q2, Frm("divmod returns different quotient than / for %r and %r", x, y)) - eq(r, r2, Frm("divmod returns different mod than %% for %r and %r", x, y)) - eq(x, q*y + r, Frm("x != q*y + r after divmod on x=%r, y=%r", x, y)) - if y > 0: - self.assertTrue(0 <= r < y, Frm("bad mod from divmod on %r and %r", x, y)) - else: - self.assertTrue(y < r <= 0, Frm("bad mod from divmod on %r and %r", x, y)) + with self.subTest(x=x, y=y): + q, r = divmod(x, y) + q2, r2 = x//y, x%y + pab, pba = x*y, y*x + eq(pab, pba, "multiplication does not commute") + eq(q, q2, "divmod returns different quotient than /") + eq(r, r2, "divmod returns different mod than %") + eq(x, q*y + r, "x != q*y + r after divmod") + if y > 0: + self.assertTrue(0 <= r < y, "bad mod from divmod") + else: + self.assertTrue(y < r <= 0, "bad mod from divmod") def test_division(self): digits = list(range(1, MAXDIGITS+1)) + list(range(KARATSUBA_CUTOFF, @@ -228,72 +220,63 @@ for bbits in bits: if bbits < abits: continue - b = (1 << bbits) - 1 - x = a * b - y = ((1 << (abits + bbits)) - - (1 << abits) - - (1 << bbits) + - 1) - self.assertEqual(x, y, - Frm("bad result for a*b: a=%r, b=%r, x=%r, y=%r", a, b, x, y)) + with self.subTest(abits=abits, bbits=bbits): + b = (1 << bbits) - 1 + x = a * b + y = ((1 << (abits + bbits)) - + (1 << abits) - + (1 << bbits) + + 1) + self.assertEqual(x, y) def check_bitop_identities_1(self, x): eq = self.assertEqual - eq(x & 0, 0, Frm("x & 0 != 0 for x=%r", x)) - eq(x | 0, x, Frm("x | 0 != x for x=%r", x)) - eq(x ^ 0, x, Frm("x ^ 0 != x for x=%r", x)) - eq(x & -1, x, Frm("x & -1 != x for x=%r", x)) - eq(x | -1, -1, Frm("x | -1 != -1 for x=%r", x)) - eq(x ^ -1, ~x, Frm("x ^ -1 != ~x for x=%r", x)) - eq(x, ~~x, Frm("x != ~~x for x=%r", x)) - eq(x & x, x, Frm("x & x != x for x=%r", x)) - eq(x | x, x, Frm("x | x != x for x=%r", x)) - eq(x ^ x, 0, Frm("x ^ x != 0 for x=%r", x)) - eq(x & ~x, 0, Frm("x & ~x != 0 for x=%r", x)) - eq(x | ~x, -1, Frm("x | ~x != -1 for x=%r", x)) - eq(x ^ ~x, -1, Frm("x ^ ~x != -1 for x=%r", x)) - eq(-x, 1 + ~x, Frm("not -x == 1 + ~x for x=%r", x)) - eq(-x, ~(x-1), Frm("not -x == ~(x-1) forx =%r", x)) + with self.subTest(x=x): + eq(x & 0, 0) + eq(x | 0, x) + eq(x ^ 0, x) + eq(x & -1, x) + eq(x | -1, -1) + eq(x ^ -1, ~x) + eq(x, ~~x) + eq(x & x, x) + eq(x | x, x) + eq(x ^ x, 0) + eq(x & ~x, 0) + eq(x | ~x, -1) + eq(x ^ ~x, -1) + eq(-x, 1 + ~x) + eq(-x, ~(x-1)) for n in range(2*SHIFT): p2 = 2 ** n - eq(x << n >> n, x, - Frm("x << n >> n != x for x=%r, n=%r", (x, n))) - eq(x // p2, x >> n, - Frm("x // p2 != x >> n for x=%r n=%r p2=%r", (x, n, p2))) - eq(x * p2, x << n, - Frm("x * p2 != x << n for x=%r n=%r p2=%r", (x, n, p2))) - eq(x & -p2, x >> n << n, - Frm("not x & -p2 == x >> n << n for x=%r n=%r p2=%r", (x, n, p2))) - eq(x & -p2, x & ~(p2 - 1), - Frm("not x & -p2 == x & ~(p2 - 1) for x=%r n=%r p2=%r", (x, n, p2))) + with self.subTest(x=x, n=n, p2=p2): + eq(x << n >> n, x) + eq(x // p2, x >> n) + eq(x * p2, x << n) + eq(x & -p2, x >> n << n) + eq(x & -p2, x & ~(p2 - 1)) def check_bitop_identities_2(self, x, y): eq = self.assertEqual - eq(x & y, y & x, Frm("x & y != y & x for x=%r, y=%r", (x, y))) - eq(x | y, y | x, Frm("x | y != y | x for x=%r, y=%r", (x, y))) - eq(x ^ y, y ^ x, Frm("x ^ y != y ^ x for x=%r, y=%r", (x, y))) - eq(x ^ y ^ x, y, Frm("x ^ y ^ x != y for x=%r, y=%r", (x, y))) - eq(x & y, ~(~x | ~y), Frm("x & y != ~(~x | ~y) for x=%r, y=%r", (x, y))) - eq(x | y, ~(~x & ~y), Frm("x | y != ~(~x & ~y) for x=%r, y=%r", (x, y))) - eq(x ^ y, (x | y) & ~(x & y), - Frm("x ^ y != (x | y) & ~(x & y) for x=%r, y=%r", (x, y))) - eq(x ^ y, (x & ~y) | (~x & y), - Frm("x ^ y == (x & ~y) | (~x & y) for x=%r, y=%r", (x, y))) - eq(x ^ y, (x | y) & (~x | ~y), - Frm("x ^ y == (x | y) & (~x | ~y) for x=%r, y=%r", (x, y))) + with self.subTest(x=x, y=y): + eq(x & y, y & x) + eq(x | y, y | x) + eq(x ^ y, y ^ x) + eq(x ^ y ^ x, y) + eq(x & y, ~(~x | ~y)) + eq(x | y, ~(~x & ~y)) + eq(x ^ y, (x | y) & ~(x & y)) + eq(x ^ y, (x & ~y) | (~x & y)) + eq(x ^ y, (x | y) & (~x | ~y)) def check_bitop_identities_3(self, x, y, z): eq = self.assertEqual - eq((x & y) & z, x & (y & z), - Frm("(x & y) & z != x & (y & z) for x=%r, y=%r, z=%r", (x, y, z))) - eq((x | y) | z, x | (y | z), - Frm("(x | y) | z != x | (y | z) for x=%r, y=%r, z=%r", (x, y, z))) - eq((x ^ y) ^ z, x ^ (y ^ z), - Frm("(x ^ y) ^ z != x ^ (y ^ z) for x=%r, y=%r, z=%r", (x, y, z))) - eq(x & (y | z), (x & y) | (x & z), - Frm("x & (y | z) != (x & y) | (x & z) for x=%r, y=%r, z=%r", (x, y, z))) - eq(x | (y & z), (x | y) & (x | z), - Frm("x | (y & z) != (x | y) & (x | z) for x=%r, y=%r, z=%r", (x, y, z))) + with self.subTest(x=x, y=y, z=z): + eq((x & y) & z, x & (y & z)) + eq((x | y) | z, x | (y | z)) + eq((x ^ y) ^ z, x ^ (y ^ z)) + eq(x & (y | z), (x & y) | (x & z)) + eq(x | (y & z), (x | y) & (x | z)) def test_bitop_identities(self): for x in special: @@ -324,11 +307,11 @@ def check_format_1(self, x): for base, mapper in (2, bin), (8, oct), (10, str), (10, repr), (16, hex): got = mapper(x) - expected = self.slow_format(x, base) - msg = Frm("%s returned %r but expected %r for %r", - mapper.__name__, got, expected, x) - self.assertEqual(got, expected, msg) - self.assertEqual(int(got, 0), x, Frm('int("%s", 0) != %r', got, x)) + with self.subTest(x=x, mapper=mapper.__name__): + expected = self.slow_format(x, base) + self.assertEqual(got, expected) + with self.subTest(got=got): + self.assertEqual(int(got, 0), x) def test_format(self): for x in special: @@ -625,14 +608,15 @@ for y in cases: Ry = Rat(y) Rcmp = (Rx > Ry) - (Rx < Ry) - xycmp = (x > y) - (x < y) - eq(Rcmp, xycmp, Frm("%r %r %d %d", x, y, Rcmp, xycmp)) - eq(x == y, Rcmp == 0, Frm("%r == %r %d", x, y, Rcmp)) - eq(x != y, Rcmp != 0, Frm("%r != %r %d", x, y, Rcmp)) - eq(x < y, Rcmp < 0, Frm("%r < %r %d", x, y, Rcmp)) - eq(x <= y, Rcmp <= 0, Frm("%r <= %r %d", x, y, Rcmp)) - eq(x > y, Rcmp > 0, Frm("%r > %r %d", x, y, Rcmp)) - eq(x >= y, Rcmp >= 0, Frm("%r >= %r %d", x, y, Rcmp)) + with self.subTest(x=x, y=y, Rcmp=Rcmp): + xycmp = (x > y) - (x < y) + eq(Rcmp, xycmp) + eq(x == y, Rcmp == 0) + eq(x != y, Rcmp != 0) + eq(x < y, Rcmp < 0) + eq(x <= y, Rcmp <= 0) + eq(x > y, Rcmp > 0) + eq(x >= y, Rcmp >= 0) def test__format__(self): self.assertEqual(format(123456789, 'd'), '123456789') -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 26 02:12:58 2015 From: python-checkins at python.org (martin.panter) Date: Sat, 26 Sep 2015 00:12:58 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2325211=3A_Merge_test=5Flong_from_3=2E5?= Message-ID: <20150926001257.81643.71436@psf.io> https://hg.python.org/cpython/rev/11ad44d4177f changeset: 98268:11ad44d4177f parent: 98265:1266e98fd04c parent: 98267:6e11708dcb3b user: Martin Panter date: Sat Sep 26 00:07:54 2015 +0000 summary: Issue #25211: Merge test_long from 3.5 files: Lib/test/test_long.py | 160 +++++++++++++---------------- 1 files changed, 72 insertions(+), 88 deletions(-) diff --git a/Lib/test/test_long.py b/Lib/test/test_long.py --- a/Lib/test/test_long.py +++ b/Lib/test/test_long.py @@ -7,15 +7,6 @@ import math import array -# Used for lazy formatting of failure messages -class Frm(object): - def __init__(self, format, *args): - self.format = format - self.args = args - - def __str__(self): - return self.format % self.args - # SHIFT should match the value in longintrepr.h for best testing. SHIFT = sys.int_info.bits_per_digit BASE = 2 ** SHIFT @@ -163,17 +154,18 @@ def check_division(self, x, y): eq = self.assertEqual - q, r = divmod(x, y) - q2, r2 = x//y, x%y - pab, pba = x*y, y*x - eq(pab, pba, Frm("multiplication does not commute for %r and %r", x, y)) - eq(q, q2, Frm("divmod returns different quotient than / for %r and %r", x, y)) - eq(r, r2, Frm("divmod returns different mod than %% for %r and %r", x, y)) - eq(x, q*y + r, Frm("x != q*y + r after divmod on x=%r, y=%r", x, y)) - if y > 0: - self.assertTrue(0 <= r < y, Frm("bad mod from divmod on %r and %r", x, y)) - else: - self.assertTrue(y < r <= 0, Frm("bad mod from divmod on %r and %r", x, y)) + with self.subTest(x=x, y=y): + q, r = divmod(x, y) + q2, r2 = x//y, x%y + pab, pba = x*y, y*x + eq(pab, pba, "multiplication does not commute") + eq(q, q2, "divmod returns different quotient than /") + eq(r, r2, "divmod returns different mod than %") + eq(x, q*y + r, "x != q*y + r after divmod") + if y > 0: + self.assertTrue(0 <= r < y, "bad mod from divmod") + else: + self.assertTrue(y < r <= 0, "bad mod from divmod") def test_division(self): digits = list(range(1, MAXDIGITS+1)) + list(range(KARATSUBA_CUTOFF, @@ -228,72 +220,63 @@ for bbits in bits: if bbits < abits: continue - b = (1 << bbits) - 1 - x = a * b - y = ((1 << (abits + bbits)) - - (1 << abits) - - (1 << bbits) + - 1) - self.assertEqual(x, y, - Frm("bad result for a*b: a=%r, b=%r, x=%r, y=%r", a, b, x, y)) + with self.subTest(abits=abits, bbits=bbits): + b = (1 << bbits) - 1 + x = a * b + y = ((1 << (abits + bbits)) - + (1 << abits) - + (1 << bbits) + + 1) + self.assertEqual(x, y) def check_bitop_identities_1(self, x): eq = self.assertEqual - eq(x & 0, 0, Frm("x & 0 != 0 for x=%r", x)) - eq(x | 0, x, Frm("x | 0 != x for x=%r", x)) - eq(x ^ 0, x, Frm("x ^ 0 != x for x=%r", x)) - eq(x & -1, x, Frm("x & -1 != x for x=%r", x)) - eq(x | -1, -1, Frm("x | -1 != -1 for x=%r", x)) - eq(x ^ -1, ~x, Frm("x ^ -1 != ~x for x=%r", x)) - eq(x, ~~x, Frm("x != ~~x for x=%r", x)) - eq(x & x, x, Frm("x & x != x for x=%r", x)) - eq(x | x, x, Frm("x | x != x for x=%r", x)) - eq(x ^ x, 0, Frm("x ^ x != 0 for x=%r", x)) - eq(x & ~x, 0, Frm("x & ~x != 0 for x=%r", x)) - eq(x | ~x, -1, Frm("x | ~x != -1 for x=%r", x)) - eq(x ^ ~x, -1, Frm("x ^ ~x != -1 for x=%r", x)) - eq(-x, 1 + ~x, Frm("not -x == 1 + ~x for x=%r", x)) - eq(-x, ~(x-1), Frm("not -x == ~(x-1) forx =%r", x)) + with self.subTest(x=x): + eq(x & 0, 0) + eq(x | 0, x) + eq(x ^ 0, x) + eq(x & -1, x) + eq(x | -1, -1) + eq(x ^ -1, ~x) + eq(x, ~~x) + eq(x & x, x) + eq(x | x, x) + eq(x ^ x, 0) + eq(x & ~x, 0) + eq(x | ~x, -1) + eq(x ^ ~x, -1) + eq(-x, 1 + ~x) + eq(-x, ~(x-1)) for n in range(2*SHIFT): p2 = 2 ** n - eq(x << n >> n, x, - Frm("x << n >> n != x for x=%r, n=%r", (x, n))) - eq(x // p2, x >> n, - Frm("x // p2 != x >> n for x=%r n=%r p2=%r", (x, n, p2))) - eq(x * p2, x << n, - Frm("x * p2 != x << n for x=%r n=%r p2=%r", (x, n, p2))) - eq(x & -p2, x >> n << n, - Frm("not x & -p2 == x >> n << n for x=%r n=%r p2=%r", (x, n, p2))) - eq(x & -p2, x & ~(p2 - 1), - Frm("not x & -p2 == x & ~(p2 - 1) for x=%r n=%r p2=%r", (x, n, p2))) + with self.subTest(x=x, n=n, p2=p2): + eq(x << n >> n, x) + eq(x // p2, x >> n) + eq(x * p2, x << n) + eq(x & -p2, x >> n << n) + eq(x & -p2, x & ~(p2 - 1)) def check_bitop_identities_2(self, x, y): eq = self.assertEqual - eq(x & y, y & x, Frm("x & y != y & x for x=%r, y=%r", (x, y))) - eq(x | y, y | x, Frm("x | y != y | x for x=%r, y=%r", (x, y))) - eq(x ^ y, y ^ x, Frm("x ^ y != y ^ x for x=%r, y=%r", (x, y))) - eq(x ^ y ^ x, y, Frm("x ^ y ^ x != y for x=%r, y=%r", (x, y))) - eq(x & y, ~(~x | ~y), Frm("x & y != ~(~x | ~y) for x=%r, y=%r", (x, y))) - eq(x | y, ~(~x & ~y), Frm("x | y != ~(~x & ~y) for x=%r, y=%r", (x, y))) - eq(x ^ y, (x | y) & ~(x & y), - Frm("x ^ y != (x | y) & ~(x & y) for x=%r, y=%r", (x, y))) - eq(x ^ y, (x & ~y) | (~x & y), - Frm("x ^ y == (x & ~y) | (~x & y) for x=%r, y=%r", (x, y))) - eq(x ^ y, (x | y) & (~x | ~y), - Frm("x ^ y == (x | y) & (~x | ~y) for x=%r, y=%r", (x, y))) + with self.subTest(x=x, y=y): + eq(x & y, y & x) + eq(x | y, y | x) + eq(x ^ y, y ^ x) + eq(x ^ y ^ x, y) + eq(x & y, ~(~x | ~y)) + eq(x | y, ~(~x & ~y)) + eq(x ^ y, (x | y) & ~(x & y)) + eq(x ^ y, (x & ~y) | (~x & y)) + eq(x ^ y, (x | y) & (~x | ~y)) def check_bitop_identities_3(self, x, y, z): eq = self.assertEqual - eq((x & y) & z, x & (y & z), - Frm("(x & y) & z != x & (y & z) for x=%r, y=%r, z=%r", (x, y, z))) - eq((x | y) | z, x | (y | z), - Frm("(x | y) | z != x | (y | z) for x=%r, y=%r, z=%r", (x, y, z))) - eq((x ^ y) ^ z, x ^ (y ^ z), - Frm("(x ^ y) ^ z != x ^ (y ^ z) for x=%r, y=%r, z=%r", (x, y, z))) - eq(x & (y | z), (x & y) | (x & z), - Frm("x & (y | z) != (x & y) | (x & z) for x=%r, y=%r, z=%r", (x, y, z))) - eq(x | (y & z), (x | y) & (x | z), - Frm("x | (y & z) != (x | y) & (x | z) for x=%r, y=%r, z=%r", (x, y, z))) + with self.subTest(x=x, y=y, z=z): + eq((x & y) & z, x & (y & z)) + eq((x | y) | z, x | (y | z)) + eq((x ^ y) ^ z, x ^ (y ^ z)) + eq(x & (y | z), (x & y) | (x & z)) + eq(x | (y & z), (x | y) & (x | z)) def test_bitop_identities(self): for x in special: @@ -324,11 +307,11 @@ def check_format_1(self, x): for base, mapper in (2, bin), (8, oct), (10, str), (10, repr), (16, hex): got = mapper(x) - expected = self.slow_format(x, base) - msg = Frm("%s returned %r but expected %r for %r", - mapper.__name__, got, expected, x) - self.assertEqual(got, expected, msg) - self.assertEqual(int(got, 0), x, Frm('int("%s", 0) != %r', got, x)) + with self.subTest(x=x, mapper=mapper.__name__): + expected = self.slow_format(x, base) + self.assertEqual(got, expected) + with self.subTest(got=got): + self.assertEqual(int(got, 0), x) def test_format(self): for x in special: @@ -625,14 +608,15 @@ for y in cases: Ry = Rat(y) Rcmp = (Rx > Ry) - (Rx < Ry) - xycmp = (x > y) - (x < y) - eq(Rcmp, xycmp, Frm("%r %r %d %d", x, y, Rcmp, xycmp)) - eq(x == y, Rcmp == 0, Frm("%r == %r %d", x, y, Rcmp)) - eq(x != y, Rcmp != 0, Frm("%r != %r %d", x, y, Rcmp)) - eq(x < y, Rcmp < 0, Frm("%r < %r %d", x, y, Rcmp)) - eq(x <= y, Rcmp <= 0, Frm("%r <= %r %d", x, y, Rcmp)) - eq(x > y, Rcmp > 0, Frm("%r > %r %d", x, y, Rcmp)) - eq(x >= y, Rcmp >= 0, Frm("%r >= %r %d", x, y, Rcmp)) + with self.subTest(x=x, y=y, Rcmp=Rcmp): + xycmp = (x > y) - (x < y) + eq(Rcmp, xycmp) + eq(x == y, Rcmp == 0) + eq(x != y, Rcmp != 0) + eq(x < y, Rcmp < 0) + eq(x <= y, Rcmp <= 0) + eq(x > y, Rcmp > 0) + eq(x >= y, Rcmp >= 0) def test__format__(self): self.assertEqual(format(123456789, 'd'), '123456789') -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 26 04:23:54 2015 From: python-checkins at python.org (terry.reedy) Date: Sat, 26 Sep 2015 02:23:54 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzI1MTcz?= =?utf-8?q?=3A_Replace_=27master=27_with_=27parent=27_in_tkinter=2Emessage?= =?utf-8?q?box_calls=2E?= Message-ID: <20150926022354.3660.82136@psf.io> https://hg.python.org/cpython/rev/e494316a9291 changeset: 98270:e494316a9291 branch: 3.4 parent: 98266:697781ff3b49 user: Terry Jan Reedy date: Fri Sep 25 22:22:55 2015 -0400 summary: Issue #25173: Replace 'master' with 'parent' in tkinter.messagebox calls. This associates the message box with the widget and is better for Mac OSX. Patch by Mark Roseman. files: Lib/idlelib/IOBinding.py | 20 ++++++++++---------- Lib/idlelib/OutputWindow.py | 2 +- Lib/idlelib/PyShell.py | 12 ++++++------ Lib/idlelib/ScriptBinding.py | 4 ++-- Lib/idlelib/run.py | 2 +- 5 files changed, 20 insertions(+), 20 deletions(-) diff --git a/Lib/idlelib/IOBinding.py b/Lib/idlelib/IOBinding.py --- a/Lib/idlelib/IOBinding.py +++ b/Lib/idlelib/IOBinding.py @@ -217,7 +217,7 @@ f.seek(0) bytes = f.read() except OSError as msg: - tkMessageBox.showerror("I/O Error", str(msg), master=self.text) + tkMessageBox.showerror("I/O Error", str(msg), parent=self.text) return False chars, converted = self._decode(two_lines, bytes) if chars is None: @@ -266,7 +266,7 @@ title="Error loading the file", message="The encoding '%s' is not known to this Python "\ "installation. The file may not display correctly" % name, - master = self.text) + parent = self.text) enc = None except UnicodeDecodeError: return None, False @@ -321,7 +321,7 @@ title="Save On Close", message=message, default=tkMessageBox.YES, - master=self.text) + parent=self.text) if confirm: reply = "yes" self.save(None) @@ -381,7 +381,7 @@ return True except OSError as msg: tkMessageBox.showerror("I/O Error", str(msg), - master=self.text) + parent=self.text) return False def encode(self, chars): @@ -418,7 +418,7 @@ tkMessageBox.showerror( "I/O Error", "%s.\nSaving as UTF-8" % failed, - master = self.text) + parent = self.text) # Fallback: save as UTF-8, with BOM - ignoring the incorrect # declared encoding return BOM_UTF8 + chars.encode("utf-8") @@ -433,7 +433,7 @@ title="Print", message="Print to Default Printer", default=tkMessageBox.OK, - master=self.text) + parent=self.text) if not confirm: self.text.focus_set() return "break" @@ -470,10 +470,10 @@ status + output if output: output = "Printing command: %s\n" % repr(command) + output - tkMessageBox.showerror("Print status", output, master=self.text) + tkMessageBox.showerror("Print status", output, parent=self.text) else: #no printing for this platform message = "Printing is not enabled for this platform: %s" % platform - tkMessageBox.showinfo("Print status", message, master=self.text) + tkMessageBox.showinfo("Print status", message, parent=self.text) if tempfilename: os.unlink(tempfilename) return "break" @@ -492,7 +492,7 @@ def askopenfile(self): dir, base = self.defaultfilename("open") if not self.opendialog: - self.opendialog = tkFileDialog.Open(master=self.text, + self.opendialog = tkFileDialog.Open(parent=self.text, filetypes=self.filetypes) filename = self.opendialog.show(initialdir=dir, initialfile=base) return filename @@ -513,7 +513,7 @@ dir, base = self.defaultfilename("save") if not self.savedialog: self.savedialog = tkFileDialog.SaveAs( - master=self.text, + parent=self.text, filetypes=self.filetypes, defaultextension=self.defaultextension) filename = self.savedialog.show(initialdir=dir, initialfile=base) diff --git a/Lib/idlelib/OutputWindow.py b/Lib/idlelib/OutputWindow.py --- a/Lib/idlelib/OutputWindow.py +++ b/Lib/idlelib/OutputWindow.py @@ -91,7 +91,7 @@ "No special line", "The line you point at doesn't look like " "a valid file name followed by a line number.", - master=self.text) + parent=self.text) return filename, lineno = result edit = self.flist.open(filename) diff --git a/Lib/idlelib/PyShell.py b/Lib/idlelib/PyShell.py --- a/Lib/idlelib/PyShell.py +++ b/Lib/idlelib/PyShell.py @@ -774,7 +774,7 @@ "Exit?", "Do you want to exit altogether?", default="yes", - master=self.tkconsole.text): + parent=self.tkconsole.text): raise else: self.showtraceback() @@ -812,7 +812,7 @@ "Run IDLE with the -n command line switch to start without a " "subprocess and refer to Help/IDLE Help 'Running without a " "subprocess' for further details.", - master=self.tkconsole.text) + parent=self.tkconsole.text) def display_no_subprocess_error(self): tkMessageBox.showerror( @@ -820,14 +820,14 @@ "IDLE's subprocess didn't make connection. Either IDLE can't " "start a subprocess or personal firewall software is blocking " "the connection.", - master=self.tkconsole.text) + parent=self.tkconsole.text) def display_executing_dialog(self): tkMessageBox.showerror( "Already executing", "The Python Shell window is already executing a command; " "please wait until it is finished.", - master=self.tkconsole.text) + parent=self.tkconsole.text) class PyShell(OutputWindow): @@ -931,7 +931,7 @@ if self.executing: tkMessageBox.showerror("Don't debug now", "You can only toggle the debugger when idle", - master=self.text) + parent=self.text) self.set_debugger_indicator() return "break" else: @@ -1239,7 +1239,7 @@ tkMessageBox.showerror("No stack trace", "There is no stack trace yet.\n" "(sys.last_traceback is not defined)", - master=self.text) + parent=self.text) return from idlelib.StackViewer import StackBrowser StackBrowser(self.root, self.flist) diff --git a/Lib/idlelib/ScriptBinding.py b/Lib/idlelib/ScriptBinding.py --- a/Lib/idlelib/ScriptBinding.py +++ b/Lib/idlelib/ScriptBinding.py @@ -197,10 +197,10 @@ confirm = tkMessageBox.askokcancel(title="Save Before Run or Check", message=msg, default=tkMessageBox.OK, - master=self.editwin.text) + parent=self.editwin.text) return confirm def errorbox(self, title, message): # XXX This should really be a function of EditorWindow... - tkMessageBox.showerror(title, message, master=self.editwin.text) + tkMessageBox.showerror(title, message, parent=self.editwin.text) self.editwin.text.focus_set() diff --git a/Lib/idlelib/run.py b/Lib/idlelib/run.py --- a/Lib/idlelib/run.py +++ b/Lib/idlelib/run.py @@ -174,7 +174,7 @@ tkMessageBox.showerror("IDLE Subprocess Error", msg, parent=root) else: tkMessageBox.showerror("IDLE Subprocess Error", - "Socket Error: %s" % err.args[1]) + "Socket Error: %s" % err.args[1], parent=root) root.destroy() def print_exception(): -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 26 04:23:54 2015 From: python-checkins at python.org (terry.reedy) Date: Sat, 26 Sep 2015 02:23:54 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Merge_with_3=2E4?= Message-ID: <20150926022354.11684.1314@psf.io> https://hg.python.org/cpython/rev/af8d79810cb6 changeset: 98271:af8d79810cb6 branch: 3.5 parent: 98267:6e11708dcb3b parent: 98270:e494316a9291 user: Terry Jan Reedy date: Fri Sep 25 22:23:19 2015 -0400 summary: Merge with 3.4 files: Lib/idlelib/IOBinding.py | 20 ++++++++++---------- Lib/idlelib/OutputWindow.py | 2 +- Lib/idlelib/PyShell.py | 12 ++++++------ Lib/idlelib/ScriptBinding.py | 4 ++-- Lib/idlelib/run.py | 2 +- 5 files changed, 20 insertions(+), 20 deletions(-) diff --git a/Lib/idlelib/IOBinding.py b/Lib/idlelib/IOBinding.py --- a/Lib/idlelib/IOBinding.py +++ b/Lib/idlelib/IOBinding.py @@ -217,7 +217,7 @@ f.seek(0) bytes = f.read() except OSError as msg: - tkMessageBox.showerror("I/O Error", str(msg), master=self.text) + tkMessageBox.showerror("I/O Error", str(msg), parent=self.text) return False chars, converted = self._decode(two_lines, bytes) if chars is None: @@ -266,7 +266,7 @@ title="Error loading the file", message="The encoding '%s' is not known to this Python "\ "installation. The file may not display correctly" % name, - master = self.text) + parent = self.text) enc = None except UnicodeDecodeError: return None, False @@ -321,7 +321,7 @@ title="Save On Close", message=message, default=tkMessageBox.YES, - master=self.text) + parent=self.text) if confirm: reply = "yes" self.save(None) @@ -381,7 +381,7 @@ return True except OSError as msg: tkMessageBox.showerror("I/O Error", str(msg), - master=self.text) + parent=self.text) return False def encode(self, chars): @@ -418,7 +418,7 @@ tkMessageBox.showerror( "I/O Error", "%s.\nSaving as UTF-8" % failed, - master = self.text) + parent = self.text) # Fallback: save as UTF-8, with BOM - ignoring the incorrect # declared encoding return BOM_UTF8 + chars.encode("utf-8") @@ -433,7 +433,7 @@ title="Print", message="Print to Default Printer", default=tkMessageBox.OK, - master=self.text) + parent=self.text) if not confirm: self.text.focus_set() return "break" @@ -470,10 +470,10 @@ status + output if output: output = "Printing command: %s\n" % repr(command) + output - tkMessageBox.showerror("Print status", output, master=self.text) + tkMessageBox.showerror("Print status", output, parent=self.text) else: #no printing for this platform message = "Printing is not enabled for this platform: %s" % platform - tkMessageBox.showinfo("Print status", message, master=self.text) + tkMessageBox.showinfo("Print status", message, parent=self.text) if tempfilename: os.unlink(tempfilename) return "break" @@ -492,7 +492,7 @@ def askopenfile(self): dir, base = self.defaultfilename("open") if not self.opendialog: - self.opendialog = tkFileDialog.Open(master=self.text, + self.opendialog = tkFileDialog.Open(parent=self.text, filetypes=self.filetypes) filename = self.opendialog.show(initialdir=dir, initialfile=base) return filename @@ -513,7 +513,7 @@ dir, base = self.defaultfilename("save") if not self.savedialog: self.savedialog = tkFileDialog.SaveAs( - master=self.text, + parent=self.text, filetypes=self.filetypes, defaultextension=self.defaultextension) filename = self.savedialog.show(initialdir=dir, initialfile=base) diff --git a/Lib/idlelib/OutputWindow.py b/Lib/idlelib/OutputWindow.py --- a/Lib/idlelib/OutputWindow.py +++ b/Lib/idlelib/OutputWindow.py @@ -91,7 +91,7 @@ "No special line", "The line you point at doesn't look like " "a valid file name followed by a line number.", - master=self.text) + parent=self.text) return filename, lineno = result edit = self.flist.open(filename) diff --git a/Lib/idlelib/PyShell.py b/Lib/idlelib/PyShell.py --- a/Lib/idlelib/PyShell.py +++ b/Lib/idlelib/PyShell.py @@ -774,7 +774,7 @@ "Exit?", "Do you want to exit altogether?", default="yes", - master=self.tkconsole.text): + parent=self.tkconsole.text): raise else: self.showtraceback() @@ -812,7 +812,7 @@ "Run IDLE with the -n command line switch to start without a " "subprocess and refer to Help/IDLE Help 'Running without a " "subprocess' for further details.", - master=self.tkconsole.text) + parent=self.tkconsole.text) def display_no_subprocess_error(self): tkMessageBox.showerror( @@ -820,14 +820,14 @@ "IDLE's subprocess didn't make connection. Either IDLE can't " "start a subprocess or personal firewall software is blocking " "the connection.", - master=self.tkconsole.text) + parent=self.tkconsole.text) def display_executing_dialog(self): tkMessageBox.showerror( "Already executing", "The Python Shell window is already executing a command; " "please wait until it is finished.", - master=self.tkconsole.text) + parent=self.tkconsole.text) class PyShell(OutputWindow): @@ -931,7 +931,7 @@ if self.executing: tkMessageBox.showerror("Don't debug now", "You can only toggle the debugger when idle", - master=self.text) + parent=self.text) self.set_debugger_indicator() return "break" else: @@ -1239,7 +1239,7 @@ tkMessageBox.showerror("No stack trace", "There is no stack trace yet.\n" "(sys.last_traceback is not defined)", - master=self.text) + parent=self.text) return from idlelib.StackViewer import StackBrowser StackBrowser(self.root, self.flist) diff --git a/Lib/idlelib/ScriptBinding.py b/Lib/idlelib/ScriptBinding.py --- a/Lib/idlelib/ScriptBinding.py +++ b/Lib/idlelib/ScriptBinding.py @@ -197,10 +197,10 @@ confirm = tkMessageBox.askokcancel(title="Save Before Run or Check", message=msg, default=tkMessageBox.OK, - master=self.editwin.text) + parent=self.editwin.text) return confirm def errorbox(self, title, message): # XXX This should really be a function of EditorWindow... - tkMessageBox.showerror(title, message, master=self.editwin.text) + tkMessageBox.showerror(title, message, parent=self.editwin.text) self.editwin.text.focus_set() diff --git a/Lib/idlelib/run.py b/Lib/idlelib/run.py --- a/Lib/idlelib/run.py +++ b/Lib/idlelib/run.py @@ -174,7 +174,7 @@ tkMessageBox.showerror("IDLE Subprocess Error", msg, parent=root) else: tkMessageBox.showerror("IDLE Subprocess Error", - "Socket Error: %s" % err.args[1]) + "Socket Error: %s" % err.args[1], parent=root) root.destroy() def print_exception(): -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 26 04:23:54 2015 From: python-checkins at python.org (terry.reedy) Date: Sat, 26 Sep 2015 02:23:54 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzI1MTcz?= =?utf-8?q?=3A_Replace_=27master=27_with_=27parent=27_in_tkinter=2Emessage?= =?utf-8?q?box_calls=2E?= Message-ID: <20150926022354.11714.61876@psf.io> https://hg.python.org/cpython/rev/e406d62014f7 changeset: 98269:e406d62014f7 branch: 2.7 parent: 98261:c1eccae07977 user: Terry Jan Reedy date: Fri Sep 25 22:22:48 2015 -0400 summary: Issue #25173: Replace 'master' with 'parent' in tkinter.messagebox calls. This associates the message box with the widget and is better for Mac OSX. Patch by Mark Roseman. files: Lib/idlelib/IOBinding.py | 22 +++++++++++----------- Lib/idlelib/OutputWindow.py | 2 +- Lib/idlelib/PyShell.py | 12 ++++++------ Lib/idlelib/ScriptBinding.py | 4 ++-- Lib/idlelib/run.py | 2 +- 5 files changed, 21 insertions(+), 21 deletions(-) diff --git a/Lib/idlelib/IOBinding.py b/Lib/idlelib/IOBinding.py --- a/Lib/idlelib/IOBinding.py +++ b/Lib/idlelib/IOBinding.py @@ -250,7 +250,7 @@ with open(filename, 'rb') as f: chars = f.read() except IOError as msg: - tkMessageBox.showerror("I/O Error", str(msg), master=self.text) + tkMessageBox.showerror("I/O Error", str(msg), parent=self.text) return False chars = self.decode(chars) @@ -297,7 +297,7 @@ title="Error loading the file", message="The encoding '%s' is not known to this Python "\ "installation. The file may not display correctly" % name, - master = self.text) + parent = self.text) enc = None if enc: try: @@ -327,7 +327,7 @@ title="Save On Close", message=message, default=tkMessageBox.YES, - master=self.text) + parent=self.text) if confirm: reply = "yes" self.save(None) @@ -386,7 +386,7 @@ return True except IOError as msg: tkMessageBox.showerror("I/O Error", str(msg), - master=self.text) + parent=self.text) return False def encode(self, chars): @@ -416,7 +416,7 @@ tkMessageBox.showerror( "I/O Error", "%s. Saving as UTF-8" % failed, - master = self.text) + parent = self.text) # If there was a UTF-8 signature, use that. This should not fail if self.fileencoding == BOM_UTF8 or failed: return BOM_UTF8 + chars.encode("utf-8") @@ -429,7 +429,7 @@ "I/O Error", "Cannot save this as '%s' anymore. Saving as UTF-8" \ % self.fileencoding, - master = self.text) + parent = self.text) return BOM_UTF8 + chars.encode("utf-8") # Nothing was declared, and we had not determined an encoding # on loading. Recommend an encoding line. @@ -473,7 +473,7 @@ title="Print", message="Print to Default Printer", default=tkMessageBox.OK, - master=self.text) + parent=self.text) if not confirm: self.text.focus_set() return "break" @@ -510,10 +510,10 @@ status + output if output: output = "Printing command: %s\n" % repr(command) + output - tkMessageBox.showerror("Print status", output, master=self.text) + tkMessageBox.showerror("Print status", output, parent=self.text) else: #no printing for this platform message = "Printing is not enabled for this platform: %s" % platform - tkMessageBox.showinfo("Print status", message, master=self.text) + tkMessageBox.showinfo("Print status", message, parent=self.text) if tempfilename: os.unlink(tempfilename) return "break" @@ -532,7 +532,7 @@ def askopenfile(self): dir, base = self.defaultfilename("open") if not self.opendialog: - self.opendialog = tkFileDialog.Open(master=self.text, + self.opendialog = tkFileDialog.Open(parent=self.text, filetypes=self.filetypes) filename = self.opendialog.show(initialdir=dir, initialfile=base) if isinstance(filename, unicode): @@ -555,7 +555,7 @@ dir, base = self.defaultfilename("save") if not self.savedialog: self.savedialog = tkFileDialog.SaveAs( - master=self.text, + parent=self.text, filetypes=self.filetypes, defaultextension=self.defaultextension) filename = self.savedialog.show(initialdir=dir, initialfile=base) diff --git a/Lib/idlelib/OutputWindow.py b/Lib/idlelib/OutputWindow.py --- a/Lib/idlelib/OutputWindow.py +++ b/Lib/idlelib/OutputWindow.py @@ -96,7 +96,7 @@ "No special line", "The line you point at doesn't look like " "a valid file name followed by a line number.", - master=self.text) + parent=self.text) return filename, lineno = result edit = self.flist.open(filename) diff --git a/Lib/idlelib/PyShell.py b/Lib/idlelib/PyShell.py --- a/Lib/idlelib/PyShell.py +++ b/Lib/idlelib/PyShell.py @@ -799,7 +799,7 @@ "Exit?", "Do you want to exit altogether?", default="yes", - master=self.tkconsole.text): + parent=self.tkconsole.text): raise else: self.showtraceback() @@ -837,7 +837,7 @@ "Run IDLE with the -n command line switch to start without a " "subprocess and refer to Help/IDLE Help 'Running without a " "subprocess' for further details.", - master=self.tkconsole.text) + parent=self.tkconsole.text) def display_no_subprocess_error(self): tkMessageBox.showerror( @@ -845,14 +845,14 @@ "IDLE's subprocess didn't make connection. Either IDLE can't " "start a subprocess or personal firewall software is blocking " "the connection.", - master=self.tkconsole.text) + parent=self.tkconsole.text) def display_executing_dialog(self): tkMessageBox.showerror( "Already executing", "The Python Shell window is already executing a command; " "please wait until it is finished.", - master=self.tkconsole.text) + parent=self.tkconsole.text) class PyShell(OutputWindow): @@ -948,7 +948,7 @@ if self.executing: tkMessageBox.showerror("Don't debug now", "You can only toggle the debugger when idle", - master=self.text) + parent=self.text) self.set_debugger_indicator() return "break" else: @@ -1256,7 +1256,7 @@ tkMessageBox.showerror("No stack trace", "There is no stack trace yet.\n" "(sys.last_traceback is not defined)", - master=self.text) + parent=self.text) return from idlelib.StackViewer import StackBrowser StackBrowser(self.root, self.flist) diff --git a/Lib/idlelib/ScriptBinding.py b/Lib/idlelib/ScriptBinding.py --- a/Lib/idlelib/ScriptBinding.py +++ b/Lib/idlelib/ScriptBinding.py @@ -213,10 +213,10 @@ confirm = tkMessageBox.askokcancel(title="Save Before Run or Check", message=msg, default=tkMessageBox.OK, - master=self.editwin.text) + parent=self.editwin.text) return confirm def errorbox(self, title, message): # XXX This should really be a function of EditorWindow... - tkMessageBox.showerror(title, message, master=self.editwin.text) + tkMessageBox.showerror(title, message, parent=self.editwin.text) self.editwin.text.focus_set() diff --git a/Lib/idlelib/run.py b/Lib/idlelib/run.py --- a/Lib/idlelib/run.py +++ b/Lib/idlelib/run.py @@ -164,7 +164,7 @@ tkMessageBox.showerror("IDLE Subprocess Error", msg, parent=root) else: tkMessageBox.showerror("IDLE Subprocess Error", - "Socket Error: %s" % err.args[1]) + "Socket Error: %s" % err.args[1], parent=root) root.destroy() def print_exception(): -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 26 04:23:55 2015 From: python-checkins at python.org (terry.reedy) Date: Sat, 26 Sep 2015 02:23:55 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge_with_3=2E5?= Message-ID: <20150926022354.3656.51048@psf.io> https://hg.python.org/cpython/rev/50a79646044b changeset: 98272:50a79646044b parent: 98268:11ad44d4177f parent: 98271:af8d79810cb6 user: Terry Jan Reedy date: Fri Sep 25 22:23:33 2015 -0400 summary: Merge with 3.5 files: Lib/idlelib/IOBinding.py | 20 ++++++++++---------- Lib/idlelib/OutputWindow.py | 2 +- Lib/idlelib/PyShell.py | 12 ++++++------ Lib/idlelib/ScriptBinding.py | 4 ++-- Lib/idlelib/run.py | 2 +- 5 files changed, 20 insertions(+), 20 deletions(-) diff --git a/Lib/idlelib/IOBinding.py b/Lib/idlelib/IOBinding.py --- a/Lib/idlelib/IOBinding.py +++ b/Lib/idlelib/IOBinding.py @@ -217,7 +217,7 @@ f.seek(0) bytes = f.read() except OSError as msg: - tkMessageBox.showerror("I/O Error", str(msg), master=self.text) + tkMessageBox.showerror("I/O Error", str(msg), parent=self.text) return False chars, converted = self._decode(two_lines, bytes) if chars is None: @@ -266,7 +266,7 @@ title="Error loading the file", message="The encoding '%s' is not known to this Python "\ "installation. The file may not display correctly" % name, - master = self.text) + parent = self.text) enc = None except UnicodeDecodeError: return None, False @@ -321,7 +321,7 @@ title="Save On Close", message=message, default=tkMessageBox.YES, - master=self.text) + parent=self.text) if confirm: reply = "yes" self.save(None) @@ -381,7 +381,7 @@ return True except OSError as msg: tkMessageBox.showerror("I/O Error", str(msg), - master=self.text) + parent=self.text) return False def encode(self, chars): @@ -418,7 +418,7 @@ tkMessageBox.showerror( "I/O Error", "%s.\nSaving as UTF-8" % failed, - master = self.text) + parent = self.text) # Fallback: save as UTF-8, with BOM - ignoring the incorrect # declared encoding return BOM_UTF8 + chars.encode("utf-8") @@ -433,7 +433,7 @@ title="Print", message="Print to Default Printer", default=tkMessageBox.OK, - master=self.text) + parent=self.text) if not confirm: self.text.focus_set() return "break" @@ -470,10 +470,10 @@ status + output if output: output = "Printing command: %s\n" % repr(command) + output - tkMessageBox.showerror("Print status", output, master=self.text) + tkMessageBox.showerror("Print status", output, parent=self.text) else: #no printing for this platform message = "Printing is not enabled for this platform: %s" % platform - tkMessageBox.showinfo("Print status", message, master=self.text) + tkMessageBox.showinfo("Print status", message, parent=self.text) if tempfilename: os.unlink(tempfilename) return "break" @@ -492,7 +492,7 @@ def askopenfile(self): dir, base = self.defaultfilename("open") if not self.opendialog: - self.opendialog = tkFileDialog.Open(master=self.text, + self.opendialog = tkFileDialog.Open(parent=self.text, filetypes=self.filetypes) filename = self.opendialog.show(initialdir=dir, initialfile=base) return filename @@ -513,7 +513,7 @@ dir, base = self.defaultfilename("save") if not self.savedialog: self.savedialog = tkFileDialog.SaveAs( - master=self.text, + parent=self.text, filetypes=self.filetypes, defaultextension=self.defaultextension) filename = self.savedialog.show(initialdir=dir, initialfile=base) diff --git a/Lib/idlelib/OutputWindow.py b/Lib/idlelib/OutputWindow.py --- a/Lib/idlelib/OutputWindow.py +++ b/Lib/idlelib/OutputWindow.py @@ -91,7 +91,7 @@ "No special line", "The line you point at doesn't look like " "a valid file name followed by a line number.", - master=self.text) + parent=self.text) return filename, lineno = result edit = self.flist.open(filename) diff --git a/Lib/idlelib/PyShell.py b/Lib/idlelib/PyShell.py --- a/Lib/idlelib/PyShell.py +++ b/Lib/idlelib/PyShell.py @@ -774,7 +774,7 @@ "Exit?", "Do you want to exit altogether?", default="yes", - master=self.tkconsole.text): + parent=self.tkconsole.text): raise else: self.showtraceback() @@ -812,7 +812,7 @@ "Run IDLE with the -n command line switch to start without a " "subprocess and refer to Help/IDLE Help 'Running without a " "subprocess' for further details.", - master=self.tkconsole.text) + parent=self.tkconsole.text) def display_no_subprocess_error(self): tkMessageBox.showerror( @@ -820,14 +820,14 @@ "IDLE's subprocess didn't make connection. Either IDLE can't " "start a subprocess or personal firewall software is blocking " "the connection.", - master=self.tkconsole.text) + parent=self.tkconsole.text) def display_executing_dialog(self): tkMessageBox.showerror( "Already executing", "The Python Shell window is already executing a command; " "please wait until it is finished.", - master=self.tkconsole.text) + parent=self.tkconsole.text) class PyShell(OutputWindow): @@ -931,7 +931,7 @@ if self.executing: tkMessageBox.showerror("Don't debug now", "You can only toggle the debugger when idle", - master=self.text) + parent=self.text) self.set_debugger_indicator() return "break" else: @@ -1239,7 +1239,7 @@ tkMessageBox.showerror("No stack trace", "There is no stack trace yet.\n" "(sys.last_traceback is not defined)", - master=self.text) + parent=self.text) return from idlelib.StackViewer import StackBrowser StackBrowser(self.root, self.flist) diff --git a/Lib/idlelib/ScriptBinding.py b/Lib/idlelib/ScriptBinding.py --- a/Lib/idlelib/ScriptBinding.py +++ b/Lib/idlelib/ScriptBinding.py @@ -197,10 +197,10 @@ confirm = tkMessageBox.askokcancel(title="Save Before Run or Check", message=msg, default=tkMessageBox.OK, - master=self.editwin.text) + parent=self.editwin.text) return confirm def errorbox(self, title, message): # XXX This should really be a function of EditorWindow... - tkMessageBox.showerror(title, message, master=self.editwin.text) + tkMessageBox.showerror(title, message, parent=self.editwin.text) self.editwin.text.focus_set() diff --git a/Lib/idlelib/run.py b/Lib/idlelib/run.py --- a/Lib/idlelib/run.py +++ b/Lib/idlelib/run.py @@ -174,7 +174,7 @@ tkMessageBox.showerror("IDLE Subprocess Error", msg, parent=root) else: tkMessageBox.showerror("IDLE Subprocess Error", - "Socket Error: %s" % err.args[1]) + "Socket Error: %s" % err.args[1], parent=root) root.destroy() def print_exception(): -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 26 07:46:21 2015 From: python-checkins at python.org (benjamin.peterson) Date: Sat, 26 Sep 2015 05:46:21 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_make_opening_b?= =?utf-8?q?race_of_container_literals_and_comprehensions_correspond_to_the?= Message-ID: <20150926054621.115545.22193@psf.io> https://hg.python.org/cpython/rev/4f14afc959df changeset: 98273:4f14afc959df branch: 3.5 parent: 98271:af8d79810cb6 user: Benjamin Peterson date: Fri Sep 25 22:44:43 2015 -0700 summary: make opening brace of container literals and comprehensions correspond to the line number and col offset of the AST node (closes #25131) files: Lib/test/test_ast.py | 18 +++++++++--------- Misc/NEWS | 3 +++ Python/ast.c | 14 ++++++++++---- 3 files changed, 22 insertions(+), 13 deletions(-) diff --git a/Lib/test/test_ast.py b/Lib/test/test_ast.py --- a/Lib/test/test_ast.py +++ b/Lib/test/test_ast.py @@ -983,15 +983,15 @@ ('Module', [('Expr', (1, 0), ('GeneratorExp', (1, 1), ('Tuple', (1, 2), [('Name', (1, 2), 'a', ('Load',)), ('Name', (1, 4), 'b', ('Load',))], ('Load',)), [('comprehension', ('Tuple', (1, 11), [('Name', (1, 11), 'a', ('Store',)), ('Name', (1, 13), 'b', ('Store',))], ('Store',)), ('Name', (1, 18), 'c', ('Load',)), [])]))]), ('Module', [('Expr', (1, 0), ('GeneratorExp', (1, 1), ('Tuple', (1, 2), [('Name', (1, 2), 'a', ('Load',)), ('Name', (1, 4), 'b', ('Load',))], ('Load',)), [('comprehension', ('Tuple', (1, 12), [('Name', (1, 12), 'a', ('Store',)), ('Name', (1, 14), 'b', ('Store',))], ('Store',)), ('Name', (1, 20), 'c', ('Load',)), [])]))]), ('Module', [('Expr', (1, 0), ('GeneratorExp', (2, 4), ('Tuple', (3, 4), [('Name', (3, 4), 'Aa', ('Load',)), ('Name', (5, 7), 'Bb', ('Load',))], ('Load',)), [('comprehension', ('Tuple', (8, 4), [('Name', (8, 4), 'Aa', ('Store',)), ('Name', (10, 4), 'Bb', ('Store',))], ('Store',)), ('Name', (10, 10), 'Cc', ('Load',)), [])]))]), -('Module', [('Expr', (1, 0), ('DictComp', (1, 1), ('Name', (1, 1), 'a', ('Load',)), ('Name', (1, 5), 'b', ('Load',)), [('comprehension', ('Name', (1, 11), 'w', ('Store',)), ('Name', (1, 16), 'x', ('Load',)), []), ('comprehension', ('Name', (1, 22), 'm', ('Store',)), ('Name', (1, 27), 'p', ('Load',)), [('Name', (1, 32), 'g', ('Load',))])]))]), -('Module', [('Expr', (1, 0), ('DictComp', (1, 1), ('Name', (1, 1), 'a', ('Load',)), ('Name', (1, 5), 'b', ('Load',)), [('comprehension', ('Tuple', (1, 11), [('Name', (1, 11), 'v', ('Store',)), ('Name', (1, 13), 'w', ('Store',))], ('Store',)), ('Name', (1, 18), 'x', ('Load',)), [])]))]), -('Module', [('Expr', (1, 0), ('SetComp', (1, 1), ('Name', (1, 1), 'r', ('Load',)), [('comprehension', ('Name', (1, 7), 'l', ('Store',)), ('Name', (1, 12), 'x', ('Load',)), [('Name', (1, 17), 'g', ('Load',))])]))]), -('Module', [('Expr', (1, 0), ('SetComp', (1, 1), ('Name', (1, 1), 'r', ('Load',)), [('comprehension', ('Tuple', (1, 7), [('Name', (1, 7), 'l', ('Store',)), ('Name', (1, 9), 'm', ('Store',))], ('Store',)), ('Name', (1, 14), 'x', ('Load',)), [])]))]), +('Module', [('Expr', (1, 0), ('DictComp', (1, 0), ('Name', (1, 1), 'a', ('Load',)), ('Name', (1, 5), 'b', ('Load',)), [('comprehension', ('Name', (1, 11), 'w', ('Store',)), ('Name', (1, 16), 'x', ('Load',)), []), ('comprehension', ('Name', (1, 22), 'm', ('Store',)), ('Name', (1, 27), 'p', ('Load',)), [('Name', (1, 32), 'g', ('Load',))])]))]), +('Module', [('Expr', (1, 0), ('DictComp', (1, 0), ('Name', (1, 1), 'a', ('Load',)), ('Name', (1, 5), 'b', ('Load',)), [('comprehension', ('Tuple', (1, 11), [('Name', (1, 11), 'v', ('Store',)), ('Name', (1, 13), 'w', ('Store',))], ('Store',)), ('Name', (1, 18), 'x', ('Load',)), [])]))]), +('Module', [('Expr', (1, 0), ('SetComp', (1, 0), ('Name', (1, 1), 'r', ('Load',)), [('comprehension', ('Name', (1, 7), 'l', ('Store',)), ('Name', (1, 12), 'x', ('Load',)), [('Name', (1, 17), 'g', ('Load',))])]))]), +('Module', [('Expr', (1, 0), ('SetComp', (1, 0), ('Name', (1, 1), 'r', ('Load',)), [('comprehension', ('Tuple', (1, 7), [('Name', (1, 7), 'l', ('Store',)), ('Name', (1, 9), 'm', ('Store',))], ('Store',)), ('Name', (1, 14), 'x', ('Load',)), [])]))]), ('Module', [('AsyncFunctionDef', (1, 6), 'f', ('arguments', [], None, [], [], None, []), [('Expr', (2, 1), ('Await', (2, 1), ('Call', (2, 7), ('Name', (2, 7), 'something', ('Load',)), [], [])))], [], None)]), ('Module', [('AsyncFunctionDef', (1, 6), 'f', ('arguments', [], None, [], [], None, []), [('AsyncFor', (2, 7), ('Name', (2, 11), 'e', ('Store',)), ('Name', (2, 16), 'i', ('Load',)), [('Expr', (2, 19), ('Num', (2, 19), 1))], [('Expr', (3, 7), ('Num', (3, 7), 2))])], [], None)]), ('Module', [('AsyncFunctionDef', (1, 6), 'f', ('arguments', [], None, [], [], None, []), [('AsyncWith', (2, 7), [('withitem', ('Name', (2, 12), 'a', ('Load',)), ('Name', (2, 17), 'b', ('Store',)))], [('Expr', (2, 20), ('Num', (2, 20), 1))])], [], None)]), -('Module', [('Expr', (1, 0), ('Dict', (1, 1), [None, ('Num', (1, 10), 2)], [('Dict', (1, 4), [('Num', (1, 4), 1)], [('Num', (1, 6), 2)]), ('Num', (1, 12), 3)]))]), -('Module', [('Expr', (1, 0), ('Set', (1, 1), [('Starred', (1, 1), ('Set', (1, 3), [('Num', (1, 3), 1), ('Num', (1, 6), 2)]), ('Load',)), ('Num', (1, 10), 3)]))]), +('Module', [('Expr', (1, 0), ('Dict', (1, 0), [None, ('Num', (1, 10), 2)], [('Dict', (1, 3), [('Num', (1, 4), 1)], [('Num', (1, 6), 2)]), ('Num', (1, 12), 3)]))]), +('Module', [('Expr', (1, 0), ('Set', (1, 0), [('Starred', (1, 1), ('Set', (1, 2), [('Num', (1, 3), 1), ('Num', (1, 6), 2)]), ('Load',)), ('Num', (1, 10), 3)]))]), ] single_results = [ ('Interactive', [('Expr', (1, 0), ('BinOp', (1, 0), ('Num', (1, 0), 1), ('Add',), ('Num', (1, 2), 2)))]), @@ -1002,10 +1002,10 @@ ('Expression', ('BinOp', (1, 0), ('Name', (1, 0), 'a', ('Load',)), ('Add',), ('Name', (1, 4), 'b', ('Load',)))), ('Expression', ('UnaryOp', (1, 0), ('Not',), ('Name', (1, 4), 'v', ('Load',)))), ('Expression', ('Lambda', (1, 0), ('arguments', [], None, [], [], None, []), ('NameConstant', (1, 7), None))), -('Expression', ('Dict', (1, 2), [('Num', (1, 2), 1)], [('Num', (1, 4), 2)])), +('Expression', ('Dict', (1, 0), [('Num', (1, 2), 1)], [('Num', (1, 4), 2)])), ('Expression', ('Dict', (1, 0), [], [])), -('Expression', ('Set', (1, 1), [('NameConstant', (1, 1), None)])), -('Expression', ('Dict', (2, 6), [('Num', (2, 6), 1)], [('Num', (4, 10), 2)])), +('Expression', ('Set', (1, 0), [('NameConstant', (1, 1), None)])), +('Expression', ('Dict', (1, 0), [('Num', (2, 6), 1)], [('Num', (4, 10), 2)])), ('Expression', ('ListComp', (1, 1), ('Name', (1, 1), 'a', ('Load',)), [('comprehension', ('Name', (1, 7), 'b', ('Store',)), ('Name', (1, 12), 'c', ('Load',)), [('Name', (1, 17), 'd', ('Load',))])])), ('Expression', ('GeneratorExp', (1, 1), ('Name', (1, 1), 'a', ('Load',)), [('comprehension', ('Name', (1, 7), 'b', ('Store',)), ('Name', (1, 12), 'c', ('Load',)), [('Name', (1, 17), 'd', ('Load',))])])), ('Expression', ('Compare', (1, 0), ('Num', (1, 0), 1), [('Lt',), ('Lt',)], [('Num', (1, 4), 2), ('Num', (1, 8), 3)])), diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -11,6 +11,9 @@ Core and Builtins ----------------- +- Issue #25131: Make the line number and column offset of set/dict literals and + comprehensions correspond to the opening brace. + - Issue #25150: Hide the private _Py_atomic_xxx symbols from the public Python.h header to fix a compilation error with OpenMP. PyThreadState_GET() becomes an alias to PyThreadState_Get() to avoid ABI incompatibilies. diff --git a/Python/ast.c b/Python/ast.c --- a/Python/ast.c +++ b/Python/ast.c @@ -2101,6 +2101,7 @@ * (comp_for | (',' (test ':' test | '**' test))* [','])) | * ((test | '*' test) * (comp_for | (',' (test | '*' test))* [','])) ) */ + expr_ty res; ch = CHILD(n, 1); if (TYPE(ch) == RBRACE) { /* It's an empty dict. */ @@ -2112,12 +2113,12 @@ (NCH(ch) > 1 && TYPE(CHILD(ch, 1)) == COMMA)) { /* It's a set display. */ - return ast_for_setdisplay(c, ch); + res = ast_for_setdisplay(c, ch); } else if (NCH(ch) > 1 && TYPE(CHILD(ch, 1)) == comp_for) { /* It's a set comprehension. */ - return ast_for_setcomp(c, ch); + res = ast_for_setcomp(c, ch); } else if (NCH(ch) > 3 - is_dict && TYPE(CHILD(ch, 3 - is_dict)) == comp_for) { @@ -2127,12 +2128,17 @@ "dict comprehension"); return NULL; } - return ast_for_dictcomp(c, ch); + res = ast_for_dictcomp(c, ch); } else { /* It's a dictionary display. */ - return ast_for_dictdisplay(c, ch); + res = ast_for_dictdisplay(c, ch); } + if (res) { + res->lineno = LINENO(n); + res->col_offset = n->n_col_offset; + } + return res; } } default: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 26 07:46:22 2015 From: python-checkins at python.org (benjamin.peterson) Date: Sat, 26 Sep 2015 05:46:22 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogbWVyZ2UgMy41ICgjMjUxMzEp?= Message-ID: <20150926054621.9933.89820@psf.io> https://hg.python.org/cpython/rev/9d895c09c08a changeset: 98274:9d895c09c08a parent: 98272:50a79646044b parent: 98273:4f14afc959df user: Benjamin Peterson date: Fri Sep 25 22:44:55 2015 -0700 summary: merge 3.5 (#25131) files: Lib/test/test_ast.py | 18 +++++++++--------- Misc/NEWS | 3 +++ Python/ast.c | 14 ++++++++++---- 3 files changed, 22 insertions(+), 13 deletions(-) diff --git a/Lib/test/test_ast.py b/Lib/test/test_ast.py --- a/Lib/test/test_ast.py +++ b/Lib/test/test_ast.py @@ -983,15 +983,15 @@ ('Module', [('Expr', (1, 0), ('GeneratorExp', (1, 1), ('Tuple', (1, 2), [('Name', (1, 2), 'a', ('Load',)), ('Name', (1, 4), 'b', ('Load',))], ('Load',)), [('comprehension', ('Tuple', (1, 11), [('Name', (1, 11), 'a', ('Store',)), ('Name', (1, 13), 'b', ('Store',))], ('Store',)), ('Name', (1, 18), 'c', ('Load',)), [])]))]), ('Module', [('Expr', (1, 0), ('GeneratorExp', (1, 1), ('Tuple', (1, 2), [('Name', (1, 2), 'a', ('Load',)), ('Name', (1, 4), 'b', ('Load',))], ('Load',)), [('comprehension', ('Tuple', (1, 12), [('Name', (1, 12), 'a', ('Store',)), ('Name', (1, 14), 'b', ('Store',))], ('Store',)), ('Name', (1, 20), 'c', ('Load',)), [])]))]), ('Module', [('Expr', (1, 0), ('GeneratorExp', (2, 4), ('Tuple', (3, 4), [('Name', (3, 4), 'Aa', ('Load',)), ('Name', (5, 7), 'Bb', ('Load',))], ('Load',)), [('comprehension', ('Tuple', (8, 4), [('Name', (8, 4), 'Aa', ('Store',)), ('Name', (10, 4), 'Bb', ('Store',))], ('Store',)), ('Name', (10, 10), 'Cc', ('Load',)), [])]))]), -('Module', [('Expr', (1, 0), ('DictComp', (1, 1), ('Name', (1, 1), 'a', ('Load',)), ('Name', (1, 5), 'b', ('Load',)), [('comprehension', ('Name', (1, 11), 'w', ('Store',)), ('Name', (1, 16), 'x', ('Load',)), []), ('comprehension', ('Name', (1, 22), 'm', ('Store',)), ('Name', (1, 27), 'p', ('Load',)), [('Name', (1, 32), 'g', ('Load',))])]))]), -('Module', [('Expr', (1, 0), ('DictComp', (1, 1), ('Name', (1, 1), 'a', ('Load',)), ('Name', (1, 5), 'b', ('Load',)), [('comprehension', ('Tuple', (1, 11), [('Name', (1, 11), 'v', ('Store',)), ('Name', (1, 13), 'w', ('Store',))], ('Store',)), ('Name', (1, 18), 'x', ('Load',)), [])]))]), -('Module', [('Expr', (1, 0), ('SetComp', (1, 1), ('Name', (1, 1), 'r', ('Load',)), [('comprehension', ('Name', (1, 7), 'l', ('Store',)), ('Name', (1, 12), 'x', ('Load',)), [('Name', (1, 17), 'g', ('Load',))])]))]), -('Module', [('Expr', (1, 0), ('SetComp', (1, 1), ('Name', (1, 1), 'r', ('Load',)), [('comprehension', ('Tuple', (1, 7), [('Name', (1, 7), 'l', ('Store',)), ('Name', (1, 9), 'm', ('Store',))], ('Store',)), ('Name', (1, 14), 'x', ('Load',)), [])]))]), +('Module', [('Expr', (1, 0), ('DictComp', (1, 0), ('Name', (1, 1), 'a', ('Load',)), ('Name', (1, 5), 'b', ('Load',)), [('comprehension', ('Name', (1, 11), 'w', ('Store',)), ('Name', (1, 16), 'x', ('Load',)), []), ('comprehension', ('Name', (1, 22), 'm', ('Store',)), ('Name', (1, 27), 'p', ('Load',)), [('Name', (1, 32), 'g', ('Load',))])]))]), +('Module', [('Expr', (1, 0), ('DictComp', (1, 0), ('Name', (1, 1), 'a', ('Load',)), ('Name', (1, 5), 'b', ('Load',)), [('comprehension', ('Tuple', (1, 11), [('Name', (1, 11), 'v', ('Store',)), ('Name', (1, 13), 'w', ('Store',))], ('Store',)), ('Name', (1, 18), 'x', ('Load',)), [])]))]), +('Module', [('Expr', (1, 0), ('SetComp', (1, 0), ('Name', (1, 1), 'r', ('Load',)), [('comprehension', ('Name', (1, 7), 'l', ('Store',)), ('Name', (1, 12), 'x', ('Load',)), [('Name', (1, 17), 'g', ('Load',))])]))]), +('Module', [('Expr', (1, 0), ('SetComp', (1, 0), ('Name', (1, 1), 'r', ('Load',)), [('comprehension', ('Tuple', (1, 7), [('Name', (1, 7), 'l', ('Store',)), ('Name', (1, 9), 'm', ('Store',))], ('Store',)), ('Name', (1, 14), 'x', ('Load',)), [])]))]), ('Module', [('AsyncFunctionDef', (1, 6), 'f', ('arguments', [], None, [], [], None, []), [('Expr', (2, 1), ('Await', (2, 1), ('Call', (2, 7), ('Name', (2, 7), 'something', ('Load',)), [], [])))], [], None)]), ('Module', [('AsyncFunctionDef', (1, 6), 'f', ('arguments', [], None, [], [], None, []), [('AsyncFor', (2, 7), ('Name', (2, 11), 'e', ('Store',)), ('Name', (2, 16), 'i', ('Load',)), [('Expr', (2, 19), ('Num', (2, 19), 1))], [('Expr', (3, 7), ('Num', (3, 7), 2))])], [], None)]), ('Module', [('AsyncFunctionDef', (1, 6), 'f', ('arguments', [], None, [], [], None, []), [('AsyncWith', (2, 7), [('withitem', ('Name', (2, 12), 'a', ('Load',)), ('Name', (2, 17), 'b', ('Store',)))], [('Expr', (2, 20), ('Num', (2, 20), 1))])], [], None)]), -('Module', [('Expr', (1, 0), ('Dict', (1, 1), [None, ('Num', (1, 10), 2)], [('Dict', (1, 4), [('Num', (1, 4), 1)], [('Num', (1, 6), 2)]), ('Num', (1, 12), 3)]))]), -('Module', [('Expr', (1, 0), ('Set', (1, 1), [('Starred', (1, 1), ('Set', (1, 3), [('Num', (1, 3), 1), ('Num', (1, 6), 2)]), ('Load',)), ('Num', (1, 10), 3)]))]), +('Module', [('Expr', (1, 0), ('Dict', (1, 0), [None, ('Num', (1, 10), 2)], [('Dict', (1, 3), [('Num', (1, 4), 1)], [('Num', (1, 6), 2)]), ('Num', (1, 12), 3)]))]), +('Module', [('Expr', (1, 0), ('Set', (1, 0), [('Starred', (1, 1), ('Set', (1, 2), [('Num', (1, 3), 1), ('Num', (1, 6), 2)]), ('Load',)), ('Num', (1, 10), 3)]))]), ] single_results = [ ('Interactive', [('Expr', (1, 0), ('BinOp', (1, 0), ('Num', (1, 0), 1), ('Add',), ('Num', (1, 2), 2)))]), @@ -1002,10 +1002,10 @@ ('Expression', ('BinOp', (1, 0), ('Name', (1, 0), 'a', ('Load',)), ('Add',), ('Name', (1, 4), 'b', ('Load',)))), ('Expression', ('UnaryOp', (1, 0), ('Not',), ('Name', (1, 4), 'v', ('Load',)))), ('Expression', ('Lambda', (1, 0), ('arguments', [], None, [], [], None, []), ('NameConstant', (1, 7), None))), -('Expression', ('Dict', (1, 2), [('Num', (1, 2), 1)], [('Num', (1, 4), 2)])), +('Expression', ('Dict', (1, 0), [('Num', (1, 2), 1)], [('Num', (1, 4), 2)])), ('Expression', ('Dict', (1, 0), [], [])), -('Expression', ('Set', (1, 1), [('NameConstant', (1, 1), None)])), -('Expression', ('Dict', (2, 6), [('Num', (2, 6), 1)], [('Num', (4, 10), 2)])), +('Expression', ('Set', (1, 0), [('NameConstant', (1, 1), None)])), +('Expression', ('Dict', (1, 0), [('Num', (2, 6), 1)], [('Num', (4, 10), 2)])), ('Expression', ('ListComp', (1, 1), ('Name', (1, 1), 'a', ('Load',)), [('comprehension', ('Name', (1, 7), 'b', ('Store',)), ('Name', (1, 12), 'c', ('Load',)), [('Name', (1, 17), 'd', ('Load',))])])), ('Expression', ('GeneratorExp', (1, 1), ('Name', (1, 1), 'a', ('Load',)), [('comprehension', ('Name', (1, 7), 'b', ('Store',)), ('Name', (1, 12), 'c', ('Load',)), [('Name', (1, 17), 'd', ('Load',))])])), ('Expression', ('Compare', (1, 0), ('Num', (1, 0), 1), [('Lt',), ('Lt',)], [('Num', (1, 4), 2), ('Num', (1, 8), 3)])), diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -131,6 +131,9 @@ Core and Builtins ----------------- +- Issue #25131: Make the line number and column offset of set/dict literals and + comprehensions correspond to the opening brace. + - Issue #25150: Hide the private _Py_atomic_xxx symbols from the public Python.h header to fix a compilation error with OpenMP. PyThreadState_GET() becomes an alias to PyThreadState_Get() to avoid ABI incompatibilies. diff --git a/Python/ast.c b/Python/ast.c --- a/Python/ast.c +++ b/Python/ast.c @@ -2107,6 +2107,7 @@ * (comp_for | (',' (test ':' test | '**' test))* [','])) | * ((test | '*' test) * (comp_for | (',' (test | '*' test))* [','])) ) */ + expr_ty res; ch = CHILD(n, 1); if (TYPE(ch) == RBRACE) { /* It's an empty dict. */ @@ -2118,12 +2119,12 @@ (NCH(ch) > 1 && TYPE(CHILD(ch, 1)) == COMMA)) { /* It's a set display. */ - return ast_for_setdisplay(c, ch); + res = ast_for_setdisplay(c, ch); } else if (NCH(ch) > 1 && TYPE(CHILD(ch, 1)) == comp_for) { /* It's a set comprehension. */ - return ast_for_setcomp(c, ch); + res = ast_for_setcomp(c, ch); } else if (NCH(ch) > 3 - is_dict && TYPE(CHILD(ch, 3 - is_dict)) == comp_for) { @@ -2133,12 +2134,17 @@ "dict comprehension"); return NULL; } - return ast_for_dictcomp(c, ch); + res = ast_for_dictcomp(c, ch); } else { /* It's a dictionary display. */ - return ast_for_dictdisplay(c, ch); + res = ast_for_dictdisplay(c, ch); } + if (res) { + res->lineno = LINENO(n); + res->col_offset = n->n_col_offset; + } + return res; } } default: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 26 09:10:18 2015 From: python-checkins at python.org (benjamin.peterson) Date: Sat, 26 Sep 2015 07:10:18 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_merge_3=2E4?= Message-ID: <20150926071018.94137.28874@psf.io> https://hg.python.org/cpython/rev/de688255f5df changeset: 98276:de688255f5df branch: 3.5 parent: 98273:4f14afc959df parent: 98275:88d98f6c2d7d user: Benjamin Peterson date: Sat Sep 26 00:09:32 2015 -0700 summary: merge 3.4 files: Misc/NEWS | 2 ++ Modules/_pickle.c | 6 ++++++ 2 files changed, 8 insertions(+), 0 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -21,6 +21,8 @@ Library ------- +- Prevent overflow in _Unpickler_Read. + - Issue #25047: The XML encoding declaration written by Element Tree now respects the letter case given by the user. This restores the ability to write encoding names in uppercase like "UTF-8", which worked in Python 2. diff --git a/Modules/_pickle.c b/Modules/_pickle.c --- a/Modules/_pickle.c +++ b/Modules/_pickle.c @@ -1193,6 +1193,12 @@ { Py_ssize_t num_read; + if (self->next_read_idx > PY_SSIZE_T_MAX - n) { + PickleState *st = _Pickle_GetGlobalState(); + PyErr_SetString(st->UnpicklingError, + "read would overflow (invalid bytecode)"); + return -1; + } if (self->next_read_idx + n <= self->input_len) { *s = self->input_buffer + self->next_read_idx; self->next_read_idx += n; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 26 09:10:18 2015 From: python-checkins at python.org (benjamin.peterson) Date: Sat, 26 Sep 2015 07:10:18 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_prevent_overfl?= =?utf-8?q?ow_in_=5FUnpickler=5FRead?= Message-ID: <20150926071018.115289.14867@psf.io> https://hg.python.org/cpython/rev/88d98f6c2d7d changeset: 98275:88d98f6c2d7d branch: 3.4 parent: 98270:e494316a9291 user: Benjamin Peterson date: Sat Sep 26 00:08:34 2015 -0700 summary: prevent overflow in _Unpickler_Read files: Misc/NEWS | 2 ++ Modules/_pickle.c | 6 ++++++ 2 files changed, 8 insertions(+), 0 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -81,6 +81,8 @@ Library ------- +- Prevent overflow in _Unpickler_Read. + - Issue #25047: The XML encoding declaration written by Element Tree now respects the letter case given by the user. This restores the ability to write encoding names in uppercase like "UTF-8", which worked in Python 2. diff --git a/Modules/_pickle.c b/Modules/_pickle.c --- a/Modules/_pickle.c +++ b/Modules/_pickle.c @@ -1182,6 +1182,12 @@ { Py_ssize_t num_read; + if (self->next_read_idx > PY_SSIZE_T_MAX - n) { + PickleState *st = _Pickle_GetGlobalState(); + PyErr_SetString(st->UnpicklingError, + "read would overflow (invalid bytecode)"); + return -1; + } if (self->next_read_idx + n <= self->input_len) { *s = self->input_buffer + self->next_read_idx; self->next_read_idx += n; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 26 09:10:19 2015 From: python-checkins at python.org (benjamin.peterson) Date: Sat, 26 Sep 2015 07:10:19 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogbWVyZ2UgMy41?= Message-ID: <20150926071018.31177.82396@psf.io> https://hg.python.org/cpython/rev/be125ed6d8b4 changeset: 98277:be125ed6d8b4 parent: 98274:9d895c09c08a parent: 98276:de688255f5df user: Benjamin Peterson date: Sat Sep 26 00:09:39 2015 -0700 summary: merge 3.5 files: Misc/NEWS | 2 ++ Modules/_pickle.c | 6 ++++++ 2 files changed, 8 insertions(+), 0 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -141,6 +141,8 @@ Library ------- +- Prevent overflow in _Unpickler_Read. + - Issue #25047: The XML encoding declaration written by Element Tree now respects the letter case given by the user. This restores the ability to write encoding names in uppercase like "UTF-8", which worked in Python 2. diff --git a/Modules/_pickle.c b/Modules/_pickle.c --- a/Modules/_pickle.c +++ b/Modules/_pickle.c @@ -1193,6 +1193,12 @@ { Py_ssize_t num_read; + if (self->next_read_idx > PY_SSIZE_T_MAX - n) { + PickleState *st = _Pickle_GetGlobalState(); + PyErr_SetString(st->UnpicklingError, + "read would overflow (invalid bytecode)"); + return -1; + } if (self->next_read_idx + n <= self->input_len) { *s = self->input_buffer + self->next_read_idx; self->next_read_idx += n; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 26 09:16:12 2015 From: python-checkins at python.org (raymond.hettinger) Date: Sat, 26 Sep 2015 07:16:12 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI1MTM1?= =?utf-8?q?=3A_Avoid_possible_reentrancy_issues_in_deque=5Fclear=2E?= Message-ID: <20150926071612.16599.81725@psf.io> https://hg.python.org/cpython/rev/005590a4a005 changeset: 98278:005590a4a005 branch: 3.5 parent: 98276:de688255f5df user: Raymond Hettinger date: Sat Sep 26 00:14:59 2015 -0700 summary: Issue #25135: Avoid possible reentrancy issues in deque_clear. files: Misc/NEWS | 3 + Modules/_collectionsmodule.c | 62 ++++++++++++++++++++++- 2 files changed, 62 insertions(+), 3 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -27,6 +27,9 @@ respects the letter case given by the user. This restores the ability to write encoding names in uppercase like "UTF-8", which worked in Python 2. +- Issue #25135: Make deque_clear() safer by emptying the deque before clearing. + This helps avoid possible reentrancy issues. + - Issue #19143: platform module now reads Windows version from kernel32.dll to avoid compatibility shims. diff --git a/Modules/_collectionsmodule.c b/Modules/_collectionsmodule.c --- a/Modules/_collectionsmodule.c +++ b/Modules/_collectionsmodule.c @@ -1038,16 +1038,72 @@ static void deque_clear(dequeobject *deque) { + block *b; + block *prevblock; + block *leftblock; + Py_ssize_t leftindex; + Py_ssize_t n; PyObject *item; + /* During the process of clearing a deque, decrefs can cause the + deque to mutate. To avoid fatal confusion, we have to make the + deque empty before clearing the blocks and never refer to + anything via deque->ref while clearing. (This is the same + technique used for clearing lists, sets, and dicts.) + + Making the deque empty requires allocating a new empty block. In + the unlikely event that memory is full, we fall back to an + alternate method that doesn't require a new block. Repeating + pops in a while-loop is slower, possibly re-entrant (and a clever + adversary could cause it to never terminate). + */ + + b = newblock(0); + if (b == NULL) { + PyErr_Clear(); + goto alternate_method; + } + + /* Remember the old size, leftblock, and leftindex */ + leftblock = deque->leftblock; + leftindex = deque->leftindex; + n = Py_SIZE(deque); + + /* Set the deque to be empty using the newly allocated block */ + MARK_END(b->leftlink); + MARK_END(b->rightlink); + Py_SIZE(deque) = 0; + deque->leftblock = b; + deque->rightblock = b; + deque->leftindex = CENTER + 1; + deque->rightindex = CENTER; + deque->state++; + + /* Now the old size, leftblock, and leftindex are disconnected from + the empty deque and we can use them to decref the pointers. + */ + while (n--) { + item = leftblock->data[leftindex]; + Py_DECREF(item); + leftindex++; + if (leftindex == BLOCKLEN && n) { + CHECK_NOT_END(leftblock->rightlink); + prevblock = leftblock; + leftblock = leftblock->rightlink; + leftindex = 0; + freeblock(prevblock); + } + } + CHECK_END(leftblock->rightlink); + freeblock(leftblock); + return; + + alternate_method: while (Py_SIZE(deque)) { item = deque_pop(deque, NULL); assert (item != NULL); Py_DECREF(item); } - assert(deque->leftblock == deque->rightblock); - assert(deque->leftindex - 1 == deque->rightindex); - assert(Py_SIZE(deque) == 0); } static int -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 26 09:16:13 2015 From: python-checkins at python.org (raymond.hettinger) Date: Sat, 26 Sep 2015 07:16:13 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_merge?= Message-ID: <20150926071613.82654.2884@psf.io> https://hg.python.org/cpython/rev/c6b36c6f91b8 changeset: 98279:c6b36c6f91b8 parent: 98277:be125ed6d8b4 parent: 98278:005590a4a005 user: Raymond Hettinger date: Sat Sep 26 00:15:46 2015 -0700 summary: merge files: Misc/NEWS | 3 + Modules/_collectionsmodule.c | 62 ++++++++++++++++++++++- 2 files changed, 62 insertions(+), 3 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -147,6 +147,9 @@ respects the letter case given by the user. This restores the ability to write encoding names in uppercase like "UTF-8", which worked in Python 2. +- Issue #25135: Make deque_clear() safer by emptying the deque before clearing. + This helps avoid possible reentrancy issues. + - Issue #19143: platform module now reads Windows version from kernel32.dll to avoid compatibility shims. diff --git a/Modules/_collectionsmodule.c b/Modules/_collectionsmodule.c --- a/Modules/_collectionsmodule.c +++ b/Modules/_collectionsmodule.c @@ -1045,16 +1045,72 @@ static void deque_clear(dequeobject *deque) { + block *b; + block *prevblock; + block *leftblock; + Py_ssize_t leftindex; + Py_ssize_t n; PyObject *item; + /* During the process of clearing a deque, decrefs can cause the + deque to mutate. To avoid fatal confusion, we have to make the + deque empty before clearing the blocks and never refer to + anything via deque->ref while clearing. (This is the same + technique used for clearing lists, sets, and dicts.) + + Making the deque empty requires allocating a new empty block. In + the unlikely event that memory is full, we fall back to an + alternate method that doesn't require a new block. Repeating + pops in a while-loop is slower, possibly re-entrant (and a clever + adversary could cause it to never terminate). + */ + + b = newblock(0); + if (b == NULL) { + PyErr_Clear(); + goto alternate_method; + } + + /* Remember the old size, leftblock, and leftindex */ + leftblock = deque->leftblock; + leftindex = deque->leftindex; + n = Py_SIZE(deque); + + /* Set the deque to be empty using the newly allocated block */ + MARK_END(b->leftlink); + MARK_END(b->rightlink); + Py_SIZE(deque) = 0; + deque->leftblock = b; + deque->rightblock = b; + deque->leftindex = CENTER + 1; + deque->rightindex = CENTER; + deque->state++; + + /* Now the old size, leftblock, and leftindex are disconnected from + the empty deque and we can use them to decref the pointers. + */ + while (n--) { + item = leftblock->data[leftindex]; + Py_DECREF(item); + leftindex++; + if (leftindex == BLOCKLEN && n) { + CHECK_NOT_END(leftblock->rightlink); + prevblock = leftblock; + leftblock = leftblock->rightlink; + leftindex = 0; + freeblock(prevblock); + } + } + CHECK_END(leftblock->rightlink); + freeblock(leftblock); + return; + + alternate_method: while (Py_SIZE(deque)) { item = deque_pop(deque, NULL); assert (item != NULL); Py_DECREF(item); } - assert(deque->leftblock == deque->rightblock); - assert(deque->leftindex - 1 == deque->rightindex); - assert(Py_SIZE(deque) == 0); } static int -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 26 09:44:51 2015 From: python-checkins at python.org (victor.stinner) Date: Sat, 26 Sep 2015 07:44:51 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2325220=3A_Create_L?= =?utf-8?q?ib/test/libregrtest/?= Message-ID: <20150926074451.16597.97108@psf.io> https://hg.python.org/cpython/rev/40667f456c6f changeset: 98280:40667f456c6f user: Victor Stinner date: Sat Sep 26 09:43:45 2015 +0200 summary: Issue #25220: Create Lib/test/libregrtest/ Start to split regrtest.py into smaller parts with the creation of Lib/test/libregrtest/cmdline.py: code to handle the command line, especially parsing command line arguments. This part of the code is tested by test_regrtest. files: Lib/test/libregrtest/__init__.py | 1 + Lib/test/libregrtest/cmdline.py | 1268 +----------------- Lib/test/regrtest.py | 332 +---- Lib/test/test_regrtest.py | 4 +- 4 files changed, 14 insertions(+), 1591 deletions(-) diff --git a/Lib/test/libregrtest/__init__.py b/Lib/test/libregrtest/__init__.py new file mode 100644 --- /dev/null +++ b/Lib/test/libregrtest/__init__.py @@ -0,0 +1,1 @@ +from test.libregrtest.cmdline import _parse_args, RESOURCE_NAMES diff --git a/Lib/test/regrtest.py b/Lib/test/libregrtest/cmdline.py old mode 100755 new mode 100644 copy from Lib/test/regrtest.py copy to Lib/test/libregrtest/cmdline.py --- a/Lib/test/regrtest.py +++ b/Lib/test/libregrtest/cmdline.py @@ -1,10 +1,8 @@ -#! /usr/bin/env python3 +import argparse +import faulthandler +import os -""" -Script to run Python regression tests. - -Run this script with -h or --help for documentation. -""" +from test import support USAGE = """\ python -m test [options] [test_name1 [test_name2 ...]] @@ -119,102 +117,16 @@ option '-uall,-gui'. """ -# We import importlib *ASAP* in order to test #15386 -import importlib - -import argparse -import builtins -import faulthandler -import io -import json -import locale -import logging -import os -import platform -import random -import re -import shutil -import signal -import sys -import sysconfig -import tempfile -import time -import traceback -import unittest -import warnings -from inspect import isabstract - -try: - import threading -except ImportError: - threading = None -try: - import _multiprocessing, multiprocessing.process -except ImportError: - multiprocessing = None - - -# Some times __path__ and __file__ are not absolute (e.g. while running from -# Lib/) and, if we change the CWD to run the tests in a temporary dir, some -# imports might fail. This affects only the modules imported before os.chdir(). -# These modules are searched first in sys.path[0] (so '' -- the CWD) and if -# they are found in the CWD their __file__ and __path__ will be relative (this -# happens before the chdir). All the modules imported after the chdir, are -# not found in the CWD, and since the other paths in sys.path[1:] are absolute -# (site.py absolutize them), the __file__ and __path__ will be absolute too. -# Therefore it is necessary to absolutize manually the __file__ and __path__ of -# the packages to prevent later imports to fail when the CWD is different. -for module in sys.modules.values(): - if hasattr(module, '__path__'): - module.__path__ = [os.path.abspath(path) for path in module.__path__] - if hasattr(module, '__file__'): - module.__file__ = os.path.abspath(module.__file__) - - -# MacOSX (a.k.a. Darwin) has a default stack size that is too small -# for deeply recursive regular expressions. We see this as crashes in -# the Python test suite when running test_re.py and test_sre.py. The -# fix is to set the stack limit to 2048. -# This approach may also be useful for other Unixy platforms that -# suffer from small default stack limits. -if sys.platform == 'darwin': - try: - import resource - except ImportError: - pass - else: - soft, hard = resource.getrlimit(resource.RLIMIT_STACK) - newsoft = min(hard, max(soft, 1024*2048)) - resource.setrlimit(resource.RLIMIT_STACK, (newsoft, hard)) - -# Test result constants. -PASSED = 1 -FAILED = 0 -ENV_CHANGED = -1 -SKIPPED = -2 -RESOURCE_DENIED = -3 -INTERRUPTED = -4 -CHILD_ERROR = -5 # error in a child process - -from test import support RESOURCE_NAMES = ('audio', 'curses', 'largefile', 'network', 'decimal', 'cpu', 'subprocess', 'urlfetch', 'gui') -# When tests are run from the Python build directory, it is best practice -# to keep the test files in a subfolder. This eases the cleanup of leftover -# files using the "make distclean" command. -if sysconfig.is_python_build(): - TEMPDIR = os.path.join(sysconfig.get_config_var('srcdir'), 'build') -else: - TEMPDIR = tempfile.gettempdir() -TEMPDIR = os.path.abspath(TEMPDIR) - class _ArgParser(argparse.ArgumentParser): def error(self, message): super().error(message + "\nPass -h or --help for complete help.") + def _create_parser(): # Set prog to prevent the uninformative "__main__.py" from displaying in # error messages when using "python -m test ...". @@ -328,11 +240,13 @@ return parser + def relative_filename(string): # CWD is replaced with a temporary dir before calling main(), so we # join it with the saved CWD so it ends up where the user expects. return os.path.join(support.SAVEDCWD, string) + def huntrleaks(string): args = string.split(':') if len(args) not in (2, 3): @@ -343,6 +257,7 @@ fname = args[2] if len(args) > 2 and args[2] else 'reflog.txt' return nwarmup, ntracked, fname + def resources_list(string): u = [x.lower() for x in string.split(',')] for r in u: @@ -354,6 +269,7 @@ raise argparse.ArgumentTypeError('invalid resource: ' + r) return u + def _parse_args(args, **kwargs): # Defaults ns = argparse.Namespace(testdir=None, verbose=0, quiet=False, @@ -422,1169 +338,3 @@ ns.randomize = True return ns - - -def run_test_in_subprocess(testname, ns): - """Run the given test in a subprocess with --slaveargs. - - ns is the option Namespace parsed from command-line arguments. regrtest - is invoked in a subprocess with the --slaveargs argument; when the - subprocess exits, its return code, stdout and stderr are returned as a - 3-tuple. - """ - from subprocess import Popen, PIPE - base_cmd = ([sys.executable] + support.args_from_interpreter_flags() + - ['-X', 'faulthandler', '-m', 'test.regrtest']) - - slaveargs = ( - (testname, ns.verbose, ns.quiet), - dict(huntrleaks=ns.huntrleaks, - use_resources=ns.use_resources, - output_on_failure=ns.verbose3, - timeout=ns.timeout, failfast=ns.failfast, - match_tests=ns.match_tests)) - # Running the child from the same working directory as regrtest's original - # invocation ensures that TEMPDIR for the child is the same when - # sysconfig.is_python_build() is true. See issue 15300. - popen = Popen(base_cmd + ['--slaveargs', json.dumps(slaveargs)], - stdout=PIPE, stderr=PIPE, - universal_newlines=True, - close_fds=(os.name != 'nt'), - cwd=support.SAVEDCWD) - stdout, stderr = popen.communicate() - retcode = popen.wait() - return retcode, stdout, stderr - - -def main(tests=None, **kwargs): - """Execute a test suite. - - This also parses command-line options and modifies its behavior - accordingly. - - tests -- a list of strings containing test names (optional) - testdir -- the directory in which to look for tests (optional) - - Users other than the Python test suite will certainly want to - specify testdir; if it's omitted, the directory containing the - Python test suite is searched for. - - If the tests argument is omitted, the tests listed on the - command-line will be used. If that's empty, too, then all *.py - files beginning with test_ will be used. - - The other default arguments (verbose, quiet, exclude, - single, randomize, findleaks, use_resources, trace, coverdir, - print_slow, and random_seed) allow programmers calling main() - directly to set the values that would normally be set by flags - on the command line. - """ - # Display the Python traceback on fatal errors (e.g. segfault) - faulthandler.enable(all_threads=True) - - # Display the Python traceback on SIGALRM or SIGUSR1 signal - signals = [] - if hasattr(signal, 'SIGALRM'): - signals.append(signal.SIGALRM) - if hasattr(signal, 'SIGUSR1'): - signals.append(signal.SIGUSR1) - for signum in signals: - faulthandler.register(signum, chain=True) - - replace_stdout() - - support.record_original_stdout(sys.stdout) - - ns = _parse_args(sys.argv[1:], **kwargs) - - if ns.huntrleaks: - # Avoid false positives due to various caches - # filling slowly with random data: - warm_caches() - if ns.memlimit is not None: - support.set_memlimit(ns.memlimit) - if ns.threshold is not None: - import gc - gc.set_threshold(ns.threshold) - if ns.nowindows: - import msvcrt - msvcrt.SetErrorMode(msvcrt.SEM_FAILCRITICALERRORS| - msvcrt.SEM_NOALIGNMENTFAULTEXCEPT| - msvcrt.SEM_NOGPFAULTERRORBOX| - msvcrt.SEM_NOOPENFILEERRORBOX) - try: - msvcrt.CrtSetReportMode - except AttributeError: - # release build - pass - else: - for m in [msvcrt.CRT_WARN, msvcrt.CRT_ERROR, msvcrt.CRT_ASSERT]: - msvcrt.CrtSetReportMode(m, msvcrt.CRTDBG_MODE_FILE) - msvcrt.CrtSetReportFile(m, msvcrt.CRTDBG_FILE_STDERR) - if ns.wait: - input("Press any key to continue...") - - if ns.slaveargs is not None: - args, kwargs = json.loads(ns.slaveargs) - if kwargs.get('huntrleaks'): - unittest.BaseTestSuite._cleanup = False - try: - result = runtest(*args, **kwargs) - except KeyboardInterrupt: - result = INTERRUPTED, '' - except BaseException as e: - traceback.print_exc() - result = CHILD_ERROR, str(e) - sys.stdout.flush() - print() # Force a newline (just in case) - print(json.dumps(result)) - sys.exit(0) - - good = [] - bad = [] - skipped = [] - resource_denieds = [] - environment_changed = [] - interrupted = False - - if ns.findleaks: - try: - import gc - except ImportError: - print('No GC available, disabling findleaks.') - ns.findleaks = False - else: - # Uncomment the line below to report garbage that is not - # freeable by reference counting alone. By default only - # garbage that is not collectable by the GC is reported. - #gc.set_debug(gc.DEBUG_SAVEALL) - found_garbage = [] - - if ns.huntrleaks: - unittest.BaseTestSuite._cleanup = False - - if ns.single: - filename = os.path.join(TEMPDIR, 'pynexttest') - try: - with open(filename, 'r') as fp: - next_test = fp.read().strip() - tests = [next_test] - except OSError: - pass - - if ns.fromfile: - tests = [] - with open(os.path.join(support.SAVEDCWD, ns.fromfile)) as fp: - count_pat = re.compile(r'\[\s*\d+/\s*\d+\]') - for line in fp: - line = count_pat.sub('', line) - guts = line.split() # assuming no test has whitespace in its name - if guts and not guts[0].startswith('#'): - tests.extend(guts) - - # Strip .py extensions. - removepy(ns.args) - removepy(tests) - - stdtests = STDTESTS[:] - nottests = NOTTESTS.copy() - if ns.exclude: - for arg in ns.args: - if arg in stdtests: - stdtests.remove(arg) - nottests.add(arg) - ns.args = [] - - # For a partial run, we do not need to clutter the output. - if ns.verbose or ns.header or not (ns.quiet or ns.single or tests or ns.args): - # Print basic platform information - print("==", platform.python_implementation(), *sys.version.split()) - print("== ", platform.platform(aliased=True), - "%s-endian" % sys.byteorder) - print("== ", "hash algorithm:", sys.hash_info.algorithm, - "64bit" if sys.maxsize > 2**32 else "32bit") - print("== ", os.getcwd()) - print("Testing with flags:", sys.flags) - - # if testdir is set, then we are not running the python tests suite, so - # don't add default tests to be executed or skipped (pass empty values) - if ns.testdir: - alltests = findtests(ns.testdir, list(), set()) - else: - alltests = findtests(ns.testdir, stdtests, nottests) - - selected = tests or ns.args or alltests - if ns.single: - selected = selected[:1] - try: - next_single_test = alltests[alltests.index(selected[0])+1] - except IndexError: - next_single_test = None - # Remove all the selected tests that precede start if it's set. - if ns.start: - try: - del selected[:selected.index(ns.start)] - except ValueError: - print("Couldn't find starting test (%s), using all tests" % ns.start) - if ns.randomize: - if ns.random_seed is None: - ns.random_seed = random.randrange(10000000) - random.seed(ns.random_seed) - print("Using random seed", ns.random_seed) - random.shuffle(selected) - if ns.trace: - import trace, tempfile - tracer = trace.Trace(ignoredirs=[sys.base_prefix, sys.base_exec_prefix, - tempfile.gettempdir()], - trace=False, count=True) - - test_times = [] - support.verbose = ns.verbose # Tell tests to be moderately quiet - support.use_resources = ns.use_resources - save_modules = sys.modules.keys() - - def accumulate_result(test, result): - ok, test_time = result - test_times.append((test_time, test)) - if ok == PASSED: - good.append(test) - elif ok == FAILED: - bad.append(test) - elif ok == ENV_CHANGED: - environment_changed.append(test) - elif ok == SKIPPED: - skipped.append(test) - elif ok == RESOURCE_DENIED: - skipped.append(test) - resource_denieds.append(test) - - if ns.forever: - def test_forever(tests=list(selected)): - while True: - for test in tests: - yield test - if bad: - return - tests = test_forever() - test_count = '' - test_count_width = 3 - else: - tests = iter(selected) - test_count = '/{}'.format(len(selected)) - test_count_width = len(test_count) - 1 - - if ns.use_mp: - try: - from threading import Thread - except ImportError: - print("Multiprocess option requires thread support") - sys.exit(2) - from queue import Queue - debug_output_pat = re.compile(r"\[\d+ refs, \d+ blocks\]$") - output = Queue() - pending = MultiprocessTests(tests) - def work(): - # A worker thread. - try: - while True: - try: - test = next(pending) - except StopIteration: - output.put((None, None, None, None)) - return - retcode, stdout, stderr = run_test_in_subprocess(test, ns) - # Strip last refcount output line if it exists, since it - # comes from the shutdown of the interpreter in the subcommand. - stderr = debug_output_pat.sub("", stderr) - stdout, _, result = stdout.strip().rpartition("\n") - if retcode != 0: - result = (CHILD_ERROR, "Exit code %s" % retcode) - output.put((test, stdout.rstrip(), stderr.rstrip(), result)) - return - if not result: - output.put((None, None, None, None)) - return - result = json.loads(result) - output.put((test, stdout.rstrip(), stderr.rstrip(), result)) - except BaseException: - output.put((None, None, None, None)) - raise - workers = [Thread(target=work) for i in range(ns.use_mp)] - for worker in workers: - worker.start() - finished = 0 - test_index = 1 - try: - while finished < ns.use_mp: - test, stdout, stderr, result = output.get() - if test is None: - finished += 1 - continue - accumulate_result(test, result) - if not ns.quiet: - fmt = "[{1:{0}}{2}/{3}] {4}" if bad else "[{1:{0}}{2}] {4}" - print(fmt.format( - test_count_width, test_index, test_count, - len(bad), test)) - if stdout: - print(stdout) - if stderr: - print(stderr, file=sys.stderr) - sys.stdout.flush() - sys.stderr.flush() - if result[0] == INTERRUPTED: - raise KeyboardInterrupt - if result[0] == CHILD_ERROR: - raise Exception("Child error on {}: {}".format(test, result[1])) - test_index += 1 - except KeyboardInterrupt: - interrupted = True - pending.interrupted = True - for worker in workers: - worker.join() - else: - for test_index, test in enumerate(tests, 1): - if not ns.quiet: - fmt = "[{1:{0}}{2}/{3}] {4}" if bad else "[{1:{0}}{2}] {4}" - print(fmt.format( - test_count_width, test_index, test_count, len(bad), test)) - sys.stdout.flush() - if ns.trace: - # If we're tracing code coverage, then we don't exit with status - # if on a false return value from main. - tracer.runctx('runtest(test, ns.verbose, ns.quiet, timeout=ns.timeout)', - globals=globals(), locals=vars()) - else: - try: - result = runtest(test, ns.verbose, ns.quiet, - ns.huntrleaks, - output_on_failure=ns.verbose3, - timeout=ns.timeout, failfast=ns.failfast, - match_tests=ns.match_tests) - accumulate_result(test, result) - except KeyboardInterrupt: - interrupted = True - break - if ns.findleaks: - gc.collect() - if gc.garbage: - print("Warning: test created", len(gc.garbage), end=' ') - print("uncollectable object(s).") - # move the uncollectable objects somewhere so we don't see - # them again - found_garbage.extend(gc.garbage) - del gc.garbage[:] - # Unload the newly imported modules (best effort finalization) - for module in sys.modules.keys(): - if module not in save_modules and module.startswith("test."): - support.unload(module) - - if interrupted: - # print a newline after ^C - print() - print("Test suite interrupted by signal SIGINT.") - omitted = set(selected) - set(good) - set(bad) - set(skipped) - print(count(len(omitted), "test"), "omitted:") - printlist(omitted) - if good and not ns.quiet: - if not bad and not skipped and not interrupted and len(good) > 1: - print("All", end=' ') - print(count(len(good), "test"), "OK.") - if ns.print_slow: - test_times.sort(reverse=True) - print("10 slowest tests:") - for time, test in test_times[:10]: - print("%s: %.1fs" % (test, time)) - if bad: - print(count(len(bad), "test"), "failed:") - printlist(bad) - if environment_changed: - print("{} altered the execution environment:".format( - count(len(environment_changed), "test"))) - printlist(environment_changed) - if skipped and not ns.quiet: - print(count(len(skipped), "test"), "skipped:") - printlist(skipped) - - if ns.verbose2 and bad: - print("Re-running failed tests in verbose mode") - for test in bad[:]: - print("Re-running test %r in verbose mode" % test) - sys.stdout.flush() - try: - ns.verbose = True - ok = runtest(test, True, ns.quiet, ns.huntrleaks, - timeout=ns.timeout) - except KeyboardInterrupt: - # print a newline separate from the ^C - print() - break - else: - if ok[0] in {PASSED, ENV_CHANGED, SKIPPED, RESOURCE_DENIED}: - bad.remove(test) - else: - if bad: - print(count(len(bad), 'test'), "failed again:") - printlist(bad) - - if ns.single: - if next_single_test: - with open(filename, 'w') as fp: - fp.write(next_single_test + '\n') - else: - os.unlink(filename) - - if ns.trace: - r = tracer.results() - r.write_results(show_missing=True, summary=True, coverdir=ns.coverdir) - - if ns.runleaks: - os.system("leaks %d" % os.getpid()) - - sys.exit(len(bad) > 0 or interrupted) - - -# small set of tests to determine if we have a basically functioning interpreter -# (i.e. if any of these fail, then anything else is likely to follow) -STDTESTS = [ - 'test_grammar', - 'test_opcodes', - 'test_dict', - 'test_builtin', - 'test_exceptions', - 'test_types', - 'test_unittest', - 'test_doctest', - 'test_doctest2', - 'test_support' -] - -# set of tests that we don't want to be executed when using regrtest -NOTTESTS = set() - -def findtests(testdir=None, stdtests=STDTESTS, nottests=NOTTESTS): - """Return a list of all applicable test modules.""" - testdir = findtestdir(testdir) - names = os.listdir(testdir) - tests = [] - others = set(stdtests) | nottests - for name in names: - mod, ext = os.path.splitext(name) - if mod[:5] == "test_" and ext in (".py", "") and mod not in others: - tests.append(mod) - return stdtests + sorted(tests) - -# We do not use a generator so multiple threads can call next(). -class MultiprocessTests(object): - - """A thread-safe iterator over tests for multiprocess mode.""" - - def __init__(self, tests): - self.interrupted = False - self.lock = threading.Lock() - self.tests = tests - - def __iter__(self): - return self - - def __next__(self): - with self.lock: - if self.interrupted: - raise StopIteration('tests interrupted') - return next(self.tests) - -def replace_stdout(): - """Set stdout encoder error handler to backslashreplace (as stderr error - handler) to avoid UnicodeEncodeError when printing a traceback""" - import atexit - - stdout = sys.stdout - sys.stdout = open(stdout.fileno(), 'w', - encoding=stdout.encoding, - errors="backslashreplace", - closefd=False, - newline='\n') - - def restore_stdout(): - sys.stdout.close() - sys.stdout = stdout - atexit.register(restore_stdout) - -def runtest(test, verbose, quiet, - huntrleaks=False, use_resources=None, - output_on_failure=False, failfast=False, match_tests=None, - timeout=None): - """Run a single test. - - test -- the name of the test - verbose -- if true, print more messages - quiet -- if true, don't print 'skipped' messages (probably redundant) - huntrleaks -- run multiple times to test for leaks; requires a debug - build; a triple corresponding to -R's three arguments - use_resources -- list of extra resources to use - output_on_failure -- if true, display test output on failure - timeout -- dump the traceback and exit if a test takes more than - timeout seconds - failfast, match_tests -- See regrtest command-line flags for these. - - Returns the tuple result, test_time, where result is one of the constants: - INTERRUPTED KeyboardInterrupt when run under -j - RESOURCE_DENIED test skipped because resource denied - SKIPPED test skipped for some other reason - ENV_CHANGED test failed because it changed the execution environment - FAILED test failed - PASSED test passed - """ - - if use_resources is not None: - support.use_resources = use_resources - use_timeout = (timeout is not None) - if use_timeout: - faulthandler.dump_traceback_later(timeout, exit=True) - try: - support.match_tests = match_tests - if failfast: - support.failfast = True - if output_on_failure: - support.verbose = True - - # Reuse the same instance to all calls to runtest(). Some - # tests keep a reference to sys.stdout or sys.stderr - # (eg. test_argparse). - if runtest.stringio is None: - stream = io.StringIO() - runtest.stringio = stream - else: - stream = runtest.stringio - stream.seek(0) - stream.truncate() - - orig_stdout = sys.stdout - orig_stderr = sys.stderr - try: - sys.stdout = stream - sys.stderr = stream - result = runtest_inner(test, verbose, quiet, huntrleaks, - display_failure=False) - if result[0] == FAILED: - output = stream.getvalue() - orig_stderr.write(output) - orig_stderr.flush() - finally: - sys.stdout = orig_stdout - sys.stderr = orig_stderr - else: - support.verbose = verbose # Tell tests to be moderately quiet - result = runtest_inner(test, verbose, quiet, huntrleaks, - display_failure=not verbose) - return result - finally: - if use_timeout: - faulthandler.cancel_dump_traceback_later() - cleanup_test_droppings(test, verbose) -runtest.stringio = None - -# Unit tests are supposed to leave the execution environment unchanged -# once they complete. But sometimes tests have bugs, especially when -# tests fail, and the changes to environment go on to mess up other -# tests. This can cause issues with buildbot stability, since tests -# are run in random order and so problems may appear to come and go. -# There are a few things we can save and restore to mitigate this, and -# the following context manager handles this task. - -class saved_test_environment: - """Save bits of the test environment and restore them at block exit. - - with saved_test_environment(testname, verbose, quiet): - #stuff - - Unless quiet is True, a warning is printed to stderr if any of - the saved items was changed by the test. The attribute 'changed' - is initially False, but is set to True if a change is detected. - - If verbose is more than 1, the before and after state of changed - items is also printed. - """ - - changed = False - - def __init__(self, testname, verbose=0, quiet=False): - self.testname = testname - self.verbose = verbose - self.quiet = quiet - - # To add things to save and restore, add a name XXX to the resources list - # and add corresponding get_XXX/restore_XXX functions. get_XXX should - # return the value to be saved and compared against a second call to the - # get function when test execution completes. restore_XXX should accept - # the saved value and restore the resource using it. It will be called if - # and only if a change in the value is detected. - # - # Note: XXX will have any '.' replaced with '_' characters when determining - # the corresponding method names. - - resources = ('sys.argv', 'cwd', 'sys.stdin', 'sys.stdout', 'sys.stderr', - 'os.environ', 'sys.path', 'sys.path_hooks', '__import__', - 'warnings.filters', 'asyncore.socket_map', - 'logging._handlers', 'logging._handlerList', 'sys.gettrace', - 'sys.warnoptions', - # multiprocessing.process._cleanup() may release ref - # to a thread, so check processes first. - 'multiprocessing.process._dangling', 'threading._dangling', - 'sysconfig._CONFIG_VARS', 'sysconfig._INSTALL_SCHEMES', - 'files', 'locale', 'warnings.showwarning', - ) - - def get_sys_argv(self): - return id(sys.argv), sys.argv, sys.argv[:] - def restore_sys_argv(self, saved_argv): - sys.argv = saved_argv[1] - sys.argv[:] = saved_argv[2] - - def get_cwd(self): - return os.getcwd() - def restore_cwd(self, saved_cwd): - os.chdir(saved_cwd) - - def get_sys_stdout(self): - return sys.stdout - def restore_sys_stdout(self, saved_stdout): - sys.stdout = saved_stdout - - def get_sys_stderr(self): - return sys.stderr - def restore_sys_stderr(self, saved_stderr): - sys.stderr = saved_stderr - - def get_sys_stdin(self): - return sys.stdin - def restore_sys_stdin(self, saved_stdin): - sys.stdin = saved_stdin - - def get_os_environ(self): - return id(os.environ), os.environ, dict(os.environ) - def restore_os_environ(self, saved_environ): - os.environ = saved_environ[1] - os.environ.clear() - os.environ.update(saved_environ[2]) - - def get_sys_path(self): - return id(sys.path), sys.path, sys.path[:] - def restore_sys_path(self, saved_path): - sys.path = saved_path[1] - sys.path[:] = saved_path[2] - - def get_sys_path_hooks(self): - return id(sys.path_hooks), sys.path_hooks, sys.path_hooks[:] - def restore_sys_path_hooks(self, saved_hooks): - sys.path_hooks = saved_hooks[1] - sys.path_hooks[:] = saved_hooks[2] - - def get_sys_gettrace(self): - return sys.gettrace() - def restore_sys_gettrace(self, trace_fxn): - sys.settrace(trace_fxn) - - def get___import__(self): - return builtins.__import__ - def restore___import__(self, import_): - builtins.__import__ = import_ - - def get_warnings_filters(self): - return id(warnings.filters), warnings.filters, warnings.filters[:] - def restore_warnings_filters(self, saved_filters): - warnings.filters = saved_filters[1] - warnings.filters[:] = saved_filters[2] - - def get_asyncore_socket_map(self): - asyncore = sys.modules.get('asyncore') - # XXX Making a copy keeps objects alive until __exit__ gets called. - return asyncore and asyncore.socket_map.copy() or {} - def restore_asyncore_socket_map(self, saved_map): - asyncore = sys.modules.get('asyncore') - if asyncore is not None: - asyncore.close_all(ignore_all=True) - asyncore.socket_map.update(saved_map) - - def get_shutil_archive_formats(self): - # we could call get_archives_formats() but that only returns the - # registry keys; we want to check the values too (the functions that - # are registered) - return shutil._ARCHIVE_FORMATS, shutil._ARCHIVE_FORMATS.copy() - def restore_shutil_archive_formats(self, saved): - shutil._ARCHIVE_FORMATS = saved[0] - shutil._ARCHIVE_FORMATS.clear() - shutil._ARCHIVE_FORMATS.update(saved[1]) - - def get_shutil_unpack_formats(self): - return shutil._UNPACK_FORMATS, shutil._UNPACK_FORMATS.copy() - def restore_shutil_unpack_formats(self, saved): - shutil._UNPACK_FORMATS = saved[0] - shutil._UNPACK_FORMATS.clear() - shutil._UNPACK_FORMATS.update(saved[1]) - - def get_logging__handlers(self): - # _handlers is a WeakValueDictionary - return id(logging._handlers), logging._handlers, logging._handlers.copy() - def restore_logging__handlers(self, saved_handlers): - # Can't easily revert the logging state - pass - - def get_logging__handlerList(self): - # _handlerList is a list of weakrefs to handlers - return id(logging._handlerList), logging._handlerList, logging._handlerList[:] - def restore_logging__handlerList(self, saved_handlerList): - # Can't easily revert the logging state - pass - - def get_sys_warnoptions(self): - return id(sys.warnoptions), sys.warnoptions, sys.warnoptions[:] - def restore_sys_warnoptions(self, saved_options): - sys.warnoptions = saved_options[1] - sys.warnoptions[:] = saved_options[2] - - # Controlling dangling references to Thread objects can make it easier - # to track reference leaks. - def get_threading__dangling(self): - if not threading: - return None - # This copies the weakrefs without making any strong reference - return threading._dangling.copy() - def restore_threading__dangling(self, saved): - if not threading: - return - threading._dangling.clear() - threading._dangling.update(saved) - - # Same for Process objects - def get_multiprocessing_process__dangling(self): - if not multiprocessing: - return None - # Unjoined process objects can survive after process exits - multiprocessing.process._cleanup() - # This copies the weakrefs without making any strong reference - return multiprocessing.process._dangling.copy() - def restore_multiprocessing_process__dangling(self, saved): - if not multiprocessing: - return - multiprocessing.process._dangling.clear() - multiprocessing.process._dangling.update(saved) - - def get_sysconfig__CONFIG_VARS(self): - # make sure the dict is initialized - sysconfig.get_config_var('prefix') - return (id(sysconfig._CONFIG_VARS), sysconfig._CONFIG_VARS, - dict(sysconfig._CONFIG_VARS)) - def restore_sysconfig__CONFIG_VARS(self, saved): - sysconfig._CONFIG_VARS = saved[1] - sysconfig._CONFIG_VARS.clear() - sysconfig._CONFIG_VARS.update(saved[2]) - - def get_sysconfig__INSTALL_SCHEMES(self): - return (id(sysconfig._INSTALL_SCHEMES), sysconfig._INSTALL_SCHEMES, - sysconfig._INSTALL_SCHEMES.copy()) - def restore_sysconfig__INSTALL_SCHEMES(self, saved): - sysconfig._INSTALL_SCHEMES = saved[1] - sysconfig._INSTALL_SCHEMES.clear() - sysconfig._INSTALL_SCHEMES.update(saved[2]) - - def get_files(self): - return sorted(fn + ('/' if os.path.isdir(fn) else '') - for fn in os.listdir()) - def restore_files(self, saved_value): - fn = support.TESTFN - if fn not in saved_value and (fn + '/') not in saved_value: - if os.path.isfile(fn): - support.unlink(fn) - elif os.path.isdir(fn): - support.rmtree(fn) - - _lc = [getattr(locale, lc) for lc in dir(locale) - if lc.startswith('LC_')] - def get_locale(self): - pairings = [] - for lc in self._lc: - try: - pairings.append((lc, locale.setlocale(lc, None))) - except (TypeError, ValueError): - continue - return pairings - def restore_locale(self, saved): - for lc, setting in saved: - locale.setlocale(lc, setting) - - def get_warnings_showwarning(self): - return warnings.showwarning - def restore_warnings_showwarning(self, fxn): - warnings.showwarning = fxn - - def resource_info(self): - for name in self.resources: - method_suffix = name.replace('.', '_') - get_name = 'get_' + method_suffix - restore_name = 'restore_' + method_suffix - yield name, getattr(self, get_name), getattr(self, restore_name) - - def __enter__(self): - self.saved_values = dict((name, get()) for name, get, restore - in self.resource_info()) - return self - - def __exit__(self, exc_type, exc_val, exc_tb): - saved_values = self.saved_values - del self.saved_values - for name, get, restore in self.resource_info(): - current = get() - original = saved_values.pop(name) - # Check for changes to the resource's value - if current != original: - self.changed = True - restore(original) - if not self.quiet: - print("Warning -- {} was modified by {}".format( - name, self.testname), - file=sys.stderr) - if self.verbose > 1: - print(" Before: {}\n After: {} ".format( - original, current), - file=sys.stderr) - return False - - -def runtest_inner(test, verbose, quiet, - huntrleaks=False, display_failure=True): - support.unload(test) - - test_time = 0.0 - refleak = False # True if the test leaked references. - try: - if test.startswith('test.'): - abstest = test - else: - # Always import it from the test package - abstest = 'test.' + test - with saved_test_environment(test, verbose, quiet) as environment: - start_time = time.time() - the_module = importlib.import_module(abstest) - # If the test has a test_main, that will run the appropriate - # tests. If not, use normal unittest test loading. - test_runner = getattr(the_module, "test_main", None) - if test_runner is None: - def test_runner(): - loader = unittest.TestLoader() - tests = loader.loadTestsFromModule(the_module) - for error in loader.errors: - print(error, file=sys.stderr) - if loader.errors: - raise Exception("errors while loading tests") - support.run_unittest(tests) - test_runner() - if huntrleaks: - refleak = dash_R(the_module, test, test_runner, huntrleaks) - test_time = time.time() - start_time - except support.ResourceDenied as msg: - if not quiet: - print(test, "skipped --", msg) - sys.stdout.flush() - return RESOURCE_DENIED, test_time - except unittest.SkipTest as msg: - if not quiet: - print(test, "skipped --", msg) - sys.stdout.flush() - return SKIPPED, test_time - except KeyboardInterrupt: - raise - except support.TestFailed as msg: - if display_failure: - print("test", test, "failed --", msg, file=sys.stderr) - else: - print("test", test, "failed", file=sys.stderr) - sys.stderr.flush() - return FAILED, test_time - except: - msg = traceback.format_exc() - print("test", test, "crashed --", msg, file=sys.stderr) - sys.stderr.flush() - return FAILED, test_time - else: - if refleak: - return FAILED, test_time - if environment.changed: - return ENV_CHANGED, test_time - return PASSED, test_time - -def cleanup_test_droppings(testname, verbose): - import shutil - import stat - import gc - - # First kill any dangling references to open files etc. - # This can also issue some ResourceWarnings which would otherwise get - # triggered during the following test run, and possibly produce failures. - gc.collect() - - # Try to clean up junk commonly left behind. While tests shouldn't leave - # any files or directories behind, when a test fails that can be tedious - # for it to arrange. The consequences can be especially nasty on Windows, - # since if a test leaves a file open, it cannot be deleted by name (while - # there's nothing we can do about that here either, we can display the - # name of the offending test, which is a real help). - for name in (support.TESTFN, - "db_home", - ): - if not os.path.exists(name): - continue - - if os.path.isdir(name): - kind, nuker = "directory", shutil.rmtree - elif os.path.isfile(name): - kind, nuker = "file", os.unlink - else: - raise SystemError("os.path says %r exists but is neither " - "directory nor file" % name) - - if verbose: - print("%r left behind %s %r" % (testname, kind, name)) - try: - # if we have chmod, fix possible permissions problems - # that might prevent cleanup - if (hasattr(os, 'chmod')): - os.chmod(name, stat.S_IRWXU | stat.S_IRWXG | stat.S_IRWXO) - nuker(name) - except Exception as msg: - print(("%r left behind %s %r and it couldn't be " - "removed: %s" % (testname, kind, name, msg)), file=sys.stderr) - -def dash_R(the_module, test, indirect_test, huntrleaks): - """Run a test multiple times, looking for reference leaks. - - Returns: - False if the test didn't leak references; True if we detected refleaks. - """ - # This code is hackish and inelegant, but it seems to do the job. - import copyreg - import collections.abc - - if not hasattr(sys, 'gettotalrefcount'): - raise Exception("Tracking reference leaks requires a debug build " - "of Python") - - # Save current values for dash_R_cleanup() to restore. - fs = warnings.filters[:] - ps = copyreg.dispatch_table.copy() - pic = sys.path_importer_cache.copy() - try: - import zipimport - except ImportError: - zdc = None # Run unmodified on platforms without zipimport support - else: - zdc = zipimport._zip_directory_cache.copy() - abcs = {} - for abc in [getattr(collections.abc, a) for a in collections.abc.__all__]: - if not isabstract(abc): - continue - for obj in abc.__subclasses__() + [abc]: - abcs[obj] = obj._abc_registry.copy() - - nwarmup, ntracked, fname = huntrleaks - fname = os.path.join(support.SAVEDCWD, fname) - repcount = nwarmup + ntracked - rc_deltas = [0] * repcount - alloc_deltas = [0] * repcount - - print("beginning", repcount, "repetitions", file=sys.stderr) - print(("1234567890"*(repcount//10 + 1))[:repcount], file=sys.stderr) - sys.stderr.flush() - for i in range(repcount): - indirect_test() - alloc_after, rc_after = dash_R_cleanup(fs, ps, pic, zdc, abcs) - sys.stderr.write('.') - sys.stderr.flush() - if i >= nwarmup: - rc_deltas[i] = rc_after - rc_before - alloc_deltas[i] = alloc_after - alloc_before - alloc_before, rc_before = alloc_after, rc_after - print(file=sys.stderr) - # These checkers return False on success, True on failure - def check_rc_deltas(deltas): - return any(deltas) - def check_alloc_deltas(deltas): - # At least 1/3rd of 0s - if 3 * deltas.count(0) < len(deltas): - return True - # Nothing else than 1s, 0s and -1s - if not set(deltas) <= {1,0,-1}: - return True - return False - failed = False - for deltas, item_name, checker in [ - (rc_deltas, 'references', check_rc_deltas), - (alloc_deltas, 'memory blocks', check_alloc_deltas)]: - if checker(deltas): - msg = '%s leaked %s %s, sum=%s' % ( - test, deltas[nwarmup:], item_name, sum(deltas)) - print(msg, file=sys.stderr) - sys.stderr.flush() - with open(fname, "a") as refrep: - print(msg, file=refrep) - refrep.flush() - failed = True - return failed - -def dash_R_cleanup(fs, ps, pic, zdc, abcs): - import gc, copyreg - import _strptime, linecache - import urllib.parse, urllib.request, mimetypes, doctest - import struct, filecmp, collections.abc - from distutils.dir_util import _path_created - from weakref import WeakSet - - # Clear the warnings registry, so they can be displayed again - for mod in sys.modules.values(): - if hasattr(mod, '__warningregistry__'): - del mod.__warningregistry__ - - # Restore some original values. - warnings.filters[:] = fs - copyreg.dispatch_table.clear() - copyreg.dispatch_table.update(ps) - sys.path_importer_cache.clear() - sys.path_importer_cache.update(pic) - try: - import zipimport - except ImportError: - pass # Run unmodified on platforms without zipimport support - else: - zipimport._zip_directory_cache.clear() - zipimport._zip_directory_cache.update(zdc) - - # clear type cache - sys._clear_type_cache() - - # Clear ABC registries, restoring previously saved ABC registries. - for abc in [getattr(collections.abc, a) for a in collections.abc.__all__]: - if not isabstract(abc): - continue - for obj in abc.__subclasses__() + [abc]: - obj._abc_registry = abcs.get(obj, WeakSet()).copy() - obj._abc_cache.clear() - obj._abc_negative_cache.clear() - - # Flush standard output, so that buffered data is sent to the OS and - # associated Python objects are reclaimed. - for stream in (sys.stdout, sys.stderr, sys.__stdout__, sys.__stderr__): - if stream is not None: - stream.flush() - - # Clear assorted module caches. - _path_created.clear() - re.purge() - _strptime._regex_cache.clear() - urllib.parse.clear_cache() - urllib.request.urlcleanup() - linecache.clearcache() - mimetypes._default_mime_types() - filecmp._cache.clear() - struct._clearcache() - doctest.master = None - try: - import ctypes - except ImportError: - # Don't worry about resetting the cache if ctypes is not supported - pass - else: - ctypes._reset_cache() - - # Collect cyclic trash and read memory statistics immediately after. - func1 = sys.getallocatedblocks - func2 = sys.gettotalrefcount - gc.collect() - return func1(), func2() - -def warm_caches(): - # char cache - s = bytes(range(256)) - for i in range(256): - s[i:i+1] - # unicode cache - x = [chr(i) for i in range(256)] - # int cache - x = list(range(-5, 257)) - -def findtestdir(path=None): - return path or os.path.dirname(__file__) or os.curdir - -def removepy(names): - if not names: - return - for idx, name in enumerate(names): - basename, ext = os.path.splitext(name) - if ext == '.py': - names[idx] = basename - -def count(n, word): - if n == 1: - return "%d %s" % (n, word) - else: - return "%d %ss" % (n, word) - -def printlist(x, width=70, indent=4): - """Print the elements of iterable x to stdout. - - Optional arg width (default 70) is the maximum line length. - Optional arg indent (default 4) is the number of blanks with which to - begin each line. - """ - - from textwrap import fill - blanks = ' ' * indent - # Print the sorted list: 'x' may be a '--random' list or a set() - print(fill(' '.join(str(elt) for elt in sorted(x)), width, - initial_indent=blanks, subsequent_indent=blanks)) - - -def main_in_temp_cwd(): - """Run main() in a temporary working directory.""" - if sysconfig.is_python_build(): - try: - os.mkdir(TEMPDIR) - except FileExistsError: - pass - - # Define a writable temp dir that will be used as cwd while running - # the tests. The name of the dir includes the pid to allow parallel - # testing (see the -j option). - test_cwd = 'test_python_{}'.format(os.getpid()) - test_cwd = os.path.join(TEMPDIR, test_cwd) - - # Run the tests in a context manager that temporarily changes the CWD to a - # temporary and writable directory. If it's not possible to create or - # change the CWD, the original CWD will be used. The original CWD is - # available from support.SAVEDCWD. - with support.temp_cwd(test_cwd, quiet=True): - main() - - -if __name__ == '__main__': - # Remove regrtest.py's own directory from the module search path. Despite - # the elimination of implicit relative imports, this is still needed to - # ensure that submodules of the test package do not inappropriately appear - # as top-level modules even when people (or buildbots!) invoke regrtest.py - # directly instead of using the -m switch - mydir = os.path.abspath(os.path.normpath(os.path.dirname(sys.argv[0]))) - i = len(sys.path) - while i >= 0: - i -= 1 - if os.path.abspath(os.path.normpath(sys.path[i])) == mydir: - del sys.path[i] - - # findtestdir() gets the dirname out of __file__, so we have to make it - # absolute before changing the working directory. - # For example __file__ may be relative when running trace or profile. - # See issue #9323. - __file__ = os.path.abspath(__file__) - - # sanity check - assert __file__ == os.path.abspath(sys.argv[0]) - - main_in_temp_cwd() diff --git a/Lib/test/regrtest.py b/Lib/test/regrtest.py --- a/Lib/test/regrtest.py +++ b/Lib/test/regrtest.py @@ -6,119 +6,6 @@ Run this script with -h or --help for documentation. """ -USAGE = """\ -python -m test [options] [test_name1 [test_name2 ...]] -python path/to/Lib/test/regrtest.py [options] [test_name1 [test_name2 ...]] -""" - -DESCRIPTION = """\ -Run Python regression tests. - -If no arguments or options are provided, finds all files matching -the pattern "test_*" in the Lib/test subdirectory and runs -them in alphabetical order (but see -M and -u, below, for exceptions). - -For more rigorous testing, it is useful to use the following -command line: - -python -E -Wd -m test [options] [test_name1 ...] -""" - -EPILOG = """\ -Additional option details: - --r randomizes test execution order. You can use --randseed=int to provide a -int seed value for the randomizer; this is useful for reproducing troublesome -test orders. - --s On the first invocation of regrtest using -s, the first test file found -or the first test file given on the command line is run, and the name of -the next test is recorded in a file named pynexttest. If run from the -Python build directory, pynexttest is located in the 'build' subdirectory, -otherwise it is located in tempfile.gettempdir(). On subsequent runs, -the test in pynexttest is run, and the next test is written to pynexttest. -When the last test has been run, pynexttest is deleted. In this way it -is possible to single step through the test files. This is useful when -doing memory analysis on the Python interpreter, which process tends to -consume too many resources to run the full regression test non-stop. - --S is used to continue running tests after an aborted run. It will -maintain the order a standard run (ie, this assumes -r is not used). -This is useful after the tests have prematurely stopped for some external -reason and you want to start running from where you left off rather -than starting from the beginning. - --f reads the names of tests from the file given as f's argument, one -or more test names per line. Whitespace is ignored. Blank lines and -lines beginning with '#' are ignored. This is especially useful for -whittling down failures involving interactions among tests. - --L causes the leaks(1) command to be run just before exit if it exists. -leaks(1) is available on Mac OS X and presumably on some other -FreeBSD-derived systems. - --R runs each test several times and examines sys.gettotalrefcount() to -see if the test appears to be leaking references. The argument should -be of the form stab:run:fname where 'stab' is the number of times the -test is run to let gettotalrefcount settle down, 'run' is the number -of times further it is run and 'fname' is the name of the file the -reports are written to. These parameters all have defaults (5, 4 and -"reflog.txt" respectively), and the minimal invocation is '-R :'. - --M runs tests that require an exorbitant amount of memory. These tests -typically try to ascertain containers keep working when containing more than -2 billion objects, which only works on 64-bit systems. There are also some -tests that try to exhaust the address space of the process, which only makes -sense on 32-bit systems with at least 2Gb of memory. The passed-in memlimit, -which is a string in the form of '2.5Gb', determines howmuch memory the -tests will limit themselves to (but they may go slightly over.) The number -shouldn't be more memory than the machine has (including swap memory). You -should also keep in mind that swap memory is generally much, much slower -than RAM, and setting memlimit to all available RAM or higher will heavily -tax the machine. On the other hand, it is no use running these tests with a -limit of less than 2.5Gb, and many require more than 20Gb. Tests that expect -to use more than memlimit memory will be skipped. The big-memory tests -generally run very, very long. - --u is used to specify which special resource intensive tests to run, -such as those requiring large file support or network connectivity. -The argument is a comma-separated list of words indicating the -resources to test. Currently only the following are defined: - - all - Enable all special resources. - - none - Disable all special resources (this is the default). - - audio - Tests that use the audio device. (There are known - cases of broken audio drivers that can crash Python or - even the Linux kernel.) - - curses - Tests that use curses and will modify the terminal's - state and output modes. - - largefile - It is okay to run some test that may create huge - files. These tests can take a long time and may - consume >2GB of disk space temporarily. - - network - It is okay to run tests that use external network - resource, e.g. testing SSL support for sockets. - - decimal - Test the decimal module against a large suite that - verifies compliance with standards. - - cpu - Used for certain CPU-heavy tests. - - subprocess Run all tests for the subprocess module. - - urlfetch - It is okay to download files required on testing. - - gui - Run tests that require a running GUI. - -To enable all resources except one, use '-uall,-'. For -example, to run all the tests except for the gui tests, give the -option '-uall,-gui'. -""" - # We import importlib *ASAP* in order to test #15386 import importlib @@ -153,6 +40,8 @@ except ImportError: multiprocessing = None +from test.libregrtest import _parse_args + # Some times __path__ and __file__ are not absolute (e.g. while running from # Lib/) and, if we change the CWD to run the tests in a temporary dir, some @@ -198,9 +87,6 @@ from test import support -RESOURCE_NAMES = ('audio', 'curses', 'largefile', 'network', - 'decimal', 'cpu', 'subprocess', 'urlfetch', 'gui') - # When tests are run from the Python build directory, it is best practice # to keep the test files in a subfolder. This eases the cleanup of leftover # files using the "make distclean" command. @@ -210,220 +96,6 @@ TEMPDIR = tempfile.gettempdir() TEMPDIR = os.path.abspath(TEMPDIR) -class _ArgParser(argparse.ArgumentParser): - - def error(self, message): - super().error(message + "\nPass -h or --help for complete help.") - -def _create_parser(): - # Set prog to prevent the uninformative "__main__.py" from displaying in - # error messages when using "python -m test ...". - parser = _ArgParser(prog='regrtest.py', - usage=USAGE, - description=DESCRIPTION, - epilog=EPILOG, - add_help=False, - formatter_class=argparse.RawDescriptionHelpFormatter) - - # Arguments with this clause added to its help are described further in - # the epilog's "Additional option details" section. - more_details = ' See the section at bottom for more details.' - - group = parser.add_argument_group('General options') - # We add help explicitly to control what argument group it renders under. - group.add_argument('-h', '--help', action='help', - help='show this help message and exit') - group.add_argument('--timeout', metavar='TIMEOUT', type=float, - help='dump the traceback and exit if a test takes ' - 'more than TIMEOUT seconds; disabled if TIMEOUT ' - 'is negative or equals to zero') - group.add_argument('--wait', action='store_true', - help='wait for user input, e.g., allow a debugger ' - 'to be attached') - group.add_argument('--slaveargs', metavar='ARGS') - group.add_argument('-S', '--start', metavar='START', - help='the name of the test at which to start.' + - more_details) - - group = parser.add_argument_group('Verbosity') - group.add_argument('-v', '--verbose', action='count', - help='run tests in verbose mode with output to stdout') - group.add_argument('-w', '--verbose2', action='store_true', - help='re-run failed tests in verbose mode') - group.add_argument('-W', '--verbose3', action='store_true', - help='display test output on failure') - group.add_argument('-q', '--quiet', action='store_true', - help='no output unless one or more tests fail') - group.add_argument('-o', '--slow', action='store_true', dest='print_slow', - help='print the slowest 10 tests') - group.add_argument('--header', action='store_true', - help='print header with interpreter info') - - group = parser.add_argument_group('Selecting tests') - group.add_argument('-r', '--randomize', action='store_true', - help='randomize test execution order.' + more_details) - group.add_argument('--randseed', metavar='SEED', - dest='random_seed', type=int, - help='pass a random seed to reproduce a previous ' - 'random run') - group.add_argument('-f', '--fromfile', metavar='FILE', - help='read names of tests to run from a file.' + - more_details) - group.add_argument('-x', '--exclude', action='store_true', - help='arguments are tests to *exclude*') - group.add_argument('-s', '--single', action='store_true', - help='single step through a set of tests.' + - more_details) - group.add_argument('-m', '--match', metavar='PAT', - dest='match_tests', - help='match test cases and methods with glob pattern PAT') - group.add_argument('-G', '--failfast', action='store_true', - help='fail as soon as a test fails (only with -v or -W)') - group.add_argument('-u', '--use', metavar='RES1,RES2,...', - action='append', type=resources_list, - help='specify which special resource intensive tests ' - 'to run.' + more_details) - group.add_argument('-M', '--memlimit', metavar='LIMIT', - help='run very large memory-consuming tests.' + - more_details) - group.add_argument('--testdir', metavar='DIR', - type=relative_filename, - help='execute test files in the specified directory ' - '(instead of the Python stdlib test suite)') - - group = parser.add_argument_group('Special runs') - group.add_argument('-l', '--findleaks', action='store_true', - help='if GC is available detect tests that leak memory') - group.add_argument('-L', '--runleaks', action='store_true', - help='run the leaks(1) command just before exit.' + - more_details) - group.add_argument('-R', '--huntrleaks', metavar='RUNCOUNTS', - type=huntrleaks, - help='search for reference leaks (needs debug build, ' - 'very slow).' + more_details) - group.add_argument('-j', '--multiprocess', metavar='PROCESSES', - dest='use_mp', type=int, - help='run PROCESSES processes at once') - group.add_argument('-T', '--coverage', action='store_true', - dest='trace', - help='turn on code coverage tracing using the trace ' - 'module') - group.add_argument('-D', '--coverdir', metavar='DIR', - type=relative_filename, - help='directory where coverage files are put') - group.add_argument('-N', '--nocoverdir', - action='store_const', const=None, dest='coverdir', - help='put coverage files alongside modules') - group.add_argument('-t', '--threshold', metavar='THRESHOLD', - type=int, - help='call gc.set_threshold(THRESHOLD)') - group.add_argument('-n', '--nowindows', action='store_true', - help='suppress error message boxes on Windows') - group.add_argument('-F', '--forever', action='store_true', - help='run the specified tests in a loop, until an ' - 'error happens') - - parser.add_argument('args', nargs=argparse.REMAINDER, - help=argparse.SUPPRESS) - - return parser - -def relative_filename(string): - # CWD is replaced with a temporary dir before calling main(), so we - # join it with the saved CWD so it ends up where the user expects. - return os.path.join(support.SAVEDCWD, string) - -def huntrleaks(string): - args = string.split(':') - if len(args) not in (2, 3): - raise argparse.ArgumentTypeError( - 'needs 2 or 3 colon-separated arguments') - nwarmup = int(args[0]) if args[0] else 5 - ntracked = int(args[1]) if args[1] else 4 - fname = args[2] if len(args) > 2 and args[2] else 'reflog.txt' - return nwarmup, ntracked, fname - -def resources_list(string): - u = [x.lower() for x in string.split(',')] - for r in u: - if r == 'all' or r == 'none': - continue - if r[0] == '-': - r = r[1:] - if r not in RESOURCE_NAMES: - raise argparse.ArgumentTypeError('invalid resource: ' + r) - return u - -def _parse_args(args, **kwargs): - # Defaults - ns = argparse.Namespace(testdir=None, verbose=0, quiet=False, - exclude=False, single=False, randomize=False, fromfile=None, - findleaks=False, use_resources=None, trace=False, coverdir='coverage', - runleaks=False, huntrleaks=False, verbose2=False, print_slow=False, - random_seed=None, use_mp=None, verbose3=False, forever=False, - header=False, failfast=False, match_tests=None) - for k, v in kwargs.items(): - if not hasattr(ns, k): - raise TypeError('%r is an invalid keyword argument ' - 'for this function' % k) - setattr(ns, k, v) - if ns.use_resources is None: - ns.use_resources = [] - - parser = _create_parser() - parser.parse_args(args=args, namespace=ns) - - if ns.single and ns.fromfile: - parser.error("-s and -f don't go together!") - if ns.use_mp and ns.trace: - parser.error("-T and -j don't go together!") - if ns.use_mp and ns.findleaks: - parser.error("-l and -j don't go together!") - if ns.use_mp and ns.memlimit: - parser.error("-M and -j don't go together!") - if ns.failfast and not (ns.verbose or ns.verbose3): - parser.error("-G/--failfast needs either -v or -W") - - if ns.quiet: - ns.verbose = 0 - if ns.timeout is not None: - if hasattr(faulthandler, 'dump_traceback_later'): - if ns.timeout <= 0: - ns.timeout = None - else: - print("Warning: The timeout option requires " - "faulthandler.dump_traceback_later") - ns.timeout = None - if ns.use_mp is not None: - if ns.use_mp <= 0: - # Use all cores + extras for tests that like to sleep - ns.use_mp = 2 + (os.cpu_count() or 1) - if ns.use_mp == 1: - ns.use_mp = None - if ns.use: - for a in ns.use: - for r in a: - if r == 'all': - ns.use_resources[:] = RESOURCE_NAMES - continue - if r == 'none': - del ns.use_resources[:] - continue - remove = False - if r[0] == '-': - remove = True - r = r[1:] - if remove: - if r in ns.use_resources: - ns.use_resources.remove(r) - elif r not in ns.use_resources: - ns.use_resources.append(r) - if ns.random_seed is not None: - ns.randomize = True - - return ns - - def run_test_in_subprocess(testname, ns): """Run the given test in a subprocess with --slaveargs. diff --git a/Lib/test/test_regrtest.py b/Lib/test/test_regrtest.py --- a/Lib/test/test_regrtest.py +++ b/Lib/test/test_regrtest.py @@ -7,7 +7,7 @@ import getopt import os.path import unittest -from test import regrtest, support +from test import regrtest, support, libregrtest class ParseArgsTestCase(unittest.TestCase): @@ -148,7 +148,7 @@ self.assertEqual(ns.use_resources, ['gui', 'network']) ns = regrtest._parse_args([opt, 'gui,none,network']) self.assertEqual(ns.use_resources, ['network']) - expected = list(regrtest.RESOURCE_NAMES) + expected = list(libregrtest.RESOURCE_NAMES) expected.remove('gui') ns = regrtest._parse_args([opt, 'all,-gui']) self.assertEqual(ns.use_resources, expected) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 26 09:53:02 2015 From: python-checkins at python.org (raymond.hettinger) Date: Sat, 26 Sep 2015 07:53:02 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzI1MTM1?= =?utf-8?q?=3A_Avoid_possible_reentrancy_issues_in_deque=5Fclear=2E?= Message-ID: <20150926075302.16569.33146@psf.io> https://hg.python.org/cpython/rev/fc6d62db8d42 changeset: 98281:fc6d62db8d42 branch: 2.7 parent: 98269:e406d62014f7 user: Raymond Hettinger date: Sat Sep 26 00:52:57 2015 -0700 summary: Issue #25135: Avoid possible reentrancy issues in deque_clear. files: Misc/NEWS | 3 + Modules/_collectionsmodule.c | 60 ++++++++++++++++++++++- 2 files changed, 60 insertions(+), 3 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -40,6 +40,9 @@ - Issue #19143: platform module now reads Windows version from kernel32.dll to avoid compatibility shims. +- Issue #25135: Make deque_clear() safer by emptying the deque before clearing. + This helps avoid possible reentrancy issues. + - Issue #24684: socket.socket.getaddrinfo() now calls PyUnicode_AsEncodedString() instead of calling the encode() method of the host, to handle correctly custom unicode string with an encode() method diff --git a/Modules/_collectionsmodule.c b/Modules/_collectionsmodule.c --- a/Modules/_collectionsmodule.c +++ b/Modules/_collectionsmodule.c @@ -644,16 +644,70 @@ static void deque_clear(dequeobject *deque) { + block *b; + block *prevblock; + block *leftblock; + Py_ssize_t leftindex; + Py_ssize_t n; PyObject *item; + /* During the process of clearing a deque, decrefs can cause the + deque to mutate. To avoid fatal confusion, we have to make the + deque empty before clearing the blocks and never refer to + anything via deque->ref while clearing. (This is the same + technique used for clearing lists, sets, and dicts.) + + Making the deque empty requires allocating a new empty block. In + the unlikely event that memory is full, we fall back to an + alternate method that doesn't require a new block. Repeating + pops in a while-loop is slower, possibly re-entrant (and a clever + adversary could cause it to never terminate). + */ + + b = newblock(NULL, NULL, 0); + if (b == NULL) { + PyErr_Clear(); + goto alternate_method; + } + + /* Remember the old size, leftblock, and leftindex */ + leftblock = deque->leftblock; + leftindex = deque->leftindex; + n = deque->len; + + /* Set the deque to be empty using the newly allocated block */ + deque->len = 0; + deque->leftblock = b; + deque->rightblock = b; + deque->leftindex = CENTER + 1; + deque->rightindex = CENTER; + deque->state++; + + /* Now the old size, leftblock, and leftindex are disconnected from + the empty deque and we can use them to decref the pointers. + */ + while (n--) { + item = leftblock->data[leftindex]; + Py_DECREF(item); + leftindex++; + if (leftindex == BLOCKLEN && n) { + assert(leftblock->rightlink != NULL); + prevblock = leftblock; + leftblock = leftblock->rightlink; + leftindex = 0; + freeblock(prevblock); + } + } + assert(leftblock->rightlink == NULL); + freeblock(leftblock); + return; + + alternate_method: while (deque->len) { item = deque_pop(deque, NULL); assert (item != NULL); Py_DECREF(item); } - assert(deque->leftblock == deque->rightblock && - deque->leftindex - 1 == deque->rightindex && - deque->len == 0); } static PyObject * -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 26 10:31:07 2015 From: python-checkins at python.org (raymond.hettinger) Date: Sat, 26 Sep 2015 08:31:07 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Hoist_constant_expression_?= =?utf-8?q?out_of_an_inner_loop=2E?= Message-ID: <20150926083056.31179.6297@psf.io> https://hg.python.org/cpython/rev/96af5df1f665 changeset: 98282:96af5df1f665 parent: 98280:40667f456c6f user: Raymond Hettinger date: Sat Sep 26 01:30:51 2015 -0700 summary: Hoist constant expression out of an inner loop. files: Modules/_collectionsmodule.c | 24 ++++++++++++++++++------ 1 files changed, 18 insertions(+), 6 deletions(-) diff --git a/Modules/_collectionsmodule.c b/Modules/_collectionsmodule.c --- a/Modules/_collectionsmodule.c +++ b/Modules/_collectionsmodule.c @@ -371,6 +371,7 @@ deque_extend(dequeobject *deque, PyObject *iterable) { PyObject *it, *item; + PyObject *(*iternext)(PyObject *); int trim = (deque->maxlen != -1); /* Handle case where id(deque) == id(iterable) */ @@ -399,7 +400,8 @@ if (deque->maxlen == 0) return consume_iterator(it); - while ((item = PyIter_Next(it)) != NULL) { + iternext = *Py_TYPE(it)->tp_iternext; + while ((item = iternext(it)) != NULL) { deque->state++; if (deque->rightindex == BLOCKLEN - 1) { block *b = newblock(Py_SIZE(deque)); @@ -422,8 +424,12 @@ deque_trim_left(deque); } if (PyErr_Occurred()) { - Py_DECREF(it); - return NULL; + if (PyErr_ExceptionMatches(PyExc_StopIteration)) + PyErr_Clear(); + else { + Py_DECREF(it); + return NULL; + } } Py_DECREF(it); Py_RETURN_NONE; @@ -436,6 +442,7 @@ deque_extendleft(dequeobject *deque, PyObject *iterable) { PyObject *it, *item; + PyObject *(*iternext)(PyObject *); int trim = (deque->maxlen != -1); /* Handle case where id(deque) == id(iterable) */ @@ -464,7 +471,8 @@ if (deque->maxlen == 0) return consume_iterator(it); - while ((item = PyIter_Next(it)) != NULL) { + iternext = *Py_TYPE(it)->tp_iternext; + while ((item = iternext(it)) != NULL) { deque->state++; if (deque->leftindex == 0) { block *b = newblock(Py_SIZE(deque)); @@ -487,8 +495,12 @@ deque_trim_right(deque); } if (PyErr_Occurred()) { - Py_DECREF(it); - return NULL; + if (PyErr_ExceptionMatches(PyExc_StopIteration)) + PyErr_Clear(); + else { + Py_DECREF(it); + return NULL; + } } Py_DECREF(it); Py_RETURN_NONE; -- Repository URL: https://hg.python.org/cpython From solipsis at pitrou.net Sat Sep 26 10:40:57 2015 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Sat, 26 Sep 2015 08:40:57 +0000 Subject: [Python-checkins] Daily reference leaks (50a79646044b): sum=17880 Message-ID: <20150926084039.9945.88678@psf.io> results for 50a79646044b on branch "default" -------------------------------------------- test_asyncio leaked [0, 0, 3] memory blocks, sum=3 test_capi leaked [1598, 1598, 1598] references, sum=4794 test_capi leaked [387, 389, 389] memory blocks, sum=1165 test_functools leaked [0, 2, 2] memory blocks, sum=4 test_threading leaked [3196, 3196, 3196] references, sum=9588 test_threading leaked [774, 776, 776] memory blocks, sum=2326 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/psf-users/antoine/refleaks/reflogfYLSaH', '--timeout', '7200'] From python-checkins at python.org Sat Sep 26 11:14:55 2015 From: python-checkins at python.org (raymond.hettinger) Date: Sat, 26 Sep 2015 09:14:55 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Precomputing_the_number_it?= =?utf-8?q?erations_allows_the_inner-loop_to_be_vectorizable=2E?= Message-ID: <20150926091455.16583.16617@psf.io> https://hg.python.org/cpython/rev/821eca6e5a33 changeset: 98283:821eca6e5a33 user: Raymond Hettinger date: Sat Sep 26 02:14:50 2015 -0700 summary: Precomputing the number iterations allows the inner-loop to be vectorizable. files: Modules/_collectionsmodule.c | 8 ++++++-- 1 files changed, 6 insertions(+), 2 deletions(-) diff --git a/Modules/_collectionsmodule.c b/Modules/_collectionsmodule.c --- a/Modules/_collectionsmodule.c +++ b/Modules/_collectionsmodule.c @@ -557,7 +557,7 @@ static PyObject * deque_inplace_repeat(dequeobject *deque, Py_ssize_t n) { - Py_ssize_t i, size; + Py_ssize_t i, m, size; PyObject *seq; PyObject *rv; @@ -598,7 +598,11 @@ MARK_END(b->rightlink); deque->rightindex = -1; } - for ( ; i < n-1 && deque->rightindex != BLOCKLEN - 1 ; i++) { + m = n - 1 - i; + if (m > BLOCKLEN - 1 - deque->rightindex) + m = BLOCKLEN - 1 - deque->rightindex; + i += m; + while (m--) { deque->rightindex++; Py_INCREF(item); deque->rightblock->data[deque->rightindex] = item; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 26 11:25:49 2015 From: python-checkins at python.org (victor.stinner) Date: Sat, 26 Sep 2015 09:25:49 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2325220=3A_Move_mos?= =?utf-8?q?t_regrtest=2Epy_code_to_libregrtest?= Message-ID: <20150926092549.81643.10101@psf.io> https://hg.python.org/cpython/rev/4a9418ed0d0c changeset: 98284:4a9418ed0d0c user: Victor Stinner date: Sat Sep 26 10:38:01 2015 +0200 summary: Issue #25220: Move most regrtest.py code to libregrtest files: Lib/test/libregrtest/__init__.py | 1 + Lib/test/libregrtest/cmdline.py | 2 +- Lib/test/libregrtest/main.py | 749 +---------- Lib/test/libregrtest/refleak.py | 1101 +---------------- Lib/test/libregrtest/runtest.py | 1005 +-------------- Lib/test/libregrtest/save_env.py | 980 +-------------- Lib/test/regrtest.py | 1225 +----------------- Lib/test/test_regrtest.py | 95 +- 8 files changed, 78 insertions(+), 5080 deletions(-) diff --git a/Lib/test/libregrtest/__init__.py b/Lib/test/libregrtest/__init__.py --- a/Lib/test/libregrtest/__init__.py +++ b/Lib/test/libregrtest/__init__.py @@ -1,1 +1,2 @@ from test.libregrtest.cmdline import _parse_args, RESOURCE_NAMES +from test.libregrtest.main import main_in_temp_cwd diff --git a/Lib/test/libregrtest/cmdline.py b/Lib/test/libregrtest/cmdline.py --- a/Lib/test/libregrtest/cmdline.py +++ b/Lib/test/libregrtest/cmdline.py @@ -1,8 +1,8 @@ import argparse import faulthandler import os +from test import support -from test import support USAGE = """\ python -m test [options] [test_name1 [test_name2 ...]] diff --git a/Lib/test/regrtest.py b/Lib/test/libregrtest/main.py old mode 100755 new mode 100644 copy from Lib/test/regrtest.py copy to Lib/test/libregrtest/main.py --- a/Lib/test/regrtest.py +++ b/Lib/test/libregrtest/main.py @@ -1,46 +1,27 @@ -#! /usr/bin/env python3 - -""" -Script to run Python regression tests. - -Run this script with -h or --help for documentation. -""" - -# We import importlib *ASAP* in order to test #15386 -import importlib - -import argparse -import builtins import faulthandler -import io import json -import locale -import logging import os +import re +import sys +import tempfile +import sysconfig +import signal +import random import platform -import random -import re -import shutil -import signal -import sys -import sysconfig -import tempfile -import time import traceback import unittest -import warnings -from inspect import isabstract - +from test.libregrtest.runtest import ( + findtests, runtest, run_test_in_subprocess, + STDTESTS, NOTTESTS, + PASSED, FAILED, ENV_CHANGED, SKIPPED, + RESOURCE_DENIED, INTERRUPTED, CHILD_ERROR) +from test.libregrtest.refleak import warm_caches +from test.libregrtest.cmdline import _parse_args +from test import support try: import threading except ImportError: threading = None -try: - import _multiprocessing, multiprocessing.process -except ImportError: - multiprocessing = None - -from test.libregrtest import _parse_args # Some times __path__ and __file__ are not absolute (e.g. while running from @@ -76,16 +57,6 @@ newsoft = min(hard, max(soft, 1024*2048)) resource.setrlimit(resource.RLIMIT_STACK, (newsoft, hard)) -# Test result constants. -PASSED = 1 -FAILED = 0 -ENV_CHANGED = -1 -SKIPPED = -2 -RESOURCE_DENIED = -3 -INTERRUPTED = -4 -CHILD_ERROR = -5 # error in a child process - -from test import support # When tests are run from the Python build directory, it is best practice # to keep the test files in a subfolder. This eases the cleanup of leftover @@ -96,37 +67,6 @@ TEMPDIR = tempfile.gettempdir() TEMPDIR = os.path.abspath(TEMPDIR) -def run_test_in_subprocess(testname, ns): - """Run the given test in a subprocess with --slaveargs. - - ns is the option Namespace parsed from command-line arguments. regrtest - is invoked in a subprocess with the --slaveargs argument; when the - subprocess exits, its return code, stdout and stderr are returned as a - 3-tuple. - """ - from subprocess import Popen, PIPE - base_cmd = ([sys.executable] + support.args_from_interpreter_flags() + - ['-X', 'faulthandler', '-m', 'test.regrtest']) - - slaveargs = ( - (testname, ns.verbose, ns.quiet), - dict(huntrleaks=ns.huntrleaks, - use_resources=ns.use_resources, - output_on_failure=ns.verbose3, - timeout=ns.timeout, failfast=ns.failfast, - match_tests=ns.match_tests)) - # Running the child from the same working directory as regrtest's original - # invocation ensures that TEMPDIR for the child is the same when - # sysconfig.is_python_build() is true. See issue 15300. - popen = Popen(base_cmd + ['--slaveargs', json.dumps(slaveargs)], - stdout=PIPE, stderr=PIPE, - universal_newlines=True, - close_fds=(os.name != 'nt'), - cwd=support.SAVEDCWD) - stdout, stderr = popen.communicate() - retcode = popen.wait() - return retcode, stdout, stderr - def main(tests=None, **kwargs): """Execute a test suite. @@ -516,36 +456,6 @@ sys.exit(len(bad) > 0 or interrupted) -# small set of tests to determine if we have a basically functioning interpreter -# (i.e. if any of these fail, then anything else is likely to follow) -STDTESTS = [ - 'test_grammar', - 'test_opcodes', - 'test_dict', - 'test_builtin', - 'test_exceptions', - 'test_types', - 'test_unittest', - 'test_doctest', - 'test_doctest2', - 'test_support' -] - -# set of tests that we don't want to be executed when using regrtest -NOTTESTS = set() - -def findtests(testdir=None, stdtests=STDTESTS, nottests=NOTTESTS): - """Return a list of all applicable test modules.""" - testdir = findtestdir(testdir) - names = os.listdir(testdir) - tests = [] - others = set(stdtests) | nottests - for name in names: - mod, ext = os.path.splitext(name) - if mod[:5] == "test_" and ext in (".py", "") and mod not in others: - tests.append(mod) - return stdtests + sorted(tests) - # We do not use a generator so multiple threads can call next(). class MultiprocessTests(object): @@ -565,6 +475,7 @@ raise StopIteration('tests interrupted') return next(self.tests) + def replace_stdout(): """Set stdout encoder error handler to backslashreplace (as stderr error handler) to avoid UnicodeEncodeError when printing a traceback""" @@ -582,609 +493,6 @@ sys.stdout = stdout atexit.register(restore_stdout) -def runtest(test, verbose, quiet, - huntrleaks=False, use_resources=None, - output_on_failure=False, failfast=False, match_tests=None, - timeout=None): - """Run a single test. - - test -- the name of the test - verbose -- if true, print more messages - quiet -- if true, don't print 'skipped' messages (probably redundant) - huntrleaks -- run multiple times to test for leaks; requires a debug - build; a triple corresponding to -R's three arguments - use_resources -- list of extra resources to use - output_on_failure -- if true, display test output on failure - timeout -- dump the traceback and exit if a test takes more than - timeout seconds - failfast, match_tests -- See regrtest command-line flags for these. - - Returns the tuple result, test_time, where result is one of the constants: - INTERRUPTED KeyboardInterrupt when run under -j - RESOURCE_DENIED test skipped because resource denied - SKIPPED test skipped for some other reason - ENV_CHANGED test failed because it changed the execution environment - FAILED test failed - PASSED test passed - """ - - if use_resources is not None: - support.use_resources = use_resources - use_timeout = (timeout is not None) - if use_timeout: - faulthandler.dump_traceback_later(timeout, exit=True) - try: - support.match_tests = match_tests - if failfast: - support.failfast = True - if output_on_failure: - support.verbose = True - - # Reuse the same instance to all calls to runtest(). Some - # tests keep a reference to sys.stdout or sys.stderr - # (eg. test_argparse). - if runtest.stringio is None: - stream = io.StringIO() - runtest.stringio = stream - else: - stream = runtest.stringio - stream.seek(0) - stream.truncate() - - orig_stdout = sys.stdout - orig_stderr = sys.stderr - try: - sys.stdout = stream - sys.stderr = stream - result = runtest_inner(test, verbose, quiet, huntrleaks, - display_failure=False) - if result[0] == FAILED: - output = stream.getvalue() - orig_stderr.write(output) - orig_stderr.flush() - finally: - sys.stdout = orig_stdout - sys.stderr = orig_stderr - else: - support.verbose = verbose # Tell tests to be moderately quiet - result = runtest_inner(test, verbose, quiet, huntrleaks, - display_failure=not verbose) - return result - finally: - if use_timeout: - faulthandler.cancel_dump_traceback_later() - cleanup_test_droppings(test, verbose) -runtest.stringio = None - -# Unit tests are supposed to leave the execution environment unchanged -# once they complete. But sometimes tests have bugs, especially when -# tests fail, and the changes to environment go on to mess up other -# tests. This can cause issues with buildbot stability, since tests -# are run in random order and so problems may appear to come and go. -# There are a few things we can save and restore to mitigate this, and -# the following context manager handles this task. - -class saved_test_environment: - """Save bits of the test environment and restore them at block exit. - - with saved_test_environment(testname, verbose, quiet): - #stuff - - Unless quiet is True, a warning is printed to stderr if any of - the saved items was changed by the test. The attribute 'changed' - is initially False, but is set to True if a change is detected. - - If verbose is more than 1, the before and after state of changed - items is also printed. - """ - - changed = False - - def __init__(self, testname, verbose=0, quiet=False): - self.testname = testname - self.verbose = verbose - self.quiet = quiet - - # To add things to save and restore, add a name XXX to the resources list - # and add corresponding get_XXX/restore_XXX functions. get_XXX should - # return the value to be saved and compared against a second call to the - # get function when test execution completes. restore_XXX should accept - # the saved value and restore the resource using it. It will be called if - # and only if a change in the value is detected. - # - # Note: XXX will have any '.' replaced with '_' characters when determining - # the corresponding method names. - - resources = ('sys.argv', 'cwd', 'sys.stdin', 'sys.stdout', 'sys.stderr', - 'os.environ', 'sys.path', 'sys.path_hooks', '__import__', - 'warnings.filters', 'asyncore.socket_map', - 'logging._handlers', 'logging._handlerList', 'sys.gettrace', - 'sys.warnoptions', - # multiprocessing.process._cleanup() may release ref - # to a thread, so check processes first. - 'multiprocessing.process._dangling', 'threading._dangling', - 'sysconfig._CONFIG_VARS', 'sysconfig._INSTALL_SCHEMES', - 'files', 'locale', 'warnings.showwarning', - ) - - def get_sys_argv(self): - return id(sys.argv), sys.argv, sys.argv[:] - def restore_sys_argv(self, saved_argv): - sys.argv = saved_argv[1] - sys.argv[:] = saved_argv[2] - - def get_cwd(self): - return os.getcwd() - def restore_cwd(self, saved_cwd): - os.chdir(saved_cwd) - - def get_sys_stdout(self): - return sys.stdout - def restore_sys_stdout(self, saved_stdout): - sys.stdout = saved_stdout - - def get_sys_stderr(self): - return sys.stderr - def restore_sys_stderr(self, saved_stderr): - sys.stderr = saved_stderr - - def get_sys_stdin(self): - return sys.stdin - def restore_sys_stdin(self, saved_stdin): - sys.stdin = saved_stdin - - def get_os_environ(self): - return id(os.environ), os.environ, dict(os.environ) - def restore_os_environ(self, saved_environ): - os.environ = saved_environ[1] - os.environ.clear() - os.environ.update(saved_environ[2]) - - def get_sys_path(self): - return id(sys.path), sys.path, sys.path[:] - def restore_sys_path(self, saved_path): - sys.path = saved_path[1] - sys.path[:] = saved_path[2] - - def get_sys_path_hooks(self): - return id(sys.path_hooks), sys.path_hooks, sys.path_hooks[:] - def restore_sys_path_hooks(self, saved_hooks): - sys.path_hooks = saved_hooks[1] - sys.path_hooks[:] = saved_hooks[2] - - def get_sys_gettrace(self): - return sys.gettrace() - def restore_sys_gettrace(self, trace_fxn): - sys.settrace(trace_fxn) - - def get___import__(self): - return builtins.__import__ - def restore___import__(self, import_): - builtins.__import__ = import_ - - def get_warnings_filters(self): - return id(warnings.filters), warnings.filters, warnings.filters[:] - def restore_warnings_filters(self, saved_filters): - warnings.filters = saved_filters[1] - warnings.filters[:] = saved_filters[2] - - def get_asyncore_socket_map(self): - asyncore = sys.modules.get('asyncore') - # XXX Making a copy keeps objects alive until __exit__ gets called. - return asyncore and asyncore.socket_map.copy() or {} - def restore_asyncore_socket_map(self, saved_map): - asyncore = sys.modules.get('asyncore') - if asyncore is not None: - asyncore.close_all(ignore_all=True) - asyncore.socket_map.update(saved_map) - - def get_shutil_archive_formats(self): - # we could call get_archives_formats() but that only returns the - # registry keys; we want to check the values too (the functions that - # are registered) - return shutil._ARCHIVE_FORMATS, shutil._ARCHIVE_FORMATS.copy() - def restore_shutil_archive_formats(self, saved): - shutil._ARCHIVE_FORMATS = saved[0] - shutil._ARCHIVE_FORMATS.clear() - shutil._ARCHIVE_FORMATS.update(saved[1]) - - def get_shutil_unpack_formats(self): - return shutil._UNPACK_FORMATS, shutil._UNPACK_FORMATS.copy() - def restore_shutil_unpack_formats(self, saved): - shutil._UNPACK_FORMATS = saved[0] - shutil._UNPACK_FORMATS.clear() - shutil._UNPACK_FORMATS.update(saved[1]) - - def get_logging__handlers(self): - # _handlers is a WeakValueDictionary - return id(logging._handlers), logging._handlers, logging._handlers.copy() - def restore_logging__handlers(self, saved_handlers): - # Can't easily revert the logging state - pass - - def get_logging__handlerList(self): - # _handlerList is a list of weakrefs to handlers - return id(logging._handlerList), logging._handlerList, logging._handlerList[:] - def restore_logging__handlerList(self, saved_handlerList): - # Can't easily revert the logging state - pass - - def get_sys_warnoptions(self): - return id(sys.warnoptions), sys.warnoptions, sys.warnoptions[:] - def restore_sys_warnoptions(self, saved_options): - sys.warnoptions = saved_options[1] - sys.warnoptions[:] = saved_options[2] - - # Controlling dangling references to Thread objects can make it easier - # to track reference leaks. - def get_threading__dangling(self): - if not threading: - return None - # This copies the weakrefs without making any strong reference - return threading._dangling.copy() - def restore_threading__dangling(self, saved): - if not threading: - return - threading._dangling.clear() - threading._dangling.update(saved) - - # Same for Process objects - def get_multiprocessing_process__dangling(self): - if not multiprocessing: - return None - # Unjoined process objects can survive after process exits - multiprocessing.process._cleanup() - # This copies the weakrefs without making any strong reference - return multiprocessing.process._dangling.copy() - def restore_multiprocessing_process__dangling(self, saved): - if not multiprocessing: - return - multiprocessing.process._dangling.clear() - multiprocessing.process._dangling.update(saved) - - def get_sysconfig__CONFIG_VARS(self): - # make sure the dict is initialized - sysconfig.get_config_var('prefix') - return (id(sysconfig._CONFIG_VARS), sysconfig._CONFIG_VARS, - dict(sysconfig._CONFIG_VARS)) - def restore_sysconfig__CONFIG_VARS(self, saved): - sysconfig._CONFIG_VARS = saved[1] - sysconfig._CONFIG_VARS.clear() - sysconfig._CONFIG_VARS.update(saved[2]) - - def get_sysconfig__INSTALL_SCHEMES(self): - return (id(sysconfig._INSTALL_SCHEMES), sysconfig._INSTALL_SCHEMES, - sysconfig._INSTALL_SCHEMES.copy()) - def restore_sysconfig__INSTALL_SCHEMES(self, saved): - sysconfig._INSTALL_SCHEMES = saved[1] - sysconfig._INSTALL_SCHEMES.clear() - sysconfig._INSTALL_SCHEMES.update(saved[2]) - - def get_files(self): - return sorted(fn + ('/' if os.path.isdir(fn) else '') - for fn in os.listdir()) - def restore_files(self, saved_value): - fn = support.TESTFN - if fn not in saved_value and (fn + '/') not in saved_value: - if os.path.isfile(fn): - support.unlink(fn) - elif os.path.isdir(fn): - support.rmtree(fn) - - _lc = [getattr(locale, lc) for lc in dir(locale) - if lc.startswith('LC_')] - def get_locale(self): - pairings = [] - for lc in self._lc: - try: - pairings.append((lc, locale.setlocale(lc, None))) - except (TypeError, ValueError): - continue - return pairings - def restore_locale(self, saved): - for lc, setting in saved: - locale.setlocale(lc, setting) - - def get_warnings_showwarning(self): - return warnings.showwarning - def restore_warnings_showwarning(self, fxn): - warnings.showwarning = fxn - - def resource_info(self): - for name in self.resources: - method_suffix = name.replace('.', '_') - get_name = 'get_' + method_suffix - restore_name = 'restore_' + method_suffix - yield name, getattr(self, get_name), getattr(self, restore_name) - - def __enter__(self): - self.saved_values = dict((name, get()) for name, get, restore - in self.resource_info()) - return self - - def __exit__(self, exc_type, exc_val, exc_tb): - saved_values = self.saved_values - del self.saved_values - for name, get, restore in self.resource_info(): - current = get() - original = saved_values.pop(name) - # Check for changes to the resource's value - if current != original: - self.changed = True - restore(original) - if not self.quiet: - print("Warning -- {} was modified by {}".format( - name, self.testname), - file=sys.stderr) - if self.verbose > 1: - print(" Before: {}\n After: {} ".format( - original, current), - file=sys.stderr) - return False - - -def runtest_inner(test, verbose, quiet, - huntrleaks=False, display_failure=True): - support.unload(test) - - test_time = 0.0 - refleak = False # True if the test leaked references. - try: - if test.startswith('test.'): - abstest = test - else: - # Always import it from the test package - abstest = 'test.' + test - with saved_test_environment(test, verbose, quiet) as environment: - start_time = time.time() - the_module = importlib.import_module(abstest) - # If the test has a test_main, that will run the appropriate - # tests. If not, use normal unittest test loading. - test_runner = getattr(the_module, "test_main", None) - if test_runner is None: - def test_runner(): - loader = unittest.TestLoader() - tests = loader.loadTestsFromModule(the_module) - for error in loader.errors: - print(error, file=sys.stderr) - if loader.errors: - raise Exception("errors while loading tests") - support.run_unittest(tests) - test_runner() - if huntrleaks: - refleak = dash_R(the_module, test, test_runner, huntrleaks) - test_time = time.time() - start_time - except support.ResourceDenied as msg: - if not quiet: - print(test, "skipped --", msg) - sys.stdout.flush() - return RESOURCE_DENIED, test_time - except unittest.SkipTest as msg: - if not quiet: - print(test, "skipped --", msg) - sys.stdout.flush() - return SKIPPED, test_time - except KeyboardInterrupt: - raise - except support.TestFailed as msg: - if display_failure: - print("test", test, "failed --", msg, file=sys.stderr) - else: - print("test", test, "failed", file=sys.stderr) - sys.stderr.flush() - return FAILED, test_time - except: - msg = traceback.format_exc() - print("test", test, "crashed --", msg, file=sys.stderr) - sys.stderr.flush() - return FAILED, test_time - else: - if refleak: - return FAILED, test_time - if environment.changed: - return ENV_CHANGED, test_time - return PASSED, test_time - -def cleanup_test_droppings(testname, verbose): - import shutil - import stat - import gc - - # First kill any dangling references to open files etc. - # This can also issue some ResourceWarnings which would otherwise get - # triggered during the following test run, and possibly produce failures. - gc.collect() - - # Try to clean up junk commonly left behind. While tests shouldn't leave - # any files or directories behind, when a test fails that can be tedious - # for it to arrange. The consequences can be especially nasty on Windows, - # since if a test leaves a file open, it cannot be deleted by name (while - # there's nothing we can do about that here either, we can display the - # name of the offending test, which is a real help). - for name in (support.TESTFN, - "db_home", - ): - if not os.path.exists(name): - continue - - if os.path.isdir(name): - kind, nuker = "directory", shutil.rmtree - elif os.path.isfile(name): - kind, nuker = "file", os.unlink - else: - raise SystemError("os.path says %r exists but is neither " - "directory nor file" % name) - - if verbose: - print("%r left behind %s %r" % (testname, kind, name)) - try: - # if we have chmod, fix possible permissions problems - # that might prevent cleanup - if (hasattr(os, 'chmod')): - os.chmod(name, stat.S_IRWXU | stat.S_IRWXG | stat.S_IRWXO) - nuker(name) - except Exception as msg: - print(("%r left behind %s %r and it couldn't be " - "removed: %s" % (testname, kind, name, msg)), file=sys.stderr) - -def dash_R(the_module, test, indirect_test, huntrleaks): - """Run a test multiple times, looking for reference leaks. - - Returns: - False if the test didn't leak references; True if we detected refleaks. - """ - # This code is hackish and inelegant, but it seems to do the job. - import copyreg - import collections.abc - - if not hasattr(sys, 'gettotalrefcount'): - raise Exception("Tracking reference leaks requires a debug build " - "of Python") - - # Save current values for dash_R_cleanup() to restore. - fs = warnings.filters[:] - ps = copyreg.dispatch_table.copy() - pic = sys.path_importer_cache.copy() - try: - import zipimport - except ImportError: - zdc = None # Run unmodified on platforms without zipimport support - else: - zdc = zipimport._zip_directory_cache.copy() - abcs = {} - for abc in [getattr(collections.abc, a) for a in collections.abc.__all__]: - if not isabstract(abc): - continue - for obj in abc.__subclasses__() + [abc]: - abcs[obj] = obj._abc_registry.copy() - - nwarmup, ntracked, fname = huntrleaks - fname = os.path.join(support.SAVEDCWD, fname) - repcount = nwarmup + ntracked - rc_deltas = [0] * repcount - alloc_deltas = [0] * repcount - - print("beginning", repcount, "repetitions", file=sys.stderr) - print(("1234567890"*(repcount//10 + 1))[:repcount], file=sys.stderr) - sys.stderr.flush() - for i in range(repcount): - indirect_test() - alloc_after, rc_after = dash_R_cleanup(fs, ps, pic, zdc, abcs) - sys.stderr.write('.') - sys.stderr.flush() - if i >= nwarmup: - rc_deltas[i] = rc_after - rc_before - alloc_deltas[i] = alloc_after - alloc_before - alloc_before, rc_before = alloc_after, rc_after - print(file=sys.stderr) - # These checkers return False on success, True on failure - def check_rc_deltas(deltas): - return any(deltas) - def check_alloc_deltas(deltas): - # At least 1/3rd of 0s - if 3 * deltas.count(0) < len(deltas): - return True - # Nothing else than 1s, 0s and -1s - if not set(deltas) <= {1,0,-1}: - return True - return False - failed = False - for deltas, item_name, checker in [ - (rc_deltas, 'references', check_rc_deltas), - (alloc_deltas, 'memory blocks', check_alloc_deltas)]: - if checker(deltas): - msg = '%s leaked %s %s, sum=%s' % ( - test, deltas[nwarmup:], item_name, sum(deltas)) - print(msg, file=sys.stderr) - sys.stderr.flush() - with open(fname, "a") as refrep: - print(msg, file=refrep) - refrep.flush() - failed = True - return failed - -def dash_R_cleanup(fs, ps, pic, zdc, abcs): - import gc, copyreg - import _strptime, linecache - import urllib.parse, urllib.request, mimetypes, doctest - import struct, filecmp, collections.abc - from distutils.dir_util import _path_created - from weakref import WeakSet - - # Clear the warnings registry, so they can be displayed again - for mod in sys.modules.values(): - if hasattr(mod, '__warningregistry__'): - del mod.__warningregistry__ - - # Restore some original values. - warnings.filters[:] = fs - copyreg.dispatch_table.clear() - copyreg.dispatch_table.update(ps) - sys.path_importer_cache.clear() - sys.path_importer_cache.update(pic) - try: - import zipimport - except ImportError: - pass # Run unmodified on platforms without zipimport support - else: - zipimport._zip_directory_cache.clear() - zipimport._zip_directory_cache.update(zdc) - - # clear type cache - sys._clear_type_cache() - - # Clear ABC registries, restoring previously saved ABC registries. - for abc in [getattr(collections.abc, a) for a in collections.abc.__all__]: - if not isabstract(abc): - continue - for obj in abc.__subclasses__() + [abc]: - obj._abc_registry = abcs.get(obj, WeakSet()).copy() - obj._abc_cache.clear() - obj._abc_negative_cache.clear() - - # Flush standard output, so that buffered data is sent to the OS and - # associated Python objects are reclaimed. - for stream in (sys.stdout, sys.stderr, sys.__stdout__, sys.__stderr__): - if stream is not None: - stream.flush() - - # Clear assorted module caches. - _path_created.clear() - re.purge() - _strptime._regex_cache.clear() - urllib.parse.clear_cache() - urllib.request.urlcleanup() - linecache.clearcache() - mimetypes._default_mime_types() - filecmp._cache.clear() - struct._clearcache() - doctest.master = None - try: - import ctypes - except ImportError: - # Don't worry about resetting the cache if ctypes is not supported - pass - else: - ctypes._reset_cache() - - # Collect cyclic trash and read memory statistics immediately after. - func1 = sys.getallocatedblocks - func2 = sys.gettotalrefcount - gc.collect() - return func1(), func2() - -def warm_caches(): - # char cache - s = bytes(range(256)) - for i in range(256): - s[i:i+1] - # unicode cache - x = [chr(i) for i in range(256)] - # int cache - x = list(range(-5, 257)) - -def findtestdir(path=None): - return path or os.path.dirname(__file__) or os.curdir def removepy(names): if not names: @@ -1194,12 +502,14 @@ if ext == '.py': names[idx] = basename + def count(n, word): if n == 1: return "%d %s" % (n, word) else: return "%d %ss" % (n, word) + def printlist(x, width=70, indent=4): """Print the elements of iterable x to stdout. @@ -1235,28 +545,3 @@ # available from support.SAVEDCWD. with support.temp_cwd(test_cwd, quiet=True): main() - - -if __name__ == '__main__': - # Remove regrtest.py's own directory from the module search path. Despite - # the elimination of implicit relative imports, this is still needed to - # ensure that submodules of the test package do not inappropriately appear - # as top-level modules even when people (or buildbots!) invoke regrtest.py - # directly instead of using the -m switch - mydir = os.path.abspath(os.path.normpath(os.path.dirname(sys.argv[0]))) - i = len(sys.path) - while i >= 0: - i -= 1 - if os.path.abspath(os.path.normpath(sys.path[i])) == mydir: - del sys.path[i] - - # findtestdir() gets the dirname out of __file__, so we have to make it - # absolute before changing the working directory. - # For example __file__ may be relative when running trace or profile. - # See issue #9323. - __file__ = os.path.abspath(__file__) - - # sanity check - assert __file__ == os.path.abspath(sys.argv[0]) - - main_in_temp_cwd() diff --git a/Lib/test/regrtest.py b/Lib/test/libregrtest/refleak.py old mode 100755 new mode 100644 copy from Lib/test/regrtest.py copy to Lib/test/libregrtest/refleak.py --- a/Lib/test/regrtest.py +++ b/Lib/test/libregrtest/refleak.py @@ -1,1031 +1,10 @@ -#! /usr/bin/env python3 - -""" -Script to run Python regression tests. - -Run this script with -h or --help for documentation. -""" - -# We import importlib *ASAP* in order to test #15386 -import importlib - -import argparse -import builtins -import faulthandler -import io -import json -import locale -import logging import os -import platform -import random import re -import shutil -import signal import sys -import sysconfig -import tempfile -import time -import traceback -import unittest import warnings from inspect import isabstract - -try: - import threading -except ImportError: - threading = None -try: - import _multiprocessing, multiprocessing.process -except ImportError: - multiprocessing = None - -from test.libregrtest import _parse_args - - -# Some times __path__ and __file__ are not absolute (e.g. while running from -# Lib/) and, if we change the CWD to run the tests in a temporary dir, some -# imports might fail. This affects only the modules imported before os.chdir(). -# These modules are searched first in sys.path[0] (so '' -- the CWD) and if -# they are found in the CWD their __file__ and __path__ will be relative (this -# happens before the chdir). All the modules imported after the chdir, are -# not found in the CWD, and since the other paths in sys.path[1:] are absolute -# (site.py absolutize them), the __file__ and __path__ will be absolute too. -# Therefore it is necessary to absolutize manually the __file__ and __path__ of -# the packages to prevent later imports to fail when the CWD is different. -for module in sys.modules.values(): - if hasattr(module, '__path__'): - module.__path__ = [os.path.abspath(path) for path in module.__path__] - if hasattr(module, '__file__'): - module.__file__ = os.path.abspath(module.__file__) - - -# MacOSX (a.k.a. Darwin) has a default stack size that is too small -# for deeply recursive regular expressions. We see this as crashes in -# the Python test suite when running test_re.py and test_sre.py. The -# fix is to set the stack limit to 2048. -# This approach may also be useful for other Unixy platforms that -# suffer from small default stack limits. -if sys.platform == 'darwin': - try: - import resource - except ImportError: - pass - else: - soft, hard = resource.getrlimit(resource.RLIMIT_STACK) - newsoft = min(hard, max(soft, 1024*2048)) - resource.setrlimit(resource.RLIMIT_STACK, (newsoft, hard)) - -# Test result constants. -PASSED = 1 -FAILED = 0 -ENV_CHANGED = -1 -SKIPPED = -2 -RESOURCE_DENIED = -3 -INTERRUPTED = -4 -CHILD_ERROR = -5 # error in a child process - from test import support -# When tests are run from the Python build directory, it is best practice -# to keep the test files in a subfolder. This eases the cleanup of leftover -# files using the "make distclean" command. -if sysconfig.is_python_build(): - TEMPDIR = os.path.join(sysconfig.get_config_var('srcdir'), 'build') -else: - TEMPDIR = tempfile.gettempdir() -TEMPDIR = os.path.abspath(TEMPDIR) - -def run_test_in_subprocess(testname, ns): - """Run the given test in a subprocess with --slaveargs. - - ns is the option Namespace parsed from command-line arguments. regrtest - is invoked in a subprocess with the --slaveargs argument; when the - subprocess exits, its return code, stdout and stderr are returned as a - 3-tuple. - """ - from subprocess import Popen, PIPE - base_cmd = ([sys.executable] + support.args_from_interpreter_flags() + - ['-X', 'faulthandler', '-m', 'test.regrtest']) - - slaveargs = ( - (testname, ns.verbose, ns.quiet), - dict(huntrleaks=ns.huntrleaks, - use_resources=ns.use_resources, - output_on_failure=ns.verbose3, - timeout=ns.timeout, failfast=ns.failfast, - match_tests=ns.match_tests)) - # Running the child from the same working directory as regrtest's original - # invocation ensures that TEMPDIR for the child is the same when - # sysconfig.is_python_build() is true. See issue 15300. - popen = Popen(base_cmd + ['--slaveargs', json.dumps(slaveargs)], - stdout=PIPE, stderr=PIPE, - universal_newlines=True, - close_fds=(os.name != 'nt'), - cwd=support.SAVEDCWD) - stdout, stderr = popen.communicate() - retcode = popen.wait() - return retcode, stdout, stderr - - -def main(tests=None, **kwargs): - """Execute a test suite. - - This also parses command-line options and modifies its behavior - accordingly. - - tests -- a list of strings containing test names (optional) - testdir -- the directory in which to look for tests (optional) - - Users other than the Python test suite will certainly want to - specify testdir; if it's omitted, the directory containing the - Python test suite is searched for. - - If the tests argument is omitted, the tests listed on the - command-line will be used. If that's empty, too, then all *.py - files beginning with test_ will be used. - - The other default arguments (verbose, quiet, exclude, - single, randomize, findleaks, use_resources, trace, coverdir, - print_slow, and random_seed) allow programmers calling main() - directly to set the values that would normally be set by flags - on the command line. - """ - # Display the Python traceback on fatal errors (e.g. segfault) - faulthandler.enable(all_threads=True) - - # Display the Python traceback on SIGALRM or SIGUSR1 signal - signals = [] - if hasattr(signal, 'SIGALRM'): - signals.append(signal.SIGALRM) - if hasattr(signal, 'SIGUSR1'): - signals.append(signal.SIGUSR1) - for signum in signals: - faulthandler.register(signum, chain=True) - - replace_stdout() - - support.record_original_stdout(sys.stdout) - - ns = _parse_args(sys.argv[1:], **kwargs) - - if ns.huntrleaks: - # Avoid false positives due to various caches - # filling slowly with random data: - warm_caches() - if ns.memlimit is not None: - support.set_memlimit(ns.memlimit) - if ns.threshold is not None: - import gc - gc.set_threshold(ns.threshold) - if ns.nowindows: - import msvcrt - msvcrt.SetErrorMode(msvcrt.SEM_FAILCRITICALERRORS| - msvcrt.SEM_NOALIGNMENTFAULTEXCEPT| - msvcrt.SEM_NOGPFAULTERRORBOX| - msvcrt.SEM_NOOPENFILEERRORBOX) - try: - msvcrt.CrtSetReportMode - except AttributeError: - # release build - pass - else: - for m in [msvcrt.CRT_WARN, msvcrt.CRT_ERROR, msvcrt.CRT_ASSERT]: - msvcrt.CrtSetReportMode(m, msvcrt.CRTDBG_MODE_FILE) - msvcrt.CrtSetReportFile(m, msvcrt.CRTDBG_FILE_STDERR) - if ns.wait: - input("Press any key to continue...") - - if ns.slaveargs is not None: - args, kwargs = json.loads(ns.slaveargs) - if kwargs.get('huntrleaks'): - unittest.BaseTestSuite._cleanup = False - try: - result = runtest(*args, **kwargs) - except KeyboardInterrupt: - result = INTERRUPTED, '' - except BaseException as e: - traceback.print_exc() - result = CHILD_ERROR, str(e) - sys.stdout.flush() - print() # Force a newline (just in case) - print(json.dumps(result)) - sys.exit(0) - - good = [] - bad = [] - skipped = [] - resource_denieds = [] - environment_changed = [] - interrupted = False - - if ns.findleaks: - try: - import gc - except ImportError: - print('No GC available, disabling findleaks.') - ns.findleaks = False - else: - # Uncomment the line below to report garbage that is not - # freeable by reference counting alone. By default only - # garbage that is not collectable by the GC is reported. - #gc.set_debug(gc.DEBUG_SAVEALL) - found_garbage = [] - - if ns.huntrleaks: - unittest.BaseTestSuite._cleanup = False - - if ns.single: - filename = os.path.join(TEMPDIR, 'pynexttest') - try: - with open(filename, 'r') as fp: - next_test = fp.read().strip() - tests = [next_test] - except OSError: - pass - - if ns.fromfile: - tests = [] - with open(os.path.join(support.SAVEDCWD, ns.fromfile)) as fp: - count_pat = re.compile(r'\[\s*\d+/\s*\d+\]') - for line in fp: - line = count_pat.sub('', line) - guts = line.split() # assuming no test has whitespace in its name - if guts and not guts[0].startswith('#'): - tests.extend(guts) - - # Strip .py extensions. - removepy(ns.args) - removepy(tests) - - stdtests = STDTESTS[:] - nottests = NOTTESTS.copy() - if ns.exclude: - for arg in ns.args: - if arg in stdtests: - stdtests.remove(arg) - nottests.add(arg) - ns.args = [] - - # For a partial run, we do not need to clutter the output. - if ns.verbose or ns.header or not (ns.quiet or ns.single or tests or ns.args): - # Print basic platform information - print("==", platform.python_implementation(), *sys.version.split()) - print("== ", platform.platform(aliased=True), - "%s-endian" % sys.byteorder) - print("== ", "hash algorithm:", sys.hash_info.algorithm, - "64bit" if sys.maxsize > 2**32 else "32bit") - print("== ", os.getcwd()) - print("Testing with flags:", sys.flags) - - # if testdir is set, then we are not running the python tests suite, so - # don't add default tests to be executed or skipped (pass empty values) - if ns.testdir: - alltests = findtests(ns.testdir, list(), set()) - else: - alltests = findtests(ns.testdir, stdtests, nottests) - - selected = tests or ns.args or alltests - if ns.single: - selected = selected[:1] - try: - next_single_test = alltests[alltests.index(selected[0])+1] - except IndexError: - next_single_test = None - # Remove all the selected tests that precede start if it's set. - if ns.start: - try: - del selected[:selected.index(ns.start)] - except ValueError: - print("Couldn't find starting test (%s), using all tests" % ns.start) - if ns.randomize: - if ns.random_seed is None: - ns.random_seed = random.randrange(10000000) - random.seed(ns.random_seed) - print("Using random seed", ns.random_seed) - random.shuffle(selected) - if ns.trace: - import trace, tempfile - tracer = trace.Trace(ignoredirs=[sys.base_prefix, sys.base_exec_prefix, - tempfile.gettempdir()], - trace=False, count=True) - - test_times = [] - support.verbose = ns.verbose # Tell tests to be moderately quiet - support.use_resources = ns.use_resources - save_modules = sys.modules.keys() - - def accumulate_result(test, result): - ok, test_time = result - test_times.append((test_time, test)) - if ok == PASSED: - good.append(test) - elif ok == FAILED: - bad.append(test) - elif ok == ENV_CHANGED: - environment_changed.append(test) - elif ok == SKIPPED: - skipped.append(test) - elif ok == RESOURCE_DENIED: - skipped.append(test) - resource_denieds.append(test) - - if ns.forever: - def test_forever(tests=list(selected)): - while True: - for test in tests: - yield test - if bad: - return - tests = test_forever() - test_count = '' - test_count_width = 3 - else: - tests = iter(selected) - test_count = '/{}'.format(len(selected)) - test_count_width = len(test_count) - 1 - - if ns.use_mp: - try: - from threading import Thread - except ImportError: - print("Multiprocess option requires thread support") - sys.exit(2) - from queue import Queue - debug_output_pat = re.compile(r"\[\d+ refs, \d+ blocks\]$") - output = Queue() - pending = MultiprocessTests(tests) - def work(): - # A worker thread. - try: - while True: - try: - test = next(pending) - except StopIteration: - output.put((None, None, None, None)) - return - retcode, stdout, stderr = run_test_in_subprocess(test, ns) - # Strip last refcount output line if it exists, since it - # comes from the shutdown of the interpreter in the subcommand. - stderr = debug_output_pat.sub("", stderr) - stdout, _, result = stdout.strip().rpartition("\n") - if retcode != 0: - result = (CHILD_ERROR, "Exit code %s" % retcode) - output.put((test, stdout.rstrip(), stderr.rstrip(), result)) - return - if not result: - output.put((None, None, None, None)) - return - result = json.loads(result) - output.put((test, stdout.rstrip(), stderr.rstrip(), result)) - except BaseException: - output.put((None, None, None, None)) - raise - workers = [Thread(target=work) for i in range(ns.use_mp)] - for worker in workers: - worker.start() - finished = 0 - test_index = 1 - try: - while finished < ns.use_mp: - test, stdout, stderr, result = output.get() - if test is None: - finished += 1 - continue - accumulate_result(test, result) - if not ns.quiet: - fmt = "[{1:{0}}{2}/{3}] {4}" if bad else "[{1:{0}}{2}] {4}" - print(fmt.format( - test_count_width, test_index, test_count, - len(bad), test)) - if stdout: - print(stdout) - if stderr: - print(stderr, file=sys.stderr) - sys.stdout.flush() - sys.stderr.flush() - if result[0] == INTERRUPTED: - raise KeyboardInterrupt - if result[0] == CHILD_ERROR: - raise Exception("Child error on {}: {}".format(test, result[1])) - test_index += 1 - except KeyboardInterrupt: - interrupted = True - pending.interrupted = True - for worker in workers: - worker.join() - else: - for test_index, test in enumerate(tests, 1): - if not ns.quiet: - fmt = "[{1:{0}}{2}/{3}] {4}" if bad else "[{1:{0}}{2}] {4}" - print(fmt.format( - test_count_width, test_index, test_count, len(bad), test)) - sys.stdout.flush() - if ns.trace: - # If we're tracing code coverage, then we don't exit with status - # if on a false return value from main. - tracer.runctx('runtest(test, ns.verbose, ns.quiet, timeout=ns.timeout)', - globals=globals(), locals=vars()) - else: - try: - result = runtest(test, ns.verbose, ns.quiet, - ns.huntrleaks, - output_on_failure=ns.verbose3, - timeout=ns.timeout, failfast=ns.failfast, - match_tests=ns.match_tests) - accumulate_result(test, result) - except KeyboardInterrupt: - interrupted = True - break - if ns.findleaks: - gc.collect() - if gc.garbage: - print("Warning: test created", len(gc.garbage), end=' ') - print("uncollectable object(s).") - # move the uncollectable objects somewhere so we don't see - # them again - found_garbage.extend(gc.garbage) - del gc.garbage[:] - # Unload the newly imported modules (best effort finalization) - for module in sys.modules.keys(): - if module not in save_modules and module.startswith("test."): - support.unload(module) - - if interrupted: - # print a newline after ^C - print() - print("Test suite interrupted by signal SIGINT.") - omitted = set(selected) - set(good) - set(bad) - set(skipped) - print(count(len(omitted), "test"), "omitted:") - printlist(omitted) - if good and not ns.quiet: - if not bad and not skipped and not interrupted and len(good) > 1: - print("All", end=' ') - print(count(len(good), "test"), "OK.") - if ns.print_slow: - test_times.sort(reverse=True) - print("10 slowest tests:") - for time, test in test_times[:10]: - print("%s: %.1fs" % (test, time)) - if bad: - print(count(len(bad), "test"), "failed:") - printlist(bad) - if environment_changed: - print("{} altered the execution environment:".format( - count(len(environment_changed), "test"))) - printlist(environment_changed) - if skipped and not ns.quiet: - print(count(len(skipped), "test"), "skipped:") - printlist(skipped) - - if ns.verbose2 and bad: - print("Re-running failed tests in verbose mode") - for test in bad[:]: - print("Re-running test %r in verbose mode" % test) - sys.stdout.flush() - try: - ns.verbose = True - ok = runtest(test, True, ns.quiet, ns.huntrleaks, - timeout=ns.timeout) - except KeyboardInterrupt: - # print a newline separate from the ^C - print() - break - else: - if ok[0] in {PASSED, ENV_CHANGED, SKIPPED, RESOURCE_DENIED}: - bad.remove(test) - else: - if bad: - print(count(len(bad), 'test'), "failed again:") - printlist(bad) - - if ns.single: - if next_single_test: - with open(filename, 'w') as fp: - fp.write(next_single_test + '\n') - else: - os.unlink(filename) - - if ns.trace: - r = tracer.results() - r.write_results(show_missing=True, summary=True, coverdir=ns.coverdir) - - if ns.runleaks: - os.system("leaks %d" % os.getpid()) - - sys.exit(len(bad) > 0 or interrupted) - - -# small set of tests to determine if we have a basically functioning interpreter -# (i.e. if any of these fail, then anything else is likely to follow) -STDTESTS = [ - 'test_grammar', - 'test_opcodes', - 'test_dict', - 'test_builtin', - 'test_exceptions', - 'test_types', - 'test_unittest', - 'test_doctest', - 'test_doctest2', - 'test_support' -] - -# set of tests that we don't want to be executed when using regrtest -NOTTESTS = set() - -def findtests(testdir=None, stdtests=STDTESTS, nottests=NOTTESTS): - """Return a list of all applicable test modules.""" - testdir = findtestdir(testdir) - names = os.listdir(testdir) - tests = [] - others = set(stdtests) | nottests - for name in names: - mod, ext = os.path.splitext(name) - if mod[:5] == "test_" and ext in (".py", "") and mod not in others: - tests.append(mod) - return stdtests + sorted(tests) - -# We do not use a generator so multiple threads can call next(). -class MultiprocessTests(object): - - """A thread-safe iterator over tests for multiprocess mode.""" - - def __init__(self, tests): - self.interrupted = False - self.lock = threading.Lock() - self.tests = tests - - def __iter__(self): - return self - - def __next__(self): - with self.lock: - if self.interrupted: - raise StopIteration('tests interrupted') - return next(self.tests) - -def replace_stdout(): - """Set stdout encoder error handler to backslashreplace (as stderr error - handler) to avoid UnicodeEncodeError when printing a traceback""" - import atexit - - stdout = sys.stdout - sys.stdout = open(stdout.fileno(), 'w', - encoding=stdout.encoding, - errors="backslashreplace", - closefd=False, - newline='\n') - - def restore_stdout(): - sys.stdout.close() - sys.stdout = stdout - atexit.register(restore_stdout) - -def runtest(test, verbose, quiet, - huntrleaks=False, use_resources=None, - output_on_failure=False, failfast=False, match_tests=None, - timeout=None): - """Run a single test. - - test -- the name of the test - verbose -- if true, print more messages - quiet -- if true, don't print 'skipped' messages (probably redundant) - huntrleaks -- run multiple times to test for leaks; requires a debug - build; a triple corresponding to -R's three arguments - use_resources -- list of extra resources to use - output_on_failure -- if true, display test output on failure - timeout -- dump the traceback and exit if a test takes more than - timeout seconds - failfast, match_tests -- See regrtest command-line flags for these. - - Returns the tuple result, test_time, where result is one of the constants: - INTERRUPTED KeyboardInterrupt when run under -j - RESOURCE_DENIED test skipped because resource denied - SKIPPED test skipped for some other reason - ENV_CHANGED test failed because it changed the execution environment - FAILED test failed - PASSED test passed - """ - - if use_resources is not None: - support.use_resources = use_resources - use_timeout = (timeout is not None) - if use_timeout: - faulthandler.dump_traceback_later(timeout, exit=True) - try: - support.match_tests = match_tests - if failfast: - support.failfast = True - if output_on_failure: - support.verbose = True - - # Reuse the same instance to all calls to runtest(). Some - # tests keep a reference to sys.stdout or sys.stderr - # (eg. test_argparse). - if runtest.stringio is None: - stream = io.StringIO() - runtest.stringio = stream - else: - stream = runtest.stringio - stream.seek(0) - stream.truncate() - - orig_stdout = sys.stdout - orig_stderr = sys.stderr - try: - sys.stdout = stream - sys.stderr = stream - result = runtest_inner(test, verbose, quiet, huntrleaks, - display_failure=False) - if result[0] == FAILED: - output = stream.getvalue() - orig_stderr.write(output) - orig_stderr.flush() - finally: - sys.stdout = orig_stdout - sys.stderr = orig_stderr - else: - support.verbose = verbose # Tell tests to be moderately quiet - result = runtest_inner(test, verbose, quiet, huntrleaks, - display_failure=not verbose) - return result - finally: - if use_timeout: - faulthandler.cancel_dump_traceback_later() - cleanup_test_droppings(test, verbose) -runtest.stringio = None - -# Unit tests are supposed to leave the execution environment unchanged -# once they complete. But sometimes tests have bugs, especially when -# tests fail, and the changes to environment go on to mess up other -# tests. This can cause issues with buildbot stability, since tests -# are run in random order and so problems may appear to come and go. -# There are a few things we can save and restore to mitigate this, and -# the following context manager handles this task. - -class saved_test_environment: - """Save bits of the test environment and restore them at block exit. - - with saved_test_environment(testname, verbose, quiet): - #stuff - - Unless quiet is True, a warning is printed to stderr if any of - the saved items was changed by the test. The attribute 'changed' - is initially False, but is set to True if a change is detected. - - If verbose is more than 1, the before and after state of changed - items is also printed. - """ - - changed = False - - def __init__(self, testname, verbose=0, quiet=False): - self.testname = testname - self.verbose = verbose - self.quiet = quiet - - # To add things to save and restore, add a name XXX to the resources list - # and add corresponding get_XXX/restore_XXX functions. get_XXX should - # return the value to be saved and compared against a second call to the - # get function when test execution completes. restore_XXX should accept - # the saved value and restore the resource using it. It will be called if - # and only if a change in the value is detected. - # - # Note: XXX will have any '.' replaced with '_' characters when determining - # the corresponding method names. - - resources = ('sys.argv', 'cwd', 'sys.stdin', 'sys.stdout', 'sys.stderr', - 'os.environ', 'sys.path', 'sys.path_hooks', '__import__', - 'warnings.filters', 'asyncore.socket_map', - 'logging._handlers', 'logging._handlerList', 'sys.gettrace', - 'sys.warnoptions', - # multiprocessing.process._cleanup() may release ref - # to a thread, so check processes first. - 'multiprocessing.process._dangling', 'threading._dangling', - 'sysconfig._CONFIG_VARS', 'sysconfig._INSTALL_SCHEMES', - 'files', 'locale', 'warnings.showwarning', - ) - - def get_sys_argv(self): - return id(sys.argv), sys.argv, sys.argv[:] - def restore_sys_argv(self, saved_argv): - sys.argv = saved_argv[1] - sys.argv[:] = saved_argv[2] - - def get_cwd(self): - return os.getcwd() - def restore_cwd(self, saved_cwd): - os.chdir(saved_cwd) - - def get_sys_stdout(self): - return sys.stdout - def restore_sys_stdout(self, saved_stdout): - sys.stdout = saved_stdout - - def get_sys_stderr(self): - return sys.stderr - def restore_sys_stderr(self, saved_stderr): - sys.stderr = saved_stderr - - def get_sys_stdin(self): - return sys.stdin - def restore_sys_stdin(self, saved_stdin): - sys.stdin = saved_stdin - - def get_os_environ(self): - return id(os.environ), os.environ, dict(os.environ) - def restore_os_environ(self, saved_environ): - os.environ = saved_environ[1] - os.environ.clear() - os.environ.update(saved_environ[2]) - - def get_sys_path(self): - return id(sys.path), sys.path, sys.path[:] - def restore_sys_path(self, saved_path): - sys.path = saved_path[1] - sys.path[:] = saved_path[2] - - def get_sys_path_hooks(self): - return id(sys.path_hooks), sys.path_hooks, sys.path_hooks[:] - def restore_sys_path_hooks(self, saved_hooks): - sys.path_hooks = saved_hooks[1] - sys.path_hooks[:] = saved_hooks[2] - - def get_sys_gettrace(self): - return sys.gettrace() - def restore_sys_gettrace(self, trace_fxn): - sys.settrace(trace_fxn) - - def get___import__(self): - return builtins.__import__ - def restore___import__(self, import_): - builtins.__import__ = import_ - - def get_warnings_filters(self): - return id(warnings.filters), warnings.filters, warnings.filters[:] - def restore_warnings_filters(self, saved_filters): - warnings.filters = saved_filters[1] - warnings.filters[:] = saved_filters[2] - - def get_asyncore_socket_map(self): - asyncore = sys.modules.get('asyncore') - # XXX Making a copy keeps objects alive until __exit__ gets called. - return asyncore and asyncore.socket_map.copy() or {} - def restore_asyncore_socket_map(self, saved_map): - asyncore = sys.modules.get('asyncore') - if asyncore is not None: - asyncore.close_all(ignore_all=True) - asyncore.socket_map.update(saved_map) - - def get_shutil_archive_formats(self): - # we could call get_archives_formats() but that only returns the - # registry keys; we want to check the values too (the functions that - # are registered) - return shutil._ARCHIVE_FORMATS, shutil._ARCHIVE_FORMATS.copy() - def restore_shutil_archive_formats(self, saved): - shutil._ARCHIVE_FORMATS = saved[0] - shutil._ARCHIVE_FORMATS.clear() - shutil._ARCHIVE_FORMATS.update(saved[1]) - - def get_shutil_unpack_formats(self): - return shutil._UNPACK_FORMATS, shutil._UNPACK_FORMATS.copy() - def restore_shutil_unpack_formats(self, saved): - shutil._UNPACK_FORMATS = saved[0] - shutil._UNPACK_FORMATS.clear() - shutil._UNPACK_FORMATS.update(saved[1]) - - def get_logging__handlers(self): - # _handlers is a WeakValueDictionary - return id(logging._handlers), logging._handlers, logging._handlers.copy() - def restore_logging__handlers(self, saved_handlers): - # Can't easily revert the logging state - pass - - def get_logging__handlerList(self): - # _handlerList is a list of weakrefs to handlers - return id(logging._handlerList), logging._handlerList, logging._handlerList[:] - def restore_logging__handlerList(self, saved_handlerList): - # Can't easily revert the logging state - pass - - def get_sys_warnoptions(self): - return id(sys.warnoptions), sys.warnoptions, sys.warnoptions[:] - def restore_sys_warnoptions(self, saved_options): - sys.warnoptions = saved_options[1] - sys.warnoptions[:] = saved_options[2] - - # Controlling dangling references to Thread objects can make it easier - # to track reference leaks. - def get_threading__dangling(self): - if not threading: - return None - # This copies the weakrefs without making any strong reference - return threading._dangling.copy() - def restore_threading__dangling(self, saved): - if not threading: - return - threading._dangling.clear() - threading._dangling.update(saved) - - # Same for Process objects - def get_multiprocessing_process__dangling(self): - if not multiprocessing: - return None - # Unjoined process objects can survive after process exits - multiprocessing.process._cleanup() - # This copies the weakrefs without making any strong reference - return multiprocessing.process._dangling.copy() - def restore_multiprocessing_process__dangling(self, saved): - if not multiprocessing: - return - multiprocessing.process._dangling.clear() - multiprocessing.process._dangling.update(saved) - - def get_sysconfig__CONFIG_VARS(self): - # make sure the dict is initialized - sysconfig.get_config_var('prefix') - return (id(sysconfig._CONFIG_VARS), sysconfig._CONFIG_VARS, - dict(sysconfig._CONFIG_VARS)) - def restore_sysconfig__CONFIG_VARS(self, saved): - sysconfig._CONFIG_VARS = saved[1] - sysconfig._CONFIG_VARS.clear() - sysconfig._CONFIG_VARS.update(saved[2]) - - def get_sysconfig__INSTALL_SCHEMES(self): - return (id(sysconfig._INSTALL_SCHEMES), sysconfig._INSTALL_SCHEMES, - sysconfig._INSTALL_SCHEMES.copy()) - def restore_sysconfig__INSTALL_SCHEMES(self, saved): - sysconfig._INSTALL_SCHEMES = saved[1] - sysconfig._INSTALL_SCHEMES.clear() - sysconfig._INSTALL_SCHEMES.update(saved[2]) - - def get_files(self): - return sorted(fn + ('/' if os.path.isdir(fn) else '') - for fn in os.listdir()) - def restore_files(self, saved_value): - fn = support.TESTFN - if fn not in saved_value and (fn + '/') not in saved_value: - if os.path.isfile(fn): - support.unlink(fn) - elif os.path.isdir(fn): - support.rmtree(fn) - - _lc = [getattr(locale, lc) for lc in dir(locale) - if lc.startswith('LC_')] - def get_locale(self): - pairings = [] - for lc in self._lc: - try: - pairings.append((lc, locale.setlocale(lc, None))) - except (TypeError, ValueError): - continue - return pairings - def restore_locale(self, saved): - for lc, setting in saved: - locale.setlocale(lc, setting) - - def get_warnings_showwarning(self): - return warnings.showwarning - def restore_warnings_showwarning(self, fxn): - warnings.showwarning = fxn - - def resource_info(self): - for name in self.resources: - method_suffix = name.replace('.', '_') - get_name = 'get_' + method_suffix - restore_name = 'restore_' + method_suffix - yield name, getattr(self, get_name), getattr(self, restore_name) - - def __enter__(self): - self.saved_values = dict((name, get()) for name, get, restore - in self.resource_info()) - return self - - def __exit__(self, exc_type, exc_val, exc_tb): - saved_values = self.saved_values - del self.saved_values - for name, get, restore in self.resource_info(): - current = get() - original = saved_values.pop(name) - # Check for changes to the resource's value - if current != original: - self.changed = True - restore(original) - if not self.quiet: - print("Warning -- {} was modified by {}".format( - name, self.testname), - file=sys.stderr) - if self.verbose > 1: - print(" Before: {}\n After: {} ".format( - original, current), - file=sys.stderr) - return False - - -def runtest_inner(test, verbose, quiet, - huntrleaks=False, display_failure=True): - support.unload(test) - - test_time = 0.0 - refleak = False # True if the test leaked references. - try: - if test.startswith('test.'): - abstest = test - else: - # Always import it from the test package - abstest = 'test.' + test - with saved_test_environment(test, verbose, quiet) as environment: - start_time = time.time() - the_module = importlib.import_module(abstest) - # If the test has a test_main, that will run the appropriate - # tests. If not, use normal unittest test loading. - test_runner = getattr(the_module, "test_main", None) - if test_runner is None: - def test_runner(): - loader = unittest.TestLoader() - tests = loader.loadTestsFromModule(the_module) - for error in loader.errors: - print(error, file=sys.stderr) - if loader.errors: - raise Exception("errors while loading tests") - support.run_unittest(tests) - test_runner() - if huntrleaks: - refleak = dash_R(the_module, test, test_runner, huntrleaks) - test_time = time.time() - start_time - except support.ResourceDenied as msg: - if not quiet: - print(test, "skipped --", msg) - sys.stdout.flush() - return RESOURCE_DENIED, test_time - except unittest.SkipTest as msg: - if not quiet: - print(test, "skipped --", msg) - sys.stdout.flush() - return SKIPPED, test_time - except KeyboardInterrupt: - raise - except support.TestFailed as msg: - if display_failure: - print("test", test, "failed --", msg, file=sys.stderr) - else: - print("test", test, "failed", file=sys.stderr) - sys.stderr.flush() - return FAILED, test_time - except: - msg = traceback.format_exc() - print("test", test, "crashed --", msg, file=sys.stderr) - sys.stderr.flush() - return FAILED, test_time - else: - if refleak: - return FAILED, test_time - if environment.changed: - return ENV_CHANGED, test_time - return PASSED, test_time - -def cleanup_test_droppings(testname, verbose): - import shutil - import stat - import gc - - # First kill any dangling references to open files etc. - # This can also issue some ResourceWarnings which would otherwise get - # triggered during the following test run, and possibly produce failures. - gc.collect() - - # Try to clean up junk commonly left behind. While tests shouldn't leave - # any files or directories behind, when a test fails that can be tedious - # for it to arrange. The consequences can be especially nasty on Windows, - # since if a test leaves a file open, it cannot be deleted by name (while - # there's nothing we can do about that here either, we can display the - # name of the offending test, which is a real help). - for name in (support.TESTFN, - "db_home", - ): - if not os.path.exists(name): - continue - - if os.path.isdir(name): - kind, nuker = "directory", shutil.rmtree - elif os.path.isfile(name): - kind, nuker = "file", os.unlink - else: - raise SystemError("os.path says %r exists but is neither " - "directory nor file" % name) - - if verbose: - print("%r left behind %s %r" % (testname, kind, name)) - try: - # if we have chmod, fix possible permissions problems - # that might prevent cleanup - if (hasattr(os, 'chmod')): - os.chmod(name, stat.S_IRWXU | stat.S_IRWXG | stat.S_IRWXO) - nuker(name) - except Exception as msg: - print(("%r left behind %s %r and it couldn't be " - "removed: %s" % (testname, kind, name, msg)), file=sys.stderr) def dash_R(the_module, test, indirect_test, huntrleaks): """Run a test multiple times, looking for reference leaks. @@ -1103,6 +82,7 @@ failed = True return failed + def dash_R_cleanup(fs, ps, pic, zdc, abcs): import gc, copyreg import _strptime, linecache @@ -1173,6 +153,7 @@ gc.collect() return func1(), func2() + def warm_caches(): # char cache s = bytes(range(256)) @@ -1182,81 +163,3 @@ x = [chr(i) for i in range(256)] # int cache x = list(range(-5, 257)) - -def findtestdir(path=None): - return path or os.path.dirname(__file__) or os.curdir - -def removepy(names): - if not names: - return - for idx, name in enumerate(names): - basename, ext = os.path.splitext(name) - if ext == '.py': - names[idx] = basename - -def count(n, word): - if n == 1: - return "%d %s" % (n, word) - else: - return "%d %ss" % (n, word) - -def printlist(x, width=70, indent=4): - """Print the elements of iterable x to stdout. - - Optional arg width (default 70) is the maximum line length. - Optional arg indent (default 4) is the number of blanks with which to - begin each line. - """ - - from textwrap import fill - blanks = ' ' * indent - # Print the sorted list: 'x' may be a '--random' list or a set() - print(fill(' '.join(str(elt) for elt in sorted(x)), width, - initial_indent=blanks, subsequent_indent=blanks)) - - -def main_in_temp_cwd(): - """Run main() in a temporary working directory.""" - if sysconfig.is_python_build(): - try: - os.mkdir(TEMPDIR) - except FileExistsError: - pass - - # Define a writable temp dir that will be used as cwd while running - # the tests. The name of the dir includes the pid to allow parallel - # testing (see the -j option). - test_cwd = 'test_python_{}'.format(os.getpid()) - test_cwd = os.path.join(TEMPDIR, test_cwd) - - # Run the tests in a context manager that temporarily changes the CWD to a - # temporary and writable directory. If it's not possible to create or - # change the CWD, the original CWD will be used. The original CWD is - # available from support.SAVEDCWD. - with support.temp_cwd(test_cwd, quiet=True): - main() - - -if __name__ == '__main__': - # Remove regrtest.py's own directory from the module search path. Despite - # the elimination of implicit relative imports, this is still needed to - # ensure that submodules of the test package do not inappropriately appear - # as top-level modules even when people (or buildbots!) invoke regrtest.py - # directly instead of using the -m switch - mydir = os.path.abspath(os.path.normpath(os.path.dirname(sys.argv[0]))) - i = len(sys.path) - while i >= 0: - i -= 1 - if os.path.abspath(os.path.normpath(sys.path[i])) == mydir: - del sys.path[i] - - # findtestdir() gets the dirname out of __file__, so we have to make it - # absolute before changing the working directory. - # For example __file__ may be relative when running trace or profile. - # See issue #9323. - __file__ = os.path.abspath(__file__) - - # sanity check - assert __file__ == os.path.abspath(sys.argv[0]) - - main_in_temp_cwd() diff --git a/Lib/test/regrtest.py b/Lib/test/libregrtest/runtest.py old mode 100755 new mode 100644 copy from Lib/test/regrtest.py copy to Lib/test/libregrtest/runtest.py --- a/Lib/test/regrtest.py +++ b/Lib/test/libregrtest/runtest.py @@ -1,80 +1,16 @@ -#! /usr/bin/env python3 - -""" -Script to run Python regression tests. - -Run this script with -h or --help for documentation. -""" - -# We import importlib *ASAP* in order to test #15386 +import faulthandler import importlib - -import argparse -import builtins -import faulthandler import io import json -import locale -import logging import os -import platform -import random -import re -import shutil -import signal import sys -import sysconfig -import tempfile import time import traceback import unittest -import warnings -from inspect import isabstract +from test import support +from test.libregrtest.refleak import dash_R +from test.libregrtest.save_env import saved_test_environment -try: - import threading -except ImportError: - threading = None -try: - import _multiprocessing, multiprocessing.process -except ImportError: - multiprocessing = None - -from test.libregrtest import _parse_args - - -# Some times __path__ and __file__ are not absolute (e.g. while running from -# Lib/) and, if we change the CWD to run the tests in a temporary dir, some -# imports might fail. This affects only the modules imported before os.chdir(). -# These modules are searched first in sys.path[0] (so '' -- the CWD) and if -# they are found in the CWD their __file__ and __path__ will be relative (this -# happens before the chdir). All the modules imported after the chdir, are -# not found in the CWD, and since the other paths in sys.path[1:] are absolute -# (site.py absolutize them), the __file__ and __path__ will be absolute too. -# Therefore it is necessary to absolutize manually the __file__ and __path__ of -# the packages to prevent later imports to fail when the CWD is different. -for module in sys.modules.values(): - if hasattr(module, '__path__'): - module.__path__ = [os.path.abspath(path) for path in module.__path__] - if hasattr(module, '__file__'): - module.__file__ = os.path.abspath(module.__file__) - - -# MacOSX (a.k.a. Darwin) has a default stack size that is too small -# for deeply recursive regular expressions. We see this as crashes in -# the Python test suite when running test_re.py and test_sre.py. The -# fix is to set the stack limit to 2048. -# This approach may also be useful for other Unixy platforms that -# suffer from small default stack limits. -if sys.platform == 'darwin': - try: - import resource - except ImportError: - pass - else: - soft, hard = resource.getrlimit(resource.RLIMIT_STACK) - newsoft = min(hard, max(soft, 1024*2048)) - resource.setrlimit(resource.RLIMIT_STACK, (newsoft, hard)) # Test result constants. PASSED = 1 @@ -85,16 +21,6 @@ INTERRUPTED = -4 CHILD_ERROR = -5 # error in a child process -from test import support - -# When tests are run from the Python build directory, it is best practice -# to keep the test files in a subfolder. This eases the cleanup of leftover -# files using the "make distclean" command. -if sysconfig.is_python_build(): - TEMPDIR = os.path.join(sysconfig.get_config_var('srcdir'), 'build') -else: - TEMPDIR = tempfile.gettempdir() -TEMPDIR = os.path.abspath(TEMPDIR) def run_test_in_subprocess(testname, ns): """Run the given test in a subprocess with --slaveargs. @@ -128,394 +54,6 @@ return retcode, stdout, stderr -def main(tests=None, **kwargs): - """Execute a test suite. - - This also parses command-line options and modifies its behavior - accordingly. - - tests -- a list of strings containing test names (optional) - testdir -- the directory in which to look for tests (optional) - - Users other than the Python test suite will certainly want to - specify testdir; if it's omitted, the directory containing the - Python test suite is searched for. - - If the tests argument is omitted, the tests listed on the - command-line will be used. If that's empty, too, then all *.py - files beginning with test_ will be used. - - The other default arguments (verbose, quiet, exclude, - single, randomize, findleaks, use_resources, trace, coverdir, - print_slow, and random_seed) allow programmers calling main() - directly to set the values that would normally be set by flags - on the command line. - """ - # Display the Python traceback on fatal errors (e.g. segfault) - faulthandler.enable(all_threads=True) - - # Display the Python traceback on SIGALRM or SIGUSR1 signal - signals = [] - if hasattr(signal, 'SIGALRM'): - signals.append(signal.SIGALRM) - if hasattr(signal, 'SIGUSR1'): - signals.append(signal.SIGUSR1) - for signum in signals: - faulthandler.register(signum, chain=True) - - replace_stdout() - - support.record_original_stdout(sys.stdout) - - ns = _parse_args(sys.argv[1:], **kwargs) - - if ns.huntrleaks: - # Avoid false positives due to various caches - # filling slowly with random data: - warm_caches() - if ns.memlimit is not None: - support.set_memlimit(ns.memlimit) - if ns.threshold is not None: - import gc - gc.set_threshold(ns.threshold) - if ns.nowindows: - import msvcrt - msvcrt.SetErrorMode(msvcrt.SEM_FAILCRITICALERRORS| - msvcrt.SEM_NOALIGNMENTFAULTEXCEPT| - msvcrt.SEM_NOGPFAULTERRORBOX| - msvcrt.SEM_NOOPENFILEERRORBOX) - try: - msvcrt.CrtSetReportMode - except AttributeError: - # release build - pass - else: - for m in [msvcrt.CRT_WARN, msvcrt.CRT_ERROR, msvcrt.CRT_ASSERT]: - msvcrt.CrtSetReportMode(m, msvcrt.CRTDBG_MODE_FILE) - msvcrt.CrtSetReportFile(m, msvcrt.CRTDBG_FILE_STDERR) - if ns.wait: - input("Press any key to continue...") - - if ns.slaveargs is not None: - args, kwargs = json.loads(ns.slaveargs) - if kwargs.get('huntrleaks'): - unittest.BaseTestSuite._cleanup = False - try: - result = runtest(*args, **kwargs) - except KeyboardInterrupt: - result = INTERRUPTED, '' - except BaseException as e: - traceback.print_exc() - result = CHILD_ERROR, str(e) - sys.stdout.flush() - print() # Force a newline (just in case) - print(json.dumps(result)) - sys.exit(0) - - good = [] - bad = [] - skipped = [] - resource_denieds = [] - environment_changed = [] - interrupted = False - - if ns.findleaks: - try: - import gc - except ImportError: - print('No GC available, disabling findleaks.') - ns.findleaks = False - else: - # Uncomment the line below to report garbage that is not - # freeable by reference counting alone. By default only - # garbage that is not collectable by the GC is reported. - #gc.set_debug(gc.DEBUG_SAVEALL) - found_garbage = [] - - if ns.huntrleaks: - unittest.BaseTestSuite._cleanup = False - - if ns.single: - filename = os.path.join(TEMPDIR, 'pynexttest') - try: - with open(filename, 'r') as fp: - next_test = fp.read().strip() - tests = [next_test] - except OSError: - pass - - if ns.fromfile: - tests = [] - with open(os.path.join(support.SAVEDCWD, ns.fromfile)) as fp: - count_pat = re.compile(r'\[\s*\d+/\s*\d+\]') - for line in fp: - line = count_pat.sub('', line) - guts = line.split() # assuming no test has whitespace in its name - if guts and not guts[0].startswith('#'): - tests.extend(guts) - - # Strip .py extensions. - removepy(ns.args) - removepy(tests) - - stdtests = STDTESTS[:] - nottests = NOTTESTS.copy() - if ns.exclude: - for arg in ns.args: - if arg in stdtests: - stdtests.remove(arg) - nottests.add(arg) - ns.args = [] - - # For a partial run, we do not need to clutter the output. - if ns.verbose or ns.header or not (ns.quiet or ns.single or tests or ns.args): - # Print basic platform information - print("==", platform.python_implementation(), *sys.version.split()) - print("== ", platform.platform(aliased=True), - "%s-endian" % sys.byteorder) - print("== ", "hash algorithm:", sys.hash_info.algorithm, - "64bit" if sys.maxsize > 2**32 else "32bit") - print("== ", os.getcwd()) - print("Testing with flags:", sys.flags) - - # if testdir is set, then we are not running the python tests suite, so - # don't add default tests to be executed or skipped (pass empty values) - if ns.testdir: - alltests = findtests(ns.testdir, list(), set()) - else: - alltests = findtests(ns.testdir, stdtests, nottests) - - selected = tests or ns.args or alltests - if ns.single: - selected = selected[:1] - try: - next_single_test = alltests[alltests.index(selected[0])+1] - except IndexError: - next_single_test = None - # Remove all the selected tests that precede start if it's set. - if ns.start: - try: - del selected[:selected.index(ns.start)] - except ValueError: - print("Couldn't find starting test (%s), using all tests" % ns.start) - if ns.randomize: - if ns.random_seed is None: - ns.random_seed = random.randrange(10000000) - random.seed(ns.random_seed) - print("Using random seed", ns.random_seed) - random.shuffle(selected) - if ns.trace: - import trace, tempfile - tracer = trace.Trace(ignoredirs=[sys.base_prefix, sys.base_exec_prefix, - tempfile.gettempdir()], - trace=False, count=True) - - test_times = [] - support.verbose = ns.verbose # Tell tests to be moderately quiet - support.use_resources = ns.use_resources - save_modules = sys.modules.keys() - - def accumulate_result(test, result): - ok, test_time = result - test_times.append((test_time, test)) - if ok == PASSED: - good.append(test) - elif ok == FAILED: - bad.append(test) - elif ok == ENV_CHANGED: - environment_changed.append(test) - elif ok == SKIPPED: - skipped.append(test) - elif ok == RESOURCE_DENIED: - skipped.append(test) - resource_denieds.append(test) - - if ns.forever: - def test_forever(tests=list(selected)): - while True: - for test in tests: - yield test - if bad: - return - tests = test_forever() - test_count = '' - test_count_width = 3 - else: - tests = iter(selected) - test_count = '/{}'.format(len(selected)) - test_count_width = len(test_count) - 1 - - if ns.use_mp: - try: - from threading import Thread - except ImportError: - print("Multiprocess option requires thread support") - sys.exit(2) - from queue import Queue - debug_output_pat = re.compile(r"\[\d+ refs, \d+ blocks\]$") - output = Queue() - pending = MultiprocessTests(tests) - def work(): - # A worker thread. - try: - while True: - try: - test = next(pending) - except StopIteration: - output.put((None, None, None, None)) - return - retcode, stdout, stderr = run_test_in_subprocess(test, ns) - # Strip last refcount output line if it exists, since it - # comes from the shutdown of the interpreter in the subcommand. - stderr = debug_output_pat.sub("", stderr) - stdout, _, result = stdout.strip().rpartition("\n") - if retcode != 0: - result = (CHILD_ERROR, "Exit code %s" % retcode) - output.put((test, stdout.rstrip(), stderr.rstrip(), result)) - return - if not result: - output.put((None, None, None, None)) - return - result = json.loads(result) - output.put((test, stdout.rstrip(), stderr.rstrip(), result)) - except BaseException: - output.put((None, None, None, None)) - raise - workers = [Thread(target=work) for i in range(ns.use_mp)] - for worker in workers: - worker.start() - finished = 0 - test_index = 1 - try: - while finished < ns.use_mp: - test, stdout, stderr, result = output.get() - if test is None: - finished += 1 - continue - accumulate_result(test, result) - if not ns.quiet: - fmt = "[{1:{0}}{2}/{3}] {4}" if bad else "[{1:{0}}{2}] {4}" - print(fmt.format( - test_count_width, test_index, test_count, - len(bad), test)) - if stdout: - print(stdout) - if stderr: - print(stderr, file=sys.stderr) - sys.stdout.flush() - sys.stderr.flush() - if result[0] == INTERRUPTED: - raise KeyboardInterrupt - if result[0] == CHILD_ERROR: - raise Exception("Child error on {}: {}".format(test, result[1])) - test_index += 1 - except KeyboardInterrupt: - interrupted = True - pending.interrupted = True - for worker in workers: - worker.join() - else: - for test_index, test in enumerate(tests, 1): - if not ns.quiet: - fmt = "[{1:{0}}{2}/{3}] {4}" if bad else "[{1:{0}}{2}] {4}" - print(fmt.format( - test_count_width, test_index, test_count, len(bad), test)) - sys.stdout.flush() - if ns.trace: - # If we're tracing code coverage, then we don't exit with status - # if on a false return value from main. - tracer.runctx('runtest(test, ns.verbose, ns.quiet, timeout=ns.timeout)', - globals=globals(), locals=vars()) - else: - try: - result = runtest(test, ns.verbose, ns.quiet, - ns.huntrleaks, - output_on_failure=ns.verbose3, - timeout=ns.timeout, failfast=ns.failfast, - match_tests=ns.match_tests) - accumulate_result(test, result) - except KeyboardInterrupt: - interrupted = True - break - if ns.findleaks: - gc.collect() - if gc.garbage: - print("Warning: test created", len(gc.garbage), end=' ') - print("uncollectable object(s).") - # move the uncollectable objects somewhere so we don't see - # them again - found_garbage.extend(gc.garbage) - del gc.garbage[:] - # Unload the newly imported modules (best effort finalization) - for module in sys.modules.keys(): - if module not in save_modules and module.startswith("test."): - support.unload(module) - - if interrupted: - # print a newline after ^C - print() - print("Test suite interrupted by signal SIGINT.") - omitted = set(selected) - set(good) - set(bad) - set(skipped) - print(count(len(omitted), "test"), "omitted:") - printlist(omitted) - if good and not ns.quiet: - if not bad and not skipped and not interrupted and len(good) > 1: - print("All", end=' ') - print(count(len(good), "test"), "OK.") - if ns.print_slow: - test_times.sort(reverse=True) - print("10 slowest tests:") - for time, test in test_times[:10]: - print("%s: %.1fs" % (test, time)) - if bad: - print(count(len(bad), "test"), "failed:") - printlist(bad) - if environment_changed: - print("{} altered the execution environment:".format( - count(len(environment_changed), "test"))) - printlist(environment_changed) - if skipped and not ns.quiet: - print(count(len(skipped), "test"), "skipped:") - printlist(skipped) - - if ns.verbose2 and bad: - print("Re-running failed tests in verbose mode") - for test in bad[:]: - print("Re-running test %r in verbose mode" % test) - sys.stdout.flush() - try: - ns.verbose = True - ok = runtest(test, True, ns.quiet, ns.huntrleaks, - timeout=ns.timeout) - except KeyboardInterrupt: - # print a newline separate from the ^C - print() - break - else: - if ok[0] in {PASSED, ENV_CHANGED, SKIPPED, RESOURCE_DENIED}: - bad.remove(test) - else: - if bad: - print(count(len(bad), 'test'), "failed again:") - printlist(bad) - - if ns.single: - if next_single_test: - with open(filename, 'w') as fp: - fp.write(next_single_test + '\n') - else: - os.unlink(filename) - - if ns.trace: - r = tracer.results() - r.write_results(show_missing=True, summary=True, coverdir=ns.coverdir) - - if ns.runleaks: - os.system("leaks %d" % os.getpid()) - - sys.exit(len(bad) > 0 or interrupted) - - # small set of tests to determine if we have a basically functioning interpreter # (i.e. if any of these fail, then anything else is likely to follow) STDTESTS = [ @@ -534,6 +72,7 @@ # set of tests that we don't want to be executed when using regrtest NOTTESTS = set() + def findtests(testdir=None, stdtests=STDTESTS, nottests=NOTTESTS): """Return a list of all applicable test modules.""" testdir = findtestdir(testdir) @@ -546,41 +85,6 @@ tests.append(mod) return stdtests + sorted(tests) -# We do not use a generator so multiple threads can call next(). -class MultiprocessTests(object): - - """A thread-safe iterator over tests for multiprocess mode.""" - - def __init__(self, tests): - self.interrupted = False - self.lock = threading.Lock() - self.tests = tests - - def __iter__(self): - return self - - def __next__(self): - with self.lock: - if self.interrupted: - raise StopIteration('tests interrupted') - return next(self.tests) - -def replace_stdout(): - """Set stdout encoder error handler to backslashreplace (as stderr error - handler) to avoid UnicodeEncodeError when printing a traceback""" - import atexit - - stdout = sys.stdout - sys.stdout = open(stdout.fileno(), 'w', - encoding=stdout.encoding, - errors="backslashreplace", - closefd=False, - newline='\n') - - def restore_stdout(): - sys.stdout.close() - sys.stdout = stdout - atexit.register(restore_stdout) def runtest(test, verbose, quiet, huntrleaks=False, use_resources=None, @@ -656,272 +160,6 @@ cleanup_test_droppings(test, verbose) runtest.stringio = None -# Unit tests are supposed to leave the execution environment unchanged -# once they complete. But sometimes tests have bugs, especially when -# tests fail, and the changes to environment go on to mess up other -# tests. This can cause issues with buildbot stability, since tests -# are run in random order and so problems may appear to come and go. -# There are a few things we can save and restore to mitigate this, and -# the following context manager handles this task. - -class saved_test_environment: - """Save bits of the test environment and restore them at block exit. - - with saved_test_environment(testname, verbose, quiet): - #stuff - - Unless quiet is True, a warning is printed to stderr if any of - the saved items was changed by the test. The attribute 'changed' - is initially False, but is set to True if a change is detected. - - If verbose is more than 1, the before and after state of changed - items is also printed. - """ - - changed = False - - def __init__(self, testname, verbose=0, quiet=False): - self.testname = testname - self.verbose = verbose - self.quiet = quiet - - # To add things to save and restore, add a name XXX to the resources list - # and add corresponding get_XXX/restore_XXX functions. get_XXX should - # return the value to be saved and compared against a second call to the - # get function when test execution completes. restore_XXX should accept - # the saved value and restore the resource using it. It will be called if - # and only if a change in the value is detected. - # - # Note: XXX will have any '.' replaced with '_' characters when determining - # the corresponding method names. - - resources = ('sys.argv', 'cwd', 'sys.stdin', 'sys.stdout', 'sys.stderr', - 'os.environ', 'sys.path', 'sys.path_hooks', '__import__', - 'warnings.filters', 'asyncore.socket_map', - 'logging._handlers', 'logging._handlerList', 'sys.gettrace', - 'sys.warnoptions', - # multiprocessing.process._cleanup() may release ref - # to a thread, so check processes first. - 'multiprocessing.process._dangling', 'threading._dangling', - 'sysconfig._CONFIG_VARS', 'sysconfig._INSTALL_SCHEMES', - 'files', 'locale', 'warnings.showwarning', - ) - - def get_sys_argv(self): - return id(sys.argv), sys.argv, sys.argv[:] - def restore_sys_argv(self, saved_argv): - sys.argv = saved_argv[1] - sys.argv[:] = saved_argv[2] - - def get_cwd(self): - return os.getcwd() - def restore_cwd(self, saved_cwd): - os.chdir(saved_cwd) - - def get_sys_stdout(self): - return sys.stdout - def restore_sys_stdout(self, saved_stdout): - sys.stdout = saved_stdout - - def get_sys_stderr(self): - return sys.stderr - def restore_sys_stderr(self, saved_stderr): - sys.stderr = saved_stderr - - def get_sys_stdin(self): - return sys.stdin - def restore_sys_stdin(self, saved_stdin): - sys.stdin = saved_stdin - - def get_os_environ(self): - return id(os.environ), os.environ, dict(os.environ) - def restore_os_environ(self, saved_environ): - os.environ = saved_environ[1] - os.environ.clear() - os.environ.update(saved_environ[2]) - - def get_sys_path(self): - return id(sys.path), sys.path, sys.path[:] - def restore_sys_path(self, saved_path): - sys.path = saved_path[1] - sys.path[:] = saved_path[2] - - def get_sys_path_hooks(self): - return id(sys.path_hooks), sys.path_hooks, sys.path_hooks[:] - def restore_sys_path_hooks(self, saved_hooks): - sys.path_hooks = saved_hooks[1] - sys.path_hooks[:] = saved_hooks[2] - - def get_sys_gettrace(self): - return sys.gettrace() - def restore_sys_gettrace(self, trace_fxn): - sys.settrace(trace_fxn) - - def get___import__(self): - return builtins.__import__ - def restore___import__(self, import_): - builtins.__import__ = import_ - - def get_warnings_filters(self): - return id(warnings.filters), warnings.filters, warnings.filters[:] - def restore_warnings_filters(self, saved_filters): - warnings.filters = saved_filters[1] - warnings.filters[:] = saved_filters[2] - - def get_asyncore_socket_map(self): - asyncore = sys.modules.get('asyncore') - # XXX Making a copy keeps objects alive until __exit__ gets called. - return asyncore and asyncore.socket_map.copy() or {} - def restore_asyncore_socket_map(self, saved_map): - asyncore = sys.modules.get('asyncore') - if asyncore is not None: - asyncore.close_all(ignore_all=True) - asyncore.socket_map.update(saved_map) - - def get_shutil_archive_formats(self): - # we could call get_archives_formats() but that only returns the - # registry keys; we want to check the values too (the functions that - # are registered) - return shutil._ARCHIVE_FORMATS, shutil._ARCHIVE_FORMATS.copy() - def restore_shutil_archive_formats(self, saved): - shutil._ARCHIVE_FORMATS = saved[0] - shutil._ARCHIVE_FORMATS.clear() - shutil._ARCHIVE_FORMATS.update(saved[1]) - - def get_shutil_unpack_formats(self): - return shutil._UNPACK_FORMATS, shutil._UNPACK_FORMATS.copy() - def restore_shutil_unpack_formats(self, saved): - shutil._UNPACK_FORMATS = saved[0] - shutil._UNPACK_FORMATS.clear() - shutil._UNPACK_FORMATS.update(saved[1]) - - def get_logging__handlers(self): - # _handlers is a WeakValueDictionary - return id(logging._handlers), logging._handlers, logging._handlers.copy() - def restore_logging__handlers(self, saved_handlers): - # Can't easily revert the logging state - pass - - def get_logging__handlerList(self): - # _handlerList is a list of weakrefs to handlers - return id(logging._handlerList), logging._handlerList, logging._handlerList[:] - def restore_logging__handlerList(self, saved_handlerList): - # Can't easily revert the logging state - pass - - def get_sys_warnoptions(self): - return id(sys.warnoptions), sys.warnoptions, sys.warnoptions[:] - def restore_sys_warnoptions(self, saved_options): - sys.warnoptions = saved_options[1] - sys.warnoptions[:] = saved_options[2] - - # Controlling dangling references to Thread objects can make it easier - # to track reference leaks. - def get_threading__dangling(self): - if not threading: - return None - # This copies the weakrefs without making any strong reference - return threading._dangling.copy() - def restore_threading__dangling(self, saved): - if not threading: - return - threading._dangling.clear() - threading._dangling.update(saved) - - # Same for Process objects - def get_multiprocessing_process__dangling(self): - if not multiprocessing: - return None - # Unjoined process objects can survive after process exits - multiprocessing.process._cleanup() - # This copies the weakrefs without making any strong reference - return multiprocessing.process._dangling.copy() - def restore_multiprocessing_process__dangling(self, saved): - if not multiprocessing: - return - multiprocessing.process._dangling.clear() - multiprocessing.process._dangling.update(saved) - - def get_sysconfig__CONFIG_VARS(self): - # make sure the dict is initialized - sysconfig.get_config_var('prefix') - return (id(sysconfig._CONFIG_VARS), sysconfig._CONFIG_VARS, - dict(sysconfig._CONFIG_VARS)) - def restore_sysconfig__CONFIG_VARS(self, saved): - sysconfig._CONFIG_VARS = saved[1] - sysconfig._CONFIG_VARS.clear() - sysconfig._CONFIG_VARS.update(saved[2]) - - def get_sysconfig__INSTALL_SCHEMES(self): - return (id(sysconfig._INSTALL_SCHEMES), sysconfig._INSTALL_SCHEMES, - sysconfig._INSTALL_SCHEMES.copy()) - def restore_sysconfig__INSTALL_SCHEMES(self, saved): - sysconfig._INSTALL_SCHEMES = saved[1] - sysconfig._INSTALL_SCHEMES.clear() - sysconfig._INSTALL_SCHEMES.update(saved[2]) - - def get_files(self): - return sorted(fn + ('/' if os.path.isdir(fn) else '') - for fn in os.listdir()) - def restore_files(self, saved_value): - fn = support.TESTFN - if fn not in saved_value and (fn + '/') not in saved_value: - if os.path.isfile(fn): - support.unlink(fn) - elif os.path.isdir(fn): - support.rmtree(fn) - - _lc = [getattr(locale, lc) for lc in dir(locale) - if lc.startswith('LC_')] - def get_locale(self): - pairings = [] - for lc in self._lc: - try: - pairings.append((lc, locale.setlocale(lc, None))) - except (TypeError, ValueError): - continue - return pairings - def restore_locale(self, saved): - for lc, setting in saved: - locale.setlocale(lc, setting) - - def get_warnings_showwarning(self): - return warnings.showwarning - def restore_warnings_showwarning(self, fxn): - warnings.showwarning = fxn - - def resource_info(self): - for name in self.resources: - method_suffix = name.replace('.', '_') - get_name = 'get_' + method_suffix - restore_name = 'restore_' + method_suffix - yield name, getattr(self, get_name), getattr(self, restore_name) - - def __enter__(self): - self.saved_values = dict((name, get()) for name, get, restore - in self.resource_info()) - return self - - def __exit__(self, exc_type, exc_val, exc_tb): - saved_values = self.saved_values - del self.saved_values - for name, get, restore in self.resource_info(): - current = get() - original = saved_values.pop(name) - # Check for changes to the resource's value - if current != original: - self.changed = True - restore(original) - if not self.quiet: - print("Warning -- {} was modified by {}".format( - name, self.testname), - file=sys.stderr) - if self.verbose > 1: - print(" Before: {}\n After: {} ".format( - original, current), - file=sys.stderr) - return False - def runtest_inner(test, verbose, quiet, huntrleaks=False, display_failure=True): @@ -985,6 +223,7 @@ return ENV_CHANGED, test_time return PASSED, test_time + def cleanup_test_droppings(testname, verbose): import shutil import stat @@ -1027,236 +266,6 @@ print(("%r left behind %s %r and it couldn't be " "removed: %s" % (testname, kind, name, msg)), file=sys.stderr) -def dash_R(the_module, test, indirect_test, huntrleaks): - """Run a test multiple times, looking for reference leaks. - - Returns: - False if the test didn't leak references; True if we detected refleaks. - """ - # This code is hackish and inelegant, but it seems to do the job. - import copyreg - import collections.abc - - if not hasattr(sys, 'gettotalrefcount'): - raise Exception("Tracking reference leaks requires a debug build " - "of Python") - - # Save current values for dash_R_cleanup() to restore. - fs = warnings.filters[:] - ps = copyreg.dispatch_table.copy() - pic = sys.path_importer_cache.copy() - try: - import zipimport - except ImportError: - zdc = None # Run unmodified on platforms without zipimport support - else: - zdc = zipimport._zip_directory_cache.copy() - abcs = {} - for abc in [getattr(collections.abc, a) for a in collections.abc.__all__]: - if not isabstract(abc): - continue - for obj in abc.__subclasses__() + [abc]: - abcs[obj] = obj._abc_registry.copy() - - nwarmup, ntracked, fname = huntrleaks - fname = os.path.join(support.SAVEDCWD, fname) - repcount = nwarmup + ntracked - rc_deltas = [0] * repcount - alloc_deltas = [0] * repcount - - print("beginning", repcount, "repetitions", file=sys.stderr) - print(("1234567890"*(repcount//10 + 1))[:repcount], file=sys.stderr) - sys.stderr.flush() - for i in range(repcount): - indirect_test() - alloc_after, rc_after = dash_R_cleanup(fs, ps, pic, zdc, abcs) - sys.stderr.write('.') - sys.stderr.flush() - if i >= nwarmup: - rc_deltas[i] = rc_after - rc_before - alloc_deltas[i] = alloc_after - alloc_before - alloc_before, rc_before = alloc_after, rc_after - print(file=sys.stderr) - # These checkers return False on success, True on failure - def check_rc_deltas(deltas): - return any(deltas) - def check_alloc_deltas(deltas): - # At least 1/3rd of 0s - if 3 * deltas.count(0) < len(deltas): - return True - # Nothing else than 1s, 0s and -1s - if not set(deltas) <= {1,0,-1}: - return True - return False - failed = False - for deltas, item_name, checker in [ - (rc_deltas, 'references', check_rc_deltas), - (alloc_deltas, 'memory blocks', check_alloc_deltas)]: - if checker(deltas): - msg = '%s leaked %s %s, sum=%s' % ( - test, deltas[nwarmup:], item_name, sum(deltas)) - print(msg, file=sys.stderr) - sys.stderr.flush() - with open(fname, "a") as refrep: - print(msg, file=refrep) - refrep.flush() - failed = True - return failed - -def dash_R_cleanup(fs, ps, pic, zdc, abcs): - import gc, copyreg - import _strptime, linecache - import urllib.parse, urllib.request, mimetypes, doctest - import struct, filecmp, collections.abc - from distutils.dir_util import _path_created - from weakref import WeakSet - - # Clear the warnings registry, so they can be displayed again - for mod in sys.modules.values(): - if hasattr(mod, '__warningregistry__'): - del mod.__warningregistry__ - - # Restore some original values. - warnings.filters[:] = fs - copyreg.dispatch_table.clear() - copyreg.dispatch_table.update(ps) - sys.path_importer_cache.clear() - sys.path_importer_cache.update(pic) - try: - import zipimport - except ImportError: - pass # Run unmodified on platforms without zipimport support - else: - zipimport._zip_directory_cache.clear() - zipimport._zip_directory_cache.update(zdc) - - # clear type cache - sys._clear_type_cache() - - # Clear ABC registries, restoring previously saved ABC registries. - for abc in [getattr(collections.abc, a) for a in collections.abc.__all__]: - if not isabstract(abc): - continue - for obj in abc.__subclasses__() + [abc]: - obj._abc_registry = abcs.get(obj, WeakSet()).copy() - obj._abc_cache.clear() - obj._abc_negative_cache.clear() - - # Flush standard output, so that buffered data is sent to the OS and - # associated Python objects are reclaimed. - for stream in (sys.stdout, sys.stderr, sys.__stdout__, sys.__stderr__): - if stream is not None: - stream.flush() - - # Clear assorted module caches. - _path_created.clear() - re.purge() - _strptime._regex_cache.clear() - urllib.parse.clear_cache() - urllib.request.urlcleanup() - linecache.clearcache() - mimetypes._default_mime_types() - filecmp._cache.clear() - struct._clearcache() - doctest.master = None - try: - import ctypes - except ImportError: - # Don't worry about resetting the cache if ctypes is not supported - pass - else: - ctypes._reset_cache() - - # Collect cyclic trash and read memory statistics immediately after. - func1 = sys.getallocatedblocks - func2 = sys.gettotalrefcount - gc.collect() - return func1(), func2() - -def warm_caches(): - # char cache - s = bytes(range(256)) - for i in range(256): - s[i:i+1] - # unicode cache - x = [chr(i) for i in range(256)] - # int cache - x = list(range(-5, 257)) def findtestdir(path=None): - return path or os.path.dirname(__file__) or os.curdir - -def removepy(names): - if not names: - return - for idx, name in enumerate(names): - basename, ext = os.path.splitext(name) - if ext == '.py': - names[idx] = basename - -def count(n, word): - if n == 1: - return "%d %s" % (n, word) - else: - return "%d %ss" % (n, word) - -def printlist(x, width=70, indent=4): - """Print the elements of iterable x to stdout. - - Optional arg width (default 70) is the maximum line length. - Optional arg indent (default 4) is the number of blanks with which to - begin each line. - """ - - from textwrap import fill - blanks = ' ' * indent - # Print the sorted list: 'x' may be a '--random' list or a set() - print(fill(' '.join(str(elt) for elt in sorted(x)), width, - initial_indent=blanks, subsequent_indent=blanks)) - - -def main_in_temp_cwd(): - """Run main() in a temporary working directory.""" - if sysconfig.is_python_build(): - try: - os.mkdir(TEMPDIR) - except FileExistsError: - pass - - # Define a writable temp dir that will be used as cwd while running - # the tests. The name of the dir includes the pid to allow parallel - # testing (see the -j option). - test_cwd = 'test_python_{}'.format(os.getpid()) - test_cwd = os.path.join(TEMPDIR, test_cwd) - - # Run the tests in a context manager that temporarily changes the CWD to a - # temporary and writable directory. If it's not possible to create or - # change the CWD, the original CWD will be used. The original CWD is - # available from support.SAVEDCWD. - with support.temp_cwd(test_cwd, quiet=True): - main() - - -if __name__ == '__main__': - # Remove regrtest.py's own directory from the module search path. Despite - # the elimination of implicit relative imports, this is still needed to - # ensure that submodules of the test package do not inappropriately appear - # as top-level modules even when people (or buildbots!) invoke regrtest.py - # directly instead of using the -m switch - mydir = os.path.abspath(os.path.normpath(os.path.dirname(sys.argv[0]))) - i = len(sys.path) - while i >= 0: - i -= 1 - if os.path.abspath(os.path.normpath(sys.path[i])) == mydir: - del sys.path[i] - - # findtestdir() gets the dirname out of __file__, so we have to make it - # absolute before changing the working directory. - # For example __file__ may be relative when running trace or profile. - # See issue #9323. - __file__ = os.path.abspath(__file__) - - # sanity check - assert __file__ == os.path.abspath(sys.argv[0]) - - main_in_temp_cwd() + return path or os.path.dirname(os.path.dirname(__file__)) or os.curdir diff --git a/Lib/test/regrtest.py b/Lib/test/libregrtest/save_env.py old mode 100755 new mode 100644 copy from Lib/test/regrtest.py copy to Lib/test/libregrtest/save_env.py --- a/Lib/test/regrtest.py +++ b/Lib/test/libregrtest/save_env.py @@ -1,36 +1,12 @@ -#! /usr/bin/env python3 - -""" -Script to run Python regression tests. - -Run this script with -h or --help for documentation. -""" - -# We import importlib *ASAP* in order to test #15386 -import importlib - -import argparse import builtins -import faulthandler -import io -import json import locale import logging import os -import platform -import random -import re import shutil -import signal import sys import sysconfig -import tempfile -import time -import traceback -import unittest import warnings -from inspect import isabstract - +from test import support try: import threading except ImportError: @@ -40,621 +16,6 @@ except ImportError: multiprocessing = None -from test.libregrtest import _parse_args - - -# Some times __path__ and __file__ are not absolute (e.g. while running from -# Lib/) and, if we change the CWD to run the tests in a temporary dir, some -# imports might fail. This affects only the modules imported before os.chdir(). -# These modules are searched first in sys.path[0] (so '' -- the CWD) and if -# they are found in the CWD their __file__ and __path__ will be relative (this -# happens before the chdir). All the modules imported after the chdir, are -# not found in the CWD, and since the other paths in sys.path[1:] are absolute -# (site.py absolutize them), the __file__ and __path__ will be absolute too. -# Therefore it is necessary to absolutize manually the __file__ and __path__ of -# the packages to prevent later imports to fail when the CWD is different. -for module in sys.modules.values(): - if hasattr(module, '__path__'): - module.__path__ = [os.path.abspath(path) for path in module.__path__] - if hasattr(module, '__file__'): - module.__file__ = os.path.abspath(module.__file__) - - -# MacOSX (a.k.a. Darwin) has a default stack size that is too small -# for deeply recursive regular expressions. We see this as crashes in -# the Python test suite when running test_re.py and test_sre.py. The -# fix is to set the stack limit to 2048. -# This approach may also be useful for other Unixy platforms that -# suffer from small default stack limits. -if sys.platform == 'darwin': - try: - import resource - except ImportError: - pass - else: - soft, hard = resource.getrlimit(resource.RLIMIT_STACK) - newsoft = min(hard, max(soft, 1024*2048)) - resource.setrlimit(resource.RLIMIT_STACK, (newsoft, hard)) - -# Test result constants. -PASSED = 1 -FAILED = 0 -ENV_CHANGED = -1 -SKIPPED = -2 -RESOURCE_DENIED = -3 -INTERRUPTED = -4 -CHILD_ERROR = -5 # error in a child process - -from test import support - -# When tests are run from the Python build directory, it is best practice -# to keep the test files in a subfolder. This eases the cleanup of leftover -# files using the "make distclean" command. -if sysconfig.is_python_build(): - TEMPDIR = os.path.join(sysconfig.get_config_var('srcdir'), 'build') -else: - TEMPDIR = tempfile.gettempdir() -TEMPDIR = os.path.abspath(TEMPDIR) - -def run_test_in_subprocess(testname, ns): - """Run the given test in a subprocess with --slaveargs. - - ns is the option Namespace parsed from command-line arguments. regrtest - is invoked in a subprocess with the --slaveargs argument; when the - subprocess exits, its return code, stdout and stderr are returned as a - 3-tuple. - """ - from subprocess import Popen, PIPE - base_cmd = ([sys.executable] + support.args_from_interpreter_flags() + - ['-X', 'faulthandler', '-m', 'test.regrtest']) - - slaveargs = ( - (testname, ns.verbose, ns.quiet), - dict(huntrleaks=ns.huntrleaks, - use_resources=ns.use_resources, - output_on_failure=ns.verbose3, - timeout=ns.timeout, failfast=ns.failfast, - match_tests=ns.match_tests)) - # Running the child from the same working directory as regrtest's original - # invocation ensures that TEMPDIR for the child is the same when - # sysconfig.is_python_build() is true. See issue 15300. - popen = Popen(base_cmd + ['--slaveargs', json.dumps(slaveargs)], - stdout=PIPE, stderr=PIPE, - universal_newlines=True, - close_fds=(os.name != 'nt'), - cwd=support.SAVEDCWD) - stdout, stderr = popen.communicate() - retcode = popen.wait() - return retcode, stdout, stderr - - -def main(tests=None, **kwargs): - """Execute a test suite. - - This also parses command-line options and modifies its behavior - accordingly. - - tests -- a list of strings containing test names (optional) - testdir -- the directory in which to look for tests (optional) - - Users other than the Python test suite will certainly want to - specify testdir; if it's omitted, the directory containing the - Python test suite is searched for. - - If the tests argument is omitted, the tests listed on the - command-line will be used. If that's empty, too, then all *.py - files beginning with test_ will be used. - - The other default arguments (verbose, quiet, exclude, - single, randomize, findleaks, use_resources, trace, coverdir, - print_slow, and random_seed) allow programmers calling main() - directly to set the values that would normally be set by flags - on the command line. - """ - # Display the Python traceback on fatal errors (e.g. segfault) - faulthandler.enable(all_threads=True) - - # Display the Python traceback on SIGALRM or SIGUSR1 signal - signals = [] - if hasattr(signal, 'SIGALRM'): - signals.append(signal.SIGALRM) - if hasattr(signal, 'SIGUSR1'): - signals.append(signal.SIGUSR1) - for signum in signals: - faulthandler.register(signum, chain=True) - - replace_stdout() - - support.record_original_stdout(sys.stdout) - - ns = _parse_args(sys.argv[1:], **kwargs) - - if ns.huntrleaks: - # Avoid false positives due to various caches - # filling slowly with random data: - warm_caches() - if ns.memlimit is not None: - support.set_memlimit(ns.memlimit) - if ns.threshold is not None: - import gc - gc.set_threshold(ns.threshold) - if ns.nowindows: - import msvcrt - msvcrt.SetErrorMode(msvcrt.SEM_FAILCRITICALERRORS| - msvcrt.SEM_NOALIGNMENTFAULTEXCEPT| - msvcrt.SEM_NOGPFAULTERRORBOX| - msvcrt.SEM_NOOPENFILEERRORBOX) - try: - msvcrt.CrtSetReportMode - except AttributeError: - # release build - pass - else: - for m in [msvcrt.CRT_WARN, msvcrt.CRT_ERROR, msvcrt.CRT_ASSERT]: - msvcrt.CrtSetReportMode(m, msvcrt.CRTDBG_MODE_FILE) - msvcrt.CrtSetReportFile(m, msvcrt.CRTDBG_FILE_STDERR) - if ns.wait: - input("Press any key to continue...") - - if ns.slaveargs is not None: - args, kwargs = json.loads(ns.slaveargs) - if kwargs.get('huntrleaks'): - unittest.BaseTestSuite._cleanup = False - try: - result = runtest(*args, **kwargs) - except KeyboardInterrupt: - result = INTERRUPTED, '' - except BaseException as e: - traceback.print_exc() - result = CHILD_ERROR, str(e) - sys.stdout.flush() - print() # Force a newline (just in case) - print(json.dumps(result)) - sys.exit(0) - - good = [] - bad = [] - skipped = [] - resource_denieds = [] - environment_changed = [] - interrupted = False - - if ns.findleaks: - try: - import gc - except ImportError: - print('No GC available, disabling findleaks.') - ns.findleaks = False - else: - # Uncomment the line below to report garbage that is not - # freeable by reference counting alone. By default only - # garbage that is not collectable by the GC is reported. - #gc.set_debug(gc.DEBUG_SAVEALL) - found_garbage = [] - - if ns.huntrleaks: - unittest.BaseTestSuite._cleanup = False - - if ns.single: - filename = os.path.join(TEMPDIR, 'pynexttest') - try: - with open(filename, 'r') as fp: - next_test = fp.read().strip() - tests = [next_test] - except OSError: - pass - - if ns.fromfile: - tests = [] - with open(os.path.join(support.SAVEDCWD, ns.fromfile)) as fp: - count_pat = re.compile(r'\[\s*\d+/\s*\d+\]') - for line in fp: - line = count_pat.sub('', line) - guts = line.split() # assuming no test has whitespace in its name - if guts and not guts[0].startswith('#'): - tests.extend(guts) - - # Strip .py extensions. - removepy(ns.args) - removepy(tests) - - stdtests = STDTESTS[:] - nottests = NOTTESTS.copy() - if ns.exclude: - for arg in ns.args: - if arg in stdtests: - stdtests.remove(arg) - nottests.add(arg) - ns.args = [] - - # For a partial run, we do not need to clutter the output. - if ns.verbose or ns.header or not (ns.quiet or ns.single or tests or ns.args): - # Print basic platform information - print("==", platform.python_implementation(), *sys.version.split()) - print("== ", platform.platform(aliased=True), - "%s-endian" % sys.byteorder) - print("== ", "hash algorithm:", sys.hash_info.algorithm, - "64bit" if sys.maxsize > 2**32 else "32bit") - print("== ", os.getcwd()) - print("Testing with flags:", sys.flags) - - # if testdir is set, then we are not running the python tests suite, so - # don't add default tests to be executed or skipped (pass empty values) - if ns.testdir: - alltests = findtests(ns.testdir, list(), set()) - else: - alltests = findtests(ns.testdir, stdtests, nottests) - - selected = tests or ns.args or alltests - if ns.single: - selected = selected[:1] - try: - next_single_test = alltests[alltests.index(selected[0])+1] - except IndexError: - next_single_test = None - # Remove all the selected tests that precede start if it's set. - if ns.start: - try: - del selected[:selected.index(ns.start)] - except ValueError: - print("Couldn't find starting test (%s), using all tests" % ns.start) - if ns.randomize: - if ns.random_seed is None: - ns.random_seed = random.randrange(10000000) - random.seed(ns.random_seed) - print("Using random seed", ns.random_seed) - random.shuffle(selected) - if ns.trace: - import trace, tempfile - tracer = trace.Trace(ignoredirs=[sys.base_prefix, sys.base_exec_prefix, - tempfile.gettempdir()], - trace=False, count=True) - - test_times = [] - support.verbose = ns.verbose # Tell tests to be moderately quiet - support.use_resources = ns.use_resources - save_modules = sys.modules.keys() - - def accumulate_result(test, result): - ok, test_time = result - test_times.append((test_time, test)) - if ok == PASSED: - good.append(test) - elif ok == FAILED: - bad.append(test) - elif ok == ENV_CHANGED: - environment_changed.append(test) - elif ok == SKIPPED: - skipped.append(test) - elif ok == RESOURCE_DENIED: - skipped.append(test) - resource_denieds.append(test) - - if ns.forever: - def test_forever(tests=list(selected)): - while True: - for test in tests: - yield test - if bad: - return - tests = test_forever() - test_count = '' - test_count_width = 3 - else: - tests = iter(selected) - test_count = '/{}'.format(len(selected)) - test_count_width = len(test_count) - 1 - - if ns.use_mp: - try: - from threading import Thread - except ImportError: - print("Multiprocess option requires thread support") - sys.exit(2) - from queue import Queue - debug_output_pat = re.compile(r"\[\d+ refs, \d+ blocks\]$") - output = Queue() - pending = MultiprocessTests(tests) - def work(): - # A worker thread. - try: - while True: - try: - test = next(pending) - except StopIteration: - output.put((None, None, None, None)) - return - retcode, stdout, stderr = run_test_in_subprocess(test, ns) - # Strip last refcount output line if it exists, since it - # comes from the shutdown of the interpreter in the subcommand. - stderr = debug_output_pat.sub("", stderr) - stdout, _, result = stdout.strip().rpartition("\n") - if retcode != 0: - result = (CHILD_ERROR, "Exit code %s" % retcode) - output.put((test, stdout.rstrip(), stderr.rstrip(), result)) - return - if not result: - output.put((None, None, None, None)) - return - result = json.loads(result) - output.put((test, stdout.rstrip(), stderr.rstrip(), result)) - except BaseException: - output.put((None, None, None, None)) - raise - workers = [Thread(target=work) for i in range(ns.use_mp)] - for worker in workers: - worker.start() - finished = 0 - test_index = 1 - try: - while finished < ns.use_mp: - test, stdout, stderr, result = output.get() - if test is None: - finished += 1 - continue - accumulate_result(test, result) - if not ns.quiet: - fmt = "[{1:{0}}{2}/{3}] {4}" if bad else "[{1:{0}}{2}] {4}" - print(fmt.format( - test_count_width, test_index, test_count, - len(bad), test)) - if stdout: - print(stdout) - if stderr: - print(stderr, file=sys.stderr) - sys.stdout.flush() - sys.stderr.flush() - if result[0] == INTERRUPTED: - raise KeyboardInterrupt - if result[0] == CHILD_ERROR: - raise Exception("Child error on {}: {}".format(test, result[1])) - test_index += 1 - except KeyboardInterrupt: - interrupted = True - pending.interrupted = True - for worker in workers: - worker.join() - else: - for test_index, test in enumerate(tests, 1): - if not ns.quiet: - fmt = "[{1:{0}}{2}/{3}] {4}" if bad else "[{1:{0}}{2}] {4}" - print(fmt.format( - test_count_width, test_index, test_count, len(bad), test)) - sys.stdout.flush() - if ns.trace: - # If we're tracing code coverage, then we don't exit with status - # if on a false return value from main. - tracer.runctx('runtest(test, ns.verbose, ns.quiet, timeout=ns.timeout)', - globals=globals(), locals=vars()) - else: - try: - result = runtest(test, ns.verbose, ns.quiet, - ns.huntrleaks, - output_on_failure=ns.verbose3, - timeout=ns.timeout, failfast=ns.failfast, - match_tests=ns.match_tests) - accumulate_result(test, result) - except KeyboardInterrupt: - interrupted = True - break - if ns.findleaks: - gc.collect() - if gc.garbage: - print("Warning: test created", len(gc.garbage), end=' ') - print("uncollectable object(s).") - # move the uncollectable objects somewhere so we don't see - # them again - found_garbage.extend(gc.garbage) - del gc.garbage[:] - # Unload the newly imported modules (best effort finalization) - for module in sys.modules.keys(): - if module not in save_modules and module.startswith("test."): - support.unload(module) - - if interrupted: - # print a newline after ^C - print() - print("Test suite interrupted by signal SIGINT.") - omitted = set(selected) - set(good) - set(bad) - set(skipped) - print(count(len(omitted), "test"), "omitted:") - printlist(omitted) - if good and not ns.quiet: - if not bad and not skipped and not interrupted and len(good) > 1: - print("All", end=' ') - print(count(len(good), "test"), "OK.") - if ns.print_slow: - test_times.sort(reverse=True) - print("10 slowest tests:") - for time, test in test_times[:10]: - print("%s: %.1fs" % (test, time)) - if bad: - print(count(len(bad), "test"), "failed:") - printlist(bad) - if environment_changed: - print("{} altered the execution environment:".format( - count(len(environment_changed), "test"))) - printlist(environment_changed) - if skipped and not ns.quiet: - print(count(len(skipped), "test"), "skipped:") - printlist(skipped) - - if ns.verbose2 and bad: - print("Re-running failed tests in verbose mode") - for test in bad[:]: - print("Re-running test %r in verbose mode" % test) - sys.stdout.flush() - try: - ns.verbose = True - ok = runtest(test, True, ns.quiet, ns.huntrleaks, - timeout=ns.timeout) - except KeyboardInterrupt: - # print a newline separate from the ^C - print() - break - else: - if ok[0] in {PASSED, ENV_CHANGED, SKIPPED, RESOURCE_DENIED}: - bad.remove(test) - else: - if bad: - print(count(len(bad), 'test'), "failed again:") - printlist(bad) - - if ns.single: - if next_single_test: - with open(filename, 'w') as fp: - fp.write(next_single_test + '\n') - else: - os.unlink(filename) - - if ns.trace: - r = tracer.results() - r.write_results(show_missing=True, summary=True, coverdir=ns.coverdir) - - if ns.runleaks: - os.system("leaks %d" % os.getpid()) - - sys.exit(len(bad) > 0 or interrupted) - - -# small set of tests to determine if we have a basically functioning interpreter -# (i.e. if any of these fail, then anything else is likely to follow) -STDTESTS = [ - 'test_grammar', - 'test_opcodes', - 'test_dict', - 'test_builtin', - 'test_exceptions', - 'test_types', - 'test_unittest', - 'test_doctest', - 'test_doctest2', - 'test_support' -] - -# set of tests that we don't want to be executed when using regrtest -NOTTESTS = set() - -def findtests(testdir=None, stdtests=STDTESTS, nottests=NOTTESTS): - """Return a list of all applicable test modules.""" - testdir = findtestdir(testdir) - names = os.listdir(testdir) - tests = [] - others = set(stdtests) | nottests - for name in names: - mod, ext = os.path.splitext(name) - if mod[:5] == "test_" and ext in (".py", "") and mod not in others: - tests.append(mod) - return stdtests + sorted(tests) - -# We do not use a generator so multiple threads can call next(). -class MultiprocessTests(object): - - """A thread-safe iterator over tests for multiprocess mode.""" - - def __init__(self, tests): - self.interrupted = False - self.lock = threading.Lock() - self.tests = tests - - def __iter__(self): - return self - - def __next__(self): - with self.lock: - if self.interrupted: - raise StopIteration('tests interrupted') - return next(self.tests) - -def replace_stdout(): - """Set stdout encoder error handler to backslashreplace (as stderr error - handler) to avoid UnicodeEncodeError when printing a traceback""" - import atexit - - stdout = sys.stdout - sys.stdout = open(stdout.fileno(), 'w', - encoding=stdout.encoding, - errors="backslashreplace", - closefd=False, - newline='\n') - - def restore_stdout(): - sys.stdout.close() - sys.stdout = stdout - atexit.register(restore_stdout) - -def runtest(test, verbose, quiet, - huntrleaks=False, use_resources=None, - output_on_failure=False, failfast=False, match_tests=None, - timeout=None): - """Run a single test. - - test -- the name of the test - verbose -- if true, print more messages - quiet -- if true, don't print 'skipped' messages (probably redundant) - huntrleaks -- run multiple times to test for leaks; requires a debug - build; a triple corresponding to -R's three arguments - use_resources -- list of extra resources to use - output_on_failure -- if true, display test output on failure - timeout -- dump the traceback and exit if a test takes more than - timeout seconds - failfast, match_tests -- See regrtest command-line flags for these. - - Returns the tuple result, test_time, where result is one of the constants: - INTERRUPTED KeyboardInterrupt when run under -j - RESOURCE_DENIED test skipped because resource denied - SKIPPED test skipped for some other reason - ENV_CHANGED test failed because it changed the execution environment - FAILED test failed - PASSED test passed - """ - - if use_resources is not None: - support.use_resources = use_resources - use_timeout = (timeout is not None) - if use_timeout: - faulthandler.dump_traceback_later(timeout, exit=True) - try: - support.match_tests = match_tests - if failfast: - support.failfast = True - if output_on_failure: - support.verbose = True - - # Reuse the same instance to all calls to runtest(). Some - # tests keep a reference to sys.stdout or sys.stderr - # (eg. test_argparse). - if runtest.stringio is None: - stream = io.StringIO() - runtest.stringio = stream - else: - stream = runtest.stringio - stream.seek(0) - stream.truncate() - - orig_stdout = sys.stdout - orig_stderr = sys.stderr - try: - sys.stdout = stream - sys.stderr = stream - result = runtest_inner(test, verbose, quiet, huntrleaks, - display_failure=False) - if result[0] == FAILED: - output = stream.getvalue() - orig_stderr.write(output) - orig_stderr.flush() - finally: - sys.stdout = orig_stdout - sys.stderr = orig_stderr - else: - support.verbose = verbose # Tell tests to be moderately quiet - result = runtest_inner(test, verbose, quiet, huntrleaks, - display_failure=not verbose) - return result - finally: - if use_timeout: - faulthandler.cancel_dump_traceback_later() - cleanup_test_droppings(test, verbose) -runtest.stringio = None # Unit tests are supposed to leave the execution environment unchanged # once they complete. But sometimes tests have bugs, especially when @@ -921,342 +282,3 @@ original, current), file=sys.stderr) return False - - -def runtest_inner(test, verbose, quiet, - huntrleaks=False, display_failure=True): - support.unload(test) - - test_time = 0.0 - refleak = False # True if the test leaked references. - try: - if test.startswith('test.'): - abstest = test - else: - # Always import it from the test package - abstest = 'test.' + test - with saved_test_environment(test, verbose, quiet) as environment: - start_time = time.time() - the_module = importlib.import_module(abstest) - # If the test has a test_main, that will run the appropriate - # tests. If not, use normal unittest test loading. - test_runner = getattr(the_module, "test_main", None) - if test_runner is None: - def test_runner(): - loader = unittest.TestLoader() - tests = loader.loadTestsFromModule(the_module) - for error in loader.errors: - print(error, file=sys.stderr) - if loader.errors: - raise Exception("errors while loading tests") - support.run_unittest(tests) - test_runner() - if huntrleaks: - refleak = dash_R(the_module, test, test_runner, huntrleaks) - test_time = time.time() - start_time - except support.ResourceDenied as msg: - if not quiet: - print(test, "skipped --", msg) - sys.stdout.flush() - return RESOURCE_DENIED, test_time - except unittest.SkipTest as msg: - if not quiet: - print(test, "skipped --", msg) - sys.stdout.flush() - return SKIPPED, test_time - except KeyboardInterrupt: - raise - except support.TestFailed as msg: - if display_failure: - print("test", test, "failed --", msg, file=sys.stderr) - else: - print("test", test, "failed", file=sys.stderr) - sys.stderr.flush() - return FAILED, test_time - except: - msg = traceback.format_exc() - print("test", test, "crashed --", msg, file=sys.stderr) - sys.stderr.flush() - return FAILED, test_time - else: - if refleak: - return FAILED, test_time - if environment.changed: - return ENV_CHANGED, test_time - return PASSED, test_time - -def cleanup_test_droppings(testname, verbose): - import shutil - import stat - import gc - - # First kill any dangling references to open files etc. - # This can also issue some ResourceWarnings which would otherwise get - # triggered during the following test run, and possibly produce failures. - gc.collect() - - # Try to clean up junk commonly left behind. While tests shouldn't leave - # any files or directories behind, when a test fails that can be tedious - # for it to arrange. The consequences can be especially nasty on Windows, - # since if a test leaves a file open, it cannot be deleted by name (while - # there's nothing we can do about that here either, we can display the - # name of the offending test, which is a real help). - for name in (support.TESTFN, - "db_home", - ): - if not os.path.exists(name): - continue - - if os.path.isdir(name): - kind, nuker = "directory", shutil.rmtree - elif os.path.isfile(name): - kind, nuker = "file", os.unlink - else: - raise SystemError("os.path says %r exists but is neither " - "directory nor file" % name) - - if verbose: - print("%r left behind %s %r" % (testname, kind, name)) - try: - # if we have chmod, fix possible permissions problems - # that might prevent cleanup - if (hasattr(os, 'chmod')): - os.chmod(name, stat.S_IRWXU | stat.S_IRWXG | stat.S_IRWXO) - nuker(name) - except Exception as msg: - print(("%r left behind %s %r and it couldn't be " - "removed: %s" % (testname, kind, name, msg)), file=sys.stderr) - -def dash_R(the_module, test, indirect_test, huntrleaks): - """Run a test multiple times, looking for reference leaks. - - Returns: - False if the test didn't leak references; True if we detected refleaks. - """ - # This code is hackish and inelegant, but it seems to do the job. - import copyreg - import collections.abc - - if not hasattr(sys, 'gettotalrefcount'): - raise Exception("Tracking reference leaks requires a debug build " - "of Python") - - # Save current values for dash_R_cleanup() to restore. - fs = warnings.filters[:] - ps = copyreg.dispatch_table.copy() - pic = sys.path_importer_cache.copy() - try: - import zipimport - except ImportError: - zdc = None # Run unmodified on platforms without zipimport support - else: - zdc = zipimport._zip_directory_cache.copy() - abcs = {} - for abc in [getattr(collections.abc, a) for a in collections.abc.__all__]: - if not isabstract(abc): - continue - for obj in abc.__subclasses__() + [abc]: - abcs[obj] = obj._abc_registry.copy() - - nwarmup, ntracked, fname = huntrleaks - fname = os.path.join(support.SAVEDCWD, fname) - repcount = nwarmup + ntracked - rc_deltas = [0] * repcount - alloc_deltas = [0] * repcount - - print("beginning", repcount, "repetitions", file=sys.stderr) - print(("1234567890"*(repcount//10 + 1))[:repcount], file=sys.stderr) - sys.stderr.flush() - for i in range(repcount): - indirect_test() - alloc_after, rc_after = dash_R_cleanup(fs, ps, pic, zdc, abcs) - sys.stderr.write('.') - sys.stderr.flush() - if i >= nwarmup: - rc_deltas[i] = rc_after - rc_before - alloc_deltas[i] = alloc_after - alloc_before - alloc_before, rc_before = alloc_after, rc_after - print(file=sys.stderr) - # These checkers return False on success, True on failure - def check_rc_deltas(deltas): - return any(deltas) - def check_alloc_deltas(deltas): - # At least 1/3rd of 0s - if 3 * deltas.count(0) < len(deltas): - return True - # Nothing else than 1s, 0s and -1s - if not set(deltas) <= {1,0,-1}: - return True - return False - failed = False - for deltas, item_name, checker in [ - (rc_deltas, 'references', check_rc_deltas), - (alloc_deltas, 'memory blocks', check_alloc_deltas)]: - if checker(deltas): - msg = '%s leaked %s %s, sum=%s' % ( - test, deltas[nwarmup:], item_name, sum(deltas)) - print(msg, file=sys.stderr) - sys.stderr.flush() - with open(fname, "a") as refrep: - print(msg, file=refrep) - refrep.flush() - failed = True - return failed - -def dash_R_cleanup(fs, ps, pic, zdc, abcs): - import gc, copyreg - import _strptime, linecache - import urllib.parse, urllib.request, mimetypes, doctest - import struct, filecmp, collections.abc - from distutils.dir_util import _path_created - from weakref import WeakSet - - # Clear the warnings registry, so they can be displayed again - for mod in sys.modules.values(): - if hasattr(mod, '__warningregistry__'): - del mod.__warningregistry__ - - # Restore some original values. - warnings.filters[:] = fs - copyreg.dispatch_table.clear() - copyreg.dispatch_table.update(ps) - sys.path_importer_cache.clear() - sys.path_importer_cache.update(pic) - try: - import zipimport - except ImportError: - pass # Run unmodified on platforms without zipimport support - else: - zipimport._zip_directory_cache.clear() - zipimport._zip_directory_cache.update(zdc) - - # clear type cache - sys._clear_type_cache() - - # Clear ABC registries, restoring previously saved ABC registries. - for abc in [getattr(collections.abc, a) for a in collections.abc.__all__]: - if not isabstract(abc): - continue - for obj in abc.__subclasses__() + [abc]: - obj._abc_registry = abcs.get(obj, WeakSet()).copy() - obj._abc_cache.clear() - obj._abc_negative_cache.clear() - - # Flush standard output, so that buffered data is sent to the OS and - # associated Python objects are reclaimed. - for stream in (sys.stdout, sys.stderr, sys.__stdout__, sys.__stderr__): - if stream is not None: - stream.flush() - - # Clear assorted module caches. - _path_created.clear() - re.purge() - _strptime._regex_cache.clear() - urllib.parse.clear_cache() - urllib.request.urlcleanup() - linecache.clearcache() - mimetypes._default_mime_types() - filecmp._cache.clear() - struct._clearcache() - doctest.master = None - try: - import ctypes - except ImportError: - # Don't worry about resetting the cache if ctypes is not supported - pass - else: - ctypes._reset_cache() - - # Collect cyclic trash and read memory statistics immediately after. - func1 = sys.getallocatedblocks - func2 = sys.gettotalrefcount - gc.collect() - return func1(), func2() - -def warm_caches(): - # char cache - s = bytes(range(256)) - for i in range(256): - s[i:i+1] - # unicode cache - x = [chr(i) for i in range(256)] - # int cache - x = list(range(-5, 257)) - -def findtestdir(path=None): - return path or os.path.dirname(__file__) or os.curdir - -def removepy(names): - if not names: - return - for idx, name in enumerate(names): - basename, ext = os.path.splitext(name) - if ext == '.py': - names[idx] = basename - -def count(n, word): - if n == 1: - return "%d %s" % (n, word) - else: - return "%d %ss" % (n, word) - -def printlist(x, width=70, indent=4): - """Print the elements of iterable x to stdout. - - Optional arg width (default 70) is the maximum line length. - Optional arg indent (default 4) is the number of blanks with which to - begin each line. - """ - - from textwrap import fill - blanks = ' ' * indent - # Print the sorted list: 'x' may be a '--random' list or a set() - print(fill(' '.join(str(elt) for elt in sorted(x)), width, - initial_indent=blanks, subsequent_indent=blanks)) - - -def main_in_temp_cwd(): - """Run main() in a temporary working directory.""" - if sysconfig.is_python_build(): - try: - os.mkdir(TEMPDIR) - except FileExistsError: - pass - - # Define a writable temp dir that will be used as cwd while running - # the tests. The name of the dir includes the pid to allow parallel - # testing (see the -j option). - test_cwd = 'test_python_{}'.format(os.getpid()) - test_cwd = os.path.join(TEMPDIR, test_cwd) - - # Run the tests in a context manager that temporarily changes the CWD to a - # temporary and writable directory. If it's not possible to create or - # change the CWD, the original CWD will be used. The original CWD is - # available from support.SAVEDCWD. - with support.temp_cwd(test_cwd, quiet=True): - main() - - -if __name__ == '__main__': - # Remove regrtest.py's own directory from the module search path. Despite - # the elimination of implicit relative imports, this is still needed to - # ensure that submodules of the test package do not inappropriately appear - # as top-level modules even when people (or buildbots!) invoke regrtest.py - # directly instead of using the -m switch - mydir = os.path.abspath(os.path.normpath(os.path.dirname(sys.argv[0]))) - i = len(sys.path) - while i >= 0: - i -= 1 - if os.path.abspath(os.path.normpath(sys.path[i])) == mydir: - del sys.path[i] - - # findtestdir() gets the dirname out of __file__, so we have to make it - # absolute before changing the working directory. - # For example __file__ may be relative when running trace or profile. - # See issue #9323. - __file__ = os.path.abspath(__file__) - - # sanity check - assert __file__ == os.path.abspath(sys.argv[0]) - - main_in_temp_cwd() diff --git a/Lib/test/regrtest.py b/Lib/test/regrtest.py --- a/Lib/test/regrtest.py +++ b/Lib/test/regrtest.py @@ -9,1232 +9,9 @@ # We import importlib *ASAP* in order to test #15386 import importlib -import argparse -import builtins -import faulthandler -import io -import json -import locale -import logging import os -import platform -import random -import re -import shutil -import signal import sys -import sysconfig -import tempfile -import time -import traceback -import unittest -import warnings -from inspect import isabstract - -try: - import threading -except ImportError: - threading = None -try: - import _multiprocessing, multiprocessing.process -except ImportError: - multiprocessing = None - -from test.libregrtest import _parse_args - - -# Some times __path__ and __file__ are not absolute (e.g. while running from -# Lib/) and, if we change the CWD to run the tests in a temporary dir, some -# imports might fail. This affects only the modules imported before os.chdir(). -# These modules are searched first in sys.path[0] (so '' -- the CWD) and if -# they are found in the CWD their __file__ and __path__ will be relative (this -# happens before the chdir). All the modules imported after the chdir, are -# not found in the CWD, and since the other paths in sys.path[1:] are absolute -# (site.py absolutize them), the __file__ and __path__ will be absolute too. -# Therefore it is necessary to absolutize manually the __file__ and __path__ of -# the packages to prevent later imports to fail when the CWD is different. -for module in sys.modules.values(): - if hasattr(module, '__path__'): - module.__path__ = [os.path.abspath(path) for path in module.__path__] - if hasattr(module, '__file__'): - module.__file__ = os.path.abspath(module.__file__) - - -# MacOSX (a.k.a. Darwin) has a default stack size that is too small -# for deeply recursive regular expressions. We see this as crashes in -# the Python test suite when running test_re.py and test_sre.py. The -# fix is to set the stack limit to 2048. -# This approach may also be useful for other Unixy platforms that -# suffer from small default stack limits. -if sys.platform == 'darwin': - try: - import resource - except ImportError: - pass - else: - soft, hard = resource.getrlimit(resource.RLIMIT_STACK) - newsoft = min(hard, max(soft, 1024*2048)) - resource.setrlimit(resource.RLIMIT_STACK, (newsoft, hard)) - -# Test result constants. -PASSED = 1 -FAILED = 0 -ENV_CHANGED = -1 -SKIPPED = -2 -RESOURCE_DENIED = -3 -INTERRUPTED = -4 -CHILD_ERROR = -5 # error in a child process - -from test import support - -# When tests are run from the Python build directory, it is best practice -# to keep the test files in a subfolder. This eases the cleanup of leftover -# files using the "make distclean" command. -if sysconfig.is_python_build(): - TEMPDIR = os.path.join(sysconfig.get_config_var('srcdir'), 'build') -else: - TEMPDIR = tempfile.gettempdir() -TEMPDIR = os.path.abspath(TEMPDIR) - -def run_test_in_subprocess(testname, ns): - """Run the given test in a subprocess with --slaveargs. - - ns is the option Namespace parsed from command-line arguments. regrtest - is invoked in a subprocess with the --slaveargs argument; when the - subprocess exits, its return code, stdout and stderr are returned as a - 3-tuple. - """ - from subprocess import Popen, PIPE - base_cmd = ([sys.executable] + support.args_from_interpreter_flags() + - ['-X', 'faulthandler', '-m', 'test.regrtest']) - - slaveargs = ( - (testname, ns.verbose, ns.quiet), - dict(huntrleaks=ns.huntrleaks, - use_resources=ns.use_resources, - output_on_failure=ns.verbose3, - timeout=ns.timeout, failfast=ns.failfast, - match_tests=ns.match_tests)) - # Running the child from the same working directory as regrtest's original - # invocation ensures that TEMPDIR for the child is the same when - # sysconfig.is_python_build() is true. See issue 15300. - popen = Popen(base_cmd + ['--slaveargs', json.dumps(slaveargs)], - stdout=PIPE, stderr=PIPE, - universal_newlines=True, - close_fds=(os.name != 'nt'), - cwd=support.SAVEDCWD) - stdout, stderr = popen.communicate() - retcode = popen.wait() - return retcode, stdout, stderr - - -def main(tests=None, **kwargs): - """Execute a test suite. - - This also parses command-line options and modifies its behavior - accordingly. - - tests -- a list of strings containing test names (optional) - testdir -- the directory in which to look for tests (optional) - - Users other than the Python test suite will certainly want to - specify testdir; if it's omitted, the directory containing the - Python test suite is searched for. - - If the tests argument is omitted, the tests listed on the - command-line will be used. If that's empty, too, then all *.py - files beginning with test_ will be used. - - The other default arguments (verbose, quiet, exclude, - single, randomize, findleaks, use_resources, trace, coverdir, - print_slow, and random_seed) allow programmers calling main() - directly to set the values that would normally be set by flags - on the command line. - """ - # Display the Python traceback on fatal errors (e.g. segfault) - faulthandler.enable(all_threads=True) - - # Display the Python traceback on SIGALRM or SIGUSR1 signal - signals = [] - if hasattr(signal, 'SIGALRM'): - signals.append(signal.SIGALRM) - if hasattr(signal, 'SIGUSR1'): - signals.append(signal.SIGUSR1) - for signum in signals: - faulthandler.register(signum, chain=True) - - replace_stdout() - - support.record_original_stdout(sys.stdout) - - ns = _parse_args(sys.argv[1:], **kwargs) - - if ns.huntrleaks: - # Avoid false positives due to various caches - # filling slowly with random data: - warm_caches() - if ns.memlimit is not None: - support.set_memlimit(ns.memlimit) - if ns.threshold is not None: - import gc - gc.set_threshold(ns.threshold) - if ns.nowindows: - import msvcrt - msvcrt.SetErrorMode(msvcrt.SEM_FAILCRITICALERRORS| - msvcrt.SEM_NOALIGNMENTFAULTEXCEPT| - msvcrt.SEM_NOGPFAULTERRORBOX| - msvcrt.SEM_NOOPENFILEERRORBOX) - try: - msvcrt.CrtSetReportMode - except AttributeError: - # release build - pass - else: - for m in [msvcrt.CRT_WARN, msvcrt.CRT_ERROR, msvcrt.CRT_ASSERT]: - msvcrt.CrtSetReportMode(m, msvcrt.CRTDBG_MODE_FILE) - msvcrt.CrtSetReportFile(m, msvcrt.CRTDBG_FILE_STDERR) - if ns.wait: - input("Press any key to continue...") - - if ns.slaveargs is not None: - args, kwargs = json.loads(ns.slaveargs) - if kwargs.get('huntrleaks'): - unittest.BaseTestSuite._cleanup = False - try: - result = runtest(*args, **kwargs) - except KeyboardInterrupt: - result = INTERRUPTED, '' - except BaseException as e: - traceback.print_exc() - result = CHILD_ERROR, str(e) - sys.stdout.flush() - print() # Force a newline (just in case) - print(json.dumps(result)) - sys.exit(0) - - good = [] - bad = [] - skipped = [] - resource_denieds = [] - environment_changed = [] - interrupted = False - - if ns.findleaks: - try: - import gc - except ImportError: - print('No GC available, disabling findleaks.') - ns.findleaks = False - else: - # Uncomment the line below to report garbage that is not - # freeable by reference counting alone. By default only - # garbage that is not collectable by the GC is reported. - #gc.set_debug(gc.DEBUG_SAVEALL) - found_garbage = [] - - if ns.huntrleaks: - unittest.BaseTestSuite._cleanup = False - - if ns.single: - filename = os.path.join(TEMPDIR, 'pynexttest') - try: - with open(filename, 'r') as fp: - next_test = fp.read().strip() - tests = [next_test] - except OSError: - pass - - if ns.fromfile: - tests = [] - with open(os.path.join(support.SAVEDCWD, ns.fromfile)) as fp: - count_pat = re.compile(r'\[\s*\d+/\s*\d+\]') - for line in fp: - line = count_pat.sub('', line) - guts = line.split() # assuming no test has whitespace in its name - if guts and not guts[0].startswith('#'): - tests.extend(guts) - - # Strip .py extensions. - removepy(ns.args) - removepy(tests) - - stdtests = STDTESTS[:] - nottests = NOTTESTS.copy() - if ns.exclude: - for arg in ns.args: - if arg in stdtests: - stdtests.remove(arg) - nottests.add(arg) - ns.args = [] - - # For a partial run, we do not need to clutter the output. - if ns.verbose or ns.header or not (ns.quiet or ns.single or tests or ns.args): - # Print basic platform information - print("==", platform.python_implementation(), *sys.version.split()) - print("== ", platform.platform(aliased=True), - "%s-endian" % sys.byteorder) - print("== ", "hash algorithm:", sys.hash_info.algorithm, - "64bit" if sys.maxsize > 2**32 else "32bit") - print("== ", os.getcwd()) - print("Testing with flags:", sys.flags) - - # if testdir is set, then we are not running the python tests suite, so - # don't add default tests to be executed or skipped (pass empty values) - if ns.testdir: - alltests = findtests(ns.testdir, list(), set()) - else: - alltests = findtests(ns.testdir, stdtests, nottests) - - selected = tests or ns.args or alltests - if ns.single: - selected = selected[:1] - try: - next_single_test = alltests[alltests.index(selected[0])+1] - except IndexError: - next_single_test = None - # Remove all the selected tests that precede start if it's set. - if ns.start: - try: - del selected[:selected.index(ns.start)] - except ValueError: - print("Couldn't find starting test (%s), using all tests" % ns.start) - if ns.randomize: - if ns.random_seed is None: - ns.random_seed = random.randrange(10000000) - random.seed(ns.random_seed) - print("Using random seed", ns.random_seed) - random.shuffle(selected) - if ns.trace: - import trace, tempfile - tracer = trace.Trace(ignoredirs=[sys.base_prefix, sys.base_exec_prefix, - tempfile.gettempdir()], - trace=False, count=True) - - test_times = [] - support.verbose = ns.verbose # Tell tests to be moderately quiet - support.use_resources = ns.use_resources - save_modules = sys.modules.keys() - - def accumulate_result(test, result): - ok, test_time = result - test_times.append((test_time, test)) - if ok == PASSED: - good.append(test) - elif ok == FAILED: - bad.append(test) - elif ok == ENV_CHANGED: - environment_changed.append(test) - elif ok == SKIPPED: - skipped.append(test) - elif ok == RESOURCE_DENIED: - skipped.append(test) - resource_denieds.append(test) - - if ns.forever: - def test_forever(tests=list(selected)): - while True: - for test in tests: - yield test - if bad: - return - tests = test_forever() - test_count = '' - test_count_width = 3 - else: - tests = iter(selected) - test_count = '/{}'.format(len(selected)) - test_count_width = len(test_count) - 1 - - if ns.use_mp: - try: - from threading import Thread - except ImportError: - print("Multiprocess option requires thread support") - sys.exit(2) - from queue import Queue - debug_output_pat = re.compile(r"\[\d+ refs, \d+ blocks\]$") - output = Queue() - pending = MultiprocessTests(tests) - def work(): - # A worker thread. - try: - while True: - try: - test = next(pending) - except StopIteration: - output.put((None, None, None, None)) - return - retcode, stdout, stderr = run_test_in_subprocess(test, ns) - # Strip last refcount output line if it exists, since it - # comes from the shutdown of the interpreter in the subcommand. - stderr = debug_output_pat.sub("", stderr) - stdout, _, result = stdout.strip().rpartition("\n") - if retcode != 0: - result = (CHILD_ERROR, "Exit code %s" % retcode) - output.put((test, stdout.rstrip(), stderr.rstrip(), result)) - return - if not result: - output.put((None, None, None, None)) - return - result = json.loads(result) - output.put((test, stdout.rstrip(), stderr.rstrip(), result)) - except BaseException: - output.put((None, None, None, None)) - raise - workers = [Thread(target=work) for i in range(ns.use_mp)] - for worker in workers: - worker.start() - finished = 0 - test_index = 1 - try: - while finished < ns.use_mp: - test, stdout, stderr, result = output.get() - if test is None: - finished += 1 - continue - accumulate_result(test, result) - if not ns.quiet: - fmt = "[{1:{0}}{2}/{3}] {4}" if bad else "[{1:{0}}{2}] {4}" - print(fmt.format( - test_count_width, test_index, test_count, - len(bad), test)) - if stdout: - print(stdout) - if stderr: - print(stderr, file=sys.stderr) - sys.stdout.flush() - sys.stderr.flush() - if result[0] == INTERRUPTED: - raise KeyboardInterrupt - if result[0] == CHILD_ERROR: - raise Exception("Child error on {}: {}".format(test, result[1])) - test_index += 1 - except KeyboardInterrupt: - interrupted = True - pending.interrupted = True - for worker in workers: - worker.join() - else: - for test_index, test in enumerate(tests, 1): - if not ns.quiet: - fmt = "[{1:{0}}{2}/{3}] {4}" if bad else "[{1:{0}}{2}] {4}" - print(fmt.format( - test_count_width, test_index, test_count, len(bad), test)) - sys.stdout.flush() - if ns.trace: - # If we're tracing code coverage, then we don't exit with status - # if on a false return value from main. - tracer.runctx('runtest(test, ns.verbose, ns.quiet, timeout=ns.timeout)', - globals=globals(), locals=vars()) - else: - try: - result = runtest(test, ns.verbose, ns.quiet, - ns.huntrleaks, - output_on_failure=ns.verbose3, - timeout=ns.timeout, failfast=ns.failfast, - match_tests=ns.match_tests) - accumulate_result(test, result) - except KeyboardInterrupt: - interrupted = True - break - if ns.findleaks: - gc.collect() - if gc.garbage: - print("Warning: test created", len(gc.garbage), end=' ') - print("uncollectable object(s).") - # move the uncollectable objects somewhere so we don't see - # them again - found_garbage.extend(gc.garbage) - del gc.garbage[:] - # Unload the newly imported modules (best effort finalization) - for module in sys.modules.keys(): - if module not in save_modules and module.startswith("test."): - support.unload(module) - - if interrupted: - # print a newline after ^C - print() - print("Test suite interrupted by signal SIGINT.") - omitted = set(selected) - set(good) - set(bad) - set(skipped) - print(count(len(omitted), "test"), "omitted:") - printlist(omitted) - if good and not ns.quiet: - if not bad and not skipped and not interrupted and len(good) > 1: - print("All", end=' ') - print(count(len(good), "test"), "OK.") - if ns.print_slow: - test_times.sort(reverse=True) - print("10 slowest tests:") - for time, test in test_times[:10]: - print("%s: %.1fs" % (test, time)) - if bad: - print(count(len(bad), "test"), "failed:") - printlist(bad) - if environment_changed: - print("{} altered the execution environment:".format( - count(len(environment_changed), "test"))) - printlist(environment_changed) - if skipped and not ns.quiet: - print(count(len(skipped), "test"), "skipped:") - printlist(skipped) - - if ns.verbose2 and bad: - print("Re-running failed tests in verbose mode") - for test in bad[:]: - print("Re-running test %r in verbose mode" % test) - sys.stdout.flush() - try: - ns.verbose = True - ok = runtest(test, True, ns.quiet, ns.huntrleaks, - timeout=ns.timeout) - except KeyboardInterrupt: - # print a newline separate from the ^C - print() - break - else: - if ok[0] in {PASSED, ENV_CHANGED, SKIPPED, RESOURCE_DENIED}: - bad.remove(test) - else: - if bad: - print(count(len(bad), 'test'), "failed again:") - printlist(bad) - - if ns.single: - if next_single_test: - with open(filename, 'w') as fp: - fp.write(next_single_test + '\n') - else: - os.unlink(filename) - - if ns.trace: - r = tracer.results() - r.write_results(show_missing=True, summary=True, coverdir=ns.coverdir) - - if ns.runleaks: - os.system("leaks %d" % os.getpid()) - - sys.exit(len(bad) > 0 or interrupted) - - -# small set of tests to determine if we have a basically functioning interpreter -# (i.e. if any of these fail, then anything else is likely to follow) -STDTESTS = [ - 'test_grammar', - 'test_opcodes', - 'test_dict', - 'test_builtin', - 'test_exceptions', - 'test_types', - 'test_unittest', - 'test_doctest', - 'test_doctest2', - 'test_support' -] - -# set of tests that we don't want to be executed when using regrtest -NOTTESTS = set() - -def findtests(testdir=None, stdtests=STDTESTS, nottests=NOTTESTS): - """Return a list of all applicable test modules.""" - testdir = findtestdir(testdir) - names = os.listdir(testdir) - tests = [] - others = set(stdtests) | nottests - for name in names: - mod, ext = os.path.splitext(name) - if mod[:5] == "test_" and ext in (".py", "") and mod not in others: - tests.append(mod) - return stdtests + sorted(tests) - -# We do not use a generator so multiple threads can call next(). -class MultiprocessTests(object): - - """A thread-safe iterator over tests for multiprocess mode.""" - - def __init__(self, tests): - self.interrupted = False - self.lock = threading.Lock() - self.tests = tests - - def __iter__(self): - return self - - def __next__(self): - with self.lock: - if self.interrupted: - raise StopIteration('tests interrupted') - return next(self.tests) - -def replace_stdout(): - """Set stdout encoder error handler to backslashreplace (as stderr error - handler) to avoid UnicodeEncodeError when printing a traceback""" - import atexit - - stdout = sys.stdout - sys.stdout = open(stdout.fileno(), 'w', - encoding=stdout.encoding, - errors="backslashreplace", - closefd=False, - newline='\n') - - def restore_stdout(): - sys.stdout.close() - sys.stdout = stdout - atexit.register(restore_stdout) - -def runtest(test, verbose, quiet, - huntrleaks=False, use_resources=None, - output_on_failure=False, failfast=False, match_tests=None, - timeout=None): - """Run a single test. - - test -- the name of the test - verbose -- if true, print more messages - quiet -- if true, don't print 'skipped' messages (probably redundant) - huntrleaks -- run multiple times to test for leaks; requires a debug - build; a triple corresponding to -R's three arguments - use_resources -- list of extra resources to use - output_on_failure -- if true, display test output on failure - timeout -- dump the traceback and exit if a test takes more than - timeout seconds - failfast, match_tests -- See regrtest command-line flags for these. - - Returns the tuple result, test_time, where result is one of the constants: - INTERRUPTED KeyboardInterrupt when run under -j - RESOURCE_DENIED test skipped because resource denied - SKIPPED test skipped for some other reason - ENV_CHANGED test failed because it changed the execution environment - FAILED test failed - PASSED test passed - """ - - if use_resources is not None: - support.use_resources = use_resources - use_timeout = (timeout is not None) - if use_timeout: - faulthandler.dump_traceback_later(timeout, exit=True) - try: - support.match_tests = match_tests - if failfast: - support.failfast = True - if output_on_failure: - support.verbose = True - - # Reuse the same instance to all calls to runtest(). Some - # tests keep a reference to sys.stdout or sys.stderr - # (eg. test_argparse). - if runtest.stringio is None: - stream = io.StringIO() - runtest.stringio = stream - else: - stream = runtest.stringio - stream.seek(0) - stream.truncate() - - orig_stdout = sys.stdout - orig_stderr = sys.stderr - try: - sys.stdout = stream - sys.stderr = stream - result = runtest_inner(test, verbose, quiet, huntrleaks, - display_failure=False) - if result[0] == FAILED: - output = stream.getvalue() - orig_stderr.write(output) - orig_stderr.flush() - finally: - sys.stdout = orig_stdout - sys.stderr = orig_stderr - else: - support.verbose = verbose # Tell tests to be moderately quiet - result = runtest_inner(test, verbose, quiet, huntrleaks, - display_failure=not verbose) - return result - finally: - if use_timeout: - faulthandler.cancel_dump_traceback_later() - cleanup_test_droppings(test, verbose) -runtest.stringio = None - -# Unit tests are supposed to leave the execution environment unchanged -# once they complete. But sometimes tests have bugs, especially when -# tests fail, and the changes to environment go on to mess up other -# tests. This can cause issues with buildbot stability, since tests -# are run in random order and so problems may appear to come and go. -# There are a few things we can save and restore to mitigate this, and -# the following context manager handles this task. - -class saved_test_environment: - """Save bits of the test environment and restore them at block exit. - - with saved_test_environment(testname, verbose, quiet): - #stuff - - Unless quiet is True, a warning is printed to stderr if any of - the saved items was changed by the test. The attribute 'changed' - is initially False, but is set to True if a change is detected. - - If verbose is more than 1, the before and after state of changed - items is also printed. - """ - - changed = False - - def __init__(self, testname, verbose=0, quiet=False): - self.testname = testname - self.verbose = verbose - self.quiet = quiet - - # To add things to save and restore, add a name XXX to the resources list - # and add corresponding get_XXX/restore_XXX functions. get_XXX should - # return the value to be saved and compared against a second call to the - # get function when test execution completes. restore_XXX should accept - # the saved value and restore the resource using it. It will be called if - # and only if a change in the value is detected. - # - # Note: XXX will have any '.' replaced with '_' characters when determining - # the corresponding method names. - - resources = ('sys.argv', 'cwd', 'sys.stdin', 'sys.stdout', 'sys.stderr', - 'os.environ', 'sys.path', 'sys.path_hooks', '__import__', - 'warnings.filters', 'asyncore.socket_map', - 'logging._handlers', 'logging._handlerList', 'sys.gettrace', - 'sys.warnoptions', - # multiprocessing.process._cleanup() may release ref - # to a thread, so check processes first. - 'multiprocessing.process._dangling', 'threading._dangling', - 'sysconfig._CONFIG_VARS', 'sysconfig._INSTALL_SCHEMES', - 'files', 'locale', 'warnings.showwarning', - ) - - def get_sys_argv(self): - return id(sys.argv), sys.argv, sys.argv[:] - def restore_sys_argv(self, saved_argv): - sys.argv = saved_argv[1] - sys.argv[:] = saved_argv[2] - - def get_cwd(self): - return os.getcwd() - def restore_cwd(self, saved_cwd): - os.chdir(saved_cwd) - - def get_sys_stdout(self): - return sys.stdout - def restore_sys_stdout(self, saved_stdout): - sys.stdout = saved_stdout - - def get_sys_stderr(self): - return sys.stderr - def restore_sys_stderr(self, saved_stderr): - sys.stderr = saved_stderr - - def get_sys_stdin(self): - return sys.stdin - def restore_sys_stdin(self, saved_stdin): - sys.stdin = saved_stdin - - def get_os_environ(self): - return id(os.environ), os.environ, dict(os.environ) - def restore_os_environ(self, saved_environ): - os.environ = saved_environ[1] - os.environ.clear() - os.environ.update(saved_environ[2]) - - def get_sys_path(self): - return id(sys.path), sys.path, sys.path[:] - def restore_sys_path(self, saved_path): - sys.path = saved_path[1] - sys.path[:] = saved_path[2] - - def get_sys_path_hooks(self): - return id(sys.path_hooks), sys.path_hooks, sys.path_hooks[:] - def restore_sys_path_hooks(self, saved_hooks): - sys.path_hooks = saved_hooks[1] - sys.path_hooks[:] = saved_hooks[2] - - def get_sys_gettrace(self): - return sys.gettrace() - def restore_sys_gettrace(self, trace_fxn): - sys.settrace(trace_fxn) - - def get___import__(self): - return builtins.__import__ - def restore___import__(self, import_): - builtins.__import__ = import_ - - def get_warnings_filters(self): - return id(warnings.filters), warnings.filters, warnings.filters[:] - def restore_warnings_filters(self, saved_filters): - warnings.filters = saved_filters[1] - warnings.filters[:] = saved_filters[2] - - def get_asyncore_socket_map(self): - asyncore = sys.modules.get('asyncore') - # XXX Making a copy keeps objects alive until __exit__ gets called. - return asyncore and asyncore.socket_map.copy() or {} - def restore_asyncore_socket_map(self, saved_map): - asyncore = sys.modules.get('asyncore') - if asyncore is not None: - asyncore.close_all(ignore_all=True) - asyncore.socket_map.update(saved_map) - - def get_shutil_archive_formats(self): - # we could call get_archives_formats() but that only returns the - # registry keys; we want to check the values too (the functions that - # are registered) - return shutil._ARCHIVE_FORMATS, shutil._ARCHIVE_FORMATS.copy() - def restore_shutil_archive_formats(self, saved): - shutil._ARCHIVE_FORMATS = saved[0] - shutil._ARCHIVE_FORMATS.clear() - shutil._ARCHIVE_FORMATS.update(saved[1]) - - def get_shutil_unpack_formats(self): - return shutil._UNPACK_FORMATS, shutil._UNPACK_FORMATS.copy() - def restore_shutil_unpack_formats(self, saved): - shutil._UNPACK_FORMATS = saved[0] - shutil._UNPACK_FORMATS.clear() - shutil._UNPACK_FORMATS.update(saved[1]) - - def get_logging__handlers(self): - # _handlers is a WeakValueDictionary - return id(logging._handlers), logging._handlers, logging._handlers.copy() - def restore_logging__handlers(self, saved_handlers): - # Can't easily revert the logging state - pass - - def get_logging__handlerList(self): - # _handlerList is a list of weakrefs to handlers - return id(logging._handlerList), logging._handlerList, logging._handlerList[:] - def restore_logging__handlerList(self, saved_handlerList): - # Can't easily revert the logging state - pass - - def get_sys_warnoptions(self): - return id(sys.warnoptions), sys.warnoptions, sys.warnoptions[:] - def restore_sys_warnoptions(self, saved_options): - sys.warnoptions = saved_options[1] - sys.warnoptions[:] = saved_options[2] - - # Controlling dangling references to Thread objects can make it easier - # to track reference leaks. - def get_threading__dangling(self): - if not threading: - return None - # This copies the weakrefs without making any strong reference - return threading._dangling.copy() - def restore_threading__dangling(self, saved): - if not threading: - return - threading._dangling.clear() - threading._dangling.update(saved) - - # Same for Process objects - def get_multiprocessing_process__dangling(self): - if not multiprocessing: - return None - # Unjoined process objects can survive after process exits - multiprocessing.process._cleanup() - # This copies the weakrefs without making any strong reference - return multiprocessing.process._dangling.copy() - def restore_multiprocessing_process__dangling(self, saved): - if not multiprocessing: - return - multiprocessing.process._dangling.clear() - multiprocessing.process._dangling.update(saved) - - def get_sysconfig__CONFIG_VARS(self): - # make sure the dict is initialized - sysconfig.get_config_var('prefix') - return (id(sysconfig._CONFIG_VARS), sysconfig._CONFIG_VARS, - dict(sysconfig._CONFIG_VARS)) - def restore_sysconfig__CONFIG_VARS(self, saved): - sysconfig._CONFIG_VARS = saved[1] - sysconfig._CONFIG_VARS.clear() - sysconfig._CONFIG_VARS.update(saved[2]) - - def get_sysconfig__INSTALL_SCHEMES(self): - return (id(sysconfig._INSTALL_SCHEMES), sysconfig._INSTALL_SCHEMES, - sysconfig._INSTALL_SCHEMES.copy()) - def restore_sysconfig__INSTALL_SCHEMES(self, saved): - sysconfig._INSTALL_SCHEMES = saved[1] - sysconfig._INSTALL_SCHEMES.clear() - sysconfig._INSTALL_SCHEMES.update(saved[2]) - - def get_files(self): - return sorted(fn + ('/' if os.path.isdir(fn) else '') - for fn in os.listdir()) - def restore_files(self, saved_value): - fn = support.TESTFN - if fn not in saved_value and (fn + '/') not in saved_value: - if os.path.isfile(fn): - support.unlink(fn) - elif os.path.isdir(fn): - support.rmtree(fn) - - _lc = [getattr(locale, lc) for lc in dir(locale) - if lc.startswith('LC_')] - def get_locale(self): - pairings = [] - for lc in self._lc: - try: - pairings.append((lc, locale.setlocale(lc, None))) - except (TypeError, ValueError): - continue - return pairings - def restore_locale(self, saved): - for lc, setting in saved: - locale.setlocale(lc, setting) - - def get_warnings_showwarning(self): - return warnings.showwarning - def restore_warnings_showwarning(self, fxn): - warnings.showwarning = fxn - - def resource_info(self): - for name in self.resources: - method_suffix = name.replace('.', '_') - get_name = 'get_' + method_suffix - restore_name = 'restore_' + method_suffix - yield name, getattr(self, get_name), getattr(self, restore_name) - - def __enter__(self): - self.saved_values = dict((name, get()) for name, get, restore - in self.resource_info()) - return self - - def __exit__(self, exc_type, exc_val, exc_tb): - saved_values = self.saved_values - del self.saved_values - for name, get, restore in self.resource_info(): - current = get() - original = saved_values.pop(name) - # Check for changes to the resource's value - if current != original: - self.changed = True - restore(original) - if not self.quiet: - print("Warning -- {} was modified by {}".format( - name, self.testname), - file=sys.stderr) - if self.verbose > 1: - print(" Before: {}\n After: {} ".format( - original, current), - file=sys.stderr) - return False - - -def runtest_inner(test, verbose, quiet, - huntrleaks=False, display_failure=True): - support.unload(test) - - test_time = 0.0 - refleak = False # True if the test leaked references. - try: - if test.startswith('test.'): - abstest = test - else: - # Always import it from the test package - abstest = 'test.' + test - with saved_test_environment(test, verbose, quiet) as environment: - start_time = time.time() - the_module = importlib.import_module(abstest) - # If the test has a test_main, that will run the appropriate - # tests. If not, use normal unittest test loading. - test_runner = getattr(the_module, "test_main", None) - if test_runner is None: - def test_runner(): - loader = unittest.TestLoader() - tests = loader.loadTestsFromModule(the_module) - for error in loader.errors: - print(error, file=sys.stderr) - if loader.errors: - raise Exception("errors while loading tests") - support.run_unittest(tests) - test_runner() - if huntrleaks: - refleak = dash_R(the_module, test, test_runner, huntrleaks) - test_time = time.time() - start_time - except support.ResourceDenied as msg: - if not quiet: - print(test, "skipped --", msg) - sys.stdout.flush() - return RESOURCE_DENIED, test_time - except unittest.SkipTest as msg: - if not quiet: - print(test, "skipped --", msg) - sys.stdout.flush() - return SKIPPED, test_time - except KeyboardInterrupt: - raise - except support.TestFailed as msg: - if display_failure: - print("test", test, "failed --", msg, file=sys.stderr) - else: - print("test", test, "failed", file=sys.stderr) - sys.stderr.flush() - return FAILED, test_time - except: - msg = traceback.format_exc() - print("test", test, "crashed --", msg, file=sys.stderr) - sys.stderr.flush() - return FAILED, test_time - else: - if refleak: - return FAILED, test_time - if environment.changed: - return ENV_CHANGED, test_time - return PASSED, test_time - -def cleanup_test_droppings(testname, verbose): - import shutil - import stat - import gc - - # First kill any dangling references to open files etc. - # This can also issue some ResourceWarnings which would otherwise get - # triggered during the following test run, and possibly produce failures. - gc.collect() - - # Try to clean up junk commonly left behind. While tests shouldn't leave - # any files or directories behind, when a test fails that can be tedious - # for it to arrange. The consequences can be especially nasty on Windows, - # since if a test leaves a file open, it cannot be deleted by name (while - # there's nothing we can do about that here either, we can display the - # name of the offending test, which is a real help). - for name in (support.TESTFN, - "db_home", - ): - if not os.path.exists(name): - continue - - if os.path.isdir(name): - kind, nuker = "directory", shutil.rmtree - elif os.path.isfile(name): - kind, nuker = "file", os.unlink - else: - raise SystemError("os.path says %r exists but is neither " - "directory nor file" % name) - - if verbose: - print("%r left behind %s %r" % (testname, kind, name)) - try: - # if we have chmod, fix possible permissions problems - # that might prevent cleanup - if (hasattr(os, 'chmod')): - os.chmod(name, stat.S_IRWXU | stat.S_IRWXG | stat.S_IRWXO) - nuker(name) - except Exception as msg: - print(("%r left behind %s %r and it couldn't be " - "removed: %s" % (testname, kind, name, msg)), file=sys.stderr) - -def dash_R(the_module, test, indirect_test, huntrleaks): - """Run a test multiple times, looking for reference leaks. - - Returns: - False if the test didn't leak references; True if we detected refleaks. - """ - # This code is hackish and inelegant, but it seems to do the job. - import copyreg - import collections.abc - - if not hasattr(sys, 'gettotalrefcount'): - raise Exception("Tracking reference leaks requires a debug build " - "of Python") - - # Save current values for dash_R_cleanup() to restore. - fs = warnings.filters[:] - ps = copyreg.dispatch_table.copy() - pic = sys.path_importer_cache.copy() - try: - import zipimport - except ImportError: - zdc = None # Run unmodified on platforms without zipimport support - else: - zdc = zipimport._zip_directory_cache.copy() - abcs = {} - for abc in [getattr(collections.abc, a) for a in collections.abc.__all__]: - if not isabstract(abc): - continue - for obj in abc.__subclasses__() + [abc]: - abcs[obj] = obj._abc_registry.copy() - - nwarmup, ntracked, fname = huntrleaks - fname = os.path.join(support.SAVEDCWD, fname) - repcount = nwarmup + ntracked - rc_deltas = [0] * repcount - alloc_deltas = [0] * repcount - - print("beginning", repcount, "repetitions", file=sys.stderr) - print(("1234567890"*(repcount//10 + 1))[:repcount], file=sys.stderr) - sys.stderr.flush() - for i in range(repcount): - indirect_test() - alloc_after, rc_after = dash_R_cleanup(fs, ps, pic, zdc, abcs) - sys.stderr.write('.') - sys.stderr.flush() - if i >= nwarmup: - rc_deltas[i] = rc_after - rc_before - alloc_deltas[i] = alloc_after - alloc_before - alloc_before, rc_before = alloc_after, rc_after - print(file=sys.stderr) - # These checkers return False on success, True on failure - def check_rc_deltas(deltas): - return any(deltas) - def check_alloc_deltas(deltas): - # At least 1/3rd of 0s - if 3 * deltas.count(0) < len(deltas): - return True - # Nothing else than 1s, 0s and -1s - if not set(deltas) <= {1,0,-1}: - return True - return False - failed = False - for deltas, item_name, checker in [ - (rc_deltas, 'references', check_rc_deltas), - (alloc_deltas, 'memory blocks', check_alloc_deltas)]: - if checker(deltas): - msg = '%s leaked %s %s, sum=%s' % ( - test, deltas[nwarmup:], item_name, sum(deltas)) - print(msg, file=sys.stderr) - sys.stderr.flush() - with open(fname, "a") as refrep: - print(msg, file=refrep) - refrep.flush() - failed = True - return failed - -def dash_R_cleanup(fs, ps, pic, zdc, abcs): - import gc, copyreg - import _strptime, linecache - import urllib.parse, urllib.request, mimetypes, doctest - import struct, filecmp, collections.abc - from distutils.dir_util import _path_created - from weakref import WeakSet - - # Clear the warnings registry, so they can be displayed again - for mod in sys.modules.values(): - if hasattr(mod, '__warningregistry__'): - del mod.__warningregistry__ - - # Restore some original values. - warnings.filters[:] = fs - copyreg.dispatch_table.clear() - copyreg.dispatch_table.update(ps) - sys.path_importer_cache.clear() - sys.path_importer_cache.update(pic) - try: - import zipimport - except ImportError: - pass # Run unmodified on platforms without zipimport support - else: - zipimport._zip_directory_cache.clear() - zipimport._zip_directory_cache.update(zdc) - - # clear type cache - sys._clear_type_cache() - - # Clear ABC registries, restoring previously saved ABC registries. - for abc in [getattr(collections.abc, a) for a in collections.abc.__all__]: - if not isabstract(abc): - continue - for obj in abc.__subclasses__() + [abc]: - obj._abc_registry = abcs.get(obj, WeakSet()).copy() - obj._abc_cache.clear() - obj._abc_negative_cache.clear() - - # Flush standard output, so that buffered data is sent to the OS and - # associated Python objects are reclaimed. - for stream in (sys.stdout, sys.stderr, sys.__stdout__, sys.__stderr__): - if stream is not None: - stream.flush() - - # Clear assorted module caches. - _path_created.clear() - re.purge() - _strptime._regex_cache.clear() - urllib.parse.clear_cache() - urllib.request.urlcleanup() - linecache.clearcache() - mimetypes._default_mime_types() - filecmp._cache.clear() - struct._clearcache() - doctest.master = None - try: - import ctypes - except ImportError: - # Don't worry about resetting the cache if ctypes is not supported - pass - else: - ctypes._reset_cache() - - # Collect cyclic trash and read memory statistics immediately after. - func1 = sys.getallocatedblocks - func2 = sys.gettotalrefcount - gc.collect() - return func1(), func2() - -def warm_caches(): - # char cache - s = bytes(range(256)) - for i in range(256): - s[i:i+1] - # unicode cache - x = [chr(i) for i in range(256)] - # int cache - x = list(range(-5, 257)) - -def findtestdir(path=None): - return path or os.path.dirname(__file__) or os.curdir - -def removepy(names): - if not names: - return - for idx, name in enumerate(names): - basename, ext = os.path.splitext(name) - if ext == '.py': - names[idx] = basename - -def count(n, word): - if n == 1: - return "%d %s" % (n, word) - else: - return "%d %ss" % (n, word) - -def printlist(x, width=70, indent=4): - """Print the elements of iterable x to stdout. - - Optional arg width (default 70) is the maximum line length. - Optional arg indent (default 4) is the number of blanks with which to - begin each line. - """ - - from textwrap import fill - blanks = ' ' * indent - # Print the sorted list: 'x' may be a '--random' list or a set() - print(fill(' '.join(str(elt) for elt in sorted(x)), width, - initial_indent=blanks, subsequent_indent=blanks)) - - -def main_in_temp_cwd(): - """Run main() in a temporary working directory.""" - if sysconfig.is_python_build(): - try: - os.mkdir(TEMPDIR) - except FileExistsError: - pass - - # Define a writable temp dir that will be used as cwd while running - # the tests. The name of the dir includes the pid to allow parallel - # testing (see the -j option). - test_cwd = 'test_python_{}'.format(os.getpid()) - test_cwd = os.path.join(TEMPDIR, test_cwd) - - # Run the tests in a context manager that temporarily changes the CWD to a - # temporary and writable directory. If it's not possible to create or - # change the CWD, the original CWD will be used. The original CWD is - # available from support.SAVEDCWD. - with support.temp_cwd(test_cwd, quiet=True): - main() +from test.libregrtest import main_in_temp_cwd if __name__ == '__main__': diff --git a/Lib/test/test_regrtest.py b/Lib/test/test_regrtest.py --- a/Lib/test/test_regrtest.py +++ b/Lib/test/test_regrtest.py @@ -7,7 +7,8 @@ import getopt import os.path import unittest -from test import regrtest, support, libregrtest +from test import libregrtest +from test import support class ParseArgsTestCase(unittest.TestCase): @@ -15,7 +16,7 @@ def checkError(self, args, msg): with support.captured_stderr() as err, self.assertRaises(SystemExit): - regrtest._parse_args(args) + libregrtest._parse_args(args) self.assertIn(msg, err.getvalue()) def test_help(self): @@ -23,82 +24,82 @@ with self.subTest(opt=opt): with support.captured_stdout() as out, \ self.assertRaises(SystemExit): - regrtest._parse_args([opt]) + libregrtest._parse_args([opt]) self.assertIn('Run Python regression tests.', out.getvalue()) @unittest.skipUnless(hasattr(faulthandler, 'dump_traceback_later'), "faulthandler.dump_traceback_later() required") def test_timeout(self): - ns = regrtest._parse_args(['--timeout', '4.2']) + ns = libregrtest._parse_args(['--timeout', '4.2']) self.assertEqual(ns.timeout, 4.2) self.checkError(['--timeout'], 'expected one argument') self.checkError(['--timeout', 'foo'], 'invalid float value') def test_wait(self): - ns = regrtest._parse_args(['--wait']) + ns = libregrtest._parse_args(['--wait']) self.assertTrue(ns.wait) def test_slaveargs(self): - ns = regrtest._parse_args(['--slaveargs', '[[], {}]']) + ns = libregrtest._parse_args(['--slaveargs', '[[], {}]']) self.assertEqual(ns.slaveargs, '[[], {}]') self.checkError(['--slaveargs'], 'expected one argument') def test_start(self): for opt in '-S', '--start': with self.subTest(opt=opt): - ns = regrtest._parse_args([opt, 'foo']) + ns = libregrtest._parse_args([opt, 'foo']) self.assertEqual(ns.start, 'foo') self.checkError([opt], 'expected one argument') def test_verbose(self): - ns = regrtest._parse_args(['-v']) + ns = libregrtest._parse_args(['-v']) self.assertEqual(ns.verbose, 1) - ns = regrtest._parse_args(['-vvv']) + ns = libregrtest._parse_args(['-vvv']) self.assertEqual(ns.verbose, 3) - ns = regrtest._parse_args(['--verbose']) + ns = libregrtest._parse_args(['--verbose']) self.assertEqual(ns.verbose, 1) - ns = regrtest._parse_args(['--verbose'] * 3) + ns = libregrtest._parse_args(['--verbose'] * 3) self.assertEqual(ns.verbose, 3) - ns = regrtest._parse_args([]) + ns = libregrtest._parse_args([]) self.assertEqual(ns.verbose, 0) def test_verbose2(self): for opt in '-w', '--verbose2': with self.subTest(opt=opt): - ns = regrtest._parse_args([opt]) + ns = libregrtest._parse_args([opt]) self.assertTrue(ns.verbose2) def test_verbose3(self): for opt in '-W', '--verbose3': with self.subTest(opt=opt): - ns = regrtest._parse_args([opt]) + ns = libregrtest._parse_args([opt]) self.assertTrue(ns.verbose3) def test_quiet(self): for opt in '-q', '--quiet': with self.subTest(opt=opt): - ns = regrtest._parse_args([opt]) + ns = libregrtest._parse_args([opt]) self.assertTrue(ns.quiet) self.assertEqual(ns.verbose, 0) def test_slow(self): for opt in '-o', '--slow': with self.subTest(opt=opt): - ns = regrtest._parse_args([opt]) + ns = libregrtest._parse_args([opt]) self.assertTrue(ns.print_slow) def test_header(self): - ns = regrtest._parse_args(['--header']) + ns = libregrtest._parse_args(['--header']) self.assertTrue(ns.header) def test_randomize(self): for opt in '-r', '--randomize': with self.subTest(opt=opt): - ns = regrtest._parse_args([opt]) + ns = libregrtest._parse_args([opt]) self.assertTrue(ns.randomize) def test_randseed(self): - ns = regrtest._parse_args(['--randseed', '12345']) + ns = libregrtest._parse_args(['--randseed', '12345']) self.assertEqual(ns.random_seed, 12345) self.assertTrue(ns.randomize) self.checkError(['--randseed'], 'expected one argument') @@ -107,7 +108,7 @@ def test_fromfile(self): for opt in '-f', '--fromfile': with self.subTest(opt=opt): - ns = regrtest._parse_args([opt, 'foo']) + ns = libregrtest._parse_args([opt, 'foo']) self.assertEqual(ns.fromfile, 'foo') self.checkError([opt], 'expected one argument') self.checkError([opt, 'foo', '-s'], "don't go together") @@ -115,42 +116,42 @@ def test_exclude(self): for opt in '-x', '--exclude': with self.subTest(opt=opt): - ns = regrtest._parse_args([opt]) + ns = libregrtest._parse_args([opt]) self.assertTrue(ns.exclude) def test_single(self): for opt in '-s', '--single': with self.subTest(opt=opt): - ns = regrtest._parse_args([opt]) + ns = libregrtest._parse_args([opt]) self.assertTrue(ns.single) self.checkError([opt, '-f', 'foo'], "don't go together") def test_match(self): for opt in '-m', '--match': with self.subTest(opt=opt): - ns = regrtest._parse_args([opt, 'pattern']) + ns = libregrtest._parse_args([opt, 'pattern']) self.assertEqual(ns.match_tests, 'pattern') self.checkError([opt], 'expected one argument') def test_failfast(self): for opt in '-G', '--failfast': with self.subTest(opt=opt): - ns = regrtest._parse_args([opt, '-v']) + ns = libregrtest._parse_args([opt, '-v']) self.assertTrue(ns.failfast) - ns = regrtest._parse_args([opt, '-W']) + ns = libregrtest._parse_args([opt, '-W']) self.assertTrue(ns.failfast) self.checkError([opt], '-G/--failfast needs either -v or -W') def test_use(self): for opt in '-u', '--use': with self.subTest(opt=opt): - ns = regrtest._parse_args([opt, 'gui,network']) + ns = libregrtest._parse_args([opt, 'gui,network']) self.assertEqual(ns.use_resources, ['gui', 'network']) - ns = regrtest._parse_args([opt, 'gui,none,network']) + ns = libregrtest._parse_args([opt, 'gui,none,network']) self.assertEqual(ns.use_resources, ['network']) expected = list(libregrtest.RESOURCE_NAMES) expected.remove('gui') - ns = regrtest._parse_args([opt, 'all,-gui']) + ns = libregrtest._parse_args([opt, 'all,-gui']) self.assertEqual(ns.use_resources, expected) self.checkError([opt], 'expected one argument') self.checkError([opt, 'foo'], 'invalid resource') @@ -158,31 +159,31 @@ def test_memlimit(self): for opt in '-M', '--memlimit': with self.subTest(opt=opt): - ns = regrtest._parse_args([opt, '4G']) + ns = libregrtest._parse_args([opt, '4G']) self.assertEqual(ns.memlimit, '4G') self.checkError([opt], 'expected one argument') def test_testdir(self): - ns = regrtest._parse_args(['--testdir', 'foo']) + ns = libregrtest._parse_args(['--testdir', 'foo']) self.assertEqual(ns.testdir, os.path.join(support.SAVEDCWD, 'foo')) self.checkError(['--testdir'], 'expected one argument') def test_runleaks(self): for opt in '-L', '--runleaks': with self.subTest(opt=opt): - ns = regrtest._parse_args([opt]) + ns = libregrtest._parse_args([opt]) self.assertTrue(ns.runleaks) def test_huntrleaks(self): for opt in '-R', '--huntrleaks': with self.subTest(opt=opt): - ns = regrtest._parse_args([opt, ':']) + ns = libregrtest._parse_args([opt, ':']) self.assertEqual(ns.huntrleaks, (5, 4, 'reflog.txt')) - ns = regrtest._parse_args([opt, '6:']) + ns = libregrtest._parse_args([opt, '6:']) self.assertEqual(ns.huntrleaks, (6, 4, 'reflog.txt')) - ns = regrtest._parse_args([opt, ':3']) + ns = libregrtest._parse_args([opt, ':3']) self.assertEqual(ns.huntrleaks, (5, 3, 'reflog.txt')) - ns = regrtest._parse_args([opt, '6:3:leaks.log']) + ns = libregrtest._parse_args([opt, '6:3:leaks.log']) self.assertEqual(ns.huntrleaks, (6, 3, 'leaks.log')) self.checkError([opt], 'expected one argument') self.checkError([opt, '6'], @@ -193,7 +194,7 @@ def test_multiprocess(self): for opt in '-j', '--multiprocess': with self.subTest(opt=opt): - ns = regrtest._parse_args([opt, '2']) + ns = libregrtest._parse_args([opt, '2']) self.assertEqual(ns.use_mp, 2) self.checkError([opt], 'expected one argument') self.checkError([opt, 'foo'], 'invalid int value') @@ -204,13 +205,13 @@ def test_coverage(self): for opt in '-T', '--coverage': with self.subTest(opt=opt): - ns = regrtest._parse_args([opt]) + ns = libregrtest._parse_args([opt]) self.assertTrue(ns.trace) def test_coverdir(self): for opt in '-D', '--coverdir': with self.subTest(opt=opt): - ns = regrtest._parse_args([opt, 'foo']) + ns = libregrtest._parse_args([opt, 'foo']) self.assertEqual(ns.coverdir, os.path.join(support.SAVEDCWD, 'foo')) self.checkError([opt], 'expected one argument') @@ -218,13 +219,13 @@ def test_nocoverdir(self): for opt in '-N', '--nocoverdir': with self.subTest(opt=opt): - ns = regrtest._parse_args([opt]) + ns = libregrtest._parse_args([opt]) self.assertIsNone(ns.coverdir) def test_threshold(self): for opt in '-t', '--threshold': with self.subTest(opt=opt): - ns = regrtest._parse_args([opt, '1000']) + ns = libregrtest._parse_args([opt, '1000']) self.assertEqual(ns.threshold, 1000) self.checkError([opt], 'expected one argument') self.checkError([opt, 'foo'], 'invalid int value') @@ -232,13 +233,13 @@ def test_nowindows(self): for opt in '-n', '--nowindows': with self.subTest(opt=opt): - ns = regrtest._parse_args([opt]) + ns = libregrtest._parse_args([opt]) self.assertTrue(ns.nowindows) def test_forever(self): for opt in '-F', '--forever': with self.subTest(opt=opt): - ns = regrtest._parse_args([opt]) + ns = libregrtest._parse_args([opt]) self.assertTrue(ns.forever) @@ -246,26 +247,26 @@ self.checkError(['--xxx'], 'usage:') def test_long_option__partial(self): - ns = regrtest._parse_args(['--qui']) + ns = libregrtest._parse_args(['--qui']) self.assertTrue(ns.quiet) self.assertEqual(ns.verbose, 0) def test_two_options(self): - ns = regrtest._parse_args(['--quiet', '--exclude']) + ns = libregrtest._parse_args(['--quiet', '--exclude']) self.assertTrue(ns.quiet) self.assertEqual(ns.verbose, 0) self.assertTrue(ns.exclude) def test_option_with_empty_string_value(self): - ns = regrtest._parse_args(['--start', '']) + ns = libregrtest._parse_args(['--start', '']) self.assertEqual(ns.start, '') def test_arg(self): - ns = regrtest._parse_args(['foo']) + ns = libregrtest._parse_args(['foo']) self.assertEqual(ns.args, ['foo']) def test_option_and_arg(self): - ns = regrtest._parse_args(['--quiet', 'foo']) + ns = libregrtest._parse_args(['--quiet', 'foo']) self.assertTrue(ns.quiet) self.assertEqual(ns.verbose, 0) self.assertEqual(ns.args, ['foo']) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 27 00:51:26 2015 From: python-checkins at python.org (terry.reedy) Date: Sat, 26 Sep 2015 22:51:26 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzI0OTg4?= =?utf-8?q?=3A_Idle_ScrolledList_context_menus_=28used_in_debugger=29?= Message-ID: <20150926225125.81623.62697@psf.io> https://hg.python.org/cpython/rev/85a4c95ad02f changeset: 98286:85a4c95ad02f branch: 3.4 parent: 98275:88d98f6c2d7d user: Terry Jan Reedy date: Sat Sep 26 18:50:26 2015 -0400 summary: Issue #24988: Idle ScrolledList context menus (used in debugger) now work on Mac Aqua. Patch by Mark Roseman. files: Lib/idlelib/ScrolledList.py | 7 ++++++- 1 files changed, 6 insertions(+), 1 deletions(-) diff --git a/Lib/idlelib/ScrolledList.py b/Lib/idlelib/ScrolledList.py --- a/Lib/idlelib/ScrolledList.py +++ b/Lib/idlelib/ScrolledList.py @@ -1,4 +1,5 @@ from tkinter import * +from idlelib import macosxSupport class ScrolledList: @@ -22,7 +23,11 @@ # Bind events to the list box listbox.bind("", self.click_event) listbox.bind("", self.double_click_event) - listbox.bind("", self.popup_event) + if macosxSupport.isAquaTk(): + listbox.bind("", self.popup_event) + listbox.bind("", self.popup_event) + else: + listbox.bind("", self.popup_event) listbox.bind("", self.up_event) listbox.bind("", self.down_event) # Mark as empty -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 27 00:51:26 2015 From: python-checkins at python.org (terry.reedy) Date: Sat, 26 Sep 2015 22:51:26 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Merge_with_3=2E4?= Message-ID: <20150926225126.94129.86813@psf.io> https://hg.python.org/cpython/rev/d2e78afd8866 changeset: 98287:d2e78afd8866 branch: 3.5 parent: 98278:005590a4a005 parent: 98286:85a4c95ad02f user: Terry Jan Reedy date: Sat Sep 26 18:50:44 2015 -0400 summary: Merge with 3.4 files: Lib/idlelib/ScrolledList.py | 7 ++++++- 1 files changed, 6 insertions(+), 1 deletions(-) diff --git a/Lib/idlelib/ScrolledList.py b/Lib/idlelib/ScrolledList.py --- a/Lib/idlelib/ScrolledList.py +++ b/Lib/idlelib/ScrolledList.py @@ -1,4 +1,5 @@ from tkinter import * +from idlelib import macosxSupport class ScrolledList: @@ -22,7 +23,11 @@ # Bind events to the list box listbox.bind("", self.click_event) listbox.bind("", self.double_click_event) - listbox.bind("", self.popup_event) + if macosxSupport.isAquaTk(): + listbox.bind("", self.popup_event) + listbox.bind("", self.popup_event) + else: + listbox.bind("", self.popup_event) listbox.bind("", self.up_event) listbox.bind("", self.down_event) # Mark as empty -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 27 00:51:25 2015 From: python-checkins at python.org (terry.reedy) Date: Sat, 26 Sep 2015 22:51:25 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzI0OTg4?= =?utf-8?q?=3A_Idle_ScrolledList_context_menus_=28used_in_debugger=29?= Message-ID: <20150926225125.115507.41587@psf.io> https://hg.python.org/cpython/rev/bf11034f0291 changeset: 98285:bf11034f0291 branch: 2.7 parent: 98281:fc6d62db8d42 user: Terry Jan Reedy date: Sat Sep 26 18:50:20 2015 -0400 summary: Issue #24988: Idle ScrolledList context menus (used in debugger) now work on Mac Aqua. Patch by Mark Roseman. files: Lib/idlelib/ScrolledList.py | 7 ++++++- 1 files changed, 6 insertions(+), 1 deletions(-) diff --git a/Lib/idlelib/ScrolledList.py b/Lib/idlelib/ScrolledList.py --- a/Lib/idlelib/ScrolledList.py +++ b/Lib/idlelib/ScrolledList.py @@ -1,4 +1,5 @@ from Tkinter import * +from idlelib import macosxSupport class ScrolledList: @@ -22,7 +23,11 @@ # Bind events to the list box listbox.bind("", self.click_event) listbox.bind("", self.double_click_event) - listbox.bind("", self.popup_event) + if macosxSupport.isAquaTk(): + listbox.bind("", self.popup_event) + listbox.bind("", self.popup_event) + else: + listbox.bind("", self.popup_event) listbox.bind("", self.up_event) listbox.bind("", self.down_event) # Mark as empty -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 27 00:51:26 2015 From: python-checkins at python.org (terry.reedy) Date: Sat, 26 Sep 2015 22:51:26 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge_with_3=2E5?= Message-ID: <20150926225126.98372.37563@psf.io> https://hg.python.org/cpython/rev/d5739600368f changeset: 98288:d5739600368f parent: 98284:4a9418ed0d0c parent: 98287:d2e78afd8866 user: Terry Jan Reedy date: Sat Sep 26 18:50:58 2015 -0400 summary: Merge with 3.5 files: Lib/idlelib/ScrolledList.py | 7 ++++++- 1 files changed, 6 insertions(+), 1 deletions(-) diff --git a/Lib/idlelib/ScrolledList.py b/Lib/idlelib/ScrolledList.py --- a/Lib/idlelib/ScrolledList.py +++ b/Lib/idlelib/ScrolledList.py @@ -1,4 +1,5 @@ from tkinter import * +from idlelib import macosxSupport class ScrolledList: @@ -22,7 +23,11 @@ # Bind events to the list box listbox.bind("", self.click_event) listbox.bind("", self.double_click_event) - listbox.bind("", self.popup_event) + if macosxSupport.isAquaTk(): + listbox.bind("", self.popup_event) + listbox.bind("", self.popup_event) + else: + listbox.bind("", self.popup_event) listbox.bind("", self.up_event) listbox.bind("", self.down_event) # Mark as empty -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 27 02:04:49 2015 From: python-checkins at python.org (terry.reedy) Date: Sun, 27 Sep 2015 00:04:49 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge_with_3=2E5?= Message-ID: <20150927000449.11704.59905@psf.io> https://hg.python.org/cpython/rev/e732c42abedf changeset: 98292:e732c42abedf parent: 98288:d5739600368f parent: 98291:fd9009f313b9 user: Terry Jan Reedy date: Sat Sep 26 20:04:23 2015 -0400 summary: Merge with 3.5 files: Lib/idlelib/AutoCompleteWindow.py | 1 + Lib/idlelib/CallTipWindow.py | 1 + 2 files changed, 2 insertions(+), 0 deletions(-) diff --git a/Lib/idlelib/AutoCompleteWindow.py b/Lib/idlelib/AutoCompleteWindow.py --- a/Lib/idlelib/AutoCompleteWindow.py +++ b/Lib/idlelib/AutoCompleteWindow.py @@ -192,6 +192,7 @@ scrollbar.config(command=listbox.yview) scrollbar.pack(side=RIGHT, fill=Y) listbox.pack(side=LEFT, fill=BOTH, expand=True) + acw.lift() # work around bug in Tk 8.5.18+ (issue #24570) # Initialize the listbox selection self.listbox.select_set(self._binary_search(self.start)) diff --git a/Lib/idlelib/CallTipWindow.py b/Lib/idlelib/CallTipWindow.py --- a/Lib/idlelib/CallTipWindow.py +++ b/Lib/idlelib/CallTipWindow.py @@ -72,6 +72,7 @@ background="#ffffe0", relief=SOLID, borderwidth=1, font = self.widget['font']) self.label.pack() + tw.lift() # work around bug in Tk 8.5.18+ (issue #24570) self.checkhideid = self.widget.bind(CHECKHIDE_VIRTUAL_EVENT_NAME, self.checkhide_event) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 27 02:04:49 2015 From: python-checkins at python.org (terry.reedy) Date: Sun, 27 Sep 2015 00:04:49 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Merge_with_3=2E4?= Message-ID: <20150927000449.94135.4312@psf.io> https://hg.python.org/cpython/rev/fd9009f313b9 changeset: 98291:fd9009f313b9 branch: 3.5 parent: 98287:d2e78afd8866 parent: 98290:6687630e201a user: Terry Jan Reedy date: Sat Sep 26 20:04:09 2015 -0400 summary: Merge with 3.4 files: Lib/idlelib/AutoCompleteWindow.py | 1 + Lib/idlelib/CallTipWindow.py | 1 + 2 files changed, 2 insertions(+), 0 deletions(-) diff --git a/Lib/idlelib/AutoCompleteWindow.py b/Lib/idlelib/AutoCompleteWindow.py --- a/Lib/idlelib/AutoCompleteWindow.py +++ b/Lib/idlelib/AutoCompleteWindow.py @@ -192,6 +192,7 @@ scrollbar.config(command=listbox.yview) scrollbar.pack(side=RIGHT, fill=Y) listbox.pack(side=LEFT, fill=BOTH, expand=True) + acw.lift() # work around bug in Tk 8.5.18+ (issue #24570) # Initialize the listbox selection self.listbox.select_set(self._binary_search(self.start)) diff --git a/Lib/idlelib/CallTipWindow.py b/Lib/idlelib/CallTipWindow.py --- a/Lib/idlelib/CallTipWindow.py +++ b/Lib/idlelib/CallTipWindow.py @@ -72,6 +72,7 @@ background="#ffffe0", relief=SOLID, borderwidth=1, font = self.widget['font']) self.label.pack() + tw.lift() # work around bug in Tk 8.5.18+ (issue #24570) self.checkhideid = self.widget.bind(CHECKHIDE_VIRTUAL_EVENT_NAME, self.checkhide_event) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 27 02:04:50 2015 From: python-checkins at python.org (terry.reedy) Date: Sun, 27 Sep 2015 00:04:50 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzI0NTcw?= =?utf-8?q?=3A_Idle=3A_make_calltip_and_completion_boxes_appear_on_Macs?= Message-ID: <20150927000448.31197.51227@psf.io> https://hg.python.org/cpython/rev/b5bc7e9dab77 changeset: 98289:b5bc7e9dab77 branch: 2.7 parent: 98285:bf11034f0291 user: Terry Jan Reedy date: Sat Sep 26 20:03:51 2015 -0400 summary: Issue #24570: Idle: make calltip and completion boxes appear on Macs affected by a tk regression. Initial patch by Mark Roseman. files: Lib/idlelib/AutoCompleteWindow.py | 1 + Lib/idlelib/CallTipWindow.py | 1 + 2 files changed, 2 insertions(+), 0 deletions(-) diff --git a/Lib/idlelib/AutoCompleteWindow.py b/Lib/idlelib/AutoCompleteWindow.py --- a/Lib/idlelib/AutoCompleteWindow.py +++ b/Lib/idlelib/AutoCompleteWindow.py @@ -192,6 +192,7 @@ scrollbar.config(command=listbox.yview) scrollbar.pack(side=RIGHT, fill=Y) listbox.pack(side=LEFT, fill=BOTH, expand=True) + acw.lift() # work around bug in Tk 8.5.18+ (issue #24570) # Initialize the listbox selection self.listbox.select_set(self._binary_search(self.start)) diff --git a/Lib/idlelib/CallTipWindow.py b/Lib/idlelib/CallTipWindow.py --- a/Lib/idlelib/CallTipWindow.py +++ b/Lib/idlelib/CallTipWindow.py @@ -72,6 +72,7 @@ background="#ffffe0", relief=SOLID, borderwidth=1, font = self.widget['font']) self.label.pack() + tw.lift() # work around bug in Tk 8.5.18+ (issue #24570) self.checkhideid = self.widget.bind(CHECKHIDE_VIRTUAL_EVENT_NAME, self.checkhide_event) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 27 02:04:50 2015 From: python-checkins at python.org (terry.reedy) Date: Sun, 27 Sep 2015 00:04:50 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzI0NTcw?= =?utf-8?q?=3A_Idle=3A_make_calltip_and_completion_boxes_appear_on_Macs?= Message-ID: <20150927000449.9951.28542@psf.io> https://hg.python.org/cpython/rev/6687630e201a changeset: 98290:6687630e201a branch: 3.4 parent: 98286:85a4c95ad02f user: Terry Jan Reedy date: Sat Sep 26 20:03:57 2015 -0400 summary: Issue #24570: Idle: make calltip and completion boxes appear on Macs affected by a tk regression. Initial patch by Mark Roseman. files: Lib/idlelib/AutoCompleteWindow.py | 1 + Lib/idlelib/CallTipWindow.py | 1 + 2 files changed, 2 insertions(+), 0 deletions(-) diff --git a/Lib/idlelib/AutoCompleteWindow.py b/Lib/idlelib/AutoCompleteWindow.py --- a/Lib/idlelib/AutoCompleteWindow.py +++ b/Lib/idlelib/AutoCompleteWindow.py @@ -192,6 +192,7 @@ scrollbar.config(command=listbox.yview) scrollbar.pack(side=RIGHT, fill=Y) listbox.pack(side=LEFT, fill=BOTH, expand=True) + acw.lift() # work around bug in Tk 8.5.18+ (issue #24570) # Initialize the listbox selection self.listbox.select_set(self._binary_search(self.start)) diff --git a/Lib/idlelib/CallTipWindow.py b/Lib/idlelib/CallTipWindow.py --- a/Lib/idlelib/CallTipWindow.py +++ b/Lib/idlelib/CallTipWindow.py @@ -72,6 +72,7 @@ background="#ffffe0", relief=SOLID, borderwidth=1, font = self.widget['font']) self.label.pack() + tw.lift() # work around bug in Tk 8.5.18+ (issue #24570) self.checkhideid = self.widget.bind(CHECKHIDE_VIRTUAL_EVENT_NAME, self.checkhide_event) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 27 02:47:08 2015 From: python-checkins at python.org (raymond.hettinger) Date: Sun, 27 Sep 2015 00:47:08 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Bump_up_the_maximum_number?= =?utf-8?q?_of_freeblocks?= Message-ID: <20150927004708.82660.95303@psf.io> https://hg.python.org/cpython/rev/919a3487fb00 changeset: 98293:919a3487fb00 user: Raymond Hettinger date: Sat Sep 26 17:47:02 2015 -0700 summary: Bump up the maximum number of freeblocks files: Modules/_collectionsmodule.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Modules/_collectionsmodule.c b/Modules/_collectionsmodule.c --- a/Modules/_collectionsmodule.c +++ b/Modules/_collectionsmodule.c @@ -121,7 +121,7 @@ added at about the same rate as old blocks are being freed. */ -#define MAXFREEBLOCKS 10 +#define MAXFREEBLOCKS 16 static Py_ssize_t numfreeblocks = 0; static block *freeblocks[MAXFREEBLOCKS]; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 27 06:11:19 2015 From: python-checkins at python.org (raymond.hettinger) Date: Sun, 27 Sep 2015 04:11:19 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Minor_tweak_to_the_order_o?= =?utf-8?q?f_variable_updates=2E?= Message-ID: <20150927041119.82642.66906@psf.io> https://hg.python.org/cpython/rev/ef71fea740a6 changeset: 98294:ef71fea740a6 user: Raymond Hettinger date: Sat Sep 26 21:11:05 2015 -0700 summary: Minor tweak to the order of variable updates. files: Modules/_collectionsmodule.c | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Modules/_collectionsmodule.c b/Modules/_collectionsmodule.c --- a/Modules/_collectionsmodule.c +++ b/Modules/_collectionsmodule.c @@ -315,8 +315,8 @@ MARK_END(b->rightlink); deque->rightindex = -1; } + Py_SIZE(deque)++; Py_INCREF(item); - Py_SIZE(deque)++; deque->rightindex++; deque->rightblock->data[deque->rightindex] = item; deque_trim_left(deque); @@ -340,8 +340,8 @@ MARK_END(b->leftlink); deque->leftindex = BLOCKLEN; } + Py_SIZE(deque)++; Py_INCREF(item); - Py_SIZE(deque)++; deque->leftindex--; deque->leftblock->data[deque->leftindex] = item; deque_trim_right(deque); -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 27 06:31:29 2015 From: python-checkins at python.org (raymond.hettinger) Date: Sun, 27 Sep 2015 04:31:29 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Move_the_copy_and_clear_fu?= =?utf-8?q?nctions_upwards_to_eliminate_unnecessary_forward?= Message-ID: <20150927043129.9955.34578@psf.io> https://hg.python.org/cpython/rev/bb1a2944bcb6 changeset: 98295:bb1a2944bcb6 user: Raymond Hettinger date: Sat Sep 26 21:31:23 2015 -0700 summary: Move the copy and clear functions upwards to eliminate unnecessary forward references. files: Modules/_collectionsmodule.c | 230 +++++++++++----------- 1 files changed, 113 insertions(+), 117 deletions(-) diff --git a/Modules/_collectionsmodule.c b/Modules/_collectionsmodule.c --- a/Modules/_collectionsmodule.c +++ b/Modules/_collectionsmodule.c @@ -522,7 +522,40 @@ return (PyObject *)deque; } -static PyObject *deque_copy(PyObject *deque); +static PyObject * +deque_copy(PyObject *deque) +{ + dequeobject *old_deque = (dequeobject *)deque; + if (Py_TYPE(deque) == &deque_type) { + dequeobject *new_deque; + PyObject *rv; + + new_deque = (dequeobject *)deque_new(&deque_type, (PyObject *)NULL, (PyObject *)NULL); + if (new_deque == NULL) + return NULL; + new_deque->maxlen = old_deque->maxlen; + /* Fast path for the deque_repeat() common case where len(deque) == 1 */ + if (Py_SIZE(deque) == 1 && new_deque->maxlen != 0) { + PyObject *item = old_deque->leftblock->data[old_deque->leftindex]; + rv = deque_append(new_deque, item); + } else { + rv = deque_extend(new_deque, deque); + } + if (rv != NULL) { + Py_DECREF(rv); + return (PyObject *)new_deque; + } + Py_DECREF(new_deque); + return NULL; + } + if (old_deque->maxlen < 0) + return PyObject_CallFunction((PyObject *)(Py_TYPE(deque)), "O", deque, NULL); + else + return PyObject_CallFunction((PyObject *)(Py_TYPE(deque)), "Oi", + deque, old_deque->maxlen, NULL); +} + +PyDoc_STRVAR(copy_doc, "Return a shallow copy of a deque."); static PyObject * deque_concat(dequeobject *deque, PyObject *other) @@ -552,7 +585,85 @@ return new_deque; } -static void deque_clear(dequeobject *deque); +static void +deque_clear(dequeobject *deque) +{ + block *b; + block *prevblock; + block *leftblock; + Py_ssize_t leftindex; + Py_ssize_t n; + PyObject *item; + + /* During the process of clearing a deque, decrefs can cause the + deque to mutate. To avoid fatal confusion, we have to make the + deque empty before clearing the blocks and never refer to + anything via deque->ref while clearing. (This is the same + technique used for clearing lists, sets, and dicts.) + + Making the deque empty requires allocating a new empty block. In + the unlikely event that memory is full, we fall back to an + alternate method that doesn't require a new block. Repeating + pops in a while-loop is slower, possibly re-entrant (and a clever + adversary could cause it to never terminate). + */ + + b = newblock(0); + if (b == NULL) { + PyErr_Clear(); + goto alternate_method; + } + + /* Remember the old size, leftblock, and leftindex */ + leftblock = deque->leftblock; + leftindex = deque->leftindex; + n = Py_SIZE(deque); + + /* Set the deque to be empty using the newly allocated block */ + MARK_END(b->leftlink); + MARK_END(b->rightlink); + Py_SIZE(deque) = 0; + deque->leftblock = b; + deque->rightblock = b; + deque->leftindex = CENTER + 1; + deque->rightindex = CENTER; + deque->state++; + + /* Now the old size, leftblock, and leftindex are disconnected from + the empty deque and we can use them to decref the pointers. + */ + while (n--) { + item = leftblock->data[leftindex]; + Py_DECREF(item); + leftindex++; + if (leftindex == BLOCKLEN && n) { + CHECK_NOT_END(leftblock->rightlink); + prevblock = leftblock; + leftblock = leftblock->rightlink; + leftindex = 0; + freeblock(prevblock); + } + } + CHECK_END(leftblock->rightlink); + freeblock(leftblock); + return; + + alternate_method: + while (Py_SIZE(deque)) { + item = deque_pop(deque, NULL); + assert (item != NULL); + Py_DECREF(item); + } +} + +static PyObject * +deque_clearmethod(dequeobject *deque) +{ + deque_clear(deque); + Py_RETURN_NONE; +} + +PyDoc_STRVAR(clear_doc, "Remove all elements from the deque."); static PyObject * deque_inplace_repeat(dequeobject *deque, Py_ssize_t n) @@ -1058,77 +1169,6 @@ PyDoc_STRVAR(remove_doc, "D.remove(value) -- remove first occurrence of value."); -static void -deque_clear(dequeobject *deque) -{ - block *b; - block *prevblock; - block *leftblock; - Py_ssize_t leftindex; - Py_ssize_t n; - PyObject *item; - - /* During the process of clearing a deque, decrefs can cause the - deque to mutate. To avoid fatal confusion, we have to make the - deque empty before clearing the blocks and never refer to - anything via deque->ref while clearing. (This is the same - technique used for clearing lists, sets, and dicts.) - - Making the deque empty requires allocating a new empty block. In - the unlikely event that memory is full, we fall back to an - alternate method that doesn't require a new block. Repeating - pops in a while-loop is slower, possibly re-entrant (and a clever - adversary could cause it to never terminate). - */ - - b = newblock(0); - if (b == NULL) { - PyErr_Clear(); - goto alternate_method; - } - - /* Remember the old size, leftblock, and leftindex */ - leftblock = deque->leftblock; - leftindex = deque->leftindex; - n = Py_SIZE(deque); - - /* Set the deque to be empty using the newly allocated block */ - MARK_END(b->leftlink); - MARK_END(b->rightlink); - Py_SIZE(deque) = 0; - deque->leftblock = b; - deque->rightblock = b; - deque->leftindex = CENTER + 1; - deque->rightindex = CENTER; - deque->state++; - - /* Now the old size, leftblock, and leftindex are disconnected from - the empty deque and we can use them to decref the pointers. - */ - while (n--) { - item = leftblock->data[leftindex]; - Py_DECREF(item); - leftindex++; - if (leftindex == BLOCKLEN && n) { - CHECK_NOT_END(leftblock->rightlink); - prevblock = leftblock; - leftblock = leftblock->rightlink; - leftindex = 0; - freeblock(prevblock); - } - } - CHECK_END(leftblock->rightlink); - freeblock(leftblock); - return; - - alternate_method: - while (Py_SIZE(deque)) { - item = deque_pop(deque, NULL); - assert (item != NULL); - Py_DECREF(item); - } -} - static int valid_index(Py_ssize_t i, Py_ssize_t limit) { @@ -1229,15 +1269,6 @@ return 0; } -static PyObject * -deque_clearmethod(dequeobject *deque) -{ - deque_clear(deque); - Py_RETURN_NONE; -} - -PyDoc_STRVAR(clear_doc, "Remove all elements from the deque."); - static void deque_dealloc(dequeobject *deque) { @@ -1277,41 +1308,6 @@ } static PyObject * -deque_copy(PyObject *deque) -{ - dequeobject *old_deque = (dequeobject *)deque; - if (Py_TYPE(deque) == &deque_type) { - dequeobject *new_deque; - PyObject *rv; - - new_deque = (dequeobject *)deque_new(&deque_type, (PyObject *)NULL, (PyObject *)NULL); - if (new_deque == NULL) - return NULL; - new_deque->maxlen = old_deque->maxlen; - /* Fast path for the deque_repeat() common case where len(deque) == 1 */ - if (Py_SIZE(deque) == 1 && new_deque->maxlen != 0) { - PyObject *item = old_deque->leftblock->data[old_deque->leftindex]; - rv = deque_append(new_deque, item); - } else { - rv = deque_extend(new_deque, deque); - } - if (rv != NULL) { - Py_DECREF(rv); - return (PyObject *)new_deque; - } - Py_DECREF(new_deque); - return NULL; - } - if (old_deque->maxlen < 0) - return PyObject_CallFunction((PyObject *)(Py_TYPE(deque)), "O", deque, NULL); - else - return PyObject_CallFunction((PyObject *)(Py_TYPE(deque)), "Oi", - deque, old_deque->maxlen, NULL); -} - -PyDoc_STRVAR(copy_doc, "Return a shallow copy of a deque."); - -static PyObject * deque_reduce(dequeobject *deque) { PyObject *dict, *result, *aslist; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 27 09:09:14 2015 From: python-checkins at python.org (benjamin.peterson) Date: Sun, 27 Sep 2015 07:09:14 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_detect_alpn_by?= =?utf-8?q?_feature_flag_not_openssl_version_=28closes_=2323329=29?= Message-ID: <20150927070914.81619.28487@psf.io> https://hg.python.org/cpython/rev/38a5b0f6531b changeset: 98296:38a5b0f6531b branch: 3.5 parent: 98291:fd9009f313b9 user: Benjamin Peterson date: Sun Sep 27 00:09:02 2015 -0700 summary: detect alpn by feature flag not openssl version (closes #23329) files: Misc/NEWS | 3 +++ Modules/_ssl.c | 3 +-- 2 files changed, 4 insertions(+), 2 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -21,6 +21,9 @@ Library ------- +- Issue #23329: Allow the ssl module to be built with older versions of + LibreSSL. + - Prevent overflow in _Unpickler_Read. - Issue #25047: The XML encoding declaration written by Element Tree now diff --git a/Modules/_ssl.c b/Modules/_ssl.c --- a/Modules/_ssl.c +++ b/Modules/_ssl.c @@ -109,8 +109,7 @@ # define HAVE_SNI 0 #endif -/* ALPN added in OpenSSL 1.0.2 */ -#if !defined(LIBRESSL_VERSION_NUMBER) && OPENSSL_VERSION_NUMBER >= 0x1000200fL && !defined(OPENSSL_NO_TLSEXT) +#ifdef TLSEXT_TYPE_application_layer_protocol_negotiation # define HAVE_ALPN #endif -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 27 09:09:14 2015 From: python-checkins at python.org (benjamin.peterson) Date: Sun, 27 Sep 2015 07:09:14 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogbWVyZ2UgMy41ICgjMjMzMjkp?= Message-ID: <20150927070914.9927.2170@psf.io> https://hg.python.org/cpython/rev/747996431c7e changeset: 98297:747996431c7e parent: 98295:bb1a2944bcb6 parent: 98296:38a5b0f6531b user: Benjamin Peterson date: Sun Sep 27 00:09:09 2015 -0700 summary: merge 3.5 (#23329) files: Misc/NEWS | 3 +++ Modules/_ssl.c | 3 +-- 2 files changed, 4 insertions(+), 2 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -141,6 +141,9 @@ Library ------- +- Issue #23329: Allow the ssl module to be built with older versions of + LibreSSL. + - Prevent overflow in _Unpickler_Read. - Issue #25047: The XML encoding declaration written by Element Tree now diff --git a/Modules/_ssl.c b/Modules/_ssl.c --- a/Modules/_ssl.c +++ b/Modules/_ssl.c @@ -109,8 +109,7 @@ # define HAVE_SNI 0 #endif -/* ALPN added in OpenSSL 1.0.2 */ -#if !defined(LIBRESSL_VERSION_NUMBER) && OPENSSL_VERSION_NUMBER >= 0x1000200fL && !defined(OPENSSL_NO_TLSEXT) +#ifdef TLSEXT_TYPE_application_layer_protocol_negotiation # define HAVE_ALPN #endif -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 27 10:16:27 2015 From: python-checkins at python.org (benjamin.peterson) Date: Sun, 27 Sep 2015 08:16:27 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_initialize_ret?= =?utf-8?q?urn_value_to_NULL_to_avoid_compiler_compliants_=28closes_=23252?= =?utf-8?b?NDUp?= Message-ID: <20150927081627.98366.8825@psf.io> https://hg.python.org/cpython/rev/f0dcf7599517 changeset: 98298:f0dcf7599517 branch: 3.4 parent: 98290:6687630e201a user: Benjamin Peterson date: Sun Sep 27 01:16:03 2015 -0700 summary: initialize return value to NULL to avoid compiler compliants (closes #25245) files: Modules/_pickle.c | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diff --git a/Modules/_pickle.c b/Modules/_pickle.c --- a/Modules/_pickle.c +++ b/Modules/_pickle.c @@ -1182,6 +1182,7 @@ { Py_ssize_t num_read; + *s = NULL; if (self->next_read_idx > PY_SSIZE_T_MAX - n) { PickleState *st = _Pickle_GetGlobalState(); PyErr_SetString(st->UnpicklingError, -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 27 10:16:28 2015 From: python-checkins at python.org (benjamin.peterson) Date: Sun, 27 Sep 2015 08:16:28 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_merge_3=2E4_=28=2325245=29?= Message-ID: <20150927081628.94117.89167@psf.io> https://hg.python.org/cpython/rev/ef3b833b98c2 changeset: 98299:ef3b833b98c2 branch: 3.5 parent: 98296:38a5b0f6531b parent: 98298:f0dcf7599517 user: Benjamin Peterson date: Sun Sep 27 01:16:12 2015 -0700 summary: merge 3.4 (#25245) files: Modules/_pickle.c | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diff --git a/Modules/_pickle.c b/Modules/_pickle.c --- a/Modules/_pickle.c +++ b/Modules/_pickle.c @@ -1193,6 +1193,7 @@ { Py_ssize_t num_read; + *s = NULL; if (self->next_read_idx > PY_SSIZE_T_MAX - n) { PickleState *st = _Pickle_GetGlobalState(); PyErr_SetString(st->UnpicklingError, -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 27 10:16:28 2015 From: python-checkins at python.org (benjamin.peterson) Date: Sun, 27 Sep 2015 08:16:28 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogbWVyZ2UgMy41ICgjMjUyNDUp?= Message-ID: <20150927081628.9937.58702@psf.io> https://hg.python.org/cpython/rev/4d23598f1428 changeset: 98300:4d23598f1428 parent: 98297:747996431c7e parent: 98299:ef3b833b98c2 user: Benjamin Peterson date: Sun Sep 27 01:16:20 2015 -0700 summary: merge 3.5 (#25245) files: Modules/_pickle.c | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diff --git a/Modules/_pickle.c b/Modules/_pickle.c --- a/Modules/_pickle.c +++ b/Modules/_pickle.c @@ -1193,6 +1193,7 @@ { Py_ssize_t num_read; + *s = NULL; if (self->next_read_idx > PY_SSIZE_T_MAX - n) { PickleState *st = _Pickle_GetGlobalState(); PyErr_SetString(st->UnpicklingError, -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 27 10:23:51 2015 From: python-checkins at python.org (benjamin.peterson) Date: Sun, 27 Sep 2015 08:23:51 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_merge_3=2E4?= Message-ID: <20150927082351.98378.76328@psf.io> https://hg.python.org/cpython/rev/bacb540ab6ba changeset: 98303:bacb540ab6ba branch: 3.5 parent: 98299:ef3b833b98c2 parent: 98302:30da62ea4d7c user: Benjamin Peterson date: Sun Sep 27 01:23:35 2015 -0700 summary: merge 3.4 files: Doc/library/hashlib.rst | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Doc/library/hashlib.rst b/Doc/library/hashlib.rst --- a/Doc/library/hashlib.rst +++ b/Doc/library/hashlib.rst @@ -176,8 +176,8 @@ compute the digests of data sharing a common initial substring. -Key Derivation Function ------------------------ +Key derivation +-------------- Key derivation and key stretching algorithms are designed for secure password hashing. Naive algorithms such as ``sha1(password)`` are not resistant against -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 27 10:23:51 2015 From: python-checkins at python.org (benjamin.peterson) Date: Sun, 27 Sep 2015 08:23:51 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_shorten_and_fi?= =?utf-8?q?x_casing_of_title?= Message-ID: <20150927082351.81619.38524@psf.io> https://hg.python.org/cpython/rev/56bd41a77fd0 changeset: 98301:56bd41a77fd0 branch: 2.7 parent: 98289:b5bc7e9dab77 user: Benjamin Peterson date: Sun Sep 27 01:23:10 2015 -0700 summary: shorten and fix casing of title files: Doc/library/hashlib.rst | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Doc/library/hashlib.rst b/Doc/library/hashlib.rst --- a/Doc/library/hashlib.rst +++ b/Doc/library/hashlib.rst @@ -153,8 +153,8 @@ compute the digests of strings that share a common initial substring. -Key Derivation Function ------------------------ +Key derivation +-------------- Key derivation and key stretching algorithms are designed for secure password hashing. Naive algorithms such as ``sha1(password)`` are not resistant against -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 27 10:23:51 2015 From: python-checkins at python.org (benjamin.peterson) Date: Sun, 27 Sep 2015 08:23:51 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_shorten_and_fi?= =?utf-8?q?x_casing_of_title?= Message-ID: <20150927082351.9947.69178@psf.io> https://hg.python.org/cpython/rev/30da62ea4d7c changeset: 98302:30da62ea4d7c branch: 3.4 parent: 98298:f0dcf7599517 user: Benjamin Peterson date: Sun Sep 27 01:23:10 2015 -0700 summary: shorten and fix casing of title files: Doc/library/hashlib.rst | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Doc/library/hashlib.rst b/Doc/library/hashlib.rst --- a/Doc/library/hashlib.rst +++ b/Doc/library/hashlib.rst @@ -176,8 +176,8 @@ compute the digests of data sharing a common initial substring. -Key Derivation Function ------------------------ +Key derivation +-------------- Key derivation and key stretching algorithms are designed for secure password hashing. Naive algorithms such as ``sha1(password)`` are not resistant against -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 27 10:23:51 2015 From: python-checkins at python.org (benjamin.peterson) Date: Sun, 27 Sep 2015 08:23:51 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogbWVyZ2UgMy41?= Message-ID: <20150927082351.9935.93497@psf.io> https://hg.python.org/cpython/rev/fcdaa1dab846 changeset: 98304:fcdaa1dab846 parent: 98300:4d23598f1428 parent: 98303:bacb540ab6ba user: Benjamin Peterson date: Sun Sep 27 01:23:40 2015 -0700 summary: merge 3.5 files: Doc/library/hashlib.rst | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Doc/library/hashlib.rst b/Doc/library/hashlib.rst --- a/Doc/library/hashlib.rst +++ b/Doc/library/hashlib.rst @@ -176,8 +176,8 @@ compute the digests of data sharing a common initial substring. -Key Derivation Function ------------------------ +Key derivation +-------------- Key derivation and key stretching algorithms are designed for secure password hashing. Naive algorithms such as ``sha1(password)`` are not resistant against -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 27 10:40:48 2015 From: python-checkins at python.org (terry.reedy) Date: Sun, 27 Sep 2015 08:40:48 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge_with_3=2E5?= Message-ID: <20150927084048.31175.97425@psf.io> https://hg.python.org/cpython/rev/1aed2444b10f changeset: 98308:1aed2444b10f parent: 98304:fcdaa1dab846 parent: 98307:a04a568f7af6 user: Terry Jan Reedy date: Sun Sep 27 04:40:34 2015 -0400 summary: Merge with 3.5 files: Lib/idlelib/help.py | 36 ++++++++++++-------------------- 1 files changed, 14 insertions(+), 22 deletions(-) diff --git a/Lib/idlelib/help.py b/Lib/idlelib/help.py --- a/Lib/idlelib/help.py +++ b/Lib/idlelib/help.py @@ -55,12 +55,11 @@ self.hdrlink = False # used so we don't show header links self.level = 0 # indentation level self.pre = False # displaying preformatted text - self.hprefix = '' # strip e.g. '25.5' from headings + self.hprefix = '' # prefix such as '25.5' to strip from headings self.nested_dl = False # if we're in a nested
      self.simplelist = False # simple list (no double spacing) - self.tocid = 1 # id for table of contents entries - self.contents = [] # map toc ids to section titles - self.data = '' # to record data within header tags for toc + self.toc = [] # pair headers with text indexes for toc + self.header = '' # text within header tags for toc def indent(self, amt=1): self.level += amt @@ -111,14 +110,10 @@ elif tag == 'a' and class_ == 'headerlink': self.hdrlink = True elif tag == 'h1': - self.text.mark_set('toc'+str(self.tocid), - self.text.index('end-1line')) self.tags = tag elif tag in ['h2', 'h3']: if self.show: - self.data = '' - self.text.mark_set('toc'+str(self.tocid), - self.text.index('end-1line')) + self.header = '' self.text.insert('end', '\n\n') self.tags = tag if self.show: @@ -128,10 +123,8 @@ "Handle endtags in help.html." if tag in ['h1', 'h2', 'h3']: self.indent(0) # clear tag, reset indent - if self.show and tag in ['h1', 'h2', 'h3']: - title = self.data - self.contents.append(('toc'+str(self.tocid), title)) - self.tocid += 1 + if self.show: + self.toc.append((self.header, self.text.index('insert'))) elif tag in ['span', 'em']: self.chartags = '' elif tag == 'a': @@ -151,7 +144,7 @@ if self.tags in ['h1', 'h2', 'h3'] and self.hprefix != '': if d[0:len(self.hprefix)] == self.hprefix: d = d[len(self.hprefix):].strip() - self.data += d + self.header += d self.text.insert('end', d, (self.tags, self.chartags)) @@ -205,19 +198,18 @@ self['background'] = text['background'] scroll = Scrollbar(self, command=text.yview) text['yscrollcommand'] = scroll.set + self.rowconfigure(0, weight=1) + self.columnconfigure(1, weight=1) # text + self.toc_menu(text).grid(column=0, row=0, sticky='nw') text.grid(column=1, row=0, sticky='nsew') scroll.grid(column=2, row=0, sticky='ns') - self.grid_columnconfigure(1, weight=1) - self.grid_rowconfigure(0, weight=1) - toc = self.contents_widget(text) - toc.grid(column=0, row=0, sticky='nw') - def contents_widget(self, text): - "Create table of contents." + def toc_menu(self, text): + "Create table of contents as drop-down menu." toc = Menubutton(self, text='TOC') drop = Menu(toc, tearoff=False) - for tag, lbl in text.parser.contents: - drop.add_command(label=lbl, command=lambda mark=tag:text.see(mark)) + for lbl, dex in text.parser.toc: + drop.add_command(label=lbl, command=lambda dex=dex:text.yview(dex)) toc['menu'] = drop return toc -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 27 10:40:48 2015 From: python-checkins at python.org (terry.reedy) Date: Sun, 27 Sep 2015 08:40:48 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Merge_with_3=2E4?= Message-ID: <20150927084048.16573.25142@psf.io> https://hg.python.org/cpython/rev/a04a568f7af6 changeset: 98307:a04a568f7af6 branch: 3.5 parent: 98303:bacb540ab6ba parent: 98306:5b635a3ca3d5 user: Terry Jan Reedy date: Sun Sep 27 04:40:23 2015 -0400 summary: Merge with 3.4 files: Lib/idlelib/help.py | 36 ++++++++++++-------------------- 1 files changed, 14 insertions(+), 22 deletions(-) diff --git a/Lib/idlelib/help.py b/Lib/idlelib/help.py --- a/Lib/idlelib/help.py +++ b/Lib/idlelib/help.py @@ -55,12 +55,11 @@ self.hdrlink = False # used so we don't show header links self.level = 0 # indentation level self.pre = False # displaying preformatted text - self.hprefix = '' # strip e.g. '25.5' from headings + self.hprefix = '' # prefix such as '25.5' to strip from headings self.nested_dl = False # if we're in a nested
      self.simplelist = False # simple list (no double spacing) - self.tocid = 1 # id for table of contents entries - self.contents = [] # map toc ids to section titles - self.data = '' # to record data within header tags for toc + self.toc = [] # pair headers with text indexes for toc + self.header = '' # text within header tags for toc def indent(self, amt=1): self.level += amt @@ -111,14 +110,10 @@ elif tag == 'a' and class_ == 'headerlink': self.hdrlink = True elif tag == 'h1': - self.text.mark_set('toc'+str(self.tocid), - self.text.index('end-1line')) self.tags = tag elif tag in ['h2', 'h3']: if self.show: - self.data = '' - self.text.mark_set('toc'+str(self.tocid), - self.text.index('end-1line')) + self.header = '' self.text.insert('end', '\n\n') self.tags = tag if self.show: @@ -128,10 +123,8 @@ "Handle endtags in help.html." if tag in ['h1', 'h2', 'h3']: self.indent(0) # clear tag, reset indent - if self.show and tag in ['h1', 'h2', 'h3']: - title = self.data - self.contents.append(('toc'+str(self.tocid), title)) - self.tocid += 1 + if self.show: + self.toc.append((self.header, self.text.index('insert'))) elif tag in ['span', 'em']: self.chartags = '' elif tag == 'a': @@ -151,7 +144,7 @@ if self.tags in ['h1', 'h2', 'h3'] and self.hprefix != '': if d[0:len(self.hprefix)] == self.hprefix: d = d[len(self.hprefix):].strip() - self.data += d + self.header += d self.text.insert('end', d, (self.tags, self.chartags)) @@ -205,19 +198,18 @@ self['background'] = text['background'] scroll = Scrollbar(self, command=text.yview) text['yscrollcommand'] = scroll.set + self.rowconfigure(0, weight=1) + self.columnconfigure(1, weight=1) # text + self.toc_menu(text).grid(column=0, row=0, sticky='nw') text.grid(column=1, row=0, sticky='nsew') scroll.grid(column=2, row=0, sticky='ns') - self.grid_columnconfigure(1, weight=1) - self.grid_rowconfigure(0, weight=1) - toc = self.contents_widget(text) - toc.grid(column=0, row=0, sticky='nw') - def contents_widget(self, text): - "Create table of contents." + def toc_menu(self, text): + "Create table of contents as drop-down menu." toc = Menubutton(self, text='TOC') drop = Menu(toc, tearoff=False) - for tag, lbl in text.parser.contents: - drop.add_command(label=lbl, command=lambda mark=tag:text.see(mark)) + for lbl, dex in text.parser.toc: + drop.add_command(label=lbl, command=lambda dex=dex:text.yview(dex)) toc['menu'] = drop return toc -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 27 10:40:48 2015 From: python-checkins at python.org (terry.reedy) Date: Sun, 27 Sep 2015 08:40:48 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzI1MTk4?= =?utf-8?q?=3A_When_using_the_Idle_dov_TOC_menu=2C_put_the_section_title_a?= =?utf-8?q?t_the?= Message-ID: <20150927084048.3656.40640@psf.io> https://hg.python.org/cpython/rev/5b635a3ca3d5 changeset: 98306:5b635a3ca3d5 branch: 3.4 parent: 98302:30da62ea4d7c user: Terry Jan Reedy date: Sun Sep 27 04:40:08 2015 -0400 summary: Issue #25198: When using the Idle dov TOC menu, put the section title at the top of the window, unless it is too near the bottom to do do. files: Lib/idlelib/help.py | 36 ++++++++++++-------------------- 1 files changed, 14 insertions(+), 22 deletions(-) diff --git a/Lib/idlelib/help.py b/Lib/idlelib/help.py --- a/Lib/idlelib/help.py +++ b/Lib/idlelib/help.py @@ -55,12 +55,11 @@ self.hdrlink = False # used so we don't show header links self.level = 0 # indentation level self.pre = False # displaying preformatted text - self.hprefix = '' # strip e.g. '25.5' from headings + self.hprefix = '' # prefix such as '25.5' to strip from headings self.nested_dl = False # if we're in a nested
      self.simplelist = False # simple list (no double spacing) - self.tocid = 1 # id for table of contents entries - self.contents = [] # map toc ids to section titles - self.data = '' # to record data within header tags for toc + self.toc = [] # pair headers with text indexes for toc + self.header = '' # text within header tags for toc def indent(self, amt=1): self.level += amt @@ -111,14 +110,10 @@ elif tag == 'a' and class_ == 'headerlink': self.hdrlink = True elif tag == 'h1': - self.text.mark_set('toc'+str(self.tocid), - self.text.index('end-1line')) self.tags = tag elif tag in ['h2', 'h3']: if self.show: - self.data = '' - self.text.mark_set('toc'+str(self.tocid), - self.text.index('end-1line')) + self.header = '' self.text.insert('end', '\n\n') self.tags = tag if self.show: @@ -128,10 +123,8 @@ "Handle endtags in help.html." if tag in ['h1', 'h2', 'h3']: self.indent(0) # clear tag, reset indent - if self.show and tag in ['h1', 'h2', 'h3']: - title = self.data - self.contents.append(('toc'+str(self.tocid), title)) - self.tocid += 1 + if self.show: + self.toc.append((self.header, self.text.index('insert'))) elif tag in ['span', 'em']: self.chartags = '' elif tag == 'a': @@ -151,7 +144,7 @@ if self.tags in ['h1', 'h2', 'h3'] and self.hprefix != '': if d[0:len(self.hprefix)] == self.hprefix: d = d[len(self.hprefix):].strip() - self.data += d + self.header += d self.text.insert('end', d, (self.tags, self.chartags)) @@ -205,19 +198,18 @@ self['background'] = text['background'] scroll = Scrollbar(self, command=text.yview) text['yscrollcommand'] = scroll.set + self.rowconfigure(0, weight=1) + self.columnconfigure(1, weight=1) # text + self.toc_menu(text).grid(column=0, row=0, sticky='nw') text.grid(column=1, row=0, sticky='nsew') scroll.grid(column=2, row=0, sticky='ns') - self.grid_columnconfigure(1, weight=1) - self.grid_rowconfigure(0, weight=1) - toc = self.contents_widget(text) - toc.grid(column=0, row=0, sticky='nw') - def contents_widget(self, text): - "Create table of contents." + def toc_menu(self, text): + "Create table of contents as drop-down menu." toc = Menubutton(self, text='TOC') drop = Menu(toc, tearoff=False) - for tag, lbl in text.parser.contents: - drop.add_command(label=lbl, command=lambda mark=tag:text.see(mark)) + for lbl, dex in text.parser.toc: + drop.add_command(label=lbl, command=lambda dex=dex:text.yview(dex)) toc['menu'] = drop return toc -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 27 10:40:49 2015 From: python-checkins at python.org (terry.reedy) Date: Sun, 27 Sep 2015 08:40:49 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzI1MTk4?= =?utf-8?q?=3A_When_using_the_Idle_dov_TOC_menu=2C_put_the_section_title_a?= =?utf-8?q?t_the?= Message-ID: <20150927084048.82644.54198@psf.io> https://hg.python.org/cpython/rev/40bab637295d changeset: 98305:40bab637295d branch: 2.7 parent: 98301:56bd41a77fd0 user: Terry Jan Reedy date: Sun Sep 27 04:40:02 2015 -0400 summary: Issue #25198: When using the Idle dov TOC menu, put the section title at the top of the window, unless it is too near the bottom to do do. files: Lib/idlelib/help.py | 36 ++++++++++++-------------------- 1 files changed, 14 insertions(+), 22 deletions(-) diff --git a/Lib/idlelib/help.py b/Lib/idlelib/help.py --- a/Lib/idlelib/help.py +++ b/Lib/idlelib/help.py @@ -55,12 +55,11 @@ self.hdrlink = False # used so we don't show header links self.level = 0 # indentation level self.pre = False # displaying preformatted text - self.hprefix = '' # strip e.g. '25.5' from headings + self.hprefix = '' # prefix such as '25.5' to strip from headings self.nested_dl = False # if we're in a nested
      self.simplelist = False # simple list (no double spacing) - self.tocid = 1 # id for table of contents entries - self.contents = [] # map toc ids to section titles - self.data = '' # to record data within header tags for toc + self.toc = [] # pair headers with text indexes for toc + self.header = '' # text within header tags for toc def indent(self, amt=1): self.level += amt @@ -111,14 +110,10 @@ elif tag == 'a' and class_ == 'headerlink': self.hdrlink = True elif tag == 'h1': - self.text.mark_set('toc'+str(self.tocid), - self.text.index('end-1line')) self.tags = tag elif tag in ['h2', 'h3']: if self.show: - self.data = '' - self.text.mark_set('toc'+str(self.tocid), - self.text.index('end-1line')) + self.header = '' self.text.insert('end', '\n\n') self.tags = tag if self.show: @@ -128,10 +123,8 @@ "Handle endtags in help.html." if tag in ['h1', 'h2', 'h3']: self.indent(0) # clear tag, reset indent - if self.show and tag in ['h1', 'h2', 'h3']: - title = self.data - self.contents.append(('toc'+str(self.tocid), title)) - self.tocid += 1 + if self.show: + self.toc.append((self.header, self.text.index('insert'))) elif tag in ['span', 'em']: self.chartags = '' elif tag == 'a': @@ -151,7 +144,7 @@ if self.tags in ['h1', 'h2', 'h3'] and self.hprefix != '': if d[0:len(self.hprefix)] == self.hprefix: d = d[len(self.hprefix):].strip() - self.data += d + self.header += d self.text.insert('end', d, (self.tags, self.chartags)) def handle_charref(self, name): @@ -208,19 +201,18 @@ self['background'] = text['background'] scroll = Scrollbar(self, command=text.yview) text['yscrollcommand'] = scroll.set + self.rowconfigure(0, weight=1) + self.columnconfigure(1, weight=1) # text + self.toc_menu(text).grid(column=0, row=0, sticky='nw') text.grid(column=1, row=0, sticky='nsew') scroll.grid(column=2, row=0, sticky='ns') - self.grid_columnconfigure(1, weight=1) - self.grid_rowconfigure(0, weight=1) - toc = self.contents_widget(text) - toc.grid(column=0, row=0, sticky='nw') - def contents_widget(self, text): - "Create table of contents." + def toc_menu(self, text): + "Create table of contents as drop-down menu." toc = Menubutton(self, text='TOC') drop = Menu(toc, tearoff=False) - for tag, lbl in text.parser.contents: - drop.add_command(label=lbl, command=lambda mark=tag:text.see(mark)) + for lbl, dex in text.parser.toc: + drop.add_command(label=lbl, command=lambda dex=dex:text.yview(dex)) toc['menu'] = drop return toc -- Repository URL: https://hg.python.org/cpython From solipsis at pitrou.net Sun Sep 27 10:42:41 2015 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Sun, 27 Sep 2015 08:42:41 +0000 Subject: [Python-checkins] Daily reference leaks (bb1a2944bcb6): sum=61482 Message-ID: <20150927084241.31175.7311@psf.io> results for bb1a2944bcb6 on branch "default" -------------------------------------------- test_capi leaked [5409, 5409, 5409] references, sum=16227 test_capi leaked [1420, 1422, 1422] memory blocks, sum=4264 test_collections leaked [0, 0, 6] references, sum=6 test_collections leaked [0, 0, 3] memory blocks, sum=3 test_functools leaked [0, 2, 2] memory blocks, sum=4 test_threading leaked [10818, 10818, 10818] references, sum=32454 test_threading leaked [2840, 2842, 2842] memory blocks, sum=8524 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/psf-users/antoine/refleaks/reflogfWxHwl', '--timeout', '7200'] From python-checkins at python.org Sun Sep 27 11:06:33 2015 From: python-checkins at python.org (berker.peksag) Date: Sun, 27 Sep 2015 09:06:33 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_Fix_broken_lin?= =?utf-8?q?k_in_Doc/extending/index=2Erst?= Message-ID: <20150927090633.81631.77733@psf.io> https://hg.python.org/cpython/rev/6c66f88e994b changeset: 98309:6c66f88e994b branch: 2.7 parent: 98305:40bab637295d user: Berker Peksag date: Sun Sep 27 12:06:26 2015 +0300 summary: Fix broken link in Doc/extending/index.rst files: Doc/extending/index.rst | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Doc/extending/index.rst b/Doc/extending/index.rst --- a/Doc/extending/index.rst +++ b/Doc/extending/index.rst @@ -26,8 +26,8 @@ This guide only covers the basic tools for creating extensions provided as part of this version of CPython. Third party tools may offer simpler alternatives. Refer to the `binary extensions section - `__ - in the Python Packaging User Guide for more information. + `__ in the Python + Packaging User Guide for more information. .. toctree:: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 27 11:08:41 2015 From: python-checkins at python.org (benjamin.peterson) Date: Sun, 27 Sep 2015 09:08:41 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_make_wikipedia?= =?utf-8?q?_link_https?= Message-ID: <20150927090841.94131.14217@psf.io> https://hg.python.org/cpython/rev/4741dfda7438 changeset: 98310:4741dfda7438 branch: 3.4 parent: 98302:30da62ea4d7c user: Benjamin Peterson date: Sun Sep 27 02:05:01 2015 -0700 summary: make wikipedia link https files: Doc/library/hashlib.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/library/hashlib.rst b/Doc/library/hashlib.rst --- a/Doc/library/hashlib.rst +++ b/Doc/library/hashlib.rst @@ -227,7 +227,7 @@ http://csrc.nist.gov/publications/fips/fips180-2/fips180-2.pdf The FIPS 180-2 publication on Secure Hash Algorithms. - http://en.wikipedia.org/wiki/Cryptographic_hash_function#Cryptographic_hash_algorithms + https://en.wikipedia.org/wiki/Cryptographic_hash_function#Cryptographic_hash_algorithms Wikipedia article with information on which algorithms have known issues and what that means regarding their use. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 27 11:08:42 2015 From: python-checkins at python.org (benjamin.peterson) Date: Sun, 27 Sep 2015 09:08:42 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogbWVyZ2UgMy41?= Message-ID: <20150927090841.82642.90199@psf.io> https://hg.python.org/cpython/rev/0de25a6e2468 changeset: 98312:0de25a6e2468 parent: 98304:fcdaa1dab846 parent: 98311:288fbf067df7 user: Benjamin Peterson date: Sun Sep 27 02:05:18 2015 -0700 summary: merge 3.5 files: Doc/library/hashlib.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/library/hashlib.rst b/Doc/library/hashlib.rst --- a/Doc/library/hashlib.rst +++ b/Doc/library/hashlib.rst @@ -227,7 +227,7 @@ http://csrc.nist.gov/publications/fips/fips180-2/fips180-2.pdf The FIPS 180-2 publication on Secure Hash Algorithms. - http://en.wikipedia.org/wiki/Cryptographic_hash_function#Cryptographic_hash_algorithms + https://en.wikipedia.org/wiki/Cryptographic_hash_function#Cryptographic_hash_algorithms Wikipedia article with information on which algorithms have known issues and what that means regarding their use. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 27 11:08:42 2015 From: python-checkins at python.org (benjamin.peterson) Date: Sun, 27 Sep 2015 09:08:42 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_make_wikipedia?= =?utf-8?q?_link_https?= Message-ID: <20150927090842.94133.35807@psf.io> https://hg.python.org/cpython/rev/5e83a76240f9 changeset: 98313:5e83a76240f9 branch: 2.7 parent: 98305:40bab637295d user: Benjamin Peterson date: Sun Sep 27 02:05:01 2015 -0700 summary: make wikipedia link https files: Doc/library/hashlib.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/library/hashlib.rst b/Doc/library/hashlib.rst --- a/Doc/library/hashlib.rst +++ b/Doc/library/hashlib.rst @@ -204,7 +204,7 @@ http://csrc.nist.gov/publications/fips/fips180-2/fips180-2.pdf The FIPS 180-2 publication on Secure Hash Algorithms. - http://en.wikipedia.org/wiki/Cryptographic_hash_function#Cryptographic_hash_algorithms + https://en.wikipedia.org/wiki/Cryptographic_hash_function#Cryptographic_hash_algorithms Wikipedia article with information on which algorithms have known issues and what that means regarding their use. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 27 11:08:41 2015 From: python-checkins at python.org (benjamin.peterson) Date: Sun, 27 Sep 2015 09:08:41 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_merge_3=2E4?= Message-ID: <20150927090841.82656.47282@psf.io> https://hg.python.org/cpython/rev/288fbf067df7 changeset: 98311:288fbf067df7 branch: 3.5 parent: 98303:bacb540ab6ba parent: 98310:4741dfda7438 user: Benjamin Peterson date: Sun Sep 27 02:05:12 2015 -0700 summary: merge 3.4 files: Doc/library/hashlib.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/library/hashlib.rst b/Doc/library/hashlib.rst --- a/Doc/library/hashlib.rst +++ b/Doc/library/hashlib.rst @@ -227,7 +227,7 @@ http://csrc.nist.gov/publications/fips/fips180-2/fips180-2.pdf The FIPS 180-2 publication on Secure Hash Algorithms. - http://en.wikipedia.org/wiki/Cryptographic_hash_function#Cryptographic_hash_algorithms + https://en.wikipedia.org/wiki/Cryptographic_hash_function#Cryptographic_hash_algorithms Wikipedia article with information on which algorithms have known issues and what that means regarding their use. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 27 11:08:43 2015 From: python-checkins at python.org (benjamin.peterson) Date: Sun, 27 Sep 2015 09:08:43 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_default_-=3E_default?= =?utf-8?q?=29=3A_merge_heads?= Message-ID: <20150927090842.82636.73433@psf.io> https://hg.python.org/cpython/rev/870e6c941713 changeset: 98316:870e6c941713 parent: 98312:0de25a6e2468 parent: 98308:1aed2444b10f user: Benjamin Peterson date: Sun Sep 27 02:07:35 2015 -0700 summary: merge heads files: Lib/idlelib/help.py | 36 ++++++++++++-------------------- 1 files changed, 14 insertions(+), 22 deletions(-) diff --git a/Lib/idlelib/help.py b/Lib/idlelib/help.py --- a/Lib/idlelib/help.py +++ b/Lib/idlelib/help.py @@ -55,12 +55,11 @@ self.hdrlink = False # used so we don't show header links self.level = 0 # indentation level self.pre = False # displaying preformatted text - self.hprefix = '' # strip e.g. '25.5' from headings + self.hprefix = '' # prefix such as '25.5' to strip from headings self.nested_dl = False # if we're in a nested
      self.simplelist = False # simple list (no double spacing) - self.tocid = 1 # id for table of contents entries - self.contents = [] # map toc ids to section titles - self.data = '' # to record data within header tags for toc + self.toc = [] # pair headers with text indexes for toc + self.header = '' # text within header tags for toc def indent(self, amt=1): self.level += amt @@ -111,14 +110,10 @@ elif tag == 'a' and class_ == 'headerlink': self.hdrlink = True elif tag == 'h1': - self.text.mark_set('toc'+str(self.tocid), - self.text.index('end-1line')) self.tags = tag elif tag in ['h2', 'h3']: if self.show: - self.data = '' - self.text.mark_set('toc'+str(self.tocid), - self.text.index('end-1line')) + self.header = '' self.text.insert('end', '\n\n') self.tags = tag if self.show: @@ -128,10 +123,8 @@ "Handle endtags in help.html." if tag in ['h1', 'h2', 'h3']: self.indent(0) # clear tag, reset indent - if self.show and tag in ['h1', 'h2', 'h3']: - title = self.data - self.contents.append(('toc'+str(self.tocid), title)) - self.tocid += 1 + if self.show: + self.toc.append((self.header, self.text.index('insert'))) elif tag in ['span', 'em']: self.chartags = '' elif tag == 'a': @@ -151,7 +144,7 @@ if self.tags in ['h1', 'h2', 'h3'] and self.hprefix != '': if d[0:len(self.hprefix)] == self.hprefix: d = d[len(self.hprefix):].strip() - self.data += d + self.header += d self.text.insert('end', d, (self.tags, self.chartags)) @@ -205,19 +198,18 @@ self['background'] = text['background'] scroll = Scrollbar(self, command=text.yview) text['yscrollcommand'] = scroll.set + self.rowconfigure(0, weight=1) + self.columnconfigure(1, weight=1) # text + self.toc_menu(text).grid(column=0, row=0, sticky='nw') text.grid(column=1, row=0, sticky='nsew') scroll.grid(column=2, row=0, sticky='ns') - self.grid_columnconfigure(1, weight=1) - self.grid_rowconfigure(0, weight=1) - toc = self.contents_widget(text) - toc.grid(column=0, row=0, sticky='nw') - def contents_widget(self, text): - "Create table of contents." + def toc_menu(self, text): + "Create table of contents as drop-down menu." toc = Menubutton(self, text='TOC') drop = Menu(toc, tearoff=False) - for tag, lbl in text.parser.contents: - drop.add_command(label=lbl, command=lambda mark=tag:text.see(mark)) + for lbl, dex in text.parser.toc: + drop.add_command(label=lbl, command=lambda dex=dex:text.yview(dex)) toc['menu'] = drop return toc -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 27 11:08:42 2015 From: python-checkins at python.org (benjamin.peterson) Date: Sun, 27 Sep 2015 09:08:42 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy41IC0+IDMuNSk6?= =?utf-8?q?_merge_heads?= Message-ID: <20150927090842.115149.17324@psf.io> https://hg.python.org/cpython/rev/5122dce8adb8 changeset: 98315:5122dce8adb8 branch: 3.5 parent: 98311:288fbf067df7 parent: 98307:a04a568f7af6 user: Benjamin Peterson date: Sun Sep 27 02:07:27 2015 -0700 summary: merge heads files: Lib/idlelib/help.py | 36 ++++++++++++-------------------- 1 files changed, 14 insertions(+), 22 deletions(-) diff --git a/Lib/idlelib/help.py b/Lib/idlelib/help.py --- a/Lib/idlelib/help.py +++ b/Lib/idlelib/help.py @@ -55,12 +55,11 @@ self.hdrlink = False # used so we don't show header links self.level = 0 # indentation level self.pre = False # displaying preformatted text - self.hprefix = '' # strip e.g. '25.5' from headings + self.hprefix = '' # prefix such as '25.5' to strip from headings self.nested_dl = False # if we're in a nested
      self.simplelist = False # simple list (no double spacing) - self.tocid = 1 # id for table of contents entries - self.contents = [] # map toc ids to section titles - self.data = '' # to record data within header tags for toc + self.toc = [] # pair headers with text indexes for toc + self.header = '' # text within header tags for toc def indent(self, amt=1): self.level += amt @@ -111,14 +110,10 @@ elif tag == 'a' and class_ == 'headerlink': self.hdrlink = True elif tag == 'h1': - self.text.mark_set('toc'+str(self.tocid), - self.text.index('end-1line')) self.tags = tag elif tag in ['h2', 'h3']: if self.show: - self.data = '' - self.text.mark_set('toc'+str(self.tocid), - self.text.index('end-1line')) + self.header = '' self.text.insert('end', '\n\n') self.tags = tag if self.show: @@ -128,10 +123,8 @@ "Handle endtags in help.html." if tag in ['h1', 'h2', 'h3']: self.indent(0) # clear tag, reset indent - if self.show and tag in ['h1', 'h2', 'h3']: - title = self.data - self.contents.append(('toc'+str(self.tocid), title)) - self.tocid += 1 + if self.show: + self.toc.append((self.header, self.text.index('insert'))) elif tag in ['span', 'em']: self.chartags = '' elif tag == 'a': @@ -151,7 +144,7 @@ if self.tags in ['h1', 'h2', 'h3'] and self.hprefix != '': if d[0:len(self.hprefix)] == self.hprefix: d = d[len(self.hprefix):].strip() - self.data += d + self.header += d self.text.insert('end', d, (self.tags, self.chartags)) @@ -205,19 +198,18 @@ self['background'] = text['background'] scroll = Scrollbar(self, command=text.yview) text['yscrollcommand'] = scroll.set + self.rowconfigure(0, weight=1) + self.columnconfigure(1, weight=1) # text + self.toc_menu(text).grid(column=0, row=0, sticky='nw') text.grid(column=1, row=0, sticky='nsew') scroll.grid(column=2, row=0, sticky='ns') - self.grid_columnconfigure(1, weight=1) - self.grid_rowconfigure(0, weight=1) - toc = self.contents_widget(text) - toc.grid(column=0, row=0, sticky='nw') - def contents_widget(self, text): - "Create table of contents." + def toc_menu(self, text): + "Create table of contents as drop-down menu." toc = Menubutton(self, text='TOC') drop = Menu(toc, tearoff=False) - for tag, lbl in text.parser.contents: - drop.add_command(label=lbl, command=lambda mark=tag:text.see(mark)) + for lbl, dex in text.parser.toc: + drop.add_command(label=lbl, command=lambda dex=dex:text.yview(dex)) toc['menu'] = drop return toc -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 27 11:08:42 2015 From: python-checkins at python.org (benjamin.peterson) Date: Sun, 27 Sep 2015 09:08:42 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNCk6?= =?utf-8?q?_merge_heads?= Message-ID: <20150927090842.98358.94905@psf.io> https://hg.python.org/cpython/rev/c65552c172e7 changeset: 98314:c65552c172e7 branch: 3.4 parent: 98310:4741dfda7438 parent: 98306:5b635a3ca3d5 user: Benjamin Peterson date: Sun Sep 27 02:07:19 2015 -0700 summary: merge heads files: Lib/idlelib/help.py | 36 ++++++++++++-------------------- 1 files changed, 14 insertions(+), 22 deletions(-) diff --git a/Lib/idlelib/help.py b/Lib/idlelib/help.py --- a/Lib/idlelib/help.py +++ b/Lib/idlelib/help.py @@ -55,12 +55,11 @@ self.hdrlink = False # used so we don't show header links self.level = 0 # indentation level self.pre = False # displaying preformatted text - self.hprefix = '' # strip e.g. '25.5' from headings + self.hprefix = '' # prefix such as '25.5' to strip from headings self.nested_dl = False # if we're in a nested
      self.simplelist = False # simple list (no double spacing) - self.tocid = 1 # id for table of contents entries - self.contents = [] # map toc ids to section titles - self.data = '' # to record data within header tags for toc + self.toc = [] # pair headers with text indexes for toc + self.header = '' # text within header tags for toc def indent(self, amt=1): self.level += amt @@ -111,14 +110,10 @@ elif tag == 'a' and class_ == 'headerlink': self.hdrlink = True elif tag == 'h1': - self.text.mark_set('toc'+str(self.tocid), - self.text.index('end-1line')) self.tags = tag elif tag in ['h2', 'h3']: if self.show: - self.data = '' - self.text.mark_set('toc'+str(self.tocid), - self.text.index('end-1line')) + self.header = '' self.text.insert('end', '\n\n') self.tags = tag if self.show: @@ -128,10 +123,8 @@ "Handle endtags in help.html." if tag in ['h1', 'h2', 'h3']: self.indent(0) # clear tag, reset indent - if self.show and tag in ['h1', 'h2', 'h3']: - title = self.data - self.contents.append(('toc'+str(self.tocid), title)) - self.tocid += 1 + if self.show: + self.toc.append((self.header, self.text.index('insert'))) elif tag in ['span', 'em']: self.chartags = '' elif tag == 'a': @@ -151,7 +144,7 @@ if self.tags in ['h1', 'h2', 'h3'] and self.hprefix != '': if d[0:len(self.hprefix)] == self.hprefix: d = d[len(self.hprefix):].strip() - self.data += d + self.header += d self.text.insert('end', d, (self.tags, self.chartags)) @@ -205,19 +198,18 @@ self['background'] = text['background'] scroll = Scrollbar(self, command=text.yview) text['yscrollcommand'] = scroll.set + self.rowconfigure(0, weight=1) + self.columnconfigure(1, weight=1) # text + self.toc_menu(text).grid(column=0, row=0, sticky='nw') text.grid(column=1, row=0, sticky='nsew') scroll.grid(column=2, row=0, sticky='ns') - self.grid_columnconfigure(1, weight=1) - self.grid_rowconfigure(0, weight=1) - toc = self.contents_widget(text) - toc.grid(column=0, row=0, sticky='nw') - def contents_widget(self, text): - "Create table of contents." + def toc_menu(self, text): + "Create table of contents as drop-down menu." toc = Menubutton(self, text='TOC') drop = Menu(toc, tearoff=False) - for tag, lbl in text.parser.contents: - drop.add_command(label=lbl, command=lambda mark=tag:text.see(mark)) + for lbl, dex in text.parser.toc: + drop.add_command(label=lbl, command=lambda dex=dex:text.yview(dex)) toc['menu'] = drop return toc -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 27 11:08:48 2015 From: python-checkins at python.org (benjamin.peterson) Date: Sun, 27 Sep 2015 09:08:48 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogbWVyZ2UgMy41?= Message-ID: <20150927090848.94109.85223@psf.io> https://hg.python.org/cpython/rev/320b83d96633 changeset: 98318:320b83d96633 parent: 98316:870e6c941713 parent: 98317:a48d8e331c38 user: Benjamin Peterson date: Sun Sep 27 02:07:50 2015 -0700 summary: merge 3.5 files: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 27 11:08:48 2015 From: python-checkins at python.org (benjamin.peterson) Date: Sun, 27 Sep 2015 09:08:48 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_merge_3=2E4?= Message-ID: <20150927090848.98370.75243@psf.io> https://hg.python.org/cpython/rev/a48d8e331c38 changeset: 98317:a48d8e331c38 branch: 3.5 parent: 98315:5122dce8adb8 parent: 98314:c65552c172e7 user: Benjamin Peterson date: Sun Sep 27 02:07:43 2015 -0700 summary: merge 3.4 files: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 27 11:08:49 2015 From: python-checkins at python.org (benjamin.peterson) Date: Sun, 27 Sep 2015 09:08:49 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMi43IC0+IDIuNyk6?= =?utf-8?q?_merge_heads?= Message-ID: <20150927090848.115545.72733@psf.io> https://hg.python.org/cpython/rev/3c3f3e25fe77 changeset: 98319:3c3f3e25fe77 branch: 2.7 parent: 98313:5e83a76240f9 parent: 98309:6c66f88e994b user: Benjamin Peterson date: Sun Sep 27 02:08:04 2015 -0700 summary: merge heads files: Doc/extending/index.rst | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Doc/extending/index.rst b/Doc/extending/index.rst --- a/Doc/extending/index.rst +++ b/Doc/extending/index.rst @@ -26,8 +26,8 @@ This guide only covers the basic tools for creating extensions provided as part of this version of CPython. Third party tools may offer simpler alternatives. Refer to the `binary extensions section - `__ - in the Python Packaging User Guide for more information. + `__ in the Python + Packaging User Guide for more information. .. toctree:: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 27 11:14:41 2015 From: python-checkins at python.org (benjamin.peterson) Date: Sun, 27 Sep 2015 09:14:41 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_fix_spacing?= Message-ID: <20150927091441.11712.7118@psf.io> https://hg.python.org/cpython/rev/0782798733ce changeset: 98320:0782798733ce branch: 3.4 parent: 98314:c65552c172e7 user: Benjamin Peterson date: Sun Sep 27 02:13:40 2015 -0700 summary: fix spacing files: Modules/_hashopenssl.c | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Modules/_hashopenssl.c b/Modules/_hashopenssl.c --- a/Modules/_hashopenssl.c +++ b/Modules/_hashopenssl.c @@ -508,8 +508,8 @@ HMAC_CTX_cleanup(&hctx_tpl); return 0; } - while(tkeylen) { - if(tkeylen > mdlen) + while (tkeylen) { + if (tkeylen > mdlen) cplen = mdlen; else cplen = tkeylen; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 27 11:14:41 2015 From: python-checkins at python.org (benjamin.peterson) Date: Sun, 27 Sep 2015 09:14:41 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_fix_spacing?= Message-ID: <20150927091441.98360.32148@psf.io> https://hg.python.org/cpython/rev/e7e54526cdfa changeset: 98321:e7e54526cdfa branch: 2.7 parent: 98319:3c3f3e25fe77 user: Benjamin Peterson date: Sun Sep 27 02:13:40 2015 -0700 summary: fix spacing files: Modules/_hashopenssl.c | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Modules/_hashopenssl.c b/Modules/_hashopenssl.c --- a/Modules/_hashopenssl.c +++ b/Modules/_hashopenssl.c @@ -532,8 +532,8 @@ HMAC_CTX_cleanup(&hctx_tpl); return 0; } - while(tkeylen) { - if(tkeylen > mdlen) + while (tkeylen) { + if (tkeylen > mdlen) cplen = mdlen; else cplen = tkeylen; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 27 11:14:41 2015 From: python-checkins at python.org (benjamin.peterson) Date: Sun, 27 Sep 2015 09:14:41 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogbWVyZ2UgMy41?= Message-ID: <20150927091441.31177.6862@psf.io> https://hg.python.org/cpython/rev/13d558e48592 changeset: 98323:13d558e48592 parent: 98318:320b83d96633 parent: 98322:ac11bf14618d user: Benjamin Peterson date: Sun Sep 27 02:14:29 2015 -0700 summary: merge 3.5 files: Modules/_hashopenssl.c | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Modules/_hashopenssl.c b/Modules/_hashopenssl.c --- a/Modules/_hashopenssl.c +++ b/Modules/_hashopenssl.c @@ -486,8 +486,8 @@ HMAC_CTX_cleanup(&hctx_tpl); return 0; } - while(tkeylen) { - if(tkeylen > mdlen) + while (tkeylen) { + if (tkeylen > mdlen) cplen = mdlen; else cplen = tkeylen; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 27 11:14:41 2015 From: python-checkins at python.org (benjamin.peterson) Date: Sun, 27 Sep 2015 09:14:41 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_merge_3=2E4?= Message-ID: <20150927091441.115474.67397@psf.io> https://hg.python.org/cpython/rev/ac11bf14618d changeset: 98322:ac11bf14618d branch: 3.5 parent: 98317:a48d8e331c38 parent: 98320:0782798733ce user: Benjamin Peterson date: Sun Sep 27 02:14:23 2015 -0700 summary: merge 3.4 files: Modules/_hashopenssl.c | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Modules/_hashopenssl.c b/Modules/_hashopenssl.c --- a/Modules/_hashopenssl.c +++ b/Modules/_hashopenssl.c @@ -486,8 +486,8 @@ HMAC_CTX_cleanup(&hctx_tpl); return 0; } - while(tkeylen) { - if(tkeylen > mdlen) + while (tkeylen) { + if (tkeylen > mdlen) cplen = mdlen; else cplen = tkeylen; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 27 11:40:12 2015 From: python-checkins at python.org (victor.stinner) Date: Sun, 27 Sep 2015 09:40:12 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2325220=3A_Fix_Lib/?= =?utf-8?q?test/autotest=2Epy?= Message-ID: <20150927094012.3652.44140@psf.io> https://hg.python.org/cpython/rev/892b4f029ac6 changeset: 98324:892b4f029ac6 user: Victor Stinner date: Sun Sep 27 11:19:08 2015 +0200 summary: Issue #25220: Fix Lib/test/autotest.py files: Lib/test/libregrtest/__init__.py | 2 +- Lib/test/regrtest.py | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Lib/test/libregrtest/__init__.py b/Lib/test/libregrtest/__init__.py --- a/Lib/test/libregrtest/__init__.py +++ b/Lib/test/libregrtest/__init__.py @@ -1,2 +1,2 @@ from test.libregrtest.cmdline import _parse_args, RESOURCE_NAMES -from test.libregrtest.main import main_in_temp_cwd +from test.libregrtest.main import main, main_in_temp_cwd diff --git a/Lib/test/regrtest.py b/Lib/test/regrtest.py --- a/Lib/test/regrtest.py +++ b/Lib/test/regrtest.py @@ -11,7 +11,7 @@ import os import sys -from test.libregrtest import main_in_temp_cwd +from test.libregrtest import main, main_in_temp_cwd if __name__ == '__main__': -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 27 12:44:53 2015 From: python-checkins at python.org (serhiy.storchaka) Date: Sun, 27 Sep 2015 10:44:53 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2325209=3A_rlcomple?= =?utf-8?q?te_now_can_add_a_space_or_a_colon_after_completed_keyword=2E?= Message-ID: <20150927104452.115507.7815@psf.io> https://hg.python.org/cpython/rev/f64ec4aac935 changeset: 98325:f64ec4aac935 user: Serhiy Storchaka date: Sun Sep 27 13:26:03 2015 +0300 summary: Issue #25209: rlcomplete now can add a space or a colon after completed keyword. files: Lib/rlcompleter.py | 6 ++++++ Lib/test/test_rlcompleter.py | 13 +++++++++---- Misc/NEWS | 2 ++ 3 files changed, 17 insertions(+), 4 deletions(-) diff --git a/Lib/rlcompleter.py b/Lib/rlcompleter.py --- a/Lib/rlcompleter.py +++ b/Lib/rlcompleter.py @@ -106,6 +106,12 @@ n = len(text) for word in keyword.kwlist: if word[:n] == text: + if word in {'finally', 'try'}: + word = word + ':' + elif word not in {'False', 'None', 'True', + 'break', 'continue', 'pass', + 'else'}: + word = word + ' ' matches.append(word) for nspace in [builtins.__dict__, self.namespace]: for word, val in nspace.items(): diff --git a/Lib/test/test_rlcompleter.py b/Lib/test/test_rlcompleter.py --- a/Lib/test/test_rlcompleter.py +++ b/Lib/test/test_rlcompleter.py @@ -67,10 +67,15 @@ def test_complete(self): completer = rlcompleter.Completer() self.assertEqual(completer.complete('', 0), '\t') - self.assertEqual(completer.complete('a', 0), 'and') - self.assertEqual(completer.complete('a', 1), 'as') - self.assertEqual(completer.complete('as', 2), 'assert') - self.assertEqual(completer.complete('an', 0), 'and') + self.assertEqual(completer.complete('a', 0), 'and ') + self.assertEqual(completer.complete('a', 1), 'as ') + self.assertEqual(completer.complete('as', 2), 'assert ') + self.assertEqual(completer.complete('an', 0), 'and ') + self.assertEqual(completer.complete('pa', 0), 'pass') + self.assertEqual(completer.complete('Fa', 0), 'False') + self.assertEqual(completer.complete('el', 0), 'elif ') + self.assertEqual(completer.complete('el', 1), 'else') + self.assertEqual(completer.complete('tr', 0), 'try:') if __name__ == '__main__': unittest.main() diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -27,6 +27,8 @@ Library ------- +- Issue #25209: rlcomplete now can add a space or a colon after completed keyword. + - Issue #22241: timezone.utc name is now plain 'UTC', not 'UTC-00:00'. - Issue #23517: fromtimestamp() and utcfromtimestamp() methods of -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 27 12:44:53 2015 From: python-checkins at python.org (serhiy.storchaka) Date: Sun, 27 Sep 2015 10:44:53 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2325011=3A_rlcomple?= =?utf-8?q?te_now_omits_private_and_special_attribute_names_unless?= Message-ID: <20150927104453.16599.23630@psf.io> https://hg.python.org/cpython/rev/4dbb315fe667 changeset: 98326:4dbb315fe667 user: Serhiy Storchaka date: Sun Sep 27 13:43:50 2015 +0300 summary: Issue #25011: rlcomplete now omits private and special attribute names unless the prefix starts with underscores. files: Doc/whatsnew/3.6.rst | 8 +++++ Lib/rlcompleter.py | 35 +++++++++++++++++------ Lib/test/test_rlcompleter.py | 15 ++++++++++ Misc/NEWS | 3 ++ 4 files changed, 51 insertions(+), 10 deletions(-) diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst --- a/Doc/whatsnew/3.6.rst +++ b/Doc/whatsnew/3.6.rst @@ -103,6 +103,14 @@ (Contributed by Joe Jevnik in :issue:`24379`.) +rlcomplete +---------- + +Private and special attribute names now are omitted unless the prefix starts +with underscores. A space or a colon can be added after completed keyword. +(Contributed by Serhiy Storchaka in :issue:`25011` and :issue:`25209`.) + + Optimizations ============= diff --git a/Lib/rlcompleter.py b/Lib/rlcompleter.py --- a/Lib/rlcompleter.py +++ b/Lib/rlcompleter.py @@ -142,20 +142,35 @@ return [] # get the content of the object, except __builtins__ - words = dir(thisobject) - if "__builtins__" in words: - words.remove("__builtins__") + words = set(dir(thisobject)) + words.discard("__builtins__") if hasattr(thisobject, '__class__'): - words.append('__class__') - words.extend(get_class_members(thisobject.__class__)) + words.add('__class__') + words.update(get_class_members(thisobject.__class__)) matches = [] n = len(attr) - for word in words: - if word[:n] == attr and hasattr(thisobject, word): - val = getattr(thisobject, word) - word = self._callable_postfix(val, "%s.%s" % (expr, word)) - matches.append(word) + if attr == '': + noprefix = '_' + elif attr == '_': + noprefix = '__' + else: + noprefix = None + while True: + for word in words: + if (word[:n] == attr and + not (noprefix and word[:n+1] == noprefix) and + hasattr(thisobject, word)): + val = getattr(thisobject, word) + word = self._callable_postfix(val, "%s.%s" % (expr, word)) + matches.append(word) + if matches or not noprefix: + break + if noprefix == '_': + noprefix = '__' + else: + noprefix = None + matches.sort() return matches def get_class_members(klass): diff --git a/Lib/test/test_rlcompleter.py b/Lib/test/test_rlcompleter.py --- a/Lib/test/test_rlcompleter.py +++ b/Lib/test/test_rlcompleter.py @@ -5,6 +5,7 @@ class CompleteMe: """ Trivial class used in testing rlcompleter.Completer. """ spam = 1 + _ham = 2 class TestRlcompleter(unittest.TestCase): @@ -51,11 +52,25 @@ ['str.{}('.format(x) for x in dir(str) if x.startswith('s')]) self.assertEqual(self.stdcompleter.attr_matches('tuple.foospamegg'), []) + expected = sorted({'None.%s%s' % (x, '(' if x != '__doc__' else '') + for x in dir(None)}) + self.assertEqual(self.stdcompleter.attr_matches('None.'), expected) + self.assertEqual(self.stdcompleter.attr_matches('None._'), expected) + self.assertEqual(self.stdcompleter.attr_matches('None.__'), expected) # test with a customized namespace self.assertEqual(self.completer.attr_matches('CompleteMe.sp'), ['CompleteMe.spam']) self.assertEqual(self.completer.attr_matches('Completeme.egg'), []) + self.assertEqual(self.completer.attr_matches('CompleteMe.'), + ['CompleteMe.mro(', 'CompleteMe.spam']) + self.assertEqual(self.completer.attr_matches('CompleteMe._'), + ['CompleteMe._ham']) + matches = self.completer.attr_matches('CompleteMe.__') + for x in matches: + self.assertTrue(x.startswith('CompleteMe.__'), x) + self.assertIn('CompleteMe.__name__', matches) + self.assertIn('CompleteMe.__new__(', matches) CompleteMe.me = CompleteMe self.assertEqual(self.completer.attr_matches('CompleteMe.me.me.sp'), diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -27,6 +27,9 @@ Library ------- +- Issue #25011: rlcomplete now omits private and special attribute names unless + the prefix starts with underscores. + - Issue #25209: rlcomplete now can add a space or a colon after completed keyword. - Issue #22241: timezone.utc name is now plain 'UTC', not 'UTC-00:00'. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 27 18:37:41 2015 From: python-checkins at python.org (r.david.murray) Date: Sun, 27 Sep 2015 16:37:41 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Merge=3A_Fix_English_phrasing=2E?= Message-ID: <20150927163741.31205.88600@psf.io> https://hg.python.org/cpython/rev/81869f8608ce changeset: 98328:81869f8608ce branch: 3.5 parent: 98322:ac11bf14618d parent: 98327:c5ee47b28a50 user: R David Murray date: Sun Sep 27 12:36:50 2015 -0400 summary: Merge: Fix English phrasing. files: Doc/library/asyncio-stream.rst | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Doc/library/asyncio-stream.rst b/Doc/library/asyncio-stream.rst --- a/Doc/library/asyncio-stream.rst +++ b/Doc/library/asyncio-stream.rst @@ -228,8 +228,8 @@ (This is a helper class instead of making :class:`StreamReader` itself a :class:`Protocol` subclass, because the :class:`StreamReader` has other - potential uses, and to prevent the user of the :class:`StreamReader` to - accidentally call inappropriate methods of the protocol.) + potential uses, and to prevent the user of the :class:`StreamReader` from + accidentally calling inappropriate methods of the protocol.) IncompleteReadError -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 27 18:37:42 2015 From: python-checkins at python.org (r.david.murray) Date: Sun, 27 Sep 2015 16:37:42 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_Fix_English_ph?= =?utf-8?q?rasing=2E?= Message-ID: <20150927163741.9951.57431@psf.io> https://hg.python.org/cpython/rev/c5ee47b28a50 changeset: 98327:c5ee47b28a50 branch: 3.4 parent: 98320:0782798733ce user: R David Murray date: Sun Sep 27 12:36:19 2015 -0400 summary: Fix English phrasing. files: Doc/library/asyncio-stream.rst | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Doc/library/asyncio-stream.rst b/Doc/library/asyncio-stream.rst --- a/Doc/library/asyncio-stream.rst +++ b/Doc/library/asyncio-stream.rst @@ -228,8 +228,8 @@ (This is a helper class instead of making :class:`StreamReader` itself a :class:`Protocol` subclass, because the :class:`StreamReader` has other - potential uses, and to prevent the user of the :class:`StreamReader` to - accidentally call inappropriate methods of the protocol.) + potential uses, and to prevent the user of the :class:`StreamReader` from + accidentally calling inappropriate methods of the protocol.) IncompleteReadError -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 27 18:37:41 2015 From: python-checkins at python.org (r.david.murray) Date: Sun, 27 Sep 2015 16:37:41 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge=3A_Fix_English_phrasing=2E?= Message-ID: <20150927163741.9947.58032@psf.io> https://hg.python.org/cpython/rev/af46be655a2e changeset: 98329:af46be655a2e parent: 98326:4dbb315fe667 parent: 98328:81869f8608ce user: R David Murray date: Sun Sep 27 12:37:20 2015 -0400 summary: Merge: Fix English phrasing. files: Doc/library/asyncio-stream.rst | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Doc/library/asyncio-stream.rst b/Doc/library/asyncio-stream.rst --- a/Doc/library/asyncio-stream.rst +++ b/Doc/library/asyncio-stream.rst @@ -228,8 +228,8 @@ (This is a helper class instead of making :class:`StreamReader` itself a :class:`Protocol` subclass, because the :class:`StreamReader` has other - potential uses, and to prevent the user of the :class:`StreamReader` to - accidentally call inappropriate methods of the protocol.) + potential uses, and to prevent the user of the :class:`StreamReader` from + accidentally calling inappropriate methods of the protocol.) IncompleteReadError -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 27 21:39:51 2015 From: python-checkins at python.org (serhiy.storchaka) Date: Sun, 27 Sep 2015 19:39:51 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzI1MjAz?= =?utf-8?q?=3A_Failed_readline=2Eset=5Fcompleter=5Fdelims=28=29_no_longer_?= =?utf-8?q?left_the?= Message-ID: <20150927193951.31187.84414@psf.io> https://hg.python.org/cpython/rev/d867ca794bdb changeset: 98331:d867ca794bdb branch: 2.7 parent: 98321:e7e54526cdfa user: Serhiy Storchaka date: Sun Sep 27 22:34:59 2015 +0300 summary: Issue #25203: Failed readline.set_completer_delims() no longer left the module in inconsistent state. files: Misc/NEWS | 3 +++ Modules/readline.c | 9 +++++---- 2 files changed, 8 insertions(+), 4 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -37,6 +37,9 @@ Library ------- +- Issue #25203: Failed readline.set_completer_delims() no longer left the + module in inconsistent state. + - Issue #19143: platform module now reads Windows version from kernel32.dll to avoid compatibility shims. diff --git a/Modules/readline.c b/Modules/readline.c --- a/Modules/readline.c +++ b/Modules/readline.c @@ -355,10 +355,11 @@ /* Keep a reference to the allocated memory in the module state in case some other module modifies rl_completer_word_break_characters (see issue #17289). */ - free(completer_word_break_characters); - completer_word_break_characters = strdup(break_chars); - if (completer_word_break_characters) { - rl_completer_word_break_characters = completer_word_break_characters; + break_chars = strdup(break_chars); + if (break_chars) { + free(completer_word_break_characters); + completer_word_break_characters = break_chars; + rl_completer_word_break_characters = break_chars; Py_RETURN_NONE; } else -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 27 21:39:51 2015 From: python-checkins at python.org (serhiy.storchaka) Date: Sun, 27 Sep 2015 19:39:51 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzI1MjAz?= =?utf-8?q?=3A_Failed_readline=2Eset=5Fcompleter=5Fdelims=28=29_no_longer_?= =?utf-8?q?left_the?= Message-ID: <20150927193950.98376.22533@psf.io> https://hg.python.org/cpython/rev/46aaff5e8945 changeset: 98330:46aaff5e8945 branch: 3.4 parent: 98327:c5ee47b28a50 user: Serhiy Storchaka date: Sun Sep 27 22:34:59 2015 +0300 summary: Issue #25203: Failed readline.set_completer_delims() no longer left the module in inconsistent state. files: Misc/NEWS | 3 +++ Modules/readline.c | 9 +++++---- 2 files changed, 8 insertions(+), 4 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -81,6 +81,9 @@ Library ------- +- Issue #25203: Failed readline.set_completer_delims() no longer left the + module in inconsistent state. + - Prevent overflow in _Unpickler_Read. - Issue #25047: The XML encoding declaration written by Element Tree now diff --git a/Modules/readline.c b/Modules/readline.c --- a/Modules/readline.c +++ b/Modules/readline.c @@ -427,10 +427,11 @@ /* Keep a reference to the allocated memory in the module state in case some other module modifies rl_completer_word_break_characters (see issue #17289). */ - free(completer_word_break_characters); - completer_word_break_characters = strdup(break_chars); - if (completer_word_break_characters) { - rl_completer_word_break_characters = completer_word_break_characters; + break_chars = strdup(break_chars); + if (break_chars) { + free(completer_word_break_characters); + completer_word_break_characters = break_chars; + rl_completer_word_break_characters = break_chars; Py_RETURN_NONE; } else -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 27 21:39:51 2015 From: python-checkins at python.org (serhiy.storchaka) Date: Sun, 27 Sep 2015 19:39:51 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Issue_=2325203=3A_Failed_readline=2Eset=5Fcompleter=5Fdelims?= =?utf-8?q?=28=29_no_longer_left_the?= Message-ID: <20150927193951.81627.16707@psf.io> https://hg.python.org/cpython/rev/0d3b64bbc82c changeset: 98332:0d3b64bbc82c branch: 3.5 parent: 98328:81869f8608ce parent: 98330:46aaff5e8945 user: Serhiy Storchaka date: Sun Sep 27 22:38:01 2015 +0300 summary: Issue #25203: Failed readline.set_completer_delims() no longer left the module in inconsistent state. files: Misc/NEWS | 3 +++ Modules/readline.c | 9 +++++---- 2 files changed, 8 insertions(+), 4 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -21,6 +21,9 @@ Library ------- +- Issue #25203: Failed readline.set_completer_delims() no longer left the + module in inconsistent state. + - Issue #23329: Allow the ssl module to be built with older versions of LibreSSL. diff --git a/Modules/readline.c b/Modules/readline.c --- a/Modules/readline.c +++ b/Modules/readline.c @@ -464,10 +464,11 @@ /* Keep a reference to the allocated memory in the module state in case some other module modifies rl_completer_word_break_characters (see issue #17289). */ - free(completer_word_break_characters); - completer_word_break_characters = strdup(break_chars); - if (completer_word_break_characters) { - rl_completer_word_break_characters = completer_word_break_characters; + break_chars = strdup(break_chars); + if (break_chars) { + free(completer_word_break_characters); + completer_word_break_characters = break_chars; + rl_completer_word_break_characters = break_chars; Py_RETURN_NONE; } else -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 27 21:39:52 2015 From: python-checkins at python.org (serhiy.storchaka) Date: Sun, 27 Sep 2015 19:39:52 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2325203=3A_Failed_readline=2Eset=5Fcompleter=5Fde?= =?utf-8?q?lims=28=29_no_longer_left_the?= Message-ID: <20150927193951.16583.75062@psf.io> https://hg.python.org/cpython/rev/48943533965e changeset: 98333:48943533965e parent: 98329:af46be655a2e parent: 98332:0d3b64bbc82c user: Serhiy Storchaka date: Sun Sep 27 22:38:33 2015 +0300 summary: Issue #25203: Failed readline.set_completer_delims() no longer left the module in inconsistent state. files: Misc/NEWS | 3 +++ Modules/readline.c | 9 +++++---- 2 files changed, 8 insertions(+), 4 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -146,6 +146,9 @@ Library ------- +- Issue #25203: Failed readline.set_completer_delims() no longer left the + module in inconsistent state. + - Issue #23329: Allow the ssl module to be built with older versions of LibreSSL. diff --git a/Modules/readline.c b/Modules/readline.c --- a/Modules/readline.c +++ b/Modules/readline.c @@ -464,10 +464,11 @@ /* Keep a reference to the allocated memory in the module state in case some other module modifies rl_completer_word_break_characters (see issue #17289). */ - free(completer_word_break_characters); - completer_word_break_characters = strdup(break_chars); - if (completer_word_break_characters) { - rl_completer_word_break_characters = completer_word_break_characters; + break_chars = strdup(break_chars); + if (break_chars) { + free(completer_word_break_characters); + completer_word_break_characters = break_chars; + rl_completer_word_break_characters = break_chars; Py_RETURN_NONE; } else -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 28 04:35:32 2015 From: python-checkins at python.org (alexander.belopolsky) Date: Mon, 28 Sep 2015 02:35:32 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_Closes_issue_?= =?utf-8?q?=2323600=3A_Wrong_results_from_tzinfo=2Efromutc=28=29=2E?= Message-ID: <20150928023531.82664.57944@psf.io> https://hg.python.org/cpython/rev/ff68705c56a8 changeset: 98334:ff68705c56a8 branch: 3.4 parent: 98327:c5ee47b28a50 user: Alexander Belopolsky date: Sun Sep 27 21:41:55 2015 -0400 summary: Closes issue #23600: Wrong results from tzinfo.fromutc(). files: Lib/test/datetimetester.py | 23 +++++++++++++++++++++++ Misc/NEWS | 3 +++ Modules/_datetimemodule.c | 2 +- 3 files changed, 27 insertions(+), 1 deletions(-) diff --git a/Lib/test/datetimetester.py b/Lib/test/datetimetester.py --- a/Lib/test/datetimetester.py +++ b/Lib/test/datetimetester.py @@ -180,6 +180,29 @@ self.assertEqual(derived.utcoffset(None), offset) self.assertEqual(derived.tzname(None), oname) + def test_issue23600(self): + DSTDIFF = DSTOFFSET = timedelta(hours=1) + + class UKSummerTime(tzinfo): + """Simple time zone which pretends to always be in summer time, since + that's what shows the failure. + """ + + def utcoffset(self, dt): + return DSTOFFSET + + def dst(self, dt): + return DSTDIFF + + def tzname(self, dt): + return 'UKSummerTime' + + tz = UKSummerTime() + u = datetime(2014, 4, 26, 12, 1, tzinfo=tz) + t = tz.fromutc(u) + self.assertEqual(t - t.utcoffset(), u) + + class TestTimeZone(unittest.TestCase): def setUp(self): diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -81,6 +81,9 @@ Library ------- +- Issue #23600: Default implementation of tzinfo.fromutc() was returning + wrong results in some cases. + - Prevent overflow in _Unpickler_Read. - Issue #25047: The XML encoding declaration written by Element Tree now diff --git a/Modules/_datetimemodule.c b/Modules/_datetimemodule.c --- a/Modules/_datetimemodule.c +++ b/Modules/_datetimemodule.c @@ -3040,7 +3040,7 @@ goto Fail; if (dst == Py_None) goto Inconsistent; - if (delta_bool(delta) != 0) { + if (delta_bool((PyDateTime_Delta *)dst) != 0) { PyObject *temp = result; result = add_datetime_timedelta((PyDateTime_DateTime *)result, (PyDateTime_Delta *)dst, 1); -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 28 04:35:32 2015 From: python-checkins at python.org (alexander.belopolsky) Date: Mon, 28 Sep 2015 02:35:32 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Closes_issue_=2323600=3A_Wrong_results_from_tzinfo=2Efro?= =?utf-8?b?bXV0YygpLg==?= Message-ID: <20150928023532.94107.77473@psf.io> https://hg.python.org/cpython/rev/4879c10ce982 changeset: 98336:4879c10ce982 parent: 98329:af46be655a2e parent: 98335:c706c062f545 user: Alexander Belopolsky date: Sun Sep 27 21:56:53 2015 -0400 summary: Closes issue #23600: Wrong results from tzinfo.fromutc(). files: Lib/test/datetimetester.py | 23 +++++++++++++++++++++++ Misc/NEWS | 3 +++ Modules/_datetimemodule.c | 2 +- 3 files changed, 27 insertions(+), 1 deletions(-) diff --git a/Lib/test/datetimetester.py b/Lib/test/datetimetester.py --- a/Lib/test/datetimetester.py +++ b/Lib/test/datetimetester.py @@ -192,6 +192,29 @@ self.assertEqual(derived.utcoffset(None), offset) self.assertEqual(derived.tzname(None), oname) + def test_issue23600(self): + DSTDIFF = DSTOFFSET = timedelta(hours=1) + + class UKSummerTime(tzinfo): + """Simple time zone which pretends to always be in summer time, since + that's what shows the failure. + """ + + def utcoffset(self, dt): + return DSTOFFSET + + def dst(self, dt): + return DSTDIFF + + def tzname(self, dt): + return 'UKSummerTime' + + tz = UKSummerTime() + u = datetime(2014, 4, 26, 12, 1, tzinfo=tz) + t = tz.fromutc(u) + self.assertEqual(t - t.utcoffset(), u) + + class TestTimeZone(unittest.TestCase): def setUp(self): diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -146,6 +146,9 @@ Library ------- +- Issue #23600: Default implementation of tzinfo.fromutc() was returning + wrong results in some cases. + - Issue #23329: Allow the ssl module to be built with older versions of LibreSSL. diff --git a/Modules/_datetimemodule.c b/Modules/_datetimemodule.c --- a/Modules/_datetimemodule.c +++ b/Modules/_datetimemodule.c @@ -3046,7 +3046,7 @@ goto Fail; if (dst == Py_None) goto Inconsistent; - if (delta_bool(delta) != 0) { + if (delta_bool((PyDateTime_Delta *)dst) != 0) { PyObject *temp = result; result = add_datetime_timedelta((PyDateTime_DateTime *)result, (PyDateTime_Delta *)dst, 1); -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 28 04:35:32 2015 From: python-checkins at python.org (alexander.belopolsky) Date: Mon, 28 Sep 2015 02:35:32 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_default_-=3E_default?= =?utf-8?q?=29=3A_merge?= Message-ID: <20150928023532.94137.63390@psf.io> https://hg.python.org/cpython/rev/0d44613b014f changeset: 98337:0d44613b014f parent: 98336:4879c10ce982 parent: 98333:48943533965e user: Alexander Belopolsky date: Sun Sep 27 22:13:28 2015 -0400 summary: merge files: Misc/NEWS | 6 ++++++ Modules/readline.c | 9 +++++---- 2 files changed, 11 insertions(+), 4 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -27,6 +27,12 @@ Library ------- +- Issue #23600: Default implementation of tzinfo.fromutc() was returning + wrong results in some cases. + +- Issue #25203: Failed readline.set_completer_delims() no longer left the + module in inconsistent state. + - Issue #25011: rlcomplete now omits private and special attribute names unless the prefix starts with underscores. diff --git a/Modules/readline.c b/Modules/readline.c --- a/Modules/readline.c +++ b/Modules/readline.c @@ -464,10 +464,11 @@ /* Keep a reference to the allocated memory in the module state in case some other module modifies rl_completer_word_break_characters (see issue #17289). */ - free(completer_word_break_characters); - completer_word_break_characters = strdup(break_chars); - if (completer_word_break_characters) { - rl_completer_word_break_characters = completer_word_break_characters; + break_chars = strdup(break_chars); + if (break_chars) { + free(completer_word_break_characters); + completer_word_break_characters = break_chars; + rl_completer_word_break_characters = break_chars; Py_RETURN_NONE; } else -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 28 04:35:32 2015 From: python-checkins at python.org (alexander.belopolsky) Date: Mon, 28 Sep 2015 02:35:32 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_Closes_issue_?= =?utf-8?q?=2323600=3A_Wrong_results_from_tzinfo=2Efromutc=28=29=2E?= Message-ID: <20150928023532.3658.58630@psf.io> https://hg.python.org/cpython/rev/848d665bc312 changeset: 98339:848d665bc312 branch: 3.5 parent: 98332:0d3b64bbc82c user: Alexander Belopolsky date: Sun Sep 27 22:32:15 2015 -0400 summary: Closes issue #23600: Wrong results from tzinfo.fromutc(). files: Lib/test/datetimetester.py | 23 +++++++++++++++++++++++ Misc/NEWS | 3 +++ Modules/_datetimemodule.c | 2 +- 3 files changed, 27 insertions(+), 1 deletions(-) diff --git a/Lib/test/datetimetester.py b/Lib/test/datetimetester.py --- a/Lib/test/datetimetester.py +++ b/Lib/test/datetimetester.py @@ -192,6 +192,29 @@ self.assertEqual(derived.utcoffset(None), offset) self.assertEqual(derived.tzname(None), oname) + def test_issue23600(self): + DSTDIFF = DSTOFFSET = timedelta(hours=1) + + class UKSummerTime(tzinfo): + """Simple time zone which pretends to always be in summer time, since + that's what shows the failure. + """ + + def utcoffset(self, dt): + return DSTOFFSET + + def dst(self, dt): + return DSTDIFF + + def tzname(self, dt): + return 'UKSummerTime' + + tz = UKSummerTime() + u = datetime(2014, 4, 26, 12, 1, tzinfo=tz) + t = tz.fromutc(u) + self.assertEqual(t - t.utcoffset(), u) + + class TestTimeZone(unittest.TestCase): def setUp(self): diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -24,6 +24,9 @@ - Issue #25203: Failed readline.set_completer_delims() no longer left the module in inconsistent state. +- Issue #23600: Default implementation of tzinfo.fromutc() was returning + wrong results in some cases. + - Issue #23329: Allow the ssl module to be built with older versions of LibreSSL. diff --git a/Modules/_datetimemodule.c b/Modules/_datetimemodule.c --- a/Modules/_datetimemodule.c +++ b/Modules/_datetimemodule.c @@ -3046,7 +3046,7 @@ goto Fail; if (dst == Py_None) goto Inconsistent; - if (delta_bool(delta) != 0) { + if (delta_bool((PyDateTime_Delta *)dst) != 0) { PyObject *temp = result; result = add_datetime_timedelta((PyDateTime_DateTime *)result, (PyDateTime_Delta *)dst, 1); -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 28 04:35:31 2015 From: python-checkins at python.org (alexander.belopolsky) Date: Mon, 28 Sep 2015 02:35:31 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Closes_issue_=2323600=3A_Wrong_results_from_tzinfo=2Efromutc?= =?utf-8?b?KCku?= Message-ID: <20150928023531.9949.7427@psf.io> https://hg.python.org/cpython/rev/c706c062f545 changeset: 98335:c706c062f545 branch: 3.5 parent: 98328:81869f8608ce parent: 98334:ff68705c56a8 user: Alexander Belopolsky date: Sun Sep 27 21:56:09 2015 -0400 summary: Closes issue #23600: Wrong results from tzinfo.fromutc(). files: Lib/test/datetimetester.py | 23 +++++++++++++++++++++++ Misc/NEWS | 3 +++ Modules/_datetimemodule.c | 2 +- 3 files changed, 27 insertions(+), 1 deletions(-) diff --git a/Lib/test/datetimetester.py b/Lib/test/datetimetester.py --- a/Lib/test/datetimetester.py +++ b/Lib/test/datetimetester.py @@ -192,6 +192,29 @@ self.assertEqual(derived.utcoffset(None), offset) self.assertEqual(derived.tzname(None), oname) + def test_issue23600(self): + DSTDIFF = DSTOFFSET = timedelta(hours=1) + + class UKSummerTime(tzinfo): + """Simple time zone which pretends to always be in summer time, since + that's what shows the failure. + """ + + def utcoffset(self, dt): + return DSTOFFSET + + def dst(self, dt): + return DSTDIFF + + def tzname(self, dt): + return 'UKSummerTime' + + tz = UKSummerTime() + u = datetime(2014, 4, 26, 12, 1, tzinfo=tz) + t = tz.fromutc(u) + self.assertEqual(t - t.utcoffset(), u) + + class TestTimeZone(unittest.TestCase): def setUp(self): diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -21,6 +21,9 @@ Library ------- +- Issue #23600: Default implementation of tzinfo.fromutc() was returning + wrong results in some cases. + - Issue #23329: Allow the ssl module to be built with older versions of LibreSSL. diff --git a/Modules/_datetimemodule.c b/Modules/_datetimemodule.c --- a/Modules/_datetimemodule.c +++ b/Modules/_datetimemodule.c @@ -3046,7 +3046,7 @@ goto Fail; if (dst == Py_None) goto Inconsistent; - if (delta_bool(delta) != 0) { + if (delta_bool((PyDateTime_Delta *)dst) != 0) { PyObject *temp = result; result = add_datetime_timedelta((PyDateTime_DateTime *)result, (PyDateTime_Delta *)dst, 1); -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 28 04:35:32 2015 From: python-checkins at python.org (alexander.belopolsky) Date: Mon, 28 Sep 2015 02:35:32 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_Closes_issue_?= =?utf-8?q?=2323600=3A_Wrong_results_from_tzinfo=2Efromutc=28=29=2E?= Message-ID: <20150928023532.3658.90204@psf.io> https://hg.python.org/cpython/rev/cbcf82f92c25 changeset: 98338:cbcf82f92c25 branch: 3.4 parent: 98330:46aaff5e8945 user: Alexander Belopolsky date: Sun Sep 27 22:31:45 2015 -0400 summary: Closes issue #23600: Wrong results from tzinfo.fromutc(). files: Lib/test/datetimetester.py | 23 +++++++++++++++++++++++ Misc/NEWS | 3 +++ Modules/_datetimemodule.c | 2 +- 3 files changed, 27 insertions(+), 1 deletions(-) diff --git a/Lib/test/datetimetester.py b/Lib/test/datetimetester.py --- a/Lib/test/datetimetester.py +++ b/Lib/test/datetimetester.py @@ -180,6 +180,29 @@ self.assertEqual(derived.utcoffset(None), offset) self.assertEqual(derived.tzname(None), oname) + def test_issue23600(self): + DSTDIFF = DSTOFFSET = timedelta(hours=1) + + class UKSummerTime(tzinfo): + """Simple time zone which pretends to always be in summer time, since + that's what shows the failure. + """ + + def utcoffset(self, dt): + return DSTOFFSET + + def dst(self, dt): + return DSTDIFF + + def tzname(self, dt): + return 'UKSummerTime' + + tz = UKSummerTime() + u = datetime(2014, 4, 26, 12, 1, tzinfo=tz) + t = tz.fromutc(u) + self.assertEqual(t - t.utcoffset(), u) + + class TestTimeZone(unittest.TestCase): def setUp(self): diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -81,6 +81,9 @@ Library ------- +- Issue #23600: Default implementation of tzinfo.fromutc() was returning + wrong results in some cases. + - Issue #25203: Failed readline.set_completer_delims() no longer left the module in inconsistent state. diff --git a/Modules/_datetimemodule.c b/Modules/_datetimemodule.c --- a/Modules/_datetimemodule.c +++ b/Modules/_datetimemodule.c @@ -3040,7 +3040,7 @@ goto Fail; if (dst == Py_None) goto Inconsistent; - if (delta_bool(delta) != 0) { + if (delta_bool((PyDateTime_Delta *)dst) != 0) { PyObject *temp = result; result = add_datetime_timedelta((PyDateTime_DateTime *)result, (PyDateTime_Delta *)dst, 1); -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 28 04:35:33 2015 From: python-checkins at python.org (alexander.belopolsky) Date: Mon, 28 Sep 2015 02:35:33 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy41IC0+IDMuNSk6?= =?utf-8?q?_merge?= Message-ID: <20150928023533.94119.44667@psf.io> https://hg.python.org/cpython/rev/439fc6c96f65 changeset: 98341:439fc6c96f65 branch: 3.5 parent: 98339:848d665bc312 parent: 98335:c706c062f545 user: Alexander Belopolsky date: Sun Sep 27 22:34:59 2015 -0400 summary: merge files: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 28 04:35:32 2015 From: python-checkins at python.org (alexander.belopolsky) Date: Mon, 28 Sep 2015 02:35:32 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNCk6?= =?utf-8?q?_merge?= Message-ID: <20150928023532.82644.65634@psf.io> https://hg.python.org/cpython/rev/1ae0e02b541d changeset: 98340:1ae0e02b541d branch: 3.4 parent: 98338:cbcf82f92c25 parent: 98334:ff68705c56a8 user: Alexander Belopolsky date: Sun Sep 27 22:34:07 2015 -0400 summary: merge files: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 28 04:51:50 2015 From: python-checkins at python.org (terry.reedy) Date: Mon, 28 Sep 2015 02:51:50 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_Backed_out_cha?= =?utf-8?q?ngeset=3A_70c01dd35100?= Message-ID: <20150928025150.11688.3222@psf.io> https://hg.python.org/cpython/rev/719dcc47b259 changeset: 98344:719dcc47b259 branch: 3.4 user: Terry Jan Reedy date: Sun Sep 27 22:50:54 2015 -0400 summary: Backed out changeset: 70c01dd35100 files: Lib/idlelib/EditorWindow.py | 31 ++++++++++++++++++++++++- 1 files changed, 30 insertions(+), 1 deletions(-) diff --git a/Lib/idlelib/EditorWindow.py b/Lib/idlelib/EditorWindow.py --- a/Lib/idlelib/EditorWindow.py +++ b/Lib/idlelib/EditorWindow.py @@ -317,6 +317,36 @@ self.askinteger = tkSimpleDialog.askinteger self.showerror = tkMessageBox.showerror + self._highlight_workaround() # Fix selection tags on Windows + + def _highlight_workaround(self): + # On Windows, Tk removes painting of the selection + # tags which is different behavior than on Linux and Mac. + # See issue14146 for more information. + if not sys.platform.startswith('win'): + return + + text = self.text + text.event_add("<>", "") + text.event_add("<>", "") + def highlight_fix(focus): + sel_range = text.tag_ranges("sel") + if sel_range: + if focus == 'out': + HILITE_CONFIG = idleConf.GetHighlight( + idleConf.CurrentTheme(), 'hilite') + text.tag_config("sel_fix", HILITE_CONFIG) + text.tag_raise("sel_fix") + text.tag_add("sel_fix", *sel_range) + elif focus == 'in': + text.tag_remove("sel_fix", "1.0", "end") + + text.bind("<>", + lambda ev: highlight_fix("out")) + text.bind("<>", + lambda ev: highlight_fix("in")) + + def _filename_to_unicode(self, filename): """Return filename as BMP unicode so diplayable in Tk.""" # Decode bytes to unicode. @@ -755,7 +785,6 @@ insertbackground=cursor_color, selectforeground=select_colors['foreground'], selectbackground=select_colors['background'], - inactiveselectbackground=select_colors['background'], ) IDENTCHARS = string.ascii_letters + string.digits + "_" -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 28 04:51:50 2015 From: python-checkins at python.org (terry.reedy) Date: Mon, 28 Sep 2015 02:51:50 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzI0OTcy?= =?utf-8?q?=3A_Inactive_selection_background_now_matches_active_selection?= Message-ID: <20150928025150.115149.87292@psf.io> https://hg.python.org/cpython/rev/70c01dd35100 changeset: 98343:70c01dd35100 branch: 3.4 parent: 98340:1ae0e02b541d user: Terry Jan Reedy date: Sun Sep 27 22:46:17 2015 -0400 summary: Issue #24972: Inactive selection background now matches active selection background, as selected by user, on all systems. This also fixes a problem with found items not highlighted on Windows. Initial patch by Mark Roseman. Fix replaces workaround with obscure but proper configuration option. files: Lib/idlelib/EditorWindow.py | 31 +------------------------ 1 files changed, 1 insertions(+), 30 deletions(-) diff --git a/Lib/idlelib/EditorWindow.py b/Lib/idlelib/EditorWindow.py --- a/Lib/idlelib/EditorWindow.py +++ b/Lib/idlelib/EditorWindow.py @@ -317,36 +317,6 @@ self.askinteger = tkSimpleDialog.askinteger self.showerror = tkMessageBox.showerror - self._highlight_workaround() # Fix selection tags on Windows - - def _highlight_workaround(self): - # On Windows, Tk removes painting of the selection - # tags which is different behavior than on Linux and Mac. - # See issue14146 for more information. - if not sys.platform.startswith('win'): - return - - text = self.text - text.event_add("<>", "") - text.event_add("<>", "") - def highlight_fix(focus): - sel_range = text.tag_ranges("sel") - if sel_range: - if focus == 'out': - HILITE_CONFIG = idleConf.GetHighlight( - idleConf.CurrentTheme(), 'hilite') - text.tag_config("sel_fix", HILITE_CONFIG) - text.tag_raise("sel_fix") - text.tag_add("sel_fix", *sel_range) - elif focus == 'in': - text.tag_remove("sel_fix", "1.0", "end") - - text.bind("<>", - lambda ev: highlight_fix("out")) - text.bind("<>", - lambda ev: highlight_fix("in")) - - def _filename_to_unicode(self, filename): """Return filename as BMP unicode so diplayable in Tk.""" # Decode bytes to unicode. @@ -785,6 +755,7 @@ insertbackground=cursor_color, selectforeground=select_colors['foreground'], selectbackground=select_colors['background'], + inactiveselectbackground=select_colors['background'], ) IDENTCHARS = string.ascii_letters + string.digits + "_" -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 28 04:51:50 2015 From: python-checkins at python.org (terry.reedy) Date: Mon, 28 Sep 2015 02:51:50 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzI0OTcy?= =?utf-8?q?=3A_Inactive_selection_background_now_matches_active_selection?= Message-ID: <20150928025150.115117.76992@psf.io> https://hg.python.org/cpython/rev/4b3356f1a261 changeset: 98342:4b3356f1a261 branch: 2.7 parent: 98331:d867ca794bdb user: Terry Jan Reedy date: Sun Sep 27 22:46:12 2015 -0400 summary: Issue #24972: Inactive selection background now matches active selection background, as selected by user, on all systems. This also fixes a problem with found items not highlighted on Windows. Initial patch by Mark Roseman. Fix replaces workaround with obscure but proper configuration option. files: Lib/idlelib/EditorWindow.py | 31 +------------------------ 1 files changed, 1 insertions(+), 30 deletions(-) diff --git a/Lib/idlelib/EditorWindow.py b/Lib/idlelib/EditorWindow.py --- a/Lib/idlelib/EditorWindow.py +++ b/Lib/idlelib/EditorWindow.py @@ -349,36 +349,6 @@ self.askinteger = tkSimpleDialog.askinteger self.showerror = tkMessageBox.showerror - self._highlight_workaround() # Fix selection tags on Windows - - def _highlight_workaround(self): - # On Windows, Tk removes painting of the selection - # tags which is different behavior than on Linux and Mac. - # See issue14146 for more information. - if not sys.platform.startswith('win'): - return - - text = self.text - text.event_add("<>", "") - text.event_add("<>", "") - def highlight_fix(focus): - sel_range = text.tag_ranges("sel") - if sel_range: - if focus == 'out': - HILITE_CONFIG = idleConf.GetHighlight( - idleConf.CurrentTheme(), 'hilite') - text.tag_config("sel_fix", HILITE_CONFIG) - text.tag_raise("sel_fix") - text.tag_add("sel_fix", *sel_range) - elif focus == 'in': - text.tag_remove("sel_fix", "1.0", "end") - - text.bind("<>", - lambda ev: highlight_fix("out")) - text.bind("<>", - lambda ev: highlight_fix("in")) - - def _filename_to_unicode(self, filename): """convert filename to unicode in order to display it in Tk""" if isinstance(filename, unicode) or not filename: @@ -800,6 +770,7 @@ insertbackground=cursor_color, selectforeground=select_colors['foreground'], selectbackground=select_colors['background'], + inactiveselectbackground=select_colors['background'], ) def ResetFont(self): -- Repository URL: https://hg.python.org/cpython From tjreedy at udel.edu Mon Sep 28 04:58:59 2015 From: tjreedy at udel.edu (Terry Reedy) Date: Sun, 27 Sep 2015 22:58:59 -0400 Subject: [Python-checkins] cpython (merge 3.4 -> 3.4): merge In-Reply-To: <20150928023532.82644.65634@psf.io> References: <20150928023532.82644.65634@psf.io> Message-ID: <5608ACF3.2070308@udel.edu> On 9/27/2015 10:35 PM, alexander.belopolsky wrote: > https://hg.python.org/cpython/rev/1ae0e02b541d > changeset: 98340:1ae0e02b541d > branch: 3.4 > parent: 98338:cbcf82f92c25 > parent: 98334:ff68705c56a8 > user: Alexander Belopolsky > date: Sun Sep 27 22:34:07 2015 -0400 > summary: > merge This 3.4 - 3.4 merge was not merged forward. When it is, there will be merge conflicts. When I failed to notice, committed to 3.4, and attempted to 3.5, I was asked to resolve your merge conflicts. Perhaps you want them null-merged, but I don't know for sure. So I reverted before pushing to get the 2.7 commit in. From tjreedy at udel.edu Mon Sep 28 05:04:09 2015 From: tjreedy at udel.edu (Terry Reedy) Date: Sun, 27 Sep 2015 23:04:09 -0400 Subject: [Python-checkins] cpython (merge 3.5 -> 3.5): merge In-Reply-To: <20150928023533.94119.44667@psf.io> References: <20150928023533.94119.44667@psf.io> Message-ID: <5608AE29.8070401@udel.edu> On 9/27/2015 10:35 PM, alexander.belopolsky wrote: > https://hg.python.org/cpython/rev/439fc6c96f65 > changeset: 98341:439fc6c96f65 > branch: 3.5 > parent: 98339:848d665bc312 > parent: 98335:c706c062f545 > user: Alexander Belopolsky > date: Sun Sep 27 22:34:59 2015 -0400 > summary: This 3.5-3.5 merge was not merged forward to default (3.6). I believe you may want to (null?) merge the unmerged 3.4-3.4 merge first, but I don't know. Perhaps someone who understands hg as we use it can help. From tjreedy at udel.edu Mon Sep 28 05:10:39 2015 From: tjreedy at udel.edu (Terry Reedy) Date: Sun, 27 Sep 2015 23:10:39 -0400 Subject: [Python-checkins] cpython (3.4): Issue #24972: Inactive selection background now matches active selection In-Reply-To: <20150928025150.115149.87292@psf.io> References: <20150928025150.115149.87292@psf.io> Message-ID: <5608AFAF.5010702@udel.edu> On 9/27/2015 10:51 PM, terry.reedy wrote: > https://hg.python.org/cpython/rev/70c01dd35100 > changeset: 98343:70c01dd35100 > branch: 3.4 > parent: 98340:1ae0e02b541d > user: Terry Jan Reedy > date: Sun Sep 27 22:46:17 2015 -0400 I committed this before noticing that Alexander Belopolsky had left a hanging 3.4 commit (and one for 3.5). When I tried to merge forward, two of his files had merge conflicts, so I could not continue (or did not know how). I backed this out and pushed, in order to get the 2.7 commit pushed, but I don't know if this was exactly the right thing to do. Without merge conflicts in other files, this patch will merge to 3.5 and default cleanly, as EditorWindow.py is identical in all 3 versions. > summary: > Issue #24972: Inactive selection background now matches active selection > background, as selected by user, on all systems. This also fixes a problem > with found items not highlighted on Windows. Initial patch by Mark Roseman. > Fix replaces workaround with obscure but proper configuration option. > > files: > Lib/idlelib/EditorWindow.py | 31 +------------------------ > 1 files changed, 1 insertions(+), 30 deletions(-) > > > diff --git a/Lib/idlelib/EditorWindow.py b/Lib/idlelib/EditorWindow.py > --- a/Lib/idlelib/EditorWindow.py > +++ b/Lib/idlelib/EditorWindow.py > @@ -317,36 +317,6 @@ > self.askinteger = tkSimpleDialog.askinteger > self.showerror = tkMessageBox.showerror > > - self._highlight_workaround() # Fix selection tags on Windows > - > - def _highlight_workaround(self): > - # On Windows, Tk removes painting of the selection > - # tags which is different behavior than on Linux and Mac. > - # See issue14146 for more information. > - if not sys.platform.startswith('win'): > - return > - > - text = self.text > - text.event_add("<>", "") > - text.event_add("<>", "") > - def highlight_fix(focus): > - sel_range = text.tag_ranges("sel") > - if sel_range: > - if focus == 'out': > - HILITE_CONFIG = idleConf.GetHighlight( > - idleConf.CurrentTheme(), 'hilite') > - text.tag_config("sel_fix", HILITE_CONFIG) > - text.tag_raise("sel_fix") > - text.tag_add("sel_fix", *sel_range) > - elif focus == 'in': > - text.tag_remove("sel_fix", "1.0", "end") > - > - text.bind("<>", > - lambda ev: highlight_fix("out")) > - text.bind("<>", > - lambda ev: highlight_fix("in")) > - > - > def _filename_to_unicode(self, filename): > """Return filename as BMP unicode so diplayable in Tk.""" > # Decode bytes to unicode. > @@ -785,6 +755,7 @@ > insertbackground=cursor_color, > selectforeground=select_colors['foreground'], > selectbackground=select_colors['background'], > + inactiveselectbackground=select_colors['background'], > ) > > IDENTCHARS = string.ascii_letters + string.digits + "_" > > > > _______________________________________________ > Python-checkins mailing list > Python-checkins at python.org > https://mail.python.org/mailman/listinfo/python-checkins > From python-checkins at python.org Mon Sep 28 08:31:51 2015 From: python-checkins at python.org (martin.panter) Date: Mon, 28 Sep 2015 06:31:51 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Merge_from_3=2E4_into_3=2E5=3B_no_file_changes?= Message-ID: <20150928063151.31203.89498@psf.io> https://hg.python.org/cpython/rev/a95be179d40d changeset: 98345:a95be179d40d branch: 3.5 parent: 98341:439fc6c96f65 parent: 98344:719dcc47b259 user: Martin Panter date: Mon Sep 28 06:28:29 2015 +0000 summary: Merge from 3.4 into 3.5; no file changes files: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 28 08:31:51 2015 From: python-checkins at python.org (martin.panter) Date: Mon, 28 Sep 2015 06:31:51 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge_from_3=2E5=3B_no_file_changes?= Message-ID: <20150928063151.9939.26419@psf.io> https://hg.python.org/cpython/rev/4d9b31b2a360 changeset: 98346:4d9b31b2a360 parent: 98337:0d44613b014f parent: 98345:a95be179d40d user: Martin Panter date: Mon Sep 28 06:30:43 2015 +0000 summary: Merge from 3.5; no file changes files: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 28 10:17:12 2015 From: python-checkins at python.org (terry.reedy) Date: Mon, 28 Sep 2015 08:17:12 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzI0OTcy?= =?utf-8?q?=3A_Inactive_selection_background_now_matches_active_selection?= Message-ID: <20150928081711.94133.64891@psf.io> https://hg.python.org/cpython/rev/2445750029df changeset: 98347:2445750029df branch: 3.4 parent: 98344:719dcc47b259 user: Terry Jan Reedy date: Mon Sep 28 04:16:32 2015 -0400 summary: Issue #24972: Inactive selection background now matches active selection background, as selected by user, on all systems. This also fixes a problem with found items not highlighted on Windows. Initial patch by Mark Roseman. Fix replaces workaround with obscure but proper configuration option. files: Lib/idlelib/EditorWindow.py | 31 +------------------------ 1 files changed, 1 insertions(+), 30 deletions(-) diff --git a/Lib/idlelib/EditorWindow.py b/Lib/idlelib/EditorWindow.py --- a/Lib/idlelib/EditorWindow.py +++ b/Lib/idlelib/EditorWindow.py @@ -317,36 +317,6 @@ self.askinteger = tkSimpleDialog.askinteger self.showerror = tkMessageBox.showerror - self._highlight_workaround() # Fix selection tags on Windows - - def _highlight_workaround(self): - # On Windows, Tk removes painting of the selection - # tags which is different behavior than on Linux and Mac. - # See issue14146 for more information. - if not sys.platform.startswith('win'): - return - - text = self.text - text.event_add("<>", "") - text.event_add("<>", "") - def highlight_fix(focus): - sel_range = text.tag_ranges("sel") - if sel_range: - if focus == 'out': - HILITE_CONFIG = idleConf.GetHighlight( - idleConf.CurrentTheme(), 'hilite') - text.tag_config("sel_fix", HILITE_CONFIG) - text.tag_raise("sel_fix") - text.tag_add("sel_fix", *sel_range) - elif focus == 'in': - text.tag_remove("sel_fix", "1.0", "end") - - text.bind("<>", - lambda ev: highlight_fix("out")) - text.bind("<>", - lambda ev: highlight_fix("in")) - - def _filename_to_unicode(self, filename): """Return filename as BMP unicode so diplayable in Tk.""" # Decode bytes to unicode. @@ -785,6 +755,7 @@ insertbackground=cursor_color, selectforeground=select_colors['foreground'], selectbackground=select_colors['background'], + inactiveselectbackground=select_colors['background'], ) IDENTCHARS = string.ascii_letters + string.digits + "_" -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 28 10:17:12 2015 From: python-checkins at python.org (terry.reedy) Date: Mon, 28 Sep 2015 08:17:12 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Merge_with_3=2E4?= Message-ID: <20150928081712.98356.40967@psf.io> https://hg.python.org/cpython/rev/2749f6b6f443 changeset: 98348:2749f6b6f443 branch: 3.5 parent: 98345:a95be179d40d parent: 98347:2445750029df user: Terry Jan Reedy date: Mon Sep 28 04:16:43 2015 -0400 summary: Merge with 3.4 files: Lib/idlelib/EditorWindow.py | 31 +------------------------ 1 files changed, 1 insertions(+), 30 deletions(-) diff --git a/Lib/idlelib/EditorWindow.py b/Lib/idlelib/EditorWindow.py --- a/Lib/idlelib/EditorWindow.py +++ b/Lib/idlelib/EditorWindow.py @@ -317,36 +317,6 @@ self.askinteger = tkSimpleDialog.askinteger self.showerror = tkMessageBox.showerror - self._highlight_workaround() # Fix selection tags on Windows - - def _highlight_workaround(self): - # On Windows, Tk removes painting of the selection - # tags which is different behavior than on Linux and Mac. - # See issue14146 for more information. - if not sys.platform.startswith('win'): - return - - text = self.text - text.event_add("<>", "") - text.event_add("<>", "") - def highlight_fix(focus): - sel_range = text.tag_ranges("sel") - if sel_range: - if focus == 'out': - HILITE_CONFIG = idleConf.GetHighlight( - idleConf.CurrentTheme(), 'hilite') - text.tag_config("sel_fix", HILITE_CONFIG) - text.tag_raise("sel_fix") - text.tag_add("sel_fix", *sel_range) - elif focus == 'in': - text.tag_remove("sel_fix", "1.0", "end") - - text.bind("<>", - lambda ev: highlight_fix("out")) - text.bind("<>", - lambda ev: highlight_fix("in")) - - def _filename_to_unicode(self, filename): """Return filename as BMP unicode so diplayable in Tk.""" # Decode bytes to unicode. @@ -785,6 +755,7 @@ insertbackground=cursor_color, selectforeground=select_colors['foreground'], selectbackground=select_colors['background'], + inactiveselectbackground=select_colors['background'], ) IDENTCHARS = string.ascii_letters + string.digits + "_" -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 28 10:17:13 2015 From: python-checkins at python.org (terry.reedy) Date: Mon, 28 Sep 2015 08:17:13 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge_with_3=2E5?= Message-ID: <20150928081712.94117.67775@psf.io> https://hg.python.org/cpython/rev/ec3c6d4d6be9 changeset: 98349:ec3c6d4d6be9 parent: 98346:4d9b31b2a360 parent: 98348:2749f6b6f443 user: Terry Jan Reedy date: Mon Sep 28 04:16:56 2015 -0400 summary: Merge with 3.5 files: Lib/idlelib/EditorWindow.py | 31 +------------------------ 1 files changed, 1 insertions(+), 30 deletions(-) diff --git a/Lib/idlelib/EditorWindow.py b/Lib/idlelib/EditorWindow.py --- a/Lib/idlelib/EditorWindow.py +++ b/Lib/idlelib/EditorWindow.py @@ -317,36 +317,6 @@ self.askinteger = tkSimpleDialog.askinteger self.showerror = tkMessageBox.showerror - self._highlight_workaround() # Fix selection tags on Windows - - def _highlight_workaround(self): - # On Windows, Tk removes painting of the selection - # tags which is different behavior than on Linux and Mac. - # See issue14146 for more information. - if not sys.platform.startswith('win'): - return - - text = self.text - text.event_add("<>", "") - text.event_add("<>", "") - def highlight_fix(focus): - sel_range = text.tag_ranges("sel") - if sel_range: - if focus == 'out': - HILITE_CONFIG = idleConf.GetHighlight( - idleConf.CurrentTheme(), 'hilite') - text.tag_config("sel_fix", HILITE_CONFIG) - text.tag_raise("sel_fix") - text.tag_add("sel_fix", *sel_range) - elif focus == 'in': - text.tag_remove("sel_fix", "1.0", "end") - - text.bind("<>", - lambda ev: highlight_fix("out")) - text.bind("<>", - lambda ev: highlight_fix("in")) - - def _filename_to_unicode(self, filename): """Return filename as BMP unicode so diplayable in Tk.""" # Decode bytes to unicode. @@ -785,6 +755,7 @@ insertbackground=cursor_color, selectforeground=select_colors['foreground'], selectbackground=select_colors['background'], + inactiveselectbackground=select_colors['background'], ) IDENTCHARS = string.ascii_letters + string.digits + "_" -- Repository URL: https://hg.python.org/cpython From solipsis at pitrou.net Mon Sep 28 10:43:20 2015 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Mon, 28 Sep 2015 08:43:20 +0000 Subject: [Python-checkins] Daily reference leaks (0d44613b014f): sum=61473 Message-ID: <20150928084319.31199.79851@psf.io> results for 0d44613b014f on branch "default" -------------------------------------------- test_capi leaked [5409, 5409, 5409] references, sum=16227 test_capi leaked [1420, 1422, 1422] memory blocks, sum=4264 test_functools leaked [0, 2, 2] memory blocks, sum=4 test_threading leaked [10818, 10818, 10818] references, sum=32454 test_threading leaked [2840, 2842, 2842] memory blocks, sum=8524 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/psf-users/antoine/refleaks/reflogkZ4knt', '--timeout', '7200'] From python-checkins at python.org Mon Sep 28 10:53:28 2015 From: python-checkins at python.org (terry.reedy) Date: Mon, 28 Sep 2015 08:53:28 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzI0OTcy?= =?utf-8?q?=3A_New_option_is_only_valid_in_tk_8=2E5+=2E?= Message-ID: <20150928085328.16581.72059@psf.io> https://hg.python.org/cpython/rev/45955adc2ed2 changeset: 98350:45955adc2ed2 branch: 2.7 parent: 98342:4b3356f1a261 user: Terry Jan Reedy date: Mon Sep 28 04:52:44 2015 -0400 summary: Issue #24972: New option is only valid in tk 8.5+. files: Lib/idlelib/EditorWindow.py | 4 +++- 1 files changed, 3 insertions(+), 1 deletions(-) diff --git a/Lib/idlelib/EditorWindow.py b/Lib/idlelib/EditorWindow.py --- a/Lib/idlelib/EditorWindow.py +++ b/Lib/idlelib/EditorWindow.py @@ -770,8 +770,10 @@ insertbackground=cursor_color, selectforeground=select_colors['foreground'], selectbackground=select_colors['background'], - inactiveselectbackground=select_colors['background'], ) + if TkVersion >= 8.5: + self.text.config( + inactiveselectbackground=select_colors['background']) def ResetFont(self): "Update the text widgets' font if it is changed" -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 28 10:53:28 2015 From: python-checkins at python.org (terry.reedy) Date: Mon, 28 Sep 2015 08:53:28 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzI0OTcy?= =?utf-8?q?=3A_New_option_is_only_valid_in_tk_8=2E5+=2E?= Message-ID: <20150928085328.31179.12382@psf.io> https://hg.python.org/cpython/rev/460e6e6fb09a changeset: 98351:460e6e6fb09a branch: 3.4 parent: 98347:2445750029df user: Terry Jan Reedy date: Mon Sep 28 04:52:49 2015 -0400 summary: Issue #24972: New option is only valid in tk 8.5+. files: Lib/idlelib/EditorWindow.py | 4 +++- 1 files changed, 3 insertions(+), 1 deletions(-) diff --git a/Lib/idlelib/EditorWindow.py b/Lib/idlelib/EditorWindow.py --- a/Lib/idlelib/EditorWindow.py +++ b/Lib/idlelib/EditorWindow.py @@ -755,8 +755,10 @@ insertbackground=cursor_color, selectforeground=select_colors['foreground'], selectbackground=select_colors['background'], - inactiveselectbackground=select_colors['background'], ) + if TkVersion >= 8.5: + self.text.config( + inactiveselectbackground=select_colors['background']) IDENTCHARS = string.ascii_letters + string.digits + "_" -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 28 10:53:28 2015 From: python-checkins at python.org (terry.reedy) Date: Mon, 28 Sep 2015 08:53:28 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge_with_3=2E5?= Message-ID: <20150928085328.82646.61418@psf.io> https://hg.python.org/cpython/rev/6895f3f1f8c8 changeset: 98353:6895f3f1f8c8 parent: 98349:ec3c6d4d6be9 parent: 98352:ab9f41deb0c8 user: Terry Jan Reedy date: Mon Sep 28 04:53:12 2015 -0400 summary: Merge with 3.5 files: Lib/idlelib/EditorWindow.py | 4 +++- 1 files changed, 3 insertions(+), 1 deletions(-) diff --git a/Lib/idlelib/EditorWindow.py b/Lib/idlelib/EditorWindow.py --- a/Lib/idlelib/EditorWindow.py +++ b/Lib/idlelib/EditorWindow.py @@ -755,8 +755,10 @@ insertbackground=cursor_color, selectforeground=select_colors['foreground'], selectbackground=select_colors['background'], - inactiveselectbackground=select_colors['background'], ) + if TkVersion >= 8.5: + self.text.config( + inactiveselectbackground=select_colors['background']) IDENTCHARS = string.ascii_letters + string.digits + "_" -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 28 10:53:28 2015 From: python-checkins at python.org (terry.reedy) Date: Mon, 28 Sep 2015 08:53:28 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Merge_with_3=2E4?= Message-ID: <20150928085328.3650.61576@psf.io> https://hg.python.org/cpython/rev/ab9f41deb0c8 changeset: 98352:ab9f41deb0c8 branch: 3.5 parent: 98348:2749f6b6f443 parent: 98351:460e6e6fb09a user: Terry Jan Reedy date: Mon Sep 28 04:53:01 2015 -0400 summary: Merge with 3.4 files: Lib/idlelib/EditorWindow.py | 4 +++- 1 files changed, 3 insertions(+), 1 deletions(-) diff --git a/Lib/idlelib/EditorWindow.py b/Lib/idlelib/EditorWindow.py --- a/Lib/idlelib/EditorWindow.py +++ b/Lib/idlelib/EditorWindow.py @@ -755,8 +755,10 @@ insertbackground=cursor_color, selectforeground=select_colors['foreground'], selectbackground=select_colors['background'], - inactiveselectbackground=select_colors['background'], ) + if TkVersion >= 8.5: + self.text.config( + inactiveselectbackground=select_colors['background']) IDENTCHARS = string.ascii_letters + string.digits + "_" -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 28 12:34:20 2015 From: python-checkins at python.org (berker.peksag) Date: Mon, 28 Sep 2015 10:34:20 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2325249=3A_Remove_unneeded_mkstemp_helper_in_test?= =?utf-8?q?=5Fsubprocess?= Message-ID: <20150928103420.81639.20774@psf.io> https://hg.python.org/cpython/rev/e863c8760501 changeset: 98356:e863c8760501 parent: 98353:6895f3f1f8c8 parent: 98355:4cd3027ffc34 user: Berker Peksag date: Mon Sep 28 13:34:17 2015 +0300 summary: Issue #25249: Remove unneeded mkstemp helper in test_subprocess The helper was added in 76641824cf05 11 years ago and it can be removed now since all supported Python versions have tempfile.mkstemp(). Patch by Nir Soffer. files: Lib/test/test_subprocess.py | 26 +++++++----------------- 1 files changed, 8 insertions(+), 18 deletions(-) diff --git a/Lib/test/test_subprocess.py b/Lib/test/test_subprocess.py --- a/Lib/test/test_subprocess.py +++ b/Lib/test/test_subprocess.py @@ -37,16 +37,6 @@ SETBINARY = '' -try: - mkstemp = tempfile.mkstemp -except AttributeError: - # tempfile.mkstemp is not available - def mkstemp(): - """Replacement for mkstemp, calling mktemp.""" - fname = tempfile.mktemp() - return os.open(fname, os.O_RDWR|os.O_CREAT), fname - - class BaseTestCase(unittest.TestCase): def setUp(self): # Try to minimize the number of children we have so this test @@ -1150,9 +1140,9 @@ def test_handles_closed_on_exception(self): # If CreateProcess exits with an error, ensure the # duplicate output handles are released - ifhandle, ifname = mkstemp() - ofhandle, ofname = mkstemp() - efhandle, efname = mkstemp() + ifhandle, ifname = tempfile.mkstemp() + ofhandle, ofname = tempfile.mkstemp() + efhandle, efname = tempfile.mkstemp() try: subprocess.Popen (["*"], stdin=ifhandle, stdout=ofhandle, stderr=efhandle) @@ -1524,7 +1514,7 @@ def test_args_string(self): # args is a string - fd, fname = mkstemp() + fd, fname = tempfile.mkstemp() # reopen in text mode with open(fd, "w", errors="surrogateescape") as fobj: fobj.write("#!/bin/sh\n") @@ -1569,7 +1559,7 @@ def test_call_string(self): # call() function with string argument on UNIX - fd, fname = mkstemp() + fd, fname = tempfile.mkstemp() # reopen in text mode with open(fd, "w", errors="surrogateescape") as fobj: fobj.write("#!/bin/sh\n") @@ -1762,7 +1752,7 @@ def test_remapping_std_fds(self): # open up some temporary files - temps = [mkstemp() for i in range(3)] + temps = [tempfile.mkstemp() for i in range(3)] try: temp_fds = [fd for fd, fname in temps] @@ -1807,7 +1797,7 @@ def check_swap_fds(self, stdin_no, stdout_no, stderr_no): # open up some temporary files - temps = [mkstemp() for i in range(3)] + temps = [tempfile.mkstemp() for i in range(3)] temp_fds = [fd for fd, fname in temps] try: # unlink the files -- we won't need to reopen them @@ -2534,7 +2524,7 @@ def setUp(self): super().setUp() - f, fname = mkstemp(".py", "te st") + f, fname = tempfile.mkstemp(".py", "te st") self.fname = fname.lower () os.write(f, b"import sys;" b"sys.stdout.write('%d %s' % (len(sys.argv), [a.lower () for a in sys.argv]))" -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 28 12:34:21 2015 From: python-checkins at python.org (berker.peksag) Date: Mon, 28 Sep 2015 10:34:21 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzI1MjQ5?= =?utf-8?q?=3A_Remove_unneeded_mkstemp_helper_in_test=5Fsubprocess?= Message-ID: <20150928103420.82654.93867@psf.io> https://hg.python.org/cpython/rev/23f4daf7a211 changeset: 98354:23f4daf7a211 branch: 3.4 parent: 98351:460e6e6fb09a user: Berker Peksag date: Mon Sep 28 13:33:14 2015 +0300 summary: Issue #25249: Remove unneeded mkstemp helper in test_subprocess The helper was added in 76641824cf05 11 years ago and it can be removed now since all supported Python versions have tempfile.mkstemp(). Patch by Nir Soffer. files: Lib/test/test_subprocess.py | 26 +++++++----------------- 1 files changed, 8 insertions(+), 18 deletions(-) diff --git a/Lib/test/test_subprocess.py b/Lib/test/test_subprocess.py --- a/Lib/test/test_subprocess.py +++ b/Lib/test/test_subprocess.py @@ -37,16 +37,6 @@ SETBINARY = '' -try: - mkstemp = tempfile.mkstemp -except AttributeError: - # tempfile.mkstemp is not available - def mkstemp(): - """Replacement for mkstemp, calling mktemp.""" - fname = tempfile.mktemp() - return os.open(fname, os.O_RDWR|os.O_CREAT), fname - - class BaseTestCase(unittest.TestCase): def setUp(self): # Try to minimize the number of children we have so this test @@ -1150,9 +1140,9 @@ def test_handles_closed_on_exception(self): # If CreateProcess exits with an error, ensure the # duplicate output handles are released - ifhandle, ifname = mkstemp() - ofhandle, ofname = mkstemp() - efhandle, efname = mkstemp() + ifhandle, ifname = tempfile.mkstemp() + ofhandle, ofname = tempfile.mkstemp() + efhandle, efname = tempfile.mkstemp() try: subprocess.Popen (["*"], stdin=ifhandle, stdout=ofhandle, stderr=efhandle) @@ -1428,7 +1418,7 @@ def test_args_string(self): # args is a string - fd, fname = mkstemp() + fd, fname = tempfile.mkstemp() # reopen in text mode with open(fd, "w", errors="surrogateescape") as fobj: fobj.write("#!/bin/sh\n") @@ -1473,7 +1463,7 @@ def test_call_string(self): # call() function with string argument on UNIX - fd, fname = mkstemp() + fd, fname = tempfile.mkstemp() # reopen in text mode with open(fd, "w", errors="surrogateescape") as fobj: fobj.write("#!/bin/sh\n") @@ -1666,7 +1656,7 @@ def test_remapping_std_fds(self): # open up some temporary files - temps = [mkstemp() for i in range(3)] + temps = [tempfile.mkstemp() for i in range(3)] try: temp_fds = [fd for fd, fname in temps] @@ -1711,7 +1701,7 @@ def check_swap_fds(self, stdin_no, stdout_no, stderr_no): # open up some temporary files - temps = [mkstemp() for i in range(3)] + temps = [tempfile.mkstemp() for i in range(3)] temp_fds = [fd for fd, fname in temps] try: # unlink the files -- we won't need to reopen them @@ -2442,7 +2432,7 @@ def setUp(self): super().setUp() - f, fname = mkstemp(".py", "te st") + f, fname = tempfile.mkstemp(".py", "te st") self.fname = fname.lower () os.write(f, b"import sys;" b"sys.stdout.write('%d %s' % (len(sys.argv), [a.lower () for a in sys.argv]))" -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 28 12:34:21 2015 From: python-checkins at python.org (berker.peksag) Date: Mon, 28 Sep 2015 10:34:21 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Issue_=2325249=3A_Remove_unneeded_mkstemp_helper_in_test=5Fsub?= =?utf-8?q?process?= Message-ID: <20150928103420.115399.97650@psf.io> https://hg.python.org/cpython/rev/4cd3027ffc34 changeset: 98355:4cd3027ffc34 branch: 3.5 parent: 98352:ab9f41deb0c8 parent: 98354:23f4daf7a211 user: Berker Peksag date: Mon Sep 28 13:33:43 2015 +0300 summary: Issue #25249: Remove unneeded mkstemp helper in test_subprocess The helper was added in 76641824cf05 11 years ago and it can be removed now since all supported Python versions have tempfile.mkstemp(). Patch by Nir Soffer. files: Lib/test/test_subprocess.py | 26 +++++++----------------- 1 files changed, 8 insertions(+), 18 deletions(-) diff --git a/Lib/test/test_subprocess.py b/Lib/test/test_subprocess.py --- a/Lib/test/test_subprocess.py +++ b/Lib/test/test_subprocess.py @@ -37,16 +37,6 @@ SETBINARY = '' -try: - mkstemp = tempfile.mkstemp -except AttributeError: - # tempfile.mkstemp is not available - def mkstemp(): - """Replacement for mkstemp, calling mktemp.""" - fname = tempfile.mktemp() - return os.open(fname, os.O_RDWR|os.O_CREAT), fname - - class BaseTestCase(unittest.TestCase): def setUp(self): # Try to minimize the number of children we have so this test @@ -1150,9 +1140,9 @@ def test_handles_closed_on_exception(self): # If CreateProcess exits with an error, ensure the # duplicate output handles are released - ifhandle, ifname = mkstemp() - ofhandle, ofname = mkstemp() - efhandle, efname = mkstemp() + ifhandle, ifname = tempfile.mkstemp() + ofhandle, ofname = tempfile.mkstemp() + efhandle, efname = tempfile.mkstemp() try: subprocess.Popen (["*"], stdin=ifhandle, stdout=ofhandle, stderr=efhandle) @@ -1524,7 +1514,7 @@ def test_args_string(self): # args is a string - fd, fname = mkstemp() + fd, fname = tempfile.mkstemp() # reopen in text mode with open(fd, "w", errors="surrogateescape") as fobj: fobj.write("#!/bin/sh\n") @@ -1569,7 +1559,7 @@ def test_call_string(self): # call() function with string argument on UNIX - fd, fname = mkstemp() + fd, fname = tempfile.mkstemp() # reopen in text mode with open(fd, "w", errors="surrogateescape") as fobj: fobj.write("#!/bin/sh\n") @@ -1762,7 +1752,7 @@ def test_remapping_std_fds(self): # open up some temporary files - temps = [mkstemp() for i in range(3)] + temps = [tempfile.mkstemp() for i in range(3)] try: temp_fds = [fd for fd, fname in temps] @@ -1807,7 +1797,7 @@ def check_swap_fds(self, stdin_no, stdout_no, stderr_no): # open up some temporary files - temps = [mkstemp() for i in range(3)] + temps = [tempfile.mkstemp() for i in range(3)] temp_fds = [fd for fd, fname in temps] try: # unlink the files -- we won't need to reopen them @@ -2534,7 +2524,7 @@ def setUp(self): super().setUp() - f, fname = mkstemp(".py", "te st") + f, fname = tempfile.mkstemp(".py", "te st") self.fname = fname.lower () os.write(f, b"import sys;" b"sys.stdout.write('%d %s' % (len(sys.argv), [a.lower () for a in sys.argv]))" -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 28 14:37:58 2015 From: python-checkins at python.org (berker.peksag) Date: Mon, 28 Sep 2015 12:37:58 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzI1MjQ5?= =?utf-8?q?=3A_Remove_unneeded_mkstemp_helper_in_test=5Fsubprocess?= Message-ID: <20150928123758.11702.3511@psf.io> https://hg.python.org/cpython/rev/ea91991c7db5 changeset: 98357:ea91991c7db5 branch: 2.7 parent: 98350:45955adc2ed2 user: Berker Peksag date: Mon Sep 28 15:37:57 2015 +0300 summary: Issue #25249: Remove unneeded mkstemp helper in test_subprocess The helper was added in 76641824cf05 11 years ago and it can be removed now since all supported Python versions have tempfile.mkstemp(). Patch by Nir Soffer. files: Lib/test/test_subprocess.py | 24 +++++++----------------- 1 files changed, 7 insertions(+), 17 deletions(-) diff --git a/Lib/test/test_subprocess.py b/Lib/test/test_subprocess.py --- a/Lib/test/test_subprocess.py +++ b/Lib/test/test_subprocess.py @@ -32,16 +32,6 @@ SETBINARY = '' -try: - mkstemp = tempfile.mkstemp -except AttributeError: - # tempfile.mkstemp is not available - def mkstemp(): - """Replacement for mkstemp, calling mktemp.""" - fname = tempfile.mktemp() - return os.open(fname, os.O_RDWR|os.O_CREAT), fname - - class BaseTestCase(unittest.TestCase): def setUp(self): # Try to minimize the number of children we have so this test @@ -666,9 +656,9 @@ def test_handles_closed_on_exception(self): # If CreateProcess exits with an error, ensure the # duplicate output handles are released - ifhandle, ifname = mkstemp() - ofhandle, ofname = mkstemp() - efhandle, efname = mkstemp() + ifhandle, ifname = tempfile.mkstemp() + ofhandle, ofname = tempfile.mkstemp() + efhandle, efname = tempfile.mkstemp() try: subprocess.Popen (["*"], stdin=ifhandle, stdout=ofhandle, stderr=efhandle) @@ -858,7 +848,7 @@ def test_args_string(self): # args is a string - f, fname = mkstemp() + f, fname = tempfile.mkstemp() os.write(f, "#!/bin/sh\n") os.write(f, "exec '%s' -c 'import sys; sys.exit(47)'\n" % sys.executable) @@ -902,7 +892,7 @@ def test_call_string(self): # call() function with string argument on UNIX - f, fname = mkstemp() + f, fname = tempfile.mkstemp() os.write(f, "#!/bin/sh\n") os.write(f, "exec '%s' -c 'import sys; sys.exit(47)'\n" % sys.executable) @@ -1058,7 +1048,7 @@ def check_swap_fds(self, stdin_no, stdout_no, stderr_no): # open up some temporary files - temps = [mkstemp() for i in range(3)] + temps = [tempfile.mkstemp() for i in range(3)] temp_fds = [fd for fd, fname in temps] try: # unlink the files -- we won't need to reopen them @@ -1379,7 +1369,7 @@ def setUp(self): super(CommandsWithSpaces, self).setUp() - f, fname = mkstemp(".py", "te st") + f, fname = tempfile.mkstemp(".py", "te st") self.fname = fname.lower () os.write(f, b"import sys;" b"sys.stdout.write('%d %s' % (len(sys.argv), [a.lower () for a in sys.argv]))" -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 28 15:04:34 2015 From: python-checkins at python.org (victor.stinner) Date: Mon, 28 Sep 2015 13:04:34 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2325122=3A_Remove_v?= =?utf-8?q?erbose_mode_of_test=5Feintr?= Message-ID: <20150928130427.98368.92244@psf.io> https://hg.python.org/cpython/rev/e48ec5f5b82b changeset: 98358:e48ec5f5b82b parent: 98356:e863c8760501 user: Victor Stinner date: Mon Sep 28 15:04:11 2015 +0200 summary: Issue #25122: Remove verbose mode of test_eintr "./python -m test -W test_eintr" wrote Lib/test/eintrdata/eintr_tester.py output to stdout which was not expected. Since test_eintr doesn't hang anymore, remove the verbose mode instead. files: Lib/test/test_eintr.py | 9 +-------- 1 files changed, 1 insertions(+), 8 deletions(-) diff --git a/Lib/test/test_eintr.py b/Lib/test/test_eintr.py --- a/Lib/test/test_eintr.py +++ b/Lib/test/test_eintr.py @@ -16,14 +16,7 @@ # Run the tester in a sub-process, to make sure there is only one # thread (for reliable signal delivery). tester = support.findfile("eintr_tester.py", subdir="eintrdata") - - if support.verbose: - args = [sys.executable, tester] - with subprocess.Popen(args) as proc: - exitcode = proc.wait() - self.assertEqual(exitcode, 0) - else: - script_helper.assert_python_ok(tester) + script_helper.assert_python_ok(tester) if __name__ == "__main__": -- Repository URL: https://hg.python.org/cpython From lp_benchmark_robot at intel.com Mon Sep 28 16:06:41 2015 From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com) Date: Mon, 28 Sep 2015 15:06:41 +0100 Subject: [Python-checkins] Benchmark Results for Python Default 2015-09-28 Message-ID: <9b773093-8035-46d6-b7e1-eb26b863e7af@irsmsx103.ger.corp.intel.com> Results for project python_default-nightly, build date 2015-09-28 03:02:36 commit: 0d44613b014fa170c38785c46289ed318e86424d revision date: 2015-09-28 02:13:28 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v3.4.3, with hash b4cbecbc0781e89a309d03b60a1f75f8499250e6 from 2015-02-25 12:15:33+00:00 ------------------------------------------------------------------------------------------ benchmark relative change since change since current rev with std_dev* last run v3.4.3 regrtest PGO ------------------------------------------------------------------------------------------ :-) django_v2 0.19454% -1.45610% 6.79332% 17.18993% :-( pybench 0.24613% -0.02967% -2.66484% 9.37407% :-( regex_v8 2.69079% 0.06527% -4.21906% 5.54128% :-| nbody 0.16058% 0.20795% -0.12879% 8.15935% :-| json_dump_v2 0.22856% 0.22196% -0.83978% 12.21277% :-| normal_startup 0.64222% -0.00843% 0.65287% 5.08974% ------------------------------------------------------------------------------------------ Note: Benchmark results are measured in seconds. * Relative Standard Deviation (Standard Deviation/Average) Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. From lp_benchmark_robot at intel.com Mon Sep 28 16:07:53 2015 From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com) Date: Mon, 28 Sep 2015 15:07:53 +0100 Subject: [Python-checkins] Benchmark Results for Python 2.7 2015-09-28 Message-ID: <7d6ca466-4232-4344-baa7-839b18cf9b6e@irsmsx103.ger.corp.intel.com> Results for project python_2.7-nightly, build date 2015-09-26 03:42:45 commit: e406d62014f7699ddfc84caa0c08d529c205ac73 revision date: 2015-09-26 02:22:48 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v2.7.10, with hash 15c95b7d81dcf821daade360741e00714667653f from 2015-05-23 16:02:14+00:00 ------------------------------------------------------------------------------------------ benchmark relative change since change since current rev with std_dev* last run v2.7.10 regrtest PGO ------------------------------------------------------------------------------------------ :-) django_v2 0.16949% 0.35958% 4.21155% 9.67450% :-) pybench 0.13793% 0.06039% 6.85153% 6.59819% :-| regex_v8 0.57408% -0.07828% -1.23873% 7.63209% :-) nbody 0.15044% 0.00640% 9.08509% 3.91032% :-) json_dump_v2 0.21479% -0.07608% 4.57865% 13.91072% :-| normal_startup 1.79762% -0.17667% -1.46201% 2.62710% :-| ssbench 1.04925% -0.61045% 1.38961% 2.99594% ------------------------------------------------------------------------------------------ Note: Benchmark results for ssbench are measured in requests/second while all other are measured in seconds. * Relative Standard Deviation (Standard Deviation/Average) Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. From python-checkins at python.org Mon Sep 28 16:46:30 2015 From: python-checkins at python.org (guido.van.rossum) Date: Mon, 28 Sep 2015 14:46:30 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzI1MjMz?= =?utf-8?q?=3A_Rewrite_the_guts_of_Queue_to_be_more_understandable_and_cor?= =?utf-8?q?rect=2E?= Message-ID: <20150928144629.31199.61028@psf.io> https://hg.python.org/cpython/rev/1ea306202d5d changeset: 98359:1ea306202d5d branch: 3.4 parent: 98354:23f4daf7a211 user: Guido van Rossum date: Mon Sep 28 07:42:34 2015 -0700 summary: Issue #25233: Rewrite the guts of Queue to be more understandable and correct. files: Lib/asyncio/queues.py | 152 ++++---------- Lib/test/test_asyncio/test_queues.py | 55 ++++- Misc/NEWS | 2 + 3 files changed, 91 insertions(+), 118 deletions(-) diff --git a/Lib/asyncio/queues.py b/Lib/asyncio/queues.py --- a/Lib/asyncio/queues.py +++ b/Lib/asyncio/queues.py @@ -47,7 +47,7 @@ # Futures. self._getters = collections.deque() - # Futures + # Futures. self._putters = collections.deque() self._unfinished_tasks = 0 self._finished = locks.Event(loop=self._loop) @@ -67,10 +67,13 @@ # End of the overridable methods. - def __put_internal(self, item): - self._put(item) - self._unfinished_tasks += 1 - self._finished.clear() + def _wakeup_next(self, waiters): + # Wake up the next waiter (if any) that isn't cancelled. + while waiters: + waiter = waiters.popleft() + if not waiter.done(): + waiter.set_result(None) + break def __repr__(self): return '<{} at {:#x} {}>'.format( @@ -91,16 +94,6 @@ result += ' tasks={}'.format(self._unfinished_tasks) return result - def _consume_done_getters(self): - # Delete waiters at the head of the get() queue who've timed out. - while self._getters and self._getters[0].done(): - self._getters.popleft() - - def _consume_done_putters(self): - # Delete waiters at the head of the put() queue who've timed out. - while self._putters and self._putters[0].done(): - self._putters.popleft() - def qsize(self): """Number of items in the queue.""" return len(self._queue) @@ -134,47 +127,31 @@ This method is a coroutine. """ - self._consume_done_getters() - if self._getters: - assert not self._queue, ( - 'queue non-empty, why are getters waiting?') - - getter = self._getters.popleft() - self.__put_internal(item) - - # getter cannot be cancelled, we just removed done getters - getter.set_result(self._get()) - - elif self._maxsize > 0 and self._maxsize <= self.qsize(): - waiter = futures.Future(loop=self._loop) - - self._putters.append(waiter) - yield from waiter - self._put(item) - - else: - self.__put_internal(item) + while self.full(): + putter = futures.Future(loop=self._loop) + self._putters.append(putter) + try: + yield from putter + except: + putter.cancel() # Just in case putter is not done yet. + if not self.full() and not putter.cancelled(): + # We were woken up by get_nowait(), but can't take + # the call. Wake up the next in line. + self._wakeup_next(self._putters) + raise + return self.put_nowait(item) def put_nowait(self, item): """Put an item into the queue without blocking. If no free slot is immediately available, raise QueueFull. """ - self._consume_done_getters() - if self._getters: - assert not self._queue, ( - 'queue non-empty, why are getters waiting?') - - getter = self._getters.popleft() - self.__put_internal(item) - - # getter cannot be cancelled, we just removed done getters - getter.set_result(self._get()) - - elif self._maxsize > 0 and self._maxsize <= self.qsize(): + if self.full(): raise QueueFull - else: - self.__put_internal(item) + self._put(item) + self._unfinished_tasks += 1 + self._finished.clear() + self._wakeup_next(self._getters) @coroutine def get(self): @@ -184,77 +161,30 @@ This method is a coroutine. """ - self._consume_done_putters() - if self._putters: - assert self.full(), 'queue not full, why are putters waiting?' - putter = self._putters.popleft() - - # When a getter runs and frees up a slot so this putter can - # run, we need to defer the put for a tick to ensure that - # getters and putters alternate perfectly. See - # ChannelTest.test_wait. - self._loop.call_soon(putter._set_result_unless_cancelled, None) - - return self._get() - - elif self.qsize(): - return self._get() - else: - waiter = futures.Future(loop=self._loop) - self._getters.append(waiter) + while self.empty(): + getter = futures.Future(loop=self._loop) + self._getters.append(getter) try: - return (yield from waiter) - except futures.CancelledError: - # if we get CancelledError, it means someone cancelled this - # get() coroutine. But there is a chance that the waiter - # already is ready and contains an item that has just been - # removed from the queue. In this case, we need to put the item - # back into the front of the queue. This get() must either - # succeed without fault or, if it gets cancelled, it must be as - # if it never happened. - if waiter.done(): - self._put_it_back(waiter.result()) + yield from getter + except: + getter.cancel() # Just in case getter is not done yet. + if not self.empty() and not getter.cancelled(): + # We were woken up by put_nowait(), but can't take + # the call. Wake up the next in line. + self._wakeup_next(self._getters) raise - - def _put_it_back(self, item): - """ - This is called when we have a waiter to get() an item and this waiter - gets cancelled. In this case, we put the item back: wake up another - waiter or put it in the _queue. - """ - self._consume_done_getters() - if self._getters: - assert not self._queue, ( - 'queue non-empty, why are getters waiting?') - - getter = self._getters.popleft() - self.__put_internal(item) - - # getter cannot be cancelled, we just removed done getters - getter.set_result(item) - else: - self._queue.appendleft(item) + return self.get_nowait() def get_nowait(self): """Remove and return an item from the queue. Return an item if one is immediately available, else raise QueueEmpty. """ - self._consume_done_putters() - if self._putters: - assert self.full(), 'queue not full, why are putters waiting?' - putter = self._putters.popleft() - # Wake putter on next tick. - - # getter cannot be cancelled, we just removed done putters - putter.set_result(None) - - return self._get() - - elif self.qsize(): - return self._get() - else: + if self.empty(): raise QueueEmpty + item = self._get() + self._wakeup_next(self._putters) + return item def task_done(self): """Indicate that a formerly enqueued task is complete. diff --git a/Lib/test/test_asyncio/test_queues.py b/Lib/test/test_asyncio/test_queues.py --- a/Lib/test/test_asyncio/test_queues.py +++ b/Lib/test/test_asyncio/test_queues.py @@ -271,6 +271,29 @@ self.assertEqual(self.loop.run_until_complete(q.get()), 'a') self.assertEqual(self.loop.run_until_complete(q.get()), 'b') + def test_why_are_getters_waiting(self): + # From issue #268. + + @asyncio.coroutine + def consumer(queue, num_expected): + for _ in range(num_expected): + yield from queue.get() + + @asyncio.coroutine + def producer(queue, num_items): + for i in range(num_items): + yield from queue.put(i) + + queue_size = 1 + producer_num_items = 5 + q = asyncio.Queue(queue_size, loop=self.loop) + + self.loop.run_until_complete( + asyncio.gather(producer(q, producer_num_items), + consumer(q, producer_num_items), + loop=self.loop), + ) + class QueuePutTests(_QueueTestBase): @@ -377,13 +400,8 @@ loop.run_until_complete(reader3) - # reader2 will receive `2`, because it was added to the - # queue of pending readers *before* put_nowaits were called. - self.assertEqual(reader2.result(), 2) - # reader3 will receive `1`, because reader1 was cancelled - # before is had a chance to execute, and `2` was already - # pushed to reader2 by second `put_nowait`. - self.assertEqual(reader3.result(), 1) + # It is undefined in which order concurrent readers receive results. + self.assertEqual({reader2.result(), reader3.result()}, {1, 2}) def test_put_cancel_drop(self): @@ -479,6 +497,29 @@ self.loop.run_until_complete(q.put('a')) self.assertEqual(self.loop.run_until_complete(t), 'a') + def test_why_are_putters_waiting(self): + # From issue #265. + + queue = asyncio.Queue(2, loop=self.loop) + + @asyncio.coroutine + def putter(item): + yield from queue.put(item) + + @asyncio.coroutine + def getter(): + yield + num = queue.qsize() + for _ in range(num): + item = queue.get_nowait() + + t0 = putter(0) + t1 = putter(1) + t2 = putter(2) + t3 = putter(3) + self.loop.run_until_complete( + asyncio.gather(getter(), t0, t1, t2, t3, loop=self.loop)) + class LifoQueueTests(_QueueTestBase): diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -81,6 +81,8 @@ Library ------- +- Issue #25233: Rewrite the guts of Queue to be more understandable and correct. + - Issue #23600: Default implementation of tzinfo.fromutc() was returning wrong results in some cases. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 28 16:46:33 2015 From: python-checkins at python.org (guido.van.rossum) Date: Mon, 28 Sep 2015 14:46:33 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Issue_=2325233=3A_Rewrite_the_guts_of_Queue_to_be_more_underst?= =?utf-8?q?andable_and_correct=2E?= Message-ID: <20150928144629.98376.19852@psf.io> https://hg.python.org/cpython/rev/a48d90049ae2 changeset: 98360:a48d90049ae2 branch: 3.5 parent: 98355:4cd3027ffc34 parent: 98359:1ea306202d5d user: Guido van Rossum date: Mon Sep 28 07:44:49 2015 -0700 summary: Issue #25233: Rewrite the guts of Queue to be more understandable and correct. (Merge 3.4->3.5.) files: Lib/asyncio/queues.py | 152 ++++---------- Lib/test/test_asyncio/test_queues.py | 55 ++++- Misc/NEWS | 2 + 3 files changed, 91 insertions(+), 118 deletions(-) diff --git a/Lib/asyncio/queues.py b/Lib/asyncio/queues.py --- a/Lib/asyncio/queues.py +++ b/Lib/asyncio/queues.py @@ -47,7 +47,7 @@ # Futures. self._getters = collections.deque() - # Futures + # Futures. self._putters = collections.deque() self._unfinished_tasks = 0 self._finished = locks.Event(loop=self._loop) @@ -67,10 +67,13 @@ # End of the overridable methods. - def __put_internal(self, item): - self._put(item) - self._unfinished_tasks += 1 - self._finished.clear() + def _wakeup_next(self, waiters): + # Wake up the next waiter (if any) that isn't cancelled. + while waiters: + waiter = waiters.popleft() + if not waiter.done(): + waiter.set_result(None) + break def __repr__(self): return '<{} at {:#x} {}>'.format( @@ -91,16 +94,6 @@ result += ' tasks={}'.format(self._unfinished_tasks) return result - def _consume_done_getters(self): - # Delete waiters at the head of the get() queue who've timed out. - while self._getters and self._getters[0].done(): - self._getters.popleft() - - def _consume_done_putters(self): - # Delete waiters at the head of the put() queue who've timed out. - while self._putters and self._putters[0].done(): - self._putters.popleft() - def qsize(self): """Number of items in the queue.""" return len(self._queue) @@ -134,47 +127,31 @@ This method is a coroutine. """ - self._consume_done_getters() - if self._getters: - assert not self._queue, ( - 'queue non-empty, why are getters waiting?') - - getter = self._getters.popleft() - self.__put_internal(item) - - # getter cannot be cancelled, we just removed done getters - getter.set_result(self._get()) - - elif self._maxsize > 0 and self._maxsize <= self.qsize(): - waiter = futures.Future(loop=self._loop) - - self._putters.append(waiter) - yield from waiter - self._put(item) - - else: - self.__put_internal(item) + while self.full(): + putter = futures.Future(loop=self._loop) + self._putters.append(putter) + try: + yield from putter + except: + putter.cancel() # Just in case putter is not done yet. + if not self.full() and not putter.cancelled(): + # We were woken up by get_nowait(), but can't take + # the call. Wake up the next in line. + self._wakeup_next(self._putters) + raise + return self.put_nowait(item) def put_nowait(self, item): """Put an item into the queue without blocking. If no free slot is immediately available, raise QueueFull. """ - self._consume_done_getters() - if self._getters: - assert not self._queue, ( - 'queue non-empty, why are getters waiting?') - - getter = self._getters.popleft() - self.__put_internal(item) - - # getter cannot be cancelled, we just removed done getters - getter.set_result(self._get()) - - elif self._maxsize > 0 and self._maxsize <= self.qsize(): + if self.full(): raise QueueFull - else: - self.__put_internal(item) + self._put(item) + self._unfinished_tasks += 1 + self._finished.clear() + self._wakeup_next(self._getters) @coroutine def get(self): @@ -184,77 +161,30 @@ This method is a coroutine. """ - self._consume_done_putters() - if self._putters: - assert self.full(), 'queue not full, why are putters waiting?' - putter = self._putters.popleft() - - # When a getter runs and frees up a slot so this putter can - # run, we need to defer the put for a tick to ensure that - # getters and putters alternate perfectly. See - # ChannelTest.test_wait. - self._loop.call_soon(putter._set_result_unless_cancelled, None) - - return self._get() - - elif self.qsize(): - return self._get() - else: - waiter = futures.Future(loop=self._loop) - self._getters.append(waiter) + while self.empty(): + getter = futures.Future(loop=self._loop) + self._getters.append(getter) try: - return (yield from waiter) - except futures.CancelledError: - # if we get CancelledError, it means someone cancelled this - # get() coroutine. But there is a chance that the waiter - # already is ready and contains an item that has just been - # removed from the queue. In this case, we need to put the item - # back into the front of the queue. This get() must either - # succeed without fault or, if it gets cancelled, it must be as - # if it never happened. - if waiter.done(): - self._put_it_back(waiter.result()) + yield from getter + except: + getter.cancel() # Just in case getter is not done yet. + if not self.empty() and not getter.cancelled(): + # We were woken up by put_nowait(), but can't take + # the call. Wake up the next in line. + self._wakeup_next(self._getters) raise - - def _put_it_back(self, item): - """ - This is called when we have a waiter to get() an item and this waiter - gets cancelled. In this case, we put the item back: wake up another - waiter or put it in the _queue. - """ - self._consume_done_getters() - if self._getters: - assert not self._queue, ( - 'queue non-empty, why are getters waiting?') - - getter = self._getters.popleft() - self.__put_internal(item) - - # getter cannot be cancelled, we just removed done getters - getter.set_result(item) - else: - self._queue.appendleft(item) + return self.get_nowait() def get_nowait(self): """Remove and return an item from the queue. Return an item if one is immediately available, else raise QueueEmpty. """ - self._consume_done_putters() - if self._putters: - assert self.full(), 'queue not full, why are putters waiting?' - putter = self._putters.popleft() - # Wake putter on next tick. - - # getter cannot be cancelled, we just removed done putters - putter.set_result(None) - - return self._get() - - elif self.qsize(): - return self._get() - else: + if self.empty(): raise QueueEmpty + item = self._get() + self._wakeup_next(self._putters) + return item def task_done(self): """Indicate that a formerly enqueued task is complete. diff --git a/Lib/test/test_asyncio/test_queues.py b/Lib/test/test_asyncio/test_queues.py --- a/Lib/test/test_asyncio/test_queues.py +++ b/Lib/test/test_asyncio/test_queues.py @@ -271,6 +271,29 @@ self.assertEqual(self.loop.run_until_complete(q.get()), 'a') self.assertEqual(self.loop.run_until_complete(q.get()), 'b') + def test_why_are_getters_waiting(self): + # From issue #268. + + @asyncio.coroutine + def consumer(queue, num_expected): + for _ in range(num_expected): + yield from queue.get() + + @asyncio.coroutine + def producer(queue, num_items): + for i in range(num_items): + yield from queue.put(i) + + queue_size = 1 + producer_num_items = 5 + q = asyncio.Queue(queue_size, loop=self.loop) + + self.loop.run_until_complete( + asyncio.gather(producer(q, producer_num_items), + consumer(q, producer_num_items), + loop=self.loop), + ) + class QueuePutTests(_QueueTestBase): @@ -377,13 +400,8 @@ loop.run_until_complete(reader3) - # reader2 will receive `2`, because it was added to the - # queue of pending readers *before* put_nowaits were called. - self.assertEqual(reader2.result(), 2) - # reader3 will receive `1`, because reader1 was cancelled - # before is had a chance to execute, and `2` was already - # pushed to reader2 by second `put_nowait`. - self.assertEqual(reader3.result(), 1) + # It is undefined in which order concurrent readers receive results. + self.assertEqual({reader2.result(), reader3.result()}, {1, 2}) def test_put_cancel_drop(self): @@ -479,6 +497,29 @@ self.loop.run_until_complete(q.put('a')) self.assertEqual(self.loop.run_until_complete(t), 'a') + def test_why_are_putters_waiting(self): + # From issue #265. + + queue = asyncio.Queue(2, loop=self.loop) + + @asyncio.coroutine + def putter(item): + yield from queue.put(item) + + @asyncio.coroutine + def getter(): + yield + num = queue.qsize() + for _ in range(num): + item = queue.get_nowait() + + t0 = putter(0) + t1 = putter(1) + t2 = putter(2) + t3 = putter(3) + self.loop.run_until_complete( + asyncio.gather(getter(), t0, t1, t2, t3, loop=self.loop)) + class LifoQueueTests(_QueueTestBase): diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -21,6 +21,8 @@ Library ------- +- Issue #25233: Rewrite the guts of Queue to be more understandable and correct. + - Issue #25203: Failed readline.set_completer_delims() no longer left the module in inconsistent state. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 28 22:36:04 2015 From: python-checkins at python.org (guido.van.rossum) Date: Mon, 28 Sep 2015 20:36:04 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2325233=3A_Rewrite_the_guts_of_Queue_to_be_more_u?= =?utf-8?q?nderstandable_and_correct=2E?= Message-ID: <20150928203603.9931.8326@psf.io> https://hg.python.org/cpython/rev/58695845e159 changeset: 98361:58695845e159 parent: 98358:e48ec5f5b82b parent: 98360:a48d90049ae2 user: Guido van Rossum date: Mon Sep 28 13:35:54 2015 -0700 summary: Issue #25233: Rewrite the guts of Queue to be more understandable and correct. (Merge 3.5->default.) files: Lib/asyncio/queues.py | 152 ++++---------- Lib/test/test_asyncio/test_queues.py | 55 ++++- 2 files changed, 89 insertions(+), 118 deletions(-) diff --git a/Lib/asyncio/queues.py b/Lib/asyncio/queues.py --- a/Lib/asyncio/queues.py +++ b/Lib/asyncio/queues.py @@ -47,7 +47,7 @@ # Futures. self._getters = collections.deque() - # Futures + # Futures. self._putters = collections.deque() self._unfinished_tasks = 0 self._finished = locks.Event(loop=self._loop) @@ -67,10 +67,13 @@ # End of the overridable methods. - def __put_internal(self, item): - self._put(item) - self._unfinished_tasks += 1 - self._finished.clear() + def _wakeup_next(self, waiters): + # Wake up the next waiter (if any) that isn't cancelled. + while waiters: + waiter = waiters.popleft() + if not waiter.done(): + waiter.set_result(None) + break def __repr__(self): return '<{} at {:#x} {}>'.format( @@ -91,16 +94,6 @@ result += ' tasks={}'.format(self._unfinished_tasks) return result - def _consume_done_getters(self): - # Delete waiters at the head of the get() queue who've timed out. - while self._getters and self._getters[0].done(): - self._getters.popleft() - - def _consume_done_putters(self): - # Delete waiters at the head of the put() queue who've timed out. - while self._putters and self._putters[0].done(): - self._putters.popleft() - def qsize(self): """Number of items in the queue.""" return len(self._queue) @@ -134,47 +127,31 @@ This method is a coroutine. """ - self._consume_done_getters() - if self._getters: - assert not self._queue, ( - 'queue non-empty, why are getters waiting?') - - getter = self._getters.popleft() - self.__put_internal(item) - - # getter cannot be cancelled, we just removed done getters - getter.set_result(self._get()) - - elif self._maxsize > 0 and self._maxsize <= self.qsize(): - waiter = futures.Future(loop=self._loop) - - self._putters.append(waiter) - yield from waiter - self._put(item) - - else: - self.__put_internal(item) + while self.full(): + putter = futures.Future(loop=self._loop) + self._putters.append(putter) + try: + yield from putter + except: + putter.cancel() # Just in case putter is not done yet. + if not self.full() and not putter.cancelled(): + # We were woken up by get_nowait(), but can't take + # the call. Wake up the next in line. + self._wakeup_next(self._putters) + raise + return self.put_nowait(item) def put_nowait(self, item): """Put an item into the queue without blocking. If no free slot is immediately available, raise QueueFull. """ - self._consume_done_getters() - if self._getters: - assert not self._queue, ( - 'queue non-empty, why are getters waiting?') - - getter = self._getters.popleft() - self.__put_internal(item) - - # getter cannot be cancelled, we just removed done getters - getter.set_result(self._get()) - - elif self._maxsize > 0 and self._maxsize <= self.qsize(): + if self.full(): raise QueueFull - else: - self.__put_internal(item) + self._put(item) + self._unfinished_tasks += 1 + self._finished.clear() + self._wakeup_next(self._getters) @coroutine def get(self): @@ -184,77 +161,30 @@ This method is a coroutine. """ - self._consume_done_putters() - if self._putters: - assert self.full(), 'queue not full, why are putters waiting?' - putter = self._putters.popleft() - - # When a getter runs and frees up a slot so this putter can - # run, we need to defer the put for a tick to ensure that - # getters and putters alternate perfectly. See - # ChannelTest.test_wait. - self._loop.call_soon(putter._set_result_unless_cancelled, None) - - return self._get() - - elif self.qsize(): - return self._get() - else: - waiter = futures.Future(loop=self._loop) - self._getters.append(waiter) + while self.empty(): + getter = futures.Future(loop=self._loop) + self._getters.append(getter) try: - return (yield from waiter) - except futures.CancelledError: - # if we get CancelledError, it means someone cancelled this - # get() coroutine. But there is a chance that the waiter - # already is ready and contains an item that has just been - # removed from the queue. In this case, we need to put the item - # back into the front of the queue. This get() must either - # succeed without fault or, if it gets cancelled, it must be as - # if it never happened. - if waiter.done(): - self._put_it_back(waiter.result()) + yield from getter + except: + getter.cancel() # Just in case getter is not done yet. + if not self.empty() and not getter.cancelled(): + # We were woken up by put_nowait(), but can't take + # the call. Wake up the next in line. + self._wakeup_next(self._getters) raise - - def _put_it_back(self, item): - """ - This is called when we have a waiter to get() an item and this waiter - gets cancelled. In this case, we put the item back: wake up another - waiter or put it in the _queue. - """ - self._consume_done_getters() - if self._getters: - assert not self._queue, ( - 'queue non-empty, why are getters waiting?') - - getter = self._getters.popleft() - self.__put_internal(item) - - # getter cannot be cancelled, we just removed done getters - getter.set_result(item) - else: - self._queue.appendleft(item) + return self.get_nowait() def get_nowait(self): """Remove and return an item from the queue. Return an item if one is immediately available, else raise QueueEmpty. """ - self._consume_done_putters() - if self._putters: - assert self.full(), 'queue not full, why are putters waiting?' - putter = self._putters.popleft() - # Wake putter on next tick. - - # getter cannot be cancelled, we just removed done putters - putter.set_result(None) - - return self._get() - - elif self.qsize(): - return self._get() - else: + if self.empty(): raise QueueEmpty + item = self._get() + self._wakeup_next(self._putters) + return item def task_done(self): """Indicate that a formerly enqueued task is complete. diff --git a/Lib/test/test_asyncio/test_queues.py b/Lib/test/test_asyncio/test_queues.py --- a/Lib/test/test_asyncio/test_queues.py +++ b/Lib/test/test_asyncio/test_queues.py @@ -271,6 +271,29 @@ self.assertEqual(self.loop.run_until_complete(q.get()), 'a') self.assertEqual(self.loop.run_until_complete(q.get()), 'b') + def test_why_are_getters_waiting(self): + # From issue #268. + + @asyncio.coroutine + def consumer(queue, num_expected): + for _ in range(num_expected): + yield from queue.get() + + @asyncio.coroutine + def producer(queue, num_items): + for i in range(num_items): + yield from queue.put(i) + + queue_size = 1 + producer_num_items = 5 + q = asyncio.Queue(queue_size, loop=self.loop) + + self.loop.run_until_complete( + asyncio.gather(producer(q, producer_num_items), + consumer(q, producer_num_items), + loop=self.loop), + ) + class QueuePutTests(_QueueTestBase): @@ -377,13 +400,8 @@ loop.run_until_complete(reader3) - # reader2 will receive `2`, because it was added to the - # queue of pending readers *before* put_nowaits were called. - self.assertEqual(reader2.result(), 2) - # reader3 will receive `1`, because reader1 was cancelled - # before is had a chance to execute, and `2` was already - # pushed to reader2 by second `put_nowait`. - self.assertEqual(reader3.result(), 1) + # It is undefined in which order concurrent readers receive results. + self.assertEqual({reader2.result(), reader3.result()}, {1, 2}) def test_put_cancel_drop(self): @@ -479,6 +497,29 @@ self.loop.run_until_complete(q.put('a')) self.assertEqual(self.loop.run_until_complete(t), 'a') + def test_why_are_putters_waiting(self): + # From issue #265. + + queue = asyncio.Queue(2, loop=self.loop) + + @asyncio.coroutine + def putter(item): + yield from queue.put(item) + + @asyncio.coroutine + def getter(): + yield + num = queue.qsize() + for _ in range(num): + item = queue.get_nowait() + + t0 = putter(0) + t1 = putter(1) + t2 = putter(2) + t3 = putter(3) + self.loop.run_until_complete( + asyncio.gather(getter(), t0, t1, t2, t3, loop=self.loop)) + class LifoQueueTests(_QueueTestBase): -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 29 00:19:09 2015 From: python-checkins at python.org (victor.stinner) Date: Mon, 28 Sep 2015 22:19:09 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2325220=3A_Add_func?= =?utf-8?q?tional_tests_to_test=5Fregrtest?= Message-ID: <20150928221908.9941.14323@psf.io> https://hg.python.org/cpython/rev/3393d86c3bb2 changeset: 98362:3393d86c3bb2 user: Victor Stinner date: Mon Sep 28 23:16:17 2015 +0200 summary: Issue #25220: Add functional tests to test_regrtest * test all available ways to run the Python test suite * test many regrtest options: --slow, --coverage, -r, -u, etc. Note: python -m test --coverage doesn't work on Windows. files: Lib/test/test_regrtest.py | 293 +++++++++++++++++++++++++- 1 files changed, 291 insertions(+), 2 deletions(-) diff --git a/Lib/test/test_regrtest.py b/Lib/test/test_regrtest.py --- a/Lib/test/test_regrtest.py +++ b/Lib/test/test_regrtest.py @@ -1,18 +1,33 @@ """ Tests of regrtest.py. + +Note: test_regrtest cannot be run twice in parallel. """ import argparse import faulthandler import getopt import os.path +import platform +import re +import subprocess +import sys +import textwrap import unittest from test import libregrtest from test import support +from test.support import script_helper + + +Py_DEBUG = hasattr(sys, 'getobjects') +ROOT_DIR = os.path.join(os.path.dirname(__file__), '..', '..') +ROOT_DIR = os.path.abspath(os.path.normpath(ROOT_DIR)) + class ParseArgsTestCase(unittest.TestCase): - - """Test regrtest's argument parsing.""" + """ + Test regrtest's argument parsing, function _parse_args(). + """ def checkError(self, args, msg): with support.captured_stderr() as err, self.assertRaises(SystemExit): @@ -272,5 +287,279 @@ self.assertEqual(ns.args, ['foo']) +class BaseTestCase(unittest.TestCase): + TEST_UNIQUE_ID = 1 + TESTNAME_PREFIX = 'test_regrtest_' + TESTNAME_REGEX = r'test_[a-z0-9_]+' + + def setUp(self): + self.testdir = os.path.join(ROOT_DIR, 'Lib', 'test') + + # When test_regrtest is interrupted by CTRL+c, it can leave + # temporary test files + remove = [entry.path + for entry in os.scandir(self.testdir) + if (entry.name.startswith(self.TESTNAME_PREFIX) + and entry.name.endswith(".py"))] + for path in remove: + print("WARNING: test_regrtest: remove %s" % path) + support.unlink(path) + + def create_test(self, name=None, code=''): + if not name: + name = 'noop%s' % BaseTestCase.TEST_UNIQUE_ID + BaseTestCase.TEST_UNIQUE_ID += 1 + + # test_regrtest cannot be run twice in parallel because + # of setUp() and create_test() + name = self.TESTNAME_PREFIX + "%s_%s" % (os.getpid(), name) + path = os.path.join(self.testdir, name + '.py') + + self.addCleanup(support.unlink, path) + # Use 'x' mode to ensure that we do not override existing tests + with open(path, 'x', encoding='utf-8') as fp: + fp.write(code) + return name + + def regex_search(self, regex, output): + match = re.search(regex, output, re.MULTILINE) + if not match: + self.fail("%r not found in %r" % (regex, output)) + return match + + def check_line(self, output, regex): + regex = re.compile(r'^' + regex, re.MULTILINE) + self.assertRegex(output, regex) + + def parse_executed_tests(self, output): + parser = re.finditer(r'^\[[0-9]+/[0-9]+\] (%s)$' % self.TESTNAME_REGEX, + output, + re.MULTILINE) + return set(match.group(1) for match in parser) + + def check_executed_tests(self, output, tests, skipped=None): + if isinstance(tests, str): + tests = [tests] + executed = self.parse_executed_tests(output) + self.assertEqual(executed, set(tests), output) + ntest = len(tests) + if skipped: + if isinstance(skipped, str): + skipped = [skipped] + nskipped = len(skipped) + + plural = 's' if nskipped != 1 else '' + names = ' '.join(sorted(skipped)) + expected = (r'%s test%s skipped:\n %s$' + % (nskipped, plural, names)) + self.check_line(output, expected) + + ok = ntest - nskipped + if ok: + self.check_line(output, r'%s test OK\.$' % ok) + else: + self.check_line(output, r'All %s tests OK\.$' % ntest) + + def parse_random_seed(self, output): + match = self.regex_search(r'Using random seed ([0-9]+)', output) + randseed = int(match.group(1)) + self.assertTrue(0 <= randseed <= 10000000, randseed) + return randseed + + +class ProgramsTestCase(BaseTestCase): + """ + Test various ways to run the Python test suite. Use options close + to options used on the buildbot. + """ + + NTEST = 4 + + def setUp(self): + super().setUp() + + # Create NTEST tests doing nothing + self.tests = [self.create_test() for index in range(self.NTEST)] + + self.python_args = ['-Wd', '-E', '-bb'] + self.regrtest_args = ['-uall', '-rwW', '--timeout', '3600', '-j4'] + if sys.platform == 'win32': + self.regrtest_args.append('-n') + + def check_output(self, output): + self.parse_random_seed(output) + self.check_executed_tests(output, self.tests) + + def run_tests(self, args): + res = script_helper.assert_python_ok(*args) + output = os.fsdecode(res.out) + self.check_output(output) + + def test_script_regrtest(self): + # Lib/test/regrtest.py + script = os.path.join(ROOT_DIR, 'Lib', 'test', 'regrtest.py') + + args = [*self.python_args, script, *self.regrtest_args, *self.tests] + self.run_tests(args) + + def test_module_test(self): + # -m test + args = [*self.python_args, '-m', 'test', + *self.regrtest_args, *self.tests] + self.run_tests(args) + + def test_module_regrtest(self): + # -m test.regrtest + args = [*self.python_args, '-m', 'test.regrtest', + *self.regrtest_args, *self.tests] + self.run_tests(args) + + def test_module_autotest(self): + # -m test.autotest + args = [*self.python_args, '-m', 'test.autotest', + *self.regrtest_args, *self.tests] + self.run_tests(args) + + def test_module_from_test_autotest(self): + # from test import autotest + code = 'from test import autotest' + args = [*self.python_args, '-c', code, + *self.regrtest_args, *self.tests] + self.run_tests(args) + + def test_script_autotest(self): + # Lib/test/autotest.py + script = os.path.join(ROOT_DIR, 'Lib', 'test', 'autotest.py') + args = [*self.python_args, script, *self.regrtest_args, *self.tests] + self.run_tests(args) + + def test_tools_script_run_tests(self): + # Tools/scripts/run_tests.py + script = os.path.join(ROOT_DIR, 'Tools', 'scripts', 'run_tests.py') + self.run_tests([script, *self.tests]) + + def run_rt_bat(self, script, *args): + rt_args = [] + if platform.architecture()[0] == '64bit': + rt_args.append('-x64') # 64-bit build + if Py_DEBUG: + rt_args.append('-d') # Debug build + + args = [script, *rt_args, *args] + proc = subprocess.run(args, + check=True, universal_newlines=True, + input='', + stdout=subprocess.PIPE, stderr=subprocess.PIPE) + self.check_output(proc.stdout) + + @unittest.skipUnless(sys.platform == 'win32', 'Windows only') + def test_tools_buildbot_test(self): + # Tools\buildbot\test.bat + script = os.path.join(ROOT_DIR, 'Tools', 'buildbot', 'test.bat') + self.run_rt_bat(script, *self.tests) + + @unittest.skipUnless(sys.platform == 'win32', 'Windows only') + def test_pcbuild_rt(self): + # PCbuild\rt.bat + script = os.path.join(ROOT_DIR, r'PCbuild\rt.bat') + # -q: quick, don't run tests twice + rt_args = ["-q"] + self.run_rt_bat(script, *rt_args, *self.regrtest_args, *self.tests) + + +class ArgsTestCase(BaseTestCase): + """ + Test arguments of the Python test suite. + """ + + def run_tests(self, *args): + args = ['-m', 'test', *args] + res = script_helper.assert_python_ok(*args) + return os.fsdecode(res.out) + + def test_resources(self): + # test -u command line option + tests = {} + for resource in ('audio', 'network'): + code = 'from test import support\nsupport.requires(%r)' % resource + tests[resource] = self.create_test(resource, code) + test_names = sorted(tests.values()) + + # -u all: 2 resources enabled + output = self.run_tests('-u', 'all', *test_names) + self.check_executed_tests(output, test_names) + + # -u audio: 1 resource enabled + output = self.run_tests('-uaudio', *test_names) + self.check_executed_tests(output, test_names, + skipped=tests['network']) + + # no option: 0 resources enabled + output = self.run_tests(*test_names) + self.check_executed_tests(output, test_names, + skipped=test_names) + + def test_random(self): + # test -r and --randseed command line option + code = textwrap.dedent(""" + import random + print("TESTRANDOM: %s" % random.randint(1, 1000)) + """) + test = self.create_test('random', code) + + # first run to get the output with the random seed + output = self.run_tests('-r', test) + randseed = self.parse_random_seed(output) + match = self.regex_search(r'TESTRANDOM: ([0-9]+)', output) + test_random = int(match.group(1)) + + # try to reproduce with the random seed + output = self.run_tests('-r', '--randseed=%s' % randseed, test) + randseed2 = self.parse_random_seed(output) + self.assertEqual(randseed2, randseed) + + match = self.regex_search(r'TESTRANDOM: ([0-9]+)', output) + test_random2 = int(match.group(1)) + self.assertEqual(test_random2, test_random) + + def test_fromfile(self): + # test --fromfile + tests = [self.create_test() for index in range(5)] + + # Write the list of files using a format similar to regrtest output: + # [1/2] test_1 + # [2/2] test_2 + filename = support.TESTFN + self.addCleanup(support.unlink, filename) + with open(filename, "w") as fp: + for index, name in enumerate(tests, 1): + print("[%s/%s] %s" % (index, len(tests), name), file=fp) + + output = self.run_tests('--fromfile', filename) + self.check_executed_tests(output, tests) + + def test_slow(self): + # test --slow + tests = [self.create_test() for index in range(3)] + output = self.run_tests("--slow", *tests) + self.check_executed_tests(output, tests) + regex = ('10 slowest tests:\n' + '(?:%s: [0-9]+\.[0-9]+s\n){%s}' + % (self.TESTNAME_REGEX, len(tests))) + self.check_line(output, regex) + + @unittest.skipIf(sys.platform == 'win32', + "FIXME: coverage doesn't work on Windows") + def test_coverage(self): + # test --coverage + test = self.create_test() + output = self.run_tests("--coverage", test) + executed = self.parse_executed_tests(output) + self.assertEqual(executed, {test}, output) + regex = ('lines +cov% +module +\(path\)\n' + '(?: *[0-9]+ *[0-9]{1,2}% *[^ ]+ +\([^)]+\)+)+') + self.check_line(output, regex) + + if __name__ == '__main__': unittest.main() -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 29 01:10:31 2015 From: python-checkins at python.org (victor.stinner) Date: Mon, 28 Sep 2015 23:10:31 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Fix_test=5Fregrtest=2Etest?= =?utf-8?q?=5Ftools=5Fbuildbot=5Ftest=28=29?= Message-ID: <20150928231030.9951.9119@psf.io> https://hg.python.org/cpython/rev/a9fe8cd339b1 changeset: 98363:a9fe8cd339b1 user: Victor Stinner date: Tue Sep 29 01:02:37 2015 +0200 summary: Fix test_regrtest.test_tools_buildbot_test() Issue #25220: Fix test_regrtest.test_tools_buildbot_test() on release build (on Windows), pass "+d" option to test.bat. files: Lib/test/test_regrtest.py | 25 +++++++++++++------------ 1 files changed, 13 insertions(+), 12 deletions(-) diff --git a/Lib/test/test_regrtest.py b/Lib/test/test_regrtest.py --- a/Lib/test/test_regrtest.py +++ b/Lib/test/test_regrtest.py @@ -438,14 +438,7 @@ script = os.path.join(ROOT_DIR, 'Tools', 'scripts', 'run_tests.py') self.run_tests([script, *self.tests]) - def run_rt_bat(self, script, *args): - rt_args = [] - if platform.architecture()[0] == '64bit': - rt_args.append('-x64') # 64-bit build - if Py_DEBUG: - rt_args.append('-d') # Debug build - - args = [script, *rt_args, *args] + def run_batch(self, *args): proc = subprocess.run(args, check=True, universal_newlines=True, input='', @@ -456,15 +449,23 @@ def test_tools_buildbot_test(self): # Tools\buildbot\test.bat script = os.path.join(ROOT_DIR, 'Tools', 'buildbot', 'test.bat') - self.run_rt_bat(script, *self.tests) + test_args = [] + if platform.architecture()[0] == '64bit': + test_args.append('-x64') # 64-bit build + if not Py_DEBUG: + test_args.append('+d') # Release build, use python.exe + self.run_batch(script, *test_args, *self.tests) @unittest.skipUnless(sys.platform == 'win32', 'Windows only') def test_pcbuild_rt(self): # PCbuild\rt.bat script = os.path.join(ROOT_DIR, r'PCbuild\rt.bat') - # -q: quick, don't run tests twice - rt_args = ["-q"] - self.run_rt_bat(script, *rt_args, *self.regrtest_args, *self.tests) + rt_args = ["-q"] # Quick, don't run tests twice + if platform.architecture()[0] == '64bit': + rt_args.append('-x64') # 64-bit build + if Py_DEBUG: + rt_args.append('-d') # Debug build, use python_d.exe + self.run_batch(script, *rt_args, *self.regrtest_args, *self.tests) class ArgsTestCase(BaseTestCase): -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 29 01:54:17 2015 From: python-checkins at python.org (guido.van.rossum) Date: Mon, 28 Sep 2015 23:54:17 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Correct_Misc/NEWS_about_asyncio=2EQueue_rewrite=2E?= Message-ID: <20150928235417.31203.99028@psf.io> https://hg.python.org/cpython/rev/977061c2334e changeset: 98365:977061c2334e branch: 3.5 parent: 98360:a48d90049ae2 parent: 98364:efe112889889 user: Guido van Rossum date: Mon Sep 28 16:51:59 2015 -0700 summary: Correct Misc/NEWS about asyncio.Queue rewrite. files: Misc/NEWS | 5 +---- 1 files changed, 1 insertions(+), 4 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -21,7 +21,7 @@ Library ------- -- Issue #25233: Rewrite the guts of Queue to be more understandable and correct. +- Issue #25233: Rewrite the guts of asyncio.Queue to be more understandable and correct. - Issue #25203: Failed readline.set_completer_delims() no longer left the module in inconsistent state. @@ -342,9 +342,6 @@ - Issue #17527: Add PATCH to wsgiref.validator. Patch from Luca Sbardella. -- Issue #23812: Fix asyncio.Queue.get() to avoid loosing items on cancellation. - Patch by Gustavo J. A. M. Carneiro. - - Issue #24791: Fix grammar regression for call syntax: 'g(*a or b)'. IDLE -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 29 01:54:17 2015 From: python-checkins at python.org (guido.van.rossum) Date: Mon, 28 Sep 2015 23:54:17 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_Correct_Misc/N?= =?utf-8?q?EWS_about_asyncio=2EQueue_rewrite=2E?= Message-ID: <20150928235417.98362.96461@psf.io> https://hg.python.org/cpython/rev/efe112889889 changeset: 98364:efe112889889 branch: 3.4 parent: 98359:1ea306202d5d user: Guido van Rossum date: Mon Sep 28 16:50:38 2015 -0700 summary: Correct Misc/NEWS about asyncio.Queue rewrite. files: Misc/NEWS | 5 +---- 1 files changed, 1 insertions(+), 4 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -75,13 +75,10 @@ - Issue #21354: PyCFunction_New function is exposed by python DLL again. -- Issue #23812: Fix asyncio.Queue.get() to avoid loosing items on cancellation. - Patch by Gustavo J. A. M. Carneiro. - Library ------- -- Issue #25233: Rewrite the guts of Queue to be more understandable and correct. +- Issue #25233: Rewrite the guts of asyncio.Queue to be more understandable and correct. - Issue #23600: Default implementation of tzinfo.fromutc() was returning wrong results in some cases. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 29 01:54:17 2015 From: python-checkins at python.org (guido.van.rossum) Date: Mon, 28 Sep 2015 23:54:17 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Correct_Misc/NEWS_about_asyncio=2EQueue_rewrite=2E?= Message-ID: <20150928235417.82658.90126@psf.io> https://hg.python.org/cpython/rev/74a9f0d242e9 changeset: 98366:74a9f0d242e9 parent: 98363:a9fe8cd339b1 parent: 98365:977061c2334e user: Guido van Rossum date: Mon Sep 28 16:53:44 2015 -0700 summary: Correct Misc/NEWS about asyncio.Queue rewrite. files: Misc/NEWS | 5 ++--- 1 files changed, 2 insertions(+), 3 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -152,6 +152,8 @@ Library ------- +- Issue #25233: Rewrite the guts of asyncio.Queue to be more understandable and correct. + - Issue #23600: Default implementation of tzinfo.fromutc() was returning wrong results in some cases. @@ -449,9 +451,6 @@ - Issue #17527: Add PATCH to wsgiref.validator. Patch from Luca Sbardella. -- Issue #23812: Fix asyncio.Queue.get() to avoid loosing items on cancellation. - Patch by Gustavo J. A. M. Carneiro. - - Issue #24791: Fix grammar regression for call syntax: 'g(*a or b)'. IDLE -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 29 05:44:54 2015 From: python-checkins at python.org (terry.reedy) Date: Tue, 29 Sep 2015 03:44:54 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_Add_recent_IDL?= =?utf-8?q?E_NEWS_items=2E?= Message-ID: <20150929034454.31201.36159@psf.io> https://hg.python.org/cpython/rev/bc894f28d9b5 changeset: 98368:bc894f28d9b5 branch: 3.4 parent: 98364:efe112889889 user: Terry Jan Reedy date: Mon Sep 28 23:38:57 2015 -0400 summary: Add recent IDLE NEWS items. files: Lib/idlelib/NEWS.txt | 40 +++++++++++++++++++++++++++++-- Misc/NEWS | 36 ++++++++++++++++++++++++++++- 2 files changed, 72 insertions(+), 4 deletions(-) diff --git a/Lib/idlelib/NEWS.txt b/Lib/idlelib/NEWS.txt --- a/Lib/idlelib/NEWS.txt +++ b/Lib/idlelib/NEWS.txt @@ -2,8 +2,42 @@ ========================= *Release date: 2015-??-??* +- Issue #24972: Inactive selection background now matches active selection + background, as configured by user, on all systems. Found items are now + always highlighted on Windows. Initial patch by Mark Roseman. + +- Issue #24570: Idle: make calltip and completion boxes appear on Macs + affected by a tk regression. Initial patch by Mark Roseman. + +- Issue #24988: Idle ScrolledList context menus (used in debugger) + now work on Mac Aqua. Patch by Mark Roseman. + +- Issue #24801: Make right-click for context menu work on Mac Aqua. + Patch by Mark Roseman. + +- Issue #25173: Associate tkinter messageboxes with a specific widget. + For Mac OSX, make them a 'sheet'. Patch by Mark Roseman. + +- Issue #25198: Enhance the initial html viewer now used for Idle Help. + * Properly indent fixed-pitch text (patch by Mark Roseman). + * Give code snippet a very Sphinx-like light blueish-gray background. + * Re-use initial width and height set by users for shell and editor. + * When the Table of Contents (TOC) menu is used, put the section header + at the top of the screen. + +- Issue #25225: Condense and rewrite Idle doc section on text colors. + +- Issue #21995: Explain some differences between IDLE and console Python. + +- Issue #22820: Explain need for *print* when running file from Idle editor. + +- Issue #25224: Doc: augment Idle feature list and no-subprocess section. + +- Issue #25219: Update doc for Idle command line options. + Some were missing and notes were not correct. + - Issue #24861: Most of idlelib is private and subject to change. - Use idleib.idle.* to start Idle. See idlelib.__init__.__doc__. + Use idleib.idle.* to start Idle. See idlelib.__init__.__doc__. - Issue #25199: Idle: add synchronization comments for future maintainers. @@ -232,7 +266,7 @@ - Issue #2665: On Windows, an IDLE installation upgraded from an old version would not start if a custom theme was defined. - + What's New in IDLE 2.7? (UNRELEASED, but merged into 3.1 releases above.) ======================= *Release date: XX-XXX-2010* @@ -524,7 +558,7 @@ - The remote procedure call module rpc.py can now access data attributes of remote registered objects. Changes to these attributes are local, however. - + What's New in IDLE 1.1? ======================= *Release date: 30-NOV-2004* diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -456,8 +456,42 @@ IDLE ---- +- Issue #24972: Inactive selection background now matches active selection + background, as configured by user, on all systems. Found items are now + always highlighted on Windows. Initial patch by Mark Roseman. + +- Issue #24570: Idle: make calltip and completion boxes appear on Macs + affected by a tk regression. Initial patch by Mark Roseman. + +- Issue #24988: Idle ScrolledList context menus (used in debugger) + now work on Mac Aqua. Patch by Mark Roseman. + +- Issue #24801: Make right-click for context menu work on Mac Aqua. + Patch by Mark Roseman. + +- Issue #25173: Associate tkinter messageboxes with a specific widget. + For Mac OSX, make them a 'sheet'. Patch by Mark Roseman. + +- Issue #25198: Enhance the initial html viewer now used for Idle Help. + * Properly indent fixed-pitch text (patch by Mark Roseman). + * Give code snippet a very Sphinx-like light blueish-gray background. + * Re-use initial width and height set by users for shell and editor. + * When the Table of Contents (TOC) menu is used, put the section header + at the top of the screen. + +- Issue #25225: Condense and rewrite Idle doc section on text colors. + +- Issue #21995: Explain some differences between IDLE and console Python. + +- Issue #22820: Explain need for *print* when running file from Idle editor. + +- Issue #25224: Doc: augment Idle feature list and no-subprocess section. + +- Issue #25219: Update doc for Idle command line options. + Some were missing and notes were not correct. + - Issue #24861: Most of idlelib is private and subject to change. - Use idleib.idle.* to start Idle. See idlelib.__init__.__doc__. + Use idleib.idle.* to start Idle. See idlelib.__init__.__doc__. - Issue #25199: Idle: add synchronization comments for future maintainers. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 29 05:44:54 2015 From: python-checkins at python.org (terry.reedy) Date: Tue, 29 Sep 2015 03:44:54 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_IDLE_NEWS_items=2E?= Message-ID: <20150929034454.9949.84624@psf.io> https://hg.python.org/cpython/rev/d27422bbde91 changeset: 98369:d27422bbde91 branch: 3.5 parent: 98365:977061c2334e parent: 98368:bc894f28d9b5 user: Terry Jan Reedy date: Mon Sep 28 23:42:56 2015 -0400 summary: IDLE NEWS items. files: Lib/idlelib/NEWS.txt | 36 +++++++++++++++++++++++++++++++- Misc/NEWS | 36 +++++++++++++++++++++++++++++++- 2 files changed, 70 insertions(+), 2 deletions(-) diff --git a/Lib/idlelib/NEWS.txt b/Lib/idlelib/NEWS.txt --- a/Lib/idlelib/NEWS.txt +++ b/Lib/idlelib/NEWS.txt @@ -2,8 +2,42 @@ ========================= *Release date: 2015-09-13* +- Issue #24972: Inactive selection background now matches active selection + background, as configured by user, on all systems. Found items are now + always highlighted on Windows. Initial patch by Mark Roseman. + +- Issue #24570: Idle: make calltip and completion boxes appear on Macs + affected by a tk regression. Initial patch by Mark Roseman. + +- Issue #24988: Idle ScrolledList context menus (used in debugger) + now work on Mac Aqua. Patch by Mark Roseman. + +- Issue #24801: Make right-click for context menu work on Mac Aqua. + Patch by Mark Roseman. + +- Issue #25173: Associate tkinter messageboxes with a specific widget. + For Mac OSX, make them a 'sheet'. Patch by Mark Roseman. + +- Issue #25198: Enhance the initial html viewer now used for Idle Help. + * Properly indent fixed-pitch text (patch by Mark Roseman). + * Give code snippet a very Sphinx-like light blueish-gray background. + * Re-use initial width and height set by users for shell and editor. + * When the Table of Contents (TOC) menu is used, put the section header + at the top of the screen. + +- Issue #25225: Condense and rewrite Idle doc section on text colors. + +- Issue #21995: Explain some differences between IDLE and console Python. + +- Issue #22820: Explain need for *print* when running file from Idle editor. + +- Issue #25224: Doc: augment Idle feature list and no-subprocess section. + +- Issue #25219: Update doc for Idle command line options. + Some were missing and notes were not correct. + - Issue #24861: Most of idlelib is private and subject to change. - Use idleib.idle.* to start Idle. See idlelib.__init__.__doc__. + Use idleib.idle.* to start Idle. See idlelib.__init__.__doc__. - Issue #25199: Idle: add synchronization comments for future maintainers. diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -123,8 +123,42 @@ IDLE ---- +- Issue #24972: Inactive selection background now matches active selection + background, as configured by user, on all systems. Found items are now + always highlighted on Windows. Initial patch by Mark Roseman. + +- Issue #24570: Idle: make calltip and completion boxes appear on Macs + affected by a tk regression. Initial patch by Mark Roseman. + +- Issue #24988: Idle ScrolledList context menus (used in debugger) + now work on Mac Aqua. Patch by Mark Roseman. + +- Issue #24801: Make right-click for context menu work on Mac Aqua. + Patch by Mark Roseman. + +- Issue #25173: Associate tkinter messageboxes with a specific widget. + For Mac OSX, make them a 'sheet'. Patch by Mark Roseman. + +- Issue #25198: Enhance the initial html viewer now used for Idle Help. + * Properly indent fixed-pitch text (patch by Mark Roseman). + * Give code snippet a very Sphinx-like light blueish-gray background. + * Re-use initial width and height set by users for shell and editor. + * When the Table of Contents (TOC) menu is used, put the section header + at the top of the screen. + +- Issue #25225: Condense and rewrite Idle doc section on text colors. + +- Issue #21995: Explain some differences between IDLE and console Python. + +- Issue #22820: Explain need for *print* when running file from Idle editor. + +- Issue #25224: Doc: augment Idle feature list and no-subprocess section. + +- Issue #25219: Update doc for Idle command line options. + Some were missing and notes were not correct. + - Issue #24861: Most of idlelib is private and subject to change. - Use idleib.idle.* to start Idle. See idlelib.__init__.__doc__. + Use idleib.idle.* to start Idle. See idlelib.__init__.__doc__. - Issue #25199: Idle: add synchronization comments for future maintainers. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 29 05:44:54 2015 From: python-checkins at python.org (terry.reedy) Date: Tue, 29 Sep 2015 03:44:54 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_IDLE_NEWS=2E?= Message-ID: <20150929034454.81637.9796@psf.io> https://hg.python.org/cpython/rev/4df9ae2153fe changeset: 98370:4df9ae2153fe parent: 98366:74a9f0d242e9 parent: 98369:d27422bbde91 user: Terry Jan Reedy date: Mon Sep 28 23:44:37 2015 -0400 summary: IDLE NEWS. files: Lib/idlelib/NEWS.txt | 36 +++++++++++++++++++++++++++++++- Misc/NEWS | 36 +++++++++++++++++++++++++++++++- 2 files changed, 70 insertions(+), 2 deletions(-) diff --git a/Lib/idlelib/NEWS.txt b/Lib/idlelib/NEWS.txt --- a/Lib/idlelib/NEWS.txt +++ b/Lib/idlelib/NEWS.txt @@ -2,8 +2,42 @@ =========================== *Release date: 2017?* +- Issue #24972: Inactive selection background now matches active selection + background, as configured by user, on all systems. Found items are now + always highlighted on Windows. Initial patch by Mark Roseman. + +- Issue #24570: Idle: make calltip and completion boxes appear on Macs + affected by a tk regression. Initial patch by Mark Roseman. + +- Issue #24988: Idle ScrolledList context menus (used in debugger) + now work on Mac Aqua. Patch by Mark Roseman. + +- Issue #24801: Make right-click for context menu work on Mac Aqua. + Patch by Mark Roseman. + +- Issue #25173: Associate tkinter messageboxes with a specific widget. + For Mac OSX, make them a 'sheet'. Patch by Mark Roseman. + +- Issue #25198: Enhance the initial html viewer now used for Idle Help. + * Properly indent fixed-pitch text (patch by Mark Roseman). + * Give code snippet a very Sphinx-like light blueish-gray background. + * Re-use initial width and height set by users for shell and editor. + * When the Table of Contents (TOC) menu is used, put the section header + at the top of the screen. + +- Issue #25225: Condense and rewrite Idle doc section on text colors. + +- Issue #21995: Explain some differences between IDLE and console Python. + +- Issue #22820: Explain need for *print* when running file from Idle editor. + +- Issue #25224: Doc: augment Idle feature list and no-subprocess section. + +- Issue #25219: Update doc for Idle command line options. + Some were missing and notes were not correct. + - Issue #24861: Most of idlelib is private and subject to change. - Use idleib.idle.* to start Idle. See idlelib.__init__.__doc__. + Use idleib.idle.* to start Idle. See idlelib.__init__.__doc__. - Issue #25199: Idle: add synchronization comments for future maintainers. diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -92,8 +92,42 @@ IDLE ---- +- Issue #24972: Inactive selection background now matches active selection + background, as configured by user, on all systems. Found items are now + always highlighted on Windows. Initial patch by Mark Roseman. + +- Issue #24570: Idle: make calltip and completion boxes appear on Macs + affected by a tk regression. Initial patch by Mark Roseman. + +- Issue #24988: Idle ScrolledList context menus (used in debugger) + now work on Mac Aqua. Patch by Mark Roseman. + +- Issue #24801: Make right-click for context menu work on Mac Aqua. + Patch by Mark Roseman. + +- Issue #25173: Associate tkinter messageboxes with a specific widget. + For Mac OSX, make them a 'sheet'. Patch by Mark Roseman. + +- Issue #25198: Enhance the initial html viewer now used for Idle Help. + * Properly indent fixed-pitch text (patch by Mark Roseman). + * Give code snippet a very Sphinx-like light blueish-gray background. + * Re-use initial width and height set by users for shell and editor. + * When the Table of Contents (TOC) menu is used, put the section header + at the top of the screen. + +- Issue #25225: Condense and rewrite Idle doc section on text colors. + +- Issue #21995: Explain some differences between IDLE and console Python. + +- Issue #22820: Explain need for *print* when running file from Idle editor. + +- Issue #25224: Doc: augment Idle feature list and no-subprocess section. + +- Issue #25219: Update doc for Idle command line options. + Some were missing and notes were not correct. + - Issue #24861: Most of idlelib is private and subject to change. - Use idleib.idle.* to start Idle. See idlelib.__init__.__doc__. + Use idleib.idle.* to start Idle. See idlelib.__init__.__doc__. - Issue #25199: Idle: add synchronization comments for future maintainers. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 29 05:44:54 2015 From: python-checkins at python.org (terry.reedy) Date: Tue, 29 Sep 2015 03:44:54 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_Add_recent_IDL?= =?utf-8?q?E_NEWS_items=2E__Move_Build_sectios_down=2E?= Message-ID: <20150929034454.9939.89250@psf.io> https://hg.python.org/cpython/rev/f9ac87a1adc3 changeset: 98367:f9ac87a1adc3 branch: 2.7 parent: 98357:ea91991c7db5 user: Terry Jan Reedy date: Mon Sep 28 23:38:46 2015 -0400 summary: Add recent IDLE NEWS items. Move Build sectios down. files: Lib/idlelib/NEWS.txt | 38 ++++++++++++++++- Misc/NEWS | 70 +++++++++++++++++++++++-------- 2 files changed, 88 insertions(+), 20 deletions(-) diff --git a/Lib/idlelib/NEWS.txt b/Lib/idlelib/NEWS.txt --- a/Lib/idlelib/NEWS.txt +++ b/Lib/idlelib/NEWS.txt @@ -2,8 +2,42 @@ ========================= *Release date: +- Issue #24972: Inactive selection background now matches active selection + background, as configured by user, on all systems. Found items are now + always highlighted on Windows. Initial patch by Mark Roseman. + +- Issue #24570: Idle: make calltip and completion boxes appear on Macs + affected by a tk regression. Initial patch by Mark Roseman. + +- Issue #24988: Idle ScrolledList context menus (used in debugger) + now work on Mac Aqua. Patch by Mark Roseman. + +- Issue #24801: Make right-click for context menu work on Mac Aqua. + Patch by Mark Roseman. + +- Issue #25173: Associate tkinter messageboxes with a specific widget. + For Mac OSX, make them a 'sheet'. Patch by Mark Roseman. + +- Issue #25198: Enhance the initial html viewer now used for Idle Help. + * Properly indent fixed-pitch text (patch by Mark Roseman). + * Give code snippet a very Sphinx-like light blueish-gray background. + * Re-use initial width and height set by users for shell and editor. + * When the Table of Contents (TOC) menu is used, put the section header + at the top of the screen. + +- Issue #25225: Condense and rewrite Idle doc section on text colors. + +- Issue #21995: Explain some differences between IDLE and console Python. + +- Issue #22820: Explain need for *print* when running file from Idle editor. + +- Issue #25224: Doc: augment Idle feature list and no-subprocess section. + +- Issue #25219: Update doc for Idle command line options. + Some were missing and notes were not correct. + - Issue #24861: Most of idlelib is private and subject to change. - Use idleib.idle.* to start Idle. See idlelib.__init__.__doc__. + Use idleib.idle.* to start Idle. See idlelib.__init__.__doc__. - Issue #25199: Idle: add synchronization comments for future maintainers. @@ -629,7 +663,7 @@ - The remote procedure call module rpc.py can now access data attributes of remote registered objects. Changes to these attributes are local, however. - + What's New in IDLE 1.1? ======================= *Release date: 30-NOV-2004* diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -156,28 +156,45 @@ - Issue #24134: Reverted issue #24134 changes. -Build ------ - -- Issue #24915: When doing a PGO build, the test suite is now used instead of - pybench; Clang support was also added as part off this work. Initial patch by - Alecsandru Patrascu of Intel. - -- Issue #24986: It is now possible to build Python on Windows without errors - when external libraries are not available. - -- Issue #24508: Backported the MSBuild project files from Python 3.5. The - backported files replace the old project files in PCbuild; the old files moved - to PC/VS9.0 and remain supported. - -- Issue #24603: Update Windows builds and OS X 10.5 installer to use OpenSSL - 1.0.2d. - IDLE ---- +- Issue #24972: Inactive selection background now matches active selection + background, as configured by user, on all systems. Found items are now + always highlighted on Windows. Initial patch by Mark Roseman. + +- Issue #24570: Idle: make calltip and completion boxes appear on Macs + affected by a tk regression. Initial patch by Mark Roseman. + +- Issue #24988: Idle ScrolledList context menus (used in debugger) + now work on Mac Aqua. Patch by Mark Roseman. + +- Issue #24801: Make right-click for context menu work on Mac Aqua. + Patch by Mark Roseman. + +- Issue #25173: Associate tkinter messageboxes with a specific widget. + For Mac OSX, make them a 'sheet'. Patch by Mark Roseman. + +- Issue #25198: Enhance the initial html viewer now used for Idle Help. + * Properly indent fixed-pitch text (patch by Mark Roseman). + * Give code snippet a very Sphinx-like light blueish-gray background. + * Re-use initial width and height set by users for shell and editor. + * When the Table of Contents (TOC) menu is used, put the section header + at the top of the screen. + +- Issue #25225: Condense and rewrite Idle doc section on text colors. + +- Issue #21995: Explain some differences between IDLE and console Python. + +- Issue #22820: Explain need for *print* when running file from Idle editor. + +- Issue #25224: Doc: augment Idle feature list and no-subprocess section. + +- Issue #25219: Update doc for Idle command line options. + Some were missing and notes were not correct. + - Issue #24861: Most of idlelib is private and subject to change. - Use idleib.idle.* to start Idle. See idlelib.__init__.__doc__. + Use idleib.idle.* to start Idle. See idlelib.__init__.__doc__. - Issue #25199: Idle: add synchronization comments for future maintainers. @@ -233,6 +250,23 @@ - PCbuild\rt.bat now accepts an unlimited number of arguments to pass along to regrtest.py. Previously there was a limit of 9. +Build +----- + +- Issue #24915: When doing a PGO build, the test suite is now used instead of + pybench; Clang support was also added as part off this work. Initial patch by + Alecsandru Patrascu of Intel. + +- Issue #24986: It is now possible to build Python on Windows without errors + when external libraries are not available. + +- Issue #24508: Backported the MSBuild project files from Python 3.5. The + backported files replace the old project files in PCbuild; the old files moved + to PC/VS9.0 and remain supported. + +- Issue #24603: Update Windows builds and OS X 10.5 installer to use OpenSSL + 1.0.2d. + Windows ------- -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 29 07:04:32 2015 From: python-checkins at python.org (terry.reedy) Date: Tue, 29 Sep 2015 05:04:32 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogbWVyZ2UgMy41?= Message-ID: <20150929050432.31183.83706@psf.io> https://hg.python.org/cpython/rev/da7b11c0cd38 changeset: 98374:da7b11c0cd38 parent: 98370:4df9ae2153fe parent: 98373:32d89683015c user: Terry Jan Reedy date: Tue Sep 29 01:04:08 2015 -0400 summary: merge 3.5 files: Misc/NEWS | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -113,7 +113,7 @@ * Give code snippet a very Sphinx-like light blueish-gray background. * Re-use initial width and height set by users for shell and editor. * When the Table of Contents (TOC) menu is used, put the section header - at the top of the screen. + at the top of the screen. - Issue #25225: Condense and rewrite Idle doc section on text colors. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 29 07:04:33 2015 From: python-checkins at python.org (terry.reedy) Date: Tue, 29 Sep 2015 05:04:33 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_Remove_indent_?= =?utf-8?q?in_news_item=2E_Error_when_building_3=2Ex_docs=2E?= Message-ID: <20150929050432.31187.31916@psf.io> https://hg.python.org/cpython/rev/992fa135ce8c changeset: 98372:992fa135ce8c branch: 3.4 parent: 98368:bc894f28d9b5 user: Terry Jan Reedy date: Tue Sep 29 01:00:31 2015 -0400 summary: Remove indent in news item. Error when building 3.x docs. files: Misc/NEWS | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -477,7 +477,7 @@ * Give code snippet a very Sphinx-like light blueish-gray background. * Re-use initial width and height set by users for shell and editor. * When the Table of Contents (TOC) menu is used, put the section header - at the top of the screen. + at the top of the screen. - Issue #25225: Condense and rewrite Idle doc section on text colors. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 29 07:04:33 2015 From: python-checkins at python.org (terry.reedy) Date: Tue, 29 Sep 2015 05:04:33 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_Remove_indent_?= =?utf-8?q?in_news_item=2E_Error_when_building_3=2Ex_docs=2E?= Message-ID: <20150929050432.16581.81330@psf.io> https://hg.python.org/cpython/rev/89df4e23ec5f changeset: 98371:89df4e23ec5f branch: 2.7 parent: 98367:f9ac87a1adc3 user: Terry Jan Reedy date: Tue Sep 29 01:00:25 2015 -0400 summary: Remove indent in news item. Error when building 3.x docs. files: Misc/NEWS | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -180,7 +180,7 @@ * Give code snippet a very Sphinx-like light blueish-gray background. * Re-use initial width and height set by users for shell and editor. * When the Table of Contents (TOC) menu is used, put the section header - at the top of the screen. + at the top of the screen. - Issue #25225: Condense and rewrite Idle doc section on text colors. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 29 07:04:33 2015 From: python-checkins at python.org (terry.reedy) Date: Tue, 29 Sep 2015 05:04:33 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Merge_with_3=2E4?= Message-ID: <20150929050432.115250.47989@psf.io> https://hg.python.org/cpython/rev/32d89683015c changeset: 98373:32d89683015c branch: 3.5 parent: 98369:d27422bbde91 parent: 98372:992fa135ce8c user: Terry Jan Reedy date: Tue Sep 29 01:01:00 2015 -0400 summary: Merge with 3.4 files: Misc/NEWS | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -144,7 +144,7 @@ * Give code snippet a very Sphinx-like light blueish-gray background. * Re-use initial width and height set by users for shell and editor. * When the Table of Contents (TOC) menu is used, put the section header - at the top of the screen. + at the top of the screen. - Issue #25225: Condense and rewrite Idle doc section on text colors. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 29 07:57:12 2015 From: python-checkins at python.org (terry.reedy) Date: Tue, 29 Sep 2015 05:57:12 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzI0MDI4?= =?utf-8?q?=3A_Add_subsection_about_Idle_calltips=2E?= Message-ID: <20150929055712.31177.3896@psf.io> https://hg.python.org/cpython/rev/395d363e34e1 changeset: 98375:395d363e34e1 branch: 2.7 parent: 98371:89df4e23ec5f user: Terry Jan Reedy date: Tue Sep 29 01:55:50 2015 -0400 summary: Issue #24028: Add subsection about Idle calltips. files: Doc/library/idle.rst | 32 +++++++++++++++++++++++-- Lib/idlelib/help.html | 38 ++++++++++++++++++++++++------ 2 files changed, 59 insertions(+), 11 deletions(-) diff --git a/Doc/library/idle.rst b/Doc/library/idle.rst --- a/Doc/library/idle.rst +++ b/Doc/library/idle.rst @@ -435,9 +435,35 @@ much can be found by default, e.g. the re module. If you don't like the ACW popping up unbidden, simply make the delay -longer or disable the extension. Or another option is the delay could -be set to zero. Another alternative to preventing ACW popups is to -disable the call tips extension. +longer or disable the extension. + +Calltips +^^^^^^^^ + +A calltip is shown when one types :kbd:`(` after the name of an *acccessible* +function. A name expression may include dots and subscripts. A calltip +remains until it is clicked, the cursor is moved out of the argument area, +or :kbd:`)` is typed. When the cursor is in the argument part of a definition, +the menu or shortcut display a calltip. + +A calltip consists of the function signature and the first line of the +docstring. For builtins without an accessible signature, the calltip +consists of all lines up the fifth line or the first blank line. These +details may change. + +The set of *accessible* functions depends on what modules have been imported +into the user process, including those imported by Idle itself, +and what definitions have been run, all since the last restart. + +For example, restart the Shell and enter ``itertools.count(``. A calltip +appears because Idle imports itertools into the user process for its own use. +(This could change.) Enter ``turtle.write(`` and nothing appears. Idle does +not import turtle. The menu or shortcut do nothing either. Enter +``import turtle`` and then ``turtle.write(`` will work. + +In an editor, import statements have no effect until one runs the file. One +might want to run a file after writing the import statements at the top, +or immediately run an existing file before editing. Python Shell window ^^^^^^^^^^^^^^^^^^^ diff --git a/Lib/idlelib/help.html b/Lib/idlelib/help.html --- a/Lib/idlelib/help.html +++ b/Lib/idlelib/help.html @@ -417,12 +417,33 @@ Note that IDLE itself places quite a few modules in sys.modules, so much can be found by default, e.g. the re module.

      If you don’t like the ACW popping up unbidden, simply make the delay -longer or disable the extension. Or another option is the delay could -be set to zero. Another alternative to preventing ACW popups is to -disable the call tips extension.

      +longer or disable the extension.

      +
  • +
    +

    24.6.2.3. Calltips?

    +

    A calltip is shown when one types ( after the name of an acccessible +function. A name expression may include dots and subscripts. A calltip +remains until it is clicked, the cursor is moved out of the argument area, +or ) is typed. When the cursor is in the argument part of a definition, +the menu or shortcut display a calltip.

    +

    A calltip consists of the function signature and the first line of the +docstring. For builtins without an accessible signature, the calltip +consists of all lines up the fifth line or the first blank line. These +details may change.

    +

    The set of accessible functions depends on what modules have been imported +into the user process, including those imported by Idle itself, +and what definitions have been run, all since the last restart.

    +

    For example, restart the Shell and enter itertools.count(. A calltip +appears because Idle imports itertools into the user process for its own use. +(This could change.) Enter turtle.write( and nothing appears. Idle does +not import turtle. The menu or shortcut do nothing either. Enter +import turtle and then turtle.write( will work.

    +

    In an editor, import statements have no effect until one runs the file. One +might want to run a file after writing the import statements at the top, +or immediately run an existing file before editing.

    -

    24.6.2.3. Python Shell window?

    +

    24.6.2.4. Python Shell window?

    • C-c interrupts executing command

    • @@ -440,7 +461,7 @@
    -

    24.6.2.4. Text colors?

    +

    24.6.2.5. Text colors?

    Idle defaults to black on white text, but colors text with special meanings. For the shell, these are shell output, shell error, user output, and user error. For Python code, at the shell prompt or in an editor, these are @@ -595,8 +616,9 @@

  • 24.6.2. Editing and navigation
  • 24.6.3. Startup and code execution
      @@ -677,7 +699,7 @@ The Python Software Foundation is a non-profit corporation. Please donate.
      - Last updated on Sep 24, 2015. + Last updated on Sep 29, 2015. Found a bug?
      Created using Sphinx 1.2.3. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 29 07:57:12 2015 From: python-checkins at python.org (terry.reedy) Date: Tue, 29 Sep 2015 05:57:12 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Merge_with_3=2E4=2C_Issue_=2324028=3A_Add_subsection_about_Idl?= =?utf-8?q?e_calltips=2E?= Message-ID: <20150929055712.31185.91267@psf.io> https://hg.python.org/cpython/rev/3481c466926a changeset: 98377:3481c466926a branch: 3.5 parent: 98373:32d89683015c parent: 98376:8103dadb9ef7 user: Terry Jan Reedy date: Tue Sep 29 01:56:35 2015 -0400 summary: Merge with 3.4, Issue #24028: Add subsection about Idle calltips. files: Doc/library/idle.rst | 32 +++++++++++++++++++++++-- Lib/idlelib/help.html | 38 ++++++++++++++++++++++++------ 2 files changed, 59 insertions(+), 11 deletions(-) diff --git a/Doc/library/idle.rst b/Doc/library/idle.rst --- a/Doc/library/idle.rst +++ b/Doc/library/idle.rst @@ -435,9 +435,35 @@ much can be found by default, e.g. the re module. If you don't like the ACW popping up unbidden, simply make the delay -longer or disable the extension. Or another option is the delay could -be set to zero. Another alternative to preventing ACW popups is to -disable the call tips extension. +longer or disable the extension. + +Calltips +^^^^^^^^ + +A calltip is shown when one types :kbd:`(` after the name of an *acccessible* +function. A name expression may include dots and subscripts. A calltip +remains until it is clicked, the cursor is moved out of the argument area, +or :kbd:`)` is typed. When the cursor is in the argument part of a definition, +the menu or shortcut display a calltip. + +A calltip consists of the function signature and the first line of the +docstring. For builtins without an accessible signature, the calltip +consists of all lines up the fifth line or the first blank line. These +details may change. + +The set of *accessible* functions depends on what modules have been imported +into the user process, including those imported by Idle itself, +and what definitions have been run, all since the last restart. + +For example, restart the Shell and enter ``itertools.count(``. A calltip +appears because Idle imports itertools into the user process for its own use. +(This could change.) Enter ``turtle.write(`` and nothing appears. Idle does +not import turtle. The menu or shortcut do nothing either. Enter +``import turtle`` and then ``turtle.write(`` will work. + +In an editor, import statements have no effect until one runs the file. One +might want to run a file after writing the import statements at the top, +or immediately run an existing file before editing. Python Shell window ^^^^^^^^^^^^^^^^^^^ diff --git a/Lib/idlelib/help.html b/Lib/idlelib/help.html --- a/Lib/idlelib/help.html +++ b/Lib/idlelib/help.html @@ -417,12 +417,33 @@ Note that IDLE itself places quite a few modules in sys.modules, so much can be found by default, e.g. the re module.

      If you don’t like the ACW popping up unbidden, simply make the delay -longer or disable the extension. Or another option is the delay could -be set to zero. Another alternative to preventing ACW popups is to -disable the call tips extension.

      +longer or disable the extension.

      +
  • +
    +

    25.5.2.3. Calltips?

    +

    A calltip is shown when one types ( after the name of an acccessible +function. A name expression may include dots and subscripts. A calltip +remains until it is clicked, the cursor is moved out of the argument area, +or ) is typed. When the cursor is in the argument part of a definition, +the menu or shortcut display a calltip.

    +

    A calltip consists of the function signature and the first line of the +docstring. For builtins without an accessible signature, the calltip +consists of all lines up the fifth line or the first blank line. These +details may change.

    +

    The set of accessible functions depends on what modules have been imported +into the user process, including those imported by Idle itself, +and what definitions have been run, all since the last restart.

    +

    For example, restart the Shell and enter itertools.count(. A calltip +appears because Idle imports itertools into the user process for its own use. +(This could change.) Enter turtle.write( and nothing appears. Idle does +not import turtle. The menu or shortcut do nothing either. Enter +import turtle and then turtle.write( will work.

    +

    In an editor, import statements have no effect until one runs the file. One +might want to run a file after writing the import statements at the top, +or immediately run an existing file before editing.

    -

    25.5.2.3. Python Shell window?

    +

    25.5.2.4. Python Shell window?

    • C-c interrupts executing command

    • @@ -440,7 +461,7 @@
    -

    25.5.2.4. Text colors?

    +

    25.5.2.5. Text colors?

    Idle defaults to black on white text, but colors text with special meanings. For the shell, these are shell output, shell error, user output, and user error. For Python code, at the shell prompt or in an editor, these are @@ -595,8 +616,9 @@

  • 25.5.2. Editing and navigation
  • 25.5.3. Startup and code execution
      @@ -677,7 +699,7 @@ The Python Software Foundation is a non-profit corporation. Please donate.
      - Last updated on Sep 24, 2015. + Last updated on Sep 29, 2015. Found a bug?
      Created using Sphinx 1.2.3. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 29 07:57:12 2015 From: python-checkins at python.org (terry.reedy) Date: Tue, 29 Sep 2015 05:57:12 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzI0MDI4?= =?utf-8?q?=3A_Add_subsection_about_Idle_calltips=2E?= Message-ID: <20150929055712.9931.34917@psf.io> https://hg.python.org/cpython/rev/8103dadb9ef7 changeset: 98376:8103dadb9ef7 branch: 3.4 parent: 98372:992fa135ce8c user: Terry Jan Reedy date: Tue Sep 29 01:55:57 2015 -0400 summary: Issue #24028: Add subsection about Idle calltips. files: Doc/library/idle.rst | 32 +++++++++++++++++++++++-- Lib/idlelib/help.html | 38 ++++++++++++++++++++++++------ 2 files changed, 59 insertions(+), 11 deletions(-) diff --git a/Doc/library/idle.rst b/Doc/library/idle.rst --- a/Doc/library/idle.rst +++ b/Doc/library/idle.rst @@ -435,9 +435,35 @@ much can be found by default, e.g. the re module. If you don't like the ACW popping up unbidden, simply make the delay -longer or disable the extension. Or another option is the delay could -be set to zero. Another alternative to preventing ACW popups is to -disable the call tips extension. +longer or disable the extension. + +Calltips +^^^^^^^^ + +A calltip is shown when one types :kbd:`(` after the name of an *acccessible* +function. A name expression may include dots and subscripts. A calltip +remains until it is clicked, the cursor is moved out of the argument area, +or :kbd:`)` is typed. When the cursor is in the argument part of a definition, +the menu or shortcut display a calltip. + +A calltip consists of the function signature and the first line of the +docstring. For builtins without an accessible signature, the calltip +consists of all lines up the fifth line or the first blank line. These +details may change. + +The set of *accessible* functions depends on what modules have been imported +into the user process, including those imported by Idle itself, +and what definitions have been run, all since the last restart. + +For example, restart the Shell and enter ``itertools.count(``. A calltip +appears because Idle imports itertools into the user process for its own use. +(This could change.) Enter ``turtle.write(`` and nothing appears. Idle does +not import turtle. The menu or shortcut do nothing either. Enter +``import turtle`` and then ``turtle.write(`` will work. + +In an editor, import statements have no effect until one runs the file. One +might want to run a file after writing the import statements at the top, +or immediately run an existing file before editing. Python Shell window ^^^^^^^^^^^^^^^^^^^ diff --git a/Lib/idlelib/help.html b/Lib/idlelib/help.html --- a/Lib/idlelib/help.html +++ b/Lib/idlelib/help.html @@ -417,12 +417,33 @@ Note that IDLE itself places quite a few modules in sys.modules, so much can be found by default, e.g. the re module.

      If you don’t like the ACW popping up unbidden, simply make the delay -longer or disable the extension. Or another option is the delay could -be set to zero. Another alternative to preventing ACW popups is to -disable the call tips extension.

      +longer or disable the extension.

      +
  • +
    +

    25.5.2.3. Calltips?

    +

    A calltip is shown when one types ( after the name of an acccessible +function. A name expression may include dots and subscripts. A calltip +remains until it is clicked, the cursor is moved out of the argument area, +or ) is typed. When the cursor is in the argument part of a definition, +the menu or shortcut display a calltip.

    +

    A calltip consists of the function signature and the first line of the +docstring. For builtins without an accessible signature, the calltip +consists of all lines up the fifth line or the first blank line. These +details may change.

    +

    The set of accessible functions depends on what modules have been imported +into the user process, including those imported by Idle itself, +and what definitions have been run, all since the last restart.

    +

    For example, restart the Shell and enter itertools.count(. A calltip +appears because Idle imports itertools into the user process for its own use. +(This could change.) Enter turtle.write( and nothing appears. Idle does +not import turtle. The menu or shortcut do nothing either. Enter +import turtle and then turtle.write( will work.

    +

    In an editor, import statements have no effect until one runs the file. One +might want to run a file after writing the import statements at the top, +or immediately run an existing file before editing.

    -

    25.5.2.3. Python Shell window?

    +

    25.5.2.4. Python Shell window?

    • C-c interrupts executing command

    • @@ -440,7 +461,7 @@
    -

    25.5.2.4. Text colors?

    +

    25.5.2.5. Text colors?

    Idle defaults to black on white text, but colors text with special meanings. For the shell, these are shell output, shell error, user output, and user error. For Python code, at the shell prompt or in an editor, these are @@ -595,8 +616,9 @@

  • 25.5.2. Editing and navigation
  • 25.5.3. Startup and code execution
      @@ -677,7 +699,7 @@ The Python Software Foundation is a non-profit corporation. Please donate.
      - Last updated on Sep 24, 2015. + Last updated on Sep 29, 2015. Found a bug?
      Created using Sphinx 1.2.3. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 29 07:57:12 2015 From: python-checkins at python.org (terry.reedy) Date: Tue, 29 Sep 2015 05:57:12 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge_with_3=2E5=2C_Issue_=2324028=3A_Add_subsection_abo?= =?utf-8?q?ut_Idle_calltips=2E?= Message-ID: <20150929055712.9957.40394@psf.io> https://hg.python.org/cpython/rev/e025bdffd71c changeset: 98378:e025bdffd71c parent: 98374:da7b11c0cd38 parent: 98377:3481c466926a user: Terry Jan Reedy date: Tue Sep 29 01:56:54 2015 -0400 summary: Merge with 3.5, Issue #24028: Add subsection about Idle calltips. files: Doc/library/idle.rst | 32 +++++++++++++++++++++++-- Lib/idlelib/help.html | 38 ++++++++++++++++++++++++------ 2 files changed, 59 insertions(+), 11 deletions(-) diff --git a/Doc/library/idle.rst b/Doc/library/idle.rst --- a/Doc/library/idle.rst +++ b/Doc/library/idle.rst @@ -435,9 +435,35 @@ much can be found by default, e.g. the re module. If you don't like the ACW popping up unbidden, simply make the delay -longer or disable the extension. Or another option is the delay could -be set to zero. Another alternative to preventing ACW popups is to -disable the call tips extension. +longer or disable the extension. + +Calltips +^^^^^^^^ + +A calltip is shown when one types :kbd:`(` after the name of an *acccessible* +function. A name expression may include dots and subscripts. A calltip +remains until it is clicked, the cursor is moved out of the argument area, +or :kbd:`)` is typed. When the cursor is in the argument part of a definition, +the menu or shortcut display a calltip. + +A calltip consists of the function signature and the first line of the +docstring. For builtins without an accessible signature, the calltip +consists of all lines up the fifth line or the first blank line. These +details may change. + +The set of *accessible* functions depends on what modules have been imported +into the user process, including those imported by Idle itself, +and what definitions have been run, all since the last restart. + +For example, restart the Shell and enter ``itertools.count(``. A calltip +appears because Idle imports itertools into the user process for its own use. +(This could change.) Enter ``turtle.write(`` and nothing appears. Idle does +not import turtle. The menu or shortcut do nothing either. Enter +``import turtle`` and then ``turtle.write(`` will work. + +In an editor, import statements have no effect until one runs the file. One +might want to run a file after writing the import statements at the top, +or immediately run an existing file before editing. Python Shell window ^^^^^^^^^^^^^^^^^^^ diff --git a/Lib/idlelib/help.html b/Lib/idlelib/help.html --- a/Lib/idlelib/help.html +++ b/Lib/idlelib/help.html @@ -417,12 +417,33 @@ Note that IDLE itself places quite a few modules in sys.modules, so much can be found by default, e.g. the re module.

      If you don’t like the ACW popping up unbidden, simply make the delay -longer or disable the extension. Or another option is the delay could -be set to zero. Another alternative to preventing ACW popups is to -disable the call tips extension.

      +longer or disable the extension.

      +
  • +
    +

    25.5.2.3. Calltips?

    +

    A calltip is shown when one types ( after the name of an acccessible +function. A name expression may include dots and subscripts. A calltip +remains until it is clicked, the cursor is moved out of the argument area, +or ) is typed. When the cursor is in the argument part of a definition, +the menu or shortcut display a calltip.

    +

    A calltip consists of the function signature and the first line of the +docstring. For builtins without an accessible signature, the calltip +consists of all lines up the fifth line or the first blank line. These +details may change.

    +

    The set of accessible functions depends on what modules have been imported +into the user process, including those imported by Idle itself, +and what definitions have been run, all since the last restart.

    +

    For example, restart the Shell and enter itertools.count(. A calltip +appears because Idle imports itertools into the user process for its own use. +(This could change.) Enter turtle.write( and nothing appears. Idle does +not import turtle. The menu or shortcut do nothing either. Enter +import turtle and then turtle.write( will work.

    +

    In an editor, import statements have no effect until one runs the file. One +might want to run a file after writing the import statements at the top, +or immediately run an existing file before editing.

    -

    25.5.2.3. Python Shell window?

    +

    25.5.2.4. Python Shell window?

    • C-c interrupts executing command

    • @@ -440,7 +461,7 @@
    -

    25.5.2.4. Text colors?

    +

    25.5.2.5. Text colors?

    Idle defaults to black on white text, but colors text with special meanings. For the shell, these are shell output, shell error, user output, and user error. For Python code, at the shell prompt or in an editor, these are @@ -595,8 +616,9 @@

  • 25.5.2. Editing and navigation
  • 25.5.3. Startup and code execution
      @@ -677,7 +699,7 @@ The Python Software Foundation is a non-profit corporation. Please donate.
      - Last updated on Sep 24, 2015. + Last updated on Sep 29, 2015. Found a bug?
      Created using Sphinx 1.2.3. -- Repository URL: https://hg.python.org/cpython From solipsis at pitrou.net Tue Sep 29 10:44:21 2015 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Tue, 29 Sep 2015 08:44:21 +0000 Subject: [Python-checkins] Daily reference leaks (da7b11c0cd38): sum=17877 Message-ID: <20150929084419.115438.72895@psf.io> results for da7b11c0cd38 on branch "default" -------------------------------------------- test_capi leaked [1598, 1598, 1598] references, sum=4794 test_capi leaked [387, 389, 389] memory blocks, sum=1165 test_functools leaked [0, 2, 2] memory blocks, sum=4 test_threading leaked [3196, 3196, 3196] references, sum=9588 test_threading leaked [774, 776, 776] memory blocks, sum=2326 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/psf-users/antoine/refleaks/reflogiDOXa7', '--timeout', '7200'] From python-checkins at python.org Tue Sep 29 12:35:09 2015 From: python-checkins at python.org (victor.stinner) Date: Tue, 29 Sep 2015 10:35:09 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Optimize_ascii/latin1+surr?= =?utf-8?q?ogateescape_encoders?= Message-ID: <20150929103509.115438.47758@psf.io> https://hg.python.org/cpython/rev/128a3f03ddeb changeset: 98379:128a3f03ddeb user: Victor Stinner date: Tue Sep 29 12:32:13 2015 +0200 summary: Optimize ascii/latin1+surrogateescape encoders Issue #25227: Optimize ASCII and latin1 encoders with the ``surrogateescape`` error handler: the encoders are now up to 3 times as fast. Initial patch written by Serhiy Storchaka. files: Doc/whatsnew/3.6.rst | 3 + Lib/test/test_codecs.py | 60 +++++++++++++++++++++++++++++ Misc/NEWS | 4 + Objects/unicodeobject.c | 16 +++++++ 4 files changed, 83 insertions(+), 0 deletions(-) diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst --- a/Doc/whatsnew/3.6.rst +++ b/Doc/whatsnew/3.6.rst @@ -117,6 +117,9 @@ * The ASCII decoder is now up to 60 times as fast for error handlers: ``surrogateescape``, ``ignore`` and ``replace``. +* The ASCII and the Latin1 encoders are now up to 3 times as fast for the error + error ``surrogateescape``. + Build and C API Changes ======================= diff --git a/Lib/test/test_codecs.py b/Lib/test/test_codecs.py --- a/Lib/test/test_codecs.py +++ b/Lib/test/test_codecs.py @@ -3060,7 +3060,31 @@ class ASCIITest(unittest.TestCase): + def test_encode(self): + self.assertEqual('abc123'.encode('ascii'), b'abc123') + + def test_encode_error(self): + for data, error_handler, expected in ( + ('[\x80\xff\u20ac]', 'ignore', b'[]'), + ('[\x80\xff\u20ac]', 'replace', b'[???]'), + ('[\x80\xff\u20ac]', 'xmlcharrefreplace', b'[€ÿ€]'), + ('[\x80\xff\u20ac]', 'backslashreplace', b'[\\x80\\xff\\u20ac]'), + ('[\udc80\udcff]', 'surrogateescape', b'[\x80\xff]'), + ): + with self.subTest(data=data, error_handler=error_handler, + expected=expected): + self.assertEqual(data.encode('ascii', error_handler), + expected) + + def test_encode_surrogateescape_error(self): + with self.assertRaises(UnicodeEncodeError): + # the first character can be decoded, but not the second + '\udc80\xff'.encode('ascii', 'surrogateescape') + def test_decode(self): + self.assertEqual(b'abc'.decode('ascii'), 'abc') + + def test_decode_error(self): for data, error_handler, expected in ( (b'[\x80\xff]', 'ignore', '[]'), (b'[\x80\xff]', 'replace', '[\ufffd\ufffd]'), @@ -3073,5 +3097,41 @@ expected) +class Latin1Test(unittest.TestCase): + def test_encode(self): + for data, expected in ( + ('abc', b'abc'), + ('\x80\xe9\xff', b'\x80\xe9\xff'), + ): + with self.subTest(data=data, expected=expected): + self.assertEqual(data.encode('latin1'), expected) + + def test_encode_errors(self): + for data, error_handler, expected in ( + ('[\u20ac\udc80]', 'ignore', b'[]'), + ('[\u20ac\udc80]', 'replace', b'[??]'), + ('[\u20ac\udc80]', 'backslashreplace', b'[\\u20ac\\udc80]'), + ('[\u20ac\udc80]', 'xmlcharrefreplace', b'[€�]'), + ('[\udc80\udcff]', 'surrogateescape', b'[\x80\xff]'), + ): + with self.subTest(data=data, error_handler=error_handler, + expected=expected): + self.assertEqual(data.encode('latin1', error_handler), + expected) + + def test_encode_surrogateescape_error(self): + with self.assertRaises(UnicodeEncodeError): + # the first character can be decoded, but not the second + '\udc80\u20ac'.encode('latin1', 'surrogateescape') + + def test_decode(self): + for data, expected in ( + (b'abc', 'abc'), + (b'[\x80\xff]', '[\x80\xff]'), + ): + with self.subTest(data=data, expected=expected): + self.assertEqual(data.decode('latin1'), expected) + + if __name__ == "__main__": unittest.main() diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -10,6 +10,10 @@ Core and Builtins ----------------- +- Issue #25227: Optimize ASCII and latin1 encoders with the ``surrogateescape`` + error handler: the encoders are now up to 3 times as fast. Initial patch + written by Serhiy Storchaka. + - Issue #25003: On Solaris 11.3 or newer, os.urandom() now uses the getrandom() function instead of the getentropy() function. The getentropy() function is blocking to generate very good quality entropy, os.urandom() diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c --- a/Objects/unicodeobject.c +++ b/Objects/unicodeobject.c @@ -6532,6 +6532,22 @@ pos = collend; break; + case _Py_ERROR_SURROGATEESCAPE: + for (i = collstart; i < collend; ++i) { + ch = PyUnicode_READ(kind, data, i); + if (ch < 0xdc80 || 0xdcff < ch) { + /* Not a UTF-8b surrogate */ + break; + } + *str++ = (char)(ch - 0xdc00); + ++pos; + } + if (i >= collend) + break; + collstart = pos; + assert(collstart != collend); + /* fallback to general error handling */ + default: repunicode = unicode_encode_call_errorhandler(errors, &error_handler_obj, encoding, reason, unicode, &exc, -- Repository URL: https://hg.python.org/cpython From lp_benchmark_robot at intel.com Tue Sep 29 13:38:24 2015 From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com) Date: Tue, 29 Sep 2015 12:38:24 +0100 Subject: [Python-checkins] Benchmark Results for Python Default 2015-09-29 Message-ID: <79278c56-e2ed-4ef4-89e1-203de5228aec@irsmsx102.ger.corp.intel.com> Results for project python_default-nightly, build date 2015-09-29 03:01:55 commit: 74a9f0d242e9069e62fd6156ea9fc3159f278ea4 revision date: 2015-09-28 23:53:44 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v3.4.3, with hash b4cbecbc0781e89a309d03b60a1f75f8499250e6 from 2015-02-25 12:15:33+00:00 ------------------------------------------------------------------------------------------ benchmark relative change since change since current rev with std_dev* last run v3.4.3 regrtest PGO ------------------------------------------------------------------------------------------ :-) django_v2 0.43865% 1.78003% 8.45243% 14.57814% :-( pybench 0.21728% 0.07045% -2.59251% 9.25452% :-( regex_v8 2.71625% 0.07997% -4.13571% 5.81206% :-| nbody 0.08496% -0.01953% -0.14835% 7.09467% :-| json_dump_v2 0.26154% -0.43010% -1.27350% 9.77629% :-| normal_startup 0.91610% -0.06412% 0.22879% 4.94675% ------------------------------------------------------------------------------------------ Note: Benchmark results are measured in seconds. * Relative Standard Deviation (Standard Deviation/Average) Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. From lp_benchmark_robot at intel.com Tue Sep 29 13:38:36 2015 From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com) Date: Tue, 29 Sep 2015 12:38:36 +0100 Subject: [Python-checkins] Benchmark Results for Python 2.7 2015-09-29 Message-ID: <1791628a-be18-41cf-80ba-5712a9e60d57@irsmsx102.ger.corp.intel.com> Results for project python_2.7-nightly, build date 2015-09-29 03:42:54 commit: f9ac87a1adc39d06fa968a572a951cea9ed5b614 revision date: 2015-09-29 03:38:46 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v2.7.10, with hash 15c95b7d81dcf821daade360741e00714667653f from 2015-05-23 16:02:14+00:00 ------------------------------------------------------------------------------------------ benchmark relative change since change since current rev with std_dev* last run v2.7.10 regrtest PGO ------------------------------------------------------------------------------------------ :-) django_v2 0.14053% 0.47516% 4.20765% 8.93285% :-) pybench 0.09717% 0.38098% 6.79149% 7.00857% :-| regex_v8 0.56924% -0.02404% -1.12807% 7.30452% :-) nbody 0.11027% 0.03655% 9.09683% 5.27030% :-) json_dump_v2 0.23874% 0.35990% 4.38236% 13.24444% :-| normal_startup 1.76808% 0.21548% -1.43716% 3.07720% :-| ssbench 0.66743% -1.46470% 1.00975% 2.92114% ------------------------------------------------------------------------------------------ Note: Benchmark results for ssbench are measured in requests/second while all other are measured in seconds. * Relative Standard Deviation (Standard Deviation/Average) Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. From python-checkins at python.org Tue Sep 29 13:43:06 2015 From: python-checkins at python.org (victor.stinner) Date: Tue, 29 Sep 2015 11:43:06 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Try_to_fix_=5FPyTime=5FAsT?= =?utf-8?q?imevalStruct=5Fimpl=28=29_on_OpenBSD?= Message-ID: <20150929114305.11708.46208@psf.io> https://hg.python.org/cpython/rev/28d3bcb1bad6 changeset: 98380:28d3bcb1bad6 user: Victor Stinner date: Tue Sep 29 13:41:46 2015 +0200 summary: Try to fix _PyTime_AsTimevalStruct_impl() on OpenBSD It looks like the check for integer overflow doesn't work on x86 OpenBSD 5.8. files: Python/pytime.c | 5 +++-- 1 files changed, 3 insertions(+), 2 deletions(-) diff --git a/Python/pytime.c b/Python/pytime.c --- a/Python/pytime.c +++ b/Python/pytime.c @@ -454,7 +454,7 @@ _PyTime_AsTimevalStruct_impl(_PyTime_t t, struct timeval *tv, _PyTime_round_t round, int raise) { - _PyTime_t secs; + _PyTime_t secs, secs2; int us; int res; @@ -467,7 +467,8 @@ #endif tv->tv_usec = us; - if (res < 0 || (_PyTime_t)tv->tv_sec != secs) { + secs2 = (_PyTime_t)tv->tv_sec; + if (res < 0 || secs2 != secs) { if (raise) error_time_t_overflow(); return -1; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 29 14:00:14 2015 From: python-checkins at python.org (victor.stinner) Date: Tue, 29 Sep 2015 12:00:14 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2318174=3A_Explain_?= =?utf-8?q?why_is=5Fvalid=5Ffd=28=29_uses_dup=28=29_instead_of_fstat=28=29?= Message-ID: <20150929120008.98368.38668@psf.io> https://hg.python.org/cpython/rev/0e7d71a3bf0d changeset: 98382:0e7d71a3bf0d user: Victor Stinner date: Tue Sep 29 13:59:50 2015 +0200 summary: Issue #18174: Explain why is_valid_fd() uses dup() instead of fstat() files: Python/pylifecycle.c | 3 +++ 1 files changed, 3 insertions(+), 0 deletions(-) diff --git a/Python/pylifecycle.c b/Python/pylifecycle.c --- a/Python/pylifecycle.c +++ b/Python/pylifecycle.c @@ -972,6 +972,9 @@ if (fd < 0 || !_PyVerify_fd(fd)) return 0; _Py_BEGIN_SUPPRESS_IPH + /* Prefer dup() over fstat(). fstat() can require input/output whereas + dup() doesn't, there is a low risk of EMFILE/ENFILE at Python + startup. */ fd2 = dup(fd); if (fd2 >= 0) close(fd2); -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 29 14:00:14 2015 From: python-checkins at python.org (victor.stinner) Date: Tue, 29 Sep 2015 12:00:14 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_test?= Message-ID: <20150929120007.98368.5987@psf.io> https://hg.python.org/cpython/rev/9afc4aca1237 changeset: 98381:9afc4aca1237 user: Victor Stinner date: Tue Sep 29 13:47:15 2015 +0200 summary: test files: Lib/test/libregrtest/main.py | 4 +--- Lib/test/test_regrtest.py | 2 -- 2 files changed, 1 insertions(+), 5 deletions(-) diff --git a/Lib/test/libregrtest/main.py b/Lib/test/libregrtest/main.py --- a/Lib/test/libregrtest/main.py +++ b/Lib/test/libregrtest/main.py @@ -246,9 +246,7 @@ random.shuffle(selected) if ns.trace: import trace, tempfile - tracer = trace.Trace(ignoredirs=[sys.base_prefix, sys.base_exec_prefix, - tempfile.gettempdir()], - trace=False, count=True) + tracer = trace.Trace(trace=False, count=True) test_times = [] support.verbose = ns.verbose # Tell tests to be moderately quiet diff --git a/Lib/test/test_regrtest.py b/Lib/test/test_regrtest.py --- a/Lib/test/test_regrtest.py +++ b/Lib/test/test_regrtest.py @@ -549,8 +549,6 @@ % (self.TESTNAME_REGEX, len(tests))) self.check_line(output, regex) - @unittest.skipIf(sys.platform == 'win32', - "FIXME: coverage doesn't work on Windows") def test_coverage(self): # test --coverage test = self.create_test() -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 29 14:02:42 2015 From: python-checkins at python.org (victor.stinner) Date: Tue, 29 Sep 2015 12:02:42 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Oops=2C_revert_unwanted_ch?= =?utf-8?q?ange=2C_sorry?= Message-ID: <20150929120241.3670.80002@psf.io> https://hg.python.org/cpython/rev/799d3939df39 changeset: 98383:799d3939df39 user: Victor Stinner date: Tue Sep 29 14:02:35 2015 +0200 summary: Oops, revert unwanted change, sorry files: Lib/test/libregrtest/main.py | 4 +++- Lib/test/test_regrtest.py | 2 ++ 2 files changed, 5 insertions(+), 1 deletions(-) diff --git a/Lib/test/libregrtest/main.py b/Lib/test/libregrtest/main.py --- a/Lib/test/libregrtest/main.py +++ b/Lib/test/libregrtest/main.py @@ -246,7 +246,9 @@ random.shuffle(selected) if ns.trace: import trace, tempfile - tracer = trace.Trace(trace=False, count=True) + tracer = trace.Trace(ignoredirs=[sys.base_prefix, sys.base_exec_prefix, + tempfile.gettempdir()], + trace=False, count=True) test_times = [] support.verbose = ns.verbose # Tell tests to be moderately quiet diff --git a/Lib/test/test_regrtest.py b/Lib/test/test_regrtest.py --- a/Lib/test/test_regrtest.py +++ b/Lib/test/test_regrtest.py @@ -549,6 +549,8 @@ % (self.TESTNAME_REGEX, len(tests))) self.check_line(output, regex) + @unittest.skipIf(sys.platform == 'win32', + "FIXME: coverage doesn't work on Windows") def test_coverage(self): # test --coverage test = self.create_test() -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 29 14:19:11 2015 From: python-checkins at python.org (victor.stinner) Date: Tue, 29 Sep 2015 12:19:11 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2325220=3A_Add_test?= =?utf-8?q?_for_--wait_in_test=5Fregrtest?= Message-ID: <20150929121905.98352.90499@psf.io> https://hg.python.org/cpython/rev/e5165dcae942 changeset: 98384:e5165dcae942 user: Victor Stinner date: Tue Sep 29 14:17:09 2015 +0200 summary: Issue #25220: Add test for --wait in test_regrtest Replace script_helper.assert_python_ok() with subprocess.run(). files: Lib/test/test_regrtest.py | 48 ++++++++++++++++++++------ 1 files changed, 36 insertions(+), 12 deletions(-) diff --git a/Lib/test/test_regrtest.py b/Lib/test/test_regrtest.py --- a/Lib/test/test_regrtest.py +++ b/Lib/test/test_regrtest.py @@ -16,7 +16,6 @@ import unittest from test import libregrtest from test import support -from test.support import script_helper Py_DEBUG = hasattr(sys, 'getobjects') @@ -366,6 +365,31 @@ self.assertTrue(0 <= randseed <= 10000000, randseed) return randseed + def run_command(self, args, input=None): + if not input: + input = '' + try: + return subprocess.run(args, + check=True, universal_newlines=True, + input=input, + stdout=subprocess.PIPE, + stderr=subprocess.PIPE) + except subprocess.CalledProcessError as exc: + self.fail("%s\n" + "\n" + "stdout:\n" + "%s\n" + "\n" + "stderr:\n" + "%s" + % (str(exc), exc.stdout, exc.stderr)) + + + def run_python(self, args, **kw): + args = [sys.executable, '-X', 'faulthandler', '-I', *args] + proc = self.run_command(args, **kw) + return proc.stdout + class ProgramsTestCase(BaseTestCase): """ @@ -391,9 +415,8 @@ self.check_executed_tests(output, self.tests) def run_tests(self, args): - res = script_helper.assert_python_ok(*args) - output = os.fsdecode(res.out) - self.check_output(output) + stdout = self.run_python(args) + self.check_output(stdout) def test_script_regrtest(self): # Lib/test/regrtest.py @@ -439,10 +462,7 @@ self.run_tests([script, *self.tests]) def run_batch(self, *args): - proc = subprocess.run(args, - check=True, universal_newlines=True, - input='', - stdout=subprocess.PIPE, stderr=subprocess.PIPE) + proc = self.run_command(args) self.check_output(proc.stdout) @unittest.skipUnless(sys.platform == 'win32', 'Windows only') @@ -473,10 +493,8 @@ Test arguments of the Python test suite. """ - def run_tests(self, *args): - args = ['-m', 'test', *args] - res = script_helper.assert_python_ok(*args) - return os.fsdecode(res.out) + def run_tests(self, *args, input=None): + return self.run_python(['-m', 'test', *args], input=input) def test_resources(self): # test -u command line option @@ -561,6 +579,12 @@ '(?: *[0-9]+ *[0-9]{1,2}% *[^ ]+ +\([^)]+\)+)+') self.check_line(output, regex) + def test_wait(self): + # test --wait + test = self.create_test() + output = self.run_tests("--wait", test, input='key') + self.check_line(output, 'Press any key to continue') + if __name__ == '__main__': unittest.main() -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 29 14:59:11 2015 From: python-checkins at python.org (serhiy.storchaka) Date: Tue, 29 Sep 2015 12:59:11 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Added_additional_unpickling_tests=2E?= Message-ID: <20150929125911.16573.7486@psf.io> https://hg.python.org/cpython/rev/a4bf231d81d3 changeset: 98390:a4bf231d81d3 branch: 3.5 parent: 98386:dc69a996ab7d parent: 98389:3c2bcdff10a2 user: Serhiy Storchaka date: Tue Sep 29 15:50:45 2015 +0300 summary: Added additional unpickling tests. files: Lib/test/pickletester.py | 266 +++++++++++++++++++++++--- 1 files changed, 229 insertions(+), 37 deletions(-) diff --git a/Lib/test/pickletester.py b/Lib/test/pickletester.py --- a/Lib/test/pickletester.py +++ b/Lib/test/pickletester.py @@ -145,7 +145,7 @@ result.reduce_args = (name, bases) return result -# DATA0 .. DATA2 are the pickles we expect under the various protocols, for +# DATA0 .. DATA4 are the pickles we expect under the various protocols, for # the object returned by create_data(). DATA0 = ( @@ -401,22 +401,172 @@ highest protocol among opcodes = 2 """ +DATA3 = ( + b'\x80\x03]q\x00(K\x00K\x01G@\x00\x00\x00\x00\x00\x00\x00c' + b'builtins\ncomplex\nq\x01G' + b'@\x08\x00\x00\x00\x00\x00\x00G\x00\x00\x00\x00\x00\x00\x00\x00\x86q\x02' + b'Rq\x03K\x01J\xff\xff\xff\xffK\xffJ\x01\xff\xff\xffJ\x00\xff' + b'\xff\xffM\xff\xffJ\x01\x00\xff\xffJ\x00\x00\xff\xffJ\xff\xff\xff\x7f' + b'J\x01\x00\x00\x80J\x00\x00\x00\x80(X\x03\x00\x00\x00abcq' + b'\x04h\x04c__main__\nC\nq\x05)\x81q' + b'\x06}q\x07(X\x03\x00\x00\x00barq\x08K\x02X\x03\x00' + b'\x00\x00fooq\tK\x01ubh\x06tq\nh\nK\x05' + b'e.' +) + +# Disassembly of DATA3 +DATA3_DIS = """\ + 0: \x80 PROTO 3 + 2: ] EMPTY_LIST + 3: q BINPUT 0 + 5: ( MARK + 6: K BININT1 0 + 8: K BININT1 1 + 10: G BINFLOAT 2.0 + 19: c GLOBAL 'builtins complex' + 37: q BINPUT 1 + 39: G BINFLOAT 3.0 + 48: G BINFLOAT 0.0 + 57: \x86 TUPLE2 + 58: q BINPUT 2 + 60: R REDUCE + 61: q BINPUT 3 + 63: K BININT1 1 + 65: J BININT -1 + 70: K BININT1 255 + 72: J BININT -255 + 77: J BININT -256 + 82: M BININT2 65535 + 85: J BININT -65535 + 90: J BININT -65536 + 95: J BININT 2147483647 + 100: J BININT -2147483647 + 105: J BININT -2147483648 + 110: ( MARK + 111: X BINUNICODE 'abc' + 119: q BINPUT 4 + 121: h BINGET 4 + 123: c GLOBAL '__main__ C' + 135: q BINPUT 5 + 137: ) EMPTY_TUPLE + 138: \x81 NEWOBJ + 139: q BINPUT 6 + 141: } EMPTY_DICT + 142: q BINPUT 7 + 144: ( MARK + 145: X BINUNICODE 'bar' + 153: q BINPUT 8 + 155: K BININT1 2 + 157: X BINUNICODE 'foo' + 165: q BINPUT 9 + 167: K BININT1 1 + 169: u SETITEMS (MARK at 144) + 170: b BUILD + 171: h BINGET 6 + 173: t TUPLE (MARK at 110) + 174: q BINPUT 10 + 176: h BINGET 10 + 178: K BININT1 5 + 180: e APPENDS (MARK at 5) + 181: . STOP +highest protocol among opcodes = 2 +""" + +DATA4 = ( + b'\x80\x04\x95\xa8\x00\x00\x00\x00\x00\x00\x00]\x94(K\x00K\x01G@' + b'\x00\x00\x00\x00\x00\x00\x00\x8c\x08builtins\x94\x8c\x07' + b'complex\x94\x93\x94G@\x08\x00\x00\x00\x00\x00\x00G' + b'\x00\x00\x00\x00\x00\x00\x00\x00\x86\x94R\x94K\x01J\xff\xff\xff\xffK' + b'\xffJ\x01\xff\xff\xffJ\x00\xff\xff\xffM\xff\xffJ\x01\x00\xff\xffJ' + b'\x00\x00\xff\xffJ\xff\xff\xff\x7fJ\x01\x00\x00\x80J\x00\x00\x00\x80(' + b'\x8c\x03abc\x94h\x06\x8c\x08__main__\x94\x8c' + b'\x01C\x94\x93\x94)\x81\x94}\x94(\x8c\x03bar\x94K\x02\x8c' + b'\x03foo\x94K\x01ubh\nt\x94h\x0eK\x05e.' +) + +# Disassembly of DATA4 +DATA4_DIS = """\ + 0: \x80 PROTO 4 + 2: \x95 FRAME 168 + 11: ] EMPTY_LIST + 12: \x94 MEMOIZE + 13: ( MARK + 14: K BININT1 0 + 16: K BININT1 1 + 18: G BINFLOAT 2.0 + 27: \x8c SHORT_BINUNICODE 'builtins' + 37: \x94 MEMOIZE + 38: \x8c SHORT_BINUNICODE 'complex' + 47: \x94 MEMOIZE + 48: \x93 STACK_GLOBAL + 49: \x94 MEMOIZE + 50: G BINFLOAT 3.0 + 59: G BINFLOAT 0.0 + 68: \x86 TUPLE2 + 69: \x94 MEMOIZE + 70: R REDUCE + 71: \x94 MEMOIZE + 72: K BININT1 1 + 74: J BININT -1 + 79: K BININT1 255 + 81: J BININT -255 + 86: J BININT -256 + 91: M BININT2 65535 + 94: J BININT -65535 + 99: J BININT -65536 + 104: J BININT 2147483647 + 109: J BININT -2147483647 + 114: J BININT -2147483648 + 119: ( MARK + 120: \x8c SHORT_BINUNICODE 'abc' + 125: \x94 MEMOIZE + 126: h BINGET 6 + 128: \x8c SHORT_BINUNICODE '__main__' + 138: \x94 MEMOIZE + 139: \x8c SHORT_BINUNICODE 'C' + 142: \x94 MEMOIZE + 143: \x93 STACK_GLOBAL + 144: \x94 MEMOIZE + 145: ) EMPTY_TUPLE + 146: \x81 NEWOBJ + 147: \x94 MEMOIZE + 148: } EMPTY_DICT + 149: \x94 MEMOIZE + 150: ( MARK + 151: \x8c SHORT_BINUNICODE 'bar' + 156: \x94 MEMOIZE + 157: K BININT1 2 + 159: \x8c SHORT_BINUNICODE 'foo' + 164: \x94 MEMOIZE + 165: K BININT1 1 + 167: u SETITEMS (MARK at 150) + 168: b BUILD + 169: h BINGET 10 + 171: t TUPLE (MARK at 119) + 172: \x94 MEMOIZE + 173: h BINGET 14 + 175: K BININT1 5 + 177: e APPENDS (MARK at 13) + 178: . STOP +highest protocol among opcodes = 4 +""" + # set([1,2]) pickled from 2.x with protocol 2 -DATA3 = b'\x80\x02c__builtin__\nset\nq\x00]q\x01(K\x01K\x02e\x85q\x02Rq\x03.' +DATA_SET = b'\x80\x02c__builtin__\nset\nq\x00]q\x01(K\x01K\x02e\x85q\x02Rq\x03.' # xrange(5) pickled from 2.x with protocol 2 -DATA4 = b'\x80\x02c__builtin__\nxrange\nq\x00K\x00K\x05K\x01\x87q\x01Rq\x02.' +DATA_XRANGE = b'\x80\x02c__builtin__\nxrange\nq\x00K\x00K\x05K\x01\x87q\x01Rq\x02.' # a SimpleCookie() object pickled from 2.x with protocol 2 -DATA5 = (b'\x80\x02cCookie\nSimpleCookie\nq\x00)\x81q\x01U\x03key' - b'q\x02cCookie\nMorsel\nq\x03)\x81q\x04(U\x07commentq\x05U' - b'\x00q\x06U\x06domainq\x07h\x06U\x06secureq\x08h\x06U\x07' - b'expiresq\th\x06U\x07max-ageq\nh\x06U\x07versionq\x0bh\x06U' - b'\x04pathq\x0ch\x06U\x08httponlyq\rh\x06u}q\x0e(U\x0b' - b'coded_valueq\x0fU\x05valueq\x10h\x10h\x10h\x02h\x02ubs}q\x11b.') +DATA_COOKIE = (b'\x80\x02cCookie\nSimpleCookie\nq\x00)\x81q\x01U\x03key' + b'q\x02cCookie\nMorsel\nq\x03)\x81q\x04(U\x07commentq\x05U' + b'\x00q\x06U\x06domainq\x07h\x06U\x06secureq\x08h\x06U\x07' + b'expiresq\th\x06U\x07max-ageq\nh\x06U\x07versionq\x0bh\x06U' + b'\x04pathq\x0ch\x06U\x08httponlyq\rh\x06u}q\x0e(U\x0b' + b'coded_valueq\x0fU\x05valueq\x10h\x10h\x10h\x02h\x02ubs}q\x11b.') # set([3]) pickled from 2.x with protocol 2 -DATA6 = b'\x80\x02c__builtin__\nset\nq\x00]q\x01K\x03a\x85q\x02Rq\x03.' +DATA_SET2 = b'\x80\x02c__builtin__\nset\nq\x00]q\x01K\x03a\x85q\x02Rq\x03.' python2_exceptions_without_args = ( ArithmeticError, @@ -468,20 +618,10 @@ exception_pickle = b'\x80\x02cexceptions\n?\nq\x00)Rq\x01.' -# Exception objects without arguments pickled from 2.x with protocol 2 -DATA7 = { - exception : - exception_pickle.replace(b'?', exception.__name__.encode("ascii")) - for exception in python2_exceptions_without_args -} - -# StandardError is mapped to Exception, test that separately -DATA8 = exception_pickle.replace(b'?', b'StandardError') - # UnicodeEncodeError object pickled from 2.x with protocol 2 -DATA9 = (b'\x80\x02cexceptions\nUnicodeEncodeError\n' - b'q\x00(U\x05asciiq\x01X\x03\x00\x00\x00fooq\x02K\x00K\x01' - b'U\x03badq\x03tq\x04Rq\x05.') +DATA_UEERR = (b'\x80\x02cexceptions\nUnicodeEncodeError\n' + b'q\x00(U\x05asciiq\x01X\x03\x00\x00\x00fooq\x02K\x00K\x01' + b'U\x03badq\x03tq\x04Rq\x05.') def create_data(): @@ -537,6 +677,12 @@ def test_load_from_data2(self): self.assert_is_copy(self._testdata, self.loads(DATA2)) + def test_load_from_data3(self): + self.assert_is_copy(self._testdata, self.loads(DATA3)) + + def test_load_from_data4(self): + self.assert_is_copy(self._testdata, self.loads(DATA4)) + def test_load_classic_instance(self): # See issue5180. Test loading 2.x pickles that # contain an instance of old style class. @@ -594,11 +740,6 @@ b'q\x00oq\x01}q\x02b.').replace(b'X', xname) self.assert_is_copy(X(*args), self.loads(pickle2)) - def test_get(self): - self.assertRaises(KeyError, self.loads, b'g0\np0') - self.assert_is_copy([(100,), (100,)], - self.loads(b'((Kdtp0\nh\x00l.))')) - def test_maxint64(self): maxint64 = (1 << 63) - 1 data = b'I' + str(maxint64).encode("ascii") + b'\n.' @@ -616,24 +757,27 @@ def test_unpickle_from_2x(self): # Unpickle non-trivial data from Python 2.x. - loaded = self.loads(DATA3) + loaded = self.loads(DATA_SET) self.assertEqual(loaded, set([1, 2])) - loaded = self.loads(DATA4) + loaded = self.loads(DATA_XRANGE) self.assertEqual(type(loaded), type(range(0))) self.assertEqual(list(loaded), list(range(5))) - loaded = self.loads(DATA5) + loaded = self.loads(DATA_COOKIE) self.assertEqual(type(loaded), SimpleCookie) self.assertEqual(list(loaded.keys()), ["key"]) self.assertEqual(loaded["key"].value, "value") - for (exc, data) in DATA7.items(): + # Exception objects without arguments pickled from 2.x with protocol 2 + for exc in python2_exceptions_without_args: + data = exception_pickle.replace(b'?', exc.__name__.encode("ascii")) loaded = self.loads(data) self.assertIs(type(loaded), exc) - loaded = self.loads(DATA8) + # StandardError is mapped to Exception, test that separately + loaded = self.loads(exception_pickle.replace(b'?', b'StandardError')) self.assertIs(type(loaded), Exception) - loaded = self.loads(DATA9) + loaded = self.loads(DATA_UEERR) self.assertIs(type(loaded), UnicodeEncodeError) self.assertEqual(loaded.object, "foo") self.assertEqual(loaded.encoding, "ascii") @@ -670,11 +814,26 @@ b'x' * 300 + pickle.STOP, encoding='bytes'), b'x' * 300) + def test_constants(self): + self.assertIsNone(self.loads(b'N.')) + self.assertIs(self.loads(b'\x88.'), True) + self.assertIs(self.loads(b'\x89.'), False) + self.assertIs(self.loads(b'I01\n.'), True) + self.assertIs(self.loads(b'I00\n.'), False) + def test_empty_bytestring(self): # issue 11286 empty = self.loads(b'\x80\x03U\x00q\x00.', encoding='koi8-r') self.assertEqual(empty, '') + def test_short_binbytes(self): + dumped = b'\x80\x03C\x04\xe2\x82\xac\x00.' + self.assertEqual(self.loads(dumped), b'\xe2\x82\xac\x00') + + def test_binbytes(self): + dumped = b'\x80\x03B\x04\x00\x00\x00\xe2\x82\xac\x00.' + self.assertEqual(self.loads(dumped), b'\xe2\x82\xac\x00') + @requires_32b def test_negative_32b_binbytes(self): # On 32-bit builds, a BINBYTES of 2**31 or more is refused @@ -689,6 +848,39 @@ with self.assertRaises((pickle.UnpicklingError, OverflowError)): self.loads(dumped) + def test_short_binunicode(self): + dumped = b'\x80\x04\x8c\x04\xe2\x82\xac\x00.' + self.assertEqual(self.loads(dumped), '\u20ac\x00') + + def test_misc_get(self): + self.assertRaises(KeyError, self.loads, b'g0\np0') + self.assert_is_copy([(100,), (100,)], + self.loads(b'((Kdtp0\nh\x00l.))')) + + def test_get(self): + pickled = b'((lp100000\ng100000\nt.' + unpickled = self.loads(pickled) + self.assertEqual(unpickled, ([],)*2) + self.assertIs(unpickled[0], unpickled[1]) + + def test_binget(self): + pickled = b'(]q\xffh\xfft.' + unpickled = self.loads(pickled) + self.assertEqual(unpickled, ([],)*2) + self.assertIs(unpickled[0], unpickled[1]) + + def test_long_binget(self): + pickled = b'(]r\x00\x00\x01\x00j\x00\x00\x01\x00t.' + unpickled = self.loads(pickled) + self.assertEqual(unpickled, ([],)*2) + self.assertIs(unpickled[0], unpickled[1]) + + def test_dup(self): + pickled = b'((l2t.' + unpickled = self.loads(pickled) + self.assertEqual(unpickled, ([],)*2) + self.assertIs(unpickled[0], unpickled[1]) + def test_negative_put(self): # Issue #12847 dumped = b'Va\np-1\n.' @@ -1501,9 +1693,9 @@ # NOTE: this test is a bit too strong since we can produce different # bytecode that 2.x will still understand. dumped = self.dumps(range(5), 2) - self.assertEqual(dumped, DATA4) + self.assertEqual(dumped, DATA_XRANGE) dumped = self.dumps(set([3]), 2) - self.assertEqual(dumped, DATA6) + self.assertEqual(dumped, DATA_SET2) def test_large_pickles(self): # Test the correctness of internal buffering routines when handling @@ -2398,7 +2590,7 @@ # Print some stuff that can be used to rewrite DATA{0,1,2} from pickletools import dis x = create_data() - for i in range(3): + for i in range(pickle.HIGHEST_PROTOCOL+1): p = pickle.dumps(x, i) print("DATA{0} = (".format(i)) for j in range(0, len(p), 20): -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 29 14:59:13 2015 From: python-checkins at python.org (serhiy.storchaka) Date: Tue, 29 Sep 2015 12:59:13 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_Backported_add?= =?utf-8?q?itional_unpickling_tests_from_3=2Ex=2E?= Message-ID: <20150929125911.3666.60380@psf.io> https://hg.python.org/cpython/rev/2ec4aa882447 changeset: 98392:2ec4aa882447 branch: 2.7 parent: 98388:157849879bce user: Serhiy Storchaka date: Tue Sep 29 15:51:40 2015 +0300 summary: Backported additional unpickling tests from 3.x. files: Lib/test/pickletester.py | 164 ++++++++++++++++++++++++++- 1 files changed, 160 insertions(+), 4 deletions(-) diff --git a/Lib/test/pickletester.py b/Lib/test/pickletester.py --- a/Lib/test/pickletester.py +++ b/Lib/test/pickletester.py @@ -1,3 +1,4 @@ +# -*- coding: utf-8 -*- import unittest import pickle import cPickle @@ -108,9 +109,21 @@ def __cmp__(self, other): return cmp(self.__dict__, other.__dict__) +class D(C): + def __init__(self, arg): + pass + +class E(C): + def __getinitargs__(self): + return () + import __main__ __main__.C = C C.__module__ = "__main__" +__main__.D = D +D.__module__ = "__main__" +__main__.E = E +E.__module__ = "__main__" class myint(int): def __init__(self, x): @@ -426,11 +439,29 @@ _testdata = create_data() + def assert_is_copy(self, obj, objcopy, msg=None): + """Utility method to verify if two objects are copies of each others. + """ + if msg is None: + msg = "{!r} is not a copy of {!r}".format(obj, objcopy) + self.assertEqual(obj, objcopy, msg=msg) + self.assertIs(type(obj), type(objcopy), msg=msg) + if hasattr(obj, '__dict__'): + self.assertDictEqual(obj.__dict__, objcopy.__dict__, msg=msg) + self.assertIsNot(obj.__dict__, objcopy.__dict__, msg=msg) + if hasattr(obj, '__slots__'): + self.assertListEqual(obj.__slots__, objcopy.__slots__, msg=msg) + for slot in obj.__slots__: + self.assertEqual( + hasattr(obj, slot), hasattr(objcopy, slot), msg=msg) + self.assertEqual(getattr(obj, slot, None), + getattr(objcopy, slot, None), msg=msg) + def test_load_from_canned_string(self): expected = self._testdata for canned in DATA0, DATA1, DATA2: got = self.loads(canned) - self.assertEqual(expected, got) + self.assert_is_copy(expected, got) def test_garyp(self): self.assertRaises(self.error, self.loads, 'garyp') @@ -452,16 +483,141 @@ "'abc\"", # open quote and close quote don't match "'abc' ?", # junk after close quote "'\\'", # trailing backslash - "'", # issue #17710 - "' ", # issue #17710 + # issue #17710 + "'", '"', + "' ", '" ', + '\'"', '"\'', + " ''", ' ""', + ' ', # some tests of the quoting rules #"'abc\"\''", #"'\\\\a\'\'\'\\\'\\\\\''", ] for s in insecure: - buf = "S" + s + "\012p0\012." + buf = "S" + s + "\n." self.assertRaises(ValueError, self.loads, buf) + def test_correctly_quoted_string(self): + goodpickles = [(b"S''\n.", ''), + (b'S""\n.', ''), + (b'S"\\n"\n.', '\n'), + (b"S'\\n'\n.", '\n')] + for p, expected in goodpickles: + self.assertEqual(self.loads(p), expected) + + def test_load_classic_instance(self): + # See issue5180. Test loading 2.x pickles that + # contain an instance of old style class. + for X, args in [(C, ()), (D, ('x',)), (E, ())]: + xname = X.__name__.encode('ascii') + # Protocol 0 (text mode pickle): + """ + 0: ( MARK + 1: i INST '__main__ X' (MARK at 0) + 13: p PUT 0 + 16: ( MARK + 17: d DICT (MARK at 16) + 18: p PUT 1 + 21: b BUILD + 22: . STOP + """ + pickle0 = (b"(i__main__\n" + b"X\n" + b"p0\n" + b"(dp1\nb.").replace(b'X', xname) + self.assert_is_copy(X(*args), self.loads(pickle0)) + + # Protocol 1 (binary mode pickle) + """ + 0: ( MARK + 1: c GLOBAL '__main__ X' + 13: q BINPUT 0 + 15: o OBJ (MARK at 0) + 16: q BINPUT 1 + 18: } EMPTY_DICT + 19: q BINPUT 2 + 21: b BUILD + 22: . STOP + """ + pickle1 = (b'(c__main__\n' + b'X\n' + b'q\x00oq\x01}q\x02b.').replace(b'X', xname) + self.assert_is_copy(X(*args), self.loads(pickle1)) + + # Protocol 2 (pickle2 = b'\x80\x02' + pickle1) + """ + 0: \x80 PROTO 2 + 2: ( MARK + 3: c GLOBAL '__main__ X' + 15: q BINPUT 0 + 17: o OBJ (MARK at 2) + 18: q BINPUT 1 + 20: } EMPTY_DICT + 21: q BINPUT 2 + 23: b BUILD + 24: . STOP + """ + pickle2 = (b'\x80\x02(c__main__\n' + b'X\n' + b'q\x00oq\x01}q\x02b.').replace(b'X', xname) + self.assert_is_copy(X(*args), self.loads(pickle2)) + + def test_pop_empty_stack(self): + # Test issue7455 + s = b'0' + self.assertRaises((cPickle.UnpicklingError, IndexError), self.loads, s) + + def test_load_str(self): + # From Python 2: pickle.dumps('a\x00\xa0', protocol=0) + self.assertEqual(self.loads(b"S'a\\x00\\xa0'\n."), b'a\x00\xa0') + # From Python 2: pickle.dumps('a\x00\xa0', protocol=1) + self.assertEqual(self.loads(b'U\x03a\x00\xa0.'), b'a\x00\xa0') + # From Python 2: pickle.dumps('a\x00\xa0', protocol=2) + self.assertEqual(self.loads(b'\x80\x02U\x03a\x00\xa0.'), b'a\x00\xa0') + + def test_load_unicode(self): + # From Python 2: pickle.dumps(u'?', protocol=0) + self.assertEqual(self.loads(b'V\\u03c0\n.'), u'?') + # From Python 2: pickle.dumps(u'?', protocol=1) + self.assertEqual(self.loads(b'X\x02\x00\x00\x00\xcf\x80.'), u'?') + # From Python 2: pickle.dumps(u'?', protocol=2) + self.assertEqual(self.loads(b'\x80\x02X\x02\x00\x00\x00\xcf\x80.'), u'?') + + def test_constants(self): + self.assertIsNone(self.loads(b'N.')) + self.assertIs(self.loads(b'\x88.'), True) + self.assertIs(self.loads(b'\x89.'), False) + self.assertIs(self.loads(b'I01\n.'), True) + self.assertIs(self.loads(b'I00\n.'), False) + + def test_misc_get(self): + self.assertRaises(self.error, self.loads, b'g0\np0\n') + self.assertRaises(self.error, self.loads, b'h\x00q\x00') + + def test_get(self): + pickled = b'((lp100000\ng100000\nt.' + unpickled = self.loads(pickled) + self.assertEqual(unpickled, ([],)*2) + self.assertIs(unpickled[0], unpickled[1]) + + def test_binget(self): + pickled = b'(]q\xffh\xfft.' + unpickled = self.loads(pickled) + self.assertEqual(unpickled, ([],)*2) + self.assertIs(unpickled[0], unpickled[1]) + + def test_long_binget(self): + pickled = b'(]r\x00\x00\x01\x00j\x00\x00\x01\x00t.' + unpickled = self.loads(pickled) + self.assertEqual(unpickled, ([],)*2) + self.assertIs(unpickled[0], unpickled[1]) + + def test_dup(self): + pickled = b'((l2t.' + unpickled = self.loads(pickled) + self.assertEqual(unpickled, ([],)*2) + self.assertIs(unpickled[0], unpickled[1]) + class AbstractPickleTests(unittest.TestCase): # Subclass must define self.dumps, self.loads. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 29 14:59:13 2015 From: python-checkins at python.org (serhiy.storchaka) Date: Tue, 29 Sep 2015 12:59:13 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Moved_unpickling_tests_with_prepickled_data_to_separate_?= =?utf-8?q?class=2E?= Message-ID: <20150929125910.82654.36019@psf.io> https://hg.python.org/cpython/rev/ce72f75275df changeset: 98387:ce72f75275df parent: 98384:e5165dcae942 parent: 98386:dc69a996ab7d user: Serhiy Storchaka date: Tue Sep 29 15:35:19 2015 +0300 summary: Moved unpickling tests with prepickled data to separate class. files: Lib/test/pickletester.py | 424 +++++++++++++------------- Lib/test/test_pickle.py | 21 +- 2 files changed, 233 insertions(+), 212 deletions(-) diff --git a/Lib/test/pickletester.py b/Lib/test/pickletester.py --- a/Lib/test/pickletester.py +++ b/Lib/test/pickletester.py @@ -18,6 +18,9 @@ from pickle import bytes_types +requires_32b = unittest.skipUnless(sys.maxsize < 2**32, + "test is only meaningful on 32-bit builds") + # Tests that try a number of pickle protocols should have a # for proto in protocols: # kind of outer loop. @@ -502,16 +505,11 @@ return x -class AbstractPickleTests(unittest.TestCase): - # Subclass must define self.dumps, self.loads. - - optimized = False +class AbstractUnpickleTests(unittest.TestCase): + # Subclass must define self.loads. _testdata = create_data() - def setUp(self): - pass - def assert_is_copy(self, obj, objcopy, msg=None): """Utility method to verify if two objects are copies of each others. """ @@ -530,33 +528,6 @@ self.assertEqual(getattr(obj, slot, None), getattr(objcopy, slot, None), msg=msg) - def test_misc(self): - # test various datatypes not tested by testdata - for proto in protocols: - x = myint(4) - s = self.dumps(x, proto) - y = self.loads(s) - self.assert_is_copy(x, y) - - x = (1, ()) - s = self.dumps(x, proto) - y = self.loads(s) - self.assert_is_copy(x, y) - - x = initarg(1, x) - s = self.dumps(x, proto) - y = self.loads(s) - self.assert_is_copy(x, y) - - # XXX test __reduce__ protocol? - - def test_roundtrip_equality(self): - expected = self._testdata - for proto in protocols: - s = self.dumps(expected, proto) - got = self.loads(s) - self.assert_is_copy(expected, got) - def test_load_from_data0(self): self.assert_is_copy(self._testdata, self.loads(DATA0)) @@ -623,6 +594,216 @@ b'q\x00oq\x01}q\x02b.').replace(b'X', xname) self.assert_is_copy(X(*args), self.loads(pickle2)) + def test_get(self): + self.assertRaises(KeyError, self.loads, b'g0\np0') + self.assert_is_copy([(100,), (100,)], + self.loads(b'((Kdtp0\nh\x00l.))')) + + def test_maxint64(self): + maxint64 = (1 << 63) - 1 + data = b'I' + str(maxint64).encode("ascii") + b'\n.' + got = self.loads(data) + self.assert_is_copy(maxint64, got) + + # Try too with a bogus literal. + data = b'I' + str(maxint64).encode("ascii") + b'JUNK\n.' + self.assertRaises(ValueError, self.loads, data) + + def test_pop_empty_stack(self): + # Test issue7455 + s = b'0' + self.assertRaises((pickle.UnpicklingError, IndexError), self.loads, s) + + def test_unpickle_from_2x(self): + # Unpickle non-trivial data from Python 2.x. + loaded = self.loads(DATA3) + self.assertEqual(loaded, set([1, 2])) + loaded = self.loads(DATA4) + self.assertEqual(type(loaded), type(range(0))) + self.assertEqual(list(loaded), list(range(5))) + loaded = self.loads(DATA5) + self.assertEqual(type(loaded), SimpleCookie) + self.assertEqual(list(loaded.keys()), ["key"]) + self.assertEqual(loaded["key"].value, "value") + + for (exc, data) in DATA7.items(): + loaded = self.loads(data) + self.assertIs(type(loaded), exc) + + loaded = self.loads(DATA8) + self.assertIs(type(loaded), Exception) + + loaded = self.loads(DATA9) + self.assertIs(type(loaded), UnicodeEncodeError) + self.assertEqual(loaded.object, "foo") + self.assertEqual(loaded.encoding, "ascii") + self.assertEqual(loaded.start, 0) + self.assertEqual(loaded.end, 1) + self.assertEqual(loaded.reason, "bad") + + def test_load_python2_str_as_bytes(self): + # From Python 2: pickle.dumps('a\x00\xa0', protocol=0) + self.assertEqual(self.loads(b"S'a\\x00\\xa0'\n.", + encoding="bytes"), b'a\x00\xa0') + # From Python 2: pickle.dumps('a\x00\xa0', protocol=1) + self.assertEqual(self.loads(b'U\x03a\x00\xa0.', + encoding="bytes"), b'a\x00\xa0') + # From Python 2: pickle.dumps('a\x00\xa0', protocol=2) + self.assertEqual(self.loads(b'\x80\x02U\x03a\x00\xa0.', + encoding="bytes"), b'a\x00\xa0') + + def test_load_python2_unicode_as_str(self): + # From Python 2: pickle.dumps(u'?', protocol=0) + self.assertEqual(self.loads(b'V\\u03c0\n.', + encoding='bytes'), '?') + # From Python 2: pickle.dumps(u'?', protocol=1) + self.assertEqual(self.loads(b'X\x02\x00\x00\x00\xcf\x80.', + encoding="bytes"), '?') + # From Python 2: pickle.dumps(u'?', protocol=2) + self.assertEqual(self.loads(b'\x80\x02X\x02\x00\x00\x00\xcf\x80.', + encoding="bytes"), '?') + + def test_load_long_python2_str_as_bytes(self): + # From Python 2: pickle.dumps('x' * 300, protocol=1) + self.assertEqual(self.loads(pickle.BINSTRING + + struct.pack("', '<\\\u1234>', '<\n>', '<\\>', '<\\\U00012345>', @@ -754,7 +930,6 @@ self.assert_is_copy(s, self.loads(p)) def test_ints(self): - import sys for proto in protocols: n = sys.maxsize while n: @@ -764,16 +939,6 @@ self.assert_is_copy(expected, n2) n = n >> 1 - def test_maxint64(self): - maxint64 = (1 << 63) - 1 - data = b'I' + str(maxint64).encode("ascii") + b'\n.' - got = self.loads(data) - self.assert_is_copy(maxint64, got) - - # Try too with a bogus literal. - data = b'I' + str(maxint64).encode("ascii") + b'JUNK\n.' - self.assertRaises(ValueError, self.loads, data) - def test_long(self): for proto in protocols: # 256 bytes is where LONG4 begins. @@ -826,11 +991,6 @@ loaded = self.loads(dumped) self.assert_is_copy(inst, loaded) - def test_pop_empty_stack(self): - # Test issue7455 - s = b'0' - self.assertRaises((pickle.UnpicklingError, IndexError), self.loads, s) - def test_metaclass(self): a = use_metaclass() for proto in protocols: @@ -1335,33 +1495,6 @@ for x_key, y_key in zip(x_keys, y_keys): self.assertIs(x_key, y_key) - def test_unpickle_from_2x(self): - # Unpickle non-trivial data from Python 2.x. - loaded = self.loads(DATA3) - self.assertEqual(loaded, set([1, 2])) - loaded = self.loads(DATA4) - self.assertEqual(type(loaded), type(range(0))) - self.assertEqual(list(loaded), list(range(5))) - loaded = self.loads(DATA5) - self.assertEqual(type(loaded), SimpleCookie) - self.assertEqual(list(loaded.keys()), ["key"]) - self.assertEqual(loaded["key"].value, "value") - - for (exc, data) in DATA7.items(): - loaded = self.loads(data) - self.assertIs(type(loaded), exc) - - loaded = self.loads(DATA8) - self.assertIs(type(loaded), Exception) - - loaded = self.loads(DATA9) - self.assertIs(type(loaded), UnicodeEncodeError) - self.assertEqual(loaded.object, "foo") - self.assertEqual(loaded.encoding, "ascii") - self.assertEqual(loaded.start, 0) - self.assertEqual(loaded.end, 1) - self.assertEqual(loaded.reason, "bad") - def test_pickle_to_2x(self): # Pickle non-trivial data with protocol 2, expecting that it yields # the same result as Python 2.x did. @@ -1372,35 +1505,6 @@ dumped = self.dumps(set([3]), 2) self.assertEqual(dumped, DATA6) - def test_load_python2_str_as_bytes(self): - # From Python 2: pickle.dumps('a\x00\xa0', protocol=0) - self.assertEqual(self.loads(b"S'a\\x00\\xa0'\n.", - encoding="bytes"), b'a\x00\xa0') - # From Python 2: pickle.dumps('a\x00\xa0', protocol=1) - self.assertEqual(self.loads(b'U\x03a\x00\xa0.', - encoding="bytes"), b'a\x00\xa0') - # From Python 2: pickle.dumps('a\x00\xa0', protocol=2) - self.assertEqual(self.loads(b'\x80\x02U\x03a\x00\xa0.', - encoding="bytes"), b'a\x00\xa0') - - def test_load_python2_unicode_as_str(self): - # From Python 2: pickle.dumps(u'?', protocol=0) - self.assertEqual(self.loads(b'V\\u03c0\n.', - encoding='bytes'), '?') - # From Python 2: pickle.dumps(u'?', protocol=1) - self.assertEqual(self.loads(b'X\x02\x00\x00\x00\xcf\x80.', - encoding="bytes"), '?') - # From Python 2: pickle.dumps(u'?', protocol=2) - self.assertEqual(self.loads(b'\x80\x02X\x02\x00\x00\x00\xcf\x80.', - encoding="bytes"), '?') - - def test_load_long_python2_str_as_bytes(self): - # From Python 2: pickle.dumps('x' * 300, protocol=1) - self.assertEqual(self.loads(pickle.BINSTRING + - struct.pack(" 2**32: - self.skipTest("test is only meaningful on 32-bit builds") - # XXX Pure Python pickle reads lengths as signed and passes - # them directly to read() (hence the EOFError) - with self.assertRaises((pickle.UnpicklingError, EOFError, - ValueError, OverflowError)): - self.loads(dumped) - - def test_negative_32b_binbytes(self): - # On 32-bit builds, a BINBYTES of 2**31 or more is refused - self.check_negative_32b_binXXX(b'\x80\x03B\xff\xff\xff\xffxyzq\x00.') - - def test_negative_32b_binunicode(self): - # On 32-bit builds, a BINUNICODE of 2**31 or more is refused - self.check_negative_32b_binXXX(b'\x80\x03X\xff\xff\xff\xffxyzq\x00.') - - def test_negative_put(self): - # Issue #12847 - dumped = b'Va\np-1\n.' - self.assertRaises(ValueError, self.loads, dumped) - - def test_negative_32b_binput(self): - # Issue #12847 - if sys.maxsize > 2**32: - self.skipTest("test is only meaningful on 32-bit builds") - dumped = b'\x80\x03X\x01\x00\x00\x00ar\xff\xff\xff\xff.' - self.assertRaises(ValueError, self.loads, dumped) - - def test_badly_escaped_string(self): - self.assertRaises(ValueError, self.loads, b"S'\\'\n.") - - def test_badly_quoted_string(self): - # Issue #17710 - badpickles = [b"S'\n.", - b'S"\n.', - b'S\' \n.', - b'S" \n.', - b'S\'"\n.', - b'S"\'\n.', - b"S' ' \n.", - b'S" " \n.', - b"S ''\n.", - b'S ""\n.', - b'S \n.', - b'S\n.', - b'S.'] - for p in badpickles: - self.assertRaises(pickle.UnpicklingError, self.loads, p) - - def test_correctly_quoted_string(self): - goodpickles = [(b"S''\n.", ''), - (b'S""\n.', ''), - (b'S"\\n"\n.', '\n'), - (b"S'\\n'\n.", '\n')] - for p, expected in goodpickles: - self.assertEqual(self.loads(p), expected) - def _check_pickling_with_opcode(self, obj, opcode, proto): pickled = self.dumps(obj, proto) self.assertTrue(opcode_in_pickle(opcode, pickled)) @@ -1599,14 +1640,6 @@ count_opcode(pickle.FRAME, pickled)) self.assertEqual(obj, self.loads(some_frames_pickle)) - def test_frame_readline(self): - pickled = b'\x80\x04\x95\x05\x00\x00\x00\x00\x00\x00\x00I42\n.' - # 0: \x80 PROTO 4 - # 2: \x95 FRAME 5 - # 11: I INT 42 - # 15: . STOP - self.assertEqual(self.loads(pickled), 42) - def test_nested_names(self): global Nested class Nested: @@ -1734,33 +1767,6 @@ self.assertIn(('c%s\n%s' % (mod, name)).encode(), pickled) self.assertIs(type(self.loads(pickled)), type(val)) - def test_compat_unpickle(self): - # xrange(1, 7) - pickled = b'\x80\x02c__builtin__\nxrange\nK\x01K\x07K\x01\x87R.' - unpickled = self.loads(pickled) - self.assertIs(type(unpickled), range) - self.assertEqual(unpickled, range(1, 7)) - self.assertEqual(list(unpickled), [1, 2, 3, 4, 5, 6]) - # reduce - pickled = b'\x80\x02c__builtin__\nreduce\n.' - self.assertIs(self.loads(pickled), functools.reduce) - # whichdb.whichdb - pickled = b'\x80\x02cwhichdb\nwhichdb\n.' - self.assertIs(self.loads(pickled), dbm.whichdb) - # Exception(), StandardError() - for name in (b'Exception', b'StandardError'): - pickled = (b'\x80\x02cexceptions\n' + name + b'\nU\x03ugh\x85R.') - unpickled = self.loads(pickled) - self.assertIs(type(unpickled), Exception) - self.assertEqual(str(unpickled), 'ugh') - # UserDict.UserDict({1: 2}), UserDict.IterableUserDict({1: 2}) - for name in (b'UserDict', b'IterableUserDict'): - pickled = (b'\x80\x02(cUserDict\n' + name + - b'\no}U\x04data}K\x01K\x02ssb.') - unpickled = self.loads(pickled) - self.assertIs(type(unpickled), collections.UserDict) - self.assertEqual(unpickled, collections.UserDict({1: 2})) - def test_local_lookup_error(self): # Test that whichmodule() errors out cleanly when looking up # an assumed globally-reachable object fails. diff --git a/Lib/test/test_pickle.py b/Lib/test/test_pickle.py --- a/Lib/test/test_pickle.py +++ b/Lib/test/test_pickle.py @@ -10,6 +10,7 @@ import unittest from test import support +from test.pickletester import AbstractUnpickleTests from test.pickletester import AbstractPickleTests from test.pickletester import AbstractPickleModuleTests from test.pickletester import AbstractPersistentPicklerTests @@ -28,6 +29,16 @@ pass +class PyUnpicklerTests(AbstractUnpickleTests): + + unpickler = pickle._Unpickler + + def loads(self, buf, **kwds): + f = io.BytesIO(buf) + u = self.unpickler(f, **kwds) + return u.load() + + class PyPicklerTests(AbstractPickleTests): pickler = pickle._Pickler @@ -46,7 +57,8 @@ return u.load() -class InMemoryPickleTests(AbstractPickleTests, BigmemPickleTests): +class InMemoryPickleTests(AbstractPickleTests, AbstractUnpickleTests, + BigmemPickleTests): pickler = pickle._Pickler unpickler = pickle._Unpickler @@ -105,6 +117,9 @@ if has_c_implementation: + class CUnpicklerTests(PyUnpicklerTests): + unpickler = _pickle.Unpickler + class CPicklerTests(PyPicklerTests): pickler = _pickle.Pickler unpickler = _pickle.Unpickler @@ -375,11 +390,11 @@ def test_main(): - tests = [PickleTests, PyPicklerTests, PyPersPicklerTests, + tests = [PickleTests, PyUnpicklerTests, PyPicklerTests, PyPersPicklerTests, PyDispatchTableTests, PyChainDispatchTableTests, CompatPickleTests] if has_c_implementation: - tests.extend([CPicklerTests, CPersPicklerTests, + tests.extend([CUnpicklerTests, CPicklerTests, CPersPicklerTests, CDumpPickle_LoadPickle, DumpPickle_CLoadPickle, PyPicklerUnpicklerObjectTests, CPicklerUnpicklerObjectTests, -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 29 14:59:13 2015 From: python-checkins at python.org (serhiy.storchaka) Date: Tue, 29 Sep 2015 12:59:13 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_Moved_unpickli?= =?utf-8?q?ng_tests_with_prepickled_data_to_separate_class=2E?= Message-ID: <20150929125910.115289.93142@psf.io> https://hg.python.org/cpython/rev/85bc9aa3a006 changeset: 98385:85bc9aa3a006 branch: 3.4 parent: 98376:8103dadb9ef7 user: Serhiy Storchaka date: Tue Sep 29 15:33:24 2015 +0300 summary: Moved unpickling tests with prepickled data to separate class. files: Lib/test/pickletester.py | 424 +++++++++++++------------- Lib/test/test_pickle.py | 21 +- 2 files changed, 233 insertions(+), 212 deletions(-) diff --git a/Lib/test/pickletester.py b/Lib/test/pickletester.py --- a/Lib/test/pickletester.py +++ b/Lib/test/pickletester.py @@ -18,6 +18,9 @@ from pickle import bytes_types +requires_32b = unittest.skipUnless(sys.maxsize < 2**32, + "test is only meaningful on 32-bit builds") + # Tests that try a number of pickle protocols should have a # for proto in protocols: # kind of outer loop. @@ -502,16 +505,11 @@ return x -class AbstractPickleTests(unittest.TestCase): - # Subclass must define self.dumps, self.loads. - - optimized = False +class AbstractUnpickleTests(unittest.TestCase): + # Subclass must define self.loads. _testdata = create_data() - def setUp(self): - pass - def assert_is_copy(self, obj, objcopy, msg=None): """Utility method to verify if two objects are copies of each others. """ @@ -530,33 +528,6 @@ self.assertEqual(getattr(obj, slot, None), getattr(objcopy, slot, None), msg=msg) - def test_misc(self): - # test various datatypes not tested by testdata - for proto in protocols: - x = myint(4) - s = self.dumps(x, proto) - y = self.loads(s) - self.assert_is_copy(x, y) - - x = (1, ()) - s = self.dumps(x, proto) - y = self.loads(s) - self.assert_is_copy(x, y) - - x = initarg(1, x) - s = self.dumps(x, proto) - y = self.loads(s) - self.assert_is_copy(x, y) - - # XXX test __reduce__ protocol? - - def test_roundtrip_equality(self): - expected = self._testdata - for proto in protocols: - s = self.dumps(expected, proto) - got = self.loads(s) - self.assert_is_copy(expected, got) - def test_load_from_data0(self): self.assert_is_copy(self._testdata, self.loads(DATA0)) @@ -623,6 +594,216 @@ b'q\x00oq\x01}q\x02b.').replace(b'X', xname) self.assert_is_copy(X(*args), self.loads(pickle2)) + def test_get(self): + self.assertRaises(KeyError, self.loads, b'g0\np0') + self.assert_is_copy([(100,), (100,)], + self.loads(b'((Kdtp0\nh\x00l.))')) + + def test_maxint64(self): + maxint64 = (1 << 63) - 1 + data = b'I' + str(maxint64).encode("ascii") + b'\n.' + got = self.loads(data) + self.assert_is_copy(maxint64, got) + + # Try too with a bogus literal. + data = b'I' + str(maxint64).encode("ascii") + b'JUNK\n.' + self.assertRaises(ValueError, self.loads, data) + + def test_pop_empty_stack(self): + # Test issue7455 + s = b'0' + self.assertRaises((pickle.UnpicklingError, IndexError), self.loads, s) + + def test_unpickle_from_2x(self): + # Unpickle non-trivial data from Python 2.x. + loaded = self.loads(DATA3) + self.assertEqual(loaded, set([1, 2])) + loaded = self.loads(DATA4) + self.assertEqual(type(loaded), type(range(0))) + self.assertEqual(list(loaded), list(range(5))) + loaded = self.loads(DATA5) + self.assertEqual(type(loaded), SimpleCookie) + self.assertEqual(list(loaded.keys()), ["key"]) + self.assertEqual(loaded["key"].value, "value") + + for (exc, data) in DATA7.items(): + loaded = self.loads(data) + self.assertIs(type(loaded), exc) + + loaded = self.loads(DATA8) + self.assertIs(type(loaded), Exception) + + loaded = self.loads(DATA9) + self.assertIs(type(loaded), UnicodeEncodeError) + self.assertEqual(loaded.object, "foo") + self.assertEqual(loaded.encoding, "ascii") + self.assertEqual(loaded.start, 0) + self.assertEqual(loaded.end, 1) + self.assertEqual(loaded.reason, "bad") + + def test_load_python2_str_as_bytes(self): + # From Python 2: pickle.dumps('a\x00\xa0', protocol=0) + self.assertEqual(self.loads(b"S'a\\x00\\xa0'\n.", + encoding="bytes"), b'a\x00\xa0') + # From Python 2: pickle.dumps('a\x00\xa0', protocol=1) + self.assertEqual(self.loads(b'U\x03a\x00\xa0.', + encoding="bytes"), b'a\x00\xa0') + # From Python 2: pickle.dumps('a\x00\xa0', protocol=2) + self.assertEqual(self.loads(b'\x80\x02U\x03a\x00\xa0.', + encoding="bytes"), b'a\x00\xa0') + + def test_load_python2_unicode_as_str(self): + # From Python 2: pickle.dumps(u'?', protocol=0) + self.assertEqual(self.loads(b'V\\u03c0\n.', + encoding='bytes'), '?') + # From Python 2: pickle.dumps(u'?', protocol=1) + self.assertEqual(self.loads(b'X\x02\x00\x00\x00\xcf\x80.', + encoding="bytes"), '?') + # From Python 2: pickle.dumps(u'?', protocol=2) + self.assertEqual(self.loads(b'\x80\x02X\x02\x00\x00\x00\xcf\x80.', + encoding="bytes"), '?') + + def test_load_long_python2_str_as_bytes(self): + # From Python 2: pickle.dumps('x' * 300, protocol=1) + self.assertEqual(self.loads(pickle.BINSTRING + + struct.pack("', '<\\\u1234>', '<\n>', '<\\>', '<\\\U00012345>', @@ -754,7 +930,6 @@ self.assert_is_copy(s, self.loads(p)) def test_ints(self): - import sys for proto in protocols: n = sys.maxsize while n: @@ -764,16 +939,6 @@ self.assert_is_copy(expected, n2) n = n >> 1 - def test_maxint64(self): - maxint64 = (1 << 63) - 1 - data = b'I' + str(maxint64).encode("ascii") + b'\n.' - got = self.loads(data) - self.assert_is_copy(maxint64, got) - - # Try too with a bogus literal. - data = b'I' + str(maxint64).encode("ascii") + b'JUNK\n.' - self.assertRaises(ValueError, self.loads, data) - def test_long(self): for proto in protocols: # 256 bytes is where LONG4 begins. @@ -826,11 +991,6 @@ loaded = self.loads(dumped) self.assert_is_copy(inst, loaded) - def test_pop_empty_stack(self): - # Test issue7455 - s = b'0' - self.assertRaises((pickle.UnpicklingError, IndexError), self.loads, s) - def test_metaclass(self): a = use_metaclass() for proto in protocols: @@ -1289,33 +1449,6 @@ for x_key, y_key in zip(x_keys, y_keys): self.assertIs(x_key, y_key) - def test_unpickle_from_2x(self): - # Unpickle non-trivial data from Python 2.x. - loaded = self.loads(DATA3) - self.assertEqual(loaded, set([1, 2])) - loaded = self.loads(DATA4) - self.assertEqual(type(loaded), type(range(0))) - self.assertEqual(list(loaded), list(range(5))) - loaded = self.loads(DATA5) - self.assertEqual(type(loaded), SimpleCookie) - self.assertEqual(list(loaded.keys()), ["key"]) - self.assertEqual(loaded["key"].value, "value") - - for (exc, data) in DATA7.items(): - loaded = self.loads(data) - self.assertIs(type(loaded), exc) - - loaded = self.loads(DATA8) - self.assertIs(type(loaded), Exception) - - loaded = self.loads(DATA9) - self.assertIs(type(loaded), UnicodeEncodeError) - self.assertEqual(loaded.object, "foo") - self.assertEqual(loaded.encoding, "ascii") - self.assertEqual(loaded.start, 0) - self.assertEqual(loaded.end, 1) - self.assertEqual(loaded.reason, "bad") - def test_pickle_to_2x(self): # Pickle non-trivial data with protocol 2, expecting that it yields # the same result as Python 2.x did. @@ -1326,35 +1459,6 @@ dumped = self.dumps(set([3]), 2) self.assertEqual(dumped, DATA6) - def test_load_python2_str_as_bytes(self): - # From Python 2: pickle.dumps('a\x00\xa0', protocol=0) - self.assertEqual(self.loads(b"S'a\\x00\\xa0'\n.", - encoding="bytes"), b'a\x00\xa0') - # From Python 2: pickle.dumps('a\x00\xa0', protocol=1) - self.assertEqual(self.loads(b'U\x03a\x00\xa0.', - encoding="bytes"), b'a\x00\xa0') - # From Python 2: pickle.dumps('a\x00\xa0', protocol=2) - self.assertEqual(self.loads(b'\x80\x02U\x03a\x00\xa0.', - encoding="bytes"), b'a\x00\xa0') - - def test_load_python2_unicode_as_str(self): - # From Python 2: pickle.dumps(u'?', protocol=0) - self.assertEqual(self.loads(b'V\\u03c0\n.', - encoding='bytes'), '?') - # From Python 2: pickle.dumps(u'?', protocol=1) - self.assertEqual(self.loads(b'X\x02\x00\x00\x00\xcf\x80.', - encoding="bytes"), '?') - # From Python 2: pickle.dumps(u'?', protocol=2) - self.assertEqual(self.loads(b'\x80\x02X\x02\x00\x00\x00\xcf\x80.', - encoding="bytes"), '?') - - def test_load_long_python2_str_as_bytes(self): - # From Python 2: pickle.dumps('x' * 300, protocol=1) - self.assertEqual(self.loads(pickle.BINSTRING + - struct.pack(" 2**32: - self.skipTest("test is only meaningful on 32-bit builds") - # XXX Pure Python pickle reads lengths as signed and passes - # them directly to read() (hence the EOFError) - with self.assertRaises((pickle.UnpicklingError, EOFError, - ValueError, OverflowError)): - self.loads(dumped) - - def test_negative_32b_binbytes(self): - # On 32-bit builds, a BINBYTES of 2**31 or more is refused - self.check_negative_32b_binXXX(b'\x80\x03B\xff\xff\xff\xffxyzq\x00.') - - def test_negative_32b_binunicode(self): - # On 32-bit builds, a BINUNICODE of 2**31 or more is refused - self.check_negative_32b_binXXX(b'\x80\x03X\xff\xff\xff\xffxyzq\x00.') - - def test_negative_put(self): - # Issue #12847 - dumped = b'Va\np-1\n.' - self.assertRaises(ValueError, self.loads, dumped) - - def test_negative_32b_binput(self): - # Issue #12847 - if sys.maxsize > 2**32: - self.skipTest("test is only meaningful on 32-bit builds") - dumped = b'\x80\x03X\x01\x00\x00\x00ar\xff\xff\xff\xff.' - self.assertRaises(ValueError, self.loads, dumped) - - def test_badly_escaped_string(self): - self.assertRaises(ValueError, self.loads, b"S'\\'\n.") - - def test_badly_quoted_string(self): - # Issue #17710 - badpickles = [b"S'\n.", - b'S"\n.', - b'S\' \n.', - b'S" \n.', - b'S\'"\n.', - b'S"\'\n.', - b"S' ' \n.", - b'S" " \n.', - b"S ''\n.", - b'S ""\n.', - b'S \n.', - b'S\n.', - b'S.'] - for p in badpickles: - self.assertRaises(pickle.UnpicklingError, self.loads, p) - - def test_correctly_quoted_string(self): - goodpickles = [(b"S''\n.", ''), - (b'S""\n.', ''), - (b'S"\\n"\n.', '\n'), - (b"S'\\n'\n.", '\n')] - for p, expected in goodpickles: - self.assertEqual(self.loads(p), expected) - def _check_pickling_with_opcode(self, obj, opcode, proto): pickled = self.dumps(obj, proto) self.assertTrue(opcode_in_pickle(opcode, pickled)) @@ -1553,14 +1594,6 @@ count_opcode(pickle.FRAME, pickled)) self.assertEqual(obj, self.loads(some_frames_pickle)) - def test_frame_readline(self): - pickled = b'\x80\x04\x95\x05\x00\x00\x00\x00\x00\x00\x00I42\n.' - # 0: \x80 PROTO 4 - # 2: \x95 FRAME 5 - # 11: I INT 42 - # 15: . STOP - self.assertEqual(self.loads(pickled), 42) - def test_nested_names(self): global Nested class Nested: @@ -1677,33 +1710,6 @@ self.assertIn(('c%s\n%s' % (mod, name)).encode(), pickled) self.assertIs(type(self.loads(pickled)), type(val)) - def test_compat_unpickle(self): - # xrange(1, 7) - pickled = b'\x80\x02c__builtin__\nxrange\nK\x01K\x07K\x01\x87R.' - unpickled = self.loads(pickled) - self.assertIs(type(unpickled), range) - self.assertEqual(unpickled, range(1, 7)) - self.assertEqual(list(unpickled), [1, 2, 3, 4, 5, 6]) - # reduce - pickled = b'\x80\x02c__builtin__\nreduce\n.' - self.assertIs(self.loads(pickled), functools.reduce) - # whichdb.whichdb - pickled = b'\x80\x02cwhichdb\nwhichdb\n.' - self.assertIs(self.loads(pickled), dbm.whichdb) - # Exception(), StandardError() - for name in (b'Exception', b'StandardError'): - pickled = (b'\x80\x02cexceptions\n' + name + b'\nU\x03ugh\x85R.') - unpickled = self.loads(pickled) - self.assertIs(type(unpickled), Exception) - self.assertEqual(str(unpickled), 'ugh') - # UserDict.UserDict({1: 2}), UserDict.IterableUserDict({1: 2}) - for name in (b'UserDict', b'IterableUserDict'): - pickled = (b'\x80\x02(cUserDict\n' + name + - b'\no}U\x04data}K\x01K\x02ssb.') - unpickled = self.loads(pickled) - self.assertIs(type(unpickled), collections.UserDict) - self.assertEqual(unpickled, collections.UserDict({1: 2})) - class BigmemPickleTests(unittest.TestCase): diff --git a/Lib/test/test_pickle.py b/Lib/test/test_pickle.py --- a/Lib/test/test_pickle.py +++ b/Lib/test/test_pickle.py @@ -10,6 +10,7 @@ import unittest from test import support +from test.pickletester import AbstractUnpickleTests from test.pickletester import AbstractPickleTests from test.pickletester import AbstractPickleModuleTests from test.pickletester import AbstractPersistentPicklerTests @@ -28,6 +29,16 @@ pass +class PyUnpicklerTests(AbstractUnpickleTests): + + unpickler = pickle._Unpickler + + def loads(self, buf, **kwds): + f = io.BytesIO(buf) + u = self.unpickler(f, **kwds) + return u.load() + + class PyPicklerTests(AbstractPickleTests): pickler = pickle._Pickler @@ -46,7 +57,8 @@ return u.load() -class InMemoryPickleTests(AbstractPickleTests, BigmemPickleTests): +class InMemoryPickleTests(AbstractPickleTests, AbstractUnpickleTests, + BigmemPickleTests): pickler = pickle._Pickler unpickler = pickle._Unpickler @@ -105,6 +117,9 @@ if has_c_implementation: + class CUnpicklerTests(PyUnpicklerTests): + unpickler = _pickle.Unpickler + class CPicklerTests(PyPicklerTests): pickler = _pickle.Pickler unpickler = _pickle.Unpickler @@ -372,11 +387,11 @@ def test_main(): - tests = [PickleTests, PyPicklerTests, PyPersPicklerTests, + tests = [PickleTests, PyUnpicklerTests, PyPicklerTests, PyPersPicklerTests, PyDispatchTableTests, PyChainDispatchTableTests, CompatPickleTests] if has_c_implementation: - tests.extend([CPicklerTests, CPersPicklerTests, + tests.extend([CUnpicklerTests, CPicklerTests, CPersPicklerTests, CDumpPickle_LoadPickle, DumpPickle_CLoadPickle, PyPicklerUnpicklerObjectTests, CPicklerUnpicklerObjectTests, -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 29 14:59:14 2015 From: python-checkins at python.org (serhiy.storchaka) Date: Tue, 29 Sep 2015 12:59:14 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Added_additional_unpickling_tests=2E?= Message-ID: <20150929125911.82666.87345@psf.io> https://hg.python.org/cpython/rev/377f200ad521 changeset: 98391:377f200ad521 parent: 98387:ce72f75275df parent: 98390:a4bf231d81d3 user: Serhiy Storchaka date: Tue Sep 29 15:51:02 2015 +0300 summary: Added additional unpickling tests. files: Lib/test/pickletester.py | 266 +++++++++++++++++++++++--- 1 files changed, 229 insertions(+), 37 deletions(-) diff --git a/Lib/test/pickletester.py b/Lib/test/pickletester.py --- a/Lib/test/pickletester.py +++ b/Lib/test/pickletester.py @@ -145,7 +145,7 @@ result.reduce_args = (name, bases) return result -# DATA0 .. DATA2 are the pickles we expect under the various protocols, for +# DATA0 .. DATA4 are the pickles we expect under the various protocols, for # the object returned by create_data(). DATA0 = ( @@ -401,22 +401,172 @@ highest protocol among opcodes = 2 """ +DATA3 = ( + b'\x80\x03]q\x00(K\x00K\x01G@\x00\x00\x00\x00\x00\x00\x00c' + b'builtins\ncomplex\nq\x01G' + b'@\x08\x00\x00\x00\x00\x00\x00G\x00\x00\x00\x00\x00\x00\x00\x00\x86q\x02' + b'Rq\x03K\x01J\xff\xff\xff\xffK\xffJ\x01\xff\xff\xffJ\x00\xff' + b'\xff\xffM\xff\xffJ\x01\x00\xff\xffJ\x00\x00\xff\xffJ\xff\xff\xff\x7f' + b'J\x01\x00\x00\x80J\x00\x00\x00\x80(X\x03\x00\x00\x00abcq' + b'\x04h\x04c__main__\nC\nq\x05)\x81q' + b'\x06}q\x07(X\x03\x00\x00\x00barq\x08K\x02X\x03\x00' + b'\x00\x00fooq\tK\x01ubh\x06tq\nh\nK\x05' + b'e.' +) + +# Disassembly of DATA3 +DATA3_DIS = """\ + 0: \x80 PROTO 3 + 2: ] EMPTY_LIST + 3: q BINPUT 0 + 5: ( MARK + 6: K BININT1 0 + 8: K BININT1 1 + 10: G BINFLOAT 2.0 + 19: c GLOBAL 'builtins complex' + 37: q BINPUT 1 + 39: G BINFLOAT 3.0 + 48: G BINFLOAT 0.0 + 57: \x86 TUPLE2 + 58: q BINPUT 2 + 60: R REDUCE + 61: q BINPUT 3 + 63: K BININT1 1 + 65: J BININT -1 + 70: K BININT1 255 + 72: J BININT -255 + 77: J BININT -256 + 82: M BININT2 65535 + 85: J BININT -65535 + 90: J BININT -65536 + 95: J BININT 2147483647 + 100: J BININT -2147483647 + 105: J BININT -2147483648 + 110: ( MARK + 111: X BINUNICODE 'abc' + 119: q BINPUT 4 + 121: h BINGET 4 + 123: c GLOBAL '__main__ C' + 135: q BINPUT 5 + 137: ) EMPTY_TUPLE + 138: \x81 NEWOBJ + 139: q BINPUT 6 + 141: } EMPTY_DICT + 142: q BINPUT 7 + 144: ( MARK + 145: X BINUNICODE 'bar' + 153: q BINPUT 8 + 155: K BININT1 2 + 157: X BINUNICODE 'foo' + 165: q BINPUT 9 + 167: K BININT1 1 + 169: u SETITEMS (MARK at 144) + 170: b BUILD + 171: h BINGET 6 + 173: t TUPLE (MARK at 110) + 174: q BINPUT 10 + 176: h BINGET 10 + 178: K BININT1 5 + 180: e APPENDS (MARK at 5) + 181: . STOP +highest protocol among opcodes = 2 +""" + +DATA4 = ( + b'\x80\x04\x95\xa8\x00\x00\x00\x00\x00\x00\x00]\x94(K\x00K\x01G@' + b'\x00\x00\x00\x00\x00\x00\x00\x8c\x08builtins\x94\x8c\x07' + b'complex\x94\x93\x94G@\x08\x00\x00\x00\x00\x00\x00G' + b'\x00\x00\x00\x00\x00\x00\x00\x00\x86\x94R\x94K\x01J\xff\xff\xff\xffK' + b'\xffJ\x01\xff\xff\xffJ\x00\xff\xff\xffM\xff\xffJ\x01\x00\xff\xffJ' + b'\x00\x00\xff\xffJ\xff\xff\xff\x7fJ\x01\x00\x00\x80J\x00\x00\x00\x80(' + b'\x8c\x03abc\x94h\x06\x8c\x08__main__\x94\x8c' + b'\x01C\x94\x93\x94)\x81\x94}\x94(\x8c\x03bar\x94K\x02\x8c' + b'\x03foo\x94K\x01ubh\nt\x94h\x0eK\x05e.' +) + +# Disassembly of DATA4 +DATA4_DIS = """\ + 0: \x80 PROTO 4 + 2: \x95 FRAME 168 + 11: ] EMPTY_LIST + 12: \x94 MEMOIZE + 13: ( MARK + 14: K BININT1 0 + 16: K BININT1 1 + 18: G BINFLOAT 2.0 + 27: \x8c SHORT_BINUNICODE 'builtins' + 37: \x94 MEMOIZE + 38: \x8c SHORT_BINUNICODE 'complex' + 47: \x94 MEMOIZE + 48: \x93 STACK_GLOBAL + 49: \x94 MEMOIZE + 50: G BINFLOAT 3.0 + 59: G BINFLOAT 0.0 + 68: \x86 TUPLE2 + 69: \x94 MEMOIZE + 70: R REDUCE + 71: \x94 MEMOIZE + 72: K BININT1 1 + 74: J BININT -1 + 79: K BININT1 255 + 81: J BININT -255 + 86: J BININT -256 + 91: M BININT2 65535 + 94: J BININT -65535 + 99: J BININT -65536 + 104: J BININT 2147483647 + 109: J BININT -2147483647 + 114: J BININT -2147483648 + 119: ( MARK + 120: \x8c SHORT_BINUNICODE 'abc' + 125: \x94 MEMOIZE + 126: h BINGET 6 + 128: \x8c SHORT_BINUNICODE '__main__' + 138: \x94 MEMOIZE + 139: \x8c SHORT_BINUNICODE 'C' + 142: \x94 MEMOIZE + 143: \x93 STACK_GLOBAL + 144: \x94 MEMOIZE + 145: ) EMPTY_TUPLE + 146: \x81 NEWOBJ + 147: \x94 MEMOIZE + 148: } EMPTY_DICT + 149: \x94 MEMOIZE + 150: ( MARK + 151: \x8c SHORT_BINUNICODE 'bar' + 156: \x94 MEMOIZE + 157: K BININT1 2 + 159: \x8c SHORT_BINUNICODE 'foo' + 164: \x94 MEMOIZE + 165: K BININT1 1 + 167: u SETITEMS (MARK at 150) + 168: b BUILD + 169: h BINGET 10 + 171: t TUPLE (MARK at 119) + 172: \x94 MEMOIZE + 173: h BINGET 14 + 175: K BININT1 5 + 177: e APPENDS (MARK at 13) + 178: . STOP +highest protocol among opcodes = 4 +""" + # set([1,2]) pickled from 2.x with protocol 2 -DATA3 = b'\x80\x02c__builtin__\nset\nq\x00]q\x01(K\x01K\x02e\x85q\x02Rq\x03.' +DATA_SET = b'\x80\x02c__builtin__\nset\nq\x00]q\x01(K\x01K\x02e\x85q\x02Rq\x03.' # xrange(5) pickled from 2.x with protocol 2 -DATA4 = b'\x80\x02c__builtin__\nxrange\nq\x00K\x00K\x05K\x01\x87q\x01Rq\x02.' +DATA_XRANGE = b'\x80\x02c__builtin__\nxrange\nq\x00K\x00K\x05K\x01\x87q\x01Rq\x02.' # a SimpleCookie() object pickled from 2.x with protocol 2 -DATA5 = (b'\x80\x02cCookie\nSimpleCookie\nq\x00)\x81q\x01U\x03key' - b'q\x02cCookie\nMorsel\nq\x03)\x81q\x04(U\x07commentq\x05U' - b'\x00q\x06U\x06domainq\x07h\x06U\x06secureq\x08h\x06U\x07' - b'expiresq\th\x06U\x07max-ageq\nh\x06U\x07versionq\x0bh\x06U' - b'\x04pathq\x0ch\x06U\x08httponlyq\rh\x06u}q\x0e(U\x0b' - b'coded_valueq\x0fU\x05valueq\x10h\x10h\x10h\x02h\x02ubs}q\x11b.') +DATA_COOKIE = (b'\x80\x02cCookie\nSimpleCookie\nq\x00)\x81q\x01U\x03key' + b'q\x02cCookie\nMorsel\nq\x03)\x81q\x04(U\x07commentq\x05U' + b'\x00q\x06U\x06domainq\x07h\x06U\x06secureq\x08h\x06U\x07' + b'expiresq\th\x06U\x07max-ageq\nh\x06U\x07versionq\x0bh\x06U' + b'\x04pathq\x0ch\x06U\x08httponlyq\rh\x06u}q\x0e(U\x0b' + b'coded_valueq\x0fU\x05valueq\x10h\x10h\x10h\x02h\x02ubs}q\x11b.') # set([3]) pickled from 2.x with protocol 2 -DATA6 = b'\x80\x02c__builtin__\nset\nq\x00]q\x01K\x03a\x85q\x02Rq\x03.' +DATA_SET2 = b'\x80\x02c__builtin__\nset\nq\x00]q\x01K\x03a\x85q\x02Rq\x03.' python2_exceptions_without_args = ( ArithmeticError, @@ -468,20 +618,10 @@ exception_pickle = b'\x80\x02cexceptions\n?\nq\x00)Rq\x01.' -# Exception objects without arguments pickled from 2.x with protocol 2 -DATA7 = { - exception : - exception_pickle.replace(b'?', exception.__name__.encode("ascii")) - for exception in python2_exceptions_without_args -} - -# StandardError is mapped to Exception, test that separately -DATA8 = exception_pickle.replace(b'?', b'StandardError') - # UnicodeEncodeError object pickled from 2.x with protocol 2 -DATA9 = (b'\x80\x02cexceptions\nUnicodeEncodeError\n' - b'q\x00(U\x05asciiq\x01X\x03\x00\x00\x00fooq\x02K\x00K\x01' - b'U\x03badq\x03tq\x04Rq\x05.') +DATA_UEERR = (b'\x80\x02cexceptions\nUnicodeEncodeError\n' + b'q\x00(U\x05asciiq\x01X\x03\x00\x00\x00fooq\x02K\x00K\x01' + b'U\x03badq\x03tq\x04Rq\x05.') def create_data(): @@ -537,6 +677,12 @@ def test_load_from_data2(self): self.assert_is_copy(self._testdata, self.loads(DATA2)) + def test_load_from_data3(self): + self.assert_is_copy(self._testdata, self.loads(DATA3)) + + def test_load_from_data4(self): + self.assert_is_copy(self._testdata, self.loads(DATA4)) + def test_load_classic_instance(self): # See issue5180. Test loading 2.x pickles that # contain an instance of old style class. @@ -594,11 +740,6 @@ b'q\x00oq\x01}q\x02b.').replace(b'X', xname) self.assert_is_copy(X(*args), self.loads(pickle2)) - def test_get(self): - self.assertRaises(KeyError, self.loads, b'g0\np0') - self.assert_is_copy([(100,), (100,)], - self.loads(b'((Kdtp0\nh\x00l.))')) - def test_maxint64(self): maxint64 = (1 << 63) - 1 data = b'I' + str(maxint64).encode("ascii") + b'\n.' @@ -616,24 +757,27 @@ def test_unpickle_from_2x(self): # Unpickle non-trivial data from Python 2.x. - loaded = self.loads(DATA3) + loaded = self.loads(DATA_SET) self.assertEqual(loaded, set([1, 2])) - loaded = self.loads(DATA4) + loaded = self.loads(DATA_XRANGE) self.assertEqual(type(loaded), type(range(0))) self.assertEqual(list(loaded), list(range(5))) - loaded = self.loads(DATA5) + loaded = self.loads(DATA_COOKIE) self.assertEqual(type(loaded), SimpleCookie) self.assertEqual(list(loaded.keys()), ["key"]) self.assertEqual(loaded["key"].value, "value") - for (exc, data) in DATA7.items(): + # Exception objects without arguments pickled from 2.x with protocol 2 + for exc in python2_exceptions_without_args: + data = exception_pickle.replace(b'?', exc.__name__.encode("ascii")) loaded = self.loads(data) self.assertIs(type(loaded), exc) - loaded = self.loads(DATA8) + # StandardError is mapped to Exception, test that separately + loaded = self.loads(exception_pickle.replace(b'?', b'StandardError')) self.assertIs(type(loaded), Exception) - loaded = self.loads(DATA9) + loaded = self.loads(DATA_UEERR) self.assertIs(type(loaded), UnicodeEncodeError) self.assertEqual(loaded.object, "foo") self.assertEqual(loaded.encoding, "ascii") @@ -670,11 +814,26 @@ b'x' * 300 + pickle.STOP, encoding='bytes'), b'x' * 300) + def test_constants(self): + self.assertIsNone(self.loads(b'N.')) + self.assertIs(self.loads(b'\x88.'), True) + self.assertIs(self.loads(b'\x89.'), False) + self.assertIs(self.loads(b'I01\n.'), True) + self.assertIs(self.loads(b'I00\n.'), False) + def test_empty_bytestring(self): # issue 11286 empty = self.loads(b'\x80\x03U\x00q\x00.', encoding='koi8-r') self.assertEqual(empty, '') + def test_short_binbytes(self): + dumped = b'\x80\x03C\x04\xe2\x82\xac\x00.' + self.assertEqual(self.loads(dumped), b'\xe2\x82\xac\x00') + + def test_binbytes(self): + dumped = b'\x80\x03B\x04\x00\x00\x00\xe2\x82\xac\x00.' + self.assertEqual(self.loads(dumped), b'\xe2\x82\xac\x00') + @requires_32b def test_negative_32b_binbytes(self): # On 32-bit builds, a BINBYTES of 2**31 or more is refused @@ -689,6 +848,39 @@ with self.assertRaises((pickle.UnpicklingError, OverflowError)): self.loads(dumped) + def test_short_binunicode(self): + dumped = b'\x80\x04\x8c\x04\xe2\x82\xac\x00.' + self.assertEqual(self.loads(dumped), '\u20ac\x00') + + def test_misc_get(self): + self.assertRaises(KeyError, self.loads, b'g0\np0') + self.assert_is_copy([(100,), (100,)], + self.loads(b'((Kdtp0\nh\x00l.))')) + + def test_get(self): + pickled = b'((lp100000\ng100000\nt.' + unpickled = self.loads(pickled) + self.assertEqual(unpickled, ([],)*2) + self.assertIs(unpickled[0], unpickled[1]) + + def test_binget(self): + pickled = b'(]q\xffh\xfft.' + unpickled = self.loads(pickled) + self.assertEqual(unpickled, ([],)*2) + self.assertIs(unpickled[0], unpickled[1]) + + def test_long_binget(self): + pickled = b'(]r\x00\x00\x01\x00j\x00\x00\x01\x00t.' + unpickled = self.loads(pickled) + self.assertEqual(unpickled, ([],)*2) + self.assertIs(unpickled[0], unpickled[1]) + + def test_dup(self): + pickled = b'((l2t.' + unpickled = self.loads(pickled) + self.assertEqual(unpickled, ([],)*2) + self.assertIs(unpickled[0], unpickled[1]) + def test_negative_put(self): # Issue #12847 dumped = b'Va\np-1\n.' @@ -1501,9 +1693,9 @@ # NOTE: this test is a bit too strong since we can produce different # bytecode that 2.x will still understand. dumped = self.dumps(range(5), 2) - self.assertEqual(dumped, DATA4) + self.assertEqual(dumped, DATA_XRANGE) dumped = self.dumps(set([3]), 2) - self.assertEqual(dumped, DATA6) + self.assertEqual(dumped, DATA_SET2) def test_large_pickles(self): # Test the correctness of internal buffering routines when handling @@ -2398,7 +2590,7 @@ # Print some stuff that can be used to rewrite DATA{0,1,2} from pickletools import dis x = create_data() - for i in range(3): + for i in range(pickle.HIGHEST_PROTOCOL+1): p = pickle.dumps(x, i) print("DATA{0} = (".format(i)) for j in range(0, len(p), 20): -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 29 14:59:14 2015 From: python-checkins at python.org (serhiy.storchaka) Date: Tue, 29 Sep 2015 12:59:14 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_Added_addition?= =?utf-8?q?al_unpickling_tests=2E?= Message-ID: <20150929125911.82640.93057@psf.io> https://hg.python.org/cpython/rev/3c2bcdff10a2 changeset: 98389:3c2bcdff10a2 branch: 3.4 parent: 98385:85bc9aa3a006 user: Serhiy Storchaka date: Tue Sep 29 15:49:58 2015 +0300 summary: Added additional unpickling tests. files: Lib/test/pickletester.py | 266 +++++++++++++++++++++++--- 1 files changed, 229 insertions(+), 37 deletions(-) diff --git a/Lib/test/pickletester.py b/Lib/test/pickletester.py --- a/Lib/test/pickletester.py +++ b/Lib/test/pickletester.py @@ -145,7 +145,7 @@ result.reduce_args = (name, bases) return result -# DATA0 .. DATA2 are the pickles we expect under the various protocols, for +# DATA0 .. DATA4 are the pickles we expect under the various protocols, for # the object returned by create_data(). DATA0 = ( @@ -401,22 +401,172 @@ highest protocol among opcodes = 2 """ +DATA3 = ( + b'\x80\x03]q\x00(K\x00K\x01G@\x00\x00\x00\x00\x00\x00\x00c' + b'builtins\ncomplex\nq\x01G' + b'@\x08\x00\x00\x00\x00\x00\x00G\x00\x00\x00\x00\x00\x00\x00\x00\x86q\x02' + b'Rq\x03K\x01J\xff\xff\xff\xffK\xffJ\x01\xff\xff\xffJ\x00\xff' + b'\xff\xffM\xff\xffJ\x01\x00\xff\xffJ\x00\x00\xff\xffJ\xff\xff\xff\x7f' + b'J\x01\x00\x00\x80J\x00\x00\x00\x80(X\x03\x00\x00\x00abcq' + b'\x04h\x04c__main__\nC\nq\x05)\x81q' + b'\x06}q\x07(X\x03\x00\x00\x00barq\x08K\x02X\x03\x00' + b'\x00\x00fooq\tK\x01ubh\x06tq\nh\nK\x05' + b'e.' +) + +# Disassembly of DATA3 +DATA3_DIS = """\ + 0: \x80 PROTO 3 + 2: ] EMPTY_LIST + 3: q BINPUT 0 + 5: ( MARK + 6: K BININT1 0 + 8: K BININT1 1 + 10: G BINFLOAT 2.0 + 19: c GLOBAL 'builtins complex' + 37: q BINPUT 1 + 39: G BINFLOAT 3.0 + 48: G BINFLOAT 0.0 + 57: \x86 TUPLE2 + 58: q BINPUT 2 + 60: R REDUCE + 61: q BINPUT 3 + 63: K BININT1 1 + 65: J BININT -1 + 70: K BININT1 255 + 72: J BININT -255 + 77: J BININT -256 + 82: M BININT2 65535 + 85: J BININT -65535 + 90: J BININT -65536 + 95: J BININT 2147483647 + 100: J BININT -2147483647 + 105: J BININT -2147483648 + 110: ( MARK + 111: X BINUNICODE 'abc' + 119: q BINPUT 4 + 121: h BINGET 4 + 123: c GLOBAL '__main__ C' + 135: q BINPUT 5 + 137: ) EMPTY_TUPLE + 138: \x81 NEWOBJ + 139: q BINPUT 6 + 141: } EMPTY_DICT + 142: q BINPUT 7 + 144: ( MARK + 145: X BINUNICODE 'bar' + 153: q BINPUT 8 + 155: K BININT1 2 + 157: X BINUNICODE 'foo' + 165: q BINPUT 9 + 167: K BININT1 1 + 169: u SETITEMS (MARK at 144) + 170: b BUILD + 171: h BINGET 6 + 173: t TUPLE (MARK at 110) + 174: q BINPUT 10 + 176: h BINGET 10 + 178: K BININT1 5 + 180: e APPENDS (MARK at 5) + 181: . STOP +highest protocol among opcodes = 2 +""" + +DATA4 = ( + b'\x80\x04\x95\xa8\x00\x00\x00\x00\x00\x00\x00]\x94(K\x00K\x01G@' + b'\x00\x00\x00\x00\x00\x00\x00\x8c\x08builtins\x94\x8c\x07' + b'complex\x94\x93\x94G@\x08\x00\x00\x00\x00\x00\x00G' + b'\x00\x00\x00\x00\x00\x00\x00\x00\x86\x94R\x94K\x01J\xff\xff\xff\xffK' + b'\xffJ\x01\xff\xff\xffJ\x00\xff\xff\xffM\xff\xffJ\x01\x00\xff\xffJ' + b'\x00\x00\xff\xffJ\xff\xff\xff\x7fJ\x01\x00\x00\x80J\x00\x00\x00\x80(' + b'\x8c\x03abc\x94h\x06\x8c\x08__main__\x94\x8c' + b'\x01C\x94\x93\x94)\x81\x94}\x94(\x8c\x03bar\x94K\x02\x8c' + b'\x03foo\x94K\x01ubh\nt\x94h\x0eK\x05e.' +) + +# Disassembly of DATA4 +DATA4_DIS = """\ + 0: \x80 PROTO 4 + 2: \x95 FRAME 168 + 11: ] EMPTY_LIST + 12: \x94 MEMOIZE + 13: ( MARK + 14: K BININT1 0 + 16: K BININT1 1 + 18: G BINFLOAT 2.0 + 27: \x8c SHORT_BINUNICODE 'builtins' + 37: \x94 MEMOIZE + 38: \x8c SHORT_BINUNICODE 'complex' + 47: \x94 MEMOIZE + 48: \x93 STACK_GLOBAL + 49: \x94 MEMOIZE + 50: G BINFLOAT 3.0 + 59: G BINFLOAT 0.0 + 68: \x86 TUPLE2 + 69: \x94 MEMOIZE + 70: R REDUCE + 71: \x94 MEMOIZE + 72: K BININT1 1 + 74: J BININT -1 + 79: K BININT1 255 + 81: J BININT -255 + 86: J BININT -256 + 91: M BININT2 65535 + 94: J BININT -65535 + 99: J BININT -65536 + 104: J BININT 2147483647 + 109: J BININT -2147483647 + 114: J BININT -2147483648 + 119: ( MARK + 120: \x8c SHORT_BINUNICODE 'abc' + 125: \x94 MEMOIZE + 126: h BINGET 6 + 128: \x8c SHORT_BINUNICODE '__main__' + 138: \x94 MEMOIZE + 139: \x8c SHORT_BINUNICODE 'C' + 142: \x94 MEMOIZE + 143: \x93 STACK_GLOBAL + 144: \x94 MEMOIZE + 145: ) EMPTY_TUPLE + 146: \x81 NEWOBJ + 147: \x94 MEMOIZE + 148: } EMPTY_DICT + 149: \x94 MEMOIZE + 150: ( MARK + 151: \x8c SHORT_BINUNICODE 'bar' + 156: \x94 MEMOIZE + 157: K BININT1 2 + 159: \x8c SHORT_BINUNICODE 'foo' + 164: \x94 MEMOIZE + 165: K BININT1 1 + 167: u SETITEMS (MARK at 150) + 168: b BUILD + 169: h BINGET 10 + 171: t TUPLE (MARK at 119) + 172: \x94 MEMOIZE + 173: h BINGET 14 + 175: K BININT1 5 + 177: e APPENDS (MARK at 13) + 178: . STOP +highest protocol among opcodes = 4 +""" + # set([1,2]) pickled from 2.x with protocol 2 -DATA3 = b'\x80\x02c__builtin__\nset\nq\x00]q\x01(K\x01K\x02e\x85q\x02Rq\x03.' +DATA_SET = b'\x80\x02c__builtin__\nset\nq\x00]q\x01(K\x01K\x02e\x85q\x02Rq\x03.' # xrange(5) pickled from 2.x with protocol 2 -DATA4 = b'\x80\x02c__builtin__\nxrange\nq\x00K\x00K\x05K\x01\x87q\x01Rq\x02.' +DATA_XRANGE = b'\x80\x02c__builtin__\nxrange\nq\x00K\x00K\x05K\x01\x87q\x01Rq\x02.' # a SimpleCookie() object pickled from 2.x with protocol 2 -DATA5 = (b'\x80\x02cCookie\nSimpleCookie\nq\x00)\x81q\x01U\x03key' - b'q\x02cCookie\nMorsel\nq\x03)\x81q\x04(U\x07commentq\x05U' - b'\x00q\x06U\x06domainq\x07h\x06U\x06secureq\x08h\x06U\x07' - b'expiresq\th\x06U\x07max-ageq\nh\x06U\x07versionq\x0bh\x06U' - b'\x04pathq\x0ch\x06U\x08httponlyq\rh\x06u}q\x0e(U\x0b' - b'coded_valueq\x0fU\x05valueq\x10h\x10h\x10h\x02h\x02ubs}q\x11b.') +DATA_COOKIE = (b'\x80\x02cCookie\nSimpleCookie\nq\x00)\x81q\x01U\x03key' + b'q\x02cCookie\nMorsel\nq\x03)\x81q\x04(U\x07commentq\x05U' + b'\x00q\x06U\x06domainq\x07h\x06U\x06secureq\x08h\x06U\x07' + b'expiresq\th\x06U\x07max-ageq\nh\x06U\x07versionq\x0bh\x06U' + b'\x04pathq\x0ch\x06U\x08httponlyq\rh\x06u}q\x0e(U\x0b' + b'coded_valueq\x0fU\x05valueq\x10h\x10h\x10h\x02h\x02ubs}q\x11b.') # set([3]) pickled from 2.x with protocol 2 -DATA6 = b'\x80\x02c__builtin__\nset\nq\x00]q\x01K\x03a\x85q\x02Rq\x03.' +DATA_SET2 = b'\x80\x02c__builtin__\nset\nq\x00]q\x01K\x03a\x85q\x02Rq\x03.' python2_exceptions_without_args = ( ArithmeticError, @@ -468,20 +618,10 @@ exception_pickle = b'\x80\x02cexceptions\n?\nq\x00)Rq\x01.' -# Exception objects without arguments pickled from 2.x with protocol 2 -DATA7 = { - exception : - exception_pickle.replace(b'?', exception.__name__.encode("ascii")) - for exception in python2_exceptions_without_args -} - -# StandardError is mapped to Exception, test that separately -DATA8 = exception_pickle.replace(b'?', b'StandardError') - # UnicodeEncodeError object pickled from 2.x with protocol 2 -DATA9 = (b'\x80\x02cexceptions\nUnicodeEncodeError\n' - b'q\x00(U\x05asciiq\x01X\x03\x00\x00\x00fooq\x02K\x00K\x01' - b'U\x03badq\x03tq\x04Rq\x05.') +DATA_UEERR = (b'\x80\x02cexceptions\nUnicodeEncodeError\n' + b'q\x00(U\x05asciiq\x01X\x03\x00\x00\x00fooq\x02K\x00K\x01' + b'U\x03badq\x03tq\x04Rq\x05.') def create_data(): @@ -537,6 +677,12 @@ def test_load_from_data2(self): self.assert_is_copy(self._testdata, self.loads(DATA2)) + def test_load_from_data3(self): + self.assert_is_copy(self._testdata, self.loads(DATA3)) + + def test_load_from_data4(self): + self.assert_is_copy(self._testdata, self.loads(DATA4)) + def test_load_classic_instance(self): # See issue5180. Test loading 2.x pickles that # contain an instance of old style class. @@ -594,11 +740,6 @@ b'q\x00oq\x01}q\x02b.').replace(b'X', xname) self.assert_is_copy(X(*args), self.loads(pickle2)) - def test_get(self): - self.assertRaises(KeyError, self.loads, b'g0\np0') - self.assert_is_copy([(100,), (100,)], - self.loads(b'((Kdtp0\nh\x00l.))')) - def test_maxint64(self): maxint64 = (1 << 63) - 1 data = b'I' + str(maxint64).encode("ascii") + b'\n.' @@ -616,24 +757,27 @@ def test_unpickle_from_2x(self): # Unpickle non-trivial data from Python 2.x. - loaded = self.loads(DATA3) + loaded = self.loads(DATA_SET) self.assertEqual(loaded, set([1, 2])) - loaded = self.loads(DATA4) + loaded = self.loads(DATA_XRANGE) self.assertEqual(type(loaded), type(range(0))) self.assertEqual(list(loaded), list(range(5))) - loaded = self.loads(DATA5) + loaded = self.loads(DATA_COOKIE) self.assertEqual(type(loaded), SimpleCookie) self.assertEqual(list(loaded.keys()), ["key"]) self.assertEqual(loaded["key"].value, "value") - for (exc, data) in DATA7.items(): + # Exception objects without arguments pickled from 2.x with protocol 2 + for exc in python2_exceptions_without_args: + data = exception_pickle.replace(b'?', exc.__name__.encode("ascii")) loaded = self.loads(data) self.assertIs(type(loaded), exc) - loaded = self.loads(DATA8) + # StandardError is mapped to Exception, test that separately + loaded = self.loads(exception_pickle.replace(b'?', b'StandardError')) self.assertIs(type(loaded), Exception) - loaded = self.loads(DATA9) + loaded = self.loads(DATA_UEERR) self.assertIs(type(loaded), UnicodeEncodeError) self.assertEqual(loaded.object, "foo") self.assertEqual(loaded.encoding, "ascii") @@ -670,11 +814,26 @@ b'x' * 300 + pickle.STOP, encoding='bytes'), b'x' * 300) + def test_constants(self): + self.assertIsNone(self.loads(b'N.')) + self.assertIs(self.loads(b'\x88.'), True) + self.assertIs(self.loads(b'\x89.'), False) + self.assertIs(self.loads(b'I01\n.'), True) + self.assertIs(self.loads(b'I00\n.'), False) + def test_empty_bytestring(self): # issue 11286 empty = self.loads(b'\x80\x03U\x00q\x00.', encoding='koi8-r') self.assertEqual(empty, '') + def test_short_binbytes(self): + dumped = b'\x80\x03C\x04\xe2\x82\xac\x00.' + self.assertEqual(self.loads(dumped), b'\xe2\x82\xac\x00') + + def test_binbytes(self): + dumped = b'\x80\x03B\x04\x00\x00\x00\xe2\x82\xac\x00.' + self.assertEqual(self.loads(dumped), b'\xe2\x82\xac\x00') + @requires_32b def test_negative_32b_binbytes(self): # On 32-bit builds, a BINBYTES of 2**31 or more is refused @@ -689,6 +848,39 @@ with self.assertRaises((pickle.UnpicklingError, OverflowError)): self.loads(dumped) + def test_short_binunicode(self): + dumped = b'\x80\x04\x8c\x04\xe2\x82\xac\x00.' + self.assertEqual(self.loads(dumped), '\u20ac\x00') + + def test_misc_get(self): + self.assertRaises(KeyError, self.loads, b'g0\np0') + self.assert_is_copy([(100,), (100,)], + self.loads(b'((Kdtp0\nh\x00l.))')) + + def test_get(self): + pickled = b'((lp100000\ng100000\nt.' + unpickled = self.loads(pickled) + self.assertEqual(unpickled, ([],)*2) + self.assertIs(unpickled[0], unpickled[1]) + + def test_binget(self): + pickled = b'(]q\xffh\xfft.' + unpickled = self.loads(pickled) + self.assertEqual(unpickled, ([],)*2) + self.assertIs(unpickled[0], unpickled[1]) + + def test_long_binget(self): + pickled = b'(]r\x00\x00\x01\x00j\x00\x00\x01\x00t.' + unpickled = self.loads(pickled) + self.assertEqual(unpickled, ([],)*2) + self.assertIs(unpickled[0], unpickled[1]) + + def test_dup(self): + pickled = b'((l2t.' + unpickled = self.loads(pickled) + self.assertEqual(unpickled, ([],)*2) + self.assertIs(unpickled[0], unpickled[1]) + def test_negative_put(self): # Issue #12847 dumped = b'Va\np-1\n.' @@ -1455,9 +1647,9 @@ # NOTE: this test is a bit too strong since we can produce different # bytecode that 2.x will still understand. dumped = self.dumps(range(5), 2) - self.assertEqual(dumped, DATA4) + self.assertEqual(dumped, DATA_XRANGE) dumped = self.dumps(set([3]), 2) - self.assertEqual(dumped, DATA6) + self.assertEqual(dumped, DATA_SET2) def test_large_pickles(self): # Test the correctness of internal buffering routines when handling @@ -2312,7 +2504,7 @@ # Print some stuff that can be used to rewrite DATA{0,1,2} from pickletools import dis x = create_data() - for i in range(3): + for i in range(pickle.HIGHEST_PROTOCOL+1): p = pickle.dumps(x, i) print("DATA{0} = (".format(i)) for j in range(0, len(p), 20): -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 29 14:59:16 2015 From: python-checkins at python.org (serhiy.storchaka) Date: Tue, 29 Sep 2015 12:59:16 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_Moved_unpickli?= =?utf-8?q?ng_tests_with_prepickled_data_to_separate_class=2E?= Message-ID: <20150929125911.94133.76564@psf.io> https://hg.python.org/cpython/rev/157849879bce changeset: 98388:157849879bce branch: 2.7 parent: 98375:395d363e34e1 user: Serhiy Storchaka date: Tue Sep 29 15:36:28 2015 +0300 summary: Moved unpickling tests with prepickled data to separate class. files: Lib/test/pickletester.py | 83 +++++++++++++++------------ Lib/test/test_cpickle.py | 35 +++++++++-- Lib/test/test_pickle.py | 17 ++++- 3 files changed, 88 insertions(+), 47 deletions(-) diff --git a/Lib/test/pickletester.py b/Lib/test/pickletester.py --- a/Lib/test/pickletester.py +++ b/Lib/test/pickletester.py @@ -420,11 +420,54 @@ x.append(5) return x -class AbstractPickleTests(unittest.TestCase): - # Subclass must define self.dumps, self.loads, self.error. + +class AbstractUnpickleTests(unittest.TestCase): + # Subclass must define self.loads, self.error. _testdata = create_data() + def test_load_from_canned_string(self): + expected = self._testdata + for canned in DATA0, DATA1, DATA2: + got = self.loads(canned) + self.assertEqual(expected, got) + + def test_garyp(self): + self.assertRaises(self.error, self.loads, 'garyp') + + def test_maxint64(self): + maxint64 = (1L << 63) - 1 + data = 'I' + str(maxint64) + '\n.' + got = self.loads(data) + self.assertEqual(got, maxint64) + + # Try too with a bogus literal. + data = 'I' + str(maxint64) + 'JUNK\n.' + self.assertRaises(ValueError, self.loads, data) + + def test_insecure_strings(self): + insecure = ["abc", "2 + 2", # not quoted + #"'abc' + 'def'", # not a single quoted string + "'abc", # quote is not closed + "'abc\"", # open quote and close quote don't match + "'abc' ?", # junk after close quote + "'\\'", # trailing backslash + "'", # issue #17710 + "' ", # issue #17710 + # some tests of the quoting rules + #"'abc\"\''", + #"'\\\\a\'\'\'\\\'\\\\\''", + ] + for s in insecure: + buf = "S" + s + "\012p0\012." + self.assertRaises(ValueError, self.loads, buf) + + +class AbstractPickleTests(unittest.TestCase): + # Subclass must define self.dumps, self.loads. + + _testdata = AbstractUnpickleTests._testdata + def setUp(self): pass @@ -455,12 +498,6 @@ got = self.loads(s) self.assertEqual(expected, got) - def test_load_from_canned_string(self): - expected = self._testdata - for canned in DATA0, DATA1, DATA2: - got = self.loads(canned) - self.assertEqual(expected, got) - # There are gratuitous differences between pickles produced by # pickle and cPickle, largely because cPickle starts PUT indices at # 1 and pickle starts them at 0. See XXX comment in cPickle's put2() -- @@ -528,26 +565,6 @@ self.assertEqual(x[0].attr.keys(), [1]) self.assertTrue(x[0].attr[1] is x) - def test_garyp(self): - self.assertRaises(self.error, self.loads, 'garyp') - - def test_insecure_strings(self): - insecure = ["abc", "2 + 2", # not quoted - #"'abc' + 'def'", # not a single quoted string - "'abc", # quote is not closed - "'abc\"", # open quote and close quote don't match - "'abc' ?", # junk after close quote - "'\\'", # trailing backslash - "'", # issue #17710 - "' ", # issue #17710 - # some tests of the quoting rules - #"'abc\"\''", - #"'\\\\a\'\'\'\\\'\\\\\''", - ] - for s in insecure: - buf = "S" + s + "\012p0\012." - self.assertRaises(ValueError, self.loads, buf) - if have_unicode: def test_unicode(self): endcases = [u'', u'<\\u>', u'<\\\u1234>', u'<\n>', @@ -576,16 +593,6 @@ self.assertEqual(expected, n2) n = n >> 1 - def test_maxint64(self): - maxint64 = (1L << 63) - 1 - data = 'I' + str(maxint64) + '\n.' - got = self.loads(data) - self.assertEqual(got, maxint64) - - # Try too with a bogus literal. - data = 'I' + str(maxint64) + 'JUNK\n.' - self.assertRaises(ValueError, self.loads, data) - def test_long(self): for proto in protocols: # 256 bytes is where LONG4 begins. diff --git a/Lib/test/test_cpickle.py b/Lib/test/test_cpickle.py --- a/Lib/test/test_cpickle.py +++ b/Lib/test/test_cpickle.py @@ -2,7 +2,8 @@ import cStringIO import io import unittest -from test.pickletester import (AbstractPickleTests, +from test.pickletester import (AbstractUnpickleTests, + AbstractPickleTests, AbstractPickleModuleTests, AbstractPicklerUnpicklerObjectTests, BigmemPickleTests) @@ -40,7 +41,8 @@ test_support.unlink(test_support.TESTFN) -class cPickleTests(AbstractPickleTests, AbstractPickleModuleTests): +class cPickleTests(AbstractUnpickleTests, AbstractPickleTests, + AbstractPickleModuleTests): def setUp(self): self.dumps = cPickle.dumps @@ -49,6 +51,28 @@ error = cPickle.BadPickleGet module = cPickle +class cPickleUnpicklerTests(AbstractUnpickleTests): + + def loads(self, buf): + f = self.input(buf) + try: + p = cPickle.Unpickler(f) + return p.load() + finally: + self.close(f) + + error = cPickle.BadPickleGet + +class cStringIOCUnpicklerTests(cStringIOMixin, cPickleUnpicklerTests): + pass + +class BytesIOCUnpicklerTests(BytesIOMixin, cPickleUnpicklerTests): + pass + +class FileIOCUnpicklerTests(FileIOMixin, cPickleUnpicklerTests): + pass + + class cPicklePicklerTests(AbstractPickleTests): def dumps(self, arg, proto=0): @@ -69,8 +93,6 @@ finally: self.close(f) - error = cPickle.BadPickleGet - class cStringIOCPicklerTests(cStringIOMixin, cPicklePicklerTests): pass @@ -129,8 +151,6 @@ finally: self.close(f) - error = cPickle.BadPickleGet - def test_recursive_list(self): self.assertRaises(ValueError, AbstractPickleTests.test_recursive_list, @@ -219,6 +239,9 @@ def test_main(): test_support.run_unittest( cPickleTests, + cStringIOCUnpicklerTests, + BytesIOCUnpicklerTests, + FileIOCUnpicklerTests, cStringIOCPicklerTests, BytesIOCPicklerTests, FileIOCPicklerTests, diff --git a/Lib/test/test_pickle.py b/Lib/test/test_pickle.py --- a/Lib/test/test_pickle.py +++ b/Lib/test/test_pickle.py @@ -3,13 +3,15 @@ from test import test_support -from test.pickletester import (AbstractPickleTests, +from test.pickletester import (AbstractUnpickleTests, + AbstractPickleTests, AbstractPickleModuleTests, AbstractPersistentPicklerTests, AbstractPicklerUnpicklerObjectTests, BigmemPickleTests) -class PickleTests(AbstractPickleTests, AbstractPickleModuleTests): +class PickleTests(AbstractUnpickleTests, AbstractPickleTests, + AbstractPickleModuleTests): def dumps(self, arg, proto=0, fast=0): # Ignore fast @@ -22,10 +24,18 @@ module = pickle error = KeyError -class PicklerTests(AbstractPickleTests): +class UnpicklerTests(AbstractUnpickleTests): error = KeyError + def loads(self, buf): + f = StringIO(buf) + u = pickle.Unpickler(f) + return u.load() + + +class PicklerTests(AbstractPickleTests): + def dumps(self, arg, proto=0, fast=0): f = StringIO() p = pickle.Pickler(f, proto) @@ -81,6 +91,7 @@ def test_main(): test_support.run_unittest( PickleTests, + UnpicklerTests, PicklerTests, PersPicklerTests, PicklerUnpicklerObjectTests, -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 29 14:59:16 2015 From: python-checkins at python.org (serhiy.storchaka) Date: Tue, 29 Sep 2015 12:59:16 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Moved_unpickling_tests_with_prepickled_data_to_separate_class?= =?utf-8?q?=2E?= Message-ID: <20150929125910.81621.59485@psf.io> https://hg.python.org/cpython/rev/dc69a996ab7d changeset: 98386:dc69a996ab7d branch: 3.5 parent: 98377:3481c466926a parent: 98385:85bc9aa3a006 user: Serhiy Storchaka date: Tue Sep 29 15:34:53 2015 +0300 summary: Moved unpickling tests with prepickled data to separate class. files: Lib/test/pickletester.py | 424 +++++++++++++------------- Lib/test/test_pickle.py | 21 +- 2 files changed, 233 insertions(+), 212 deletions(-) diff --git a/Lib/test/pickletester.py b/Lib/test/pickletester.py --- a/Lib/test/pickletester.py +++ b/Lib/test/pickletester.py @@ -18,6 +18,9 @@ from pickle import bytes_types +requires_32b = unittest.skipUnless(sys.maxsize < 2**32, + "test is only meaningful on 32-bit builds") + # Tests that try a number of pickle protocols should have a # for proto in protocols: # kind of outer loop. @@ -502,16 +505,11 @@ return x -class AbstractPickleTests(unittest.TestCase): - # Subclass must define self.dumps, self.loads. - - optimized = False +class AbstractUnpickleTests(unittest.TestCase): + # Subclass must define self.loads. _testdata = create_data() - def setUp(self): - pass - def assert_is_copy(self, obj, objcopy, msg=None): """Utility method to verify if two objects are copies of each others. """ @@ -530,33 +528,6 @@ self.assertEqual(getattr(obj, slot, None), getattr(objcopy, slot, None), msg=msg) - def test_misc(self): - # test various datatypes not tested by testdata - for proto in protocols: - x = myint(4) - s = self.dumps(x, proto) - y = self.loads(s) - self.assert_is_copy(x, y) - - x = (1, ()) - s = self.dumps(x, proto) - y = self.loads(s) - self.assert_is_copy(x, y) - - x = initarg(1, x) - s = self.dumps(x, proto) - y = self.loads(s) - self.assert_is_copy(x, y) - - # XXX test __reduce__ protocol? - - def test_roundtrip_equality(self): - expected = self._testdata - for proto in protocols: - s = self.dumps(expected, proto) - got = self.loads(s) - self.assert_is_copy(expected, got) - def test_load_from_data0(self): self.assert_is_copy(self._testdata, self.loads(DATA0)) @@ -623,6 +594,216 @@ b'q\x00oq\x01}q\x02b.').replace(b'X', xname) self.assert_is_copy(X(*args), self.loads(pickle2)) + def test_get(self): + self.assertRaises(KeyError, self.loads, b'g0\np0') + self.assert_is_copy([(100,), (100,)], + self.loads(b'((Kdtp0\nh\x00l.))')) + + def test_maxint64(self): + maxint64 = (1 << 63) - 1 + data = b'I' + str(maxint64).encode("ascii") + b'\n.' + got = self.loads(data) + self.assert_is_copy(maxint64, got) + + # Try too with a bogus literal. + data = b'I' + str(maxint64).encode("ascii") + b'JUNK\n.' + self.assertRaises(ValueError, self.loads, data) + + def test_pop_empty_stack(self): + # Test issue7455 + s = b'0' + self.assertRaises((pickle.UnpicklingError, IndexError), self.loads, s) + + def test_unpickle_from_2x(self): + # Unpickle non-trivial data from Python 2.x. + loaded = self.loads(DATA3) + self.assertEqual(loaded, set([1, 2])) + loaded = self.loads(DATA4) + self.assertEqual(type(loaded), type(range(0))) + self.assertEqual(list(loaded), list(range(5))) + loaded = self.loads(DATA5) + self.assertEqual(type(loaded), SimpleCookie) + self.assertEqual(list(loaded.keys()), ["key"]) + self.assertEqual(loaded["key"].value, "value") + + for (exc, data) in DATA7.items(): + loaded = self.loads(data) + self.assertIs(type(loaded), exc) + + loaded = self.loads(DATA8) + self.assertIs(type(loaded), Exception) + + loaded = self.loads(DATA9) + self.assertIs(type(loaded), UnicodeEncodeError) + self.assertEqual(loaded.object, "foo") + self.assertEqual(loaded.encoding, "ascii") + self.assertEqual(loaded.start, 0) + self.assertEqual(loaded.end, 1) + self.assertEqual(loaded.reason, "bad") + + def test_load_python2_str_as_bytes(self): + # From Python 2: pickle.dumps('a\x00\xa0', protocol=0) + self.assertEqual(self.loads(b"S'a\\x00\\xa0'\n.", + encoding="bytes"), b'a\x00\xa0') + # From Python 2: pickle.dumps('a\x00\xa0', protocol=1) + self.assertEqual(self.loads(b'U\x03a\x00\xa0.', + encoding="bytes"), b'a\x00\xa0') + # From Python 2: pickle.dumps('a\x00\xa0', protocol=2) + self.assertEqual(self.loads(b'\x80\x02U\x03a\x00\xa0.', + encoding="bytes"), b'a\x00\xa0') + + def test_load_python2_unicode_as_str(self): + # From Python 2: pickle.dumps(u'?', protocol=0) + self.assertEqual(self.loads(b'V\\u03c0\n.', + encoding='bytes'), '?') + # From Python 2: pickle.dumps(u'?', protocol=1) + self.assertEqual(self.loads(b'X\x02\x00\x00\x00\xcf\x80.', + encoding="bytes"), '?') + # From Python 2: pickle.dumps(u'?', protocol=2) + self.assertEqual(self.loads(b'\x80\x02X\x02\x00\x00\x00\xcf\x80.', + encoding="bytes"), '?') + + def test_load_long_python2_str_as_bytes(self): + # From Python 2: pickle.dumps('x' * 300, protocol=1) + self.assertEqual(self.loads(pickle.BINSTRING + + struct.pack("', '<\\\u1234>', '<\n>', '<\\>', '<\\\U00012345>', @@ -754,7 +930,6 @@ self.assert_is_copy(s, self.loads(p)) def test_ints(self): - import sys for proto in protocols: n = sys.maxsize while n: @@ -764,16 +939,6 @@ self.assert_is_copy(expected, n2) n = n >> 1 - def test_maxint64(self): - maxint64 = (1 << 63) - 1 - data = b'I' + str(maxint64).encode("ascii") + b'\n.' - got = self.loads(data) - self.assert_is_copy(maxint64, got) - - # Try too with a bogus literal. - data = b'I' + str(maxint64).encode("ascii") + b'JUNK\n.' - self.assertRaises(ValueError, self.loads, data) - def test_long(self): for proto in protocols: # 256 bytes is where LONG4 begins. @@ -826,11 +991,6 @@ loaded = self.loads(dumped) self.assert_is_copy(inst, loaded) - def test_pop_empty_stack(self): - # Test issue7455 - s = b'0' - self.assertRaises((pickle.UnpicklingError, IndexError), self.loads, s) - def test_metaclass(self): a = use_metaclass() for proto in protocols: @@ -1335,33 +1495,6 @@ for x_key, y_key in zip(x_keys, y_keys): self.assertIs(x_key, y_key) - def test_unpickle_from_2x(self): - # Unpickle non-trivial data from Python 2.x. - loaded = self.loads(DATA3) - self.assertEqual(loaded, set([1, 2])) - loaded = self.loads(DATA4) - self.assertEqual(type(loaded), type(range(0))) - self.assertEqual(list(loaded), list(range(5))) - loaded = self.loads(DATA5) - self.assertEqual(type(loaded), SimpleCookie) - self.assertEqual(list(loaded.keys()), ["key"]) - self.assertEqual(loaded["key"].value, "value") - - for (exc, data) in DATA7.items(): - loaded = self.loads(data) - self.assertIs(type(loaded), exc) - - loaded = self.loads(DATA8) - self.assertIs(type(loaded), Exception) - - loaded = self.loads(DATA9) - self.assertIs(type(loaded), UnicodeEncodeError) - self.assertEqual(loaded.object, "foo") - self.assertEqual(loaded.encoding, "ascii") - self.assertEqual(loaded.start, 0) - self.assertEqual(loaded.end, 1) - self.assertEqual(loaded.reason, "bad") - def test_pickle_to_2x(self): # Pickle non-trivial data with protocol 2, expecting that it yields # the same result as Python 2.x did. @@ -1372,35 +1505,6 @@ dumped = self.dumps(set([3]), 2) self.assertEqual(dumped, DATA6) - def test_load_python2_str_as_bytes(self): - # From Python 2: pickle.dumps('a\x00\xa0', protocol=0) - self.assertEqual(self.loads(b"S'a\\x00\\xa0'\n.", - encoding="bytes"), b'a\x00\xa0') - # From Python 2: pickle.dumps('a\x00\xa0', protocol=1) - self.assertEqual(self.loads(b'U\x03a\x00\xa0.', - encoding="bytes"), b'a\x00\xa0') - # From Python 2: pickle.dumps('a\x00\xa0', protocol=2) - self.assertEqual(self.loads(b'\x80\x02U\x03a\x00\xa0.', - encoding="bytes"), b'a\x00\xa0') - - def test_load_python2_unicode_as_str(self): - # From Python 2: pickle.dumps(u'?', protocol=0) - self.assertEqual(self.loads(b'V\\u03c0\n.', - encoding='bytes'), '?') - # From Python 2: pickle.dumps(u'?', protocol=1) - self.assertEqual(self.loads(b'X\x02\x00\x00\x00\xcf\x80.', - encoding="bytes"), '?') - # From Python 2: pickle.dumps(u'?', protocol=2) - self.assertEqual(self.loads(b'\x80\x02X\x02\x00\x00\x00\xcf\x80.', - encoding="bytes"), '?') - - def test_load_long_python2_str_as_bytes(self): - # From Python 2: pickle.dumps('x' * 300, protocol=1) - self.assertEqual(self.loads(pickle.BINSTRING + - struct.pack(" 2**32: - self.skipTest("test is only meaningful on 32-bit builds") - # XXX Pure Python pickle reads lengths as signed and passes - # them directly to read() (hence the EOFError) - with self.assertRaises((pickle.UnpicklingError, EOFError, - ValueError, OverflowError)): - self.loads(dumped) - - def test_negative_32b_binbytes(self): - # On 32-bit builds, a BINBYTES of 2**31 or more is refused - self.check_negative_32b_binXXX(b'\x80\x03B\xff\xff\xff\xffxyzq\x00.') - - def test_negative_32b_binunicode(self): - # On 32-bit builds, a BINUNICODE of 2**31 or more is refused - self.check_negative_32b_binXXX(b'\x80\x03X\xff\xff\xff\xffxyzq\x00.') - - def test_negative_put(self): - # Issue #12847 - dumped = b'Va\np-1\n.' - self.assertRaises(ValueError, self.loads, dumped) - - def test_negative_32b_binput(self): - # Issue #12847 - if sys.maxsize > 2**32: - self.skipTest("test is only meaningful on 32-bit builds") - dumped = b'\x80\x03X\x01\x00\x00\x00ar\xff\xff\xff\xff.' - self.assertRaises(ValueError, self.loads, dumped) - - def test_badly_escaped_string(self): - self.assertRaises(ValueError, self.loads, b"S'\\'\n.") - - def test_badly_quoted_string(self): - # Issue #17710 - badpickles = [b"S'\n.", - b'S"\n.', - b'S\' \n.', - b'S" \n.', - b'S\'"\n.', - b'S"\'\n.', - b"S' ' \n.", - b'S" " \n.', - b"S ''\n.", - b'S ""\n.', - b'S \n.', - b'S\n.', - b'S.'] - for p in badpickles: - self.assertRaises(pickle.UnpicklingError, self.loads, p) - - def test_correctly_quoted_string(self): - goodpickles = [(b"S''\n.", ''), - (b'S""\n.', ''), - (b'S"\\n"\n.', '\n'), - (b"S'\\n'\n.", '\n')] - for p, expected in goodpickles: - self.assertEqual(self.loads(p), expected) - def _check_pickling_with_opcode(self, obj, opcode, proto): pickled = self.dumps(obj, proto) self.assertTrue(opcode_in_pickle(opcode, pickled)) @@ -1599,14 +1640,6 @@ count_opcode(pickle.FRAME, pickled)) self.assertEqual(obj, self.loads(some_frames_pickle)) - def test_frame_readline(self): - pickled = b'\x80\x04\x95\x05\x00\x00\x00\x00\x00\x00\x00I42\n.' - # 0: \x80 PROTO 4 - # 2: \x95 FRAME 5 - # 11: I INT 42 - # 15: . STOP - self.assertEqual(self.loads(pickled), 42) - def test_nested_names(self): global Nested class Nested: @@ -1734,33 +1767,6 @@ self.assertIn(('c%s\n%s' % (mod, name)).encode(), pickled) self.assertIs(type(self.loads(pickled)), type(val)) - def test_compat_unpickle(self): - # xrange(1, 7) - pickled = b'\x80\x02c__builtin__\nxrange\nK\x01K\x07K\x01\x87R.' - unpickled = self.loads(pickled) - self.assertIs(type(unpickled), range) - self.assertEqual(unpickled, range(1, 7)) - self.assertEqual(list(unpickled), [1, 2, 3, 4, 5, 6]) - # reduce - pickled = b'\x80\x02c__builtin__\nreduce\n.' - self.assertIs(self.loads(pickled), functools.reduce) - # whichdb.whichdb - pickled = b'\x80\x02cwhichdb\nwhichdb\n.' - self.assertIs(self.loads(pickled), dbm.whichdb) - # Exception(), StandardError() - for name in (b'Exception', b'StandardError'): - pickled = (b'\x80\x02cexceptions\n' + name + b'\nU\x03ugh\x85R.') - unpickled = self.loads(pickled) - self.assertIs(type(unpickled), Exception) - self.assertEqual(str(unpickled), 'ugh') - # UserDict.UserDict({1: 2}), UserDict.IterableUserDict({1: 2}) - for name in (b'UserDict', b'IterableUserDict'): - pickled = (b'\x80\x02(cUserDict\n' + name + - b'\no}U\x04data}K\x01K\x02ssb.') - unpickled = self.loads(pickled) - self.assertIs(type(unpickled), collections.UserDict) - self.assertEqual(unpickled, collections.UserDict({1: 2})) - def test_local_lookup_error(self): # Test that whichmodule() errors out cleanly when looking up # an assumed globally-reachable object fails. diff --git a/Lib/test/test_pickle.py b/Lib/test/test_pickle.py --- a/Lib/test/test_pickle.py +++ b/Lib/test/test_pickle.py @@ -10,6 +10,7 @@ import unittest from test import support +from test.pickletester import AbstractUnpickleTests from test.pickletester import AbstractPickleTests from test.pickletester import AbstractPickleModuleTests from test.pickletester import AbstractPersistentPicklerTests @@ -28,6 +29,16 @@ pass +class PyUnpicklerTests(AbstractUnpickleTests): + + unpickler = pickle._Unpickler + + def loads(self, buf, **kwds): + f = io.BytesIO(buf) + u = self.unpickler(f, **kwds) + return u.load() + + class PyPicklerTests(AbstractPickleTests): pickler = pickle._Pickler @@ -46,7 +57,8 @@ return u.load() -class InMemoryPickleTests(AbstractPickleTests, BigmemPickleTests): +class InMemoryPickleTests(AbstractPickleTests, AbstractUnpickleTests, + BigmemPickleTests): pickler = pickle._Pickler unpickler = pickle._Unpickler @@ -105,6 +117,9 @@ if has_c_implementation: + class CUnpicklerTests(PyUnpicklerTests): + unpickler = _pickle.Unpickler + class CPicklerTests(PyPicklerTests): pickler = _pickle.Pickler unpickler = _pickle.Unpickler @@ -375,11 +390,11 @@ def test_main(): - tests = [PickleTests, PyPicklerTests, PyPersPicklerTests, + tests = [PickleTests, PyUnpicklerTests, PyPicklerTests, PyPersPicklerTests, PyDispatchTableTests, PyChainDispatchTableTests, CompatPickleTests] if has_c_implementation: - tests.extend([CPicklerTests, CPersPicklerTests, + tests.extend([CUnpicklerTests, CPicklerTests, CPersPicklerTests, CDumpPickle_LoadPickle, DumpPickle_CLoadPickle, PyPicklerUnpicklerObjectTests, CPicklerUnpicklerObjectTests, -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 29 16:31:19 2015 From: python-checkins at python.org (eric.smith) Date: Tue, 29 Sep 2015 14:31:19 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Issue_=2325034=3A_Merge_from_3=2E4=2E?= Message-ID: <20150929143105.115474.1362@psf.io> https://hg.python.org/cpython/rev/65d7b4fd0332 changeset: 98394:65d7b4fd0332 branch: 3.5 parent: 98390:a4bf231d81d3 parent: 98393:9eae18e8af66 user: Eric V. Smith date: Tue Sep 29 10:30:04 2015 -0400 summary: Issue #25034: Merge from 3.4. files: Lib/string.py | 11 ++++++----- Lib/test/test_string.py | 2 ++ Misc/ACKS | 1 + Misc/NEWS | 3 +++ 4 files changed, 12 insertions(+), 5 deletions(-) diff --git a/Lib/string.py b/Lib/string.py --- a/Lib/string.py +++ b/Lib/string.py @@ -188,7 +188,7 @@ def vformat(self, format_string, args, kwargs): used_args = set() - result = self._vformat(format_string, args, kwargs, used_args, 2) + result, _ = self._vformat(format_string, args, kwargs, used_args, 2) self.check_unused_args(used_args, args, kwargs) return result @@ -235,14 +235,15 @@ obj = self.convert_field(obj, conversion) # expand the format spec, if needed - format_spec = self._vformat(format_spec, args, kwargs, - used_args, recursion_depth-1, - auto_arg_index=auto_arg_index) + format_spec, auto_arg_index = self._vformat( + format_spec, args, kwargs, + used_args, recursion_depth-1, + auto_arg_index=auto_arg_index) # format the object and append to the result result.append(self.format_field(obj, format_spec)) - return ''.join(result) + return ''.join(result), auto_arg_index def get_value(self, key, args, kwargs): diff --git a/Lib/test/test_string.py b/Lib/test/test_string.py --- a/Lib/test/test_string.py +++ b/Lib/test/test_string.py @@ -58,6 +58,8 @@ 'foo{1}{num}{1}'.format(None, 'bar', num=6)) self.assertEqual(fmt.format('{:^{}}', 'bar', 6), '{:^{}}'.format('bar', 6)) + self.assertEqual(fmt.format('{:^{}} {}', 'bar', 6, 'X'), + '{:^{}} {}'.format('bar', 6, 'X')) self.assertEqual(fmt.format('{:^{pad}}{}', 'foo', 'bar', pad=6), '{:^{pad}}{}'.format('foo', 'bar', pad=6)) diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -1012,6 +1012,7 @@ Trent Nelson Chad Netzer Max Neunh?ffer +Anthon van der Neut George Neville-Neil Hieu Nguyen Johannes Nicolai diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -21,6 +21,9 @@ Library ------- +- Issue #25034: Fix string.Formatter problem with auto-numbering and + nested format_specs. Patch by Anthon van der Neut. + - Issue #25233: Rewrite the guts of asyncio.Queue to be more understandable and correct. - Issue #25203: Failed readline.set_completer_delims() no longer left the -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 29 16:31:19 2015 From: python-checkins at python.org (eric.smith) Date: Tue, 29 Sep 2015 14:31:19 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2325034=3A_Merge_from_3=2E5=2E?= Message-ID: <20150929143105.81647.53031@psf.io> https://hg.python.org/cpython/rev/aef6365294c8 changeset: 98395:aef6365294c8 parent: 98391:377f200ad521 parent: 98394:65d7b4fd0332 user: Eric V. Smith date: Tue Sep 29 10:30:47 2015 -0400 summary: Issue #25034: Merge from 3.5. files: Lib/string.py | 11 ++++++----- Lib/test/test_string.py | 2 ++ Misc/ACKS | 1 + Misc/NEWS | 3 +++ 4 files changed, 12 insertions(+), 5 deletions(-) diff --git a/Lib/string.py b/Lib/string.py --- a/Lib/string.py +++ b/Lib/string.py @@ -183,7 +183,7 @@ def vformat(self, format_string, args, kwargs): used_args = set() - result = self._vformat(format_string, args, kwargs, used_args, 2) + result, _ = self._vformat(format_string, args, kwargs, used_args, 2) self.check_unused_args(used_args, args, kwargs) return result @@ -230,14 +230,15 @@ obj = self.convert_field(obj, conversion) # expand the format spec, if needed - format_spec = self._vformat(format_spec, args, kwargs, - used_args, recursion_depth-1, - auto_arg_index=auto_arg_index) + format_spec, auto_arg_index = self._vformat( + format_spec, args, kwargs, + used_args, recursion_depth-1, + auto_arg_index=auto_arg_index) # format the object and append to the result result.append(self.format_field(obj, format_spec)) - return ''.join(result) + return ''.join(result), auto_arg_index def get_value(self, key, args, kwargs): diff --git a/Lib/test/test_string.py b/Lib/test/test_string.py --- a/Lib/test/test_string.py +++ b/Lib/test/test_string.py @@ -58,6 +58,8 @@ 'foo{1}{num}{1}'.format(None, 'bar', num=6)) self.assertEqual(fmt.format('{:^{}}', 'bar', 6), '{:^{}}'.format('bar', 6)) + self.assertEqual(fmt.format('{:^{}} {}', 'bar', 6, 'X'), + '{:^{}} {}'.format('bar', 6, 'X')) self.assertEqual(fmt.format('{:^{pad}}{}', 'foo', 'bar', pad=6), '{:^{pad}}{}'.format('foo', 'bar', pad=6)) diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -1013,6 +1013,7 @@ Trent Nelson Chad Netzer Max Neunh?ffer +Anthon van der Neut George Neville-Neil Hieu Nguyen Johannes Nicolai diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -190,6 +190,9 @@ Library ------- +- Issue #25034: Fix string.Formatter problem with auto-numbering and + nested format_specs. Patch by Anthon van der Neut. + - Issue #25233: Rewrite the guts of asyncio.Queue to be more understandable and correct. - Issue #23600: Default implementation of tzinfo.fromutc() was returning -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 29 16:31:19 2015 From: python-checkins at python.org (eric.smith) Date: Tue, 29 Sep 2015 14:31:19 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_Fixed_issue_?= =?utf-8?q?=2325034=3A_Fix_string=2EFormatter_problem_with_auto-numbering?= Message-ID: <20150929143105.81633.79442@psf.io> https://hg.python.org/cpython/rev/9eae18e8af66 changeset: 98393:9eae18e8af66 branch: 3.4 parent: 98389:3c2bcdff10a2 user: Eric V. Smith date: Tue Sep 29 10:27:38 2015 -0400 summary: Fixed issue #25034: Fix string.Formatter problem with auto-numbering and nested format_specs. Patch by Anthon van der Neut. files: Lib/string.py | 11 ++++++----- Lib/test/test_string.py | 2 ++ Misc/ACKS | 1 + Misc/NEWS | 3 +++ 4 files changed, 12 insertions(+), 5 deletions(-) diff --git a/Lib/string.py b/Lib/string.py --- a/Lib/string.py +++ b/Lib/string.py @@ -185,7 +185,7 @@ def vformat(self, format_string, args, kwargs): used_args = set() - result = self._vformat(format_string, args, kwargs, used_args, 2) + result, _ = self._vformat(format_string, args, kwargs, used_args, 2) self.check_unused_args(used_args, args, kwargs) return result @@ -232,14 +232,15 @@ obj = self.convert_field(obj, conversion) # expand the format spec, if needed - format_spec = self._vformat(format_spec, args, kwargs, - used_args, recursion_depth-1, - auto_arg_index=auto_arg_index) + format_spec, auto_arg_index = self._vformat( + format_spec, args, kwargs, + used_args, recursion_depth-1, + auto_arg_index=auto_arg_index) # format the object and append to the result result.append(self.format_field(obj, format_spec)) - return ''.join(result) + return ''.join(result), auto_arg_index def get_value(self, key, args, kwargs): diff --git a/Lib/test/test_string.py b/Lib/test/test_string.py --- a/Lib/test/test_string.py +++ b/Lib/test/test_string.py @@ -54,6 +54,8 @@ 'foo{1}{num}{1}'.format(None, 'bar', num=6)) self.assertEqual(fmt.format('{:^{}}', 'bar', 6), '{:^{}}'.format('bar', 6)) + self.assertEqual(fmt.format('{:^{}} {}', 'bar', 6, 'X'), + '{:^{}} {}'.format('bar', 6, 'X')) self.assertEqual(fmt.format('{:^{pad}}{}', 'foo', 'bar', pad=6), '{:^{pad}}{}'.format('foo', 'bar', pad=6)) diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -984,6 +984,7 @@ Trent Nelson Chad Netzer Max Neunh?ffer +Anthon van der Neut George Neville-Neil Hieu Nguyen Johannes Nicolai diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -78,6 +78,9 @@ Library ------- +- Issue #25034: Fix string.Formatter problem with auto-numbering and + nested format_specs. Patch by Anthon van der Neut. + - Issue #25233: Rewrite the guts of asyncio.Queue to be more understandable and correct. - Issue #23600: Default implementation of tzinfo.fromutc() was returning -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 29 18:24:44 2015 From: python-checkins at python.org (andrew.svetlov) Date: Tue, 29 Sep 2015 16:24:44 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge_3=2E5_-=3E_default?= Message-ID: <20150929153833.81633.19446@psf.io> https://hg.python.org/cpython/rev/d325c372161a changeset: 98398:d325c372161a parent: 98395:aef6365294c8 parent: 98397:03b9579be90f user: Andrew Svetlov date: Tue Sep 29 18:38:22 2015 +0300 summary: Merge 3.5 -> default files: Lib/asyncio/streams.py | 2 +- Lib/test/test_asyncio/test_streams.py | 42 +++++++++++++++ 2 files changed, 43 insertions(+), 1 deletions(-) diff --git a/Lib/asyncio/streams.py b/Lib/asyncio/streams.py --- a/Lib/asyncio/streams.py +++ b/Lib/asyncio/streams.py @@ -324,7 +324,7 @@ def __repr__(self): info = ['StreamReader'] if self._buffer: - info.append('%d bytes' % len(info)) + info.append('%d bytes' % len(self._buffer)) if self._eof: info.append('eof') if self._limit != _DEFAULT_LIMIT: diff --git a/Lib/test/test_asyncio/test_streams.py b/Lib/test/test_asyncio/test_streams.py --- a/Lib/test/test_asyncio/test_streams.py +++ b/Lib/test/test_asyncio/test_streams.py @@ -632,6 +632,48 @@ protocol = asyncio.StreamReaderProtocol(reader) self.assertIs(protocol._loop, self.loop) + def test___repr__(self): + stream = asyncio.StreamReader(loop=self.loop) + self.assertEqual("", repr(stream)) + + def test___repr__nondefault_limit(self): + stream = asyncio.StreamReader(loop=self.loop, limit=123) + self.assertEqual("", repr(stream)) + + def test___repr__eof(self): + stream = asyncio.StreamReader(loop=self.loop) + stream.feed_eof() + self.assertEqual("", repr(stream)) + + def test___repr__data(self): + stream = asyncio.StreamReader(loop=self.loop) + stream.feed_data(b'data') + self.assertEqual("", repr(stream)) + + def test___repr__exception(self): + stream = asyncio.StreamReader(loop=self.loop) + exc = RuntimeError() + stream.set_exception(exc) + self.assertEqual("", repr(stream)) + + def test___repr__waiter(self): + stream = asyncio.StreamReader(loop=self.loop) + stream._waiter = asyncio.Future(loop=self.loop) + self.assertRegex( + repr(stream), + ">") + stream._waiter.set_result(None) + self.loop.run_until_complete(stream._waiter) + stream._waiter = None + self.assertEqual("", repr(stream)) + + def test___repr__transport(self): + stream = asyncio.StreamReader(loop=self.loop) + stream._transport = mock.Mock() + stream._transport.__repr__ = mock.Mock() + stream._transport.__repr__.return_value = "" + self.assertEqual(">", repr(stream)) + if __name__ == '__main__': unittest.main() -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 29 18:24:45 2015 From: python-checkins at python.org (andrew.svetlov) Date: Tue, 29 Sep 2015 16:24:45 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_Fix_StreamRead?= =?utf-8?b?ZXIuX19yZXByX18=?= Message-ID: <20150929153833.94115.44417@psf.io> https://hg.python.org/cpython/rev/64905df6d6b6 changeset: 98396:64905df6d6b6 branch: 3.4 parent: 98393:9eae18e8af66 user: Andrew Svetlov date: Tue Sep 29 18:36:00 2015 +0300 summary: Fix StreamReader.__repr__ files: Lib/asyncio/streams.py | 2 +- Lib/test/test_asyncio/test_streams.py | 42 +++++++++++++++ 2 files changed, 43 insertions(+), 1 deletions(-) diff --git a/Lib/asyncio/streams.py b/Lib/asyncio/streams.py --- a/Lib/asyncio/streams.py +++ b/Lib/asyncio/streams.py @@ -324,7 +324,7 @@ def __repr__(self): info = ['StreamReader'] if self._buffer: - info.append('%d bytes' % len(info)) + info.append('%d bytes' % len(self._buffer)) if self._eof: info.append('eof') if self._limit != _DEFAULT_LIMIT: diff --git a/Lib/test/test_asyncio/test_streams.py b/Lib/test/test_asyncio/test_streams.py --- a/Lib/test/test_asyncio/test_streams.py +++ b/Lib/test/test_asyncio/test_streams.py @@ -632,6 +632,48 @@ protocol = asyncio.StreamReaderProtocol(reader) self.assertIs(protocol._loop, self.loop) + def test___repr__(self): + stream = asyncio.StreamReader(loop=self.loop) + self.assertEqual("", repr(stream)) + + def test___repr__nondefault_limit(self): + stream = asyncio.StreamReader(loop=self.loop, limit=123) + self.assertEqual("", repr(stream)) + + def test___repr__eof(self): + stream = asyncio.StreamReader(loop=self.loop) + stream.feed_eof() + self.assertEqual("", repr(stream)) + + def test___repr__data(self): + stream = asyncio.StreamReader(loop=self.loop) + stream.feed_data(b'data') + self.assertEqual("", repr(stream)) + + def test___repr__exception(self): + stream = asyncio.StreamReader(loop=self.loop) + exc = RuntimeError() + stream.set_exception(exc) + self.assertEqual("", repr(stream)) + + def test___repr__waiter(self): + stream = asyncio.StreamReader(loop=self.loop) + stream._waiter = asyncio.Future(loop=self.loop) + self.assertRegex( + repr(stream), + ">") + stream._waiter.set_result(None) + self.loop.run_until_complete(stream._waiter) + stream._waiter = None + self.assertEqual("", repr(stream)) + + def test___repr__transport(self): + stream = asyncio.StreamReader(loop=self.loop) + stream._transport = mock.Mock() + stream._transport.__repr__ = mock.Mock() + stream._transport.__repr__.return_value = "" + self.assertEqual(">", repr(stream)) + if __name__ == '__main__': unittest.main() -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 29 18:24:45 2015 From: python-checkins at python.org (andrew.svetlov) Date: Tue, 29 Sep 2015 16:24:45 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Merge_3=2E4_-=3E_3=2E5?= Message-ID: <20150929153833.81631.49382@psf.io> https://hg.python.org/cpython/rev/03b9579be90f changeset: 98397:03b9579be90f branch: 3.5 parent: 98394:65d7b4fd0332 parent: 98396:64905df6d6b6 user: Andrew Svetlov date: Tue Sep 29 18:36:44 2015 +0300 summary: Merge 3.4 -> 3.5 files: Lib/asyncio/streams.py | 2 +- Lib/test/test_asyncio/test_streams.py | 42 +++++++++++++++ 2 files changed, 43 insertions(+), 1 deletions(-) diff --git a/Lib/asyncio/streams.py b/Lib/asyncio/streams.py --- a/Lib/asyncio/streams.py +++ b/Lib/asyncio/streams.py @@ -324,7 +324,7 @@ def __repr__(self): info = ['StreamReader'] if self._buffer: - info.append('%d bytes' % len(info)) + info.append('%d bytes' % len(self._buffer)) if self._eof: info.append('eof') if self._limit != _DEFAULT_LIMIT: diff --git a/Lib/test/test_asyncio/test_streams.py b/Lib/test/test_asyncio/test_streams.py --- a/Lib/test/test_asyncio/test_streams.py +++ b/Lib/test/test_asyncio/test_streams.py @@ -632,6 +632,48 @@ protocol = asyncio.StreamReaderProtocol(reader) self.assertIs(protocol._loop, self.loop) + def test___repr__(self): + stream = asyncio.StreamReader(loop=self.loop) + self.assertEqual("", repr(stream)) + + def test___repr__nondefault_limit(self): + stream = asyncio.StreamReader(loop=self.loop, limit=123) + self.assertEqual("", repr(stream)) + + def test___repr__eof(self): + stream = asyncio.StreamReader(loop=self.loop) + stream.feed_eof() + self.assertEqual("", repr(stream)) + + def test___repr__data(self): + stream = asyncio.StreamReader(loop=self.loop) + stream.feed_data(b'data') + self.assertEqual("", repr(stream)) + + def test___repr__exception(self): + stream = asyncio.StreamReader(loop=self.loop) + exc = RuntimeError() + stream.set_exception(exc) + self.assertEqual("", repr(stream)) + + def test___repr__waiter(self): + stream = asyncio.StreamReader(loop=self.loop) + stream._waiter = asyncio.Future(loop=self.loop) + self.assertRegex( + repr(stream), + ">") + stream._waiter.set_result(None) + self.loop.run_until_complete(stream._waiter) + stream._waiter = None + self.assertEqual("", repr(stream)) + + def test___repr__transport(self): + stream = asyncio.StreamReader(loop=self.loop) + stream._transport = mock.Mock() + stream._transport.__repr__ = mock.Mock() + stream._transport.__repr__.return_value = "" + self.assertEqual(">", repr(stream)) + if __name__ == '__main__': unittest.main() -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 29 21:02:38 2015 From: python-checkins at python.org (guido.van.rossum) Date: Tue, 29 Sep 2015 19:02:38 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_Also_rewrote_t?= =?utf-8?q?he_guts_of_asyncio=2ESemaphore_=28patch_by_manipopopo=29=2E?= Message-ID: <20150929190234.3654.63064@psf.io> https://hg.python.org/cpython/rev/1ab732cb4643 changeset: 98399:1ab732cb4643 branch: 3.4 parent: 98396:64905df6d6b6 user: Guido van Rossum date: Tue Sep 29 11:54:45 2015 -0700 summary: Also rewrote the guts of asyncio.Semaphore (patch by manipopopo). files: Lib/asyncio/locks.py | 37 ++++++----- Lib/test/test_asyncio/test_locks.py | 52 ++++++++++++++-- Misc/NEWS | 3 +- 3 files changed, 66 insertions(+), 26 deletions(-) diff --git a/Lib/asyncio/locks.py b/Lib/asyncio/locks.py --- a/Lib/asyncio/locks.py +++ b/Lib/asyncio/locks.py @@ -411,6 +411,13 @@ extra = '{},waiters:{}'.format(extra, len(self._waiters)) return '<{} [{}]>'.format(res[1:-1], extra) + def _wake_up_next(self): + while self._waiters: + waiter = self._waiters.popleft() + if not waiter.done(): + waiter.set_result(None) + return + def locked(self): """Returns True if semaphore can not be acquired immediately.""" return self._value == 0 @@ -425,18 +432,19 @@ called release() to make it larger than 0, and then return True. """ - if not self._waiters and self._value > 0: - self._value -= 1 - return True - - fut = futures.Future(loop=self._loop) - self._waiters.append(fut) - try: - yield from fut - self._value -= 1 - return True - finally: - self._waiters.remove(fut) + while self._value <= 0: + fut = futures.Future(loop=self._loop) + self._waiters.append(fut) + try: + yield from fut + except: + # See the similar code in Queue.get. + fut.cancel() + if self._value > 0 and not fut.cancelled(): + self._wake_up_next() + raise + self._value -= 1 + return True def release(self): """Release a semaphore, incrementing the internal counter by one. @@ -444,10 +452,7 @@ become larger than zero again, wake up that coroutine. """ self._value += 1 - for waiter in self._waiters: - if not waiter.done(): - waiter.set_result(True) - break + self._wake_up_next() class BoundedSemaphore(Semaphore): diff --git a/Lib/test/test_asyncio/test_locks.py b/Lib/test/test_asyncio/test_locks.py --- a/Lib/test/test_asyncio/test_locks.py +++ b/Lib/test/test_asyncio/test_locks.py @@ -7,7 +7,6 @@ import asyncio from asyncio import test_utils - STR_RGX_REPR = ( r'^<(?P.*?) object at (?P
      .*?)' r'\[(?P' @@ -783,22 +782,20 @@ test_utils.run_briefly(self.loop) self.assertEqual(0, sem._value) - self.assertEqual([1, 2, 3], result) + self.assertEqual(3, len(result)) self.assertTrue(sem.locked()) self.assertEqual(1, len(sem._waiters)) self.assertEqual(0, sem._value) self.assertTrue(t1.done()) self.assertTrue(t1.result()) - self.assertTrue(t2.done()) - self.assertTrue(t2.result()) - self.assertTrue(t3.done()) - self.assertTrue(t3.result()) - self.assertFalse(t4.done()) + race_tasks = [t2, t3, t4] + done_tasks = [t for t in race_tasks if t.done() and t.result()] + self.assertTrue(2, len(done_tasks)) # cleanup locked semaphore sem.release() - self.loop.run_until_complete(t4) + self.loop.run_until_complete(asyncio.gather(*race_tasks)) def test_acquire_cancel(self): sem = asyncio.Semaphore(loop=self.loop) @@ -809,7 +806,44 @@ self.assertRaises( asyncio.CancelledError, self.loop.run_until_complete, acquire) - self.assertFalse(sem._waiters) + self.assertTrue((not sem._waiters) or + all(waiter.done() for waiter in sem._waiters)) + + def test_acquire_cancel_before_awoken(self): + sem = asyncio.Semaphore(value=0, loop=self.loop) + + t1 = asyncio.Task(sem.acquire(), loop=self.loop) + t2 = asyncio.Task(sem.acquire(), loop=self.loop) + t3 = asyncio.Task(sem.acquire(), loop=self.loop) + t4 = asyncio.Task(sem.acquire(), loop=self.loop) + + test_utils.run_briefly(self.loop) + + sem.release() + t1.cancel() + t2.cancel() + + test_utils.run_briefly(self.loop) + num_done = sum(t.done() for t in [t3, t4]) + self.assertEqual(num_done, 1) + + t3.cancel() + t4.cancel() + test_utils.run_briefly(self.loop) + + def test_acquire_hang(self): + sem = asyncio.Semaphore(value=0, loop=self.loop) + + t1 = asyncio.Task(sem.acquire(), loop=self.loop) + t2 = asyncio.Task(sem.acquire(), loop=self.loop) + + test_utils.run_briefly(self.loop) + + sem.release() + t1.cancel() + + test_utils.run_briefly(self.loop) + self.assertTrue(sem.locked()) def test_release_not_acquired(self): sem = asyncio.BoundedSemaphore(loop=self.loop) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -81,7 +81,8 @@ - Issue #25034: Fix string.Formatter problem with auto-numbering and nested format_specs. Patch by Anthon van der Neut. -- Issue #25233: Rewrite the guts of asyncio.Queue to be more understandable and correct. +- Issue #25233: Rewrite the guts of asyncio.Queue and + asyncio.Semaphore to be more understandable and correct. - Issue #23600: Default implementation of tzinfo.fromutc() was returning wrong results in some cases. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 29 21:02:46 2015 From: python-checkins at python.org (guido.van.rossum) Date: Tue, 29 Sep 2015 19:02:46 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Also_rewrote_the_guts_of_asyncio=2ESemaphore_=28patch_by?= =?utf-8?q?_manipopopo=29=2E_=28Merge?= Message-ID: <20150929190235.115080.34603@psf.io> https://hg.python.org/cpython/rev/4000d2dc4406 changeset: 98401:4000d2dc4406 parent: 98398:d325c372161a parent: 98400:a36914230396 user: Guido van Rossum date: Tue Sep 29 12:01:55 2015 -0700 summary: Also rewrote the guts of asyncio.Semaphore (patch by manipopopo). (Merge 3.5->3.6.) files: Lib/asyncio/locks.py | 37 ++++++----- Lib/test/test_asyncio/test_locks.py | 52 ++++++++++++++-- Misc/NEWS | 3 +- 3 files changed, 66 insertions(+), 26 deletions(-) diff --git a/Lib/asyncio/locks.py b/Lib/asyncio/locks.py --- a/Lib/asyncio/locks.py +++ b/Lib/asyncio/locks.py @@ -411,6 +411,13 @@ extra = '{},waiters:{}'.format(extra, len(self._waiters)) return '<{} [{}]>'.format(res[1:-1], extra) + def _wake_up_next(self): + while self._waiters: + waiter = self._waiters.popleft() + if not waiter.done(): + waiter.set_result(None) + return + def locked(self): """Returns True if semaphore can not be acquired immediately.""" return self._value == 0 @@ -425,18 +432,19 @@ called release() to make it larger than 0, and then return True. """ - if not self._waiters and self._value > 0: - self._value -= 1 - return True - - fut = futures.Future(loop=self._loop) - self._waiters.append(fut) - try: - yield from fut - self._value -= 1 - return True - finally: - self._waiters.remove(fut) + while self._value <= 0: + fut = futures.Future(loop=self._loop) + self._waiters.append(fut) + try: + yield from fut + except: + # See the similar code in Queue.get. + fut.cancel() + if self._value > 0 and not fut.cancelled(): + self._wake_up_next() + raise + self._value -= 1 + return True def release(self): """Release a semaphore, incrementing the internal counter by one. @@ -444,10 +452,7 @@ become larger than zero again, wake up that coroutine. """ self._value += 1 - for waiter in self._waiters: - if not waiter.done(): - waiter.set_result(True) - break + self._wake_up_next() class BoundedSemaphore(Semaphore): diff --git a/Lib/test/test_asyncio/test_locks.py b/Lib/test/test_asyncio/test_locks.py --- a/Lib/test/test_asyncio/test_locks.py +++ b/Lib/test/test_asyncio/test_locks.py @@ -7,7 +7,6 @@ import asyncio from asyncio import test_utils - STR_RGX_REPR = ( r'^<(?P.*?) object at (?P
      .*?)' r'\[(?P' @@ -783,22 +782,20 @@ test_utils.run_briefly(self.loop) self.assertEqual(0, sem._value) - self.assertEqual([1, 2, 3], result) + self.assertEqual(3, len(result)) self.assertTrue(sem.locked()) self.assertEqual(1, len(sem._waiters)) self.assertEqual(0, sem._value) self.assertTrue(t1.done()) self.assertTrue(t1.result()) - self.assertTrue(t2.done()) - self.assertTrue(t2.result()) - self.assertTrue(t3.done()) - self.assertTrue(t3.result()) - self.assertFalse(t4.done()) + race_tasks = [t2, t3, t4] + done_tasks = [t for t in race_tasks if t.done() and t.result()] + self.assertTrue(2, len(done_tasks)) # cleanup locked semaphore sem.release() - self.loop.run_until_complete(t4) + self.loop.run_until_complete(asyncio.gather(*race_tasks)) def test_acquire_cancel(self): sem = asyncio.Semaphore(loop=self.loop) @@ -809,7 +806,44 @@ self.assertRaises( asyncio.CancelledError, self.loop.run_until_complete, acquire) - self.assertFalse(sem._waiters) + self.assertTrue((not sem._waiters) or + all(waiter.done() for waiter in sem._waiters)) + + def test_acquire_cancel_before_awoken(self): + sem = asyncio.Semaphore(value=0, loop=self.loop) + + t1 = asyncio.Task(sem.acquire(), loop=self.loop) + t2 = asyncio.Task(sem.acquire(), loop=self.loop) + t3 = asyncio.Task(sem.acquire(), loop=self.loop) + t4 = asyncio.Task(sem.acquire(), loop=self.loop) + + test_utils.run_briefly(self.loop) + + sem.release() + t1.cancel() + t2.cancel() + + test_utils.run_briefly(self.loop) + num_done = sum(t.done() for t in [t3, t4]) + self.assertEqual(num_done, 1) + + t3.cancel() + t4.cancel() + test_utils.run_briefly(self.loop) + + def test_acquire_hang(self): + sem = asyncio.Semaphore(value=0, loop=self.loop) + + t1 = asyncio.Task(sem.acquire(), loop=self.loop) + t2 = asyncio.Task(sem.acquire(), loop=self.loop) + + test_utils.run_briefly(self.loop) + + sem.release() + t1.cancel() + + test_utils.run_briefly(self.loop) + self.assertTrue(sem.locked()) def test_release_not_acquired(self): sem = asyncio.BoundedSemaphore(loop=self.loop) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -193,7 +193,8 @@ - Issue #25034: Fix string.Formatter problem with auto-numbering and nested format_specs. Patch by Anthon van der Neut. -- Issue #25233: Rewrite the guts of asyncio.Queue to be more understandable and correct. +- Issue #25233: Rewrite the guts of asyncio.Queue and + asyncio.Semaphore to be more understandable and correct. - Issue #23600: Default implementation of tzinfo.fromutc() was returning wrong results in some cases. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 29 21:02:49 2015 From: python-checkins at python.org (guido.van.rossum) Date: Tue, 29 Sep 2015 19:02:49 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Also_rewrote_the_guts_of_asyncio=2ESemaphore_=28patch_by_manip?= =?utf-8?q?opopo=29=2E_=28Merge?= Message-ID: <20150929190234.82642.91851@psf.io> https://hg.python.org/cpython/rev/a36914230396 changeset: 98400:a36914230396 branch: 3.5 parent: 98397:03b9579be90f parent: 98399:1ab732cb4643 user: Guido van Rossum date: Tue Sep 29 12:00:01 2015 -0700 summary: Also rewrote the guts of asyncio.Semaphore (patch by manipopopo). (Merge 3.4->3.5.) files: Lib/asyncio/locks.py | 37 ++++++----- Lib/test/test_asyncio/test_locks.py | 52 ++++++++++++++-- Misc/NEWS | 3 +- 3 files changed, 66 insertions(+), 26 deletions(-) diff --git a/Lib/asyncio/locks.py b/Lib/asyncio/locks.py --- a/Lib/asyncio/locks.py +++ b/Lib/asyncio/locks.py @@ -411,6 +411,13 @@ extra = '{},waiters:{}'.format(extra, len(self._waiters)) return '<{} [{}]>'.format(res[1:-1], extra) + def _wake_up_next(self): + while self._waiters: + waiter = self._waiters.popleft() + if not waiter.done(): + waiter.set_result(None) + return + def locked(self): """Returns True if semaphore can not be acquired immediately.""" return self._value == 0 @@ -425,18 +432,19 @@ called release() to make it larger than 0, and then return True. """ - if not self._waiters and self._value > 0: - self._value -= 1 - return True - - fut = futures.Future(loop=self._loop) - self._waiters.append(fut) - try: - yield from fut - self._value -= 1 - return True - finally: - self._waiters.remove(fut) + while self._value <= 0: + fut = futures.Future(loop=self._loop) + self._waiters.append(fut) + try: + yield from fut + except: + # See the similar code in Queue.get. + fut.cancel() + if self._value > 0 and not fut.cancelled(): + self._wake_up_next() + raise + self._value -= 1 + return True def release(self): """Release a semaphore, incrementing the internal counter by one. @@ -444,10 +452,7 @@ become larger than zero again, wake up that coroutine. """ self._value += 1 - for waiter in self._waiters: - if not waiter.done(): - waiter.set_result(True) - break + self._wake_up_next() class BoundedSemaphore(Semaphore): diff --git a/Lib/test/test_asyncio/test_locks.py b/Lib/test/test_asyncio/test_locks.py --- a/Lib/test/test_asyncio/test_locks.py +++ b/Lib/test/test_asyncio/test_locks.py @@ -7,7 +7,6 @@ import asyncio from asyncio import test_utils - STR_RGX_REPR = ( r'^<(?P.*?) object at (?P
      .*?)' r'\[(?P' @@ -783,22 +782,20 @@ test_utils.run_briefly(self.loop) self.assertEqual(0, sem._value) - self.assertEqual([1, 2, 3], result) + self.assertEqual(3, len(result)) self.assertTrue(sem.locked()) self.assertEqual(1, len(sem._waiters)) self.assertEqual(0, sem._value) self.assertTrue(t1.done()) self.assertTrue(t1.result()) - self.assertTrue(t2.done()) - self.assertTrue(t2.result()) - self.assertTrue(t3.done()) - self.assertTrue(t3.result()) - self.assertFalse(t4.done()) + race_tasks = [t2, t3, t4] + done_tasks = [t for t in race_tasks if t.done() and t.result()] + self.assertTrue(2, len(done_tasks)) # cleanup locked semaphore sem.release() - self.loop.run_until_complete(t4) + self.loop.run_until_complete(asyncio.gather(*race_tasks)) def test_acquire_cancel(self): sem = asyncio.Semaphore(loop=self.loop) @@ -809,7 +806,44 @@ self.assertRaises( asyncio.CancelledError, self.loop.run_until_complete, acquire) - self.assertFalse(sem._waiters) + self.assertTrue((not sem._waiters) or + all(waiter.done() for waiter in sem._waiters)) + + def test_acquire_cancel_before_awoken(self): + sem = asyncio.Semaphore(value=0, loop=self.loop) + + t1 = asyncio.Task(sem.acquire(), loop=self.loop) + t2 = asyncio.Task(sem.acquire(), loop=self.loop) + t3 = asyncio.Task(sem.acquire(), loop=self.loop) + t4 = asyncio.Task(sem.acquire(), loop=self.loop) + + test_utils.run_briefly(self.loop) + + sem.release() + t1.cancel() + t2.cancel() + + test_utils.run_briefly(self.loop) + num_done = sum(t.done() for t in [t3, t4]) + self.assertEqual(num_done, 1) + + t3.cancel() + t4.cancel() + test_utils.run_briefly(self.loop) + + def test_acquire_hang(self): + sem = asyncio.Semaphore(value=0, loop=self.loop) + + t1 = asyncio.Task(sem.acquire(), loop=self.loop) + t2 = asyncio.Task(sem.acquire(), loop=self.loop) + + test_utils.run_briefly(self.loop) + + sem.release() + t1.cancel() + + test_utils.run_briefly(self.loop) + self.assertTrue(sem.locked()) def test_release_not_acquired(self): sem = asyncio.BoundedSemaphore(loop=self.loop) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -24,7 +24,8 @@ - Issue #25034: Fix string.Formatter problem with auto-numbering and nested format_specs. Patch by Anthon van der Neut. -- Issue #25233: Rewrite the guts of asyncio.Queue to be more understandable and correct. +- Issue #25233: Rewrite the guts of asyncio.Queue and + asyncio.Semaphore to be more understandable and correct. - Issue #25203: Failed readline.set_completer_delims() no longer left the module in inconsistent state. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 29 21:14:25 2015 From: python-checkins at python.org (serhiy.storchaka) Date: Tue, 29 Sep 2015 19:14:25 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2325262=2E_Added_support_for_BINBYTES8_opcode_in_?= =?utf-8?q?Python_implementation_of?= Message-ID: <20150929191425.3670.95403@psf.io> https://hg.python.org/cpython/rev/8de1967edfdb changeset: 98404:8de1967edfdb parent: 98401:4000d2dc4406 parent: 98403:da9ad20dd470 user: Serhiy Storchaka date: Tue Sep 29 22:13:01 2015 +0300 summary: Issue #25262. Added support for BINBYTES8 opcode in Python implementation of unpickler. Highest 32 bits of 64-bit size for BINUNICODE8 and BINBYTES8 opcodes no longer silently ignored on 32-bit platforms in C implementation. files: Lib/pickle.py | 8 ++++++++ Lib/test/pickletester.py | 20 ++++++++++++++++++++ Misc/NEWS | 4 ++++ Modules/_pickle.c | 14 ++++++++++++-- 4 files changed, 44 insertions(+), 2 deletions(-) diff --git a/Lib/pickle.py b/Lib/pickle.py --- a/Lib/pickle.py +++ b/Lib/pickle.py @@ -1205,6 +1205,14 @@ self.append(str(self.read(len), 'utf-8', 'surrogatepass')) dispatch[BINUNICODE8[0]] = load_binunicode8 + def load_binbytes8(self): + len, = unpack(' maxsize: + raise UnpicklingError("BINBYTES8 exceeds system's maximum size " + "of %d bytes" % maxsize) + self.append(self.read(len)) + dispatch[BINBYTES8[0]] = load_binbytes8 + def load_short_binstring(self): len = self.read(1)[0] data = self.read(len) diff --git a/Lib/test/pickletester.py b/Lib/test/pickletester.py --- a/Lib/test/pickletester.py +++ b/Lib/test/pickletester.py @@ -857,6 +857,26 @@ self.assert_is_copy([(100,), (100,)], self.loads(b'((Kdtp0\nh\x00l.))')) + def test_binbytes8(self): + dumped = b'\x80\x04\x8e\4\0\0\0\0\0\0\0\xe2\x82\xac\x00.' + self.assertEqual(self.loads(dumped), b'\xe2\x82\xac\x00') + + def test_binunicode8(self): + dumped = b'\x80\x04\x8d\4\0\0\0\0\0\0\0\xe2\x82\xac\x00.' + self.assertEqual(self.loads(dumped), '\u20ac\x00') + + @requires_32b + def test_large_32b_binbytes8(self): + dumped = b'\x80\x04\x8e\4\0\0\0\1\0\0\0\xe2\x82\xac\x00.' + with self.assertRaises((pickle.UnpicklingError, OverflowError)): + self.loads(dumped) + + @requires_32b + def test_large_32b_binunicode8(self): + dumped = b'\x80\x04\x8d\4\0\0\0\1\0\0\0\xe2\x82\xac\x00.' + with self.assertRaises((pickle.UnpicklingError, OverflowError)): + self.loads(dumped) + def test_get(self): pickled = b'((lp100000\ng100000\nt.' unpickled = self.loads(pickled) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -190,6 +190,10 @@ Library ------- +- Issue #25262. Added support for BINBYTES8 opcode in Python implementation of + unpickler. Highest 32 bits of 64-bit size for BINUNICODE8 and BINBYTES8 + opcodes no longer silently ignored on 32-bit platforms in C implementation. + - Issue #25034: Fix string.Formatter problem with auto-numbering and nested format_specs. Patch by Anthon van der Neut. diff --git a/Modules/_pickle.c b/Modules/_pickle.c --- a/Modules/_pickle.c +++ b/Modules/_pickle.c @@ -4606,10 +4606,20 @@ calc_binsize(char *bytes, int nbytes) { unsigned char *s = (unsigned char *)bytes; - Py_ssize_t i; + int i; size_t x = 0; - for (i = 0; i < nbytes && (size_t)i < sizeof(size_t); i++) { + if (nbytes > (int)sizeof(size_t)) { + /* Check for integer overflow. BINBYTES8 and BINUNICODE8 opcodes + * have 64-bit size that can't be represented on 32-bit platform. + */ + for (i = (int)sizeof(size_t); i < nbytes; i++) { + if (s[i]) + return -1; + } + nbytes = (int)sizeof(size_t); + } + for (i = 0; i < nbytes; i++) { x |= (size_t) s[i] << (8 * i); } -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 29 21:14:25 2015 From: python-checkins at python.org (serhiy.storchaka) Date: Tue, 29 Sep 2015 19:14:25 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzI1MjYy?= =?utf-8?q?=2E_Added_support_for_BINBYTES8_opcode_in_Python_implementation?= =?utf-8?q?_of?= Message-ID: <20150929191425.9927.47856@psf.io> https://hg.python.org/cpython/rev/d4f8316d0860 changeset: 98402:d4f8316d0860 branch: 3.4 parent: 98399:1ab732cb4643 user: Serhiy Storchaka date: Tue Sep 29 22:10:07 2015 +0300 summary: Issue #25262. Added support for BINBYTES8 opcode in Python implementation of unpickler. Highest 32 bits of 64-bit size for BINUNICODE8 and BINBYTES8 opcodes no longer silently ignored on 32-bit platforms in C implementation. files: Lib/pickle.py | 8 ++++++++ Lib/test/pickletester.py | 20 ++++++++++++++++++++ Misc/NEWS | 4 ++++ Modules/_pickle.c | 12 +++++++++++- 4 files changed, 43 insertions(+), 1 deletions(-) diff --git a/Lib/pickle.py b/Lib/pickle.py --- a/Lib/pickle.py +++ b/Lib/pickle.py @@ -1204,6 +1204,14 @@ self.append(str(self.read(len), 'utf-8', 'surrogatepass')) dispatch[BINUNICODE8[0]] = load_binunicode8 + def load_binbytes8(self): + len, = unpack(' maxsize: + raise UnpicklingError("BINBYTES8 exceeds system's maximum size " + "of %d bytes" % maxsize) + self.append(self.read(len)) + dispatch[BINBYTES8[0]] = load_binbytes8 + def load_short_binstring(self): len = self.read(1)[0] data = self.read(len) diff --git a/Lib/test/pickletester.py b/Lib/test/pickletester.py --- a/Lib/test/pickletester.py +++ b/Lib/test/pickletester.py @@ -857,6 +857,26 @@ self.assert_is_copy([(100,), (100,)], self.loads(b'((Kdtp0\nh\x00l.))')) + def test_binbytes8(self): + dumped = b'\x80\x04\x8e\4\0\0\0\0\0\0\0\xe2\x82\xac\x00.' + self.assertEqual(self.loads(dumped), b'\xe2\x82\xac\x00') + + def test_binunicode8(self): + dumped = b'\x80\x04\x8d\4\0\0\0\0\0\0\0\xe2\x82\xac\x00.' + self.assertEqual(self.loads(dumped), '\u20ac\x00') + + @requires_32b + def test_large_32b_binbytes8(self): + dumped = b'\x80\x04\x8e\4\0\0\0\1\0\0\0\xe2\x82\xac\x00.' + with self.assertRaises((pickle.UnpicklingError, OverflowError)): + self.loads(dumped) + + @requires_32b + def test_large_32b_binunicode8(self): + dumped = b'\x80\x04\x8d\4\0\0\0\1\0\0\0\xe2\x82\xac\x00.' + with self.assertRaises((pickle.UnpicklingError, OverflowError)): + self.loads(dumped) + def test_get(self): pickled = b'((lp100000\ng100000\nt.' unpickled = self.loads(pickled) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -78,6 +78,10 @@ Library ------- +- Issue #25262. Added support for BINBYTES8 opcode in Python implementation of + unpickler. Highest 32 bits of 64-bit size for BINUNICODE8 and BINBYTES8 + opcodes no longer silently ignored on 32-bit platforms in C implementation. + - Issue #25034: Fix string.Formatter problem with auto-numbering and nested format_specs. Patch by Anthon van der Neut. diff --git a/Modules/_pickle.c b/Modules/_pickle.c --- a/Modules/_pickle.c +++ b/Modules/_pickle.c @@ -4540,7 +4540,17 @@ int i; size_t x = 0; - for (i = 0; i < nbytes && i < sizeof(size_t); i++) { + if (nbytes > (int)sizeof(size_t)) { + /* Check for integer overflow. BINBYTES8 and BINUNICODE8 opcodes + * have 64-bit size that can't be represented on 32-bit platform. + */ + for (i = (int)sizeof(size_t); i < nbytes; i++) { + if (s[i]) + return -1; + } + nbytes = (int)sizeof(size_t); + } + for (i = 0; i < nbytes; i++) { x |= (size_t) s[i] << (8 * i); } -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 29 21:14:25 2015 From: python-checkins at python.org (serhiy.storchaka) Date: Tue, 29 Sep 2015 19:14:25 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Issue_=2325262=2E_Added_support_for_BINBYTES8_opcode_in_Python?= =?utf-8?q?_implementation_of?= Message-ID: <20150929191425.3650.67960@psf.io> https://hg.python.org/cpython/rev/da9ad20dd470 changeset: 98403:da9ad20dd470 branch: 3.5 parent: 98400:a36914230396 parent: 98402:d4f8316d0860 user: Serhiy Storchaka date: Tue Sep 29 22:12:29 2015 +0300 summary: Issue #25262. Added support for BINBYTES8 opcode in Python implementation of unpickler. Highest 32 bits of 64-bit size for BINUNICODE8 and BINBYTES8 opcodes no longer silently ignored on 32-bit platforms in C implementation. files: Lib/pickle.py | 8 ++++++++ Lib/test/pickletester.py | 20 ++++++++++++++++++++ Misc/NEWS | 4 ++++ Modules/_pickle.c | 14 ++++++++++++-- 4 files changed, 44 insertions(+), 2 deletions(-) diff --git a/Lib/pickle.py b/Lib/pickle.py --- a/Lib/pickle.py +++ b/Lib/pickle.py @@ -1205,6 +1205,14 @@ self.append(str(self.read(len), 'utf-8', 'surrogatepass')) dispatch[BINUNICODE8[0]] = load_binunicode8 + def load_binbytes8(self): + len, = unpack(' maxsize: + raise UnpicklingError("BINBYTES8 exceeds system's maximum size " + "of %d bytes" % maxsize) + self.append(self.read(len)) + dispatch[BINBYTES8[0]] = load_binbytes8 + def load_short_binstring(self): len = self.read(1)[0] data = self.read(len) diff --git a/Lib/test/pickletester.py b/Lib/test/pickletester.py --- a/Lib/test/pickletester.py +++ b/Lib/test/pickletester.py @@ -857,6 +857,26 @@ self.assert_is_copy([(100,), (100,)], self.loads(b'((Kdtp0\nh\x00l.))')) + def test_binbytes8(self): + dumped = b'\x80\x04\x8e\4\0\0\0\0\0\0\0\xe2\x82\xac\x00.' + self.assertEqual(self.loads(dumped), b'\xe2\x82\xac\x00') + + def test_binunicode8(self): + dumped = b'\x80\x04\x8d\4\0\0\0\0\0\0\0\xe2\x82\xac\x00.' + self.assertEqual(self.loads(dumped), '\u20ac\x00') + + @requires_32b + def test_large_32b_binbytes8(self): + dumped = b'\x80\x04\x8e\4\0\0\0\1\0\0\0\xe2\x82\xac\x00.' + with self.assertRaises((pickle.UnpicklingError, OverflowError)): + self.loads(dumped) + + @requires_32b + def test_large_32b_binunicode8(self): + dumped = b'\x80\x04\x8d\4\0\0\0\1\0\0\0\xe2\x82\xac\x00.' + with self.assertRaises((pickle.UnpicklingError, OverflowError)): + self.loads(dumped) + def test_get(self): pickled = b'((lp100000\ng100000\nt.' unpickled = self.loads(pickled) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -21,6 +21,10 @@ Library ------- +- Issue #25262. Added support for BINBYTES8 opcode in Python implementation of + unpickler. Highest 32 bits of 64-bit size for BINUNICODE8 and BINBYTES8 + opcodes no longer silently ignored on 32-bit platforms in C implementation. + - Issue #25034: Fix string.Formatter problem with auto-numbering and nested format_specs. Patch by Anthon van der Neut. diff --git a/Modules/_pickle.c b/Modules/_pickle.c --- a/Modules/_pickle.c +++ b/Modules/_pickle.c @@ -4606,10 +4606,20 @@ calc_binsize(char *bytes, int nbytes) { unsigned char *s = (unsigned char *)bytes; - Py_ssize_t i; + int i; size_t x = 0; - for (i = 0; i < nbytes && (size_t)i < sizeof(size_t); i++) { + if (nbytes > (int)sizeof(size_t)) { + /* Check for integer overflow. BINBYTES8 and BINUNICODE8 opcodes + * have 64-bit size that can't be represented on 32-bit platform. + */ + for (i = (int)sizeof(size_t); i < nbytes; i++) { + if (s[i]) + return -1; + } + nbytes = (int)sizeof(size_t); + } + for (i = 0; i < nbytes; i++) { x |= (size_t) s[i] << (8 * i); } -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 29 21:36:30 2015 From: python-checkins at python.org (serhiy.storchaka) Date: Tue, 29 Sep 2015 19:36:30 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI1MTEx?= =?utf-8?q?=3A_Fixed_comparison_of_traceback=2EFrameSummary=2E?= Message-ID: <20150929193630.3642.1255@psf.io> https://hg.python.org/cpython/rev/2ecb7d4d9e0b changeset: 98405:2ecb7d4d9e0b branch: 3.5 parent: 98403:da9ad20dd470 user: Serhiy Storchaka date: Tue Sep 29 22:33:36 2015 +0300 summary: Issue #25111: Fixed comparison of traceback.FrameSummary. files: Lib/test/test_traceback.py | 16 +++++++++++----- Lib/traceback.py | 12 ++++++++---- Misc/NEWS | 2 ++ 3 files changed, 21 insertions(+), 9 deletions(-) diff --git a/Lib/test/test_traceback.py b/Lib/test/test_traceback.py --- a/Lib/test/test_traceback.py +++ b/Lib/test/test_traceback.py @@ -640,7 +640,7 @@ return traceback.extract_stack() result = extract() lineno = extract.__code__.co_firstlineno - self.assertEqual([tuple(x) for x in result[-2:]], [ + self.assertEqual(result[-2:], [ (__file__, lineno+2, 'test_extract_stack', 'result = extract()'), (__file__, lineno+1, 'extract', 'return traceback.extract_stack()'), ]) @@ -652,10 +652,16 @@ linecache.clearcache() linecache.lazycache("f", globals()) f = traceback.FrameSummary("f", 1, "dummy") - self.assertEqual( - ("f", 1, "dummy", '"""Test cases for traceback module"""'), - tuple(f)) - self.assertEqual(None, f.locals) + self.assertEqual(f, + ("f", 1, "dummy", '"""Test cases for traceback module"""')) + self.assertEqual(tuple(f), + ("f", 1, "dummy", '"""Test cases for traceback module"""')) + self.assertEqual(f, traceback.FrameSummary("f", 1, "dummy")) + self.assertEqual(f, tuple(f)) + # Since tuple.__eq__ doesn't support FrameSummary, the equality + # operator fallbacks to FrameSummary.__eq__. + self.assertEqual(tuple(f), f) + self.assertIsNone(f.locals) def test_lazy_lines(self): linecache.clearcache() diff --git a/Lib/traceback.py b/Lib/traceback.py --- a/Lib/traceback.py +++ b/Lib/traceback.py @@ -257,10 +257,14 @@ dict((k, repr(v)) for k, v in locals.items()) if locals else None def __eq__(self, other): - return (self.filename == other.filename and - self.lineno == other.lineno and - self.name == other.name and - self.locals == other.locals) + if isinstance(other, FrameSummary): + return (self.filename == other.filename and + self.lineno == other.lineno and + self.name == other.name and + self.locals == other.locals) + if isinstance(other, tuple): + return (self.filename, self.lineno, self.name, self.line) == other + return NotImplemented def __getitem__(self, pos): return (self.filename, self.lineno, self.name, self.line)[pos] diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -21,6 +21,8 @@ Library ------- +- Issue #25111: Fixed comparison of traceback.FrameSummary. + - Issue #25262. Added support for BINBYTES8 opcode in Python implementation of unpickler. Highest 32 bits of 64-bit size for BINUNICODE8 and BINBYTES8 opcodes no longer silently ignored on 32-bit platforms in C implementation. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 29 21:36:32 2015 From: python-checkins at python.org (serhiy.storchaka) Date: Tue, 29 Sep 2015 19:36:32 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2325111=3A_Fixed_comparison_of_traceback=2EFrameS?= =?utf-8?q?ummary=2E?= Message-ID: <20150929193630.81619.31767@psf.io> https://hg.python.org/cpython/rev/f043182a81d7 changeset: 98406:f043182a81d7 parent: 98404:8de1967edfdb parent: 98405:2ecb7d4d9e0b user: Serhiy Storchaka date: Tue Sep 29 22:34:16 2015 +0300 summary: Issue #25111: Fixed comparison of traceback.FrameSummary. files: Lib/test/test_traceback.py | 16 +++++++++++----- Lib/traceback.py | 12 ++++++++---- Misc/NEWS | 2 ++ 3 files changed, 21 insertions(+), 9 deletions(-) diff --git a/Lib/test/test_traceback.py b/Lib/test/test_traceback.py --- a/Lib/test/test_traceback.py +++ b/Lib/test/test_traceback.py @@ -640,7 +640,7 @@ return traceback.extract_stack() result = extract() lineno = extract.__code__.co_firstlineno - self.assertEqual([tuple(x) for x in result[-2:]], [ + self.assertEqual(result[-2:], [ (__file__, lineno+2, 'test_extract_stack', 'result = extract()'), (__file__, lineno+1, 'extract', 'return traceback.extract_stack()'), ]) @@ -652,10 +652,16 @@ linecache.clearcache() linecache.lazycache("f", globals()) f = traceback.FrameSummary("f", 1, "dummy") - self.assertEqual( - ("f", 1, "dummy", '"""Test cases for traceback module"""'), - tuple(f)) - self.assertEqual(None, f.locals) + self.assertEqual(f, + ("f", 1, "dummy", '"""Test cases for traceback module"""')) + self.assertEqual(tuple(f), + ("f", 1, "dummy", '"""Test cases for traceback module"""')) + self.assertEqual(f, traceback.FrameSummary("f", 1, "dummy")) + self.assertEqual(f, tuple(f)) + # Since tuple.__eq__ doesn't support FrameSummary, the equality + # operator fallbacks to FrameSummary.__eq__. + self.assertEqual(tuple(f), f) + self.assertIsNone(f.locals) def test_lazy_lines(self): linecache.clearcache() diff --git a/Lib/traceback.py b/Lib/traceback.py --- a/Lib/traceback.py +++ b/Lib/traceback.py @@ -257,10 +257,14 @@ dict((k, repr(v)) for k, v in locals.items()) if locals else None def __eq__(self, other): - return (self.filename == other.filename and - self.lineno == other.lineno and - self.name == other.name and - self.locals == other.locals) + if isinstance(other, FrameSummary): + return (self.filename == other.filename and + self.lineno == other.lineno and + self.name == other.name and + self.locals == other.locals) + if isinstance(other, tuple): + return (self.filename, self.lineno, self.name, self.line) == other + return NotImplemented def __getitem__(self, pos): return (self.filename, self.lineno, self.name, self.line)[pos] diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -190,6 +190,8 @@ Library ------- +- Issue #25111: Fixed comparison of traceback.FrameSummary. + - Issue #25262. Added support for BINBYTES8 opcode in Python implementation of unpickler. Highest 32 bits of 64-bit size for BINUNICODE8 and BINBYTES8 opcodes no longer silently ignored on 32-bit platforms in C implementation. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 29 22:54:24 2015 From: python-checkins at python.org (serhiy.storchaka) Date: Tue, 29 Sep 2015 20:54:24 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzIyNjA5?= =?utf-8?q?=3A_Constructor_and_the_update_method_of_collections=2EUserDict?= =?utf-8?q?_now?= Message-ID: <20150929205423.3656.69579@psf.io> https://hg.python.org/cpython/rev/4c5407e1b0ec changeset: 98407:4c5407e1b0ec branch: 2.7 parent: 98392:2ec4aa882447 user: Serhiy Storchaka date: Tue Sep 29 23:33:03 2015 +0300 summary: Issue #22609: Constructor and the update method of collections.UserDict now accept the self keyword argument. files: Lib/UserDict.py | 37 +++++++++++++++++++++++++- Lib/test/test_userdict.py | 35 ++++++++++++++++++++++++- Misc/NEWS | 3 ++ 3 files changed, 72 insertions(+), 3 deletions(-) diff --git a/Lib/UserDict.py b/Lib/UserDict.py --- a/Lib/UserDict.py +++ b/Lib/UserDict.py @@ -1,7 +1,24 @@ """A more or less complete user-defined wrapper around dictionary objects.""" class UserDict: - def __init__(self, dict=None, **kwargs): + def __init__(*args, **kwargs): + if not args: + raise TypeError("descriptor '__init__' of 'UserDict' object " + "needs an argument") + self = args[0] + args = args[1:] + if len(args) > 1: + raise TypeError('expected at most 1 arguments, got %d' % len(args)) + if args: + dict = args[0] + elif 'dict' in kwargs: + dict = kwargs.pop('dict') + import warnings + warnings.warn("Passing 'dict' as keyword argument is " + "deprecated", PendingDeprecationWarning, + stacklevel=2) + else: + dict = None self.data = {} if dict is not None: self.update(dict) @@ -43,7 +60,23 @@ def itervalues(self): return self.data.itervalues() def values(self): return self.data.values() def has_key(self, key): return key in self.data - def update(self, dict=None, **kwargs): + def update(*args, **kwargs): + if not args: + raise TypeError("descriptor 'update' of 'UserDict' object " + "needs an argument") + self = args[0] + args = args[1:] + if len(args) > 1: + raise TypeError('expected at most 1 arguments, got %d' % len(args)) + if args: + dict = args[0] + elif 'dict' in kwargs: + dict = kwargs.pop('dict') + import warnings + warnings.warn("Passing 'dict' as keyword argument is deprecated", + PendingDeprecationWarning, stacklevel=2) + else: + dict = None if dict is None: pass elif isinstance(dict, UserDict): diff --git a/Lib/test/test_userdict.py b/Lib/test/test_userdict.py --- a/Lib/test/test_userdict.py +++ b/Lib/test/test_userdict.py @@ -2,6 +2,7 @@ from test import test_support, mapping_tests import UserDict +import warnings d0 = {} d1 = {"one": 1} @@ -29,7 +30,9 @@ self.assertEqual(UserDict.UserDict(one=1, two=2), d2) # item sequence constructor self.assertEqual(UserDict.UserDict([('one',1), ('two',2)]), d2) - self.assertEqual(UserDict.UserDict(dict=[('one',1), ('two',2)]), d2) + with test_support.check_warnings((".*'dict'.*", + PendingDeprecationWarning)): + self.assertEqual(UserDict.UserDict(dict=[('one',1), ('two',2)]), d2) # both together self.assertEqual(UserDict.UserDict([('one',1), ('two',2)], two=3, three=5), d3) @@ -148,6 +151,36 @@ self.assertEqual(t.popitem(), ("x", 42)) self.assertRaises(KeyError, t.popitem) + def test_init(self): + for kw in 'self', 'other', 'iterable': + self.assertEqual(list(UserDict.UserDict(**{kw: 42}).items()), + [(kw, 42)]) + self.assertEqual(list(UserDict.UserDict({}, dict=42).items()), + [('dict', 42)]) + self.assertEqual(list(UserDict.UserDict({}, dict=None).items()), + [('dict', None)]) + with test_support.check_warnings((".*'dict'.*", + PendingDeprecationWarning)): + self.assertEqual(list(UserDict.UserDict(dict={'a': 42}).items()), + [('a', 42)]) + self.assertRaises(TypeError, UserDict.UserDict, 42) + self.assertRaises(TypeError, UserDict.UserDict, (), ()) + self.assertRaises(TypeError, UserDict.UserDict.__init__) + + def test_update(self): + for kw in 'self', 'other', 'iterable': + d = UserDict.UserDict() + d.update(**{kw: 42}) + self.assertEqual(list(d.items()), [(kw, 42)]) + d = UserDict.UserDict() + with test_support.check_warnings((".*'dict'.*", + PendingDeprecationWarning)): + d.update(dict={'a': 42}) + self.assertEqual(list(d.items()), [('a', 42)]) + self.assertRaises(TypeError, UserDict.UserDict().update, 42) + self.assertRaises(TypeError, UserDict.UserDict().update, {}, {}) + self.assertRaises(TypeError, UserDict.UserDict.update) + def test_missing(self): # Make sure UserDict doesn't have a __missing__ method self.assertEqual(hasattr(UserDict, "__missing__"), False) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -37,6 +37,9 @@ Library ------- +- Issue #22609: Constructor and the update method of collections.UserDict now + accept the self keyword argument. + - Issue #25203: Failed readline.set_completer_delims() no longer left the module in inconsistent state. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 29 22:54:24 2015 From: python-checkins at python.org (serhiy.storchaka) Date: Tue, 29 Sep 2015 20:54:24 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2322609=3A_Constructor_of_collections=2EUserDict_?= =?utf-8?q?now_accepts_the_self_keyword?= Message-ID: <20150929205424.11714.69888@psf.io> https://hg.python.org/cpython/rev/901964295066 changeset: 98410:901964295066 parent: 98406:f043182a81d7 parent: 98409:ab7e3f1f9f88 user: Serhiy Storchaka date: Tue Sep 29 23:38:34 2015 +0300 summary: Issue #22609: Constructor of collections.UserDict now accepts the self keyword argument. files: Lib/collections/__init__.py | 17 ++++++++++++++- Lib/test/test_userdict.py | 27 ++++++++++++++++++++++++- Misc/NEWS | 3 ++ 3 files changed, 45 insertions(+), 2 deletions(-) diff --git a/Lib/collections/__init__.py b/Lib/collections/__init__.py --- a/Lib/collections/__init__.py +++ b/Lib/collections/__init__.py @@ -939,7 +939,22 @@ class UserDict(MutableMapping): # Start by filling-out the abstract methods - def __init__(self, dict=None, **kwargs): + def __init__(*args, **kwargs): + if not args: + raise TypeError("descriptor '__init__' of 'UserDict' object " + "needs an argument") + self, *args = args + if len(args) > 1: + raise TypeError('expected at most 1 arguments, got %d' % len(args)) + if args: + dict = args[0] + elif 'dict' in kwargs: + dict = kwargs.pop('dict') + import warnings + warnings.warn("Passing 'dict' as keyword argument is deprecated", + DeprecationWarning, stacklevel=2) + else: + dict = None self.data = {} if dict is not None: self.update(dict) diff --git a/Lib/test/test_userdict.py b/Lib/test/test_userdict.py --- a/Lib/test/test_userdict.py +++ b/Lib/test/test_userdict.py @@ -29,7 +29,8 @@ self.assertEqual(collections.UserDict(one=1, two=2), d2) # item sequence constructor self.assertEqual(collections.UserDict([('one',1), ('two',2)]), d2) - self.assertEqual(collections.UserDict(dict=[('one',1), ('two',2)]), d2) + with self.assertWarnsRegex(DeprecationWarning, "'dict'"): + self.assertEqual(collections.UserDict(dict=[('one',1), ('two',2)]), d2) # both together self.assertEqual(collections.UserDict([('one',1), ('two',2)], two=3, three=5), d3) @@ -139,6 +140,30 @@ self.assertEqual(t.popitem(), ("x", 42)) self.assertRaises(KeyError, t.popitem) + def test_init(self): + for kw in 'self', 'other', 'iterable': + self.assertEqual(list(collections.UserDict(**{kw: 42}).items()), + [(kw, 42)]) + self.assertEqual(list(collections.UserDict({}, dict=42).items()), + [('dict', 42)]) + self.assertEqual(list(collections.UserDict({}, dict=None).items()), + [('dict', None)]) + with self.assertWarnsRegex(DeprecationWarning, "'dict'"): + self.assertEqual(list(collections.UserDict(dict={'a': 42}).items()), + [('a', 42)]) + self.assertRaises(TypeError, collections.UserDict, 42) + self.assertRaises(TypeError, collections.UserDict, (), ()) + self.assertRaises(TypeError, collections.UserDict.__init__) + + def test_update(self): + for kw in 'self', 'dict', 'other', 'iterable': + d = collections.UserDict() + d.update(**{kw: 42}) + self.assertEqual(list(d.items()), [(kw, 42)]) + self.assertRaises(TypeError, collections.UserDict().update, 42) + self.assertRaises(TypeError, collections.UserDict().update, {}, {}) + self.assertRaises(TypeError, collections.UserDict.update) + def test_missing(self): # Make sure UserDict doesn't have a __missing__ method self.assertEqual(hasattr(collections.UserDict, "__missing__"), False) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -190,6 +190,9 @@ Library ------- +- Issue #22609: Constructor of collections.UserDict now accepts the self keyword + argument. + - Issue #25111: Fixed comparison of traceback.FrameSummary. - Issue #25262. Added support for BINBYTES8 opcode in Python implementation of -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 29 22:54:25 2015 From: python-checkins at python.org (serhiy.storchaka) Date: Tue, 29 Sep 2015 20:54:25 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzIyNjA5?= =?utf-8?q?=3A_Constructor_of_collections=2EUserDict_now_accepts_the_self_?= =?utf-8?q?keyword?= Message-ID: <20150929205423.82656.32001@psf.io> https://hg.python.org/cpython/rev/1869f5625392 changeset: 98408:1869f5625392 branch: 3.4 parent: 98402:d4f8316d0860 user: Serhiy Storchaka date: Tue Sep 29 23:36:06 2015 +0300 summary: Issue #22609: Constructor of collections.UserDict now accepts the self keyword argument. files: Lib/collections/__init__.py | 17 ++++++++++++++- Lib/test/test_userdict.py | 27 ++++++++++++++++++++++++- Misc/NEWS | 3 ++ 3 files changed, 45 insertions(+), 2 deletions(-) diff --git a/Lib/collections/__init__.py b/Lib/collections/__init__.py --- a/Lib/collections/__init__.py +++ b/Lib/collections/__init__.py @@ -883,7 +883,22 @@ class UserDict(MutableMapping): # Start by filling-out the abstract methods - def __init__(self, dict=None, **kwargs): + def __init__(*args, **kwargs): + if not args: + raise TypeError("descriptor '__init__' of 'UserDict' object " + "needs an argument") + self, *args = args + if len(args) > 1: + raise TypeError('expected at most 1 arguments, got %d' % len(args)) + if args: + dict = args[0] + elif 'dict' in kwargs: + dict = kwargs.pop('dict') + import warnings + warnings.warn("Passing 'dict' as keyword argument is deprecated", + PendingDeprecationWarning, stacklevel=2) + else: + dict = None self.data = {} if dict is not None: self.update(dict) diff --git a/Lib/test/test_userdict.py b/Lib/test/test_userdict.py --- a/Lib/test/test_userdict.py +++ b/Lib/test/test_userdict.py @@ -29,7 +29,8 @@ self.assertEqual(collections.UserDict(one=1, two=2), d2) # item sequence constructor self.assertEqual(collections.UserDict([('one',1), ('two',2)]), d2) - self.assertEqual(collections.UserDict(dict=[('one',1), ('two',2)]), d2) + with self.assertWarnsRegex(PendingDeprecationWarning, "'dict'"): + self.assertEqual(collections.UserDict(dict=[('one',1), ('two',2)]), d2) # both together self.assertEqual(collections.UserDict([('one',1), ('two',2)], two=3, three=5), d3) @@ -139,6 +140,30 @@ self.assertEqual(t.popitem(), ("x", 42)) self.assertRaises(KeyError, t.popitem) + def test_init(self): + for kw in 'self', 'other', 'iterable': + self.assertEqual(list(collections.UserDict(**{kw: 42}).items()), + [(kw, 42)]) + self.assertEqual(list(collections.UserDict({}, dict=42).items()), + [('dict', 42)]) + self.assertEqual(list(collections.UserDict({}, dict=None).items()), + [('dict', None)]) + with self.assertWarnsRegex(PendingDeprecationWarning, "'dict'"): + self.assertEqual(list(collections.UserDict(dict={'a': 42}).items()), + [('a', 42)]) + self.assertRaises(TypeError, collections.UserDict, 42) + self.assertRaises(TypeError, collections.UserDict, (), ()) + self.assertRaises(TypeError, collections.UserDict.__init__) + + def test_update(self): + for kw in 'self', 'dict', 'other', 'iterable': + d = collections.UserDict() + d.update(**{kw: 42}) + self.assertEqual(list(d.items()), [(kw, 42)]) + self.assertRaises(TypeError, collections.UserDict().update, 42) + self.assertRaises(TypeError, collections.UserDict().update, {}, {}) + self.assertRaises(TypeError, collections.UserDict.update) + def test_missing(self): # Make sure UserDict doesn't have a __missing__ method self.assertEqual(hasattr(collections.UserDict, "__missing__"), False) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -78,6 +78,9 @@ Library ------- +- Issue #22609: Constructor of collections.UserDict now accepts the self keyword + argument. + - Issue #25262. Added support for BINBYTES8 opcode in Python implementation of unpickler. Highest 32 bits of 64-bit size for BINUNICODE8 and BINBYTES8 opcodes no longer silently ignored on 32-bit platforms in C implementation. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 29 22:54:25 2015 From: python-checkins at python.org (serhiy.storchaka) Date: Tue, 29 Sep 2015 20:54:25 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Issue_=2322609=3A_Constructor_of_collections=2EUserDict_now_ac?= =?utf-8?q?cepts_the_self_keyword?= Message-ID: <20150929205424.9937.30503@psf.io> https://hg.python.org/cpython/rev/ab7e3f1f9f88 changeset: 98409:ab7e3f1f9f88 branch: 3.5 parent: 98405:2ecb7d4d9e0b parent: 98408:1869f5625392 user: Serhiy Storchaka date: Tue Sep 29 23:37:09 2015 +0300 summary: Issue #22609: Constructor of collections.UserDict now accepts the self keyword argument. files: Lib/collections/__init__.py | 17 ++++++++++++++- Lib/test/test_userdict.py | 27 ++++++++++++++++++++++++- Misc/NEWS | 3 ++ 3 files changed, 45 insertions(+), 2 deletions(-) diff --git a/Lib/collections/__init__.py b/Lib/collections/__init__.py --- a/Lib/collections/__init__.py +++ b/Lib/collections/__init__.py @@ -939,7 +939,22 @@ class UserDict(MutableMapping): # Start by filling-out the abstract methods - def __init__(self, dict=None, **kwargs): + def __init__(*args, **kwargs): + if not args: + raise TypeError("descriptor '__init__' of 'UserDict' object " + "needs an argument") + self, *args = args + if len(args) > 1: + raise TypeError('expected at most 1 arguments, got %d' % len(args)) + if args: + dict = args[0] + elif 'dict' in kwargs: + dict = kwargs.pop('dict') + import warnings + warnings.warn("Passing 'dict' as keyword argument is deprecated", + PendingDeprecationWarning, stacklevel=2) + else: + dict = None self.data = {} if dict is not None: self.update(dict) diff --git a/Lib/test/test_userdict.py b/Lib/test/test_userdict.py --- a/Lib/test/test_userdict.py +++ b/Lib/test/test_userdict.py @@ -29,7 +29,8 @@ self.assertEqual(collections.UserDict(one=1, two=2), d2) # item sequence constructor self.assertEqual(collections.UserDict([('one',1), ('two',2)]), d2) - self.assertEqual(collections.UserDict(dict=[('one',1), ('two',2)]), d2) + with self.assertWarnsRegex(PendingDeprecationWarning, "'dict'"): + self.assertEqual(collections.UserDict(dict=[('one',1), ('two',2)]), d2) # both together self.assertEqual(collections.UserDict([('one',1), ('two',2)], two=3, three=5), d3) @@ -139,6 +140,30 @@ self.assertEqual(t.popitem(), ("x", 42)) self.assertRaises(KeyError, t.popitem) + def test_init(self): + for kw in 'self', 'other', 'iterable': + self.assertEqual(list(collections.UserDict(**{kw: 42}).items()), + [(kw, 42)]) + self.assertEqual(list(collections.UserDict({}, dict=42).items()), + [('dict', 42)]) + self.assertEqual(list(collections.UserDict({}, dict=None).items()), + [('dict', None)]) + with self.assertWarnsRegex(PendingDeprecationWarning, "'dict'"): + self.assertEqual(list(collections.UserDict(dict={'a': 42}).items()), + [('a', 42)]) + self.assertRaises(TypeError, collections.UserDict, 42) + self.assertRaises(TypeError, collections.UserDict, (), ()) + self.assertRaises(TypeError, collections.UserDict.__init__) + + def test_update(self): + for kw in 'self', 'dict', 'other', 'iterable': + d = collections.UserDict() + d.update(**{kw: 42}) + self.assertEqual(list(d.items()), [(kw, 42)]) + self.assertRaises(TypeError, collections.UserDict().update, 42) + self.assertRaises(TypeError, collections.UserDict().update, {}, {}) + self.assertRaises(TypeError, collections.UserDict.update) + def test_missing(self): # Make sure UserDict doesn't have a __missing__ method self.assertEqual(hasattr(collections.UserDict, "__missing__"), False) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -21,6 +21,9 @@ Library ------- +- Issue #22609: Constructor of collections.UserDict now accepts the self keyword + argument. + - Issue #25111: Fixed comparison of traceback.FrameSummary. - Issue #25262. Added support for BINBYTES8 opcode in Python implementation of -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 29 22:54:25 2015 From: python-checkins at python.org (serhiy.storchaka) Date: Tue, 29 Sep 2015 20:54:25 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzIyOTU4?= =?utf-8?q?=3A_Constructor_and_update_method_of_weakref=2EWeakValueDiction?= =?utf-8?q?ary?= Message-ID: <20150929205425.94113.48032@psf.io> https://hg.python.org/cpython/rev/8274fc521e69 changeset: 98411:8274fc521e69 branch: 2.7 parent: 98407:4c5407e1b0ec user: Serhiy Storchaka date: Tue Sep 29 23:51:27 2015 +0300 summary: Issue #22958: Constructor and update method of weakref.WeakValueDictionary now accept the self keyword argument. files: Lib/test/test_weakref.py | 25 +++++++++++++++++++++++++ Lib/weakref.py | 19 +++++++++++++++++-- Misc/NEWS | 3 +++ 3 files changed, 45 insertions(+), 2 deletions(-) diff --git a/Lib/test/test_weakref.py b/Lib/test/test_weakref.py --- a/Lib/test/test_weakref.py +++ b/Lib/test/test_weakref.py @@ -1197,6 +1197,18 @@ dict[o] = o.arg return dict, objects + def test_make_weak_valued_dict_misc(self): + # errors + self.assertRaises(TypeError, weakref.WeakValueDictionary.__init__) + self.assertRaises(TypeError, weakref.WeakValueDictionary, {}, {}) + self.assertRaises(TypeError, weakref.WeakValueDictionary, (), ()) + # special keyword arguments + o = Object(3) + for kw in 'self', 'other', 'iterable': + d = weakref.WeakValueDictionary(**{kw: o}) + self.assertEqual(list(d.keys()), [kw]) + self.assertEqual(d[kw], o) + def make_weak_valued_dict(self): dict = weakref.WeakValueDictionary() objects = map(Object, range(self.COUNT)) @@ -1279,6 +1291,19 @@ def test_weak_valued_dict_update(self): self.check_update(weakref.WeakValueDictionary, {1: C(), 'a': C(), C(): C()}) + # errors + self.assertRaises(TypeError, weakref.WeakValueDictionary.update) + d = weakref.WeakValueDictionary() + self.assertRaises(TypeError, d.update, {}, {}) + self.assertRaises(TypeError, d.update, (), ()) + self.assertEqual(list(d.keys()), []) + # special keyword arguments + o = Object(3) + for kw in 'self', 'dict', 'other', 'iterable': + d = weakref.WeakValueDictionary() + d.update(**{kw: o}) + self.assertEqual(list(d.keys()), [kw]) + self.assertEqual(d[kw], o) def test_weak_keyed_dict_update(self): self.check_update(weakref.WeakKeyDictionary, diff --git a/Lib/weakref.py b/Lib/weakref.py --- a/Lib/weakref.py +++ b/Lib/weakref.py @@ -44,7 +44,14 @@ # objects are unwrapped on the way out, and we always wrap on the # way in). - def __init__(self, *args, **kw): + def __init__(*args, **kw): + if not args: + raise TypeError("descriptor '__init__' of 'WeakValueDictionary' " + "object needs an argument") + self = args[0] + args = args[1:] + if len(args) > 1: + raise TypeError('expected at most 1 arguments, got %d' % len(args)) def remove(wr, selfref=ref(self)): self = selfref() if self is not None: @@ -214,7 +221,15 @@ else: return wr() - def update(self, dict=None, **kwargs): + def update(*args, **kwargs): + if not args: + raise TypeError("descriptor 'update' of 'WeakValueDictionary' " + "object needs an argument") + self = args[0] + args = args[1:] + if len(args) > 1: + raise TypeError('expected at most 1 arguments, got %d' % len(args)) + dict = args[0] if args else None if self._pending_removals: self._commit_removals() d = self.data diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -37,6 +37,9 @@ Library ------- +- Issue #22958: Constructor and update method of weakref.WeakValueDictionary + now accept the self keyword argument. + - Issue #22609: Constructor and the update method of collections.UserDict now accept the self keyword argument. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 29 22:54:25 2015 From: python-checkins at python.org (serhiy.storchaka) Date: Tue, 29 Sep 2015 20:54:25 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzIyOTU4?= =?utf-8?q?=3A_Constructor_and_update_method_of_weakref=2EWeakValueDiction?= =?utf-8?q?ary?= Message-ID: <20150929205425.115507.40122@psf.io> https://hg.python.org/cpython/rev/01c79072d671 changeset: 98412:01c79072d671 branch: 3.4 parent: 98408:1869f5625392 user: Serhiy Storchaka date: Tue Sep 29 23:52:09 2015 +0300 summary: Issue #22958: Constructor and update method of weakref.WeakValueDictionary now accept the self and the dict keyword arguments. files: Lib/test/test_weakref.py | 25 +++++++++++++++++++++++++ Lib/weakref.py | 17 +++++++++++++++-- Misc/NEWS | 3 +++ 3 files changed, 43 insertions(+), 2 deletions(-) diff --git a/Lib/test/test_weakref.py b/Lib/test/test_weakref.py --- a/Lib/test/test_weakref.py +++ b/Lib/test/test_weakref.py @@ -1408,6 +1408,18 @@ dict2 = weakref.WeakValueDictionary(dict) self.assertEqual(dict[364], o) + def test_make_weak_valued_dict_misc(self): + # errors + self.assertRaises(TypeError, weakref.WeakValueDictionary.__init__) + self.assertRaises(TypeError, weakref.WeakValueDictionary, {}, {}) + self.assertRaises(TypeError, weakref.WeakValueDictionary, (), ()) + # special keyword arguments + o = Object(3) + for kw in 'self', 'dict', 'other', 'iterable': + d = weakref.WeakValueDictionary(**{kw: o}) + self.assertEqual(list(d.keys()), [kw]) + self.assertEqual(d[kw], o) + def make_weak_valued_dict(self): dict = weakref.WeakValueDictionary() objects = list(map(Object, range(self.COUNT))) @@ -1488,6 +1500,19 @@ def test_weak_valued_dict_update(self): self.check_update(weakref.WeakValueDictionary, {1: C(), 'a': C(), C(): C()}) + # errors + self.assertRaises(TypeError, weakref.WeakValueDictionary.update) + d = weakref.WeakValueDictionary() + self.assertRaises(TypeError, d.update, {}, {}) + self.assertRaises(TypeError, d.update, (), ()) + self.assertEqual(list(d.keys()), []) + # special keyword arguments + o = Object(3) + for kw in 'self', 'dict', 'other', 'iterable': + d = weakref.WeakValueDictionary() + d.update(**{kw: o}) + self.assertEqual(list(d.keys()), [kw]) + self.assertEqual(d[kw], o) def test_weak_keyed_dict_update(self): self.check_update(weakref.WeakKeyDictionary, diff --git a/Lib/weakref.py b/Lib/weakref.py --- a/Lib/weakref.py +++ b/Lib/weakref.py @@ -98,7 +98,13 @@ # objects are unwrapped on the way out, and we always wrap on the # way in). - def __init__(self, *args, **kw): + def __init__(*args, **kw): + if not args: + raise TypeError("descriptor '__init__' of 'WeakValueDictionary' " + "object needs an argument") + self, *args = args + if len(args) > 1: + raise TypeError('expected at most 1 arguments, got %d' % len(args)) def remove(wr, selfref=ref(self)): self = selfref() if self is not None: @@ -252,7 +258,14 @@ else: return wr() - def update(self, dict=None, **kwargs): + def update(*args, **kwargs): + if not args: + raise TypeError("descriptor 'update' of 'WeakValueDictionary' " + "object needs an argument") + self, *args = args + if len(args) > 1: + raise TypeError('expected at most 1 arguments, got %d' % len(args)) + dict = args[0] if args else None if self._pending_removals: self._commit_removals() d = self.data diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -78,6 +78,9 @@ Library ------- +- Issue #22958: Constructor and update method of weakref.WeakValueDictionary + now accept the self and the dict keyword arguments. + - Issue #22609: Constructor of collections.UserDict now accepts the self keyword argument. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 29 22:54:26 2015 From: python-checkins at python.org (serhiy.storchaka) Date: Tue, 29 Sep 2015 20:54:26 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Issue_=2322958=3A_Constructor_and_update_method_of_weakref=2EW?= =?utf-8?q?eakValueDictionary?= Message-ID: <20150929205425.115250.25261@psf.io> https://hg.python.org/cpython/rev/73b6b88ac28a changeset: 98413:73b6b88ac28a branch: 3.5 parent: 98409:ab7e3f1f9f88 parent: 98412:01c79072d671 user: Serhiy Storchaka date: Tue Sep 29 23:52:42 2015 +0300 summary: Issue #22958: Constructor and update method of weakref.WeakValueDictionary now accept the self and the dict keyword arguments. files: Lib/test/test_weakref.py | 25 +++++++++++++++++++++++++ Lib/weakref.py | 17 +++++++++++++++-- Misc/NEWS | 3 +++ 3 files changed, 43 insertions(+), 2 deletions(-) diff --git a/Lib/test/test_weakref.py b/Lib/test/test_weakref.py --- a/Lib/test/test_weakref.py +++ b/Lib/test/test_weakref.py @@ -1421,6 +1421,18 @@ dict2 = weakref.WeakValueDictionary(dict) self.assertEqual(dict[364], o) + def test_make_weak_valued_dict_misc(self): + # errors + self.assertRaises(TypeError, weakref.WeakValueDictionary.__init__) + self.assertRaises(TypeError, weakref.WeakValueDictionary, {}, {}) + self.assertRaises(TypeError, weakref.WeakValueDictionary, (), ()) + # special keyword arguments + o = Object(3) + for kw in 'self', 'dict', 'other', 'iterable': + d = weakref.WeakValueDictionary(**{kw: o}) + self.assertEqual(list(d.keys()), [kw]) + self.assertEqual(d[kw], o) + def make_weak_valued_dict(self): dict = weakref.WeakValueDictionary() objects = list(map(Object, range(self.COUNT))) @@ -1501,6 +1513,19 @@ def test_weak_valued_dict_update(self): self.check_update(weakref.WeakValueDictionary, {1: C(), 'a': C(), C(): C()}) + # errors + self.assertRaises(TypeError, weakref.WeakValueDictionary.update) + d = weakref.WeakValueDictionary() + self.assertRaises(TypeError, d.update, {}, {}) + self.assertRaises(TypeError, d.update, (), ()) + self.assertEqual(list(d.keys()), []) + # special keyword arguments + o = Object(3) + for kw in 'self', 'dict', 'other', 'iterable': + d = weakref.WeakValueDictionary() + d.update(**{kw: o}) + self.assertEqual(list(d.keys()), [kw]) + self.assertEqual(d[kw], o) def test_weak_keyed_dict_update(self): self.check_update(weakref.WeakKeyDictionary, diff --git a/Lib/weakref.py b/Lib/weakref.py --- a/Lib/weakref.py +++ b/Lib/weakref.py @@ -98,7 +98,13 @@ # objects are unwrapped on the way out, and we always wrap on the # way in). - def __init__(self, *args, **kw): + def __init__(*args, **kw): + if not args: + raise TypeError("descriptor '__init__' of 'WeakValueDictionary' " + "object needs an argument") + self, *args = args + if len(args) > 1: + raise TypeError('expected at most 1 arguments, got %d' % len(args)) def remove(wr, selfref=ref(self)): self = selfref() if self is not None: @@ -252,7 +258,14 @@ else: return wr() - def update(self, dict=None, **kwargs): + def update(*args, **kwargs): + if not args: + raise TypeError("descriptor 'update' of 'WeakValueDictionary' " + "object needs an argument") + self, *args = args + if len(args) > 1: + raise TypeError('expected at most 1 arguments, got %d' % len(args)) + dict = args[0] if args else None if self._pending_removals: self._commit_removals() d = self.data diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -21,6 +21,9 @@ Library ------- +- Issue #22958: Constructor and update method of weakref.WeakValueDictionary + now accept the self and the dict keyword arguments. + - Issue #22609: Constructor of collections.UserDict now accepts the self keyword argument. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 29 22:54:28 2015 From: python-checkins at python.org (serhiy.storchaka) Date: Tue, 29 Sep 2015 20:54:28 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2322958=3A_Constructor_and_update_method_of_weakr?= =?utf-8?q?ef=2EWeakValueDictionary?= Message-ID: <20150929205426.115182.22300@psf.io> https://hg.python.org/cpython/rev/815bb6a2d69e changeset: 98414:815bb6a2d69e parent: 98410:901964295066 parent: 98413:73b6b88ac28a user: Serhiy Storchaka date: Tue Sep 29 23:53:25 2015 +0300 summary: Issue #22958: Constructor and update method of weakref.WeakValueDictionary now accept the self and the dict keyword arguments. files: Lib/test/test_weakref.py | 25 +++++++++++++++++++++++++ Lib/weakref.py | 17 +++++++++++++++-- Misc/NEWS | 3 +++ 3 files changed, 43 insertions(+), 2 deletions(-) diff --git a/Lib/test/test_weakref.py b/Lib/test/test_weakref.py --- a/Lib/test/test_weakref.py +++ b/Lib/test/test_weakref.py @@ -1421,6 +1421,18 @@ dict2 = weakref.WeakValueDictionary(dict) self.assertEqual(dict[364], o) + def test_make_weak_valued_dict_misc(self): + # errors + self.assertRaises(TypeError, weakref.WeakValueDictionary.__init__) + self.assertRaises(TypeError, weakref.WeakValueDictionary, {}, {}) + self.assertRaises(TypeError, weakref.WeakValueDictionary, (), ()) + # special keyword arguments + o = Object(3) + for kw in 'self', 'dict', 'other', 'iterable': + d = weakref.WeakValueDictionary(**{kw: o}) + self.assertEqual(list(d.keys()), [kw]) + self.assertEqual(d[kw], o) + def make_weak_valued_dict(self): dict = weakref.WeakValueDictionary() objects = list(map(Object, range(self.COUNT))) @@ -1501,6 +1513,19 @@ def test_weak_valued_dict_update(self): self.check_update(weakref.WeakValueDictionary, {1: C(), 'a': C(), C(): C()}) + # errors + self.assertRaises(TypeError, weakref.WeakValueDictionary.update) + d = weakref.WeakValueDictionary() + self.assertRaises(TypeError, d.update, {}, {}) + self.assertRaises(TypeError, d.update, (), ()) + self.assertEqual(list(d.keys()), []) + # special keyword arguments + o = Object(3) + for kw in 'self', 'dict', 'other', 'iterable': + d = weakref.WeakValueDictionary() + d.update(**{kw: o}) + self.assertEqual(list(d.keys()), [kw]) + self.assertEqual(d[kw], o) def test_weak_keyed_dict_update(self): self.check_update(weakref.WeakKeyDictionary, diff --git a/Lib/weakref.py b/Lib/weakref.py --- a/Lib/weakref.py +++ b/Lib/weakref.py @@ -98,7 +98,13 @@ # objects are unwrapped on the way out, and we always wrap on the # way in). - def __init__(self, *args, **kw): + def __init__(*args, **kw): + if not args: + raise TypeError("descriptor '__init__' of 'WeakValueDictionary' " + "object needs an argument") + self, *args = args + if len(args) > 1: + raise TypeError('expected at most 1 arguments, got %d' % len(args)) def remove(wr, selfref=ref(self)): self = selfref() if self is not None: @@ -252,7 +258,14 @@ else: return wr() - def update(self, dict=None, **kwargs): + def update(*args, **kwargs): + if not args: + raise TypeError("descriptor 'update' of 'WeakValueDictionary' " + "object needs an argument") + self, *args = args + if len(args) > 1: + raise TypeError('expected at most 1 arguments, got %d' % len(args)) + dict = args[0] if args else None if self._pending_removals: self._commit_removals() d = self.data diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -190,6 +190,9 @@ Library ------- +- Issue #22958: Constructor and update method of weakref.WeakValueDictionary + now accept the self and the dict keyword arguments. + - Issue #22609: Constructor of collections.UserDict now accepts the self keyword argument. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 29 22:57:27 2015 From: python-checkins at python.org (victor.stinner) Date: Tue, 29 Sep 2015 20:57:27 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2325220=3A_Split_th?= =?utf-8?q?e_huge_main=28=29_function_of_libregrtest=2Emain_into_a_class?= Message-ID: <20150929205724.82658.68242@psf.io> https://hg.python.org/cpython/rev/817e25bd34d0 changeset: 98415:817e25bd34d0 user: Victor Stinner date: Tue Sep 29 22:48:52 2015 +0200 summary: Issue #25220: Split the huge main() function of libregrtest.main into a class with attributes and methods. The --threshold command line option is now ignored if the gc module is missing. * Convert main() variables to Regrtest attributes, document some attributes * Convert accumulate_result() function to a method * Create setup_python() function and setup_regrtest() method. * Import gc at top level * Move resource.setrlimit() and the code to make the module paths absolute into the new setup_python() function. So this code is no more executed when the module is imported, only when main() is executed. We have a better control on when the setup is done. * Move textwrap import from printlist() to the top level. * Some other minor cleanup. files: Lib/test/libregrtest/main.py | 689 ++++++++++++---------- 1 files changed, 381 insertions(+), 308 deletions(-) diff --git a/Lib/test/libregrtest/main.py b/Lib/test/libregrtest/main.py --- a/Lib/test/libregrtest/main.py +++ b/Lib/test/libregrtest/main.py @@ -1,13 +1,14 @@ import faulthandler import json import os +import platform +import random import re +import signal import sys +import sysconfig import tempfile -import sysconfig -import signal -import random -import platform +import textwrap import traceback import unittest from test.libregrtest.runtest import ( @@ -19,45 +20,15 @@ from test.libregrtest.cmdline import _parse_args from test import support try: + import gc +except ImportError: + gc = None +try: import threading except ImportError: threading = None -# Some times __path__ and __file__ are not absolute (e.g. while running from -# Lib/) and, if we change the CWD to run the tests in a temporary dir, some -# imports might fail. This affects only the modules imported before os.chdir(). -# These modules are searched first in sys.path[0] (so '' -- the CWD) and if -# they are found in the CWD their __file__ and __path__ will be relative (this -# happens before the chdir). All the modules imported after the chdir, are -# not found in the CWD, and since the other paths in sys.path[1:] are absolute -# (site.py absolutize them), the __file__ and __path__ will be absolute too. -# Therefore it is necessary to absolutize manually the __file__ and __path__ of -# the packages to prevent later imports to fail when the CWD is different. -for module in sys.modules.values(): - if hasattr(module, '__path__'): - module.__path__ = [os.path.abspath(path) for path in module.__path__] - if hasattr(module, '__file__'): - module.__file__ = os.path.abspath(module.__file__) - - -# MacOSX (a.k.a. Darwin) has a default stack size that is too small -# for deeply recursive regular expressions. We see this as crashes in -# the Python test suite when running test_re.py and test_sre.py. The -# fix is to set the stack limit to 2048. -# This approach may also be useful for other Unixy platforms that -# suffer from small default stack limits. -if sys.platform == 'darwin': - try: - import resource - except ImportError: - pass - else: - soft, hard = resource.getrlimit(resource.RLIMIT_STACK) - newsoft = min(hard, max(soft, 1024*2048)) - resource.setrlimit(resource.RLIMIT_STACK, (newsoft, hard)) - - # When tests are run from the Python build directory, it is best practice # to keep the test files in a subfolder. This eases the cleanup of leftover # files using the "make distclean" command. @@ -68,7 +39,73 @@ TEMPDIR = os.path.abspath(TEMPDIR) -def main(tests=None, **kwargs): +def slave_runner(slaveargs): + args, kwargs = json.loads(slaveargs) + if kwargs.get('huntrleaks'): + unittest.BaseTestSuite._cleanup = False + try: + result = runtest(*args, **kwargs) + except KeyboardInterrupt: + result = INTERRUPTED, '' + except BaseException as e: + traceback.print_exc() + result = CHILD_ERROR, str(e) + sys.stdout.flush() + print() # Force a newline (just in case) + print(json.dumps(result)) + sys.exit(0) + + +def setup_python(): + # Display the Python traceback on fatal errors (e.g. segfault) + faulthandler.enable(all_threads=True) + + # Display the Python traceback on SIGALRM or SIGUSR1 signal + signals = [] + if hasattr(signal, 'SIGALRM'): + signals.append(signal.SIGALRM) + if hasattr(signal, 'SIGUSR1'): + signals.append(signal.SIGUSR1) + for signum in signals: + faulthandler.register(signum, chain=True) + + replace_stdout() + support.record_original_stdout(sys.stdout) + + # Some times __path__ and __file__ are not absolute (e.g. while running from + # Lib/) and, if we change the CWD to run the tests in a temporary dir, some + # imports might fail. This affects only the modules imported before os.chdir(). + # These modules are searched first in sys.path[0] (so '' -- the CWD) and if + # they are found in the CWD their __file__ and __path__ will be relative (this + # happens before the chdir). All the modules imported after the chdir, are + # not found in the CWD, and since the other paths in sys.path[1:] are absolute + # (site.py absolutize them), the __file__ and __path__ will be absolute too. + # Therefore it is necessary to absolutize manually the __file__ and __path__ of + # the packages to prevent later imports to fail when the CWD is different. + for module in sys.modules.values(): + if hasattr(module, '__path__'): + module.__path__ = [os.path.abspath(path) for path in module.__path__] + if hasattr(module, '__file__'): + module.__file__ = os.path.abspath(module.__file__) + + # MacOSX (a.k.a. Darwin) has a default stack size that is too small + # for deeply recursive regular expressions. We see this as crashes in + # the Python test suite when running test_re.py and test_sre.py. The + # fix is to set the stack limit to 2048. + # This approach may also be useful for other Unixy platforms that + # suffer from small default stack limits. + if sys.platform == 'darwin': + try: + import resource + except ImportError: + pass + else: + soft, hard = resource.getrlimit(resource.RLIMIT_STACK) + newsoft = min(hard, max(soft, 1024*2048)) + resource.setrlimit(resource.RLIMIT_STACK, (newsoft, hard)) + + +class Regrtest: """Execute a test suite. This also parses command-line options and modifies its behavior @@ -91,210 +128,257 @@ directly to set the values that would normally be set by flags on the command line. """ - # Display the Python traceback on fatal errors (e.g. segfault) - faulthandler.enable(all_threads=True) + def __init__(self): + # Namespace of command line options + self.ns = None - # Display the Python traceback on SIGALRM or SIGUSR1 signal - signals = [] - if hasattr(signal, 'SIGALRM'): - signals.append(signal.SIGALRM) - if hasattr(signal, 'SIGUSR1'): - signals.append(signal.SIGUSR1) - for signum in signals: - faulthandler.register(signum, chain=True) + # tests + self.tests = [] + self.selected = [] - replace_stdout() + # test results + self.good = [] + self.bad = [] + self.skipped = [] + self.resource_denieds = [] + self.environment_changed = [] + self.interrupted = False - support.record_original_stdout(sys.stdout) + # used by --slow + self.test_times = [] - ns = _parse_args(sys.argv[1:], **kwargs) + # used by --coverage, trace.Trace instance + self.tracer = None - if ns.huntrleaks: - # Avoid false positives due to various caches - # filling slowly with random data: - warm_caches() - if ns.memlimit is not None: - support.set_memlimit(ns.memlimit) - if ns.threshold is not None: - import gc - gc.set_threshold(ns.threshold) - if ns.nowindows: - import msvcrt - msvcrt.SetErrorMode(msvcrt.SEM_FAILCRITICALERRORS| - msvcrt.SEM_NOALIGNMENTFAULTEXCEPT| - msvcrt.SEM_NOGPFAULTERRORBOX| - msvcrt.SEM_NOOPENFILEERRORBOX) - try: - msvcrt.CrtSetReportMode - except AttributeError: - # release build - pass + # used by --findleaks, store for gc.garbage + self.found_garbage = [] + + # used to display the progress bar "[ 3/100]" + self.test_count = '' + self.test_count_width = 1 + + # used by --single + self.next_single_test = None + self.next_single_filename = None + + def accumulate_result(self, test, result): + ok, test_time = result + self.test_times.append((test_time, test)) + if ok == PASSED: + self.good.append(test) + elif ok == FAILED: + self.bad.append(test) + elif ok == ENV_CHANGED: + self.environment_changed.append(test) + elif ok == SKIPPED: + self.skipped.append(test) + elif ok == RESOURCE_DENIED: + self.skipped.append(test) + self.resource_denieds.append(test) + + def display_progress(self, test_index, test): + if self.ns.quiet: + return + fmt = "[{1:{0}}{2}/{3}] {4}" if self.bad else "[{1:{0}}{2}] {4}" + print(fmt.format( + self.test_count_width, test_index, self.test_count, len(self.bad), test)) + sys.stdout.flush() + + def setup_regrtest(self): + if self.ns.huntrleaks: + # Avoid false positives due to various caches + # filling slowly with random data: + warm_caches() + + if self.ns.memlimit is not None: + support.set_memlimit(self.ns.memlimit) + + if self.ns.threshold is not None: + if gc is not None: + gc.set_threshold(self.ns.threshold) + else: + print('No GC available, ignore --threshold.') + + if self.ns.nowindows: + import msvcrt + msvcrt.SetErrorMode(msvcrt.SEM_FAILCRITICALERRORS| + msvcrt.SEM_NOALIGNMENTFAULTEXCEPT| + msvcrt.SEM_NOGPFAULTERRORBOX| + msvcrt.SEM_NOOPENFILEERRORBOX) + try: + msvcrt.CrtSetReportMode + except AttributeError: + # release build + pass + else: + for m in [msvcrt.CRT_WARN, msvcrt.CRT_ERROR, msvcrt.CRT_ASSERT]: + msvcrt.CrtSetReportMode(m, msvcrt.CRTDBG_MODE_FILE) + msvcrt.CrtSetReportFile(m, msvcrt.CRTDBG_FILE_STDERR) + + if self.ns.findleaks: + if gc is not None: + # Uncomment the line below to report garbage that is not + # freeable by reference counting alone. By default only + # garbage that is not collectable by the GC is reported. + pass + #gc.set_debug(gc.DEBUG_SAVEALL) + else: + print('No GC available, disabling --findleaks') + self.ns.findleaks = False + + if self.ns.huntrleaks: + unittest.BaseTestSuite._cleanup = False + + # Strip .py extensions. + removepy(self.ns.args) + + if self.ns.trace: + import trace + self.tracer = trace.Trace(ignoredirs=[sys.base_prefix, + sys.base_exec_prefix, + tempfile.gettempdir()], + trace=False, count=True) + + def find_tests(self, tests): + self.tests = tests + + if self.ns.single: + self.next_single_filename = os.path.join(TEMPDIR, 'pynexttest') + try: + with open(self.next_single_filename, 'r') as fp: + next_test = fp.read().strip() + self.tests = [next_test] + except OSError: + pass + + if self.ns.fromfile: + self.tests = [] + with open(os.path.join(support.SAVEDCWD, self.ns.fromfile)) as fp: + count_pat = re.compile(r'\[\s*\d+/\s*\d+\]') + for line in fp: + line = count_pat.sub('', line) + guts = line.split() # assuming no test has whitespace in its name + if guts and not guts[0].startswith('#'): + self.tests.extend(guts) + + removepy(self.tests) + + stdtests = STDTESTS[:] + nottests = NOTTESTS.copy() + if self.ns.exclude: + for arg in self.ns.args: + if arg in stdtests: + stdtests.remove(arg) + nottests.add(arg) + self.ns.args = [] + + # For a partial run, we do not need to clutter the output. + if self.ns.verbose or self.ns.header or not (self.ns.quiet or self.ns.single or self.tests or self.ns.args): + # Print basic platform information + print("==", platform.python_implementation(), *sys.version.split()) + print("== ", platform.platform(aliased=True), + "%s-endian" % sys.byteorder) + print("== ", "hash algorithm:", sys.hash_info.algorithm, + "64bit" if sys.maxsize > 2**32 else "32bit") + print("== ", os.getcwd()) + print("Testing with flags:", sys.flags) + + # if testdir is set, then we are not running the python tests suite, so + # don't add default tests to be executed or skipped (pass empty values) + if self.ns.testdir: + alltests = findtests(self.ns.testdir, list(), set()) else: - for m in [msvcrt.CRT_WARN, msvcrt.CRT_ERROR, msvcrt.CRT_ASSERT]: - msvcrt.CrtSetReportMode(m, msvcrt.CRTDBG_MODE_FILE) - msvcrt.CrtSetReportFile(m, msvcrt.CRTDBG_FILE_STDERR) - if ns.wait: - input("Press any key to continue...") + alltests = findtests(self.ns.testdir, stdtests, nottests) - if ns.slaveargs is not None: - args, kwargs = json.loads(ns.slaveargs) - if kwargs.get('huntrleaks'): - unittest.BaseTestSuite._cleanup = False - try: - result = runtest(*args, **kwargs) - except KeyboardInterrupt: - result = INTERRUPTED, '' - except BaseException as e: - traceback.print_exc() - result = CHILD_ERROR, str(e) - sys.stdout.flush() - print() # Force a newline (just in case) - print(json.dumps(result)) - sys.exit(0) + self.selected = self.tests or self.ns.args or alltests + if self.ns.single: + self.selected = self.selected[:1] + try: + pos = alltests.index(self.selected[0]) + self.next_single_test = alltests[pos + 1] + except IndexError: + pass - good = [] - bad = [] - skipped = [] - resource_denieds = [] - environment_changed = [] - interrupted = False + # Remove all the self.selected tests that precede start if it's set. + if self.ns.start: + try: + del self.selected[:self.selected.index(self.ns.start)] + except ValueError: + print("Couldn't find starting test (%s), using all tests" % self.ns.start) - if ns.findleaks: - try: - import gc - except ImportError: - print('No GC available, disabling findleaks.') - ns.findleaks = False - else: - # Uncomment the line below to report garbage that is not - # freeable by reference counting alone. By default only - # garbage that is not collectable by the GC is reported. - #gc.set_debug(gc.DEBUG_SAVEALL) - found_garbage = [] + if self.ns.randomize: + if self.ns.random_seed is None: + self.ns.random_seed = random.randrange(10000000) + random.seed(self.ns.random_seed) + print("Using random seed", self.ns.random_seed) + random.shuffle(self.selected) - if ns.huntrleaks: - unittest.BaseTestSuite._cleanup = False + def display_result(self): + if self.interrupted: + # print a newline after ^C + print() + print("Test suite interrupted by signal SIGINT.") + omitted = set(self.selected) - set(self.good) - set(self.bad) - set(self.skipped) + print(count(len(omitted), "test"), "omitted:") + printlist(omitted) - if ns.single: - filename = os.path.join(TEMPDIR, 'pynexttest') - try: - with open(filename, 'r') as fp: - next_test = fp.read().strip() - tests = [next_test] - except OSError: - pass + if self.good and not self.ns.quiet: + if not self.bad and not self.skipped and not self.interrupted and len(self.good) > 1: + print("All", end=' ') + print(count(len(self.good), "test"), "OK.") - if ns.fromfile: - tests = [] - with open(os.path.join(support.SAVEDCWD, ns.fromfile)) as fp: - count_pat = re.compile(r'\[\s*\d+/\s*\d+\]') - for line in fp: - line = count_pat.sub('', line) - guts = line.split() # assuming no test has whitespace in its name - if guts and not guts[0].startswith('#'): - tests.extend(guts) + if self.ns.print_slow: + self.test_times.sort(reverse=True) + print("10 slowest tests:") + for time, test in self.test_times[:10]: + print("%s: %.1fs" % (test, time)) - # Strip .py extensions. - removepy(ns.args) - removepy(tests) + if self.bad: + print(count(len(self.bad), "test"), "failed:") + printlist(self.bad) - stdtests = STDTESTS[:] - nottests = NOTTESTS.copy() - if ns.exclude: - for arg in ns.args: - if arg in stdtests: - stdtests.remove(arg) - nottests.add(arg) - ns.args = [] + if self.environment_changed: + print("{} altered the execution environment:".format( + count(len(self.environment_changed), "test"))) + printlist(self.environment_changed) - # For a partial run, we do not need to clutter the output. - if ns.verbose or ns.header or not (ns.quiet or ns.single or tests or ns.args): - # Print basic platform information - print("==", platform.python_implementation(), *sys.version.split()) - print("== ", platform.platform(aliased=True), - "%s-endian" % sys.byteorder) - print("== ", "hash algorithm:", sys.hash_info.algorithm, - "64bit" if sys.maxsize > 2**32 else "32bit") - print("== ", os.getcwd()) - print("Testing with flags:", sys.flags) + if self.skipped and not self.ns.quiet: + print(count(len(self.skipped), "test"), "skipped:") + printlist(self.skipped) - # if testdir is set, then we are not running the python tests suite, so - # don't add default tests to be executed or skipped (pass empty values) - if ns.testdir: - alltests = findtests(ns.testdir, list(), set()) - else: - alltests = findtests(ns.testdir, stdtests, nottests) + if self.ns.verbose2 and self.bad: + print("Re-running failed tests in verbose mode") + for test in self.bad[:]: + print("Re-running test %r in verbose mode" % test) + sys.stdout.flush() + try: + self.ns.verbose = True + ok = runtest(test, True, self.ns.quiet, self.ns.huntrleaks, + timeout=self.ns.timeout) + except KeyboardInterrupt: + # print a newline separate from the ^C + print() + break + else: + if ok[0] in {PASSED, ENV_CHANGED, SKIPPED, RESOURCE_DENIED}: + self.bad.remove(test) + else: + if self.bad: + print(count(len(self.bad), 'test'), "failed again:") + printlist(self.bad) - selected = tests or ns.args or alltests - if ns.single: - selected = selected[:1] - try: - next_single_test = alltests[alltests.index(selected[0])+1] - except IndexError: - next_single_test = None - # Remove all the selected tests that precede start if it's set. - if ns.start: - try: - del selected[:selected.index(ns.start)] - except ValueError: - print("Couldn't find starting test (%s), using all tests" % ns.start) - if ns.randomize: - if ns.random_seed is None: - ns.random_seed = random.randrange(10000000) - random.seed(ns.random_seed) - print("Using random seed", ns.random_seed) - random.shuffle(selected) - if ns.trace: - import trace, tempfile - tracer = trace.Trace(ignoredirs=[sys.base_prefix, sys.base_exec_prefix, - tempfile.gettempdir()], - trace=False, count=True) - - test_times = [] - support.verbose = ns.verbose # Tell tests to be moderately quiet - support.use_resources = ns.use_resources - save_modules = sys.modules.keys() - - def accumulate_result(test, result): - ok, test_time = result - test_times.append((test_time, test)) - if ok == PASSED: - good.append(test) - elif ok == FAILED: - bad.append(test) - elif ok == ENV_CHANGED: - environment_changed.append(test) - elif ok == SKIPPED: - skipped.append(test) - elif ok == RESOURCE_DENIED: - skipped.append(test) - resource_denieds.append(test) - - if ns.forever: - def test_forever(tests=list(selected)): - while True: - for test in tests: - yield test - if bad: - return - tests = test_forever() - test_count = '' - test_count_width = 3 - else: - tests = iter(selected) - test_count = '/{}'.format(len(selected)) - test_count_width = len(test_count) - 1 - - if ns.use_mp: + def _run_tests_mp(self): try: from threading import Thread except ImportError: print("Multiprocess option requires thread support") sys.exit(2) from queue import Queue + debug_output_pat = re.compile(r"\[\d+ refs, \d+ blocks\]$") output = Queue() - pending = MultiprocessTests(tests) + pending = MultiprocessTests(self.tests) + def work(): # A worker thread. try: @@ -304,7 +388,7 @@ except StopIteration: output.put((None, None, None, None)) return - retcode, stdout, stderr = run_test_in_subprocess(test, ns) + retcode, stdout, stderr = run_test_in_subprocess(test, self.ns) # Strip last refcount output line if it exists, since it # comes from the shutdown of the interpreter in the subcommand. stderr = debug_output_pat.sub("", stderr) @@ -321,23 +405,20 @@ except BaseException: output.put((None, None, None, None)) raise - workers = [Thread(target=work) for i in range(ns.use_mp)] + + workers = [Thread(target=work) for i in range(self.ns.use_mp)] for worker in workers: worker.start() finished = 0 test_index = 1 try: - while finished < ns.use_mp: + while finished < self.ns.use_mp: test, stdout, stderr, result = output.get() if test is None: finished += 1 continue - accumulate_result(test, result) - if not ns.quiet: - fmt = "[{1:{0}}{2}/{3}] {4}" if bad else "[{1:{0}}{2}] {4}" - print(fmt.format( - test_count_width, test_index, test_count, - len(bad), test)) + self.accumulate_result(test, result) + self.display_progress(test_index, test) if stdout: print(stdout) if stderr: @@ -350,110 +431,99 @@ raise Exception("Child error on {}: {}".format(test, result[1])) test_index += 1 except KeyboardInterrupt: - interrupted = True + self.interrupted = True pending.interrupted = True for worker in workers: worker.join() - else: - for test_index, test in enumerate(tests, 1): - if not ns.quiet: - fmt = "[{1:{0}}{2}/{3}] {4}" if bad else "[{1:{0}}{2}] {4}" - print(fmt.format( - test_count_width, test_index, test_count, len(bad), test)) - sys.stdout.flush() - if ns.trace: + + def _run_tests_sequential(self): + save_modules = sys.modules.keys() + + for test_index, test in enumerate(self.tests, 1): + self.display_progress(test_index, test) + if self.ns.trace: # If we're tracing code coverage, then we don't exit with status # if on a false return value from main. - tracer.runctx('runtest(test, ns.verbose, ns.quiet, timeout=ns.timeout)', - globals=globals(), locals=vars()) + cmd = 'runtest(test, self.ns.verbose, self.ns.quiet, timeout=self.ns.timeout)' + self.tracer.runctx(cmd, globals=globals(), locals=vars()) else: try: - result = runtest(test, ns.verbose, ns.quiet, - ns.huntrleaks, - output_on_failure=ns.verbose3, - timeout=ns.timeout, failfast=ns.failfast, - match_tests=ns.match_tests) - accumulate_result(test, result) + result = runtest(test, self.ns.verbose, self.ns.quiet, + self.ns.huntrleaks, + output_on_failure=self.ns.verbose3, + timeout=self.ns.timeout, failfast=self.ns.failfast, + match_tests=self.ns.match_tests) + self.accumulate_result(test, result) except KeyboardInterrupt: - interrupted = True + self.interrupted = True break - if ns.findleaks: + if self.ns.findleaks: gc.collect() if gc.garbage: print("Warning: test created", len(gc.garbage), end=' ') print("uncollectable object(s).") # move the uncollectable objects somewhere so we don't see # them again - found_garbage.extend(gc.garbage) + self.found_garbage.extend(gc.garbage) del gc.garbage[:] # Unload the newly imported modules (best effort finalization) for module in sys.modules.keys(): if module not in save_modules and module.startswith("test."): support.unload(module) - if interrupted: - # print a newline after ^C - print() - print("Test suite interrupted by signal SIGINT.") - omitted = set(selected) - set(good) - set(bad) - set(skipped) - print(count(len(omitted), "test"), "omitted:") - printlist(omitted) - if good and not ns.quiet: - if not bad and not skipped and not interrupted and len(good) > 1: - print("All", end=' ') - print(count(len(good), "test"), "OK.") - if ns.print_slow: - test_times.sort(reverse=True) - print("10 slowest tests:") - for time, test in test_times[:10]: - print("%s: %.1fs" % (test, time)) - if bad: - print(count(len(bad), "test"), "failed:") - printlist(bad) - if environment_changed: - print("{} altered the execution environment:".format( - count(len(environment_changed), "test"))) - printlist(environment_changed) - if skipped and not ns.quiet: - print(count(len(skipped), "test"), "skipped:") - printlist(skipped) + def run_tests(self): + support.verbose = self.ns.verbose # Tell tests to be moderately quiet + support.use_resources = self.ns.use_resources - if ns.verbose2 and bad: - print("Re-running failed tests in verbose mode") - for test in bad[:]: - print("Re-running test %r in verbose mode" % test) - sys.stdout.flush() - try: - ns.verbose = True - ok = runtest(test, True, ns.quiet, ns.huntrleaks, - timeout=ns.timeout) - except KeyboardInterrupt: - # print a newline separate from the ^C - print() - break + if self.ns.forever: + def test_forever(tests): + while True: + for test in tests: + yield test + if self.bad: + return + self.tests = test_forever(list(self.selected)) + self.test_count = '' + self.test_count_width = 3 + else: + self.tests = iter(self.selected) + self.test_count = '/{}'.format(len(self.selected)) + self.test_count_width = len(self.test_count) - 1 + + if self.ns.use_mp: + self._run_tests_mp() + else: + self._run_tests_sequential() + + def finalize(self): + if self.next_single_filename: + if self.next_single_test: + with open(self.next_single_filename, 'w') as fp: + fp.write(self.next_single_test + '\n') else: - if ok[0] in {PASSED, ENV_CHANGED, SKIPPED, RESOURCE_DENIED}: - bad.remove(test) - else: - if bad: - print(count(len(bad), 'test'), "failed again:") - printlist(bad) + os.unlink(self.next_single_filename) - if ns.single: - if next_single_test: - with open(filename, 'w') as fp: - fp.write(next_single_test + '\n') - else: - os.unlink(filename) + if self.ns.trace: + r = self.tracer.results() + r.write_results(show_missing=True, summary=True, + coverdir=self.ns.coverdir) - if ns.trace: - r = tracer.results() - r.write_results(show_missing=True, summary=True, coverdir=ns.coverdir) + if self.ns.runleaks: + os.system("leaks %d" % os.getpid()) - if ns.runleaks: - os.system("leaks %d" % os.getpid()) - - sys.exit(len(bad) > 0 or interrupted) + def main(self, tests=None, **kwargs): + setup_python() + self.ns = _parse_args(sys.argv[1:], **kwargs) + self.setup_regrtest() + if self.ns.wait: + input("Press any key to continue...") + if self.ns.slaveargs is not None: + slave_runner(self.ns.slaveargs) + self.find_tests(tests) + self.run_tests() + self.display_result() + self.finalize() + sys.exit(len(self.bad) > 0 or self.interrupted) # We do not use a generator so multiple threads can call next(). @@ -518,11 +588,14 @@ begin each line. """ - from textwrap import fill blanks = ' ' * indent # Print the sorted list: 'x' may be a '--random' list or a set() - print(fill(' '.join(str(elt) for elt in sorted(x)), width, - initial_indent=blanks, subsequent_indent=blanks)) + print(textwrap.fill(' '.join(str(elt) for elt in sorted(x)), width, + initial_indent=blanks, subsequent_indent=blanks)) + + +def main(tests=None, **kwargs): + Regrtest().main(tests=tests, **kwargs) def main_in_temp_cwd(): -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 29 23:25:04 2015 From: python-checkins at python.org (victor.stinner) Date: Tue, 29 Sep 2015 21:25:04 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2325220=3A_Create_l?= =?utf-8?q?ibregrtest/runtest=5Fmp=2Epy?= Message-ID: <20150929212503.82658.3693@psf.io> https://hg.python.org/cpython/rev/12c666eea556 changeset: 98416:12c666eea556 user: Victor Stinner date: Tue Sep 29 23:15:38 2015 +0200 summary: Issue #25220: Create libregrtest/runtest_mp.py Move the code to run tests in multiple processes using threading and subprocess to a new submodule. Move also slave_runner() (renamed to run_tests_slave()) and run_test_in_subprocess() (renamed to run_tests_in_subprocess()) there. files: Lib/test/libregrtest/main.py | 120 +------------ Lib/test/libregrtest/runtest.py | 33 --- Lib/test/libregrtest/runtest_mp.py | 158 +++++++++++++++++ 3 files changed, 164 insertions(+), 147 deletions(-) diff --git a/Lib/test/libregrtest/main.py b/Lib/test/libregrtest/main.py --- a/Lib/test/libregrtest/main.py +++ b/Lib/test/libregrtest/main.py @@ -1,5 +1,4 @@ import faulthandler -import json import os import platform import random @@ -9,13 +8,10 @@ import sysconfig import tempfile import textwrap -import traceback import unittest from test.libregrtest.runtest import ( - findtests, runtest, run_test_in_subprocess, - STDTESTS, NOTTESTS, - PASSED, FAILED, ENV_CHANGED, SKIPPED, - RESOURCE_DENIED, INTERRUPTED, CHILD_ERROR) + findtests, runtest, + STDTESTS, NOTTESTS, PASSED, FAILED, ENV_CHANGED, SKIPPED, RESOURCE_DENIED) from test.libregrtest.refleak import warm_caches from test.libregrtest.cmdline import _parse_args from test import support @@ -39,23 +35,6 @@ TEMPDIR = os.path.abspath(TEMPDIR) -def slave_runner(slaveargs): - args, kwargs = json.loads(slaveargs) - if kwargs.get('huntrleaks'): - unittest.BaseTestSuite._cleanup = False - try: - result = runtest(*args, **kwargs) - except KeyboardInterrupt: - result = INTERRUPTED, '' - except BaseException as e: - traceback.print_exc() - result = CHILD_ERROR, str(e) - sys.stdout.flush() - print() # Force a newline (just in case) - print(json.dumps(result)) - sys.exit(0) - - def setup_python(): # Display the Python traceback on fatal errors (e.g. segfault) faulthandler.enable(all_threads=True) @@ -367,75 +346,6 @@ print(count(len(self.bad), 'test'), "failed again:") printlist(self.bad) - def _run_tests_mp(self): - try: - from threading import Thread - except ImportError: - print("Multiprocess option requires thread support") - sys.exit(2) - from queue import Queue - - debug_output_pat = re.compile(r"\[\d+ refs, \d+ blocks\]$") - output = Queue() - pending = MultiprocessTests(self.tests) - - def work(): - # A worker thread. - try: - while True: - try: - test = next(pending) - except StopIteration: - output.put((None, None, None, None)) - return - retcode, stdout, stderr = run_test_in_subprocess(test, self.ns) - # Strip last refcount output line if it exists, since it - # comes from the shutdown of the interpreter in the subcommand. - stderr = debug_output_pat.sub("", stderr) - stdout, _, result = stdout.strip().rpartition("\n") - if retcode != 0: - result = (CHILD_ERROR, "Exit code %s" % retcode) - output.put((test, stdout.rstrip(), stderr.rstrip(), result)) - return - if not result: - output.put((None, None, None, None)) - return - result = json.loads(result) - output.put((test, stdout.rstrip(), stderr.rstrip(), result)) - except BaseException: - output.put((None, None, None, None)) - raise - - workers = [Thread(target=work) for i in range(self.ns.use_mp)] - for worker in workers: - worker.start() - finished = 0 - test_index = 1 - try: - while finished < self.ns.use_mp: - test, stdout, stderr, result = output.get() - if test is None: - finished += 1 - continue - self.accumulate_result(test, result) - self.display_progress(test_index, test) - if stdout: - print(stdout) - if stderr: - print(stderr, file=sys.stderr) - sys.stdout.flush() - sys.stderr.flush() - if result[0] == INTERRUPTED: - raise KeyboardInterrupt - if result[0] == CHILD_ERROR: - raise Exception("Child error on {}: {}".format(test, result[1])) - test_index += 1 - except KeyboardInterrupt: - self.interrupted = True - pending.interrupted = True - for worker in workers: - worker.join() - def _run_tests_sequential(self): save_modules = sys.modules.keys() @@ -491,7 +401,8 @@ self.test_count_width = len(self.test_count) - 1 if self.ns.use_mp: - self._run_tests_mp() + from test.libregrtest.runtest_mp import run_tests_multiprocess + run_tests_multiprocess(self) else: self._run_tests_sequential() @@ -518,7 +429,8 @@ if self.ns.wait: input("Press any key to continue...") if self.ns.slaveargs is not None: - slave_runner(self.ns.slaveargs) + from test.libregrtest.runtest_mp import run_tests_slave + run_tests_slave(self.ns.slaveargs) self.find_tests(tests) self.run_tests() self.display_result() @@ -526,26 +438,6 @@ sys.exit(len(self.bad) > 0 or self.interrupted) -# We do not use a generator so multiple threads can call next(). -class MultiprocessTests(object): - - """A thread-safe iterator over tests for multiprocess mode.""" - - def __init__(self, tests): - self.interrupted = False - self.lock = threading.Lock() - self.tests = tests - - def __iter__(self): - return self - - def __next__(self): - with self.lock: - if self.interrupted: - raise StopIteration('tests interrupted') - return next(self.tests) - - def replace_stdout(): """Set stdout encoder error handler to backslashreplace (as stderr error handler) to avoid UnicodeEncodeError when printing a traceback""" diff --git a/Lib/test/libregrtest/runtest.py b/Lib/test/libregrtest/runtest.py --- a/Lib/test/libregrtest/runtest.py +++ b/Lib/test/libregrtest/runtest.py @@ -1,7 +1,6 @@ import faulthandler import importlib import io -import json import os import sys import time @@ -22,38 +21,6 @@ CHILD_ERROR = -5 # error in a child process -def run_test_in_subprocess(testname, ns): - """Run the given test in a subprocess with --slaveargs. - - ns is the option Namespace parsed from command-line arguments. regrtest - is invoked in a subprocess with the --slaveargs argument; when the - subprocess exits, its return code, stdout and stderr are returned as a - 3-tuple. - """ - from subprocess import Popen, PIPE - base_cmd = ([sys.executable] + support.args_from_interpreter_flags() + - ['-X', 'faulthandler', '-m', 'test.regrtest']) - - slaveargs = ( - (testname, ns.verbose, ns.quiet), - dict(huntrleaks=ns.huntrleaks, - use_resources=ns.use_resources, - output_on_failure=ns.verbose3, - timeout=ns.timeout, failfast=ns.failfast, - match_tests=ns.match_tests)) - # Running the child from the same working directory as regrtest's original - # invocation ensures that TEMPDIR for the child is the same when - # sysconfig.is_python_build() is true. See issue 15300. - popen = Popen(base_cmd + ['--slaveargs', json.dumps(slaveargs)], - stdout=PIPE, stderr=PIPE, - universal_newlines=True, - close_fds=(os.name != 'nt'), - cwd=support.SAVEDCWD) - stdout, stderr = popen.communicate() - retcode = popen.wait() - return retcode, stdout, stderr - - # small set of tests to determine if we have a basically functioning interpreter # (i.e. if any of these fail, then anything else is likely to follow) STDTESTS = [ diff --git a/Lib/test/libregrtest/runtest_mp.py b/Lib/test/libregrtest/runtest_mp.py new file mode 100644 --- /dev/null +++ b/Lib/test/libregrtest/runtest_mp.py @@ -0,0 +1,158 @@ +import json +import os +import re +import sys +import traceback +import unittest +from queue import Queue +from test import support +try: + import threading +except ImportError: + print("Multiprocess option requires thread support") + sys.exit(2) + +from test.libregrtest.runtest import runtest, INTERRUPTED, CHILD_ERROR + + +debug_output_pat = re.compile(r"\[\d+ refs, \d+ blocks\]$") + + +def run_tests_in_subprocess(testname, ns): + """Run the given test in a subprocess with --slaveargs. + + ns is the option Namespace parsed from command-line arguments. regrtest + is invoked in a subprocess with the --slaveargs argument; when the + subprocess exits, its return code, stdout and stderr are returned as a + 3-tuple. + """ + from subprocess import Popen, PIPE + base_cmd = ([sys.executable] + support.args_from_interpreter_flags() + + ['-X', 'faulthandler', '-m', 'test.regrtest']) + + slaveargs = ( + (testname, ns.verbose, ns.quiet), + dict(huntrleaks=ns.huntrleaks, + use_resources=ns.use_resources, + output_on_failure=ns.verbose3, + timeout=ns.timeout, failfast=ns.failfast, + match_tests=ns.match_tests)) + # Running the child from the same working directory as regrtest's original + # invocation ensures that TEMPDIR for the child is the same when + # sysconfig.is_python_build() is true. See issue 15300. + popen = Popen(base_cmd + ['--slaveargs', json.dumps(slaveargs)], + stdout=PIPE, stderr=PIPE, + universal_newlines=True, + close_fds=(os.name != 'nt'), + cwd=support.SAVEDCWD) + stdout, stderr = popen.communicate() + retcode = popen.wait() + return retcode, stdout, stderr + + +def run_tests_slave(slaveargs): + args, kwargs = json.loads(slaveargs) + if kwargs.get('huntrleaks'): + unittest.BaseTestSuite._cleanup = False + try: + result = runtest(*args, **kwargs) + except KeyboardInterrupt: + result = INTERRUPTED, '' + except BaseException as e: + traceback.print_exc() + result = CHILD_ERROR, str(e) + sys.stdout.flush() + print() # Force a newline (just in case) + print(json.dumps(result)) + sys.exit(0) + + +# We do not use a generator so multiple threads can call next(). +class MultiprocessIterator: + + """A thread-safe iterator over tests for multiprocess mode.""" + + def __init__(self, tests): + self.interrupted = False + self.lock = threading.Lock() + self.tests = tests + + def __iter__(self): + return self + + def __next__(self): + with self.lock: + if self.interrupted: + raise StopIteration('tests interrupted') + return next(self.tests) + + +class MultiprocessThread(threading.Thread): + def __init__(self, pending, output, ns): + super().__init__() + self.pending = pending + self.output = output + self.ns = ns + + def run(self): + # A worker thread. + try: + while True: + try: + test = next(self.pending) + except StopIteration: + self.output.put((None, None, None, None)) + return + retcode, stdout, stderr = run_tests_in_subprocess(test, self.ns) + # Strip last refcount output line if it exists, since it + # comes from the shutdown of the interpreter in the subcommand. + stderr = debug_output_pat.sub("", stderr) + stdout, _, result = stdout.strip().rpartition("\n") + if retcode != 0: + result = (CHILD_ERROR, "Exit code %s" % retcode) + self.output.put((test, stdout.rstrip(), stderr.rstrip(), result)) + return + if not result: + self.output.put((None, None, None, None)) + return + result = json.loads(result) + self.output.put((test, stdout.rstrip(), stderr.rstrip(), result)) + except BaseException: + self.output.put((None, None, None, None)) + raise + + +def run_tests_multiprocess(regrtest): + output = Queue() + pending = MultiprocessIterator(regrtest.tests) + + workers = [MultiprocessThread(pending, output, regrtest.ns) + for i in range(regrtest.ns.use_mp)] + for worker in workers: + worker.start() + finished = 0 + test_index = 1 + try: + while finished < regrtest.ns.use_mp: + test, stdout, stderr, result = output.get() + if test is None: + finished += 1 + continue + regrtest.accumulate_result(test, result) + regrtest.display_progress(test_index, test) + if stdout: + print(stdout) + if stderr: + print(stderr, file=sys.stderr) + sys.stdout.flush() + sys.stderr.flush() + if result[0] == INTERRUPTED: + raise KeyboardInterrupt + if result[0] == CHILD_ERROR: + raise Exception("Child error on {}: {}".format(test, result[1])) + test_index += 1 + except KeyboardInterrupt: + regrtest.interrupted = True + pending.interrupted = True + for worker in workers: + worker.join() -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 29 23:37:49 2015 From: python-checkins at python.org (victor.stinner) Date: Tue, 29 Sep 2015 21:37:49 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2325220=3A_regrtest?= =?utf-8?q?_setups_Python_after_parsing_command_line_options?= Message-ID: <20150929213749.115474.47145@psf.io> https://hg.python.org/cpython/rev/45c43b9d50c5 changeset: 98418:45c43b9d50c5 user: Victor Stinner date: Tue Sep 29 23:37:14 2015 +0200 summary: Issue #25220: regrtest setups Python after parsing command line options files: Lib/test/libregrtest/main.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Lib/test/libregrtest/main.py b/Lib/test/libregrtest/main.py --- a/Lib/test/libregrtest/main.py +++ b/Lib/test/libregrtest/main.py @@ -431,8 +431,8 @@ os.system("leaks %d" % os.getpid()) def main(self, tests=None, **kwargs): + self.ns = _parse_args(sys.argv[1:], **kwargs) setup_python() - self.ns = _parse_args(sys.argv[1:], **kwargs) self.setup_regrtest() if self.ns.wait: input("Press any key to continue...") -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 29 23:37:53 2015 From: python-checkins at python.org (victor.stinner) Date: Tue, 29 Sep 2015 21:37:53 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2325220=3A_Enhance_?= =?utf-8?q?regrtest_--coverage?= Message-ID: <20150929213749.94131.10726@psf.io> https://hg.python.org/cpython/rev/281ab7954d7c changeset: 98417:281ab7954d7c user: Victor Stinner date: Tue Sep 29 23:36:27 2015 +0200 summary: Issue #25220: Enhance regrtest --coverage Add a new Regrtest.run_test() method to ensure that --coverage pass the same options to the runtest() function. files: Lib/test/libregrtest/main.py | 28 +++++++++++++++-------- 1 files changed, 18 insertions(+), 10 deletions(-) diff --git a/Lib/test/libregrtest/main.py b/Lib/test/libregrtest/main.py --- a/Lib/test/libregrtest/main.py +++ b/Lib/test/libregrtest/main.py @@ -346,7 +346,18 @@ print(count(len(self.bad), 'test'), "failed again:") printlist(self.bad) - def _run_tests_sequential(self): + def run_test(self, test): + result = runtest(test, + self.ns.verbose, + self.ns.quiet, + self.ns.huntrleaks, + output_on_failure=self.ns.verbose3, + timeout=self.ns.timeout, + failfast=self.ns.failfast, + match_tests=self.ns.match_tests) + self.accumulate_result(test, result) + + def run_tests_sequential(self): save_modules = sys.modules.keys() for test_index, test in enumerate(self.tests, 1): @@ -354,19 +365,15 @@ if self.ns.trace: # If we're tracing code coverage, then we don't exit with status # if on a false return value from main. - cmd = 'runtest(test, self.ns.verbose, self.ns.quiet, timeout=self.ns.timeout)' + cmd = 'self.run_test(test)' self.tracer.runctx(cmd, globals=globals(), locals=vars()) else: try: - result = runtest(test, self.ns.verbose, self.ns.quiet, - self.ns.huntrleaks, - output_on_failure=self.ns.verbose3, - timeout=self.ns.timeout, failfast=self.ns.failfast, - match_tests=self.ns.match_tests) - self.accumulate_result(test, result) + self.run_test(test) except KeyboardInterrupt: self.interrupted = True break + if self.ns.findleaks: gc.collect() if gc.garbage: @@ -376,13 +383,14 @@ # them again self.found_garbage.extend(gc.garbage) del gc.garbage[:] + # Unload the newly imported modules (best effort finalization) for module in sys.modules.keys(): if module not in save_modules and module.startswith("test."): support.unload(module) def run_tests(self): - support.verbose = self.ns.verbose # Tell tests to be moderately quiet + support.verbose = self.ns.verbose # Tell tests to be moderately quiet support.use_resources = self.ns.use_resources if self.ns.forever: @@ -404,7 +412,7 @@ from test.libregrtest.runtest_mp import run_tests_multiprocess run_tests_multiprocess(self) else: - self._run_tests_sequential() + self.run_tests_sequential() def finalize(self): if self.next_single_filename: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 29 23:56:30 2015 From: python-checkins at python.org (victor.stinner) Date: Tue, 29 Sep 2015 21:56:30 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Don=27t_strip_refcount_in_?= =?utf-8?q?libregrtest/runtest=5Fmp=2Epy?= Message-ID: <20150929215630.94115.2696@psf.io> https://hg.python.org/cpython/rev/eddf85b65996 changeset: 98421:eddf85b65996 user: Victor Stinner date: Tue Sep 29 23:52:33 2015 +0200 summary: Don't strip refcount in libregrtest/runtest_mp.py Python doesn't display the refcount anymore by default. It only displays it when -X showrefcount command line option is used, which is not the case here. regrtest can be run with -X showrefcount, the option is not inherited by child processes. files: Lib/test/libregrtest/runtest_mp.py | 6 ------ 1 files changed, 0 insertions(+), 6 deletions(-) diff --git a/Lib/test/libregrtest/runtest_mp.py b/Lib/test/libregrtest/runtest_mp.py --- a/Lib/test/libregrtest/runtest_mp.py +++ b/Lib/test/libregrtest/runtest_mp.py @@ -15,9 +15,6 @@ from test.libregrtest.runtest import runtest, INTERRUPTED, CHILD_ERROR -debug_output_pat = re.compile(r"\[\d+ refs, \d+ blocks\]$") - - def run_tests_in_subprocess(testname, ns): """Run the given test in a subprocess with --slaveargs. @@ -105,9 +102,6 @@ return retcode, stdout, stderr = run_tests_in_subprocess(test, self.ns) - # Strip last refcount output line if it exists, since it - # comes from the shutdown of the interpreter in the subcommand. - stderr = debug_output_pat.sub("", stderr) stdout, _, result = stdout.strip().rpartition("\n") if retcode != 0: result = (CHILD_ERROR, "Exit code %s" % retcode) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 29 23:56:30 2015 From: python-checkins at python.org (victor.stinner) Date: Tue, 29 Sep 2015 21:56:30 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2325220=2C_libregrt?= =?utf-8?q?est=3A_Remove_unused_import?= Message-ID: <20150929215630.11704.33522@psf.io> https://hg.python.org/cpython/rev/2c53c8dcde3f changeset: 98420:2c53c8dcde3f user: Victor Stinner date: Tue Sep 29 23:50:19 2015 +0200 summary: Issue #25220, libregrtest: Remove unused import files: Lib/test/libregrtest/main.py | 4 ---- 1 files changed, 0 insertions(+), 4 deletions(-) diff --git a/Lib/test/libregrtest/main.py b/Lib/test/libregrtest/main.py --- a/Lib/test/libregrtest/main.py +++ b/Lib/test/libregrtest/main.py @@ -19,10 +19,6 @@ import gc except ImportError: gc = None -try: - import threading -except ImportError: - threading = None # When tests are run from the Python build directory, it is best practice -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 29 23:56:30 2015 From: python-checkins at python.org (victor.stinner) Date: Tue, 29 Sep 2015 21:56:30 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2325220=3A_truncate?= =?utf-8?q?_some_long_lines_in_libregrtest/*=2Epy?= Message-ID: <20150929215629.115050.16817@psf.io> https://hg.python.org/cpython/rev/e6b48bfd6d8e changeset: 98419:e6b48bfd6d8e user: Victor Stinner date: Tue Sep 29 23:43:33 2015 +0200 summary: Issue #25220: truncate some long lines in libregrtest/*.py files: Lib/test/libregrtest/main.py | 23 ++++++++++++----- Lib/test/libregrtest/runtest_mp.py | 12 ++++++--- 2 files changed, 24 insertions(+), 11 deletions(-) diff --git a/Lib/test/libregrtest/main.py b/Lib/test/libregrtest/main.py --- a/Lib/test/libregrtest/main.py +++ b/Lib/test/libregrtest/main.py @@ -63,7 +63,8 @@ # the packages to prevent later imports to fail when the CWD is different. for module in sys.modules.values(): if hasattr(module, '__path__'): - module.__path__ = [os.path.abspath(path) for path in module.__path__] + module.__path__ = [os.path.abspath(path) + for path in module.__path__] if hasattr(module, '__file__'): module.__file__ = os.path.abspath(module.__file__) @@ -159,8 +160,8 @@ if self.ns.quiet: return fmt = "[{1:{0}}{2}/{3}] {4}" if self.bad else "[{1:{0}}{2}] {4}" - print(fmt.format( - self.test_count_width, test_index, self.test_count, len(self.bad), test)) + print(fmt.format(self.test_count_width, test_index, + self.test_count, len(self.bad), test)) sys.stdout.flush() def setup_regrtest(self): @@ -252,7 +253,10 @@ self.ns.args = [] # For a partial run, we do not need to clutter the output. - if self.ns.verbose or self.ns.header or not (self.ns.quiet or self.ns.single or self.tests or self.ns.args): + if (self.ns.verbose + or self.ns.header + or not (self.ns.quiet or self.ns.single + or self.tests or self.ns.args)): # Print basic platform information print("==", platform.python_implementation(), *sys.version.split()) print("== ", platform.platform(aliased=True), @@ -283,7 +287,8 @@ try: del self.selected[:self.selected.index(self.ns.start)] except ValueError: - print("Couldn't find starting test (%s), using all tests" % self.ns.start) + print("Couldn't find starting test (%s), using all tests" + % self.ns.start) if self.ns.randomize: if self.ns.random_seed is None: @@ -297,12 +302,16 @@ # print a newline after ^C print() print("Test suite interrupted by signal SIGINT.") - omitted = set(self.selected) - set(self.good) - set(self.bad) - set(self.skipped) + executed = set(self.good) | set(self.bad) | set(self.skipped) + omitted = set(self.selected) - executed print(count(len(omitted), "test"), "omitted:") printlist(omitted) if self.good and not self.ns.quiet: - if not self.bad and not self.skipped and not self.interrupted and len(self.good) > 1: + if (not self.bad + and not self.skipped + and not self.interrupted + and len(self.good) > 1): print("All", end=' ') print(count(len(self.good), "test"), "OK.") diff --git a/Lib/test/libregrtest/runtest_mp.py b/Lib/test/libregrtest/runtest_mp.py --- a/Lib/test/libregrtest/runtest_mp.py +++ b/Lib/test/libregrtest/runtest_mp.py @@ -103,20 +103,23 @@ except StopIteration: self.output.put((None, None, None, None)) return - retcode, stdout, stderr = run_tests_in_subprocess(test, self.ns) + retcode, stdout, stderr = run_tests_in_subprocess(test, + self.ns) # Strip last refcount output line if it exists, since it # comes from the shutdown of the interpreter in the subcommand. stderr = debug_output_pat.sub("", stderr) stdout, _, result = stdout.strip().rpartition("\n") if retcode != 0: result = (CHILD_ERROR, "Exit code %s" % retcode) - self.output.put((test, stdout.rstrip(), stderr.rstrip(), result)) + self.output.put((test, stdout.rstrip(), stderr.rstrip(), + result)) return if not result: self.output.put((None, None, None, None)) return result = json.loads(result) - self.output.put((test, stdout.rstrip(), stderr.rstrip(), result)) + self.output.put((test, stdout.rstrip(), stderr.rstrip(), + result)) except BaseException: self.output.put((None, None, None, None)) raise @@ -149,7 +152,8 @@ if result[0] == INTERRUPTED: raise KeyboardInterrupt if result[0] == CHILD_ERROR: - raise Exception("Child error on {}: {}".format(test, result[1])) + msg = "Child error on {}: {}".format(test, result[1]) + raise Exception(msg) test_index += 1 except KeyboardInterrupt: regrtest.interrupted = True -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 30 00:41:21 2015 From: python-checkins at python.org (victor.stinner) Date: Tue, 29 Sep 2015 22:41:21 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2325220=3A_Enhance_?= =?utf-8?q?regrtest_-jN?= Message-ID: <20150929224121.81625.72298@psf.io> https://hg.python.org/cpython/rev/8bd9422ef41e changeset: 98422:8bd9422ef41e user: Victor Stinner date: Wed Sep 30 00:33:29 2015 +0200 summary: Issue #25220: Enhance regrtest -jN Running the Python test suite with -jN now: - Display the duration of tests which took longer than 30 seconds - Display the tests currently running since at least 30 seconds - Display the tests we are waiting for when the test suite is interrupted Clenaup also run_test_in_subprocess() code. files: Lib/test/libregrtest/runtest_mp.py | 125 ++++++++++++---- 1 files changed, 89 insertions(+), 36 deletions(-) diff --git a/Lib/test/libregrtest/runtest_mp.py b/Lib/test/libregrtest/runtest_mp.py --- a/Lib/test/libregrtest/runtest_mp.py +++ b/Lib/test/libregrtest/runtest_mp.py @@ -1,7 +1,7 @@ import json import os -import re import sys +import time import traceback import unittest from queue import Queue @@ -15,7 +15,12 @@ from test.libregrtest.runtest import runtest, INTERRUPTED, CHILD_ERROR -def run_tests_in_subprocess(testname, ns): +# Minimum duration of a test to display its duration or to mention that +# the test is running in background +PROGRESS_MIN_TIME = 30.0 # seconds + + +def run_test_in_subprocess(testname, ns): """Run the given test in a subprocess with --slaveargs. ns is the option Namespace parsed from command-line arguments. regrtest @@ -24,26 +29,33 @@ 3-tuple. """ from subprocess import Popen, PIPE - base_cmd = ([sys.executable] + support.args_from_interpreter_flags() + - ['-X', 'faulthandler', '-m', 'test.regrtest']) - slaveargs = ( - (testname, ns.verbose, ns.quiet), - dict(huntrleaks=ns.huntrleaks, - use_resources=ns.use_resources, - output_on_failure=ns.verbose3, - timeout=ns.timeout, failfast=ns.failfast, - match_tests=ns.match_tests)) + args = (testname, ns.verbose, ns.quiet) + kwargs = dict(huntrleaks=ns.huntrleaks, + use_resources=ns.use_resources, + output_on_failure=ns.verbose3, + timeout=ns.timeout, + failfast=ns.failfast, + match_tests=ns.match_tests) + slaveargs = (args, kwargs) + slaveargs = json.dumps(slaveargs) + + cmd = [sys.executable, *support.args_from_interpreter_flags(), + '-X', 'faulthandler', + '-m', 'test.regrtest', + '--slaveargs', slaveargs] + # Running the child from the same working directory as regrtest's original # invocation ensures that TEMPDIR for the child is the same when # sysconfig.is_python_build() is true. See issue 15300. - popen = Popen(base_cmd + ['--slaveargs', json.dumps(slaveargs)], + popen = Popen(cmd, stdout=PIPE, stderr=PIPE, universal_newlines=True, close_fds=(os.name != 'nt'), cwd=support.SAVEDCWD) - stdout, stderr = popen.communicate() - retcode = popen.wait() + with popen: + stdout, stderr = popen.communicate() + retcode = popen.wait() return retcode, stdout, stderr @@ -90,30 +102,45 @@ self.pending = pending self.output = output self.ns = ns + self.current_test = None + self.start_time = None + + def _runtest(self): + try: + test = next(self.pending) + except StopIteration: + self.output.put((None, None, None, None)) + return True + + try: + self.start_time = time.monotonic() + self.current_test = test + + retcode, stdout, stderr = run_test_in_subprocess(test, self.ns) + finally: + self.current_test = None + + stdout, _, result = stdout.strip().rpartition("\n") + if retcode != 0: + result = (CHILD_ERROR, "Exit code %s" % retcode) + self.output.put((test, stdout.rstrip(), stderr.rstrip(), + result)) + return True + + if not result: + self.output.put((None, None, None, None)) + return True + + result = json.loads(result) + self.output.put((test, stdout.rstrip(), stderr.rstrip(), + result)) + return False def run(self): - # A worker thread. try: - while True: - try: - test = next(self.pending) - except StopIteration: - self.output.put((None, None, None, None)) - return - retcode, stdout, stderr = run_tests_in_subprocess(test, - self.ns) - stdout, _, result = stdout.strip().rpartition("\n") - if retcode != 0: - result = (CHILD_ERROR, "Exit code %s" % retcode) - self.output.put((test, stdout.rstrip(), stderr.rstrip(), - result)) - return - if not result: - self.output.put((None, None, None, None)) - return - result = json.loads(result) - self.output.put((test, stdout.rstrip(), stderr.rstrip(), - result)) + stop = False + while not stop: + stop = self._runtest() except BaseException: self.output.put((None, None, None, None)) raise @@ -136,13 +163,33 @@ finished += 1 continue regrtest.accumulate_result(test, result) - regrtest.display_progress(test_index, test) + + # Display progress + text = test + ok, test_time = result + if (ok not in (CHILD_ERROR, INTERRUPTED) + and test_time >= PROGRESS_MIN_TIME): + text += ' (%.0f sec)' % test_time + running = [] + for worker in workers: + current_test = worker.current_test + if not current_test: + continue + dt = time.monotonic() - worker.start_time + if dt >= PROGRESS_MIN_TIME: + running.append('%s (%.0f sec)' % (current_test, dt)) + if running: + text += ' -- running: %s' % ', '.join(running) + regrtest.display_progress(test_index, text) + + # Copy stdout and stderr from the child process if stdout: print(stdout) if stderr: print(stderr, file=sys.stderr) sys.stdout.flush() sys.stderr.flush() + if result[0] == INTERRUPTED: raise KeyboardInterrupt if result[0] == CHILD_ERROR: @@ -152,5 +199,11 @@ except KeyboardInterrupt: regrtest.interrupted = True pending.interrupted = True + print() + + running = [worker.current_test for worker in workers] + running = list(filter(bool, running)) + if running: + print("Waiting for %s" % ', '.join(running)) for worker in workers: worker.join() -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 30 02:05:10 2015 From: python-checkins at python.org (victor.stinner) Date: Wed, 30 Sep 2015 00:05:10 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2325274=3A_Workarou?= =?utf-8?q?nd_test=5Fsys_crash_just_to_keep_buildbots_usable?= Message-ID: <20150930000510.3668.55282@psf.io> https://hg.python.org/cpython/rev/00c1cd1f0131 changeset: 98428:00c1cd1f0131 user: Victor Stinner date: Wed Sep 30 02:02:49 2015 +0200 summary: Issue #25274: Workaround test_sys crash just to keep buildbots usable files: Lib/test/test_sys.py | 5 ++++- 1 files changed, 4 insertions(+), 1 deletions(-) diff --git a/Lib/test/test_sys.py b/Lib/test/test_sys.py --- a/Lib/test/test_sys.py +++ b/Lib/test/test_sys.py @@ -208,7 +208,10 @@ def f(): f() try: - for i in (50, 1000): + # FIXME: workaround crash for the issue #25274 + # FIXME: until the crash is fixed + #for i in (50, 1000): + for i in (150, 1000): # Issue #5392: stack overflow after hitting recursion limit twice sys.setrecursionlimit(i) self.assertRaises(RecursionError, f) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 30 02:05:10 2015 From: python-checkins at python.org (victor.stinner) Date: Wed, 30 Sep 2015 00:05:10 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2325220=2C_libregrt?= =?utf-8?q?est=3A_Move_setup=5Fpython=28=29_to_a_new_submodule?= Message-ID: <20150930000510.3664.6281@psf.io> https://hg.python.org/cpython/rev/b7d27c3c9e65 changeset: 98425:b7d27c3c9e65 user: Victor Stinner date: Wed Sep 30 01:13:53 2015 +0200 summary: Issue #25220, libregrtest: Move setup_python() to a new submodule files: Lib/test/libregrtest/main.py | 125 ++------------------- Lib/test/libregrtest/setup.py | 108 +++++++++++++++++++ 2 files changed, 123 insertions(+), 110 deletions(-) diff --git a/Lib/test/libregrtest/main.py b/Lib/test/libregrtest/main.py --- a/Lib/test/libregrtest/main.py +++ b/Lib/test/libregrtest/main.py @@ -1,19 +1,16 @@ -import faulthandler import os import platform import random import re -import signal import sys import sysconfig import tempfile import textwrap -import unittest from test.libregrtest.runtest import ( findtests, runtest, STDTESTS, NOTTESTS, PASSED, FAILED, ENV_CHANGED, SKIPPED, RESOURCE_DENIED) -from test.libregrtest.refleak import warm_caches from test.libregrtest.cmdline import _parse_args +from test.libregrtest.setup import setup_python from test import support try: import gc @@ -31,88 +28,6 @@ TEMPDIR = os.path.abspath(TEMPDIR) -def setup_python(ns): - # Display the Python traceback on fatal errors (e.g. segfault) - faulthandler.enable(all_threads=True) - - # Display the Python traceback on SIGALRM or SIGUSR1 signal - signals = [] - if hasattr(signal, 'SIGALRM'): - signals.append(signal.SIGALRM) - if hasattr(signal, 'SIGUSR1'): - signals.append(signal.SIGUSR1) - for signum in signals: - faulthandler.register(signum, chain=True) - - replace_stdout() - support.record_original_stdout(sys.stdout) - - # Some times __path__ and __file__ are not absolute (e.g. while running from - # Lib/) and, if we change the CWD to run the tests in a temporary dir, some - # imports might fail. This affects only the modules imported before os.chdir(). - # These modules are searched first in sys.path[0] (so '' -- the CWD) and if - # they are found in the CWD their __file__ and __path__ will be relative (this - # happens before the chdir). All the modules imported after the chdir, are - # not found in the CWD, and since the other paths in sys.path[1:] are absolute - # (site.py absolutize them), the __file__ and __path__ will be absolute too. - # Therefore it is necessary to absolutize manually the __file__ and __path__ of - # the packages to prevent later imports to fail when the CWD is different. - for module in sys.modules.values(): - if hasattr(module, '__path__'): - module.__path__ = [os.path.abspath(path) - for path in module.__path__] - if hasattr(module, '__file__'): - module.__file__ = os.path.abspath(module.__file__) - - # MacOSX (a.k.a. Darwin) has a default stack size that is too small - # for deeply recursive regular expressions. We see this as crashes in - # the Python test suite when running test_re.py and test_sre.py. The - # fix is to set the stack limit to 2048. - # This approach may also be useful for other Unixy platforms that - # suffer from small default stack limits. - if sys.platform == 'darwin': - try: - import resource - except ImportError: - pass - else: - soft, hard = resource.getrlimit(resource.RLIMIT_STACK) - newsoft = min(hard, max(soft, 1024*2048)) - resource.setrlimit(resource.RLIMIT_STACK, (newsoft, hard)) - - if ns.huntrleaks: - unittest.BaseTestSuite._cleanup = False - - # Avoid false positives due to various caches - # filling slowly with random data: - warm_caches() - - if ns.memlimit is not None: - support.set_memlimit(ns.memlimit) - - if ns.threshold is not None: - if gc is not None: - gc.set_threshold(ns.threshold) - else: - print('No GC available, ignore --threshold.') - - if ns.nowindows: - import msvcrt - msvcrt.SetErrorMode(msvcrt.SEM_FAILCRITICALERRORS| - msvcrt.SEM_NOALIGNMENTFAULTEXCEPT| - msvcrt.SEM_NOGPFAULTERRORBOX| - msvcrt.SEM_NOOPENFILEERRORBOX) - try: - msvcrt.CrtSetReportMode - except AttributeError: - # release build - pass - else: - for m in [msvcrt.CRT_WARN, msvcrt.CRT_ERROR, msvcrt.CRT_ASSERT]: - msvcrt.CrtSetReportMode(m, msvcrt.CRTDBG_MODE_FILE) - msvcrt.CrtSetReportFile(m, msvcrt.CRTDBG_FILE_STDERR) - - class Regrtest: """Execute a test suite. @@ -192,8 +107,14 @@ self.test_count, len(self.bad), test), flush=True) - def setup_regrtest(self): - if self.ns.findleaks: + def parse_args(self, kwargs): + ns = _parse_args(sys.argv[1:], **kwargs) + + if ns.threshold is not None and gc is None: + print('No GC available, ignore --threshold.') + ns.threshold = None + + if ns.findleaks: if gc is not None: # Uncomment the line below to report garbage that is not # freeable by reference counting alone. By default only @@ -202,10 +123,12 @@ #gc.set_debug(gc.DEBUG_SAVEALL) else: print('No GC available, disabling --findleaks') - self.ns.findleaks = False + ns.findleaks = False # Strip .py extensions. - removepy(self.ns.args) + removepy(ns.args) + + return ns def find_tests(self, tests): self.tests = tests @@ -434,9 +357,9 @@ os.system("leaks %d" % os.getpid()) def main(self, tests=None, **kwargs): - self.ns = _parse_args(sys.argv[1:], **kwargs) + self.ns = self.parse_args(kwargs) + setup_python(self.ns) - self.setup_regrtest() if self.ns.slaveargs is not None: from test.libregrtest.runtest_mp import run_tests_slave @@ -452,24 +375,6 @@ sys.exit(len(self.bad) > 0 or self.interrupted) -def replace_stdout(): - """Set stdout encoder error handler to backslashreplace (as stderr error - handler) to avoid UnicodeEncodeError when printing a traceback""" - import atexit - - stdout = sys.stdout - sys.stdout = open(stdout.fileno(), 'w', - encoding=stdout.encoding, - errors="backslashreplace", - closefd=False, - newline='\n') - - def restore_stdout(): - sys.stdout.close() - sys.stdout = stdout - atexit.register(restore_stdout) - - def removepy(names): if not names: return diff --git a/Lib/test/libregrtest/setup.py b/Lib/test/libregrtest/setup.py new file mode 100644 --- /dev/null +++ b/Lib/test/libregrtest/setup.py @@ -0,0 +1,108 @@ +import atexit +import faulthandler +import os +import signal +import sys +import unittest +from test import support +try: + import gc +except ImportError: + gc = None + +from test.libregrtest.refleak import warm_caches + + +def setup_python(ns): + # Display the Python traceback on fatal errors (e.g. segfault) + faulthandler.enable(all_threads=True) + + # Display the Python traceback on SIGALRM or SIGUSR1 signal + signals = [] + if hasattr(signal, 'SIGALRM'): + signals.append(signal.SIGALRM) + if hasattr(signal, 'SIGUSR1'): + signals.append(signal.SIGUSR1) + for signum in signals: + faulthandler.register(signum, chain=True) + + replace_stdout() + support.record_original_stdout(sys.stdout) + + # Some times __path__ and __file__ are not absolute (e.g. while running from + # Lib/) and, if we change the CWD to run the tests in a temporary dir, some + # imports might fail. This affects only the modules imported before os.chdir(). + # These modules are searched first in sys.path[0] (so '' -- the CWD) and if + # they are found in the CWD their __file__ and __path__ will be relative (this + # happens before the chdir). All the modules imported after the chdir, are + # not found in the CWD, and since the other paths in sys.path[1:] are absolute + # (site.py absolutize them), the __file__ and __path__ will be absolute too. + # Therefore it is necessary to absolutize manually the __file__ and __path__ of + # the packages to prevent later imports to fail when the CWD is different. + for module in sys.modules.values(): + if hasattr(module, '__path__'): + module.__path__ = [os.path.abspath(path) + for path in module.__path__] + if hasattr(module, '__file__'): + module.__file__ = os.path.abspath(module.__file__) + + # MacOSX (a.k.a. Darwin) has a default stack size that is too small + # for deeply recursive regular expressions. We see this as crashes in + # the Python test suite when running test_re.py and test_sre.py. The + # fix is to set the stack limit to 2048. + # This approach may also be useful for other Unixy platforms that + # suffer from small default stack limits. + if sys.platform == 'darwin': + try: + import resource + except ImportError: + pass + else: + soft, hard = resource.getrlimit(resource.RLIMIT_STACK) + newsoft = min(hard, max(soft, 1024*2048)) + resource.setrlimit(resource.RLIMIT_STACK, (newsoft, hard)) + + if ns.huntrleaks: + unittest.BaseTestSuite._cleanup = False + + # Avoid false positives due to various caches + # filling slowly with random data: + warm_caches() + + if ns.memlimit is not None: + support.set_memlimit(ns.memlimit) + + if ns.threshold is not None: + gc.set_threshold(ns.threshold) + + if ns.nowindows: + import msvcrt + msvcrt.SetErrorMode(msvcrt.SEM_FAILCRITICALERRORS| + msvcrt.SEM_NOALIGNMENTFAULTEXCEPT| + msvcrt.SEM_NOGPFAULTERRORBOX| + msvcrt.SEM_NOOPENFILEERRORBOX) + try: + msvcrt.CrtSetReportMode + except AttributeError: + # release build + pass + else: + for m in [msvcrt.CRT_WARN, msvcrt.CRT_ERROR, msvcrt.CRT_ASSERT]: + msvcrt.CrtSetReportMode(m, msvcrt.CRTDBG_MODE_FILE) + msvcrt.CrtSetReportFile(m, msvcrt.CRTDBG_FILE_STDERR) + + +def replace_stdout(): + """Set stdout encoder error handler to backslashreplace (as stderr error + handler) to avoid UnicodeEncodeError when printing a traceback""" + stdout = sys.stdout + sys.stdout = open(stdout.fileno(), 'w', + encoding=stdout.encoding, + errors="backslashreplace", + closefd=False, + newline='\n') + + def restore_stdout(): + sys.stdout.close() + sys.stdout = stdout + atexit.register(restore_stdout) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 30 02:05:11 2015 From: python-checkins at python.org (victor.stinner) Date: Wed, 30 Sep 2015 00:05:11 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2325220=2C_libregrt?= =?utf-8?q?est=3A_Cleanup_setup_code?= Message-ID: <20150930000510.11714.2821@psf.io> https://hg.python.org/cpython/rev/e2ed6e9163d5 changeset: 98424:e2ed6e9163d5 user: Victor Stinner date: Wed Sep 30 00:59:35 2015 +0200 summary: Issue #25220, libregrtest: Cleanup setup code files: Lib/test/libregrtest/main.py | 96 ++++++++++++----------- 1 files changed, 49 insertions(+), 47 deletions(-) diff --git a/Lib/test/libregrtest/main.py b/Lib/test/libregrtest/main.py --- a/Lib/test/libregrtest/main.py +++ b/Lib/test/libregrtest/main.py @@ -31,7 +31,7 @@ TEMPDIR = os.path.abspath(TEMPDIR) -def setup_python(): +def setup_python(ns): # Display the Python traceback on fatal errors (e.g. segfault) faulthandler.enable(all_threads=True) @@ -80,6 +80,38 @@ newsoft = min(hard, max(soft, 1024*2048)) resource.setrlimit(resource.RLIMIT_STACK, (newsoft, hard)) + if ns.huntrleaks: + unittest.BaseTestSuite._cleanup = False + + # Avoid false positives due to various caches + # filling slowly with random data: + warm_caches() + + if ns.memlimit is not None: + support.set_memlimit(ns.memlimit) + + if ns.threshold is not None: + if gc is not None: + gc.set_threshold(ns.threshold) + else: + print('No GC available, ignore --threshold.') + + if ns.nowindows: + import msvcrt + msvcrt.SetErrorMode(msvcrt.SEM_FAILCRITICALERRORS| + msvcrt.SEM_NOALIGNMENTFAULTEXCEPT| + msvcrt.SEM_NOGPFAULTERRORBOX| + msvcrt.SEM_NOOPENFILEERRORBOX) + try: + msvcrt.CrtSetReportMode + except AttributeError: + # release build + pass + else: + for m in [msvcrt.CRT_WARN, msvcrt.CRT_ERROR, msvcrt.CRT_ASSERT]: + msvcrt.CrtSetReportMode(m, msvcrt.CRTDBG_MODE_FILE) + msvcrt.CrtSetReportFile(m, msvcrt.CRTDBG_FILE_STDERR) + class Regrtest: """Execute a test suite. @@ -161,36 +193,6 @@ flush=True) def setup_regrtest(self): - if self.ns.huntrleaks: - # Avoid false positives due to various caches - # filling slowly with random data: - warm_caches() - - if self.ns.memlimit is not None: - support.set_memlimit(self.ns.memlimit) - - if self.ns.threshold is not None: - if gc is not None: - gc.set_threshold(self.ns.threshold) - else: - print('No GC available, ignore --threshold.') - - if self.ns.nowindows: - import msvcrt - msvcrt.SetErrorMode(msvcrt.SEM_FAILCRITICALERRORS| - msvcrt.SEM_NOALIGNMENTFAULTEXCEPT| - msvcrt.SEM_NOGPFAULTERRORBOX| - msvcrt.SEM_NOOPENFILEERRORBOX) - try: - msvcrt.CrtSetReportMode - except AttributeError: - # release build - pass - else: - for m in [msvcrt.CRT_WARN, msvcrt.CRT_ERROR, msvcrt.CRT_ASSERT]: - msvcrt.CrtSetReportMode(m, msvcrt.CRTDBG_MODE_FILE) - msvcrt.CrtSetReportFile(m, msvcrt.CRTDBG_FILE_STDERR) - if self.ns.findleaks: if gc is not None: # Uncomment the line below to report garbage that is not @@ -202,19 +204,9 @@ print('No GC available, disabling --findleaks') self.ns.findleaks = False - if self.ns.huntrleaks: - unittest.BaseTestSuite._cleanup = False - # Strip .py extensions. removepy(self.ns.args) - if self.ns.trace: - import trace - self.tracer = trace.Trace(ignoredirs=[sys.base_prefix, - sys.base_exec_prefix, - tempfile.gettempdir()], - trace=False, count=True) - def find_tests(self, tests): self.tests = tests @@ -278,7 +270,7 @@ except IndexError: pass - # Remove all the self.selected tests that precede start if it's set. + # Remove all the selected tests that precede start if it's set. if self.ns.start: try: del self.selected[:self.selected.index(self.ns.start)] @@ -362,11 +354,18 @@ self.accumulate_result(test, result) def run_tests_sequential(self): + if self.ns.trace: + import trace + self.tracer = trace.Trace(ignoredirs=[sys.base_prefix, + sys.base_exec_prefix, + tempfile.gettempdir()], + trace=False, count=True) + save_modules = sys.modules.keys() for test_index, test in enumerate(self.tests, 1): self.display_progress(test_index, test) - if self.ns.trace: + if self.tracer: # If we're tracing code coverage, then we don't exit with status # if on a false return value from main. cmd = 'self.run_test(test)' @@ -426,7 +425,7 @@ else: os.unlink(self.next_single_filename) - if self.ns.trace: + if self.tracer: r = self.tracer.results() r.write_results(show_missing=True, summary=True, coverdir=self.ns.coverdir) @@ -436,15 +435,18 @@ def main(self, tests=None, **kwargs): self.ns = _parse_args(sys.argv[1:], **kwargs) - setup_python() + setup_python(self.ns) self.setup_regrtest() - if self.ns.wait: - input("Press any key to continue...") + if self.ns.slaveargs is not None: from test.libregrtest.runtest_mp import run_tests_slave run_tests_slave(self.ns.slaveargs) + if self.ns.wait: + input("Press any key to continue...") + self.find_tests(tests) self.run_tests() + self.display_result() self.finalize() sys.exit(len(self.bad) > 0 or self.interrupted) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 30 02:05:13 2015 From: python-checkins at python.org (victor.stinner) Date: Wed, 30 Sep 2015 00:05:13 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2325220=3A_Use_prin?= =?utf-8?q?t=28flush=3DTrue=29_in_libregrtest?= Message-ID: <20150930000510.82664.62213@psf.io> https://hg.python.org/cpython/rev/7e07c51d8fc6 changeset: 98423:7e07c51d8fc6 user: Victor Stinner date: Wed Sep 30 00:48:27 2015 +0200 summary: Issue #25220: Use print(flush=True) in libregrtest files: Lib/test/libregrtest/main.py | 7 +++---- Lib/test/libregrtest/refleak.py | 10 ++++------ Lib/test/libregrtest/runtest.py | 14 +++++--------- Lib/test/libregrtest/runtest_mp.py | 9 +++------ 4 files changed, 15 insertions(+), 25 deletions(-) diff --git a/Lib/test/libregrtest/main.py b/Lib/test/libregrtest/main.py --- a/Lib/test/libregrtest/main.py +++ b/Lib/test/libregrtest/main.py @@ -157,8 +157,8 @@ return fmt = "[{1:{0}}{2}/{3}] {4}" if self.bad else "[{1:{0}}{2}] {4}" print(fmt.format(self.test_count_width, test_index, - self.test_count, len(self.bad), test)) - sys.stdout.flush() + self.test_count, len(self.bad), test), + flush=True) def setup_regrtest(self): if self.ns.huntrleaks: @@ -333,8 +333,7 @@ if self.ns.verbose2 and self.bad: print("Re-running failed tests in verbose mode") for test in self.bad[:]: - print("Re-running test %r in verbose mode" % test) - sys.stdout.flush() + print("Re-running test %r in verbose mode" % test, flush=True) try: self.ns.verbose = True ok = runtest(test, True, self.ns.quiet, self.ns.huntrleaks, diff --git a/Lib/test/libregrtest/refleak.py b/Lib/test/libregrtest/refleak.py --- a/Lib/test/libregrtest/refleak.py +++ b/Lib/test/libregrtest/refleak.py @@ -44,13 +44,12 @@ alloc_deltas = [0] * repcount print("beginning", repcount, "repetitions", file=sys.stderr) - print(("1234567890"*(repcount//10 + 1))[:repcount], file=sys.stderr) - sys.stderr.flush() + print(("1234567890"*(repcount//10 + 1))[:repcount], file=sys.stderr, + flush=True) for i in range(repcount): indirect_test() alloc_after, rc_after = dash_R_cleanup(fs, ps, pic, zdc, abcs) - sys.stderr.write('.') - sys.stderr.flush() + print('.', end='', flush=True) if i >= nwarmup: rc_deltas[i] = rc_after - rc_before alloc_deltas[i] = alloc_after - alloc_before @@ -74,8 +73,7 @@ if checker(deltas): msg = '%s leaked %s %s, sum=%s' % ( test, deltas[nwarmup:], item_name, sum(deltas)) - print(msg, file=sys.stderr) - sys.stderr.flush() + print(msg, file=sys.stderr, flush=True) with open(fname, "a") as refrep: print(msg, file=refrep) refrep.flush() diff --git a/Lib/test/libregrtest/runtest.py b/Lib/test/libregrtest/runtest.py --- a/Lib/test/libregrtest/runtest.py +++ b/Lib/test/libregrtest/runtest.py @@ -161,27 +161,23 @@ test_time = time.time() - start_time except support.ResourceDenied as msg: if not quiet: - print(test, "skipped --", msg) - sys.stdout.flush() + print(test, "skipped --", msg, flush=True) return RESOURCE_DENIED, test_time except unittest.SkipTest as msg: if not quiet: - print(test, "skipped --", msg) - sys.stdout.flush() + print(test, "skipped --", msg, flush=True) return SKIPPED, test_time except KeyboardInterrupt: raise except support.TestFailed as msg: if display_failure: - print("test", test, "failed --", msg, file=sys.stderr) + print("test", test, "failed --", msg, file=sys.stderr, flush=True) else: - print("test", test, "failed", file=sys.stderr) - sys.stderr.flush() + print("test", test, "failed", file=sys.stderr, flush=True) return FAILED, test_time except: msg = traceback.format_exc() - print("test", test, "crashed --", msg, file=sys.stderr) - sys.stderr.flush() + print("test", test, "crashed --", msg, file=sys.stderr, flush=True) return FAILED, test_time else: if refleak: diff --git a/Lib/test/libregrtest/runtest_mp.py b/Lib/test/libregrtest/runtest_mp.py --- a/Lib/test/libregrtest/runtest_mp.py +++ b/Lib/test/libregrtest/runtest_mp.py @@ -70,9 +70,8 @@ except BaseException as e: traceback.print_exc() result = CHILD_ERROR, str(e) - sys.stdout.flush() print() # Force a newline (just in case) - print(json.dumps(result)) + print(json.dumps(result), flush=True) sys.exit(0) @@ -184,11 +183,9 @@ # Copy stdout and stderr from the child process if stdout: - print(stdout) + print(stdout, flush=True) if stderr: - print(stderr, file=sys.stderr) - sys.stdout.flush() - sys.stderr.flush() + print(stderr, file=sys.stderr, flush=True) if result[0] == INTERRUPTED: raise KeyboardInterrupt -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 30 02:05:13 2015 From: python-checkins at python.org (victor.stinner) Date: Wed, 30 Sep 2015 00:05:13 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2325220=2C_libregrt?= =?utf-8?q?est=3A_Call_setup=5Fpython=28ns=29_in_the_slaves?= Message-ID: <20150930000510.81621.52064@psf.io> https://hg.python.org/cpython/rev/b48ae54ed5be changeset: 98427:b48ae54ed5be user: Victor Stinner date: Wed Sep 30 01:39:28 2015 +0200 summary: Issue #25220, libregrtest: Call setup_python(ns) in the slaves Slaves (child processes running tests for regrtest -jN) now inherit --memlimit/-M, --threshold/-t and --nowindows/-n options. * -M, -t and -n are now supported with -jN * Factorize code to run tests. * run_test_in_subprocess() now pass the whole "ns" namespace to the child process. files: Lib/test/libregrtest/cmdline.py | 2 -- Lib/test/libregrtest/main.py | 5 +++-- Lib/test/libregrtest/runtest_mp.py | 8 ++++---- Lib/test/test_regrtest.py | 1 - 4 files changed, 7 insertions(+), 9 deletions(-) diff --git a/Lib/test/libregrtest/cmdline.py b/Lib/test/libregrtest/cmdline.py --- a/Lib/test/libregrtest/cmdline.py +++ b/Lib/test/libregrtest/cmdline.py @@ -295,8 +295,6 @@ parser.error("-T and -j don't go together!") if ns.use_mp and ns.findleaks: parser.error("-l and -j don't go together!") - if ns.use_mp and ns.memlimit: - parser.error("-M and -j don't go together!") if ns.failfast and not (ns.verbose or ns.verbose3): parser.error("-G/--failfast needs either -v or -W") diff --git a/Lib/test/libregrtest/main.py b/Lib/test/libregrtest/main.py --- a/Lib/test/libregrtest/main.py +++ b/Lib/test/libregrtest/main.py @@ -354,14 +354,15 @@ def main(self, tests=None, **kwargs): self.ns = self.parse_args(kwargs) - setup_python(self.ns) - if self.ns.slaveargs is not None: from test.libregrtest.runtest_mp import run_tests_slave run_tests_slave(self.ns.slaveargs) + if self.ns.wait: input("Press any key to continue...") + setup_python(self.ns) + self.find_tests(tests) self.run_tests() diff --git a/Lib/test/libregrtest/runtest_mp.py b/Lib/test/libregrtest/runtest_mp.py --- a/Lib/test/libregrtest/runtest_mp.py +++ b/Lib/test/libregrtest/runtest_mp.py @@ -13,7 +13,8 @@ print("Multiprocess option requires thread support") sys.exit(2) -from test.libregrtest.runtest import runtest, INTERRUPTED, CHILD_ERROR +from test.libregrtest.runtest import runtest_ns, INTERRUPTED, CHILD_ERROR +from test.libregrtest.setup import setup_python # Minimum duration of a test to display its duration or to mention that @@ -58,11 +59,10 @@ ns_dict, testname = json.loads(slaveargs) ns = types.SimpleNamespace(**ns_dict) - if ns.huntrleaks: - unittest.BaseTestSuite._cleanup = False + setup_python(ns) try: - result = runtest_ns(testname, ns.verbose, ns.quiet, ns, + result = runtest_ns(testname, ns.verbose, ns, use_resources=ns.use_resources, output_on_failure=ns.verbose3, failfast=ns.failfast, diff --git a/Lib/test/test_regrtest.py b/Lib/test/test_regrtest.py --- a/Lib/test/test_regrtest.py +++ b/Lib/test/test_regrtest.py @@ -214,7 +214,6 @@ self.checkError([opt, 'foo'], 'invalid int value') self.checkError([opt, '2', '-T'], "don't go together") self.checkError([opt, '2', '-l'], "don't go together") - self.checkError([opt, '2', '-M', '4G'], "don't go together") def test_coverage(self): for opt in '-T', '--coverage': -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 30 02:05:12 2015 From: python-checkins at python.org (victor.stinner) Date: Wed, 30 Sep 2015 00:05:12 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2325220=2C_libregrt?= =?utf-8?q?est=3A_Add_runtest=5Fns=28=29_function?= Message-ID: <20150930000510.81647.3526@psf.io> https://hg.python.org/cpython/rev/ff012c1b8068 changeset: 98426:ff012c1b8068 user: Victor Stinner date: Wed Sep 30 01:32:39 2015 +0200 summary: Issue #25220, libregrtest: Add runtest_ns() function * Factorize code to run tests. * run_test_in_subprocess() now pass the whole "ns" namespace to the child process. files: Lib/test/libregrtest/main.py | 17 ++++-------- Lib/test/libregrtest/runtest.py | 7 +++++ Lib/test/libregrtest/runtest_mp.py | 24 +++++++++-------- 3 files changed, 26 insertions(+), 22 deletions(-) diff --git a/Lib/test/libregrtest/main.py b/Lib/test/libregrtest/main.py --- a/Lib/test/libregrtest/main.py +++ b/Lib/test/libregrtest/main.py @@ -7,7 +7,7 @@ import tempfile import textwrap from test.libregrtest.runtest import ( - findtests, runtest, + findtests, runtest_ns, STDTESTS, NOTTESTS, PASSED, FAILED, ENV_CHANGED, SKIPPED, RESOURCE_DENIED) from test.libregrtest.cmdline import _parse_args from test.libregrtest.setup import setup_python @@ -251,8 +251,7 @@ print("Re-running test %r in verbose mode" % test, flush=True) try: self.ns.verbose = True - ok = runtest(test, True, self.ns.quiet, self.ns.huntrleaks, - timeout=self.ns.timeout) + ok = runtest_ns(test, True, self.ns) except KeyboardInterrupt: # print a newline separate from the ^C print() @@ -266,14 +265,10 @@ printlist(self.bad) def run_test(self, test): - result = runtest(test, - self.ns.verbose, - self.ns.quiet, - self.ns.huntrleaks, - output_on_failure=self.ns.verbose3, - timeout=self.ns.timeout, - failfast=self.ns.failfast, - match_tests=self.ns.match_tests) + result = runtest_ns(test, self.ns.verbose, self.ns, + output_on_failure=self.ns.verbose3, + failfast=self.ns.failfast, + match_tests=self.ns.match_tests) self.accumulate_result(test, result) def run_tests_sequential(self): diff --git a/Lib/test/libregrtest/runtest.py b/Lib/test/libregrtest/runtest.py --- a/Lib/test/libregrtest/runtest.py +++ b/Lib/test/libregrtest/runtest.py @@ -53,6 +53,13 @@ return stdtests + sorted(tests) +def runtest_ns(test, verbose, ns, **kw): + return runtest(test, verbose, ns.quiet, + huntrleaks=ns.huntrleaks, + timeout=ns.timeout, + **kw) + + def runtest(test, verbose, quiet, huntrleaks=False, use_resources=None, output_on_failure=False, failfast=False, match_tests=None, diff --git a/Lib/test/libregrtest/runtest_mp.py b/Lib/test/libregrtest/runtest_mp.py --- a/Lib/test/libregrtest/runtest_mp.py +++ b/Lib/test/libregrtest/runtest_mp.py @@ -3,6 +3,7 @@ import sys import time import traceback +import types import unittest from queue import Queue from test import support @@ -30,14 +31,8 @@ """ from subprocess import Popen, PIPE - args = (testname, ns.verbose, ns.quiet) - kwargs = dict(huntrleaks=ns.huntrleaks, - use_resources=ns.use_resources, - output_on_failure=ns.verbose3, - timeout=ns.timeout, - failfast=ns.failfast, - match_tests=ns.match_tests) - slaveargs = (args, kwargs) + ns_dict = vars(ns) + slaveargs = (ns_dict, testname) slaveargs = json.dumps(slaveargs) cmd = [sys.executable, *support.args_from_interpreter_flags(), @@ -60,11 +55,18 @@ def run_tests_slave(slaveargs): - args, kwargs = json.loads(slaveargs) - if kwargs.get('huntrleaks'): + ns_dict, testname = json.loads(slaveargs) + ns = types.SimpleNamespace(**ns_dict) + + if ns.huntrleaks: unittest.BaseTestSuite._cleanup = False + try: - result = runtest(*args, **kwargs) + result = runtest_ns(testname, ns.verbose, ns.quiet, ns, + use_resources=ns.use_resources, + output_on_failure=ns.verbose3, + failfast=ns.failfast, + match_tests=ns.match_tests) except KeyboardInterrupt: result = INTERRUPTED, '' except BaseException as e: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 30 02:40:30 2015 From: python-checkins at python.org (victor.stinner) Date: Wed, 30 Sep 2015 00:40:30 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2325220=2C_libregrt?= =?utf-8?q?est=3A_Cleanup?= Message-ID: <20150930004030.98362.5838@psf.io> https://hg.python.org/cpython/rev/8e985fb19724 changeset: 98431:8e985fb19724 user: Victor Stinner date: Wed Sep 30 02:39:22 2015 +0200 summary: Issue #25220, libregrtest: Cleanup No need to support.verbose in Regrtest.run_tests(), it's always set in runtest(). files: Lib/test/libregrtest/main.py | 17 ++++++++--------- Lib/test/libregrtest/runtest_mp.py | 1 + 2 files changed, 9 insertions(+), 9 deletions(-) diff --git a/Lib/test/libregrtest/main.py b/Lib/test/libregrtest/main.py --- a/Lib/test/libregrtest/main.py +++ b/Lib/test/libregrtest/main.py @@ -310,17 +310,16 @@ if module not in save_modules and module.startswith("test."): support.unload(module) + def _test_forever(self, tests): + while True: + for test in tests: + yield test + if self.bad: + return + def run_tests(self): - support.verbose = self.ns.verbose # Tell tests to be moderately quiet - if self.ns.forever: - def test_forever(tests): - while True: - for test in tests: - yield test - if self.bad: - return - self.tests = test_forever(list(self.selected)) + self.tests = _test_forever(list(self.selected)) self.test_count = '' self.test_count_width = 3 else: diff --git a/Lib/test/libregrtest/runtest_mp.py b/Lib/test/libregrtest/runtest_mp.py --- a/Lib/test/libregrtest/runtest_mp.py +++ b/Lib/test/libregrtest/runtest_mp.py @@ -68,6 +68,7 @@ except BaseException as e: traceback.print_exc() result = CHILD_ERROR, str(e) + print() # Force a newline (just in case) print(json.dumps(result), flush=True) sys.exit(0) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 30 02:40:30 2015 From: python-checkins at python.org (victor.stinner) Date: Wed, 30 Sep 2015 00:40:30 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2325220=2C_libregrt?= =?utf-8?q?est=3A_Set_support=2Euse=5Fresources_in_setup=5Ftests=28=29?= Message-ID: <20150930004030.94129.19857@psf.io> https://hg.python.org/cpython/rev/6ec81abb8e6a changeset: 98429:6ec81abb8e6a user: Victor Stinner date: Wed Sep 30 02:17:28 2015 +0200 summary: Issue #25220, libregrtest: Set support.use_resources in setup_tests() * Rename setup_python() to setup_tests() * Remove use_resources parameter of runtest() files: Lib/test/libregrtest/main.py | 5 ++--- Lib/test/libregrtest/runtest.py | 5 +---- Lib/test/libregrtest/runtest_mp.py | 5 ++--- Lib/test/libregrtest/setup.py | 4 +++- 4 files changed, 8 insertions(+), 11 deletions(-) diff --git a/Lib/test/libregrtest/main.py b/Lib/test/libregrtest/main.py --- a/Lib/test/libregrtest/main.py +++ b/Lib/test/libregrtest/main.py @@ -10,7 +10,7 @@ findtests, runtest_ns, STDTESTS, NOTTESTS, PASSED, FAILED, ENV_CHANGED, SKIPPED, RESOURCE_DENIED) from test.libregrtest.cmdline import _parse_args -from test.libregrtest.setup import setup_python +from test.libregrtest.setup import setup_tests from test import support try: import gc @@ -312,7 +312,6 @@ def run_tests(self): support.verbose = self.ns.verbose # Tell tests to be moderately quiet - support.use_resources = self.ns.use_resources if self.ns.forever: def test_forever(tests): @@ -361,7 +360,7 @@ if self.ns.wait: input("Press any key to continue...") - setup_python(self.ns) + setup_tests(self.ns) self.find_tests(tests) self.run_tests() diff --git a/Lib/test/libregrtest/runtest.py b/Lib/test/libregrtest/runtest.py --- a/Lib/test/libregrtest/runtest.py +++ b/Lib/test/libregrtest/runtest.py @@ -61,7 +61,7 @@ def runtest(test, verbose, quiet, - huntrleaks=False, use_resources=None, + huntrleaks=False, output_on_failure=False, failfast=False, match_tests=None, timeout=None): """Run a single test. @@ -71,7 +71,6 @@ quiet -- if true, don't print 'skipped' messages (probably redundant) huntrleaks -- run multiple times to test for leaks; requires a debug build; a triple corresponding to -R's three arguments - use_resources -- list of extra resources to use output_on_failure -- if true, display test output on failure timeout -- dump the traceback and exit if a test takes more than timeout seconds @@ -86,8 +85,6 @@ PASSED test passed """ - if use_resources is not None: - support.use_resources = use_resources use_timeout = (timeout is not None) if use_timeout: faulthandler.dump_traceback_later(timeout, exit=True) diff --git a/Lib/test/libregrtest/runtest_mp.py b/Lib/test/libregrtest/runtest_mp.py --- a/Lib/test/libregrtest/runtest_mp.py +++ b/Lib/test/libregrtest/runtest_mp.py @@ -14,7 +14,7 @@ sys.exit(2) from test.libregrtest.runtest import runtest_ns, INTERRUPTED, CHILD_ERROR -from test.libregrtest.setup import setup_python +from test.libregrtest.setup import setup_tests # Minimum duration of a test to display its duration or to mention that @@ -59,11 +59,10 @@ ns_dict, testname = json.loads(slaveargs) ns = types.SimpleNamespace(**ns_dict) - setup_python(ns) + setup_tests(ns) try: result = runtest_ns(testname, ns.verbose, ns, - use_resources=ns.use_resources, output_on_failure=ns.verbose3, failfast=ns.failfast, match_tests=ns.match_tests) diff --git a/Lib/test/libregrtest/setup.py b/Lib/test/libregrtest/setup.py --- a/Lib/test/libregrtest/setup.py +++ b/Lib/test/libregrtest/setup.py @@ -13,7 +13,7 @@ from test.libregrtest.refleak import warm_caches -def setup_python(ns): +def setup_tests(ns): # Display the Python traceback on fatal errors (e.g. segfault) faulthandler.enable(all_threads=True) @@ -91,6 +91,8 @@ msvcrt.CrtSetReportMode(m, msvcrt.CRTDBG_MODE_FILE) msvcrt.CrtSetReportFile(m, msvcrt.CRTDBG_FILE_STDERR) + support.use_resources = ns.use_resources + def replace_stdout(): """Set stdout encoder error handler to backslashreplace (as stderr error -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 30 02:40:31 2015 From: python-checkins at python.org (victor.stinner) Date: Wed, 30 Sep 2015 00:40:31 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2325220=2C_libregrt?= =?utf-8?q?est=3A_Pass_directly_ns_to_runtest=28=29?= Message-ID: <20150930004030.3652.16266@psf.io> https://hg.python.org/cpython/rev/e765b6c16e1c changeset: 98430:e765b6c16e1c user: Victor Stinner date: Wed Sep 30 02:32:11 2015 +0200 summary: Issue #25220, libregrtest: Pass directly ns to runtest() * Remove runtest_ns(): pass directly ns to runtest(). * Create also Regrtest.rerun_failed_tests() method. * Inline again Regrtest.run_test(): it's no more justified to have a method files: Lib/test/libregrtest/main.py | 62 +++++++++-------- Lib/test/libregrtest/runtest.py | 20 ++--- Lib/test/libregrtest/runtest_mp.py | 7 +- 3 files changed, 44 insertions(+), 45 deletions(-) diff --git a/Lib/test/libregrtest/main.py b/Lib/test/libregrtest/main.py --- a/Lib/test/libregrtest/main.py +++ b/Lib/test/libregrtest/main.py @@ -7,7 +7,7 @@ import tempfile import textwrap from test.libregrtest.runtest import ( - findtests, runtest_ns, + findtests, runtest, STDTESTS, NOTTESTS, PASSED, FAILED, ENV_CHANGED, SKIPPED, RESOURCE_DENIED) from test.libregrtest.cmdline import _parse_args from test.libregrtest.setup import setup_tests @@ -208,6 +208,30 @@ print("Using random seed", self.ns.random_seed) random.shuffle(self.selected) + def rerun_failed_tests(self): + self.ns.verbose = True + self.ns.failfast = False + self.ns.verbose3 = False + self.ns.match_tests = None + + print("Re-running failed tests in verbose mode") + for test in self.bad[:]: + print("Re-running test %r in verbose mode" % test, flush=True) + try: + self.ns.verbose = True + ok = runtest(self.ns, test) + except KeyboardInterrupt: + # print a newline separate from the ^C + print() + break + else: + if ok[0] in {PASSED, ENV_CHANGED, SKIPPED, RESOURCE_DENIED}: + self.bad.remove(test) + else: + if self.bad: + print(count(len(self.bad), 'test'), "failed again:") + printlist(self.bad) + def display_result(self): if self.interrupted: # print a newline after ^C @@ -245,32 +269,6 @@ print(count(len(self.skipped), "test"), "skipped:") printlist(self.skipped) - if self.ns.verbose2 and self.bad: - print("Re-running failed tests in verbose mode") - for test in self.bad[:]: - print("Re-running test %r in verbose mode" % test, flush=True) - try: - self.ns.verbose = True - ok = runtest_ns(test, True, self.ns) - except KeyboardInterrupt: - # print a newline separate from the ^C - print() - break - else: - if ok[0] in {PASSED, ENV_CHANGED, SKIPPED, RESOURCE_DENIED}: - self.bad.remove(test) - else: - if self.bad: - print(count(len(self.bad), 'test'), "failed again:") - printlist(self.bad) - - def run_test(self, test): - result = runtest_ns(test, self.ns.verbose, self.ns, - output_on_failure=self.ns.verbose3, - failfast=self.ns.failfast, - match_tests=self.ns.match_tests) - self.accumulate_result(test, result) - def run_tests_sequential(self): if self.ns.trace: import trace @@ -286,11 +284,13 @@ if self.tracer: # If we're tracing code coverage, then we don't exit with status # if on a false return value from main. - cmd = 'self.run_test(test)' + cmd = ('result = runtest(self.ns, test); ' + 'self.accumulate_result(test, result)') self.tracer.runctx(cmd, globals=globals(), locals=vars()) else: try: - self.run_test(test) + result = runtest(self.ns, test) + self.accumulate_result(test, result) except KeyboardInterrupt: self.interrupted = True break @@ -366,6 +366,10 @@ self.run_tests() self.display_result() + + if self.ns.verbose2 and self.bad: + self.rerun_failed_tests() + self.finalize() sys.exit(len(self.bad) > 0 or self.interrupted) diff --git a/Lib/test/libregrtest/runtest.py b/Lib/test/libregrtest/runtest.py --- a/Lib/test/libregrtest/runtest.py +++ b/Lib/test/libregrtest/runtest.py @@ -53,17 +53,7 @@ return stdtests + sorted(tests) -def runtest_ns(test, verbose, ns, **kw): - return runtest(test, verbose, ns.quiet, - huntrleaks=ns.huntrleaks, - timeout=ns.timeout, - **kw) - - -def runtest(test, verbose, quiet, - huntrleaks=False, - output_on_failure=False, failfast=False, match_tests=None, - timeout=None): +def runtest(ns, test): """Run a single test. test -- the name of the test @@ -85,6 +75,14 @@ PASSED test passed """ + verbose = ns.verbose + quiet = ns.quiet + huntrleaks = ns.huntrleaks + output_on_failure = ns.verbose3 + failfast = ns.failfast + match_tests = ns.match_tests + timeout = ns.timeout + use_timeout = (timeout is not None) if use_timeout: faulthandler.dump_traceback_later(timeout, exit=True) diff --git a/Lib/test/libregrtest/runtest_mp.py b/Lib/test/libregrtest/runtest_mp.py --- a/Lib/test/libregrtest/runtest_mp.py +++ b/Lib/test/libregrtest/runtest_mp.py @@ -13,7 +13,7 @@ print("Multiprocess option requires thread support") sys.exit(2) -from test.libregrtest.runtest import runtest_ns, INTERRUPTED, CHILD_ERROR +from test.libregrtest.runtest import runtest, INTERRUPTED, CHILD_ERROR from test.libregrtest.setup import setup_tests @@ -62,10 +62,7 @@ setup_tests(ns) try: - result = runtest_ns(testname, ns.verbose, ns, - output_on_failure=ns.verbose3, - failfast=ns.failfast, - match_tests=ns.match_tests) + result = runtest(ns, testname) except KeyboardInterrupt: result = INTERRUPTED, '' except BaseException as e: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 30 03:05:57 2015 From: python-checkins at python.org (victor.stinner) Date: Wed, 30 Sep 2015 01:05:57 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2325220=2C_libregrt?= =?utf-8?q?est=3A_more_verbose_output_for_-jN?= Message-ID: <20150930010557.11690.6714@psf.io> https://hg.python.org/cpython/rev/ed0734966dad changeset: 98432:ed0734966dad user: Victor Stinner date: Wed Sep 30 03:05:43 2015 +0200 summary: Issue #25220, libregrtest: more verbose output for -jN When the -jN command line option is used, display tests running since at least 30 seconds every minute. files: Lib/test/libregrtest/runtest_mp.py | 39 ++++++++++++----- 1 files changed, 28 insertions(+), 11 deletions(-) diff --git a/Lib/test/libregrtest/runtest_mp.py b/Lib/test/libregrtest/runtest_mp.py --- a/Lib/test/libregrtest/runtest_mp.py +++ b/Lib/test/libregrtest/runtest_mp.py @@ -1,11 +1,11 @@ import json import os +import queue import sys import time import traceback import types import unittest -from queue import Queue from test import support try: import threading @@ -21,6 +21,9 @@ # the test is running in background PROGRESS_MIN_TIME = 30.0 # seconds +# Display the running tests if nothing happened last N seconds +PROGRESS_UPDATE = 60.0 # seconds + def run_test_in_subprocess(testname, ns): """Run the given test in a subprocess with --slaveargs. @@ -145,18 +148,39 @@ def run_tests_multiprocess(regrtest): - output = Queue() + output = queue.Queue() pending = MultiprocessIterator(regrtest.tests) workers = [MultiprocessThread(pending, output, regrtest.ns) for i in range(regrtest.ns.use_mp)] for worker in workers: worker.start() + + def get_running(workers): + running = [] + for worker in workers: + current_test = worker.current_test + if not current_test: + continue + dt = time.monotonic() - worker.start_time + if dt >= PROGRESS_MIN_TIME: + running.append('%s (%.0f sec)' % (current_test, dt)) + return running + finished = 0 test_index = 1 + timeout = max(PROGRESS_UPDATE, PROGRESS_MIN_TIME) try: while finished < regrtest.ns.use_mp: - test, stdout, stderr, result = output.get() + try: + item = output.get(timeout=PROGRESS_UPDATE) + except queue.Empty: + running = get_running(workers) + if running: + print('running: %s' % ', '.join(running)) + continue + + test, stdout, stderr, result = item if test is None: finished += 1 continue @@ -168,14 +192,7 @@ if (ok not in (CHILD_ERROR, INTERRUPTED) and test_time >= PROGRESS_MIN_TIME): text += ' (%.0f sec)' % test_time - running = [] - for worker in workers: - current_test = worker.current_test - if not current_test: - continue - dt = time.monotonic() - worker.start_time - if dt >= PROGRESS_MIN_TIME: - running.append('%s (%.0f sec)' % (current_test, dt)) + running = get_running(workers) if running: text += ' -- running: %s' % ', '.join(running) regrtest.display_progress(test_index, text) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 30 07:45:11 2015 From: python-checkins at python.org (raymond.hettinger) Date: Wed, 30 Sep 2015 05:45:11 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Add_an_early-out_for_deque?= =?utf-8?b?X2NsZWFyKCk=?= Message-ID: <20150930054511.98358.76241@psf.io> https://hg.python.org/cpython/rev/91259f061cfb changeset: 98433:91259f061cfb user: Raymond Hettinger date: Tue Sep 29 22:45:05 2015 -0700 summary: Add an early-out for deque_clear() files: Modules/_collectionsmodule.c | 3 +++ 1 files changed, 3 insertions(+), 0 deletions(-) diff --git a/Modules/_collectionsmodule.c b/Modules/_collectionsmodule.c --- a/Modules/_collectionsmodule.c +++ b/Modules/_collectionsmodule.c @@ -595,6 +595,9 @@ Py_ssize_t n; PyObject *item; + if (Py_SIZE(deque) == 0) + return; + /* During the process of clearing a deque, decrefs can cause the deque to mutate. To avoid fatal confusion, we have to make the deque empty before clearing the blocks and never refer to -- Repository URL: https://hg.python.org/cpython From solipsis at pitrou.net Wed Sep 30 10:43:34 2015 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Wed, 30 Sep 2015 08:43:34 +0000 Subject: [Python-checkins] Daily reference leaks (ed0734966dad): sum=17880 Message-ID: <20150930084333.11700.49117@psf.io> results for ed0734966dad on branch "default" -------------------------------------------- test_asyncio leaked [0, 3, 0] memory blocks, sum=3 test_capi leaked [1598, 1598, 1598] references, sum=4794 test_capi leaked [387, 389, 389] memory blocks, sum=1165 test_functools leaked [0, 2, 2] memory blocks, sum=4 test_threading leaked [3196, 3196, 3196] references, sum=9588 test_threading leaked [774, 776, 776] memory blocks, sum=2326 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/psf-users/antoine/refleaks/refloguHbdEF', '--timeout', '7200'] From lp_benchmark_robot at intel.com Wed Sep 30 13:41:52 2015 From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com) Date: Wed, 30 Sep 2015 12:41:52 +0100 Subject: [Python-checkins] Benchmark Results for Python Default 2015-09-30 Message-ID: <7eee085b-e3c4-4017-b01e-1b591414890f@irsmsx151.ger.corp.intel.com> Results for project python_default-nightly, build date 2015-09-30 08:51:08 commit: 91259f061cfb347aeaf93daa98cd21477d555a8a revision date: 2015-09-30 05:45:05 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v3.4.3, with hash b4cbecbc0781e89a309d03b60a1f75f8499250e6 from 2015-02-25 12:15:33+00:00 ------------------------------------------------------------------------------------------ benchmark relative change since change since current rev with std_dev* last run v3.4.3 regrtest PGO ------------------------------------------------------------------------------------------ :-) django_v2 0.31717% 1.79726% 7.44575% 16.75906% :-| pybench 0.09855% 0.18706% -1.56845% 8.35832% :-( regex_v8 2.79408% -0.06455% -5.09787% 6.64279% :-| nbody 0.11134% -0.13852% -0.79860% 10.67541% :-| json_dump_v2 0.27503% 0.67736% -1.51425% 10.12527% :-| normal_startup 0.82980% 0.09625% 1.03598% 4.76778% ------------------------------------------------------------------------------------------ Note: Benchmark results are measured in seconds. * Relative Standard Deviation (Standard Deviation/Average) Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. From lp_benchmark_robot at intel.com Wed Sep 30 13:42:21 2015 From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com) Date: Wed, 30 Sep 2015 12:42:21 +0100 Subject: [Python-checkins] Benchmark Results for Python 2.7 2015-09-30 Message-ID: <6ea42d61-8d5d-4a42-a7bf-4f27476f4c7d@irsmsx151.ger.corp.intel.com> Results for project python_2.7-nightly, build date 2015-09-30 09:31:56 commit: 8274fc521e69cb6baf2932ffd3d81bef4a45b5e8 revision date: 2015-09-29 20:51:27 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v2.7.10, with hash 15c95b7d81dcf821daade360741e00714667653f from 2015-05-23 16:02:14+00:00 ------------------------------------------------------------------------------------------ benchmark relative change since change since current rev with std_dev* last run v2.7.10 regrtest PGO ------------------------------------------------------------------------------------------ :-) django_v2 0.37820% -0.19226% 4.02348% 9.24847% :-) pybench 0.18635% -0.18920% 6.61514% 7.24446% :-| regex_v8 1.06512% -0.76123% -1.89789% 8.19914% :-) nbody 0.11606% -0.51994% 8.62419% 3.90676% :-) json_dump_v2 0.28493% -0.97989% 3.44541% 12.26374% :-| normal_startup 1.57922% 0.15100% -1.28399% 2.33192% :-| ssbench 0.43283% 0.56654% 1.58202% 2.52947% ------------------------------------------------------------------------------------------ Note: Benchmark results for ssbench are measured in requests/second while all other are measured in seconds. * Relative Standard Deviation (Standard Deviation/Average) Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. From python-checkins at python.org Wed Sep 30 14:30:07 2015 From: python-checkins at python.org (victor.stinner) Date: Wed, 30 Sep 2015 12:30:07 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2325220=3A_Fix_=22-?= =?utf-8?q?m_test_--forever=22?= Message-ID: <20150930123005.98356.79789@psf.io> https://hg.python.org/cpython/rev/ec02ccffd1dc changeset: 98434:ec02ccffd1dc user: Victor Stinner date: Wed Sep 30 13:51:17 2015 +0200 summary: Issue #25220: Fix "-m test --forever" * Fix "-m test --forever": replace _test_forever() with self._test_forever() * Add unit test for --forever * Add unit test for a failing test * Fix also some pyflakes warnings in libregrtest files: Lib/test/libregrtest/main.py | 2 +- Lib/test/libregrtest/refleak.py | 6 +- Lib/test/libregrtest/runtest_mp.py | 3 +- Lib/test/test_regrtest.py | 129 ++++++++++++---- 4 files changed, 98 insertions(+), 42 deletions(-) diff --git a/Lib/test/libregrtest/main.py b/Lib/test/libregrtest/main.py --- a/Lib/test/libregrtest/main.py +++ b/Lib/test/libregrtest/main.py @@ -319,7 +319,7 @@ def run_tests(self): if self.ns.forever: - self.tests = _test_forever(list(self.selected)) + self.tests = self._test_forever(list(self.selected)) self.test_count = '' self.test_count_width = 3 else: diff --git a/Lib/test/libregrtest/refleak.py b/Lib/test/libregrtest/refleak.py --- a/Lib/test/libregrtest/refleak.py +++ b/Lib/test/libregrtest/refleak.py @@ -46,6 +46,8 @@ print("beginning", repcount, "repetitions", file=sys.stderr) print(("1234567890"*(repcount//10 + 1))[:repcount], file=sys.stderr, flush=True) + # initialize variables to make pyflakes quiet + rc_before = alloc_before = 0 for i in range(repcount): indirect_test() alloc_after, rc_after = dash_R_cleanup(fs, ps, pic, zdc, abcs) @@ -158,6 +160,6 @@ for i in range(256): s[i:i+1] # unicode cache - x = [chr(i) for i in range(256)] + [chr(i) for i in range(256)] # int cache - x = list(range(-5, 257)) + list(range(-5, 257)) diff --git a/Lib/test/libregrtest/runtest_mp.py b/Lib/test/libregrtest/runtest_mp.py --- a/Lib/test/libregrtest/runtest_mp.py +++ b/Lib/test/libregrtest/runtest_mp.py @@ -5,7 +5,6 @@ import time import traceback import types -import unittest from test import support try: import threading @@ -173,7 +172,7 @@ try: while finished < regrtest.ns.use_mp: try: - item = output.get(timeout=PROGRESS_UPDATE) + item = output.get(timeout=timeout) except queue.Empty: running = get_running(workers) if running: diff --git a/Lib/test/test_regrtest.py b/Lib/test/test_regrtest.py --- a/Lib/test/test_regrtest.py +++ b/Lib/test/test_regrtest.py @@ -330,33 +330,52 @@ self.assertRegex(output, regex) def parse_executed_tests(self, output): - parser = re.finditer(r'^\[[0-9]+/[0-9]+\] (%s)$' % self.TESTNAME_REGEX, - output, - re.MULTILINE) - return set(match.group(1) for match in parser) + regex = r'^\[ *[0-9]+(?:/ *[0-9]+)?\] (%s)$' % self.TESTNAME_REGEX + parser = re.finditer(regex, output, re.MULTILINE) + return list(match.group(1) for match in parser) - def check_executed_tests(self, output, tests, skipped=None): + def check_executed_tests(self, output, tests, skipped=(), failed=(), + randomize=False): if isinstance(tests, str): tests = [tests] + if isinstance(skipped, str): + skipped = [skipped] + if isinstance(failed, str): + failed = [failed] + ntest = len(tests) + nskipped = len(skipped) + nfailed = len(failed) + executed = self.parse_executed_tests(output) - self.assertEqual(executed, set(tests), output) - ntest = len(tests) + if randomize: + self.assertEqual(set(executed), set(tests), output) + else: + self.assertEqual(executed, tests, output) + + def plural(count): + return 's' if count != 1 else '' + + def list_regex(line_format, tests): + count = len(tests) + names = ' '.join(sorted(tests)) + regex = line_format % (count, plural(count)) + regex = r'%s:\n %s$' % (regex, names) + return regex + if skipped: - if isinstance(skipped, str): - skipped = [skipped] - nskipped = len(skipped) + regex = list_regex('%s test%s skipped', skipped) + self.check_line(output, regex) - plural = 's' if nskipped != 1 else '' - names = ' '.join(sorted(skipped)) - expected = (r'%s test%s skipped:\n %s$' - % (nskipped, plural, names)) - self.check_line(output, expected) + if failed: + regex = list_regex('%s test%s failed', failed) + self.check_line(output, regex) - ok = ntest - nskipped - if ok: - self.check_line(output, r'%s test OK\.$' % ok) - else: - self.check_line(output, r'All %s tests OK\.$' % ntest) + good = ntest - nskipped - nfailed + if good: + regex = r'%s test%s OK\.$' % (good, plural(good)) + if not skipped and not failed and good > 1: + regex = 'All %s' % regex + self.check_line(output, regex) def parse_random_seed(self, output): match = self.regex_search(r'Using random seed ([0-9]+)', output) @@ -364,24 +383,28 @@ self.assertTrue(0 <= randseed <= 10000000, randseed) return randseed - def run_command(self, args, input=None): + def run_command(self, args, input=None, exitcode=0): if not input: input = '' - try: - return subprocess.run(args, - check=True, universal_newlines=True, - input=input, - stdout=subprocess.PIPE, - stderr=subprocess.PIPE) - except subprocess.CalledProcessError as exc: - self.fail("%s\n" + proc = subprocess.run(args, + universal_newlines=True, + input=input, + stdout=subprocess.PIPE, + stderr=subprocess.PIPE) + if proc.returncode != exitcode: + self.fail("Command %s failed with exit code %s\n" "\n" "stdout:\n" + "---\n" "%s\n" + "---\n" "\n" "stderr:\n" + "---\n" "%s" - % (str(exc), exc.stdout, exc.stderr)) + "---\n" + % (str(args), proc.returncode, proc.stdout, proc.stderr)) + return proc def run_python(self, args, **kw): @@ -411,11 +434,11 @@ def check_output(self, output): self.parse_random_seed(output) - self.check_executed_tests(output, self.tests) + self.check_executed_tests(output, self.tests, randomize=True) def run_tests(self, args): - stdout = self.run_python(args) - self.check_output(stdout) + output = self.run_python(args) + self.check_output(output) def test_script_regrtest(self): # Lib/test/regrtest.py @@ -492,8 +515,24 @@ Test arguments of the Python test suite. """ - def run_tests(self, *args, input=None): - return self.run_python(['-m', 'test', *args], input=input) + def run_tests(self, *args, **kw): + return self.run_python(['-m', 'test', *args], **kw) + + def test_failing_test(self): + # test a failing test + code = textwrap.dedent(""" + import unittest + + class FailingTest(unittest.TestCase): + def test_failing(self): + self.fail("bug") + """) + test_ok = self.create_test() + test_failing = self.create_test(code=code) + tests = [test_ok, test_failing] + + output = self.run_tests(*tests, exitcode=1) + self.check_executed_tests(output, tests, failed=test_failing) def test_resources(self): # test -u command line option @@ -572,8 +611,7 @@ # test --coverage test = self.create_test() output = self.run_tests("--coverage", test) - executed = self.parse_executed_tests(output) - self.assertEqual(executed, {test}, output) + self.check_executed_tests(output, [test]) regex = ('lines +cov% +module +\(path\)\n' '(?: *[0-9]+ *[0-9]{1,2}% *[^ ]+ +\([^)]+\)+)+') self.check_line(output, regex) @@ -584,6 +622,23 @@ output = self.run_tests("--wait", test, input='key') self.check_line(output, 'Press any key to continue') + def test_forever(self): + # test --forever + code = textwrap.dedent(""" + import unittest + + class ForeverTester(unittest.TestCase): + RUN = 1 + + def test_run(self): + ForeverTester.RUN += 1 + if ForeverTester.RUN > 3: + self.fail("fail at the 3rd runs") + """) + test = self.create_test(code=code) + output = self.run_tests('--forever', test, exitcode=1) + self.check_executed_tests(output, [test]*3, failed=test) + if __name__ == '__main__': unittest.main() -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 30 15:03:41 2015 From: python-checkins at python.org (serhiy.storchaka) Date: Wed, 30 Sep 2015 13:03:41 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2325182=3A_The_stdprinter_=28used_as_sys=2Estderr?= =?utf-8?q?_before_the_io_module_is?= Message-ID: <20150930125138.115545.95198@psf.io> https://hg.python.org/cpython/rev/0b0945c8de36 changeset: 98437:0b0945c8de36 parent: 98434:ec02ccffd1dc parent: 98436:e8b6c6c433a4 user: Serhiy Storchaka date: Wed Sep 30 15:51:01 2015 +0300 summary: Issue #25182: The stdprinter (used as sys.stderr before the io module is imported at startup) now uses the backslashreplace error handler. files: Misc/NEWS | 3 +++ Objects/fileobject.c | 25 +++++++++++++++++++++---- 2 files changed, 24 insertions(+), 4 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -180,6 +180,9 @@ Core and Builtins ----------------- +- Issue #25182: The stdprinter (used as sys.stderr before the io module is + imported at startup) now uses the backslashreplace error handler. + - Issue #25131: Make the line number and column offset of set/dict literals and comprehensions correspond to the opening brace. diff --git a/Objects/fileobject.c b/Objects/fileobject.c --- a/Objects/fileobject.c +++ b/Objects/fileobject.c @@ -372,8 +372,11 @@ static PyObject * stdprinter_write(PyStdPrinter_Object *self, PyObject *args) { + PyObject *unicode; + PyObject *bytes = NULL; char *str; Py_ssize_t n; + int _errno; if (self->fd < 0) { /* fd might be invalid on Windows @@ -383,13 +386,27 @@ Py_RETURN_NONE; } - /* encode Unicode to UTF-8 */ - if (!PyArg_ParseTuple(args, "s", &str)) + if (!PyArg_ParseTuple(args, "U", &unicode)) return NULL; - n = _Py_write(self->fd, str, strlen(str)); + /* encode Unicode to UTF-8 */ + str = PyUnicode_AsUTF8AndSize(unicode, &n); + if (str == NULL) { + PyErr_Clear(); + bytes = _PyUnicode_AsUTF8String(unicode, "backslashreplace"); + if (bytes == NULL) + return NULL; + if (PyBytes_AsStringAndSize(bytes, &str, &n) < 0) { + Py_DECREF(bytes); + return NULL; + } + } + + n = _Py_write(self->fd, str, n); + _errno = errno; + Py_XDECREF(bytes); if (n == -1) { - if (errno == EAGAIN) { + if (_errno == EAGAIN) { PyErr_Clear(); Py_RETURN_NONE; } -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 30 15:03:40 2015 From: python-checkins at python.org (serhiy.storchaka) Date: Wed, 30 Sep 2015 13:03:40 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzI1MTgy?= =?utf-8?q?=3A_The_stdprinter_=28used_as_sys=2Estderr_before_the_io_module?= =?utf-8?q?_is?= Message-ID: <20150930125137.98366.25360@psf.io> https://hg.python.org/cpython/rev/6347b154dd67 changeset: 98435:6347b154dd67 branch: 3.4 parent: 98412:01c79072d671 user: Serhiy Storchaka date: Wed Sep 30 15:46:53 2015 +0300 summary: Issue #25182: The stdprinter (used as sys.stderr before the io module is imported at startup) now uses the backslashreplace error handler. files: Misc/NEWS | 3 +++ Objects/fileobject.c | 28 ++++++++++++++++++++++------ 2 files changed, 25 insertions(+), 6 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -10,6 +10,9 @@ Core and Builtins ----------------- +- Issue #25182: The stdprinter (used as sys.stderr before the io module is + imported at startup) now uses the backslashreplace error handler. + - Issue #24891: Fix a race condition at Python startup if the file descriptor of stdin (0), stdout (1) or stderr (2) is closed while Python is creating sys.stdin, sys.stdout and sys.stderr objects. These attributes are now set diff --git a/Objects/fileobject.c b/Objects/fileobject.c --- a/Objects/fileobject.c +++ b/Objects/fileobject.c @@ -372,8 +372,11 @@ static PyObject * stdprinter_write(PyStdPrinter_Object *self, PyObject *args) { - char *c; + PyObject *unicode; + PyObject *bytes = NULL; + char *str; Py_ssize_t n; + int _errno; if (self->fd < 0) { /* fd might be invalid on Windows @@ -383,24 +386,37 @@ Py_RETURN_NONE; } - if (!PyArg_ParseTuple(args, "s", &c)) { + if (!PyArg_ParseTuple(args, "U", &unicode)) return NULL; + + /* encode Unicode to UTF-8 */ + str = PyUnicode_AsUTF8AndSize(unicode, &n); + if (str == NULL) { + PyErr_Clear(); + bytes = _PyUnicode_AsUTF8String(unicode, "backslashreplace"); + if (bytes == NULL) + return NULL; + if (PyBytes_AsStringAndSize(bytes, &str, &n) < 0) { + Py_DECREF(bytes); + return NULL; + } } - n = strlen(c); Py_BEGIN_ALLOW_THREADS errno = 0; #ifdef MS_WINDOWS if (n > INT_MAX) n = INT_MAX; - n = write(self->fd, c, (int)n); + n = write(self->fd, str, (int)n); #else - n = write(self->fd, c, n); + n = write(self->fd, str, n); #endif + _errno = errno; Py_END_ALLOW_THREADS + Py_XDECREF(bytes); if (n < 0) { - if (errno == EAGAIN) + if (_errno == EAGAIN) Py_RETURN_NONE; PyErr_SetFromErrno(PyExc_IOError); return NULL; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 30 15:03:41 2015 From: python-checkins at python.org (serhiy.storchaka) Date: Wed, 30 Sep 2015 13:03:41 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Issue_=2325182=3A_The_stdprinter_=28used_as_sys=2Estderr_befor?= =?utf-8?q?e_the_io_module_is?= Message-ID: <20150930125137.82656.77553@psf.io> https://hg.python.org/cpython/rev/e8b6c6c433a4 changeset: 98436:e8b6c6c433a4 branch: 3.5 parent: 98413:73b6b88ac28a parent: 98435:6347b154dd67 user: Serhiy Storchaka date: Wed Sep 30 15:50:32 2015 +0300 summary: Issue #25182: The stdprinter (used as sys.stderr before the io module is imported at startup) now uses the backslashreplace error handler. files: Misc/NEWS | 3 +++ Objects/fileobject.c | 25 +++++++++++++++++++++---- 2 files changed, 24 insertions(+), 4 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -11,6 +11,9 @@ Core and Builtins ----------------- +- Issue #25182: The stdprinter (used as sys.stderr before the io module is + imported at startup) now uses the backslashreplace error handler. + - Issue #25131: Make the line number and column offset of set/dict literals and comprehensions correspond to the opening brace. diff --git a/Objects/fileobject.c b/Objects/fileobject.c --- a/Objects/fileobject.c +++ b/Objects/fileobject.c @@ -372,8 +372,11 @@ static PyObject * stdprinter_write(PyStdPrinter_Object *self, PyObject *args) { + PyObject *unicode; + PyObject *bytes = NULL; char *str; Py_ssize_t n; + int _errno; if (self->fd < 0) { /* fd might be invalid on Windows @@ -383,13 +386,27 @@ Py_RETURN_NONE; } - /* encode Unicode to UTF-8 */ - if (!PyArg_ParseTuple(args, "s", &str)) + if (!PyArg_ParseTuple(args, "U", &unicode)) return NULL; - n = _Py_write(self->fd, str, strlen(str)); + /* encode Unicode to UTF-8 */ + str = PyUnicode_AsUTF8AndSize(unicode, &n); + if (str == NULL) { + PyErr_Clear(); + bytes = _PyUnicode_AsUTF8String(unicode, "backslashreplace"); + if (bytes == NULL) + return NULL; + if (PyBytes_AsStringAndSize(bytes, &str, &n) < 0) { + Py_DECREF(bytes); + return NULL; + } + } + + n = _Py_write(self->fd, str, n); + _errno = errno; + Py_XDECREF(bytes); if (n == -1) { - if (errno == EAGAIN) { + if (_errno == EAGAIN) { PyErr_Clear(); Py_RETURN_NONE; } -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 30 15:04:25 2015 From: python-checkins at python.org (victor.stinner) Date: Wed, 30 Sep 2015 13:04:25 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_=28Merge_3=2E5=29_Issue_=2325182=3A_Fix_compilation_on_W?= =?utf-8?q?indows?= Message-ID: <20150930130424.31179.24100@psf.io> https://hg.python.org/cpython/rev/d1090d733d39 changeset: 98440:d1090d733d39 parent: 98437:0b0945c8de36 parent: 98439:0eb26a4d5ffa user: Victor Stinner date: Wed Sep 30 15:03:50 2015 +0200 summary: (Merge 3.5) Issue #25182: Fix compilation on Windows files: Objects/fileobject.c | 9 ++++++--- 1 files changed, 6 insertions(+), 3 deletions(-) diff --git a/Objects/fileobject.c b/Objects/fileobject.c --- a/Objects/fileobject.c +++ b/Objects/fileobject.c @@ -376,7 +376,7 @@ PyObject *bytes = NULL; char *str; Py_ssize_t n; - int _errno; + int err; if (self->fd < 0) { /* fd might be invalid on Windows @@ -403,10 +403,13 @@ } n = _Py_write(self->fd, str, n); - _errno = errno; + /* save errno, it can be modified indirectly by Py_XDECREF() */ + err = errno; + Py_XDECREF(bytes); + if (n == -1) { - if (_errno == EAGAIN) { + if (err == EAGAIN) { PyErr_Clear(); Py_RETURN_NONE; } -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 30 15:04:27 2015 From: python-checkins at python.org (victor.stinner) Date: Wed, 30 Sep 2015 13:04:27 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzI1MTgy?= =?utf-8?q?=3A_Fix_compilation_on_Windows?= Message-ID: <20150930130424.98370.88496@psf.io> https://hg.python.org/cpython/rev/2652c1798f7d changeset: 98438:2652c1798f7d branch: 3.4 parent: 98435:6347b154dd67 user: Victor Stinner date: Wed Sep 30 15:01:34 2015 +0200 summary: Issue #25182: Fix compilation on Windows Restore also errno value before calling PyErr_SetFromErrno(). files: Objects/fileobject.c | 9 ++++++--- 1 files changed, 6 insertions(+), 3 deletions(-) diff --git a/Objects/fileobject.c b/Objects/fileobject.c --- a/Objects/fileobject.c +++ b/Objects/fileobject.c @@ -376,7 +376,7 @@ PyObject *bytes = NULL; char *str; Py_ssize_t n; - int _errno; + int err; if (self->fd < 0) { /* fd might be invalid on Windows @@ -411,13 +411,16 @@ #else n = write(self->fd, str, n); #endif - _errno = errno; + /* save errno, it can be modified indirectly by Py_XDECREF() */ + err = errno; Py_END_ALLOW_THREADS + Py_XDECREF(bytes); if (n < 0) { - if (_errno == EAGAIN) + if (err == EAGAIN) Py_RETURN_NONE; + errno = err; PyErr_SetFromErrno(PyExc_IOError); return NULL; } -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 30 15:04:34 2015 From: python-checkins at python.org (victor.stinner) Date: Wed, 30 Sep 2015 13:04:34 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_=28Merge_3=2E4=29_Issue_=2325182=3A_Fix_compilation_on_Windows?= Message-ID: <20150930130424.81633.81121@psf.io> https://hg.python.org/cpython/rev/0eb26a4d5ffa changeset: 98439:0eb26a4d5ffa branch: 3.5 parent: 98436:e8b6c6c433a4 parent: 98438:2652c1798f7d user: Victor Stinner date: Wed Sep 30 15:03:31 2015 +0200 summary: (Merge 3.4) Issue #25182: Fix compilation on Windows files: Objects/fileobject.c | 9 ++++++--- 1 files changed, 6 insertions(+), 3 deletions(-) diff --git a/Objects/fileobject.c b/Objects/fileobject.c --- a/Objects/fileobject.c +++ b/Objects/fileobject.c @@ -376,7 +376,7 @@ PyObject *bytes = NULL; char *str; Py_ssize_t n; - int _errno; + int err; if (self->fd < 0) { /* fd might be invalid on Windows @@ -403,10 +403,13 @@ } n = _Py_write(self->fd, str, n); - _errno = errno; + /* save errno, it can be modified indirectly by Py_XDECREF() */ + err = errno; + Py_XDECREF(bytes); + if (n == -1) { - if (_errno == EAGAIN) { + if (err == EAGAIN) { PyErr_Clear(); Py_RETURN_NONE; } -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 30 22:07:57 2015 From: python-checkins at python.org (victor.stinner) Date: Wed, 30 Sep 2015 20:07:57 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2325171=3A_Fix_comp?= =?utf-8?q?ilation_issue_on_OpenBSD_in_random=2Ec?= Message-ID: <20150930200757.115050.87357@psf.io> https://hg.python.org/cpython/rev/e4ac5a899657 changeset: 98441:e4ac5a899657 user: Victor Stinner date: Wed Sep 30 22:06:51 2015 +0200 summary: Issue #25171: Fix compilation issue on OpenBSD in random.c Patch written by Remi Pointel. files: Python/random.c | 6 +++--- 1 files changed, 3 insertions(+), 3 deletions(-) diff --git a/Python/random.c b/Python/random.c --- a/Python/random.c +++ b/Python/random.c @@ -364,7 +364,7 @@ #ifdef MS_WINDOWS return win32_urandom((unsigned char *)buffer, size, 1); -#elif PY_GETENTROPY +#elif defined(PY_GETENTROPY) return py_getentropy(buffer, size, 0); #else return dev_urandom_python((char*)buffer, size); @@ -411,7 +411,7 @@ else { #ifdef MS_WINDOWS (void)win32_urandom(secret, secret_size, 0); -#elif PY_GETENTROPY +#elif defined(PY_GETENTROPY) (void)py_getentropy(secret, secret_size, 1); #else dev_urandom_noraise(secret, secret_size); @@ -427,7 +427,7 @@ CryptReleaseContext(hCryptProv, 0); hCryptProv = 0; } -#elif PY_GETENTROPY +#elif defined(PY_GETENTROPY) /* nothing to clean */ #else dev_urandom_close(); -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 30 22:33:01 2015 From: python-checkins at python.org (barry.warsaw) Date: Wed, 30 Sep 2015 20:33:01 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_Add_some_open_issues=2E?= Message-ID: <20150930203300.3652.69897@psf.io> https://hg.python.org/peps/rev/2fb211ea5cb1 changeset: 6104:2fb211ea5cb1 user: Barry Warsaw date: Wed Sep 30 16:32:56 2015 -0400 summary: Add some open issues. files: pep-0507.txt | 28 ++++++++++++++++++++++++++++ 1 files changed, 28 insertions(+), 0 deletions(-) diff --git a/pep-0507.txt b/pep-0507.txt --- a/pep-0507.txt +++ b/pep-0507.txt @@ -278,6 +278,34 @@ possible to submit and commit changes. +Open issues +=========== + +* What level of hosted support will GitLab offer? The PEP author has been in + contact with the GitLab CEO, with positive interest on their part. The + details of the hosting offer would have to be discussed. + +* What happens to Roundup and do we switch to the GitLab issue tracker? + Currently, this PEP is *not* suggesting we move from Roundup to GitLab + issues. We have way too much invested in Roundup right now and migrating + the data would be a huge effort. GitLab does support webhooks, so we will + probably want to use webhooks to integrate merges and other events with + updates to Roundup (e.g. to include pointers to commits, close issues, + etc. similar to what is currently done). + +* What happens to wiki.python.org? Nothing! While GitLab does support wikis + in repositories, there's no reason for us to migration our Moin wikis. + +* What happens to the existing GitHub mirrors? We'd probably want to + regenerate them once the official upstream branches are natively hosted in + Git. This may change commit ids, but after that, it should be easy to + mirror the official Git branches and repositories far and wide. + +* Where would the GitLab instance live? Physically, in whatever hosting + provider GitLab chooses. We would point gitlab.python.org (or + git.python.org?) to this host. + + References ========== -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Wed Sep 30 22:33:02 2015 From: python-checkins at python.org (barry.warsaw) Date: Wed, 30 Sep 2015 20:33:02 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_PEP_507=2C_GitLab?= Message-ID: <20150930203300.82664.62567@psf.io> https://hg.python.org/peps/rev/7a68ed3b135b changeset: 6103:7a68ed3b135b user: Barry Warsaw date: Wed Sep 30 16:03:03 2015 -0400 summary: PEP 507, GitLab files: pep-0507.txt | 303 +++++++++++++++++++++++++++++++++++++++ 1 files changed, 303 insertions(+), 0 deletions(-) diff --git a/pep-0507.txt b/pep-0507.txt new file mode 100644 --- /dev/null +++ b/pep-0507.txt @@ -0,0 +1,303 @@ +PEP: 507 +Title: Migrate CPython to Git and GitLab +Version: $Revision$ +Last-Modified: $Date$ +Author: Barry Warsaw +Status: Draft +Type: Process +Content-Type: text/x-rst +Created: 2015-09-30 +Post-History: + + +Abstract +======== + +This PEP proposes migrating the repository hosting of CPython and the +supporting repositories to Git. Further, it proposes adopting a +hosted GitLab instance as the primary way of handling merge requests, +code reviews, and code hosting. It is similar in intent to PEP 481 +but proposes an open source alternative to GitHub and omits the +proposal to run Phabricator. As with PEP 481, this particular PEP is +offered as an alternative to PEP 474 and PEP 462. + + +Rationale +========= + +CPython is an open source project which relies on a number of +volunteers donating their time. As with any healthy, vibrant open +source project, it relies on attracting new volunteers as well as +retaining existing developers. Given that volunteer time is the most +scarce resource, providing a process that maximizes the efficiency of +contributors and reduces the friction for contributions, is of vital +importance for the long-term health of the project. + +The current tool chain of the CPython project is a custom and unique +combination of tools. This has two critical implications: + +* The unique nature of the tool chain means that contributors must + remember or relearn, the process, workflow, and tools whenever they + contribute to CPython, without the advantage of leveraging long-term + memory and familiarity they retain by working with other projects in + the FLOSS ecosystem. The knowledge they gain in working with + CPython is unlikely to be applicable to other projects. + +* The burden on the Python/PSF infrastructure team is much greater in + order to continue to maintain custom tools, improve them over time, + fix bugs, address security issues, and more generally adapt to new + standards in online software development with global collaboration. + +These limitations act as a barrier to contribution both for highly +engaged contributors (e.g. core Python developers) and especially for +more casual "drive-by" contributors, who care more about getting their +bug fix than learning a new suite of tools and workflows. + +By proposing the adoption of both a different version control system +and a modern, well-maintained hosting solution, this PEP addresses +these limitations. It aims to enable a modern, well-understood +process that will carry CPython development for many years. + + +Version Control System +---------------------- + +Currently the CPython and supporting repositories use Mercurial. As a +modern distributed version control system, it has served us well since +the migration from Subversion. However, when evaluating the VCS we +must consider the capabilities of the VCS itself as well as the +network effect and mindshare of the community around that VCS. + +There are really only two real options for this, Mercurial and Git. +The technical capabilities of the two systems are largely equivalent, +therefore this PEP instead focuses on their social aspects. + +It is not possible to get exact numbers for the number of projects or +people which are using a particular VCS, however we can infer this by +looking at several sources of information for what VCS projects are +using. + +The Open Hub (previously Ohloh) statistics [#openhub-stats]_ show that +37% of the repositories indexed by The Open Hub are using Git (second +only to Subversion which has 48%) while Mercurial has just 2%, beating +only Bazaar which has 1%. This has Git being just over 18 times as +popular as Mercurial on The Open Hub. + +Another source of information on VCS popularity is PyPI itself. This +source is more targeted at the Python community itself since it +represents projects developed for Python. Unfortunately PyPI does not +have a standard location for representing this information, so this +requires manual processing. If we limit our search to the top 100 +projects on PyPI (ordered by download counts) we can see that 62% of +them use Git, while 22% of them use Mercurial, and 13% use something +else. This has Git being just under 3 times as popular as Mercurial +for the top 100 projects on PyPI. + +These numbers back up the anecdotal evidence for Git as the far more +popular DVCS for open source projects. Choosing the more popular VCS +has a number of positive benefits. + +For new contributors it increases the likelihood that they will have already +learned the basics of Git as part of working with another project or if they +are just now learning Git, that they'll be able to take that knowledge and +apply it to other projects. Additionally a larger community means more people +writing how to guides, answering questions, and writing articles about Git +which makes it easier for a new user to find answers and information about the +tool they are trying to learn and use. Given its popularity, there may also +be more auxiliary tooling written *around* Git. This increases options for +everything from GUI clients, helper scripts, repository hosting, etc. + +Further, the adoption of Git as the proposed back-end repository +format doesn't prohibit the use of Mercurial by fans of that VCS! +Mercurial users have the [#hg-git]_ plugin which allows them to push +and pull from a Git server using the Mercurial front-end. It's a +well-maintained and highly functional plugin that seems to be +well-liked by Mercurial users. + + +Repository Hosting +------------------ + +Where and how the official repositories for CPython are hosted is in +someways determined by the choice of VCS. With Git there are several +options. In fact, once the repository is hosted in Git, branches can +be mirrored in many locations, within many free, open, and proprietary +code hosting sites. + +It's still important for CPython to adopt a single, official +repository, with a web front-end that allows for many convenient and +common interactions entirely through the web, without always requiring +local VCS manipulations. These interactions include as a minimum, +code review with inline comments, branch diffing, CI integration, and +auto-merging. + +This PEP proposes to adopt a [#GitLab]_ instance, run within the +python.org domain, accessible to and with ultimate control from the +PSF and the Python infrastructure team, but donated, hosted, and +primarily maintained by GitLab, Inc. + +Why GitLab? Because it is a fully functional Git hosting system, that +sports modern web interactions, software workflows, and CI +integration. GitLab's Community Edition (CE) is open source software, +and thus is closely aligned with the principles of the CPython +community. + + +Code Review +----------- + +Currently CPython uses a custom fork of Rietveld modified to not run +on Google App Engine and which is currently only really maintained by +one person. It is missing common features present in many modern code +review tools. + +This PEP proposes to utilize GitLab's built-in merge requests and +online code review features to facilitate reviews of all proposed +changes. + + +GitLab merge requests +--------------------- + +The normal workflow for a GitLab hosted project is to submit a *merge request* +asking that a feature or bug fix branch be merged into a target branch, +usually one or more of the stable maintenance branches or the next-version +master branch for new features. GitLab's merge requests are similar in form +and function to GitHub's pull requests, so anybody who is already familiar +with the latter should be able to immediately utilize the former. + +Once submitted, a conversation about the change can be had between the +submitter and reviewer. This includes both general comments, and inline +comments attached to a particular line of the diff between the source and +target branches. Projects can also be configured to automatically run +continuous integration on the submitted branch, the results of which are +readily visible from the merge request page. Thus both the reviewer and +submitter can immediately see the results of the tests, making it much easier +to only land branches with passing tests. Each new push to the source branch +(e.g. to respond to a commenter's feedback or to fix a failing test) results +in a new run of the CI, so that the state of the request always reflects the +latest commit. + +Merge requests have a fairly major advantage over the older "submit a patch to +a bug tracker" model. They allow developers to work completely within the VCS +using standard VCS tooling, without requiring the creation of a patch file or +figuring out the right location to upload the patch to. This lowers the +barrier for sending a change to be reviewed. + +Merge requests are far easier to review. For example, they provide nice +syntax highlighted diffs which can operate in either unified or side by side +views. They allow commenting inline and on the merge request as a whole and +they present that in a nice unified way which will also hide comments which no +longer apply. Comments can be hidden and revealed. + +Actually merging a merge request is quite simple, if the source branch applies +cleanly to the target branch. A core reviewer simply needs to press the +"Merge" button for GitLab to automatically perform the merge. The source +branch can be optionally rebased, and once the merge is completed, the source +branch can be automatically deleted. + +GitLab also has a good workflow for submitting pull requests to a project +completely through their web interface. This would enable the Python +documentation to have "Edit on GitLab" buttons on every page and people who +discover things like typos, inaccuracies, or just want to make improvements to +the docs they are currently reading. They can simply hit that button and get +an in browser editor that will let them make changes and submit a merge +request all from the comfort of their browser. + + +Criticism +========= + +X is not written in Python +-------------------------- + +One feature that the current tooling (Mercurial, Rietveld) has is that the +primary language for all of the pieces are written in Python. This PEP +focuses more on the *best* tools for the job and not necessarily on the *best* +tools that happen to be written in Python. Volunteer time is the most +precious resource for any open source project and we can best respect and +utilize that time by focusing on the benefits and downsides of the tools +themselves rather than what language their authors happened to write them in. + +One concern is the ability to modify tools to work for us, however one of the +Goals here is to *not* modify software to work for us and instead adapt +ourselves to a more standardized workflow. This standardization pays off in +the ability to re-use tools out of the box freeing up developer time to +actually work on Python itself as well as enabling knowledge sharing between +projects. + +However if we do need to modify the tooling, Git itself is largely written in +C the same as CPython itself. It can also have commands written for it using +any language, including Python. GitLab itself is largely written in Ruby and +since it is Open Source software, we would have the ability to submit merge +requests to the upstream Community Edition, albeit in language potentially +unfamiliar to most Python programmers. + + +Mercurial is better than Git +---------------------------- + +Whether Mercurial or Git is better on a technical level is a highly subjective +opinion. This PEP does not state whether the mechanics of Git or Mercurial +are better, and instead focuses on the network effect that is available for +either option. While this PEP proposes switching to Git, Mercurial users are +not left completely out of the loop. By using the hg-git extension for +Mercurial, working with server-side Git repositories is fairly easy and +straightforward. + + +CPython Workflow is too Complicated +----------------------------------- + +One sentiment that came out of previous discussions was that the multi-branch +model of CPython was too complicated for GitLab style merge requests. This +PEP disagrees with that sentiment. + +Currently any particular change requires manually creating a patch for 2.7 and +3.x which won't change at all in this regards. + +If someone submits a fix for the current stable branch (e.g. 3.5) the merge +request workflow can be used to create a request to merge the current stable +branch into the master branch, assuming there is no merge conflicts. As +always, merge conflicts must be manually and locally resolved. Because +developers also have the *option* of performing the merge locally, this +provides an improvement over the current situation where the merge *must* +always happen locally. + +For fixes in the current development branch that must also be applied to +stable release branches, it is possible in many situations to locally cherry +pick and apply the change to other branches, with merge requests submitted for +each stable branch. It is also possible just cherry pick and complete the +merge locally. These are all accomplished with standard Git commands and +techniques, with the advantage that all such changes can go through the review +and CI test workflows, even for merges to stable branches. Minor changes may +be easily accomplished in the GitLab web editor. + +No system can hide all the complexities involved in maintaining several long +lived branches. The only thing that the tooling can do is make it as easy as +possible to submit and commit changes. + + +References +========== + +.. [#openhub-stats] `Open Hub Statistics ` +.. [#hg-git] `Hg-Git mercurial plugin ` +.. [#GitLab] `https://about.gitlab.com/` + + +Copyright +========= + +This document has been placed in the public domain. + + + +.. + Local Variables: + mode: indented-text + indent-tabs-mode: nil + sentence-end-double-space: t + fill-column: 70 + coding: utf-8 + End: -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Wed Sep 30 23:01:00 2015 From: python-checkins at python.org (victor.stinner) Date: Wed, 30 Sep 2015 21:01:00 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Backout_change_28d3bcb1bad?= =?utf-8?q?6=3A_=22Try_to_fix_=5FPyTime=5FAsTimevalStruct=5Fimpl=28=29_on?= Message-ID: <20150930210058.3652.89639@psf.io> https://hg.python.org/cpython/rev/4bbe95f0a57b changeset: 98442:4bbe95f0a57b user: Victor Stinner date: Wed Sep 30 22:50:12 2015 +0200 summary: Backout change 28d3bcb1bad6: "Try to fix _PyTime_AsTimevalStruct_impl() on OpenBSD", I'm not sure that the change was really needed. I read the test result of an old build because the OpenBSD was 100 builds late. files: Python/pytime.c | 5 ++--- 1 files changed, 2 insertions(+), 3 deletions(-) diff --git a/Python/pytime.c b/Python/pytime.c --- a/Python/pytime.c +++ b/Python/pytime.c @@ -454,7 +454,7 @@ _PyTime_AsTimevalStruct_impl(_PyTime_t t, struct timeval *tv, _PyTime_round_t round, int raise) { - _PyTime_t secs, secs2; + _PyTime_t secs; int us; int res; @@ -467,8 +467,7 @@ #endif tv->tv_usec = us; - secs2 = (_PyTime_t)tv->tv_sec; - if (res < 0 || secs2 != secs) { + if (res < 0 || (_PyTime_t)tv->tv_sec != secs) { if (raise) error_time_t_overflow(); return -1; -- Repository URL: https://hg.python.org/cpython